System inspection and analysis of robotic systems with the use of low cost RGB-D sensor

• Grant number: 17/00882-4
• Support Opportunities: Research Grants - Innovative Research in Small Business - PIPE
• Start date: October 01, 2017
• End date: May 31, 2018
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Glaydson Luiz Bertoze Lima
• Grantee: Glaydson Luiz Bertoze Lima
• Company: It Fast Consultoria em TI Ltda. - ME
• City: São José dos Campos
• Pesquisadores principais: Osamu Saotome
• Associated researchers: IJAR MILAGRE DA FONSECA

Abstract

This project aims to develop a system that measures and tracks real runtime objects in 3D video scenes, viewed from low-cost intelligent sensors (in this project the Microsoft Kinect sensor). The solution is proposed for a computer system, responsible for the reception of luminous signals (lumens), luminous intensity (luxs) and ultraviolet (UV) rays from a network of wireless sensors installed in the environment. The idea is to perform the automatic calibration of the filters used in the image processing and computational vision applied to the proposed system. Once the calibration is done, the system receives the video (RGB) and depth (IR) images of the Kinect sensor, searches the points of interest marked on objects (for example, quadcopters, robots and / or air-supported platforms) and starts The coordinates X, Y and Z of each point of interest displayed in the video. These coordinates are sent through the UDP protocol to another computer, with simulator software (Mathlab, Labview, X-Plane), which receives this real positioning information and compares them to the expected real ones (estimated in the simulation), perceived and processed by the Kinect sensors). With the comparison results, it is possible to improve and validate models as, for example, of landing and automatic take-off, autonomous flights, and in the formation flying of quadcopters as well as in the stability of moving robots among others. With the proposed system, it is possible to compare the estimated robotic system under real conditions. IT FAST developed a primary prototype (basic communication software with Kinect) whose basic functionality was presented to the teachers of Electronic Engineering / Computing and Aeronautical Mechanics at ITA - Instituto Tecnológico de Aeronáutica (aeronautics technological Institute). The proposed research project intends to present a better finished prototype for tracking using low cost sensors, with improvement aiming the state of the art of this technology in the world. The main technological improvements are: (1) the automatic calibration of software filters in relation to the environment; (2) support for Kinects version 1 (Kinect 360) and version 2 (Kinect One) sensors, the second used only when high resolution is required; (3) tool developed closer to hardware, in C / C ++ language, higher speed in X, Y and Z positioning, supported by FPGA; (4) Flexible, modular, extensible and easy-to-use software with specific functions for each audience (robotics and robotics), with emphasis on the markets for teaching, scientific research and industrial automation. It is expected that at the end of the project it will be obtained: a detailed list of the most appropriate components to the project, hardware prototype validation, prototype software verification, calibration limits, and use of low cost sensors for follow-up at real runtime. So that in the sequence, we can evolve with this solution for a market product. (AU)