Computer vision analysis of unconstrained urban ground-level images

Author(s)

Éric Keiji Tokuda

• Document type: Doctoral Thesis
• Press: São Paulo.
• Institution: Universidade de São Paulo (USP). Instituto de Matemática e Estatística (IME/SBI)
• Defense date: 2019-07-11

Examining board members

Roberto Marcondes Cesar Junior; Roberto Hirata Junior; João Paulo Papa; Hélio Pedrini; Anderson de Rezende Rocha

• Advisor: Roberto Marcondes Cesar Junior

Abstract

Nowadays, images are generated on a large scale and in a decentralized way. Such modality of data carries valuable information but extracting this information is not always trivial. In this thesis, we tackle computer vision challenges when using ground-level images. The first challenge is the high-cost annotation for evaluating object detection methods. In the context of image degradation imposed by weather, the second issue is the lack of analysis that evaluates the impact of de-raining methods to the object detection algorithms on rainy images. The third challenge is the evaluation of the reliability of the density estimation results from a real sensor network. The emergence of sensor network data motivates the last problem, of estimating the urban degradation in the city using city images. These challenges define the scope of this thesis. For the first problem, we proposed an approach with cheap annotation cost for object detectors comparison and we applied it in a semi-supervised learning approach using surveillance images. To address challenge two, we established a protocol and performed an extensive benchmark of object detection preceded by de-raining methods. We find strong indicators that no current de-raining method can robustly improve the posterior object detection accuracy when applied in this naive way. The third issue was tackled by creating a probabilistic sensing model to establish theoretical bounds for the errors of the sensed distributions. The approach has been validated using simulation and applied to compute the pedestrian density map in Manhattan. To attack the last problem, we systematically collected public images of São Paulo and segmented the regions affected by tagging, as an indicator of the urban degradation of the region. The source code is fully released. (AU)

• FAPESP's process: 14/24918-0 - Weakly supervised learning for face and person attributes detection
• Grantee: Eric Keiji Tokuda
• Support Opportunities: Scholarships in Brazil - Doctorate