ight-Field Imaging Reconstruction Using Deep Learning Enabling Intelligent Autonomous Transportation Syste

Author(s)

Silva, Juan Casavilca ^[1] ; Saadi, Muhammad ^[2] ; Wuttisittikulkij, Lunchakorn ^[3] ; Militani, Davi Ribeiro ^[4] ; Rosa, Renata Lopes ^[4] ; Rodriguez, Demostenes Zegarra ^[4] ; Al Otaibi, Sattam ^[5]

Affiliation

^[1] Pontificia Univ Catolica Peru, Dept Sci, Lima 15088 - Peru

^[2] Univ Cent Punjab, Dept Elect Engn, Fac Engn, Lahore 54000 - Pakistan

^[3] Chulalongkorn Univ, Dept Elect Engn, Wireless Commun Ecosyst Res Unit, Bangkok 10330 - Thailand

^[4] Univ Fed Lavras, Dept Comp Sci, BR-37200000 Lavras - Brazil

^[5] Taif Univ, Dept Elect Engn, Coll Engn, At Taif 21944 - Saudi Arabia

• Document type: Journal article
• Source: IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS; v. 23, n. 2 MAY 2021.
• Web of Science Citations: 1

Abstract

Light-field (LF) cameras, also known as plenoptic cameras, permit the recording of the 4D LF distribution of target scenes. However, many times, surface errors of a microlens array (MLA) are responsible for degradation in the images captured by a plenoptic camera. Additionally, the limited pixel count of the sensor can cause missing parallax information. The aforementioned issues are crucial for creating accurate maps for Intelligent Autonomous Transport System (IATS), because they cause loss of LF information, and need to be addressed. To tackle this problem, a learning-based framework by directly simulating the LF distribution is proposed. A high-dimensional convolution layer with densely sampled LFs in 4D space and considering a soft activation function based on ReLU segmentation correction is used to generate a superresolution (SR) LF image, improving the convergence rate in the deep learning network. Experimental results show that our proposed LF image reconstruction framework outperforms the existing state-of-the-art approaches; specifically, it is effective for learning the LF distribution and generating high-quality LF images. Different image quality assessment methods are used to evaluate the performance of the proposed framework, such as PSNR, SSIM, IWSSIM, FSIM, GFM, MDFM, and HDR-VDP. Additionally, the computational efficiency was evaluated in terms of number of parameters and FLOPs, and experimental results demonstrated that our proposed framework reached the highest performance in most of the datasets used. (AU)

• FAPESP's process: 18/26455-8 - Audio-Visual Speech Processing by Machine Learning
• Grantee: Miguel Arjona Ramírez
• Support Opportunities: Regular Research Grants