Nonlinear effect compensation in digital coherent optical communication systems employing neural networks with unsupervised training

• Grant number: 25/00062-3
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: March 01, 2025
• End date: February 28, 2026
• Field of knowledge: Engineering - Electrical Engineering - Telecommunications
• Principal Investigator: Ivan Aritz Aldaya Garde
• Grantee: Ana Laura Jorge das Graças
• Host Institution: Faculdade de Engenharia. Universidade Estadual Paulista (UNESP). Campus Experimental São João da Boa Vista. São João da Boa Vista , SP, Brazil
• Associated research grant: 24/01712-9 - Compensation for transmission impairments in advanced optical communication systems employing artificial intelligence, AP.R

Abstract

Digital coherent optical communication systems have revolutionized the design of optical networks, particularly through the adoption of advanced modulation formats and the efficient compensation of impairments. These innovations enable the achievement of unprecedented spectral efficiencies. However, the combination of additive noise and nonlinear distortions remains one of the main limiting factors of the channel's maximum capacity. In this context, various nonlinear impairment mitigation techniques have been proposed in recent years. Among these, machine learning-based methods have shown a good balance between performance and computational cost. For instance, clustering methods have the advantage of not requiring training sequences, but they exhibit limited mitigation capability. On the other hand, supervised regression techniques offer superior performance but rely on training sequences, which reduce the net transmission rate. This project proposes evaluating the integration of clustering techniques with supervised regression methods. The objective is to analyze whether this combined approach can overcome the individual limitations and achieve better performance compared to mitigation based solely on clustering. The research aims to contribute to the development of more efficient and robust strategies for nonlinear impairment compensation in digital coherent optical systems.