Deep Learning Models for Tracking Players and Calibration of Drone Cameras in Soccer

• Grant number: 25/03268-1
• Support Opportunities: Scholarships in Brazil - Master
• Start date: January 01, 2026
• End date: January 31, 2027
• Field of knowledge: Health Sciences - Physical Education
• Principal Investigator: Paulo Roberto Pereira Santiago
• Grantee: Lennin Abrão Sousa Santos
• Host Institution: Faculdade de Medicina de Ribeirão Preto (FMRP). Universidade de São Paulo (USP). Ribeirão Preto , SP, Brazil

Abstract

This master's degree project presents a computer vision system for analyzing top-down dronesoccer videos, focusing on two pillars: multi-object tracking (MOT) and automatic camera-to-fieldcalibration. For tracking, we adopt the tracking-by-detection paradigm, where objects are detectedin each frame and associated over time. We employ a You Only Look Once (YOLO) detector tailoredto aerial imagery, with optimized anchor design and domain-specific data augmentations , togetherwith a Re-Identification (ReID) head to mitigate identity switches . A multi-person pose estimationmodule is also assessed to enhance robustness under occlusions. For calibration, the pipelineintegrates semantic segmentation of field lines and grass, detection of keypoints (intersections,penalty marks, circles), robust homography estimation, and geometric consistency checks. Thedataset consists of 4K youth-game videos annotated with bounding boxes, player identities (IDs),and field markings, complemented by public datasets. Training is performed on high-performanceGPUs (e.g., NVIDIA RTX 3090/4090). The solution will be released within the vailá framework, withopen-source code, pretrained weights, and documentation. Validation uses established metrics:mean Average Precision (mAP@[0.50:0.95]) for detection, Identification F1 score (IDF1) and HigherOrder Tracking Accuracy (HOTA) for tracking, plus reprojection and calibration errors, evaluated withcross-validation and ablation studies. Target benchmarks are mAP@[0.50:0.95] ¿ 0.55, IDF1 ¿ 0.60,and reprojection error ¿ 12 px (target range: 8-12 px). Looking ahead, we explore aerial-domainself-supervised pretraining, deployment on low-cost edge devices, and open-science practices toenable adoption beyond the lab. As a social contribution, delivering advanced AI technologies toschools and grassroots clubs fosters scientific and digital literacy, critical awareness, and a senseof belonging among youth who rarely have access to such tools, thereby reducing the technologygap. (AU)