Real-Time On-Device Multi-Vessel Tracking for UAV with Trapezoid Convolution

Abstract

Visual multi-vessel tracking is critical for intelligent maritime surveillance yet challenging due to the complexity of efficiently modeling rigid vessel structures across diverse scales and viewpoints. Existing methods struggle to reconcile fine-grained geometric modeling with the real-time constraints of resource-constrained UAV edge platforms, where performance is further hindered by scale variations and viewpoint perturbations. To address these challenges, a novel visual multi-vessel tracking framework (MobileMVT) is proposed for real-time deployment on UAV-mounted edge electro-optical devices. At its core, the Trapezoid Convolution (TConv) encodes vessel-specific geometric priors through trapezoid-aligned sampling, enhancing shape preservation and feature discrimination. Building upon it, TConvNet serves as a compact backbone to improve recognition accuracy and tracking consistency with minimal computational cost. Furthermore, an uncertainty-aware viewpoint compensator is integrated to mitigate UAV-induced shifts and motion uncertainties, ensuring stable trajectory estimation. To facilitate comprehensive evaluation, we introduce VesselMOT, a large-scale benchmark comprising 155 sequences and over 170k annotated frames featuring diverse scenarios. Extensive experiments and real-world tests demonstrate that MobileMVT achieves superior tracking accuracy and real-time performance in dynamic maritime environments. The code and new dataset are available at https://github.com/vision4robotics/MobileMVT.