In an era where maritime infrastructures are paramount to societies, the need for advanced maritime situational awareness solutions has become increasingly important. Existing ship monitoring procedures, such as the Automatic Identification System (AIS), have limitations, suffer from delayed updates, and are vulnerable to cyberattacks. Other technologies, such as satellite imagery and radar, face challenges in real-time applications due to delays in acquiring and processing data. The use of optical camera systems and image processing can improve situational awareness, allowing real-time usage of maritime infrastructure footage. However, the number of video streams available poses a challenge for maritime operators, who could benefit from summarized spatial information of recognized ships, irrespective of their size and type, presented on a map in real-time. This motivates the development of automated ship recognition and georeferencing technologies. Moreover, the deployment of such camera systems, equipped with embedded devices, allows for local data processing on the edge to minimize network demand, reduce energy usage, decrease latency, cut costs, and enhance data protection. This thesis, integrating six of my publications, presents a comprehensive investigation into leveraging deep learning and computer vision to advance real-time ship recognition and georeferencing for the improvement of maritime situational awareness. I present a novel dataset for ship recognition and georeferencing, ShipSG, which facilitates the development and validation of recognition and georeferencing methodologies. The dataset contains 3,505 images and 11,625 ship masks with their corresponding class, geographic position, and length. Through a series of studies of state-of-the-art deep-learning-based object recognition algorithms, I introduce a custom real-time segmentation architecture, ScatYOLOv8+CBAM. This architecture was created and optimized for the NVIDIA Jetson AGX Xavier as an embedded system. ScatYOLOv8+CBAM incorporates the 2D scattering transform, a novel addition that enhances YOLOv8 in real-world applications such as ship segmentation. Additionally, the performance is further improved with the integration of attention mechanisms. The proposed architecture exceeds state-of-the-art methods by more than 5%, achieving a mean Average Precision (mAP) of 75.46%. The inference speed, once the customized architecture is deployed on the embedded system using TensorRT, is 25.3 ms per frame. Furthermore, I address the need for precision in recognizing small and distant ships and their real-time processing of full-resolution images on embedded systems with an enhanced slicing mechanism that performs batch inference and merges predictions, achieving mAP improvements ranging from 8% to 11%. The recognized ships are georeferenced using my proposed method, which automatically calculates the georeferencing pixel of the recognized ships and uses homographies to provide the geographic position of ships from single images without prior camera knowledge. In the quantitative analysis, the georeferencing method achieved a positioning error of 18 m ± 10 m for ranges inside the port basin (up to 400 m) and 44 m ± 27 m outside (from 400 m to 1200 m). The main findings reveal significant advancements in maritime situational awareness with the practical demonstration of the applicability of the methodologies in real-world scenarios, such as the detection of abnormal ship behavior, camera integrity assessment, and 3D reconstruction. The approach not only outperforms existing methods in terms of accuracy and processing speed but also provides a framework for seamlessly integrating recognized and georeferenced ships into real-time systems, enhancing operational effectiveness and decision-making for maritime authorities. The integration of these methodologies into embedded systems represents a pivotal advancement in the domain, offering a scalable and efficient solution for improving maritime situational awareness and response capabilities. This thesis contributes to the maritime computer vision field by establishing a benchmark for ship segmentation and georeferencing research, demonstrating the viability of deep-learning-based recognition and georeferencing methods for real-time maritime monitoring.


    Access

    Download


    Export, share and cite



    Title :

    Real-time ship recognition and georeferencing for the improvement of maritime situational awareness


    Contributors:

    Publication date :

    2024



    Type of media :

    Miscellaneous


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    0




    Maritime situational awareness

    North Atlantic Treaty Organization | British Library Conference Proceedings | 2008


    Near Real Time Applications for Maritime Situational Awareness

    E. Schwarz / D. Krause / M. Berg et al. | DOAJ | 2015

    Free access

    Maritime Situational Awareness of the students for the Target Ship

    Nakashima, Hiroki | British Library Online Contents | 2016


    Event Processing for Maritime Situational AwarenessMaritime situation awareness (MSA)

    Pitsikalis, Manolis / Bereta, Konstantina / Vodas, Marios et al. | Springer Verlag | 2020


    MARITIME SITUATIONAL AWARENESS АS A KEY MEASURE FOR SAFE SHIP OPERATION

    Oleksiy MELNYK / Yuriy BYCHKOVSKY / Andrii VOLOSHYN | DOAJ | 2022

    Free access