A robust and efficient traffic monitoring system is essential for smart cities and Intelligent Transportation Systems (ITS), using sensors and cameras to track vehicle movements, optimize traffic flow, reduce congestion, enhance road safety, and enable real-time adaptive traffic control. Traffic monitoring models must comprehensively understand dynamic urban conditions and provide an intuitive user interface for effective management. This research leverages the LLaVA visual grounding multimodal large language model (LLM) for traffic monitoring tasks on the real-time Quanser Interactive Lab simulation platform, covering scenarios like intersections, congestion, and collisions. Cameras placed at multiple urban locations collect real-time images from the simulation, which are fed into the LLaVA model with queries for analysis. An instance segmentation model integrated into the cameras highlights key elements such as vehicles and pedestrians, enhancing training and throughput. The system achieves 84.3% accuracy in recognizing vehicle locations and 76.4% in determining steering direction, outperforming traditional models.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Leveraging Multimodal-LLMs Assisted by Instance Segmentation for Intelligent Traffic Monitoring


    Beteiligte:
    Onsu, Murat Arda (Autor:in) / Lohan, Poonam (Autor:in) / Kantarci, Burak (Autor:in) / Syed, Aisha (Autor:in) / Andrews, Matthew (Autor:in) / Kennedy, Sean (Autor:in)

    Erscheinungsdatum :

    2025


    Anmerkungen:

    6 pages, 7 figures, submitted to 30th IEEE International Symposium on Computers and Communications (ISCC) 2025


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Leveraging Large Language Models (LLMs) for Traffic Management at Urban Intersections: The Case of Mixed Traffic Scenarios

    Masri, Sari / Ashqar, Huthaifa I. / Elhenawy, Mohammed | ArXiv | 2024

    Freier Zugriff

    Joint Semantic-Instance Segmentation Method for Intelligent Transportation System

    Li, Yujie / Cai, Jintong / Zhou, Quan et al. | IEEE | 2023


    RTMDet-R: A Robust Instance Segmentation Network for Complex Traffic Scenarios

    Wang, Hai / Qin, Qirui / Chen, Long et al. | IEEE | 2025


    Instance segmentation imaging system

    PHILION JONAH / ZHAO YIBIAO | Europäisches Patentamt | 2025

    Freier Zugriff

    Instance segmentation imaging system

    PHILION JONAH / ZHAO YIBIAO | Europäisches Patentamt | 2023

    Freier Zugriff