There is a growing demand for the virtualization of real-world scenes to become more realistic, driven by improvements in data acquisition systems and availability of analytical tools and computational power to treat that data. Despite the growing number of studies which tackle specific steps in this process, there is still a lack of solutions which leverage recent advances in computer vision techniques to facilitate the digital reconstruction of a real-world environment. Such an approach has obvious advantages, e.g., being able of operating on simple video data and not requiring an expensive data acquisition system, such as LIDAR cameras. However, it comes with several challenges, including the need to individualize and categorize objects in an image, estimate their position with respect to the camera and also deal with artifacts. To tackle these challenges, we conceived an innovative workflow for extracting and positioning in real-world coordinates relevant elements in a video scene, in our case of video captured from the front of a moving train. Starting from a semantic segmentation task, our method then leverages the object segmentation model SegmentAnything to assign a unique identifier to each instance of each class, which are subsequently tracked along the video sequence using a video segmentation algorithm. Each unique object instance can be positioned in real-world coordinates by integrating the output with its depth calibrated to metric units by leveraging scene reconstruction from Structure from Motion (SfM). The full approach is validated by comparing the estimated positions of buildings and traffic signs with their real positions based on an open-source database, achieving in both a sub-meter accuracy. This novel approach provides a comprehensive framework for the positioning of any object from a sequence of video frames and can be applied to a wide-range of domains beyond the one tackled here.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Object Localization and Tracking Pipeline for the Realistic Rendering of Railway Environments




    Publication date :

    2024-09-24


    Size :

    3502586 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Realistic range rendering for object hypothesis verification

    Flynn, P. J. | British Library Online Contents | 1996


    Realistic range rendering

    Flynn, | IEEE | 1994


    Realistic Range Rendering

    Flynn, P. J. / Institute of Electrical and Electronics Engineers; Computer Society | British Library Conference Proceedings | 1994


    Online Localization and Mapping with Moving Object Tracking in Dynamic Outdoor Environments

    Vu, Trung-Dung / Aycard, Olivier / Appenrodt, Nils | IEEE | 2007


    A SYSTEM FOR REALISTIC THREE-DIMENSIONAL RENDERING

    N BALASUBRAMANIYAN / B KARTHIKEYAN / THANGAMANI ARUNVEL | European Patent Office | 2020

    Free access