Understanding and predicting pedestrian behavior is an important and challenging area of research for realizing safe and effective navigation strategies in automated and advanced driver assistance technologies in urban scenes. This paper focuses on monocular pedestrian action recognition and 3D localization from an egocentric view for the purpose of predicting intention and forecasting future trajectory. A challenge in addressing this problem in urban traffic scenes is attributed to the unpredictable behavior of pedestrians, whereby actions and intentions are constantly in flux and depend on the pedestrians pose, their 3D spatial relations, and their interaction with other agents as well as with the environment. To partially address these challenges, we consider the importance of pose toward recognition and 3D localization of pedestrian actions. In particular, we propose an action recognition framework using a two-stream temporal relation network with inputs corresponding to the raw RGB image sequence of the tracked pedestrian as well as the pedestrian pose. The proposed method outperforms methods using a single-stream temporal relation network based on evaluations using the JAAD public dataset. The estimated pose and associated body key-points are also used as input to a network that estimates the 3D location of the pedestrian using a unique loss function. The evaluation of our 3D localization method on the KITTI dataset indicates the improvement of the average localization error as compared to existing state-of-the-art methods. Finally, we conduct qualitative tests of action recognition and 3D localization on HRI’s H3D driving dataset.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Recognition and 3D Localization of Pedestrian Actions from Monocular Video


    Contributors:


    Publication date :

    2020-09-20


    Size :

    3090087 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Monocular pedestrian recognition using motion parallax

    Enzweiler, M. / Kanter, P. / Gavrila, D. M. | IEEE | 2008


    Monocular Pedestrian Recognition Using Motion Parallax

    Enzweiler, M. / Kanter, P. / Gavrila, D.M. | British Library Conference Proceedings | 2008


    Pedestrian detection with monocular vision system

    Schick,J. / Bosch,Leonberg,DE | Automotive engineering | 2006


    Pedestrian candidates generation using monocular cues

    Cheda, Diego / Ponsa, Daniel / Lopez, Antonio M. | IEEE | 2012


    Pedestrian detection with monocular vision system

    Schick, Jens | Tema Archive | 2006