The control of self-driving cars has received growing attention recently. Although existing research shows promising results in the vehicle control using video from a monocular dash camera, there has been very limited work on directly learning vehicle control from motion-based cues. Such cues are powerful features for visual representations, as they encode the per-pixel movement between two consecutive images, allowing a system to effectively map the features into the control signal. The authors propose a new framework that exploits the use of a motion-based feature known as optical flow extracted from the dash camera and demonstrates that such a feature is effective in significantly improving the accuracy of the control signals. The proposed framework involves two main components. The flow predictor, as a self-supervised deep network, models the underlying scene structure from consecutive frames and generates the optical flow. The controller, as a supervised multi-task deep network, predicts both steer angle and speed. The authors demonstrate that the proposed framework using the optical flow features can effectively predict control signals from a dash camera video. Using the Cityscapes data set, the authors validate that the system prediction has errors as low as 0.0130 rad/s on steer angle and 0.0615 m/s on speed, outperforming existing research.


    Access

    Access via TIB


    Export, share and cite



    Title :

    Multi-task deep learning with optical flow features for self-driving cars


    Contributors:

    Published in:

    Publication date :

    2021-01-06


    Size :

    10 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Multi‐task deep learning with optical flow features for self‐driving cars

    Hu, Yuan / Shum, Hubert P. H. / Ho, Edmond S. L. | Wiley | 2020

    Free access

    Deep Learning Based Self Driving Cars Using Computer Vision

    Bhaggiaraj, S. / Priyadharsini, M. / Karuppasamy, K. et al. | IEEE | 2023


    Deep Learning for Self-Driving Cars: Chances and Challenges

    Rao, Qing / Frtunikj, Jelena | IEEE | 2018


    Deep Learning Based Caching for Self-Driving Cars in Multi-Access Edge Computing

    Ndikumana, Anselme / Tran, Nguyen H. / Kim, Do Hyeon et al. | IEEE | 2021


    Predicting Steering Actions for Self-Driving Cars Through Deep Learning

    Ou, Chaojie / Bedawi, Safaa Mahmoud / Koesdwiady, Arief B. et al. | IEEE | 2018