The control of self‐driving cars has received growing attention recently. Although existing research shows promising results in the vehicle control using video from a monocular dash camera, there has been very limited work on directly learning vehicle control from motion‐based cues. Such cues are powerful features for visual representations, as they encode the per‐pixel movement between two consecutive images, allowing a system to effectively map the features into the control signal. The authors propose a new framework that exploits the use of a motion‐based feature known as optical flow extracted from the dash camera and demonstrates that such a feature is effective in significantly improving the accuracy of the control signals. The proposed framework involves two main components. The flow predictor, as a self‐supervised deep network, models the underlying scene structure from consecutive frames and generates the optical flow. The controller, as a supervised multi‐task deep network, predicts both steer angle and speed. The authors demonstrate that the proposed framework using the optical flow features can effectively predict control signals from a dash camera video. Using the Cityscapes data set, the authors validate that the system prediction has errors as low as 0.0130 rad/s on steer angle and 0.0615 m/s on speed, outperforming existing research.
Multi‐task deep learning with optical flow features for self‐driving cars
IET Intelligent Transport Systems ; 14 , 13 ; 1845-1854
01.12.2020
10 pages
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
dash camera video , cameras , image sequences , control signal , self‐supervised deep network , image motion analysis , optical flow features , multitask deep learning , monocular dash camera , supervised multitask deep network , automobiles , traffic engineering computing , feature extraction , vehicle control , video signal processing , learning (artificial intelligence) , consecutive images , flow predictor , self‐driving cars , motion‐based feature
Multi-task deep learning with optical flow features for self-driving cars
IET | 2021
|