The control of self-driving cars has received growing attention recently. Although existing research shows promising results in the vehicle control using video from a monocular dash camera, there has been very limited work on directly learning vehicle control from motion-based cues. Such cues are powerful features for visual representations, as they encode the per-pixel movement between two consecutive images, allowing a system to effectively map the features into the control signal. The authors propose a new framework that exploits the use of a motion-based feature known as optical flow extracted from the dash camera and demonstrates that such a feature is effective in significantly improving the accuracy of the control signals. The proposed framework involves two main components. The flow predictor, as a self-supervised deep network, models the underlying scene structure from consecutive frames and generates the optical flow. The controller, as a supervised multi-task deep network, predicts both steer angle and speed. The authors demonstrate that the proposed framework using the optical flow features can effectively predict control signals from a dash camera video. Using the Cityscapes data set, the authors validate that the system prediction has errors as low as 0.0130 rad/s on steer angle and 0.0615 m/s on speed, outperforming existing research.
Multi-task deep learning with optical flow features for self-driving cars
IET Intelligent Transport Systems ; 14 , 13 ; 1845-1854
2021-01-06
10 pages
Article (Journal)
Electronic Resource
English
multitask deep learning , video signal processing , consecutive images , image motion analysis , motion-based feature , flow predictor , learning (artificial intelligence) , vehicle control , supervised multitask deep network , optical flow features , cameras , self-driving cars , monocular dash camera , dash camera video , self-supervised deep network , traffic engineering computing , feature extraction , automobiles , image sequences , control signal
Metadata by IET is licensed under CC BY 3.0
Multi‐task deep learning with optical flow features for self‐driving cars
Wiley | 2020
|