Automotive scene understanding under adverse weather conditions raises a realistic and challenging problem attributable to poor outdoor scene visibility (e.g. foggy weather). However, because most contemporary scene understanding approaches are applied under ideal-weather conditions, such approaches may not provide genuinely optimal performance when compared to established a priori insights on extreme-weather understanding. In this paper, we propose a complex but competitive multi-task learning approach capable of performing in real-time semantic scene understanding and monocular depth estimation under foggy weather conditions by leveraging both recent advances in adversarial training and domain adaptation. As an end-to-end pipeline, our model provides a novel solution to surpass degraded visibility in foggy weather conditions by transferring scenes from foggy to normal using a GAN-based model. For optimal performance in semantic segmentation, our model generates depth to be used as complementary source information with RGB in the segmentation network. We provide a robust method for foggy scene understanding by training two models (normal and foggy) simultaneously with shared weights (each model is trained on each weather condition). Our model incorporates RGB colour, depth, and luminance images via distinct encoders with dense connectivity and features fusing, and leverages skip connections to produce consistent depth and segmentation predictions. Using this architectural formulation with light computational complexity at inference time, we are able to achieve comparable performance to contemporary approaches at a fraction of the overall model complexity. Evaluation over several foggy weather condition datasets including synthetic and real-world examples illustrates our approach competitive performance compared to other contemporary state-of-the-art approaches.
Competitive Simplicity for Multi-Task Learning for Real-Time Foggy Scene Understanding via Domain Adaptation
2021 IEEE Intelligent Vehicles Symposium (IV) ; 1413-1420
2021-07-11
49591347 byte
Conference paper
Electronic Resource
English
Real-Time Traffic Sign Detection Under Foggy Condition
Springer Verlag | 2022
|Object Detection in Foggy Scenes by Embedding Depth and Reconstruction into Domain Adaptation
British Library Conference Proceedings | 2023
|IuD Bahn | 2007
|