In the area of autonomous driving, sensing the environment is most important for self-localization and egomotion estimation. Visual odometry/SLAM methods have proven capable to achieve good results, even in real-time applications by operating in a sparse mode. Running on a sequence, these methods need to continuously incorporate new features well distributed over the image. Therefore, the performance of these methods can be further improved, if they are supplied with coarse but dense initial depth information, that can be utilized at arbitrary sparse image positions. Previously triangulated depths and even high quality depth measurements of a LIDAR sensor are not suitable for this task, since they only provide a sparse depth map. To solve this issue, we introduce a novel interpolation method called Spatio-Temporal Depth Interpolation (STDI), which exploits spatial and temporal correlations of the data (e.g. sequences of sparse depth maps) to give a consistent dense output including associated uncertainties. STDI is a fused approach, which makes use of the most important components of a principal component analysis (PCA) (spatial information) and additionally is capable to re-use information of previously interpolated depth maps in a regression based approach (temporal information). We evaluate the quality of STDI on the KITTI visual odometry benchmark, where a sequence of extremely sparsely sampled depth maps $(\approx 40$ depth values) is densified and on the KITTI depth completion benchmark. The latter deals with the densification of sparse LIDAR input. Of course, our method is not limited to these applications and can be used for any densification of sparse sequential data which is expected to contain spatial and/or temporal correlations (e.g. initialization for dense optical flow methods based on a sparse measurement).
Spatio-Temporal Depth Interpolation (STDI)
2018 IEEE Intelligent Vehicles Symposium (IV) ; 1742-1748
2018-06-01
7403307 byte
Conference paper
Electronic Resource
English
SPATIO-TEMPORAL DEPTH INTERPOLATION (STDI)
British Library Conference Proceedings | 2018
|Spatio-Temporal Fusion of LiDAR and Camera Data for Omnidirectional Depth Perception
Transportation Research Record | 2023
|Nonlinear Spatio-temporal Processing
British Library Conference Proceedings | 1999
|