In order to operate in human environments, a robot’s semantic perception has to overcome open-world challenges such as novel objects and domain gaps. Autonomous deployment to such environments therefore requires robots to update their knowledge and learn without supervision. We investigate how a robot can autonomously discover novel semantic classes and improve accuracy on known classes when exploring an unknown environment. To this end, we develop a general framework for mapping and clustering that we then use to generate a self-supervised learning signal to update a semantic segmentation model. In particular, we show how clustering parameters can be optimized during deployment and that fusion of multiple observation modalities improves novel object discovery compared to prior work. Models, data, and implementations can be found at github.com/hermannsblum/scim.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    SCIM: Simultaneous Clustering, Inference, and Mapping for Open-World Semantic Scene Understanding


    Additional title:

    Springer Proceedings in Advanced Robotics


    Contributors:

    Conference:

    The International Symposium of Robotics Research ; 2022 ; Geneva, Switzerland September 25, 2022 - September 30, 2022


    Published in:

    Robotics Research ; Chapter : 9 ; 119-135


    Publication date :

    2023-03-08


    Size :

    17 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English




    Rainy Night Scene Understanding With Near Scene Semantic Adaptation

    Di, Shuai / Feng, Qi / Li, Chun-Guang et al. | IEEE | 2021


    Indoor Scene Understanding with Geometric and Semantic Contexts

    Choi, W. / Chao, Y. W. / Pantofaru, C. et al. | British Library Online Contents | 2015



    Understanding Road Scene Situation and Semantic Representation of Road Scene Situation for Reliable Sharing

    YALLA VEERAGANESH / PARUNDEKAR RAHUL RAVI / PILLAI PREETI J et al. | European Patent Office | 2017

    Free access

    Semantic Segmentation for Traffic Scene Understanding Based on Mobile Networks

    Hao, Peihan / Chen, Sihan / Bai, Jie et al. | British Library Conference Proceedings | 2018