For robots to be generally useful, they must be able to find arbitrary objects described by people (i.e., be language-driven) even without expensive navigation training on in-domain data (i.e., perform zero-shot inference). We explore these capabilities in a unified setting: language-driven zero-shot object navigation (L-ZSON). Inspired by the recent success of open-vocabulary models for image classification, we investigate a straightforward framework, CLIP on Wheels (CoW), to adapt open-vocabulary models to this task without fine-tuning. To better evaluate L-ZSON, we introduce the Pasture benchmark, which considers finding uncommon objects, objects described by spatial and appearance attributes, and hidden objects described relative to visible objects. We conduct an in-depth empirical study by directly deploying 21 CoW baselines across Habitat, RoboTHOR, and Pasture. In total, we evaluate over 90k navigation episodes and find that (1) CoW baselines often struggle to leverage language descriptions, but are proficient at finding uncommon objects. (2) A simple CoW, with CLIP-based object localization and classical exploration -- and no additional training -- matches the navigation efficiency of a state-of-the-art ZSON method trained for 500M steps on Habitat MP3D data. This same CoW provides a 15.6 percentage point improvement in success over a state-of-the-art RoboTHOR ZSON model.


    Access

    Download


    Export, share and cite



    Title :

    CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation


    Contributors:

    Publication date :

    2022


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Training for Inter-Rater Reliability: Baselines and Benchmarks

    Williams, D. M. / Holt, R. W. / Boehm-Davis, D. A. | British Library Conference Proceedings | 1997



    GPS-based Relative Navigation of LEO formations with Varying Baselines

    Tancredi, Urbano / Renga, Alfredo / Grassi, Michele | AIAA | 2010


    GPS-Based Relative Navigation of LEO Formations with Varying Baselines

    Tancredi, U. / Renga, A. / Grassi, M. et al. | British Library Conference Proceedings | 2010


    Agrivoltaics to shade cows in a pasture-based dairy system

    Heins, Bradley J. / Sharpe, Kirsten T. / Buchanan, Eric S. et al. | TIBKAT | 2022