In Vision-and-Language Navigation (VLN), an embodied agent needs to reach a target destination with the only guidance of a natural language instruction. To explore the environment and progress towards the target location, the agent must perform a series of low-level actions, such as rotate, before stepping ahead. In this paper, we propose to exploit dynamic convolutional filters to encode the visual information and the lingual description in an efficient way. Differently from some previous works that abstract from the agent perspective and use high-level navigation spaces, we design a policy which decodes the information provided by dynamic convolution into a series of low-level, agent friendly actions. Results show that our model exploiting dynamic filters performs better than other architectures with traditional convolution, being the new state of the art for embodied VLN in the low-level action space. Additionally, we attempt to categorize recent work on VLN depending on their architectural choices and distinguish two main groups: we call them low-level actions and high-level actions models. To the best of our knowledge, we are the first to propose this analysis and categorization for VLN.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters



    Erscheinungsdatum :

    2019-01-01


    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    629



    Cognitive vision: The case for embodied perception

    Vernon, D. | British Library Online Contents | 2008


    Vision-based MAV Navigation in Underground Mine Using Convolutional Neural Network

    Mansouri, Sina Sharif / Karvelis, Petros / Kanellakis, Christoforos et al. | BASE | 2019

    Freier Zugriff

    A Convolutional Neural Network Vision System Approach to Indoor Autonomous Quadrotor Navigation

    Garcia, Adriano / Mittal, Sandeep S. / Kiewra, Edward et al. | IEEE | 2019


    MOVING: A MOdular and Flexible Platform for Embodied Visual NaviGation

    Rosano, Marco / Ragusa, Francesco / Furnari, Antonino et al. | British Library Conference Proceedings | 2023


    Multimodal Attention Networks for Low-Level Vision-and-Language Navigation

    Federico Landi / Lorenzo Baraldi / Marcella Cornia et al. | BASE | 2021

    Freier Zugriff