The large majority of previous work on human-robot conversations in a second language has been performed with a human wizard-of-Oz. The reasons are that automatic speech recognition of non-native conversa- tional speech is considered to be unreliable and that the dialogue management task of selecting robot ut- terances that are adequate at a given turn is com- plex in social conversations. This study therefore in- vestigates if robot-led conversation practice in a sec- ond language with pairs of adult learners could po- tentially be managed by an autonomous robot. We first investigate how correct and understandable tran- scriptions of second language learner utterances are when made by a state-of-the-art speech recogniser. We find both a relatively high word error rate (41%) and that a substantial share (42%) of the utterances are judged to be incomprehensible or only partially understandable by a human reader. We then evaluate how adequate the robot utterance selection is, when performed manually based on the speech recognition transcriptions or autonomously using a) predefined sequences of robot utterances, b) a general state-of- the-art language model that selects utterances based on learner input or the preceding robot utterance, or c) a custom-made statistical method that is trained on observations of the wizard’s choices in previous con- versations. It is shown that adequate or at least ac- ceptable robot utterances are selected by the human wizard in most cases (96%), even though the ASR transcriptions have a high word error rate. Further, the custom-made statistical method performs as well as manual selection of robot utterances based on ASR transcriptions. It was also found that the interaction strategy that the robot employed, which differed re- garding how much the robot maintained the initiative in the conversation and if the focus of the conversa- tion was on the robot or the learners, had marginal effects on the word error rate and understandability of the transcriptions but larger effects on the adequate- ness of the utterance selection. Autonomous robot-led conversations may hence work better with some robot interaction strategies. ; QC 20211124


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Is a wizard-of-Oz required for robot-led conversations in a second language?


    Beteiligte:

    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629



    Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?

    Engwall, Olov / Águas Lopes, José David / Cumbal, Ronald | BASE | 2022

    Freier Zugriff

    Identification of low-engaged learners in robot-led second language conversations with adults

    Engwall, Olov / Cumbal, Ronald / Águas Lopes, José David et al. | BASE

    Freier Zugriff

    Hillman "wizard" chassis

    Engineering Index Backfile | 1932


    SWS (SNOW WIZARD SYSTEM)

    CALLAHAN THOMAS / EDMUNDS LYLE | Europäisches Patentamt | 2023

    Freier Zugriff

    IRWoZ: Constructing An Industrial Robot Wizard-of-OZ Dialoguing Dataset

    LI, Chen / Chrysostomou, Dimitrios / Zhang, Xiaochun et al. | BASE | 2023

    Freier Zugriff