The transformer architecture based on self-attention offers a versatile structure which has led to the definition of multiple deep learning models for various tasks or applications of natural language processing. The purpose of this work is to analyze two language models for training bidirectional encoders like BERT: the Masked Language Model (MLM) and the Conditional Masked Language Model (CMLM) for learning sentence embeddings. How sentence-level representations impact the task of sequence classification is the main focus of interest in our investigation: is there any significant difference in quality between these two pretrained language models? We evaluate via fine-tuning these pretrained models on a downstream task as sequence classification.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Fine-Tuning of BERT models for Sequence Classification


    Beteiligte:
    Uribe, Diego (Autor:in) / Cuan, Enrique (Autor:in) / Urquizo, Elisa (Autor:in)


    Erscheinungsdatum :

    05.12.2022


    Format / Umfang :

    358821 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    BERT for Aviation Text Classification

    Jing, Xiao / Chennakesavan, Akul / Chandra, Chetan et al. | TIBKAT | 2023


    BERT for Aviation Text Classification

    Jing, Xiao / Chennakesavan, Akul / Chandra, Chetan et al. | AIAA | 2023



    Fine tuning mechanical design

    McCormick,D. | Kraftfahrwesen | 1981


    Platform fine-tuning mechanism

    MA HAIHUA / GE CHUNMING / GU HAIYANG et al. | Europäisches Patentamt | 2020

    Freier Zugriff