Image Quality Assessment (IQA) and Image Aesthetic Assessment (IAA) aim to simulate human subjective perception of image visual quality and aesthetic appeal. Existing methods typically address these tasks independently due to distinct learning objectives. However, they neglect the underlying interconnectedness of both tasks, which hinders the learning of task-agnostic shared representations for human subjective perception. To confront this challenge, we propose Unified vision-language pre-training of Quality and Aesthetics (UniQA), to learn general perceptions of two tasks, thereby benefiting them simultaneously. Addressing the absence of text in the IQA datasets and the presence of textual noise in the IAA datasets, (1) we utilize multimodal large language models (MLLMs) to generate high-quality text descriptions; (2) the generated text for IAA serves as metadata to purify noisy IAA data. To effectively adapt the pre-trained UniQA to downstream tasks, we further propose a lightweight adapter that utilizes versatile cues to fully exploit the extensive knowledge of the pre-trained model. Extensive experiments demonstrate that our approach attains a new state-of-the-art performance on both IQA and IAA tasks, while concurrently showcasing exceptional zero-shot and few-label image assessment capabilities. The source code will be available at https://github.com/zht8506/UniQA.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment


    Beteiligte:
    Zhou, Hantao (Autor:in) / Tang, Longxiang (Autor:in) / Yang, Rui (Autor:in) / Qin, Guanyi (Autor:in) / Zhang, Yan (Autor:in) / Hu, Runze (Autor:in) / Li, Xiu (Autor:in)

    Erscheinungsdatum :

    2024


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark

    Zhou, Zhaokun / Wang, Qiulin / Lin, Bin et al. | ArXiv | 2024

    Freier Zugriff


    Deep Multimodality Learning for UAV Video Aesthetic Quality Assessment

    Kuang, Qi / Jin, Xin / Zhao, Qinping et al. | ArXiv | 2020

    Freier Zugriff

    Drawing a baseline in aesthetic quality assessment [10696-106]

    British Library Conference Proceedings | 2018


    Vision models for image quality assessment: one is not enough

    Bremond, R. / Tarel, J.-P. / Dumont, E. et al. | British Library Online Contents | 2010