As an alternative to expensive expert evaluation, Image Aesthetic Assessment (IAA) stands out as a crucial task in computer vision. However, traditional IAA methods are typically constrained to a single data source or task, restricting the universality and broader application. In this work, to better align with human aesthetics, we propose a Unified Multi-modal Image Aesthetic Assessment (UNIAA) framework, including a Multi-modal Large Language Model (MLLM) named UNIAA-LLaVA and a comprehensive benchmark named UNIAA-Bench. We choose MLLMs with both visual perception and language ability for IAA and establish a low-cost paradigm for transforming the existing datasets into unified and high-quality visual instruction tuning data, from which the UNIAA-LLaVA is trained. To further evaluate the IAA capability of MLLMs, we construct the UNIAA-Bench, which consists of three aesthetic levels: Perception, Description, and Assessment. Extensive experiments validate the effectiveness and rationality of UNIAA. UNIAA-LLaVA achieves competitive performance on all levels of UNIAA-Bench, compared with existing MLLMs. Specifically, our model performs better than GPT-4V in aesthetic perception and even approaches the junior-level human. We find MLLMs have great potential in IAA, yet there remains plenty of room for further improvement. The UNIAA-LLaVA and UNIAA-Bench will be released.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark


    Beteiligte:
    Zhou, Zhaokun (Autor:in) / Wang, Qiulin (Autor:in) / Lin, Bin (Autor:in) / Su, Yiwei (Autor:in) / Chen, Rui (Autor:in) / Tao, Xin (Autor:in) / Zheng, Amin (Autor:in) / Yuan, Li (Autor:in) / Wan, Pengfei (Autor:in) / Zhang, Di (Autor:in)

    Erscheinungsdatum :

    2024


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Drawing a baseline in aesthetic quality assessment [10696-106]

    British Library Conference Proceedings | 2018


    UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment

    Zhou, Hantao / Tang, Longxiang / Yang, Rui et al. | ArXiv | 2024

    Freier Zugriff

    Human running detection: Benchmark and baseline

    Lao, Shihong / Wang, Dong / li, Fu et al. | British Library Online Contents | 2016


    Human running detection: Benchmark and baseline

    Lao, Shihong / Wang, Dong / li, Fu et al. | British Library Online Contents | 2016