Although transformer structure has become the defacto standard of the natural language processing task, it still has limited application in computer vision. In vision, attention is either combined with or replaces certain modules in a convolutional network, while keeping its overall framework intact. It proves that this reliance on CNNs is unnecessary for the pure transformer directly applied to image patch sequence can successfully classify the images. When large amounts of data are pre-trained and transmitted to multiple small and medium-sized image recognition benchmark (ImageNet, CIFAR-100, VTAB, etc.), the Vision Transformer (ViT) achieves remarkable results with less computational resources required than existing convolutional networks. Unlike the previous calculation that weighs attention, the model in this paper has been optimized, reducing the time complexity from O(n2) to O(n • logn), largely improving the model’s speed. Therefore, a new model is developed—Fast Vision Transformer (Fast VIT).


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Fast Vision Transformer via Query Vector Decoupling


    Contributors:
    Sun, Donghao (author) / Liu, Jiashun (author) / Liu, Jiaxing (author) / Zhang, Zixuan (author)


    Publication date :

    2021-10-20


    Size :

    1365557 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Decoupling and Vector Control of Induction Motor

    Jianrong, C. / Lie, Y. / Youbai, X. | British Library Online Contents | 2000


    Decoupling Recognition and Localization in CAD-Based Vision

    Dickinson, S. J. / Metaxas, D. / IEEE Computer Society; Technical Committee on Pattern Analysis and Machine Intelligence | British Library Conference Proceedings | 1994


    Motion estimation by decoupling rotation and translation in catadioptric vision

    Bazin, J. C. / Demonceaux, C. / Vasseur, P. et al. | British Library Online Contents | 2010


    AnchorPoint: Query Design for Transformer-Based 3D Object Detection and Tracking

    Liu, Hao / Ma, Yanni / Wang, Hanyun et al. | IEEE | 2023


    Omnidirectional decoupling annular vector tilt rotor aircraft and control method thereof

    ZHU XINNING / ZHENG XIANGMING / LU KAIJIE et al. | European Patent Office | 2024

    Free access