Previous knowledge distillation (KD) methods for object detection mostly focus on feature imitation instead of mimicking the prediction logits due to its inefficiency in distilling the localization information. In this paper, we investigate whether logit mimicking always lags behind feature imitation. Towards this goal, we first present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. Second, we introduce the concept of valuable localization region that can aid to selectively distill the classification and localization knowledge for a certain region. Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and the absence of localization distillation is a critical reason for why logit mimicking underperforms for years. The thorough studies exhibit the great potential of logit mimicking that can significantly alleviate the localization ambiguity, learn robust feature representation, and ease the training difficulty in the early stage. We also provide the theoretical connection between the proposed LD and the classification KD, that they share the equivalent optimization effect. Our distillation scheme is simple as well as effective and can be easily applied to both dense horizontal object detectors and rotated object detectors. Extensive experiments on the MS COCO, PASCAL VOC, and DOTA benchmarks demonstrate that our method can achieve considerable AP improvement without any sacrifice on the inference speed. Our source code and pretrained models are publicly available at https://github.com/HikariTJU/LD.


    Access

    Download


    Export, share and cite



    Title :

    Localization Distillation for Object Detection


    Contributors:
    Zheng, Zhaohui (author) / Ye, Rongguang (author) / Hou, Qibin (author) / Ren, Dongwei (author) / Wang, Ping (author) / Zuo, Wangmeng (author) / Cheng, Ming-Ming (author)

    Publication date :

    2022


    Remarks:

    Journal extension of our previous conference paper arXiv:2102.12252


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Pre-pruned Distillation for Point Cloud-based 3D Object Detection

    Li, Fuyang / Min, Chen / Xiao, Liang et al. | IEEE | 2024


    Empowering Object Detection: Unleashing the Potential of Decoupled and Interactive Distillation

    Qian, Fulan / Hong, Jiacheng / Yan, Huanqian et al. | IEEE | 2025


    NEURAL NETWORK FOR LOCALIZATION AND OBJECT DETECTION

    PANDEY GAURAV / JAIPURIA NIKITA / NARAYANAN PRAVEEN et al. | European Patent Office | 2021

    Free access

    Neural network for localization and object detection

    PANDEY GAURAV / JAIPURIA NIKITA / NARAYANAN PRAVEEN et al. | European Patent Office | 2023

    Free access

    MULTICLASS CONFIDENCE AND LOCALIZATION CALIBRATION FOR OBJECT DETECTION

    PATHIRAJA BIMSARA / GUNAWARDHANA MALITHA / KHAN MUHAMMAD HARIS | European Patent Office | 2025

    Free access