In the dynamic field of machine learning, foundation models have recently gained prominence, particularly for their application in natural language processing and computer vision. The foundational Segment Anything Model (SAM), known for its interactive image segmentation via prompts, serves as the basis for this study. We introduce ChangeSAM, a tailored adaptation of SAM for street scene image change detection (CD). ChangeSAM utilizes the versatility and vast knowledge of SAM, adapting it to effectively identify semantic changes in image pairs. Two architectural adaptations are introduced - Pre Decoder Fusion (PreDF) and Post Decoder Fusion (PostDF) - enabling ChangeSAM to process dual images. Enhancements through Prompt Tuning and Low-Rank Adaptation (LoRA) are integrated, achieving a balance between reusability, computational efficiency, and accuracy. Our evaluation on the VL-CMU-CD dataset shows that with minimal parameter adjustments, ChangeSAM achieves accuracy on par with fully fine-tuned models. This work contributes to the ongoing development of foundation models in practical applications, illustrating the viability and potential of adaptable, efficient models in scenarios with limited computational resources, such as intelligent vehicles.
ChangeSAM: Adapting the Segment Anything Model to Street Scene Image Change Detection
2024-09-24
2962266 byte
Conference paper
Electronic Resource
English
Adapting indexing features to scene structures for relevant image retrieval
British Library Conference Proceedings | 2003
|