This paper investigates the potential of PaliGemma, a recently released open-source Visual-Language Model (VLM), for detecting phone-based driver distraction. We evaluate PaliGemma’s performance on the State Farm Distracted Driver Detection (SFDDD) dataset. The proposed method involves three steps: (1) generating a textual description of the input image using PaliGemma, (2) create prompts related to phone distraction based on the text description (e.g., “Is the driver distracted by talking or using the phone?”), and (3) identifying the driver’s phone use status (talking or texting) based on PaliGemma’s response. We compare the accuracy of three PaliGemma models for classifying four driver actions: texting/talking on the phone with left/right hand. All models achieved high accuracy (>95%) for most categories, with PaliGemma-3b-pt-896 achieving impressive 100% accuracy. Our findings suggest that PaliGemma VLMs offer promising potential for detecting phone distraction while driving.
Detection of Phone Distraction While Driving Using Open Visual-Language Models
Proceedings in Automotive Engineering
International Congress of Automotive and Transport Engineering ; 2024 ; Brasov, Romania November 06, 2024 - November 08, 2024
CONAT 2024 International Congress of Automotive and Transport Engineering ; Chapter : 24 ; 281-286
2024-11-20
6 pages
Article/Chapter (Book)
Electronic Resource
English
Analysis of visual distraction related to smartphone use while driving
Elsevier | 2025
|Predicting visual distraction using driving performance data
Tema Archive | 2010
|Predicting Visual Distraction Using Driving Performance Data
British Library Conference Proceedings | 2008
|