Transit riders’ feedback provided in ridership surveys, customer relationship management (CRM) channels, and, in more recent times, through social media, is key for transit agencies to better gauge the efficacy of their services and initiatives. Getting a holistic understanding of riders’ experience through the feedback shared in those instruments is often challenging, mostly because of the open-ended, unstructured nature of text feedback. In this paper, we propose leveraging traditional transit CRM feedback to develop and deploy a transit-topic-aware large language model (LLM) capable of classifying open-ended text feedback to relevant transit-specific topics. First, we utilize semi-supervised learning to engineer a training dataset of 11 broad transit topics detected in a corpus of 6 years of customer feedback provided to the Washington Metropolitan Area Transit Authority. We then use this dataset to train and thoroughly evaluate a language model based on the RoBERTa architecture. We compare our LLM—MetRoBERTa—with classical machine learning approaches utilizing keyword-based and lexicon representations. Our model outperforms those methods across all evaluation metrics, providing an average topic classification accuracy of 90%. Finally, we provide a value proposition of this work demonstrating how the language model, alongside additional text processing tools, can be applied to add structure to open-ended text sources of feedback such as Twitter. The framework and results we present provide a pathway for an automated, generalizable approach for ingesting, visualizing, and reporting transit riders’ feedback at scale, enabling agencies to better understand and improve customer experience.
MetRoBERTa: Leveraging Traditional Customer Relationship Management Data to Develop a Transit-Topic-Aware Language Model
Transportation Research Record: Journal of the Transportation Research Board
2024-02-02
Article (Journal)
Electronic Resource
English
Leveraging the General Transit Feed Specification for Efficient Transit Analysis
Online Contents | 2013
|Leveraging the General Transit Feed Specification for Efficient Transit Analysis
Transportation Research Record | 2013
|BRT TOD: Leveraging transit oriented development with bus rapid transit investments
Online Contents | 2014
|Overview: Mass Transit Customers / Automated Transit Customer
IuD Bahn | 1994
|20 US cities develop transit systems
Online Contents | 1996