Data quality is essential for its authentic usage in analysis and applications. The large volume of automated collection data inevidently suffers from data quality issues including data missing and invalidity. This paper deals with an invalid data problem in the automated fare collection (AFC) database caused by the erroneous association between the fare machines and metro stations, e.g., a fare machine located at Station A is wrongly associated with Station B in the AFC database. It could lead to inappropriate fare charges in a distance-based fare system and cause analysis bias for planning/operation practice. We propose a tensor decomposition and isolation forest-based approach to detect and correct the invalid associated fare machines in the system. The tensor decomposition extracts features of passenger flows and travel times passing through fare machines. The isolation forest coupled with a neural network (NN) takes these features as inputs to detect the wrongly associated fare machines and infer the correct association stations. Case studies using data from a metro system show that the proposed detection approach achieves over 90% accuracy in detecting the invalid associations for up to 35% invalid associations. The inferred association has a 90% accuracy even when the invalid association ratio reaches 40%. The proposed data-driven invalid data detection method is useful for large-scale data management in terms of data quality check and fix.


    Access

    Download


    Export, share and cite



    Title :

    Detecting Invalid Associations between Fare Machines and Metro Stations Using Smart Card Data


    Contributors:


    Publication date :

    2021




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Recovering the Association Between Unlinked Fare Machines and Stations Using Automated Fare Collection Data in Metro Systems

    Zhang, Pengfei / Ma, Zhenliang / Weng, Xiaoxiong et al. | Transportation Research Record | 2021


    Moscow Metro smart card

    Marchenko, A. | British Library Conference Proceedings | 2002



    Stochastic bus traffic modelling and validation using smart card fare collection data

    Ivanchev, Jordan / Aydt, Heiko / Knoll, Alois | IEEE | 2014


    Recognizing metro-bus transfers from smart card data

    Zhao, De / Wang, Wei / Li, Chenyang et al. | Taylor & Francis Verlag | 2019