This paper explores the background of constructing a railway data lake and presents a comprehensive framework for its architecture. The core components of this framework include three key elements: data ingestion standards, data ingestion strategies, and data storage. The study investigates two methods of data ingestion: physical ingestion and logical ingestion. Physical ingestion is particularly suitable for scenarios involving smaller data volumes and lower real-time requirements, while logical ingestion is designed for handling large datasets with high real-time demands and elevated security levels.The paper clearly defines six standards for data ingestion and outlines the specific workflows associated with each of the two ingestion methods. Establishing a railway data lake plays a crucial role in addressing issues such as data silos and weak data governance. Furthermore, the implementation process of the data lake is designed to integrate various technologies and management strategies, thereby enhancing the overall optimization of railway data management.By creating a data lake, the railway sector can not only improve data accessibility and interoperability but also foster a data-driven culture that promotes informed decision-making. This initiative is expected to streamline operations, enhance data quality, and ultimately contribute to the efficiency and safety of railway systems. The insights gained from this research provide a valuable reference for the future development of data management frameworks within the railway industry and other sectors facing similar challenges.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Research on the Construction and Data Ingestion Methodology of the Railway Data Lake


    Additional title:

    Lect. Notes Electrical Eng.


    Contributors:
    Liu, Jun (editor) / Yang, Jianjian (editor) / Xu, Minyi (editor) / Yu, Quan (editor) / Shen, Wenchao (editor) / Dan, Zou (author) / Peiran, Wang (author) / Jiang, Wu (author) / Siqi, Sun (author) / Li, Guohua (author)

    Conference:

    International Conference on Artificial Intelligence and Autonomous Transportation ; 2024 ; Beijing, China December 06, 2024 - December 08, 2024



    Publication date :

    2025-03-28


    Size :

    10 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English




    Research on Secure Storage Architecture of Railway Data Based on Data Lake

    Wu, Jiang / Zou, Dan / Sun, Siqi et al. | Springer Verlag | 2025


    Centralized Lake Warehouse Architecture for Railway Data Services

    Ma, Yinjun / Wang, Jiaqi / Zhou, Yubin et al. | Springer Verlag | 2025



    IRI Data Ingestion and Ionospheric Tomography

    Bust, G. S. / Coker, C. / Coco, D. S. et al. | British Library Conference Proceedings | 2001


    IRI data ingestion and ionospheric tomography

    Bust, Gary S. | Online Contents | 2001