This paper explores the background of constructing a railway data lake and presents a comprehensive framework for its architecture. The core components of this framework include three key elements: data ingestion standards, data ingestion strategies, and data storage. The study investigates two methods of data ingestion: physical ingestion and logical ingestion. Physical ingestion is particularly suitable for scenarios involving smaller data volumes and lower real-time requirements, while logical ingestion is designed for handling large datasets with high real-time demands and elevated security levels.The paper clearly defines six standards for data ingestion and outlines the specific workflows associated with each of the two ingestion methods. Establishing a railway data lake plays a crucial role in addressing issues such as data silos and weak data governance. Furthermore, the implementation process of the data lake is designed to integrate various technologies and management strategies, thereby enhancing the overall optimization of railway data management.By creating a data lake, the railway sector can not only improve data accessibility and interoperability but also foster a data-driven culture that promotes informed decision-making. This initiative is expected to streamline operations, enhance data quality, and ultimately contribute to the efficiency and safety of railway systems. The insights gained from this research provide a valuable reference for the future development of data management frameworks within the railway industry and other sectors facing similar challenges.
Research on the Construction and Data Ingestion Methodology of the Railway Data Lake
Lect. Notes Electrical Eng.
International Conference on Artificial Intelligence and Autonomous Transportation ; 2024 ; Beijing, China December 06, 2024 - December 08, 2024
The Proceedings of 2024 International Conference on Artificial Intelligence and Autonomous Transportation ; Chapter : 52 ; 486-495
2025-03-28
10 pages
Article/Chapter (Book)
Electronic Resource
English
Research on Secure Storage Architecture of Railway Data Based on Data Lake
Springer Verlag | 2025
|Centralized Lake Warehouse Architecture for Railway Data Services
Springer Verlag | 2025
|JPL PO.DAAC automated data ingestion
NTRS | 2001
|IRI Data Ingestion and Ionospheric Tomography
British Library Conference Proceedings | 2001
|IRI data ingestion and ionospheric tomography
Online Contents | 2001
|