The invention discloses a method for analyzing portal transaction data in real time based on streaming computation, which comprises the following steps of: consuming kafka real-time portal transaction data, and grouping according to a portal; real-time vehicle passing data; performing portal transaction volume statistics: respectively performing statistics on the flow of the portal in each hour and each day by continuously adding 1 in a corresponding time interval; road section transaction volume statistics: when traffic volume statistics is carried out on a road section, de-weighting operation needs to be carried out, one vehicle passes through three gantries of one road section, and the traffic volume of the road section can only be 1; and carrying out transaction volume statistics. The method has the beneficial effects that the requirements of solving the data volume problem, the real-time problem, the accuracy problem and the problem that the data capacity needs to be solved for calculating the traffic flow are met by adopting a streaming calculation + kv storage mode, the intermediate state is continuously stored in the data processing process, and the processed data position and result are stored at a certain time point, so that the data processing efficiency is improved. Inaccurate statistical results caused by missed calculation or recalculation of system faults are prevented, and functions are completed with fewer servers.
本发明公开了一种基于流式计算实时分析门架交易数据方法,包括消费kafka实时门架交易数据,按照门架进行分组;实时过车数据;门架交易量统计:分别统计门架每小时和每天的流量,通过在相应的时间区间不断加1完成;路段交易量统计:路段在统计通行量时,需要做去重操作,一辆车通过一个路段的3个门架,路段的通行量只能算1个;关于交易量的统计。本发明有益之处在于:采用流式计算+kv存储方式来实现解决数据量问题、实时问题、准确性问题,计算通行流量需要解决数据容量的问题的需求,在数据处理过程中不断保存中间状态,在某个时间点,把处理的数据位置和结果都保存下,防止系统故障漏算或者重算导致的统计结果不准确,用较少的服务器完成功能。
Method for analyzing portal transaction data in real time based on streaming computation
一种基于流式计算实时分析门架交易数据方法
2023-03-03
Patent
Electronic Resource
Chinese
Congestion analyzing method and equipment based on portal passing data
European Patent Office | 2021
|REAL TIME STREAMING ANALYTICS FOR FLIGHT DATA PROCESSING
European Patent Office | 2019
|Real-Time Car Sharing and Car Rental Portal
Trans Tech Publications | 2023
|SYSTEMS AND METHODS FOR REAL-TIME STREAMING OF FLIGHT DATA
European Patent Office | 2019
|Systems and methods for real-time streaming of flight data
European Patent Office | 2021
|