CN-122019608-A - Real-time warehousing processing method and device for rail transit data and electronic equipment

CN122019608ACN 122019608 ACN122019608 ACN 122019608ACN-122019608-A

Abstract

The invention provides a real-time warehousing processing method, device and electronic equipment for rail transit data, which comprise the steps of collecting line side data generated by a rail vehicle in real time, storing the line side data into a line side distributed event stream platform, wherein the line side data comprises operation data information and operation alarm information of the rail vehicle, unidirectionally transmitting the line side data stored in the line side distributed event stream platform to a line network side distributed event stream platform based on a unidirectional transmission device, carrying out stream processing on the line side data in the line network side distributed event stream platform to obtain processed line side data, and warehousing the processed line side data. Therefore, the defects of insufficient real-time performance and low safety in rail transit data processing are overcome, and the timeliness and safety in rail transit data warehousing processing are improved.

Inventors

WEI TINGSAN
Shi Weishi
JIANG ZEXIN
LIN SHAOYONG
WU JIN

Assignees

广西交控智维科技发展有限公司

Dates

Publication Date: 20260512
Application Date: 20251231

Claims (10)

1. The real-time warehousing processing method for the rail transit data is characterized by comprising the following steps of: Line side data generated by a railway vehicle are collected in real time and stored to a line side distributed event stream platform, wherein the line side data comprise running data information and running alarm information of the railway vehicle; Transmitting the line-side data stored in the line-side distributed event stream platform to a line-network-side distributed event stream platform in a unidirectional manner based on a unidirectional transmission device; carrying out stream processing on line side data in the line network side distributed event stream platform to obtain processed line side data; and warehousing and storing the processed line side data.
2. The method for real-time warehousing of rail transit data according to claim 1, wherein before the storing of the processed line side data, the method further comprises: Classifying the processed line side data according to the data type of the processed line side data to obtain classified line side data, wherein the classified line side data at least comprises first classified line side data with a structured data type and second classified line side data with a time sequence data type; The step of warehousing and storing the processed line side data comprises the following steps: Storing the first classified line side data with the structured data type to a structured database; And storing the second classified line-side data with the time-ordered data type into a time-ordered database.
3. The method for real-time warehousing of rail traffic data according to claim 1 or 2, wherein before the streaming of line-side data in the line-side distributed event stream platform, the method further comprises: carrying out data verification on line side data in the line network side distributed event stream platform to obtain line side data after verification, wherein the data verification comprises any one or more of data validity verification and data integrity verification; The streaming processing of the line-side data in the line-side distributed event stream platform comprises the following steps: and carrying out stream processing on the data passing the line side after the verification in the line side distributed event stream platform.
4. The method of real-time warehousing of rail transit data according to claim 1 or 2, wherein prior to said storing of said line-side data to a line-side distributed event stream platform, the method further comprises: acquiring current time information; the storing the line side data to a line side distributed event stream platform includes: and under the condition that the current time information meets the preset time requirement, storing the line-side data to a line-side distributed event stream platform.
5. The method of real-time warehousing of rail transit data according to claim 1 or 2, wherein prior to said storing of said line-side data to a line-side distributed event stream platform, the method further comprises: monitoring the running state of a preset part of the railway vehicle in real time; the storing the line side data to a line side distributed event stream platform includes: and under the condition that the preset part of the railway vehicle is monitored to send out operation alarm information, storing the line-side data to a line-side distributed event stream platform.
6. The method for real-time warehousing of rail transit data according to claim 1, wherein the performing streaming processing on the line-side data in the network-side distributed event stream platform to obtain processed line-side data comprises: and based on a distributed stream processing engine, carrying out any one or more of data format standardization, data de-duplication and data filtering on the line-side data in the line-side distributed event stream platform to obtain the processed line-side data.
7. The method for real-time warehousing of rail transit data according to claim 6, wherein before the data format standardization, data deduplication and data filtering processing are performed on the line-side data in the line-side distributed event stream platform based on the distributed stream processing engine to obtain the processed line-side data, the method further comprises: acquiring the data quantity of line side data in the line network side distributed event stream platform; determining a processing rate of the distributed stream processing engine based on the data volume; the distributed stream processing engine is used for carrying out any one or more of data format standardization, data de-duplication and data filtering on line side data in the line network side distributed event stream platform to obtain processed line side data, and the method comprises the following steps: And based on a distributed stream processing engine, carrying out any one or more of data format standardization, data de-duplication and data filtering processing on line side data in the line network side distributed event stream platform according to the processing rate to obtain processed line side data.
8. A real-time warehousing processing device for rail traffic data, the device comprising: the system comprises an acquisition module, a data processing module and a data processing module, wherein the acquisition module is used for acquiring line side data generated by a railway vehicle in real time and storing the line side data to a line side distributed event stream platform, and the line side data comprises operation data information and operation alarm information of the railway vehicle; The transmission module is used for unidirectionally transmitting the line-side data stored in the line-side distributed event stream platform to the line-network-side distributed event stream platform based on a unidirectional transmission device; the processing module is used for carrying out stream processing on the line side data in the line network side distributed event stream platform to obtain processed line side data; and the warehousing module is used for warehousing and storing the processed line side data.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a real-time warehousing method of rail transit data according to any one of claims 1 to 7 when the computer program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements a real-time warehousing processing method of rail transit data according to any one of claims 1 to 7.

Description

Real-time warehousing processing method and device for rail transit data and electronic equipment Technical Field The present invention relates to the field of rail transit technologies, and in particular, to a method and an apparatus for real-time warehousing processing of rail transit data, and an electronic device. Background In a rail transit operation management system, efficient processing and accurate summarization of data are core supports for guaranteeing line safety and improving operation efficiency. Related art shows that the batch processing mode and the simple message queue transmission mode are often adopted for processing and summarizing the data of each line in the current industry. However, performing data operations in a timed or batch manner in batch mode results in data processing delays. The simple message queue transmission mode is adopted, a unidirectional isolation mechanism for data transmission is not arranged, and if a network side network has a safety problem, the stability of a line side data acquisition system can be affected reversely. Disclosure of Invention The invention provides a method, a device and electronic equipment for real-time warehousing processing of rail transit data, which are used for solving the defects of insufficient real-time performance and low safety in rail transit data processing in the prior art and realizing improvement of timeliness and safety of rail transit data warehousing processing. The invention provides a real-time warehousing processing method of rail transit data, which comprises the steps of collecting line side data generated by a rail vehicle in real time, storing the line side data into a line side distributed event stream platform, wherein the line side data comprises operation data information and operation alarm information of the rail vehicle, unidirectionally transmitting the line side data stored in the line side distributed event stream platform to a line network side distributed event stream platform based on a unidirectional transmission device, carrying out stream processing on the line side data in the line network side distributed event stream platform to obtain processed line side data, and warehousing the processed line side data. The method for real-time warehousing processing of the track traffic data comprises the steps of classifying the processed line side data according to the data type of the processed line side data to obtain classified line side data, wherein the classified line side data at least comprises first classified line side data with a structured data type and second classified line side data with a time sequence data type, and the step of warehousing the processed line side data comprises the steps of storing the first classified line side data with the structured data type into a structured database and storing the second classified line side data with the time sequence data type into the time sequence database. The method further comprises the step of carrying out data verification on the line side data in the line network side distributed event stream platform to obtain verified line side data before carrying out streaming processing on the line side data in the line network side distributed event stream platform, wherein the data verification comprises any one or more of data validity verification and data integrity verification, and the step of carrying out streaming processing on the line side data in the line network side distributed event stream platform comprises the step of carrying out streaming processing on the verified line side data in the line network side distributed event stream platform. The real-time warehousing processing method for the rail transit data, provided by the invention, further comprises the steps of acquiring current time information before the line side data are stored in the line side distributed event stream platform, and storing the line side data in the line side distributed event stream platform under the condition that the current time information is monitored to meet the preset time requirement. The real-time warehousing processing method for the rail transit data, provided by the invention, further comprises the step of monitoring the running state of the preset part of the rail vehicle in real time before the line-side data are stored in the line-side distributed event stream platform, wherein the step of storing the line-side data in the line-side distributed event stream platform comprises the step of storing the line-side data in the line-side distributed event stream platform under the condition that running alarm information is sent out by the preset part of the rail vehicle. The invention provides a real-time warehousing processing method for rail transit data, which is characterized in that line side data in a line network side distributed event stream platform is subjected to streaming processing to obtain processed line side data, and the method comp