Search

CN-121996503-A - Cross-platform data real-time monitoring and intelligent analysis method and system

CN121996503ACN 121996503 ACN121996503 ACN 121996503ACN-121996503-A

Abstract

The invention relates to the technical field of data monitoring, in particular to a cross-platform data real-time monitoring and intelligent analysis method and system, comprising the following steps: obtaining a field structure and a protocol sequence, calibrating an offset state, extracting channel numbers and continuous numbered fragments according to offset information, combining source fragment mapping path field positions, analyzing field calling sequence and processing state according to path mapping, comparing task fields and path field coverage relations, and outputting task state information. According to the invention, the field difference state is calibrated by constructing offset association of the field numbers and the structure positions, the continuous numbered sections are divided to form source fragments, the corresponding relation between the field range and the path positions is established, the coverage comparison and the difference extraction of the task fields and the path fields are completed by combining the state information, the field adaptation elasticity and the path number association are enhanced, the path logic connection is maintained in a multi-channel and structure change scene, and the dynamic judgment of the task state is supported.

Inventors

  • GAO YAO

Assignees

  • 北京外思特科技有限公司

Dates

Publication Date
20260508
Application Date
20251231

Claims (10)

  1. 1. The cross-platform data real-time monitoring and intelligent analysis method is characterized by comprising the following steps of: S1, acquiring field structure information and protocol identification content in a source channel, extracting a protocol field sequence, comparing corresponding positions of fields in the structure, identifying field numbers with misplaced fields, positioning to a field state information list, and obtaining a field offset state registration table; S2, based on the field offset state registration table, extracting a channel number and a field serial number corresponding to a field, analyzing a continuously numbered section in the sequence, splitting a field fragment according to a source number, and obtaining source field number fragment information; S3, extracting a task path field number based on a field range in the source field number fragment information, positioning a path field position corresponding to the repeated number, and mapping the path position to obtain a path field number corresponding information table; S4, based on the information table corresponding to the path field numbers, reading configuration content of a path where an associated field is located in a task scheduling table, identifying calling precedence relation of the field in the path and positioning the field sequence to obtain a path field data processing state table; s5, based on the path field data processing state table, comparing the task field with the path field number, judging whether the path field number covers the task field number, and obtaining task state information.
  2. 2. The method for real-time monitoring and intelligent analysis of cross-platform data according to claim 1, wherein the field offset state registry comprises an offset field number set, a field sequence difference identifier, and a channel field state identifier, the source field number fragment information comprises a fragment source identifier, a field number start-stop range, and a number continuous section feature, the path field number correspondence information table comprises a path field positioning number, a cross number mapping relationship, and a path structure field association, the path field data processing state table comprises a field processing sequence number, a field scheduling allocation identifier, and a path participation sequence item, and the task state information comprises a task field coverage state, a path execution integrity identifier, and a field difference set feature.
  3. 3. The method for monitoring and analyzing data in real time across platforms according to claim 1, wherein the sequence of the protocol fields refers to the arrangement sequence of the fields in the data structure according to the protocol definition; The field state information list refers to a table for registering a field position offset state.
  4. 4. The method for monitoring and analyzing data in real time across platforms according to claim 1, wherein the task schedule refers to a configuration table of management field distribution and execution sequence in a path; The configuration content of the path refers to the structure and sequence information of description fields in the task path.
  5. 5. The method for monitoring and intelligent analysis of cross-platform data in real time according to claim 1, wherein the specific steps of S1 are as follows: s101, acquiring field header information and a transmission structure format identifier in a source data channel, extracting field index and field length parameters in a structure format, and comparing field sequence with a defined sequence position in a structure to obtain a field sequence offset comparison group; S102, extracting adjacent field items in a channel field set based on offset field indexes and corresponding channel numbers in the field sequence offset comparison group, and judging whether the adjacent field items are offset continuously to obtain a continuous offset field mapping set; And S103, extracting the field and the channel position which are subjected to sequential offset based on the field index and the channel number in the continuous offset field mapping set, and correspondingly analyzing the field number and the channel information to obtain a field offset state registration table.
  6. 6. The method for monitoring and intelligent analysis of cross-platform data in real time according to claim 1, wherein the specific steps of S2 are as follows: s201, based on field number information in the field offset state registration table, identifying a data channel number corresponding to a field and a structure position index, comparing the front-back relation of the field numbers in the same channel, identifying field group segments with continuous indexes, and obtaining field continuous number segment information; s202, analyzing source identifiers of fields in channels based on field group segments in the field continuous numbering segment information, judging the sequence of the differentiated source identifier fields in the structure, and dividing the field paragraph range corresponding to the source identifiers to obtain source field sequence distribution content; S203, analyzing the start-stop position range of the field section in the channel structure based on the field section range in the source field sequence distribution content, and comparing the structure position separation condition among the source field sections to obtain source field number fragment information.
  7. 7. The method for monitoring and analyzing cross-platform data in real time according to claim 1, wherein the specific steps of S3 are as follows: S301, screening the appearance state of the number in the field range in the task path field list based on the field number range in the source field number fragment information, and comparing whether the number range is overlapped with the number between the path fields to obtain a number cross section list; S302, positioning a field index position in a corresponding task path structure based on each item number in the number cross section list, and performing position mapping on the cross number and a field site in the path structure to obtain a field path site corresponding table; S303, based on the mapping content in the field path position corresponding table, the cross information of the butt joint numbers and the field positions in the task path structure, and the positions of the fields in the path structure are designated, so that the path field number corresponding information table is obtained.
  8. 8. The method for monitoring and analyzing cross-platform data in real time according to claim 1, wherein the specific steps of S4 are as follows: S401, based on a field number set and a task path index value in the path field number corresponding information table, screening the attribution position of a field number under a task path from a task schedule table, and obtaining a field path attribution data set according to the mapping relation between the field number and the path index value; S402, analyzing the position range of the field in the path sequence based on the path attribution information of the field numbers in the field path attribution data set, and comparing the sequence relation of the field positions in the path to obtain a field path sequence relation table; S403, comparing the positions of each field in the path sequence based on the field path sequence relation table, checking the sequence, synchronizing the path index information, and obtaining a path field data processing state table.
  9. 9. The method for monitoring and analyzing cross-platform data in real time according to claim 1, wherein the specific steps of S5 are as follows: S501, extracting field numbers in a path field data processing state table based on the field numbers in the path field data processing state table and task path information, and comparing states of corresponding fields in the task path information to obtain a path missing field number set; S502, based on the field numbers in the path missing field number set, correspondingly matching with the numbers in the task field number set, and reserving the number information of the field numbers which are not covered by the path field set to obtain a path number coverage state; s503, based on the path number coverage state, reading the corresponding condition of the task field number set and the path field set, judging whether the path is in a completion state, and obtaining task state information.
  10. 10. A cross-platform data real-time monitoring and intelligent analysis system for implementing the cross-platform data real-time monitoring and intelligent analysis method according to any one of claims 1-9, the system comprising: The field structure extraction module acquires field structure information and protocol identification content in the source channel, extracts a protocol field sequence, compares corresponding positions of fields in the structure, identifies field numbers with misplaced fields, positions the field numbers to a field state information list, and obtains a field offset state registration table; The field segment disassembling module extracts a channel corresponding to a field and a field serial number based on the field number in the field offset state registration table, analyzes continuously numbered sections in the sequence, and disassembles the field segments according to the source number to obtain source field number segment information; the path field mapping module extracts a task path field number based on a field number range in the source field number fragment information, positions a path field position corresponding to the repeated number, and maps the path position to obtain a path field number corresponding information table; the field sequence positioning module reads the configuration content of the path where the associated field is located in the task scheduling table based on the field number and the path identifier in the path field number corresponding information table, identifies the calling precedence relationship of the field in the path and positions the field sequence, and obtains a path field data processing state table; And the task execution judging module compares the task field with the path field number based on the field number and the path task information in the path field data processing state table, judges whether the path field number covers the task field number, and obtains the task state information.

Description

Cross-platform data real-time monitoring and intelligent analysis method and system Technical Field The invention relates to the technical field of data monitoring, in particular to a cross-platform data real-time monitoring and intelligent analysis method and system. Background The technical field of data monitoring relates to real-time acquisition, unified standard conversion, high-speed transmission and structural processing of data with various sources, wherein core matters comprise a multi-source data access mode, a data protocol compatible strategy, a data format analysis and normalization, a streaming data caching mechanism, a continuous data verification flow and a graphical presentation method of result data, the technical field generally adopts a data adaptation scheme taking a middleware as a bridge, the technical field generally adopts a data adaptation scheme taking the middleware as a bridge to carry out format mapping and field verification on the access data, carries out asynchronous data pushing by utilizing a network transmission protocol, carries out aggregation and segmentation processing on the received data by using a timing task in background service, and finally completes data display and alarm response in a fixed-value triggering mode, the traditional cross-platform data real-time monitoring and intelligent analysis method is a technical scheme for uniformly processing and analyzing data from different operating systems or hardware architecture platforms, mainly aims at technical matters such as inconsistent communication protocols among platforms, inconsistent data formats, difficult coordination of data processing flows and the like, generally adopts a fixed format configuration file to realize data structure mapping, relies on a script-driven polling mechanism to periodically read target data source contents, carries out field-level warehousing processing on the data through a database predefined table structure, and carries out rule screening and logic judgment on the data in an analysis stage by calling single-thread processing logic or a batch reading mode based on a time window, and a visual graphical interface is rendered by virtue of a script language capable of being embedded into a page so as to present an analysis result. In the prior art, field structure mapping is performed based on a static configuration file, when a field structure has a dynamic change or a channel structure has an offset, a direct corresponding relation between a field and a task path is difficult to establish, so that a field processing sequence is fuzzy, the path mapping capability is limited, the path structure cannot be dynamically adjusted according to the field state change in the execution process, a difference recognition means is absent between a task field and a path field, the processing logic is difficult to cover all task requirements, and especially under the condition that a plurality of channels are transmitted in parallel and the field sequence is different, the problems of field mismatching, path calling confusion, task state misjudgment and the like are easily caused, and the accuracy and the response efficiency of the whole processing process are restricted. Disclosure of Invention In order to solve the technical problems in the prior art, the embodiment of the invention provides a cross-platform data real-time monitoring and intelligent analysis method; In order to achieve the above purpose, the invention adopts the following technical scheme that the cross-platform data real-time monitoring and intelligent analysis method comprises the following steps: S1, acquiring field structure information and protocol identification content in a source channel, extracting a protocol field sequence, comparing corresponding positions of fields in the structure, identifying field numbers with misplaced fields, positioning to a field state information list, and obtaining a field offset state registration table; S2, based on the field offset state registration table, extracting a channel number and a field serial number corresponding to a field, analyzing a continuously numbered section in the sequence, splitting a field fragment according to a source number, and obtaining source field number fragment information; S3, extracting a task path field number based on a field range in the source field number fragment information, positioning a path field position corresponding to the repeated number, and mapping the path position to obtain a path field number corresponding information table; S4, based on the information table corresponding to the path field numbers, reading configuration content of a path where an associated field is located in a task scheduling table, identifying calling precedence relation of the field in the path and positioning the field sequence to obtain a path field data processing state table; s5, based on the path field data processing state table, comparing the task field w