CN-121582756-B - Industrial visual data processing method of self-adaptive space-time window based on Netty

CN121582756BCN 121582756 BCN121582756 BCN 121582756BCN-121582756-B

Abstract

The invention discloses a Netty-based industrial visual data processing method of a self-adaptive space-time window, which relates to the technical field of industrial visual data processing. The invention effectively solves the problem of frame disorder of data transmission in the same station, remarkably reduces cross-station quality detection false detection and omission caused by network delay jitter, realizes strong self-adaptive capacity to dynamic network environment, and improves the instantaneity, stability and quality detection reliability of an industrial vision detection system.

Inventors

XUE JIAHUI
KANG MINGZHU
LAI YIJUN
WANG YING
MA JIAN
WANG BINGHAN
ZHU WEIMING

Assignees

杭州聚玻科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260127

Claims (9)

1. The industrial visual data processing method based on the Netty self-adaptive space-time window is characterized by comprising the following steps of, Establishing a data acquisition layer, and performing time stamp synchronization and structured packaging on the image data of each station at the data acquisition layer; The method comprises the steps of carrying out space-time header analysis on a data packet through a protocol decoding module, and setting a space-time constraint processor, wherein the space-time constraint processor comprises an event mode and a timing mode, and concretely comprises the steps of shunting data of different stations into independent receiving queues according to the checking validity of a 2-byte station ID, sequencing and storing the data of different stations in each receiving queue according to the sequence of a frame sequence number, and completing the time sequence consistency of the data in the same station ID; And carrying out dynamic window verification on the image data of each station to finish the industrial visual data processing of the self-adaptive space-time window.
2. The method for industrial visual data processing based on Netty adaptive space-time window of claim 1, wherein the timestamp synchronization comprises performing microsecond clock synchronization using IEEE 1588 protocol, and the structured packing comprises embedding an 8-byte timestamp, a 2-byte station ID, and a 4-byte frame number in the data packet.
3. The Netty-based adaptive space-time window industrial vision data processing method of claim 2, wherein: The event mode comprises the following technical steps: When the data arrives, notifying a network monitor to update the network index; According to the sub-queues of different stations, storing the data into the corresponding queues according to the frame numbers; taking out the head data of the queue, and obtaining a dynamic window from a dynamic window calculator; the dynamic window calculator acquires the dynamic window, which comprises the following technical steps: subtracting the absolute value of the current time from the data time stamp as a time difference; when the time difference is smaller than or equal to the time window, the verification of the time window is passed, data is pushed to TensorRT for reasoning, the verified data is transferred to the time window, and the calculation of the time window is optimized; Otherwise, if the time window verification fails, the time window verification failure problem is processed through the dynamic window calculator, and the time window calculation is optimized.
4. The method for industrial visual data processing based on Netty adaptive space-time window of claim 3 wherein the timing pattern comprises re-checking queue status for all different stations based on every fixed time interval; If the processing time difference between the current time and the previous time is more than 5 seconds and the queue size is 0, the idle state is determined, and the time window value is reset to be a basic window value; When the backlog of the number of the queues of different stations exceeds a threshold value And when the time window is in the time, recalculating a dynamic time window value, wherein the recalculating the dynamic time window value refers to a part of a dynamic time window self-adaption mechanism and is used for adjusting a time tolerance threshold according to the queue load state of the station.
5. The Netty-based adaptive space-time window industrial vision data processing method of claim 4, wherein: the network monitor is used for updating network indexes, and the updating network indexes comprises the following steps: Calculating the total transmission delay experienced by each data packet from the acquisition end to the receiving end; calculating the average degree of arrival delay change between data packets continuously arriving at the same station, namely the average value of arrival interval jitter; the self-adaptive adjustment for the dynamic time window comprises an exponential weighted moving average algorithm recommended by IETF and a calculation of the disorder rate; When the disorder rate is less than 1%, the basic safety margin is used, and when the disorder rate is more than 1%, the dynamic expansion is carried out proportionally.
6. The method for industrial visual data processing based on Netty adaptive space-time window according to claim 5, wherein the basic window value is used for reflecting the minimum buffer value under the network condition, so that the network is buffered to be 20ms; the average jitter comprises the steps of configuring a jitter amplification coefficient and acquiring a station average arrival interval jitter value of the average jitter; The disorder rate comprises a safety margin obtained based on the disorder rate and is used for coping with jitter or measurement errors; The current queue backlog comprises the steps of carrying out current queue backlog analysis based on the item of the current queue backlog condition, increasing a dynamic time window when the queue backlog is increased, allowing data to be processed, and reducing the dynamic time window when the queue backlog is reduced; The dynamic window calculator also comprises a trigger mechanism for calculating a dynamic time window, wherein the trigger mechanism comprises timing trigger and event trigger; The timing triggering is to default to 200ms at fixed time intervals to recalculate the size of the dynamic time window of all stations once, so that the dynamic time window can adapt to gradual change of network state; The event trigger comprises the following steps: when the dynamic time window fails to verify, namely under the condition of overtime, acquiring a current latest dynamic time window, and amplifying the current latest dynamic time window to 1.2 times; If the minimum value is taken as a new dynamic time window with the maximum dynamic time window, the latest dynamic time window value of the current station is recalculated.
7. The Netty-based adaptive space-time window industrial visual data processing system is characterized by comprising the following steps of, The data acquisition module establishes a data acquisition layer, and performs time stamp synchronization and structured encapsulation on the image data of each station at the data acquisition layer; The protocol analysis and processing module is used for carrying out space-time header analysis on the data packet through the protocol decoding module and setting a space-time constraint processor, wherein the space-time constraint processor comprises an event mode and a timing mode, and specifically comprises the steps of distributing data of different stations into independent receiving queues according to the checking validity of a 2-byte station ID, sequencing and storing the data in each receiving queue according to the sequence of a frame sequence number, and completing the time sequence consistency of the data in the same station ID; and the dynamic window verification module is used for carrying out dynamic window verification on the image data of each station and completing the industrial visual data processing of the self-adaptive space-time window.
8. A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that the processor realizes the steps of the industrial vision data processing method based on the Netty adaptive space-time window according to any one of claims 1-6 when executing the computer program.
9. A computer readable storage medium having a computer program stored thereon, wherein the computer program when executed by a processor implements the steps of the Netty-based adaptive space-time window industrial vision data processing method of any one of claims 1-6.

Description

Industrial visual data processing method of self-adaptive space-time window based on Netty Technical Field The invention relates to the technical field of industrial visual data processing, in particular to a Netty-based industrial visual data processing method of a self-adaptive space-time window. Background Currently, in the state of the art and the defects, high-performance industrial cameras, intelligent sensors and edge computing equipment are adopted in traditional detection hardware, and a deep learning model and a real-time operating system (RTOS) are combined in software to realize efficient processing. Although the prior art has realized high-efficiency detection, the method has the following pain points that the network dependence is strong, the edge equipment depends on a stable network, network delay or jitter causes disorder, a fixed threshold value is stiff, a traditional time window cannot adapt to a dynamic network environment, delay redundancy in a stable period and omission rate in a jitter period are high, multi-station coordination is weak, a global unified buffer strategy ignores network difference among stations, and the resource utilization rate is low. Disclosure of Invention The invention is provided in view of the problems existing in the existing Netty-based adaptive space-time window industrial visual data processing method. Therefore, the invention aims to solve the problems of missed detection, false detection and low resource utilization rate caused by disordered data, stiff fixed time window and weak multi-station cooperation caused by network delay jitter in the existing industrial vision detection system, and realize a space-time window data processing method capable of adapting to a dynamic network environment and independently adjusting according to stations. In order to solve the technical problems, the invention provides the following technical scheme: In a first aspect, an embodiment of the present invention provides a Netty-based adaptive space-time window industrial visual data processing method, which includes establishing a data acquisition layer, and performing timestamp synchronization and structured encapsulation on image data of each station in the data acquisition layer; Carrying out space-time header analysis on the data packet through a protocol decoding module, and setting a space-time constraint processor, wherein the space-time constraint processor comprises an event mode and a timing mode; And carrying out dynamic window verification on the image data of each station to finish the industrial visual data processing of the self-adaptive space-time window. As an optimal scheme of the industrial visual data processing method based on the Netty self-adaptive space-time window, the timestamp synchronization comprises the step of completing microsecond clock synchronization by adopting an IEEE 1588 protocol, and the structured encapsulation comprises the step of embedding an 8-byte timestamp, a 2-byte station ID and a 4-byte frame number into a data packet. The invention relates to a Netty-based adaptive space-time window industrial visual data processing method, which comprises the steps that space-time header analysis is carried out on a data packet through a protocol decoding module, the data of different stations are shunted into independent receiving queues according to the checking validity of a 2-byte station ID, and the interior of each receiving queue is sequenced and stored according to the sequence of frame sequence numbers, so that the time sequence consistency of the data in the same station ID is completed. As a preferable scheme of the industrial visual data processing method based on the Netty self-adaptive space-time window, the space-time constraint processor comprises an event mode and a timing mode; The event mode comprises the following technical steps: When the data arrives, notifying a network monitor to update the network index; According to the sub-queues of different stations, storing the data into the corresponding queues according to the frame numbers; taking out the head data of the queue, and obtaining a dynamic window from a dynamic window calculator; the dynamic window calculator acquires the dynamic window, which comprises the following technical steps: subtracting the absolute value of the current time from the data time stamp as a time difference; when the time difference is smaller than or equal to the time window, the verification of the time window is passed, data is pushed to TensorRT for reasoning, the verified data is transferred to the time window, and the calculation of the time window is optimized Otherwise, if the time window verification fails, the time window verification failure problem is processed through the dynamic window calculator, and the time window calculation is optimized. As a preferable scheme of the industrial visual data processing method based on the Netty self-adaptive space-time window, the timin