CN-121074745-B - Automatic equipment abnormality monitoring method and system based on image processing

CN121074745BCN 121074745 BCN121074745 BCN 121074745BCN-121074745-B

Abstract

The invention provides an automatic equipment abnormality monitoring method and system based on image processing, which relate to the technical field of automatic equipment abnormality monitoring and convert equipment operation video into multidimensional physical field characteristics, wherein in the dimension of a sports field, a full-period displacement statistical histogram and a space thermodynamic diagram are extracted based on optical flow field analysis, and the motion instability characteristic of a mechanical transmission system is accurately quantized; in the dimension of the structural field, edge gradient analysis and texture feature extraction technology are fused, a time sequence structural thermodynamic diagram sequence is constructed to capture the progressive damage evolution rule, a three-field abnormal index dynamic weighting fusion mechanism overcomes the limitation of traditional single-point monitoring, and the connected domain analysis of the fusion thermodynamic diagram is combined with LBP texture and morphological feature decision trees to realize the abnormal monitoring of the automatic equipment based on image processing.

Inventors

QI YARONG
YU TAO
YANG ZHENXING
LI WEI
LI YAN
LI ZHONG

Assignees

本钢高远实业发展有限公司

Dates

Publication Date: 20260508
Application Date: 20250814

Claims (9)

1. An automatic equipment abnormality monitoring method based on image processing is characterized by comprising the following steps: S1, decoding an operation video stream of equipment to be monitored into a continuous frame sequence, and dividing the continuous frame sequence into single-period frame sequence fragments based on a track period of a preset equipment motion reference point; S2, based on a registration key frame in a single-period frame sequence segment determined by a preset registration phase point, registering the single-period frame sequence segment to a unified coordinate system through image affine transformation to obtain a registration frame sequence segment set; S3, calculating continuous optical flow fields of all registration frame sequence fragments in the registration frame sequence fragment set, generating a motion histogram and a motion abnormality thermodynamic diagram based on the continuous optical flow fields, determining frequency spectrum monitoring points based on the continuous optical flow fields, extracting displacement signals of the frequency spectrum monitoring points, and generating a vibration energy spectrum through windowed Fourier transform; S4, extracting an LBP histogram and an edge gradient map of each frame image of the registration frame sequence segment, carrying out amplitude normalization and thermodynamic color level mapping on the edge gradient map, generating a structural anomaly thermodynamic diagram, and combining the structural anomaly thermodynamic diagram into a structural anomaly thermodynamic diagram sequence according to time sequence; S5, generating a motion abnormality index based on the motion histogram, generating a vibration abnormality thermodynamic diagram and a vibration abnormality index based on the vibration energy spectrum, and generating a structural abnormality index through spatial self-reference analysis of a structural abnormality thermodynamic diagram sequence; S6, weighting and fusing the motion abnormality thermodynamic diagram, the vibration abnormality thermodynamic diagram and the structure abnormality thermodynamic diagram based on the motion abnormality index, the vibration abnormality index and the structure abnormality index to generate a fusion thermodynamic diagram, and analyzing a connected domain positioning abnormality region of the fusion thermodynamic diagram; In step S5, the motion abnormality index is generated by calculating KL divergences of actual displacement distribution and theoretical uniform distribution characterized by the motion histogram; In step S5, the generation step of the vibration abnormality index includes: identifying the frequency corresponding to the maximum energy peak in the vibration energy spectrum, and determining the fundamental frequency of the equipment through parabolic interpolation; Determining a sensitive frequency band range based on the equipment type and the equipment fundamental frequency, calculating the proportion of energy in the sensitive frequency band range to the energy in the whole frequency band, and generating a vibration abnormality index; in step S5, the generating step of the structural abnormality index includes: dividing each frame thermodynamic diagram in the thermodynamic diagram sequence into a plurality of blocks, sequencing all pixel gradient values in each block, and taking a median value as a median value of the block; For each block, taking the median of the neighborhood blocks as a health reference standard; Calculating the absolute difference value of the median value of each block and the healthy reference standard, and dividing the absolute difference value by the standard deviation of the median value of the field block to obtain a standardized deviation degree; and marking the block with the standardized deviation exceeding a preset deviation threshold as an abnormal block, and counting the proportion of the abnormal block to generate a structural abnormality index.
2. The method for monitoring abnormality of an automated apparatus based on image processing according to claim 1, wherein in step S1, perspective projection calculation is implemented by using a CAD model of the apparatus to be monitored and camera calibration parameters, and three-dimensional trajectory feature points of a preset apparatus motion reference point are mapped to an image coordinate system.
3. The method for monitoring anomalies in an automated apparatus based on image processing according to claim 2, wherein in step S3, the step of determining the spectrum monitoring points comprises: acquiring optical flow fields of a first frame image and a second frame image of a registration frame sequence segment based on continuous inter-frame optical flow fields; selecting a preselected pixel block of which the optical flow displacement amplitude is larger than a preset displacement amplitude and the SIFT feature of the first frame image is larger than a preset feature point number; And carrying out morphological clustering on the preselected pixel blocks, and taking the centroid of the maximum connected domain as a frequency spectrum monitoring point.
4. The method for monitoring abnormality of an automated apparatus based on image processing according to claim 3, wherein in step S3, the step of generating the vibration energy spectrum includes: extracting displacement amplitude values of optical flow fields between continuous frames at the coordinates of the frequency spectrum monitoring points to form time sequence signals; Applying a hanning window function to the time sequence signal to obtain a windowed signal, and performing fast Fourier transform on the windowed signal to obtain a complex frequency spectrum; and squaring and normalizing the complex frequency spectrum modes to generate a unilateral energy distribution spectrum, namely a vibration energy spectrum.
5. The method for monitoring abnormality of an automated equipment based on image processing according to claim 4, wherein in step S4, the step of extracting the LBP histogram includes: dynamically selecting an analysis scale based on local gradient variances of each frame image of the registered frame sequence segments; capturing fine texture features by adopting a small-radius operator for the high gradient variance region; Extracting macroscopic texture features from the low gradient variance region by adopting a large-radius operator; The fine texture features and the macro texture features are fused to generate a unified texture histogram, i.e., an LBP histogram.
6. The method for monitoring abnormality of an automated apparatus based on image processing according to claim 5, wherein in step S4, the step of extracting the edge gradient map includes: converting each frame image of the registration frame sequence segment into a registration gray scale image; Applying Laplace second order differential operator to the registration gray level image to calculate and obtain an initial gradient image; And calculating a local curvature field based on the registration gray level image, performing exponential gain modulation on the initial gradient map by using the local curvature field, and obtaining the edge gradient map through normalization processing.
7. The method for monitoring abnormality of an automated apparatus based on image processing according to claim 6, wherein in step S5, the step of calculating the vibration abnormality index includes: identifying the frequency corresponding to the maximum energy peak in the vibration energy spectrum, and determining the fundamental frequency of the equipment through parabolic interpolation; a sensitive frequency band range is determined based on the device type and the device fundamental frequency, and a vibration abnormality index is generated by calculating the proportion of sensitive frequency band energy to full frequency band energy falling within the sensitive frequency band range.
8. The method for monitoring anomalies in an automated apparatus based on image processing as recited in claim 7, wherein step S6 further comprises determining a fault type by extracting LBP texture features and morphology features within the localized anomaly region.
9. An image processing-based abnormality monitoring system for an automated apparatus, which is applied to the image processing-based abnormality monitoring method for an automated apparatus according to any one of claims 1 to 8, comprising: the data segmentation module is used for decoding an operation video stream of the equipment to be monitored into a continuous frame sequence, and segmenting the continuous frame sequence into single-period frame sequence fragments based on the track period of a preset equipment motion reference point; the image registration module is used for registering the single-period frame sequence fragments to a unified coordinate system through image affine transformation based on registration key frames in the single-period frame sequence fragments determined by the preset registration phase points, so as to obtain a registration frame sequence fragment set; the motion and vibration field generation module is used for calculating continuous optical flow fields of all registration frame sequence fragments in the registration frame sequence fragment set, generating a motion histogram and a motion abnormality thermodynamic diagram based on the continuous optical flow fields, determining frequency spectrum monitoring points based on the continuous optical flow fields, extracting displacement signals of the frequency spectrum monitoring points, and generating a vibration energy spectrum through windowing Fourier transform; The structural field analysis module is used for extracting an LBP histogram and an edge gradient map of each frame image of the registration frame sequence segment, carrying out amplitude normalization and thermodynamic color level mapping on the edge gradient map, generating a structural anomaly thermodynamic diagram, and combining the structural anomaly thermodynamic diagram into a structural anomaly thermodynamic diagram sequence according to time sequence; The multi-domain anomaly quantification module is used for generating a motion anomaly index based on the motion histogram, generating a vibration anomaly thermodynamic diagram and a vibration anomaly index based on the vibration energy spectrum, and generating a structural anomaly index through spatial self-reference analysis of a structural anomaly thermodynamic diagram sequence; the anomaly fusion diagnosis module is used for carrying out weighted fusion on the motion anomaly thermodynamic diagram, the vibration anomaly thermodynamic diagram and the structure anomaly thermodynamic diagram based on the motion anomaly index, the vibration anomaly index and the structure anomaly index to generate a fusion thermodynamic diagram, and analyzing a connected domain positioning anomaly region of the fusion thermodynamic diagram.

Description

Automatic equipment abnormality monitoring method and system based on image processing Technical Field The invention relates to the technical field of automatic equipment abnormality monitoring, in particular to an automatic equipment abnormality monitoring method and system based on image processing. Background Along with the rapid development of industry and intelligent manufacturing, the health monitoring technology of automatic equipment becomes a core link for guaranteeing production safety and efficiency increasingly. The traditional equipment abnormality monitoring method mainly relies on a single sensor such as an accelerometer and an acoustic emission sensor to collect vibration or acoustic signals, and fault diagnosis is achieved through spectrum analysis or a machine learning model. However, such methods have significant limitations, such as difficulty in locating small lesions due to insufficient spatial resolution, difficulty in installing invasiveness to limit their application in complex structural devices, and difficulty in comprehensively capturing coupled fault features of motion anomalies, mechanical vibrations and structural deformations due to lack of collaborative analysis capability of multiple physical fields. The traditional vibration sensor cannot accurately position the root microcrack, and the motion instability and structural damage of the high-speed rotating part often show space-time correlation, and single signal source analysis is easy to cause misjudgment or omission. In addition, the diagnosis scheme based on the fixed threshold value or the static model is difficult to adapt to the dynamic change of the working condition of the equipment, so that the false alarm rate is high. In recent years, although the machine vision technology is introduced into the field of equipment monitoring, the existing scheme focuses on single dimension, namely, analysis of motion track abnormality only through an optical flow field or static detection of structural defects by means of thermodynamic diagram, and the core challenges of motion, vibration and structural multi-field data fusion cannot be solved. Therefore, it is necessary to provide an image processing-based method and system for monitoring abnormality of an automation device to solve the above technical problems. Disclosure of Invention In order to solve the technical problems, the invention provides an automatic equipment abnormality monitoring method and system based on image processing, which achieve the beneficial effect of fusing multi-field data to perform abnormality monitoring on equipment. The invention provides an image processing-based automatic equipment abnormality monitoring method, which comprises the following steps: S1, decoding an operation video stream of equipment to be monitored into a continuous frame sequence, and dividing the continuous frame sequence into single-period frame sequence fragments based on a track period of a preset equipment motion reference point; S2, based on a registration key frame in a single-period frame sequence segment determined by a preset registration phase point, registering the single-period frame sequence segment to a unified coordinate system through image affine transformation to obtain a registration frame sequence segment set; S3, calculating continuous optical flow fields of all registration frame sequence fragments in the registration frame sequence fragment set, generating a motion histogram and a motion abnormality thermodynamic diagram based on the continuous optical flow fields, determining frequency spectrum monitoring points based on the continuous optical flow fields, extracting displacement signals of the frequency spectrum monitoring points, and generating a vibration energy spectrum through windowed Fourier transform; S4, extracting an LBP histogram and an edge gradient map of each frame image of the registration frame sequence segment, carrying out amplitude normalization and thermodynamic color level mapping on the edge gradient map, generating a structural anomaly thermodynamic diagram, and combining the structural anomaly thermodynamic diagram into a structural anomaly thermodynamic diagram sequence according to time sequence; S5, generating a motion abnormality index based on the motion histogram, generating a vibration abnormality thermodynamic diagram and a vibration abnormality index based on the vibration energy spectrum, and generating a structural abnormality index through spatial self-reference analysis of a structural abnormality thermodynamic diagram sequence; and S6, weighting and fusing the motion abnormality thermodynamic diagram, the vibration abnormality thermodynamic diagram and the structure abnormality thermodynamic diagram based on the motion abnormality index, the vibration abnormality index and the structure abnormality index to generate a fusion thermodynamic diagram, and analyzing a connected domain positioning abnormality region of the