CN-121980352-A - Multi-mode data fusion-based auxiliary monitoring system and method for power equipment

CN121980352ACN 121980352 ACN121980352 ACN 121980352ACN-121980352-A

Abstract

The invention provides an auxiliary monitoring system and method for power equipment based on multi-mode data fusion, wherein the auxiliary monitoring system comprises three types of data including an infrared image, a sound signal and a partial discharge signal of the power equipment to be monitored, when all mode characteristics are extracted, a sound signal reliability index is used for generating weight, a time-frequency characteristic is used for generating a spatial attention mask, the infrared image characteristic is enabled to be extracted after weight adjustment, then the three types of mode characteristics are input into a shared encoder and a private encoder, mode common characteristics and all mode characteristic characteristics are separated, mode invariance and inter-mode difference constraint are synchronously applied, multi-mode weight is built according to all data reliability indexes, all characteristic characteristics are weighted and adjusted, the time sequence information fusion module is input together with the common characteristics, and a fusion state characteristic sequence is output, so that the running state of the power equipment is judged.

Inventors

CHEN XIAOGANG
RONG HAO
YANG HENGFEI
SONG HANG

Assignees

河南恩克姆科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260126

Claims (10)

1. The auxiliary monitoring system for the electric power equipment based on the multi-mode data fusion is characterized by comprising the following modules: the acquisition module is used for acquiring infrared image data, sound signal data and partial discharge signal data of the power equipment to be monitored; The generating module is used for respectively extracting the infrared image data, the sound signal data and the partial discharge signal data to obtain an infrared mode characteristic, a sound mode characteristic and a partial discharge mode characteristic, wherein the process of generating the infrared mode characteristic comprises the steps of generating sound mode reliability weight according to the reliability index of the sound signal data, generating a spatial attention mask based on the time-frequency characteristic of the sound signal data, adjusting the spatial attention mask by utilizing the sound mode reliability weight, and applying the adjusted spatial attention mask to the characteristic extracting process of the infrared image data; The input module is used for inputting the infrared mode characteristics, the sound mode characteristics and the partial discharge mode characteristics into the shared encoder and the plurality of private encoders, and separating to obtain mode common characteristics and mode specific characteristics corresponding to the mode characteristics respectively; The power equipment comprises a construction module, a time sequence information fusion module and a fusion state feature sequence, wherein the construction module is used for constructing a multi-mode reliability weight based on respective reliability indexes of the infrared image data, the sound signal data and the partial discharge signal data, carrying out weighted adjustment on the special features of each mode by utilizing the multi-mode reliability weight, inputting the common features of the modes and the special features of each mode after weighted adjustment to the time sequence information fusion module to obtain a fusion state feature sequence, and determining the running state of the power equipment based on the fusion state feature sequence.
2. The system of claim 1, wherein the generating sound modality reliability weights from the sound signal data reliability metrics comprises: Calculating the signal-to-noise ratio of the sound signal data as the reliability index; inputting the signal-to-noise ratio into an activation function to obtain a value normalized to a preset value interval As the sound modality reliability weight.
3. The system of claim 1, wherein the generating a spatial attention mask based on time-frequency characteristics of the sound signal data comprises: performing time-frequency analysis on the sound signal data to obtain a sound signal time frequency spectrum; inputting the time spectrum of the sound signal to a feature extraction network to extract low-dimensional space coding; The low-dimensional spatial code is upsampled and reshaped into a matrix that is spatially consistent with the feature map of the infrared image data as the spatial attention mask.
4. The system of claim 1, wherein the adjusting the spatial attention mask with the sound modality reliability weights and applying the adjusted spatial attention mask to the feature extraction process of the infrared image data comprises: scalar multiplication is carried out on the spatial attention mask and the sound modality reliability weight to obtain an adjusted spatial attention mask; And when the characteristic extraction network is adopted to extract the characteristic of the infrared image data, multiplying the adjusted spatial attention mask with the characteristic diagram output by the middle layer of the characteristic extraction network element by element.
5. The system of any of claims 1-4, wherein the applying a modality invariance constraint to the modality common feature comprises: adopting an countermeasure training strategy, and carrying out modal classification on the modal common characteristics through a modal discriminator; The modal discriminators are trained to maximize modal classification accuracy while the shared encoder is trained to minimize modal classification accuracy of the modal discriminators.
6. The system of any one of claims 1-4, wherein said applying a differential constraint between modality-specific features comprises: Characteristic features of different modes are combined pairwise; calculating cosine similarity or other similarity measures between the features of each combination; Minimizing the similarity measure in model training enhances the variability between modality-specific features.
7. The system of any of claims 1-4, wherein the process of constructing a multi-modal reliability weight based on the respective reliability metrics of the infrared image data, sound signal data, and partial discharge signal data comprises: Respectively calculating quality evaluation indexes of the infrared image data, the sound signal data and the partial discharge signal data as respective reliability indexes Wherein ; Reliability index of each mode Forming a reliability vector after normalization processing; processing the reliability vector by adopting a Softmax function to obtain the reliability weight corresponding to each mode The calculation formula is as follows: Ownership weight Together forming the multi-modal reliability weight.
8. The system of any one of claims 1-4, wherein the weighting adjustment of the modality specific features using the multi-modality reliability weights, inputting the modality common features and the weighted adjusted modality specific features to a timing information fusion module to obtain a fusion state feature sequence, comprises: Scalar multiplication is carried out on each weight value in the multi-mode reliability weight and the corresponding mode specific feature respectively, so that weighted mode specific features are obtained; Splicing the common modal characteristics with all weighted modal characteristic characteristics in characteristic dimension to form combined characteristics; and inputting the combined characteristics into a long-term memory network or other time sequence processing network, and taking the output of the network as the fusion state characteristic sequence.
9. The auxiliary monitoring method for the power equipment based on the multi-mode data fusion is characterized by comprising the following steps of: Acquiring infrared image data, sound signal data and partial discharge signal data of the power equipment to be monitored; Respectively extracting the infrared image data, the sound signal data and the partial discharge signal data to obtain an infrared mode feature, a sound mode feature and a partial discharge mode feature, wherein the process of generating the infrared mode feature comprises the steps of generating sound mode reliability weight according to the reliability index of the sound signal data, generating a spatial attention mask based on the time-frequency characteristic of the sound signal data, adjusting the spatial attention mask by utilizing the sound mode reliability weight, and acting the adjusted spatial attention mask on the feature extraction process of the infrared image data; Inputting the infrared mode characteristics, the sound mode characteristics and the partial discharge mode characteristics into a shared encoder and a plurality of private encoders, and separating to obtain mode common characteristics and mode specific characteristics corresponding to the mode characteristics respectively; the method comprises the steps of constructing multi-mode reliability weights based on reliability indexes of infrared image data, sound signal data and partial discharge signal data, carrying out weighted adjustment on specific characteristics of each mode by utilizing the multi-mode reliability weights, inputting common characteristics of the modes and the specific characteristics of each mode after weighted adjustment to a time sequence information fusion module to obtain a fusion state characteristic sequence, and determining the running state of the power equipment based on the fusion state characteristic sequence.
10. The method of claim 9, wherein the generating sound modality reliability weights from the sound signal data reliability metrics comprises: Calculating the signal-to-noise ratio of the sound signal data as the reliability index; inputting the signal-to-noise ratio into an activation function to obtain a value normalized to a preset value interval As the sound modality reliability weight.

Description

Multi-mode data fusion-based auxiliary monitoring system and method for power equipment Technical Field The application belongs to the field of monitoring, and particularly relates to an auxiliary monitoring system and method for power equipment based on multi-mode data fusion. Background The power equipment is the basis of stable operation of a power system, traditional power equipment state monitoring such as manual inspection, overheat faults are diagnosed by detecting temperature distribution on the surface of equipment through an infrared thermal imaging technology, and the power equipment is easily interfered by environmental factors such as weather, illumination reflection and the like and is insensitive to faults such as internal discharge, mechanical looseness and the like of non-thermal effects. Sound monitoring, particularly ultrasonic detection, can identify problems such as corona discharge, mechanical vibration and the like, but in a strong-noise industrial site, the signal-to-noise ratio of signals is low, the feature extraction is difficult, and erroneous judgment is easy to cause. Partial discharge detection is a means of evaluating the insulation state of a device, and can sensitively find early insulation defects, but is extremely sensitive to electromagnetic interference and cannot reflect the mechanical state or the overall thermal state of the device. The single-mode monitoring method has the defects of single information dimension, weak anti-interference capability, incomplete information acquisition, limited diagnosis precision and the like. The data are collected in parallel by adopting a plurality of sensors, and the state of the equipment is judged by splicing the feature layers or fusing the decision layers, so that the problems can be reduced. But the data quality and reliability of different modalities may vary, e.g. sound signals may be distorted by sudden noise and infrared images may appear as pseudo-hot spots due to abrupt illumination changes. Most of the existing methods treat the data of each mode equally, and lack of evaluation and weighting mechanisms for the reliability of the information of each mode leads to the influence of low-quality data on the fusion result. The existing fusion strategy, such as the time-frequency characteristic of a sound signal, may have a corresponding relation with the local abnormal hot spot in the infrared image in space, but the traditional method cannot utilize the characteristic of one mode to guide and optimize the characteristic extraction process of the other mode. The data of different modes not only contains common information reflecting the same state of the equipment, but also contains respective unique supplementary information, and the prior art is difficult to reliably separate and utilize the two characteristics, so that information redundancy or key information is lost, and further improvement of multi-mode fusion performance is limited. Disclosure of Invention The invention provides an auxiliary monitoring system of electric equipment based on multi-mode data fusion, which is used for solving the problem that the prior art cannot reflect the mechanical state or the whole thermal state of the equipment and lacks an evaluation and weighting mechanism for the reliability of information of each mode, and comprises the following modules: the acquisition module is used for acquiring infrared image data, sound signal data and partial discharge signal data of the power equipment to be monitored; The generating module is used for respectively extracting the infrared image data, the sound signal data and the partial discharge signal data to obtain an infrared mode characteristic, a sound mode characteristic and a partial discharge mode characteristic, wherein the process of generating the infrared mode characteristic comprises the steps of generating sound mode reliability weight according to the reliability index of the sound signal data, generating a spatial attention mask based on the time-frequency characteristic of the sound signal data, adjusting the spatial attention mask by utilizing the sound mode reliability weight, and applying the adjusted spatial attention mask to the characteristic extracting process of the infrared image data; The input module is used for inputting the infrared mode characteristics, the sound mode characteristics and the partial discharge mode characteristics into the shared encoder and the plurality of private encoders, and separating to obtain mode common characteristics and mode specific characteristics corresponding to the mode characteristics respectively; The power equipment comprises a construction module, a time sequence information fusion module and a fusion state feature sequence, wherein the construction module is used for constructing a multi-mode reliability weight based on respective reliability indexes of the infrared image data, the sound signal data and the partial discharge signal da