CN-121980475-A - Multi-mode data fusion-based power equipment defect diagnosis method and system

Abstract

The invention provides a power equipment defect diagnosis method and system based on multi-mode data fusion, and relates to the technical field of power equipment detection, wherein a multi-mode original signal set and a spatial position mark set synchronously acquired by power equipment are firstly acquired, mapped to a three-dimensional space grid and a physical field component sequence is constructed; the method comprises the steps of analyzing a space-time evolution rule, extracting a defect early multi-mode precursor feature set, inputting a pre-training model to deduce future multi-mode physical field distribution, comparing predicted and actual physical field components, identifying residual error abnormal areas, judging defect types and generating a final diagnosis result. The invention can comprehensively and accurately diagnose the defects of the power equipment, discover potential problems in advance and predict the development trend.

Inventors

• LI JUNYAN
• Jiao Yinsheng
• YANG DONGYANG
• DING CHENG
• ZHANG YU
• Qiang Ziyu
• DUANMU QINGWEN
• ZHAO GUANGMING
• YANG QIZHEN
• WANG LEI
• WANG YU
• WANG LEXIAO
• KANG KE
• Lei Mingchuan
• ZHAO YINGJIU
• LIU XIAOHU

Assignees

• 四川盐源华电新能源有限公司
• 中国华电集团有限公司四川分公司
• 华电科工股份有限公司
• 四川鸿华睿橙智能电气有限公司

Dates

Publication Date

20260505

Application Date

20260407

Claims (10)

1. 1. A method for diagnosing defects of an electrical device based on multi-modal data fusion, the method comprising: Acquiring a multi-mode original signal set synchronously acquired aiming at power equipment and a space position mark set corresponding to the multi-mode original signal set, wherein the multi-mode original signal set comprises an ultrasonic original signal, an ultrahigh frequency original signal, an infrared thermal image original frame and a visible light video original frame; Mapping the multi-mode original signal set to a unified three-dimensional space grid according to the space position mark set, and respectively constructing corresponding physical field components on each three-dimensional grid node to obtain a physical field component sequence, wherein the physical field component sequence comprises ultrasonic energy density, ultrahigh frequency field intensity amplitude, infrared radiation temperature and visible light reflection intensity, and each physical field component jointly represents the multi-mode physical state of the three-dimensional grid node; performing space-time evolution rule analysis on the physical field component sequence, extracting the variation trend of each modal physical quantity on a continuous time sequence and the coupling disturbance characteristics of interaction among modalities, and generating a multi-modal precursor characteristic set in early defect; Inputting the multi-modal precursor feature set into a pre-trained defect evolution trend prediction model, and deducing multi-modal physical field distribution at future time to obtain a predicted physical field component sequence; Comparing the predicted physical field component sequence with the actually collected real-time physical field components, calculating multi-mode prediction residual errors on each three-dimensional grid node, identifying residual error abnormal areas according to the spatial distribution aggregation degree of the multi-mode prediction residual errors, judging defect types, and generating a final diagnosis result containing defect position coordinates and defect category identifiers.
2. 2. The method for diagnosing defects of a power apparatus based on multi-modal data fusion according to claim 1, wherein the mapping the multi-modal raw signal set to a unified three-dimensional space grid according to the spatial position marker set, and constructing a corresponding physical field component on each three-dimensional grid node respectively, comprises: Performing space grid subdivision processing according to the three-dimensional structure model of the power equipment, generating a three-dimensional space grid formed by a plurality of hexahedral grid units in the power equipment and in a surface area, wherein the vertex of each hexahedral grid unit is used as a three-dimensional grid node and is endowed with a unique space coordinate identifier; Analyzing the ultrasonic sensor space coordinates and beam pointing parameters in the space position mark set, carrying out energy envelope extraction processing on each ultrasonic original signal to obtain an ultrasonic instantaneous energy value at each sampling moment, reversely propagating the ultrasonic instantaneous energy value to each three-dimensional grid node of the three-dimensional space grid according to a propagation attenuation model of ultrasonic waves in a medium, calculating an ultrasonic energy density contribution value received by each three-dimensional grid node at a corresponding moment, carrying out superposition fusion on the contribution values of all the ultrasonic original signals, and generating ultrasonic energy density tensor components of each three-dimensional grid node on a continuous time sequence; Analyzing the space coordinates and antenna pattern parameters of the ultrahigh frequency sensor in the space position marker set, carrying out field intensity amplitude demodulation processing on each ultrahigh frequency original signal to obtain an ultrahigh frequency instantaneous field intensity amplitude at each sampling moment, mapping the ultrahigh frequency instantaneous field intensity amplitude to each three-dimensional grid node of the three-dimensional grid according to a radiation propagation model of the ultrahigh frequency electromagnetic wave in space, calculating a field intensity amplitude contribution value of each three-dimensional grid node at a corresponding moment, carrying out vector synthesis on the contribution values of all the ultrahigh frequency original signals, and generating an ultrahigh frequency field intensity amplitude tensor component of each three-dimensional grid node on a continuous time sequence; Analyzing the thermal infrared imager space coordinates and imaging projection matrix in the space position mark set, carrying out non-uniformity correction processing on each thermal infrared image original frame to obtain corrected infrared temperature distribution matrix, establishing a mapping relation between infrared image pixel coordinates and three-dimensional space three-dimensional grid node coordinates according to the imaging projection matrix, extracting an infrared radiation temperature value corresponding to each three-dimensional grid node from the infrared temperature distribution matrix through a bilinear interpolation method, and generating an infrared radiation temperature tensor component of each three-dimensional grid node on a continuous time sequence; Analyzing the space coordinates of a visible light camera in the space position mark set and an imaging projection matrix, carrying out illumination uniformity compensation treatment on each visible light video original frame to obtain a compensated visible light reflection intensity distribution matrix, establishing a mapping relation between the pixel coordinates of a visible light image and the coordinates of three-dimensional grid nodes of a three-dimensional space according to the imaging projection matrix, extracting visible light reflection intensity values corresponding to each three-dimensional grid node from the visible light reflection intensity distribution matrix by a bilinear interpolation method, and generating visible light reflection intensity tensor components of each three-dimensional grid node on a continuous time sequence; the ultrasonic energy density tensor component, the ultrahigh frequency field intensity amplitude tensor component, the infrared radiation temperature tensor component and the visible light reflection intensity tensor component of each three-dimensional grid node at the same moment are used as independent data channels aligned on the space position to organize, so that multichannel data representing the state of the node at the moment is formed; Recombining multi-channel data of all three-dimensional grid nodes at the same moment according to the spatial arrangement sequence of the three-dimensional grid nodes to generate a three-dimensional spatial data body with a plurality of independent channels, wherein each channel corresponds to the physical field distribution of one mode; And stacking the physical field components at a plurality of continuous moments according to a time sequence to generate the physical field component sequence containing space-time dimension information.
3. 3. The method for diagnosing defects of a power device based on multi-modal data fusion according to claim 1, wherein the performing a space-time evolution rule analysis on the physical field component sequence extracts a variation trend of each modal physical quantity on a continuous time sequence and a coupling disturbance feature of interaction between modes to generate a multi-modal precursor feature set of early defects, includes: Trend decomposition is carried out on the ultrasonic energy density time sequence of each three-dimensional grid node in the physical field component sequence, the ultrasonic energy density time sequence is decomposed into a long-term trend component, a periodic fluctuation component and a random disturbance component by adopting a local weighted regression scattered point smoothing method, and the slope change rate of the long-term trend component is extracted to be used as an ultrasonic energy accumulation rate characteristic; Performing pulse analysis processing on the ultrahigh frequency field intensity amplitude time sequence of each three-dimensional grid node in the physical field component sequence, detecting transient pulse events in the ultrahigh frequency field intensity amplitude time sequence, and counting occurrence frequency and amplitude distribution of the transient pulse events in each time window to generate ultrahigh frequency pulse activity intensity characteristics and ultrahigh frequency pulse phase distribution characteristics; Carrying out thermodynamic analysis processing on the infrared radiation temperature time sequence of each three-dimensional grid node in the physical field component sequence, calculating a first-order differential sequence and a second-order differential sequence of the infrared radiation temperature time sequence, extracting temperature rise rate peak value characteristics from the first-order differential sequence, and extracting temperature acceleration change starting point characteristics from the second-order differential sequence; Performing texture change analysis processing on the visible light reflection intensity time sequence of each three-dimensional grid node in the physical field component sequence, calculating the structural similarity index of visible light reflection intensity tensor components at adjacent moments, and extracting texture mutation moment and mutation amplitude characteristics according to the time change curve of the structural similarity index; Respectively reserving ultrasonic energy accumulation rate characteristics extracted from ultrasonic signals, ultrahigh frequency pulse activity intensity characteristics and ultrahigh frequency pulse phase distribution characteristics extracted from ultrahigh frequency signals, temperature rise rate peak value characteristics and temperature acceleration change starting point characteristics extracted from infrared signals, texture mutation moment and mutation amplitude characteristics extracted from visible light signals on each three-dimensional grid node, wherein the characteristics jointly form a multi-mode trend characteristic set of the three-dimensional grid node; Respectively carrying out space correlation analysis on trend characteristics of adjacent three-dimensional grid nodes under the same mode, respectively calculating a space covariance matrix of each three-dimensional grid node and adjacent three-dimensional grid nodes on each characteristic aiming at ultrasonic energy accumulation rate, ultrahigh frequency pulse activity intensity and infrared temperature rising rate peak characteristics, and extracting space coupling strength characteristics reflecting local region cooperative change according to characteristic value decomposition results of the space covariance matrix; calculating a time-varying cross-correlation function between the ultrasonic energy density and the ultrahigh frequency field intensity amplitude on each three-dimensional grid node, and extracting a maximum correlation coefficient and a delay time corresponding to the maximum correlation coefficient from the time-varying cross-correlation function as an ultrasonic-ultrahigh frequency coupling delay characteristic; Calculating a local gradient mutual information value between infrared radiation temperature and visible light reflection intensity on each three-dimensional grid node, and extracting a thermal-optical coupling abnormality index characteristic according to a time evolution curve of the local gradient mutual information value; the calculated spatial coupling strength characteristic reflecting the spatial cooperative change, the ultrasonic-ultrahigh frequency coupling delay characteristic reflecting the cross-modal interaction and the thermal-optical coupling abnormal index characteristic are used as an enhanced multi-modal evolution characteristic set of the node together with a multi-modal trend characteristic set of the corresponding three-dimensional grid node; Performing cluster analysis on the enhanced multi-mode evolution feature sets of all three-dimensional grid nodes, identifying three-dimensional grid node clusters showing abnormal evolution modes, taking the area covered by the three-dimensional grid node clusters as a defect precursor area, and taking the enhanced multi-mode evolution feature sets of all three-dimensional grid nodes in the defect precursor area as the early-stage multi-mode precursor feature set of the defect.
4. 4. The method for diagnosing defects of a power device based on multi-modal data fusion according to claim 1, wherein the inputting the multi-modal precursor feature set into a pre-trained defect evolution trend prediction model, deducing multi-modal physical field distribution at a future time to obtain a predicted physical field component sequence, comprises: Constructing the defect evolution trend prediction model, wherein the defect evolution trend prediction model is based on an encoder-predictor-decoder framework, the encoder consists of a space-time convolution long-short-term memory network, the predictor consists of a plurality of parallel modal prediction branches, and the decoder consists of a three-dimensional deconvolution network; reorganizing the multi-mode precursor feature set according to the spatial sequence of three-dimensional grid nodes of a three-dimensional space to generate precursor feature tensors with spatial structures, wherein each channel of the precursor feature tensors corresponds to one precursor feature type; Inputting the precursor feature tensor into the encoder, modeling the dependency relationship of the precursor feature on time and space dimensions through a gating mechanism of the space-time convolution long-short-term memory network, and extracting a hidden state tensor containing history evolution information; Inputting the hidden state tensor into each mode prediction branch of the predictor, capturing a unique evolution rule of a corresponding mode by adopting a recurrent neural network with different structures by each mode prediction branch, processing a nonlinear growth trend of ultrasonic energy accumulation by using a two-way long-short-term memory network by using an ultrasonic mode prediction branch, processing an intermittent burst mode of ultrahigh frequency pulse activity by using a gating circulation unit network by using an ultrahigh frequency mode prediction branch, processing a spatial propagation process of thermal field diffusion by using a convolution long-short term memory network by using an infrared mode prediction branch, and processing a mutation characteristic of texture change by using a self-attention network by using a visible mode prediction branch; splicing and fusing the intermediate prediction features output by each modal prediction branch on the feature dimension to obtain a comprehensive prediction feature tensor fused with multi-modal evolution information; Inputting the comprehensive prediction feature tensor into the decoder, and carrying out spatial resolution recovery and detail reconstruction on the comprehensive prediction feature through the three-dimensional deconvolution network to generate an initial prediction physical field component at a first future time; Combining the initial predicted physical field component at the first future time with the actual physical field component at the historical time to form a new input sequence, repeatedly executing the forward calculation processes of the encoder, the predictor and the decoder, and iteratively generating predicted physical field components at a plurality of subsequent future times; Carrying out mode consistency adjustment on the predicted physical field component generated at each future moment, ensuring that physical quantities of different modes at the same moment meet a preset physical constraint relation on spatial distribution, and generating an adjusted predicted physical field component; and arranging the adjusted predicted physical field components at all future moments according to a time sequence to form the predicted physical field component sequence.
5. 5. The method for diagnosing defects of a power device based on multi-modal data fusion according to claim 1, wherein comparing the predicted physical field component sequence with the real-time physical field components actually collected, calculating multi-modal predicted residuals on each three-dimensional grid node, identifying residual anomaly regions according to the spatial distribution concentration degree of the multi-modal predicted residuals, and determining defect types, and generating a final diagnosis result including defect position coordinates and defect category identifiers, comprises: at each future time instant, acquiring an actual acquired real-time physical field component corresponding to the future time instant; For each three-dimensional grid node, calculating an absolute difference value between an ultrasonic energy density value of the three-dimensional grid node in a real-time physical field component and an ultrasonic energy density predicted value of the three-dimensional grid node at a corresponding moment in a predicted physical field component sequence, and taking the absolute difference value as an ultrasonic mode residual error of the three-dimensional grid node; For each three-dimensional grid node, calculating an absolute difference value between the ultrahigh frequency field intensity amplitude of the three-dimensional grid node in the real-time physical field component and the ultrahigh frequency field intensity amplitude predicted value of the three-dimensional grid node at the corresponding moment in the predicted physical field component sequence, and taking the absolute difference value as an ultrahigh frequency modal residual error of the three-dimensional grid node; for each three-dimensional grid node, calculating an absolute difference value between an infrared radiation temperature value of the three-dimensional grid node in a real-time physical field component and an infrared radiation temperature predicted value of the three-dimensional grid node at a corresponding moment in a predicted physical field component sequence, and taking the absolute difference value as an infrared mode residual error of the three-dimensional grid node; for each three-dimensional grid node, calculating an absolute difference value between a visible light reflection intensity value of the three-dimensional grid node in a real-time physical field component and a visible light reflection intensity predicted value of the three-dimensional grid node at a corresponding moment in a predicted physical field component sequence, and taking the absolute difference value as a visible light mode residual error of the three-dimensional grid node; For each mode, performing Z-score normalization processing on the residual error of the mode by using the average value and standard deviation obtained by calculating the residual error values of all three-dimensional grid nodes of the mode under the historical normal working condition to obtain normalized mode residual error, and respectively reserving normalized ultrasonic mode residual error, ultrahigh frequency mode residual error, infrared mode residual error and visible mode residual error of each three-dimensional grid node to form a multi-mode residual error spectrum of the three-dimensional grid node; Carrying out spatial distribution analysis on normalized mode residuals of all three-dimensional grid nodes for each mode to generate a normalized residual distribution map of each mode; detecting local maxima of the normalized residual distribution diagram, and for any three-dimensional grid node, identifying the three-dimensional grid node as a candidate abnormal seed point if the residual value of the three-dimensional grid node is larger than the residual values of all adjacent three-dimensional grid nodes and the difference value between the residual value of the three-dimensional grid node and the average value of the residual values of all adjacent three-dimensional grid nodes exceeds a preset neighborhood residual threshold; taking each candidate abnormal seed point as a starting point, respectively combining residual error distribution of each mode, and incorporating three-dimensional grid nodes which are similar in surrounding residual error value with seed points in at least one mode and are spatially communicated into the same residual error abnormal region to obtain a plurality of residual error abnormal regions; For each residual error abnormal region, calculating average values of normalized mode residual errors of all three-dimensional grid nodes in the residual error abnormal region on each mode respectively to obtain multi-mode abnormal intensity spectrum of the residual error abnormal region, and calculating average values of space coordinates of all three-dimensional grid nodes in the residual error abnormal region as region center coordinates; Extracting a ratio distribution histogram between an ultrasonic mode residual error and an ultrahigh frequency mode residual error of a three-dimensional grid node in each residual error abnormal region, matching the shape characteristic of the ratio distribution histogram with a preset defect type template library, and taking the defect type with the highest matching degree as a defect type identifier of the residual error abnormal region; And taking the region center coordinates of the residual error abnormal region with the region abnormal intensity exceeding a preset threshold value in the multi-mode abnormal intensity spectrum as defect position coordinates, and associating and combining the defect type identification of the residual error abnormal region with the defect position coordinates to generate the final diagnosis result containing the defect position coordinates and the defect type identification.
6. 6. The method for diagnosing defects of a power device based on multi-modal data fusion according to claim 5, wherein extracting a ratio distribution histogram between an ultrasonic modal residual and an ultrahigh frequency modal residual of three-dimensional grid nodes in each residual anomaly region, matching the shape feature of the ratio distribution histogram with a preset defect type template library, and using the defect type with the highest matching degree as the defect type identifier of the residual anomaly region comprises: For each residual error abnormal region, collecting ultrasonic mode residual error values of all three-dimensional grid nodes in the residual error abnormal region to form an ultrasonic residual error set, and collecting ultrahigh frequency mode residual error values of all three-dimensional grid nodes in the residual error abnormal region to form an ultrahigh frequency residual error set; element pairing is carried out on the ultrasonic residual error set and the ultrahigh frequency residual error set, so that each three-dimensional grid node corresponds to a pair of ultrasonic residual values and ultrahigh frequency residual values; Calculating the ratio of the ultrasonic residual value to the ultrahigh frequency residual value on each three-dimensional grid node to obtain the residual ratio of the three-dimensional grid node, and forming a residual ratio set of the residual abnormal region; carrying out statistical distribution analysis on the residual ratio set, dividing the value range of the residual ratio into a plurality of continuous equal-width intervals, counting the number of the residual ratio falling into each interval, and generating a residual ratio distribution histogram with the interval as an abscissa and the frequency as an ordinate; Carrying out normalization processing on the residual ratio distribution histogram, dividing the frequency of each interval by the total node number to obtain a normalized residual ratio probability density histogram; Loading a plurality of reference ratio distribution templates corresponding to defect types from a preset defect type template library, wherein each reference ratio distribution template is a normalized probability density histogram which is obtained by pre-calculating according to the same method on a sample area of known defect types; calculating the pasteurization distance between the residual ratio probability density histogram and each reference ratio distribution template to obtain a plurality of pasteurization distance values, wherein the smaller the pasteurization distance is, the more similar the two distributions are; taking the defect type corresponding to the reference ratio distribution template with the minimum Babbitt distance as a candidate defect type; Further calculating a correlation coefficient between the residual ratio probability density histogram and a candidate defect type reference ratio distribution template, and if the correlation coefficient is larger than a preset similarity threshold, confirming that the candidate defect type is a defect type identification of the residual abnormal region; And if the correlation coefficient is not greater than the similarity threshold, selecting a reference ratio distribution template with the next smaller Babbitt distance, repeating the correlation coefficient calculation until a defect type meeting the requirement of the correlation coefficient threshold is found, and if the correlation coefficients of all the reference ratio distribution templates do not meet the requirement, marking the defect type identification of the residual error abnormal region as an unknown type.
7. 7. The method for diagnosing a defect of a power device based on multi-modal data fusion according to claim 3, wherein the performing cluster analysis on the enhanced multi-modal evolution characteristics of all three-dimensional grid nodes, identifying a three-dimensional grid node cluster showing an abnormal evolution mode, taking a region covered by the three-dimensional grid node cluster as a defect precursor zone, and taking an enhanced multi-modal evolution characteristic set of all three-dimensional grid nodes in the defect precursor zone as the multi-modal precursor characteristic set of early defect, comprises: Combining enhanced multi-mode evolution feature vectors of all three-dimensional grid nodes into a point set in a high-dimensional feature space, wherein each point corresponds to one three-dimensional grid node, and coordinates of the points are jointly formed by ultrasonic energy accumulation rate features, ultrahigh frequency pulse activity intensity features, ultrahigh frequency pulse phase distribution features, temperature rise rate peak features, temperature acceleration change starting point features, texture mutation moment and mutation amplitude features, space coupling intensity features, ultrasonic-ultrahigh frequency coupling delay features and thermal-optical coupling abnormality index features of the three-dimensional grid nodes; Performing density-based spatial clustering operation on a point set in the high-dimensional feature space, setting a neighborhood radius parameter and a minimum neighborhood point parameter, identifying core points by traversing each point and calculating the number of adjacent points contained in the neighborhood radius, and connecting mutually reachable core points and boundary points in the adjacent points into clusters to generate a plurality of initial feature cluster clusters; Carrying out space continuity verification on each initial feature cluster, extracting space coordinates of all three-dimensional grid nodes contained in the initial feature cluster, constructing a space adjacency graph of the initial feature cluster, checking whether isolated nodes or fragmented areas exist in the space adjacency graph, if so, eliminating the isolated nodes or the fragmented areas from the current initial feature cluster, and recalculating space areas corresponding to the eliminated initial feature clusters to obtain candidate feature clusters with continuous space; for each candidate feature cluster, calculating the mean value vector of all three-dimensional grid node enhanced multi-mode evolution feature vectors in the candidate feature cluster as a cluster center, calculating the Euclidean distance between each three-dimensional grid node feature vector and the cluster center, and taking the square sum of all Euclidean distances as a cluster compactness index of the candidate feature cluster; Calculating the mean value point of the space coordinates of all three-dimensional grid nodes in each candidate feature cluster as a cluster space center, calculating the space distance between each three-dimensional grid node and the cluster space center, and taking the square sum of all the space distances as a space condensation index of the candidate feature cluster; Calculating the comprehensive anomaly score of each candidate feature cluster according to the weighted sum of the intra-cluster compactness index and the spatial aggregation index, wherein the weighting coefficient is preset according to the importance of feature similarity and spatial aggregation in the historical defect sample; Judging candidate feature cluster with the comprehensive anomaly score exceeding a preset dynamic threshold as an anomaly evolution mode cluster, wherein the preset dynamic threshold is adaptively determined according to the statistical distribution of the comprehensive anomaly scores of all the current candidate feature cluster; for each abnormal evolution mode cluster, analyzing the time evolution track of the characteristic vector of the three-dimensional grid node in the cluster, backtracking and extracting the change curve of the physical quantity of each mode of the three-dimensional grid node in the past continuous time period from a physical field component sequence, calculating the first derivative and the second derivative of each curve in the time dimension, identifying the starting moment of inflection points or acceleration changes of the change curve, and taking the earliest inflection point or acceleration change starting moment as the defect precursor starting time of the abnormal evolution mode cluster; marking a corresponding space region and a time window on the three-dimensional space grid according to the cluster space center coordinates of each abnormal evolution mode cluster and the defect precursor starting time, taking the marked space region as a defect precursor region, and organizing an enhanced multi-mode evolution feature set of all three-dimensional grid nodes in the defect precursor region into a data structure with space-time labels to serve as the multi-mode precursor feature set in early defect; And carrying out similarity retrieval on the multi-mode precursor feature set and a historical defect case library of the power equipment, calculating the mahalanobis distance between the current precursor feature set and each historical case precursor feature set, selecting the first K historical cases with the minimum mahalanobis distance and the corresponding final defect types thereof as reference defect type candidates of the current defect early stage, and adding the reference defect type candidates into the multi-mode precursor feature set.
8. 8. The method for diagnosing defects of a power unit based on multi-modal data fusion according to claim 1, wherein after the step of comparing the predicted physical field component sequence with the real-time physical field components actually collected, calculating multi-modal prediction residuals on each three-dimensional grid node, identifying residual anomaly regions according to the spatial distribution concentration degree of the multi-modal prediction residuals, and determining defect types, and generating a final diagnosis result including defect location coordinates and defect category identifications, the method further comprises: extracting ultrasonic original signal fragments and ultrahigh frequency original signal fragments corresponding to space positions from the multi-mode original signal set according to defect position coordinates in the final diagnosis result, performing deconvolution processing on the ultrasonic original signal fragments, removing the repeated reflection aliasing effect of the ultrasonic signals on a propagation path, recovering waveform characteristics of original acoustic emission signals at a defect source, and obtaining defect source ultrasonic waveforms; performing time-frequency analysis on the defect source ultrasonic waveform, extracting energy distribution of the defect source ultrasonic waveform on different frequency components, and constructing an ultrasonic fingerprint feature vector representing a defect microscopic motion mode according to the number of main frequency peaks in the energy distribution and the proportional relation between the main frequencies; Carrying out phase analysis processing on the ultrahigh frequency original signal segments, synchronously dividing the ultrahigh frequency original signal segments according to power frequency periods, extracting the amplitude of an ultrahigh frequency signal at equal phase intervals in each power frequency period, generating a phase resolution amplitude matrix of the ultrahigh frequency signal in the power frequency period, carrying out singular value decomposition on the phase resolution amplitude matrix, extracting a first singular vector reflecting discharge phase stability and a second singular vector reflecting discharge amplitude fluctuation, and combining the first singular vector and the second singular vector into an ultrahigh frequency fingerprint feature vector representing a defect discharge mode; Extracting an infrared thermal image original frame sequence corresponding to a space position from the multi-mode original signal set according to the defect position coordinates, carrying out dynamic modeling on a temperature change curve of a region where a defect is located in the infrared thermal image original frame sequence, calculating a second derivative zero crossing point moment of temperature change along with time, taking the zero crossing point moment as a heat source energy release rate turning point characteristic, and constructing an infrared fingerprint characteristic vector representing the thermal characteristic of the defect by combining the contrast ratio of a temperature field of the defect region and a surrounding normal region temperature field; Extracting a visible light video original frame sequence corresponding to a space position from the multi-mode original signal set according to the defect position coordinates, performing fractal dimension calculation on a texture image of a region where a defect is located in the visible light video original frame sequence to obtain a fractal dimension change curve of the texture image under different scales, extracting scale parameters and dimension parameters corresponding to curve slope mutation points from the fractal dimension change curve, and constructing a visible light fingerprint feature vector representing the evolution of the defect surface morphology; Inputting the ultrasonic fingerprint feature vector, the ultrahigh frequency fingerprint feature vector, the infrared fingerprint feature vector and the visible light fingerprint feature vector into a pre-constructed multi-mode fingerprint fusion network, performing reduction and reconstruction on each mode fingerprint feature through a self-encoder structure of the multi-mode fingerprint fusion network, and calculating a low-dimensional hidden space vector with the minimum reconstruction error as a unified microscopic fingerprint code of a defect; Performing similarity matching on the unified microscopic fingerprint codes of the defects and all mechanism fingerprint codes in a prestored defect physical mechanism library, wherein the defect physical mechanism library comprises standard microscopic fingerprint codes corresponding to different defect types in different development stages, and determining the microscopic evolution mechanism type of the current defects according to the defect physical mechanism corresponding to the mechanism fingerprint code with the highest matching degree, wherein the microscopic evolution mechanism type comprises an electric tree channel step-by-step growth mechanism, a partial discharge induced hot spot formation mechanism or a fatigue cracking mechanism caused by mechanical vibration; according to the microscopic evolution mechanism type, a corresponding defect development dynamics equation is called from the defect physical mechanism library, wherein the defect development dynamics equation comprises a differential equation set describing the change of key feature quantity of the defect along with time; Taking partial components in the unified microscopic fingerprint code as initial state parameters of the defect development kinetic equation, carrying out numerical integration solution on the differential equation set in a time domain, and deducing a size expansion curve, a discharge intensity evolution curve and a temperature rise curve of the defect on a future time sequence; and carrying out consistency comparison on the deduced defect size expansion curve, the discharge intensity evolution curve and the temperature rise curve and the change trend of the corresponding physical quantity in the predicted physical field component sequence obtained by the defect evolution trend prediction model, calculating the dynamic time warping distance between multiple curves, and outputting the deduced curve as a complementary prediction result of the defect development track and guiding the subsequent operation and maintenance decision together with the final diagnosis result if the dynamic time warping distance is smaller than a preset matching threshold value.
9. 9. A multi-modal data fusion-based power equipment defect diagnosis system, comprising: A processor; a machine-readable storage medium storing machine-executable instructions for the processor; Wherein the processor is configured to perform the multi-modal data fusion-based power device defect diagnosis method of any one of claims 1 to 8 via execution of the machine-executable instructions.
10. 10. A computer program product, characterized in that the computer program product comprises machine executable instructions stored in a computer readable storage medium, from which a processor of a multi-modal data fusion based power device defect diagnosis system reads the machine executable instructions, which processor executes the machine executable instructions such that the multi-modal data fusion based power device defect diagnosis system performs the multi-modal data fusion based power device defect diagnosis method according to any one of claims 1 to 8.

Description

Multi-mode data fusion-based power equipment defect diagnosis method and system Technical Field The invention relates to the technical field of power equipment detection, in particular to a power equipment defect diagnosis method and system based on multi-mode data fusion. Background In operation and maintenance of power equipment, a conventional power equipment defect diagnosis method mainly relies on a single-mode detection signal, for example, only one mode of ultrasonic detection, ultrahigh frequency detection, infrared thermal image detection or visible light detection is used for acquiring equipment state information. However, there are significant limitations to single modality detection signals. Although the ultrasonic detection can effectively detect defects such as partial discharge and the like in equipment, the detection capability of the ultrasonic detection on defects such as abnormal temperature and the like on the surface of the equipment is limited, the ultrahigh frequency detection is sensitive to the partial discharge signals and is difficult to comprehensively reflect the whole physical state of the equipment, the infrared thermal image detection can intuitively display the temperature distribution of the equipment, but can not effectively identify some non-heating defects, and the visible light detection can provide an appearance image of the equipment and is difficult to detect potential defects in the equipment. In addition, most of the existing diagnosis methods only analyze the detection data at the current moment, lack excavation of early precursor features of defects and prediction of future development trend of the defects, are difficult to discover and take measures in time when the defects just appear, often wait until the defects develop to a serious degree and increase the risk of equipment faults and maintenance cost. Disclosure of Invention In view of the above-mentioned problems, in combination with a first aspect of the present invention, the present invention provides a method for diagnosing defects of an electrical device based on multi-modal data fusion, the method comprising: Acquiring a multi-mode original signal set synchronously acquired aiming at power equipment and a space position mark set corresponding to the multi-mode original signal set, wherein the multi-mode original signal set comprises an ultrasonic original signal, an ultrahigh frequency original signal, an infrared thermal image original frame and a visible light video original frame; Mapping the multi-mode original signal set to a unified three-dimensional space grid according to the space position mark set, and respectively constructing corresponding physical field components on each three-dimensional grid node to obtain a physical field component sequence, wherein the physical field component sequence comprises ultrasonic energy density, ultrahigh frequency field intensity amplitude, infrared radiation temperature and visible light reflection intensity, and each physical field component jointly represents the multi-mode physical state of the three-dimensional grid node; performing space-time evolution rule analysis on the physical field component sequence, extracting the variation trend of each modal physical quantity on a continuous time sequence and the coupling disturbance characteristics of interaction among modalities, and generating a multi-modal precursor characteristic set in early defect; Inputting the multi-modal precursor feature set into a pre-trained defect evolution trend prediction model, and deducing multi-modal physical field distribution at future time to obtain a predicted physical field component sequence; Comparing the predicted physical field component sequence with the actually collected real-time physical field components, calculating multi-mode prediction residual errors on each three-dimensional grid node, identifying residual error abnormal areas according to the spatial distribution aggregation degree of the multi-mode prediction residual errors, judging defect types, and generating a final diagnosis result containing defect position coordinates and defect category identifiers. In still another aspect, the present invention further provides a power equipment defect diagnosis system based on multi-mode data fusion, including: The system comprises a processor, a machine-readable storage medium for storing machine-executable instructions of the processor, wherein the processor is configured to perform the above-described multi-modal data fusion-based power device defect diagnosis method via execution of the machine-executable instructions. In still another aspect, the present invention further provides a computer program product including machine executable instructions stored in a computer readable storage medium, from which a processor of a multi-modal data fusion based power device defect diagnosis system reads the machine executable instructions, the processor executing