CN-122023978-A - Infrared-visible light image fusion method based on deformation field, storage medium and electronic equipment

CN122023978ACN 122023978 ACN122023978 ACN 122023978ACN-122023978-A

Abstract

The invention relates to an infrared-visible light image fusion method based on a deformation field, a storage medium and electronic equipment, which comprise the steps of collecting an infrared image and a visible light image of a high-voltage switch cabinet; the method comprises the steps of adopting a cross-modal sensing style migration network in a multi-modal image registration model to process a visible light image to generate a pseudo infrared image, inputting the pseudo infrared image and the infrared image into a multi-level refinement registration network of the model to generate a distortion displacement vector deformation field, carrying out registration reconstruction on the infrared image to obtain a registered infrared image, respectively inputting the registered infrared image and the registered visible light image into a feature extraction network to extract features, and adopting a multi-modal image fusion network to fuse the features to obtain a fusion image output. According to the method, the deformation field is generated through the pseudo infrared image so as to realize infrared and visible light image registration and fusion, the method is constructed and applied to the infrared and visible light image fusion of the switch cabinet, the image registration and fusion are automatically realized, and the manual workload is reduced.

Inventors

YUAN JUNFENG
DONG CHUNLEI
CHEN SIYUAN
ZHAN WENCHAO
MA WENBIN
GUO GUANG

Assignees

国网河北省电力有限公司衡水供电分公司
国家电网有限公司
北京中科创益科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251222

Claims (10)

1. An infrared-visible light image fusion method based on a deformation field, which is characterized by comprising the following steps: S1, acquiring an infrared image and a visible light image of a high-voltage switch cabinet; S2, constructing a multi-mode image registration model, and processing the visible light image by adopting a cross-mode perception style migration network in the model to generate a pseudo infrared image; S3, inputting the pseudo infrared image and the infrared image into a multi-level refined registration network of the model, comparing the pseudo infrared image and the infrared image to generate a distortion displacement vector deformation field, and carrying out registration reconstruction on the infrared image by adopting the deformation field to obtain a registered infrared image; S4, respectively inputting the registered infrared image and visible light image into a feature extraction network, extracting features, and fusing the features by adopting a multi-mode image fusion network to obtain a fused image; s5, outputting the fusion image.
2. The method of claim 1, wherein the capturing is performed using an infrared camera and a visible light camera.
3. A method according to claim 1 or 2, wherein the cross-modal perceived style migration network comprises an image generator for processing the visible light image to obtain the pseudo-infrared image and an image discriminator for determining the pseudo-infrared image, which remains as a final pseudo-infrared image when belonging to a set target class.
4. The method according to claim 3, wherein the image generator comprises an encoder, a converter and a decoder, wherein the visible light picture enters the encoder formed by three convolution layers, and the encoder performs feature extraction on the visible light picture and outputs a feature vector The characteristic vector is input into a converter composed of nine layers Resnet residual modules, and the converter converts the characteristic vector into the characteristic vector required by the pseudo-infrared image And finally, the required feature vector The two deconvolution layers input to the decoder are restored to low-level features, and then the low-level features are converted into the pseudo-infrared picture through a convolution layer.
5. The method of claim 3, wherein the image discriminator is a convolutional neural network, and comprises four convolutional layers and a one-dimensional output convolutional layer, the pseudo-infrared image is input into the four convolutional layers for processing, then image features are extracted, and the one-dimensional output convolutional layer outputs and judges the image features to determine whether the image features belong to a target class.
6. The method of claim 1, wherein the multi-level refined registration network comprises a feature extraction network, two deformation field prediction components C2F-DFE and a resampling layer, wherein the feature extraction network performs feature extraction on the pseudo-infrared image and the infrared image respectively to obtain pseudo-infrared image features and infrared image features, the pseudo-infrared image features and the infrared image features are input to the two deformation field prediction components C2F-DFE respectively to obtain a predicted deformation field, and the predicted deformation field is input to the resampling layer to be registered with the infrared image to obtain a registered infrared image.
7. The method of claim 6, wherein each of said deformation field prediction components C2F-DFE comprises a coarse deformation field prediction mode connected thereto And a fine deformation field prediction module for predicting a coarse deformation field for the input image, and the fine deformation field prediction module further predicts the coarse deformation field to obtain the predicted deformation field.
8. The method according to claim 1, wherein the multi-modal fusion network comprises an encoder network, a fusion network and a decoder network, and the encoder network performs feature extraction on the registered infrared image and visible light image to obtain an infrared image feature map and a visible light image feature map; The fusion network adopts an L1 norm and Softmax strategy to fuse the infrared image feature map and the visible light image feature map to obtain an intermediate image, and the intermediate image is input to the decoder network for reconstruction to obtain a fusion image.
9. A computer storage medium, characterized in that the medium has stored thereon a computer program which is executed by a processor to implement the method of any of claims 1-8.
10. An electronic device, the electronic device comprising: A memory storing executable instructions; a processor executing the executable instructions in the memory to implement the method of any of claims 1-8.

Description

Infrared-visible light image fusion method based on deformation field, storage medium and electronic equipment Technical Field The invention belongs to the field of high-voltage electrical equipment monitoring, and particularly relates to an infrared-visible light image fusion method based on a deformation field, a storage medium and electronic equipment. Background The high-voltage switch cabinet is core equipment for ensuring the safe and stable operation of a power system, and potential thermal faults (such as poor contact, overload and insulation ageing accompanied with heat generation) in the high-voltage switch cabinet are one of main causes for causing equipment accidents. The infrared thermal imaging technology can intuitively display the temperature distribution of the equipment, effectively detect abnormal heating points, but has low spatial resolution and lacks of equipment structural details, so that operation and maintenance personnel are difficult to accurately position the heating points to specific physical components (such as specific bolts, contacts and bus connecting sheets) in the switch cabinet. While visible light imaging can provide high-resolution equipment structure information, temperature anomalies cannot be directly reflected. Therefore, the infrared heat map and the visible light image are accurately registered and information fused to form a visible image with a heat-structure overlapped, and the infrared heat map and the visible light image have irreplaceable values for accurately positioning a heating source, understanding the corresponding relation between heat abnormality and equipment structure and guiding efficient overhaul. However, currently, image registration fusion in a switch cabinet scene mainly faces the following challenges and limitations: 1. the automatic degree is low, the current fusion scheme mainly depends on manual work, the efficiency is low, and no automatic means exists in feature point division. 2. The method has poor adaptability, namely lens parameters of the infrared sensor and the visible light sensor need to be determined to realize fusion, but fusion can not be carried out on images without parameters. Thus, a reliable method is needed to solve the current problem. Disclosure of Invention In order to overcome the above problems in the prior art, the present invention provides an infrared-visible light image fusion method, a storage medium and an electronic device based on a deformation field, which are used for solving the above problems in the prior art. S1, acquiring an infrared image and a visible light image of a high-voltage switch cabinet; S2, constructing a multi-mode image registration model, and processing the visible light image by adopting a cross-mode perception style migration network in the model to generate a pseudo infrared image; S3, inputting the pseudo infrared image and the infrared image into a multi-level refined registration network of the model, comparing the pseudo infrared image and the infrared image to generate a distortion displacement vector deformation field, and carrying out registration reconstruction on the infrared image by adopting the deformation field to obtain a registered infrared image; S4, respectively inputting the registered infrared image and visible light image into a feature extraction network, extracting features, and fusing the features by adopting a multi-mode image fusion network to obtain a fused image; s5, outputting the fusion image. Aspects and any one of the possible implementations as described above, further providing an implementation, the acquiring is performed with an infrared camera and a visible light camera. In accordance with the aspects and any possible implementation manner of the foregoing, there is further provided an implementation manner, where the cross-modal sensing style migration network includes an image generator and an image discriminator, where the image generator is configured to process the visible light image to obtain the pseudo-infrared image, and the image discriminator is configured to determine the pseudo-infrared image, and to retain the pseudo-infrared image as a final pseudo-infrared image when the pseudo-infrared image belongs to a set target class. In the aspect and any possible implementation manner, there is further provided an implementation manner, wherein the image generator comprises an encoder, a converter and a decoder, the visible light picture enters the encoder formed by three convolution layers, and the encoder performs feature extraction on the visible light picture and outputs a feature vectorThe characteristic vector is input into a converter composed of nine layers Resnet residual modules, and the converter converts the characteristic vector into the characteristic vector required by the pseudo-infrared imageAnd finally, the required feature vectorThe two deconvolution layers input to the decoder are restored to low-level features, and then