Search

CN-121998846-A - Training method of image fusion model, image fusion method and related device

CN121998846ACN 121998846 ACN121998846 ACN 121998846ACN-121998846-A

Abstract

The application provides a training method of an image fusion model, the image fusion method and a related device, wherein the training method of the image fusion model comprises the steps of obtaining a first visible light image and a first infrared image, wherein the first visible light image and the first infrared image are noise-free images, carrying out image layering on the first visible light image based on image characteristics to obtain a visible light image layer combination, carrying out image layering on the first infrared image to obtain an infrared image layer combination, carrying out fusion on the visible light image layer combination and the infrared image layer combination to obtain a sample fusion image, and training an initial image fusion model by utilizing the sample fusion image to obtain the image fusion model. The method can obtain the high-quality fusion image of the visible light image and the infrared image, train the image fusion model by using the fusion image, and improve the performance of the image fusion model.

Inventors

  • YU KEQIANG
  • ZHOU JUNJIE
  • FENG LIANG

Assignees

  • 浙江大华技术股份有限公司

Dates

Publication Date
20260508
Application Date
20251209

Claims (13)

  1. 1. A method for training an image fusion model, comprising: Acquiring a first visible light image and a first infrared image, wherein the first visible light image and the first infrared image are noiseless images; Image layering is carried out on the first visible light image based on image characteristics to obtain a visible light image layer combination, and image layering is carried out on the first infrared image to obtain an infrared image layer combination; Fusing the visible light image layer combination and the infrared image layer combination to obtain a sample fused image; training an initial image fusion model by using the sample fusion image to obtain the image fusion model.
  2. 2. The method of claim 1, wherein image layering the first visible light image based on image features to obtain a visible light image layer combination, and image layering the first infrared image to obtain an infrared image layer combination, comprises: performing image layering on the first visible light image based on image gradient characteristics to obtain a visible light image layer combination, and performing image layering on the first infrared image to obtain an infrared image layer combination; The visible light image layer combination and the infrared image layer combination respectively comprise at least part of a texture image layer, a small edge image layer, a large edge image layer and a flat image layer.
  3. 3. The method of claim 2, wherein fusing the visible light image layer combination with the infrared image layer combination to obtain a sample fused image comprises: Fusing a visible light texture image layer in the visible light image layer combination and an infrared texture image layer in the infrared image layer combination to obtain a first fused image; Fusing the visible light small edge image layer in the visible light image layer combination and the infrared small edge image layer in the infrared image layer combination to obtain a second fused image; Fusing the visible light large-edge image layer in the visible light image layer combination and the infrared large-edge image layer in the infrared image layer combination to obtain a third fused image; fusing the visible light flat image layer in the visible light image layer combination and the infrared flat image layer in the infrared image layer combination to obtain a fourth fused image; And fusing the first fused image, the second fused image, the third fused image and the fourth fused image to obtain the sample fused image.
  4. 4. The method of claim 3, wherein fusing the visible texture image layer in the visible image layer combination and the infrared texture image layer in the infrared image layer combination to obtain a first fused image comprises: Calculating the ratio of the infrared texture image layer to the visible light texture image layer to obtain a first fusion weight; Dividing the brightness of the first visible light image into a plurality of brightness intervals, mapping the visible light texture value of the visible light texture image layer and the infrared texture value of the infrared texture image layer into the plurality of brightness intervals, calculating interval fusion weights corresponding to each brightness interval based on the visible light texture value and the infrared texture value in each brightness interval, and carrying out bilinear sampling on the interval fusion weights corresponding to all brightness intervals to obtain second fusion weights; Obtaining texture fusion weights based on the first fusion weights and the second fusion weights; And fusing the visible texture image layer and the infrared texture image layer based on the texture fusion weight to obtain the first fused image.
  5. 5. The method of claim 3, wherein fusing the visible light small edge image layer in the visible light image layer combination and the infrared small edge image layer in the infrared image layer combination to obtain a second fused image comprises: Taking the brightness of the first visible light image as the weight of the infrared small-edge image layer, and taking the brightness of the first infrared image as the weight of the visible light small-edge image layer; And fusing the visible light small edge image layer and the infrared small edge image layer based on the weight of the infrared small edge image layer and the weight of the visible light small edge image layer to obtain the second fused image.
  6. 6. The method of claim 3, wherein fusing the visible large edge image layer in the visible image layer combination and the infrared large edge image layer in the infrared image layer combination to obtain a third fused image comprises: And fusing the visible light large-edge image layer and the infrared large-edge image layer based on a preset weight value to obtain the third fused image, wherein the preset weight value is positively correlated with the duty ratio of the visible light large-edge image layer in the third fused image.
  7. 7. The method of claim 3, wherein fusing the visible flat image layer in the visible image layer combination with the infrared flat image layer in the infrared image layer combination to obtain a fourth fused image comprises: Converting the first visible light image into an HSV image, and determining a flat fusion weight based on saturation characteristics in the HSV image; And fusing the visible light flat image layer and the infrared flat image layer based on the flat fusion weight to obtain the fourth fused image.
  8. 8. The method according to any one of claims 1 to 7, wherein the initial image fusion model comprises an image fusion module, training the initial image fusion model by using the sample fusion image to obtain the image fusion model, and the method comprises the steps of: acquiring a second visible light image and a second infrared image, wherein the second visible light image and the second infrared image are noisy images; Processing the second visible light image and the second infrared image by using an initial image fusion model to obtain an initial fusion image; And calculating a fusion loss value by using the initial fusion image and the sample fusion image, thereby obtaining the image fusion model.
  9. 9. The method of claim 8, wherein the initial image fusion model includes a visible light noise reduction module coupled to the image fusion module, and wherein the step of processing the second visible light image and the second infrared image using the initial image fusion model to obtain an initial fusion image comprises: The visible light noise reduction module is used for reducing noise of the second visible light image to obtain a visible light noise reduction image; processing the visible light noise reduction image and the second infrared image by using the image fusion module to obtain the initial fusion image; and calculating a fusion loss value by using the initial fusion image and the sample fusion image, thereby obtaining the image fusion model, comprising the following steps: And calculating a visible light noise reduction loss value by using the visible light noise reduction image and the first visible light image, and calculating a fusion loss value by using the initial fusion image and the sample fusion image, so as to obtain the image fusion model.
  10. 10. The method of claim 8, wherein the initial image fusion model includes a visible light noise reduction module and an infrared noise reduction module coupled to the image fusion module, and wherein the step of processing the second visible light image and the second infrared image using the initial image fusion model to obtain an initial fusion image includes: The visible light noise reduction module is used for carrying out noise reduction on the second visible light image to obtain a visible light noise reduction image; processing the visible light noise reduction image and the infrared noise reduction image by using the image fusion module to obtain the initial fusion image; and calculating a fusion loss value by using the initial fusion image and the sample fusion image, thereby obtaining the image fusion model, comprising the following steps: Calculating a visible light noise reduction loss value by using the visible light noise reduction image and the first visible light image, and calculating an infrared noise reduction loss value by using the infrared noise reduction image and the first infrared image; and calculating a fusion loss value by using the initial fusion image and the sample fusion image, thereby obtaining the image fusion model.
  11. 11. An image fusion method, characterized in that the image fusion method comprises: processing the visible light image to be fused and the infrared image to be fused by using an image fusion model to obtain a fused image, wherein the image fusion model is obtained by training the method of any one of claims 1-10.
  12. 12. An electronic terminal comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory, the processor being configured to execute program data to implement steps in the training method of the image fusion model according to any one of claims 1 to 10 or steps in the image fusion method according to claim 11.
  13. 13. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, which when executed by a processor, implements the steps of the training method of the image fusion model according to any one of claims 1 to 10 or the steps of the image fusion method according to claim 11.

Description

Training method of image fusion model, image fusion method and related device Technical Field The present invention relates to the field of image technologies, and in particular, to a training method for an image fusion model, an image fusion method and a related device. Background With the development of the deep learning algorithm, the deep learning algorithm is increasingly applied in practice. In the field of image processing, compared with the traditional algorithm, the deep learning algorithm has the advantages of larger calculation power consumption and better processing effect. But deep learning algorithms require training samples to be constructed and trained. The quality of the training samples directly determines the application effect of the deep learning algorithm. Supervised learning requires pairs of training samples, which has the advantage that the trained network works better in practical applications, and the disadvantage that the paired samples need to be constructed, which is more labor intensive. The depth learning algorithm of fusion of the visible light image and the infrared image needs to directly acquire the noisy visible light image, the noisy infrared image and the fusion image of the noiseless visible light image and the noiseless infrared image through hardware. However, it is generally difficult to acquire a fused image of a noiseless visible light image and a noiseless infrared image which meet the demands of human eyes, because in practice, the band that human eyes can sense is 390-780 nm, and the infrared band cannot be sensed basically. The infrared transmitting filter is directly used, and the acquisition of the visible light image with infrared information does not meet the requirement of human eyes, because the image has the problems of color cast and abnormal brightness, the quality of a training sample is reduced, and the performance of a deep learning algorithm for fusion of the visible light image and the infrared image is further influenced. Disclosure of Invention The application provides a training method of an image fusion model, an image fusion method and a related device, the method can obtain the high-quality fusion image of the visible light image and the infrared image, thereby improving the performance of the image fusion model. In order to solve the technical problems, the first technical scheme adopted by the invention is to provide a training method of an image fusion model, which comprises the following steps: Acquiring a first visible light image and a first infrared image, wherein the first visible light image and the first infrared image are noiseless images; Image layering is carried out on the first visible light image based on image characteristics to obtain a visible light image layer combination, and image layering is carried out on the first infrared image to obtain an infrared image layer combination; Fusing the visible light image layer combination and the infrared image layer combination to obtain a sample fused image; training an initial image fusion model by using the sample fusion image to obtain the image fusion model. In order to solve the technical problems, the second technical scheme adopted by the invention is to provide an image fusion method, which comprises the following steps: And processing the visible light image to be fused and the infrared image to be fused by using an image fusion model to obtain a fused image, wherein the image fusion model is obtained by training by the method of any one of the above. In order to solve the technical problem, a third technical scheme adopted by the invention is that an electronic terminal is provided, wherein the electronic terminal comprises a memory and a processor which are mutually coupled, the processor is used for executing program instructions stored in the memory, and the processor is used for executing program data to realize the steps in the training method of the image fusion model or the steps in the image fusion method. In order to solve the technical problem, a fourth technical scheme adopted by the invention is to provide a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program realizes the steps in the training method of the image fusion model or the steps in the image fusion method when being executed by a processor. The training method for the image fusion model has the advantages that the training method is different from the prior art, the training method comprises the steps of obtaining a first visible light image and a first infrared image, enabling the first visible light image and the first infrared image to be noise-free images, conducting image layering on the first visible light image based on image features to obtain a visible light image layer combination, conducting image layering on the first infrared image to obtain an infrared image layer combination, fusing the visible ligh