CN-117218047-B - Image fusion method and device, electronic equipment and storage medium

CN117218047BCN 117218047 BCN117218047 BCN 117218047BCN-117218047-B

Abstract

The embodiment of the application provides an image fusion method, device, electronic equipment and storage medium, wherein the method comprises the steps of obtaining a visible light image and a thermal imaging image, determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image, wherein the relation between the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image is a negative correlation relation, fusing the color component of the pixel in the thermal imaging image and the color component of the pixel in the visible light image according to the first color weight and the second color weight, and fusing the brightness component of the pixel in the thermal imaging image and the brightness component of the pixel in the visible light image to obtain a fused image. The embodiment of the application can reduce the problem of color cast of the image obtained by fusing the thermal imaging image and the visible light image.

Inventors

Jin Yiru
TANG JIE
JIANG HENGYU
Tang Ningqiu

Assignees

杭州微影软件有限公司

Dates

Publication Date: 20260505
Application Date: 20230918

Claims (13)

1. A method of image fusion, the method comprising: obtaining a visible light image and a thermal imaging image; Determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image, wherein the relationship between the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image is a negative correlation, and the negative correlation comprises the steps of dividing the temperature into a plurality of temperature intervals, wherein the first color weight corresponding to each temperature interval is in a negative correlation with the temperature of the temperature interval, and the first color weight of the color component corresponding to the pixel of the temperature reflected by the pixel in the thermal imaging image in the same temperature interval is the same or different, or dividing the temperature into a plurality of temperature intervals, and the temperature reflected by the pixel in the thermal imaging image and the first color weight of the color component corresponding to the pixel are in a negative correlation in at least one temperature interval; And according to the first color weight and the second color weight, fusing the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image, and fusing the brightness components of the pixels in the thermal imaging image and the brightness components of the pixels in the visible light image to obtain a fused image.
2. The method of claim 1, wherein the thermal imaging image is a gray scale image; The step of determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image comprises the following steps: based on the gray value of the pixel in the gray image, performing pseudo-color conversion processing on the gray image to obtain a color thermal image; And determining a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image.
3. The method of claim 2, wherein the step of determining a first color weight for the color component of the pixel in the thermal imaging image and a second color weight for the color component of the pixel in the visible light image based on the temperature reflected by the luminance component of each pixel in the color thermal image comprises: Calculating a background mean value threshold value based on the brightness components of all pixels in the color thermal image, wherein the background mean value threshold value represents the mean value of pixel values of background pixels in the thermal imaging image; For each pixel of the color thermal image, if the brightness component corresponding to the pixel is smaller than the background mean value threshold value, determining that the first color weight of the color component of the pixel corresponding to the pixel point in the thermal imaging image is1, and determining that the second color weight of the color component of the pixel corresponding to the pixel point in the visible light image is 0.
4. A method according to claim 3, characterized in that the method further comprises: Calculating to obtain a high Wen Mubiao threshold value based on the brightness component of each pixel in the color thermal image, wherein the high-temperature target threshold value represents a pixel value corresponding to a target with the temperature higher than a preset temperature in the thermal imaging image; For each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold and the brightness component of the pixel is not greater than the high-temperature target threshold, determining that a first color weight of a color component of a pixel point corresponding to the pixel in the thermal imaging image is a first weight, and determining that a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image is a second weight, wherein the first weight and the second weight enable a pixel value corresponding to the pixel point in the fused image to be in inverse proportion to temperature information in the thermal imaging image.
5. The method according to claim 4, wherein the method further comprises: For each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high-temperature target threshold, determining that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is a first preset weight, and determining that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is a second preset weight.
6. The method of claim 5, wherein the first preset weight is The second preset weight is ; The step of determining that a first color weight of a color component of a pixel corresponding to the pixel in the thermal imaging image is a first weight and determining that a second color weight of a color component of a pixel corresponding to the pixel in the visible light image is a second weight includes: the first weight is calculated according to the following formula And a second weight : Wherein, the For the maximum value of the luminance component of each pixel in the color thermal image, For the average value of the luminance components of the pixels in the color thermal image, For the high temperature target threshold value in question, For the luminance component corresponding to the pixel.
7. The method according to any one of claims 4-6, wherein the means for calculating the background mean threshold and the high temperature target threshold comprises: based on the brightness components of each pixel in the color thermal image, calculating to obtain a background mean threshold according to the following formula High Wen Mubiao threshold : Wherein, the For the maximum value of the luminance component of each pixel in the color thermal image, Is the average value of the brightness components of each pixel in the color thermal image.
8. The method according to any one of claims 4-6, wherein the step of fusing the luminance component of the pixels in the thermographic image with the luminance component of the pixels in the visible light image comprises: And fusing the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image according to the first preset brightness weight of the brightness components of the pixels in the thermal imaging image and the second preset brightness weight of the brightness components of the pixels in the visible light image.
9. The method of claim 2, wherein the color space of the color thermal image does not include a luminance space; Before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image, the method further includes: and performing color space conversion processing on the color thermal image according to a conversion mode corresponding to a target color space to obtain the color thermal image of the target color space, wherein the target color space comprises a brightness space.
10. An image fusion apparatus, the apparatus comprising: the image acquisition module is used for acquiring visible light images and thermal imaging images; The color weight determining module is used for determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image, wherein the relation between the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image is a negative correlation relation; the negative correlation includes dividing the temperature into a plurality of temperature intervals, wherein the first color weight corresponding to each temperature interval is in negative correlation with the temperature of the temperature interval, and the first color weights of the color components corresponding to the pixels of which the temperature is reflected in the pixels in the same temperature interval in the thermal imaging image are the same or different; And the component fusion module is used for fusing the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image according to the first color weight and the second color weight, and fusing the brightness components of the pixels in the thermal imaging image and the brightness components of the pixels in the visible light image to obtain a fused image.
11. The apparatus of claim 10, wherein the thermal imaging image is a gray scale image; the color weight determination module includes: The color thermal image acquisition sub-module is used for carrying out pseudo-color conversion processing on the gray level image based on the gray level value of the pixel in the gray level image to obtain a color thermal image; a color weight determining sub-module, configured to determine a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to a temperature reflected by a brightness component of each pixel in the color thermal image; the color weight determination submodule includes: the background mean value threshold value calculation unit is used for calculating a background mean value threshold value based on the brightness components of all pixels in the color thermal image, wherein the background mean value threshold value represents the pixel value mean value of background pixels in the thermal imaging image; A first determining unit, configured to determine, for each pixel of the color thermal image, if a luminance component corresponding to the pixel is less than the background mean threshold, that a first color weight of a color component of a pixel point corresponding to the pixel in the thermal imaging image is 1, and that a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image is 0; The apparatus further comprises: A high Wen Mubiao threshold calculating unit, configured to calculate a high Wen Mubiao threshold based on a luminance component of each pixel in the color thermal image, where the high temperature target threshold represents a pixel value corresponding to a target whose temperature is higher than a preset temperature in the thermal imaging image; a second determining unit, configured to determine, for each pixel of the color thermal image, a first color weight of a color component of a pixel corresponding to the pixel in the thermal imaging image as a first weight and a second color weight of a color component of a pixel corresponding to the pixel in the visible light image as a second weight if a luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, where the first weight and the second weight make a pixel value corresponding to the pixel in the fused image inversely proportional to temperature information in the thermal imaging image; The apparatus further comprises: A third determining unit, configured to determine, for each pixel of the color thermal image, if a luminance component corresponding to the pixel is greater than the high temperature target threshold, a first color weight of a color component of a pixel point corresponding to the pixel in the thermal imaging image as a first preset weight, and determine a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image as a second preset weight; The first preset weight is The second preset weight is ; The second determination unit includes: A weight determining subunit for calculating a first weight according to the following formula And a second weight : Wherein, the For the maximum value of the luminance component of each pixel in the color thermal image, For the average value of the luminance components of the pixels in the color thermal image, For the high temperature target threshold value in question, A luminance component corresponding to the pixel; The apparatus further includes a threshold calculation module for calculating a background mean threshold and a high Wen Mubiao threshold, including: a threshold calculation unit for calculating a background mean threshold based on the brightness components of each pixel in the color thermal image according to the following formula High Wen Mubiao threshold : Wherein, the For the maximum value of the luminance component of each pixel in the color thermal image, A mean value of brightness components of pixels in the color thermal image; the component fusion module comprises: A luminance component fusion sub-module, configured to fuse a luminance component of a pixel in the thermal imaging image with a luminance component of a pixel in the visible light image according to a first preset luminance weight of the luminance component of the pixel in the thermal imaging image and a second preset luminance weight of the luminance component of the pixel in the visible light image; the color space of the color thermal image does not include a luminance space; The apparatus further comprises: And the color thermal image acquisition module is used for carrying out color space conversion processing on the color thermal image according to a conversion mode corresponding to a target color space before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image, so as to obtain the color thermal image of the target color space, wherein the target color space comprises the brightness space.
12. An electronic device, comprising: a memory for storing a computer program; a processor for implementing the method of any of claims 1-9 when executing a program stored on a memory.
13. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-9.

Description

Image fusion method and device, electronic equipment and storage medium Technical Field The present application relates to the field of image fusion technologies, and in particular, to an image fusion method, an image fusion device, an electronic device, and a storage medium. Background The visible light image has rich brightness and texture information, can keep the color and the outline of the object, and can obtain most of information contained in the image, such as the size, the position and the like of the object through human eyes. A thermal imaging image is an image formed by a thermal radiation scanner receiving and recording thermal radiation energy emitted from a target object, for example, a color thermal image may be acquired using a thermal infrared scanner, or the like. The targets and the backgrounds in the thermal imaging images can be distinguished according to the thermal radiation difference, so that the thermal imaging images have stronger anti-interference capability and penetrability. But the thermal imaging image resolution is low and only the approximate contours of the target can be obtained, and most of the background information is lost. In order to enable the image to have good target indicating capability while having visible light texture information, the visible light image and the thermal imaging image can be fused, the fusion of the thermal imaging image and the visible light image is generally based on the brightness space of the image at present, the problem of color cast is easy to generate, and the image fusion effect is not ideal. Disclosure of Invention The embodiment of the application aims to provide an image fusion method, an image fusion device, electronic equipment and a storage medium, so as to reduce the problem of color cast of an image obtained by fusing a thermal imaging image and a visible light image and improve the image fusion effect. The specific technical scheme is as follows: in a first aspect, an embodiment of the present application provides an image fusion method, where the method includes: obtaining a visible light image and a thermal imaging image; Determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image, wherein the relationship between the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image is a negative correlation relationship; And according to the first color weight and the second color weight, fusing the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image, and fusing the brightness components of the pixels in the thermal imaging image and the brightness components of the pixels in the visible light image to obtain a fused image. Optionally, the thermal imaging image is a gray scale image; The step of determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by each pixel of the thermal imaging image comprises the following steps: based on the gray value of the pixel in the gray image, performing pseudo-color conversion processing on the gray image to obtain a color thermal image; And determining a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image. Optionally, the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image includes: Calculating a background mean value threshold value based on the brightness components of all pixels in the color thermal image, wherein the background mean value threshold value represents the mean value of pixel values of background pixels in the thermal imaging image; For each pixel of the color thermal image, if the brightness component corresponding to the pixel is smaller than the background mean value threshold value, determining that the first color weight of the color component of the pixel corresponding to the pixel point in the thermal imaging image is1, and determining that the second color weight of the color component of the pixel corresponding to the pixel point in the visible light image is 0. Optionally, the method further comprises: Calculating to obtain a high Wen Mubiao threshold value based on