CN-121998842-A - Image fusion method and system with low light enhancement and overexposure inhibition

CN121998842ACN 121998842 ACN121998842 ACN 121998842ACN-121998842-A

Abstract

The invention discloses an image fusion method and system with low light enhancement and overexposure inhibition, which belong to the technical field of image processing, wherein the method comprises the steps of extracting a reflectivity characteristic image, a visible light characteristic image, a dark channel characteristic image and an infrared characteristic image of a visible light image in the training process of an image fusion model, and obtaining an enhanced reflectivity image according to the reflectivity characteristic image; the method comprises the steps of fusing a reflectivity characteristic image, a visible light characteristic image and an infrared characteristic image, performing color space conversion to obtain a color fused image, determining an overexposure area and a non-overexposure area of a dark channel characteristic image, replacing the overexposure area with the characteristics of the infrared characteristic image, replacing the non-overexposure area with the characteristics of the enhanced reflectivity image to obtain an overexposure-restrained fused image, and participating the overexposure-restrained fused image into model loss calculation. The quality of the color fusion image is improved, and the technical problem that the image fusion quality is affected due to overexposure at present is solved.

Inventors

PAN JINGSHAN
LV GUOHUA
TANG SHAOHONG

Assignees

济南超级计算中心有限公司
济南超级计算技术研究院

Dates

Publication Date: 20260508
Application Date: 20260407

Claims (10)

1. An image fusion method with both low light enhancement and overexposure suppression, comprising: obtaining a plurality of paired visible light images and infrared images; Training the image fusion model by using the paired visible light image and infrared image, and obtaining a trained image fusion model after training is completed; Extracting a reflectivity characteristic image, a visible light characteristic image, a dark channel characteristic image and an infrared characteristic image of the infrared image in the training process of the image fusion model, and decoding the reflectivity characteristic image to obtain an enhanced reflectivity image; Fusing the reflectivity characteristic diagram, the visible light characteristic diagram and the infrared characteristic diagram to obtain a fused characteristic diagram; the overexposure area of the dark channel feature map is replaced by the features in the infrared feature map, and the non-overexposure area is replaced by the features in the enhanced reflectivity image, so that a fusion image with overexposure inhibition is obtained; and participating the fusion image with overexposure inhibition into image fusion model loss calculation.
2. The image fusion method with both low light enhancement and overexposure suppression according to claim 1, characterized in that the global average value of dark channel feature map pixels is calculated and determined, and a set multiple of the average value is taken as a judging condition of overexposure areas and non-overexposure areas; judging the area of which the pixels in the dark channel characteristic image are larger than the judging condition as an overexposure area; and judging the area of which the pixel is smaller than or equal to the judgment condition in the dark channel characteristic diagram as a non-overexposure area.
3. The image fusion method with both low light enhancement and overexposure suppression of claim 1, wherein an overexposure area mask of a visible light image is acquired; separating R, G, B three channels of features of the visible light image through channel splitting operation, and sequentially calculating minimum values among the three channels of features by adopting pixel-by-pixel minimization operation to obtain a dark channel initial feature map; noise suppression and dark area strengthening are carried out on the initial characteristic diagram of the dark channel, and a coarse characteristic diagram of the dark channel is output; and filtering the dark channel rough feature map by taking the fusion feature map as a guide image to obtain a final dark channel feature map.
4. The method for image fusion with both low light enhancement and overexposure suppression as set forth in claim 1, wherein the process of extracting the reflectance profile of the visible light image includes: Carrying out convolution on the visible light image for multiple times to obtain initial visible light characteristics; Extracting reflectivity initial characteristics which are not influenced by illumination change from the initial visible light characteristics; And multiplying the reflectivity initial characteristic by the initial visible light characteristic to obtain a reflectivity characteristic map.
5. The method for image fusion with both low light enhancement and overexposure suppression according to claim 1, further comprising the steps of extracting an illuminance characteristic map of a visible light image in an image fusion model training process, decoding the illuminance characteristic map to obtain an enhanced illuminance image, and participating the enhanced illuminance image and the enhanced reflectivity image into an image fusion model loss calculation.
6. The image fusion method with both low light enhancement and overexposure suppression as set forth in claim 1, wherein the process of fusing the reflectivity map, the visible light map and the infrared map to obtain the fused map includes: The method comprises the steps of multiplying a reflectivity characteristic diagram and an infrared characteristic diagram, then carrying out dense connection convolution processing to obtain reflectivity dense connection characteristic, multiplying the reflectivity dense connection characteristic and a visible light characteristic diagram to obtain fusion characteristic after primary injection, connecting the fusion characteristic after primary injection with the visible light characteristic diagram through an attention mechanism to obtain fusion characteristic after secondary injection, and decoding the fusion characteristic after secondary injection to obtain the fusion characteristic diagram.
7. An image fusion system having both low light enhancement and overexposure suppression, comprising: The image acquisition unit is used for acquiring a plurality of paired visible light images and infrared images; The model training unit is used for training an image fusion model by using paired visible light images and infrared images, obtaining a trained image fusion model, extracting a reflectivity characteristic image of the visible light images, a reflectivity characteristic image of a dark channel characteristic image and an infrared characteristic image of the infrared images in the image fusion model training process, decoding the reflectivity characteristic image to obtain an enhanced reflectivity image, fusing the reflectivity characteristic image, the visible light characteristic image and the infrared characteristic image to obtain a fused characteristic image, performing color space conversion on the fused characteristic image to obtain a color fused image, determining an overexposure area and a non-overexposure area of the dark channel characteristic image, replacing the overexposure area of the dark channel characteristic image with the characteristics in the infrared characteristic image, replacing the non-overexposure area with the characteristics in the enhanced reflectivity image, obtaining the fused image with overexposure inhibition, and participating the fused image with overexposure inhibition into image fusion model loss calculation.
8. An electronic device, the device comprising: A processor adapted to execute a computer program; A computer readable storage medium having a computer program stored therein, which when executed by the processor, implements an image fusion method having both low light enhancement and overexposure suppression as claimed in any one of claims 1-6.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor and to perform an image fusion method with both low light enhancement and overexposure suppression according to any one of claims 1-6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements an image fusion method as claimed in any one of claims 1-6 with both low light enhancement and overexposure suppression.

Description

Image fusion method and system with low light enhancement and overexposure inhibition Technical Field The invention relates to the technical field of image data processing, in particular to an image fusion method and system with low light enhancement and overexposure inhibition. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. At present, the fusion method of the visible light image and the infrared image is mainly divided into two major categories, namely a traditional image fusion method and a deep learning-based method. However, no matter the traditional image fusion method or the existing image fusion method based on deep learning is used for carrying out image fusion on a visible light image and an infrared image in a low light scene, the problem of excessive exposure of the images is not considered, and the flare phenomenon caused by a night light is avoided, so that the quality of the acquired fusion image is low. Disclosure of Invention In order to solve the problems, the invention provides an image fusion method and an image fusion system with both low light enhancement and overexposure inhibition, which can improve the quality of color fusion images by inhibiting overexposed areas in the images. In order to achieve the above purpose, the invention adopts the following technical scheme: in a first aspect, an image fusion method with both low light enhancement and overexposure suppression is provided, including: obtaining a plurality of paired visible light images and infrared images; Training the image fusion model by using the paired visible light image and infrared image, and obtaining a trained image fusion model after training is completed; Extracting a reflectivity characteristic image, a visible light characteristic image, a dark channel characteristic image and an infrared characteristic image of the infrared image in the training process of the image fusion model, and decoding the reflectivity characteristic image to obtain an enhanced reflectivity image; Fusing the reflectivity characteristic diagram, the visible light characteristic diagram and the infrared characteristic diagram to obtain a fused characteristic diagram; the overexposure area of the dark channel feature map is replaced by the features in the infrared feature map, and the non-overexposure area is replaced by the features in the enhanced reflectivity image, so that a fusion image with overexposure inhibition is obtained; and participating the fusion image with overexposure inhibition into image fusion model loss calculation. Further, calculating and determining the global average value of the dark channel feature map pixels, and taking the set multiple of the average value as a judging condition of the overexposure region and the non-overexposure region; judging the area of which the pixels in the dark channel characteristic image are larger than the judging condition as an overexposure area; and judging the area of which the pixel is smaller than or equal to the judgment condition in the dark channel characteristic diagram as a non-overexposure area. Further, obtaining an overexposed area mask of the visible light image; separating R, G, B three channels of features of the visible light image through channel splitting operation, and sequentially calculating minimum values among the three channels of features by adopting pixel-by-pixel minimization operation to obtain a dark channel initial feature map; noise suppression and dark area strengthening are carried out on the initial characteristic diagram of the dark channel, and a coarse characteristic diagram of the dark channel is output; and filtering the dark channel rough feature map by taking the fusion feature map as a guide image to obtain a final dark channel feature map. Further, the process of extracting the reflectivity characteristic map of the visible light image comprises the following steps: Carrying out convolution on the visible light image for multiple times to obtain initial visible light characteristics; Extracting reflectivity initial characteristics which are not influenced by illumination change from the initial visible light characteristics; And multiplying the reflectivity initial characteristic by the initial visible light characteristic to obtain a reflectivity characteristic map. Further, in the training process of the image fusion model, an illuminance characteristic diagram of the visible light image is also extracted, the illuminance characteristic diagram is decoded, an enhanced illuminance image is obtained, and the enhanced illuminance image and the enhanced reflectivity image are involved in the loss calculation of the image fusion model. Further, fusing the reflectivity characteristic map, the visible light characteristic map and the infrared characteristic map, and obtaining the fused characteristic map includes: The method compri