Search

CN-122023990-A - Missile-borne infrared-visible light multi-mode image fusion method

CN122023990ACN 122023990 ACN122023990 ACN 122023990ACN-122023990-A

Abstract

The invention relates to the technical field of image processing and information fusion, and discloses a missile-borne infrared-visible light multi-mode image fusion method which comprises the following steps of S1, S2, S3, S4, S5, FPGA (field programmable gate array) parallelization design, memory optimization and other hardware acceleration strategies to realize optimization of missile-borne instantaneity, and S5, wherein the method can be adapted to a multi-type missile guidance system to remarkably enhance target locking precision and real-time response capability in a complex battlefield environment and expand to the fields of unmanned plane reconnaissance, armored vehicle multispectral perception and other national defense application.

Inventors

  • SONG SHENMIN
  • YANG YAHU
  • LIU JINGANG
  • ZHANG YANSONG
  • LI JIAPENG

Assignees

  • 哈尔滨工业大学

Dates

Publication Date
20260512
Application Date
20260202

Claims (7)

  1. 1. The missile-borne infrared-visible light multi-mode image fusion method is characterized by comprising the following steps of: step S1, respectively preprocessing an infrared image and a visible light image by adopting different algorithms; s2, carrying out multi-scale decomposition and feature extraction on the infrared image and the visible light image; S3, performing low-frequency fusion and high-frequency fusion on the infrared image and the visible light image by adopting a self-adaptive fusion rule; S4, performing image reconstruction and post-optimization on the fused low-frequency information and high-frequency information; and S5, optimizing the missile-borne instantaneity by adopting hardware acceleration strategies such as FPGA parallelization design, memory optimization and the like.
  2. 2. The method for fusion of missile-borne infrared-visible light multi-mode images according to claim 1, wherein the step S1 of image preprocessing specifically comprises infrared image enhancement and visible light image illumination correction, and the infrared image is subjected to contrast-limited adaptive histogram equalization (CLAHE) algorithm Enhancement is performed by first dividing the image into Sub-blocks, each sub-block independently calculating a histogram, and then setting a threshold value for a gray level distribution of each sub-block histogram Finally, eliminating the blocking effect through bilinear interpolation to obtain an enhanced infrared image The mathematical expression is as follows: , Wherein, the As a local sub-block histogram equalization function, Is an interpolation weight.
  3. 3. The method for fusion of missile-borne infrared-visible light multi-mode images according to claim 2, wherein the visible light image illumination correction is specifically carried out by decomposing the visible light image with a multi-scale Retinex algorithm (MSR) aiming at the non-uniformity of the visible light image under complex illumination, and three sets of scale parameters are adopted first And respectively extracting illumination components in different ranges, and then, calculating reflection components according to the following formula: , Wherein, the As the weight coefficient of the light-emitting diode, As a Gaussian filter, finally, for the reflected component Gamma correction is performed Outputting the enhanced visible light image 。
  4. 4. The method for multi-modal image fusion of airborne infrared and visible light of claim 3, wherein said step S2 of multi-scale decomposition and feature extraction first decomposes the image into a low frequency subband L and a set of high frequency subbands by a non-downsampling pyramid NSP The high frequency subbands are then direction refined using a bank of shearing filters, described mathematically as follows: , Where s=3 is the number of decomposition levels, For the direction number of the s-th layer, the fixed link strength of the traditional improved pulse coupled neural network PCNN Instead, dynamic parameters related to local contrast: , Wherein the method comprises the steps of In pixels Is central Window variance, threshold The update formula is: , Wherein the method comprises the steps of In order for the attenuation coefficient to be a factor, As the threshold value amplification factor, For neuron output, pixels in high frequency sub-bands Is characterized by PCNN ignition frequency Characterization: , Wherein the method comprises the steps of Is the total number of iterations.
  5. 5. The method for fusion of missile-borne infrared-visible light multi-modal images according to claim 4, wherein the step S3 adaptive fusion rule is implemented by first Calculating the area energy value in the window: , , then, the gradient magnitude is calculated using the Sobel operator: , finally, weight synthesis is carried out: , Wherein, the For the balance coefficient of energy and gradient, the local contrast LC is calculated as follows: , Wherein, the Is a local mean value The window-opening is provided with a window, For the global average value of the sub-bands, combining the PCNN ignition frequency and the contrast, and selecting a coefficient with higher significance: , wherein the threshold is set as the mean of the two coefficient significance products.
  6. 6. The method for fusion of missile-borne infrared-visible light multi-mode images according to claim 5, wherein said step S4 comprises the specific steps of first fusing the low-frequency signals With high frequency Synthesized by inverse NSST: , Then, denoising is performed by a bilateral filtering algorithm: , Wherein, the For the normalized coefficient to be a function of the normalized coefficient, Is the spatial domain standard deviation; is the standard deviation of the brightness domain.
  7. 7. The method for fusion of missile-borne infrared-visible light multi-mode images according to claim 6, wherein NSST decomposition and PCNN feature extraction modules in the step S5 are of pipeline structures, and each layer of decomposition is independently distributed with a computing unit.

Description

Missile-borne infrared-visible light multi-mode image fusion method Technical Field The invention relates to the technical field of image processing and information fusion, in particular to a missile-borne infrared-visible light multi-mode image fusion method. Background In missile guidance systems, infrared and visible light image sensors are widely used due to their complementary characteristics. The infrared image can penetrate smoke and image at night through thermal radiation information, but has low sensitivity to texture details, the visible light image has high resolution and abundant details, but is easily influenced by illumination and weather conditions, the existing fusion method mainly has the following problems that ① has insufficient instantaneity, the traditional multi-scale decomposition method (such as wavelet transformation) has high computational complexity, the requirement of a missile-borne environment on instantaneity is difficult to meet, ② has single fusion rule, the existing method has poor adaptability to a dynamic scene, the complementary characteristics of multi-mode characteristics are not fully utilized, ③ has weak anti-interference capability, and image blurring and noise caused by high-speed movement of the missile-borne sensor are not effectively inhibited. Based on the problems existing in the prior art, the invention provides a missile-borne infrared-visible light multi-mode image fusion method. Disclosure of Invention The invention aims to provide a missile-borne infrared-visible light multi-mode image fusion method for solving the problems in the background technology. In order to achieve the purpose, the invention provides a missile-borne infrared-visible light multi-mode image fusion method, which comprises the following steps: step S1, respectively preprocessing an infrared image and a visible light image by adopting different algorithms; s2, carrying out multi-scale decomposition and feature extraction on the infrared image and the visible light image; S3, performing low-frequency fusion and high-frequency fusion on the infrared image and the visible light image by adopting a self-adaptive fusion rule; S4, performing image reconstruction and post-optimization on the fused low-frequency information and high-frequency information; and S5, optimizing the missile-borne instantaneity by adopting hardware acceleration strategies such as FPGA parallelization design, memory optimization and the like. Preferably, the step S1 of image preprocessing specifically comprises infrared image enhancement and visible light image illumination correction, and the contrast-limited self-adaptive histogram equalization (CLAHE) algorithm is adopted for infrared imagesEnhancement is performed by first dividing the image intoSub-blocks, each sub-block independently calculating a histogram, and then setting a threshold value for a gray level distribution of each sub-block histogramFinally, eliminating the blocking effect through bilinear interpolation to obtain an enhanced infrared imageThe mathematical expression is as follows: Wherein, the As a local sub-block histogram equalization function,Is an interpolation weight. Preferably, the visible light image illumination correction is carried out by adopting a multi-scale Retinex algorithm, namely MSR, to decompose aiming at the non-uniformity of the visible light image under complex illumination, and firstly adopting three groups of scale parametersAnd respectively extracting illumination components in different ranges, and then, calculating reflection components according to the following formula: Wherein, the As the weight coefficient of the light-emitting diode,As a Gaussian filter, finally, for the reflected componentGamma correction is performedOutputting the enhanced visible light image。 Preferably, the step S2 multi-scale decomposition and feature extraction first decomposes the image into low frequency sub-band L and high frequency sub-band set by non-downsampling pyramid NSPThe high frequency subbands are then direction refined using a bank of shearing filters, described mathematically as follows: Where s=3 is the number of decomposition levels, For the direction number of the s-th layer, the fixed link strength of the traditional improved pulse coupled neural network PCNNInstead, dynamic parameters related to local contrast: Wherein the method comprises the steps of In pixelsIs centralWindow variance, thresholdThe update formula is: Wherein the method comprises the steps of In order for the attenuation coefficient to be a factor,As the threshold value amplification factor,For neuron output, pixels in high frequency sub-bandsIs characterized by PCNN ignition frequencyCharacterization: Wherein the method comprises the steps of Is the total number of iterations. Preferably, the adaptive fusion rule of the step S3 is that firstlyCalculating the area energy value in the window: , then, the gradient magnitude is calculated using the Sobe