CN-122023145-A - Image processing method and device based on traditional algorithm and neural network

CN122023145ACN 122023145 ACN122023145 ACN 122023145ACN-122023145-A

Abstract

The application belongs to the technical field of image processing, and discloses an image processing method and device based on a traditional algorithm and a neural network, wherein the method comprises the steps of acquiring an infrared image and a visible light image, and processing according to the neural network and a self-attention mechanism to obtain degradation information; selecting a preset algorithm according to the degradation information to obtain a current algorithm, processing an infrared image and a visible light image according to the current algorithm to obtain a bimodal image, carrying out weighted fusion on the bimodal image to obtain a fused image, and carrying out iterative computation on the fused image according to the current algorithm to obtain an image result. The application can complete the high-quality restoration and fusion of the infrared image and the visible light image under the multi-environment degradation condition by combining the traditional algorithm with the neural network, thereby improving the quality of the image result and the image precision.

Inventors

LOU WANGYANG
YUAN CHANGSHUN
LI YAXIN
CAO QIDONG
HE YUQIANG
ZHOU YANG
ZHANG CHUNLIN
BI YANXIAN

Assignees

北京航空航天大学杭州创新研究院

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (10)

1. An image processing method based on a traditional algorithm and a neural network, which is characterized by comprising the following steps: acquiring an infrared image and a visible light image, and processing according to a neural network and a self-attention mechanism to obtain degradation information; Selecting a preset algorithm according to the degradation information to obtain a current algorithm; Processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image; weighting and fusing the bimodal images to obtain fused images; And carrying out iterative computation on the fusion image according to the current algorithm to obtain an image result.
2. The image processing method based on the conventional algorithm and the neural network according to claim 1, wherein the acquiring the infrared image and the visible light image and processing according to the neural network and the self-attention mechanism to obtain the degradation information comprises: Acquiring an infrared image and a visible light image, and processing according to a multi-scale convolutional neural network and a self-attention mechanism to obtain a multi-scale feature map; processing the multi-scale feature map by adopting an attention mechanism to obtain a degradation intensity map and degradation type probability distribution; constructing a degradation association graph according to the degradation intensity graph and the degradation type probability distribution; And obtaining degradation information according to the degradation intensity graph, the degradation type probability distribution and the degradation association graph.
3. The image processing method based on the conventional algorithm and the neural network according to claim 2, wherein the selecting a preset algorithm according to the degradation information to obtain the current algorithm includes: judging the degradation type according to the degradation information; If the degradation type has a physical model, a fixed rule or a predictable mode, the current algorithm comprises a dark channel defogging algorithm, a multi-scale Retinex algorithm and an adaptive median filtering algorithm.
4. The image processing method based on a conventional algorithm and a neural network according to claim 3, wherein the processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image comprises: Processing the infrared image and the visible light image based on Fourier transform to obtain a global offset; Processing the global offset based on a RANSAC algorithm to obtain a geometric alignment image; And processing the geometric alignment image according to the current algorithm to obtain a bimodal image.
5. The image processing method based on the conventional algorithm and the neural network according to claim 4, wherein the selecting a preset algorithm according to the degradation information to obtain a current algorithm further comprises: judging the degradation type according to the degradation information; If the degradation type is a nonlinear complex combination, the current algorithm is a neural network.
6. The image processing method based on a conventional algorithm and a neural network according to claim 5, wherein the processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image further comprises: Processing the degradation intensity map, the infrared image and the visible light image based on the degradation perception encoder to obtain dynamic offset characteristics; processing the dynamic offset feature based on the deformable convolution layer to obtain a correction amount; and supplementing the geometrically aligned image according to the correction quantity to obtain a bimodal image.
7. The method for processing an image based on a conventional algorithm and a neural network according to claim 6, wherein the performing weighted fusion on the bimodal image to obtain a fused image comprises: processing based on the degradation intensity map and the feature significance in the bimodal image to obtain a pixel-level fusion weight; adjusting the pixel-level fusion weight according to the degradation degree to obtain fusion characteristics; and processing the fusion characteristic according to a decoder to obtain a fusion image.
8. An image processing apparatus based on a conventional algorithm and a neural network, the apparatus comprising: The processing module is used for acquiring an infrared image and a visible light image and processing according to the neural network and the self-attention mechanism to obtain degradation information; the algorithm module is used for selecting a preset algorithm according to the degradation information to obtain a current algorithm; The computing module is used for processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image; the fusion module is used for carrying out weighted fusion on the bimodal images to obtain fusion images; and the iteration module is used for carrying out iterative computation on the fusion image according to the current algorithm to obtain an image result.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the conventional algorithm and neural network based image processing method according to any one of claims 1 to 7 when the computer program is executed.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the image processing method based on a conventional algorithm and a neural network as claimed in any one of claims 1 to 7.

Description

Image processing method and device based on traditional algorithm and neural network Technical Field The application relates to the technical field of image processing, in particular to an image processing method and device based on a traditional algorithm and a neural network. Background The fusion of infrared and visible light images plays an important role in the fields of night vision monitoring, automatic driving, remote sensing imaging, complex meteorological observation and the like. However, the image photographed in the actual environment is often affected by various factors such as darkness, fog, rain, snow and the like, and the environmental degradations are mutually affected, so that the comprehensive restoration is difficult to realize through a single algorithm. The traditional algorithm has better interpretability and stability in the aspects of distortion, haze, noise and the like, but is difficult to treat complex nonlinear degradation, and the neural network is better treated in a complex degradation environment, but often has the problems of over fitting, under fitting and the like. In the fusion process of infrared and visible light images, the fusion can be conflicted due to the difference of the responses of different modes to the same scene. The existing method lacks the cooperation of dynamic sensing and self-adaption aiming at multi-environment degradation, so that the problems of artifact, color shift or detail loss and the like occur when the infrared light and the visible light of the mixed degradation environment are processed in a fusion mode. Therefore, how to realize high-quality restoration and fusion of infrared and visible light images under the multi-environment degradation condition becomes a problem to be solved. Disclosure of Invention The application provides an image processing method and device based on a traditional algorithm and a neural network, which can select the traditional algorithm or the neural network according to degradation information corresponding to an infrared image and a visible light image, combine the two to process the image, keep the stability of the traditional algorithm, and simultaneously adopt the neural network to increase the flexibility of the method, adaptively adjust the image processing strategy according to the difference of the infrared image and the visible light image, effectively solve the problem of multiple degradation and coexistence, and complete the high-quality restoration and fusion of the infrared image and the visible light image under the multi-environment degradation condition by the processing mode of combining the traditional algorithm and the neural network, improve the quality of an image result and improve the image precision. In a first aspect, an embodiment of the present application provides an image processing method based on a conventional algorithm and a neural network, where the method includes: acquiring an infrared image and a visible light image, and processing according to a neural network and a self-attention mechanism to obtain degradation information; Selecting a preset algorithm according to the degradation information to obtain a current algorithm; Processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image; weighting and fusing the bimodal images to obtain fused images; And carrying out iterative computation on the fusion image according to the current algorithm to obtain an image result. Further, acquiring an infrared image and a visible light image and processing according to a neural network and a self-attention mechanism to obtain degradation information, including: Acquiring an infrared image and a visible light image, and processing according to a multi-scale convolutional neural network and a self-attention mechanism to obtain a multi-scale feature map; adopting an attention mechanism to process the multi-scale feature map to obtain a degradation intensity map and degradation type probability distribution; constructing a degradation association graph according to the degradation intensity graph and the degradation type probability distribution; And obtaining degradation information according to the degradation intensity graph, the degradation type probability distribution and the degradation association graph. Further, selecting a preset algorithm according to the degradation information to obtain a current algorithm, including: judging the degradation type according to the degradation information; if the degradation type has a physical model, a fixed rule or a predictable mode, the current algorithm comprises a dark channel defogging algorithm, a multi-scale Retinex algorithm and an adaptive median filtering algorithm. Further, processing the infrared image and the visible light image according to the current algorithm to obtain a bimodal image includes: processing the infrared image and the visible light image based on Fourier tra