CN-122023135-A - Industrial detection-oriented infrared and visible light image self-adaptive alignment system and method

CN122023135ACN 122023135 ACN122023135 ACN 122023135ACN-122023135-A

Abstract

The invention discloses an infrared and visible light image self-adaptive alignment system and method for industrial detection, which belong to the technical field of industrial detection and comprise a multi-mode image acquisition module, an image preprocessing module, a feature extraction and alignment calculation module and an image transformation and fusion output module. The feature extraction and alignment calculation module constructs a space enhancement Raney mutual information objective function O (T) based on the extracted feature points, and adopts a hybrid optimization strategy to solve a space transformation parameter which maximizes the objective function. Through the mode, the method successfully solves the problems of high-precision and robust alignment of large-mode-difference images such as infrared and visible light. The system and the method do not depend on a large amount of training data, have global robustness and local accuracy, and provide strong and practical technical support for deep application of multi-mode information fusion in the fields of industrial detection, security and the like.

Inventors

ZHOU CAIJIAN
XU QINGYANG
YU JIAHUI
ZHAO SHUWEN
ZENG DONGSHENG
CHEN AN
CHEN ENZAN

Assignees

杭州汇萃智能科技有限公司
苏州汇萃智能科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260116

Claims (10)

1. Industrial detection-oriented infrared and visible light image self-adaptive alignment system is characterized by comprising: the multi-mode image acquisition module is used for acquiring an infrared image and a visible light image of the same scene; the image preprocessing module is used for receiving the original infrared image and visible light image and performing preprocessing operation; The feature extraction and alignment calculation module is used for extracting layered feature points of the image processed by the image preprocessing module, extracting thermal contour points of the infrared image and extracting structural edge points of the visible light image, constructing a space enhancement Raney mutual information objective function O (T) based on the extracted feature points, solving a space transformation parameter for maximizing the objective function by adopting a mixed optimization strategy, and finally outputting an optimal transformation parameter T * ; the formula of the spatially enhanced Raney mutual information objective function O (T) is as follows: ; Wherein: is the spatial transformation to be solved; And Is a weight coefficient; Representing a set of infrared points; Feature point set representing image to be seen light Through transformation A new point set is obtained; Is a Raney mutual information item; Is a spatially consistent term; The image transformation and fusion output module is used for resampling and geometrically transforming the infrared image by utilizing the optimal transformation parameter T * to enable the infrared image to be aligned with the visible light image in space, registering and fusing the two aligned images, evaluating an alignment result and finally outputting a fusion enhanced image.
2. The industrial inspection-oriented infrared and visible light image self-adaptive alignment system of claim 1, wherein the multi-modality image acquisition module comprises a thermal infrared imager and a high definition visible light camera rigidly secured together by mechanical structure.
3. The industrial inspection-oriented infrared and visible light image self-adaptive alignment system according to claim 2, wherein the specific steps of the image preprocessing module are as follows: 1) For visible light images, gray processing is firstly carried out, and color three-channel images are converted into single-channel gray images; 2) The converted visible light image and infrared image are subjected to improved self-adaptive Gaussian filtering by adopting a filter, and the standard deviation of the filter Based on variance of local areas of the image Dynamically adjusting; 3) Image enhancement is carried out by utilizing the Laplace operator, and edges and contours are highlighted; 4) And outputting an edge and contour enhanced image.
4. The system for adaptively aligning infrared and visible light images for industrial detection according to claim 3, wherein the feature extraction and alignment calculation module comprises the following specific steps: Step one, hierarchical feature point extraction For an infrared image, extracting a main contour in the image by using a Canny edge detector, and taking curvature extreme points, crossing points and end points of contour lines as characteristic points; for visible light images, extracting structural edges with potential corresponding relation with infrared contours; Let the feature point set extracted from the infrared image be The feature point set extracted from the visible light image is Wherein And The two-dimensional coordinates of the feature points in the respective images are that N 1 、N 2 respectively represent the number of the feature points extracted from the infrared image and the visible light image; step two, constructing a space enhancement Raney mutual information objective function O (T); Step three, solving the space transformation parameters by a hybrid optimization strategy, wherein the aim is to find an optimal transformation parameter T * so as to maximize a space enhancement Raney mutual information objective function O (T): 。
5. the adaptive alignment system for infrared and visible light images for industrial detection according to claim 4, wherein in the first step, the structural edges having potential correspondence with the infrared contour are extracted, specifically, the edge segments having high similarity with the shape of the infrared contour are screened out by calculating the intensity and direction of the edges, and then feature points are extracted on the edge segments.
6. The adaptive alignment system for infrared and visible light images for industrial detection according to claim 5, wherein in the second step, the formula of the rani mutual information item is: ; Wherein, the Is a joint probability distribution, a is the order of the entropy of renyi, and b is the adjustment parameter.
7. The adaptive alignment system for infrared and visible light images for industrial inspection according to claim 6, wherein in step three, a two-stage hybrid optimization strategy is adopted: 1) Global exploration phase transforms parameters using particle swarm optimization algorithm Encoded as the position of the particle, as an objective function As a function of fitness; 2) And in the local refining stage, a Powell local search algorithm is adopted to obtain a high-precision optimal transformation parameter T * .
8. The industrial inspection-oriented infrared and visible light image adaptive alignment system of claim 7, wherein the alignment results are quantitatively evaluated using peak signal-to-noise ratio, structural similarity index, or root mean square error index of manually labeled points.
9. An infrared and visible light image self-adaptive alignment method for industrial detection, the industrial detection-oriented infrared and visible light image self-adaptive alignment system according to any one of claims 1 to 8, comprising the steps of: S1, acquiring an infrared image and a visible light image of the same scene; Step S2, performing gray level processing on the visible light image to convert the color three-channel image into a single-channel gray level image, and performing improved adaptive Gaussian filtering on the converted visible light image and infrared image by using a filter, wherein the standard deviation of the filter Based on variance of local areas of the image Dynamically adjusting; S3, extracting layered characteristic points of the image processed by the image preprocessing module, and constructing a space enhancement Raney mutual information objective function O (T), solving a space transformation parameter maximizing the objective function by adopting a mixed optimization strategy, and outputting an optimal transformation parameter T * ; And S4, resampling and geometrically transforming the infrared image by utilizing the optimal transformation parameter T * to enable the infrared image to be aligned with the visible light image in space, registering and fusing the two aligned images, evaluating an alignment result, and finally outputting a fused enhanced image.
10. The method for adaptively aligning infrared and visible light images for industrial detection according to claim 9, wherein the step S3 specifically comprises the following steps: Step one, hierarchical feature point extraction For an infrared image, firstly extracting a main contour in the image by using a Canny edge detector, and then taking curvature extreme points, crossing points and end points of contour lines as characteristic points; for visible light images, extracting structural edges with potential corresponding relation with infrared contours; Let the feature point set extracted from the infrared image be The feature point set extracted from the visible light image is Wherein And The two-dimensional coordinates of the feature points in the respective images are that N 1 、N 2 respectively represent the number of the feature points extracted from the infrared image and the visible light image; step two, constructing a space enhancement Raney mutual information objective function O (T), wherein the formula is as follows: ; Wherein: is the spatial transformation to be solved; And Is a weight coefficient; Representing a set of infrared points; Feature point set representing image to be seen light Through transformation A new point set is obtained; Is a Raney mutual information item; Is a spatially consistent term; Step three, solving the space transformation parameters by a hybrid optimization strategy, wherein the aim is to find an optimal transformation parameter T * so as to maximize a space enhancement Raney mutual information objective function O (T): ; specifically, the hybrid optimization strategy adopts a two-stage hybrid optimization strategy, namely, a global exploration stage uses a particle swarm optimization algorithm to transform parameters Encoded as the position of the particle, as an objective function And the local refining stage adopts a Powell local search algorithm to obtain an optimal transformation parameter T * as a fitness function.

Description

Industrial detection-oriented infrared and visible light image self-adaptive alignment system and method Technical Field The invention relates to the technical field of industrial detection, in particular to an infrared and visible light image self-adaptive alignment system and method for industrial detection. Background The multi-mode sensor fusion technology is widely applied in the fields of modern industrial detection, security monitoring and the like. While the combination of an infrared thermal imaging camera and a visible light camera is one of the most classical and critical configurations. The infrared camera can capture temperature distribution information on the surface of an object, is not influenced by ambient light, can penetrate smoke and dust, can effectively detect overheat of equipment, energy leakage or find a camouflage target, and can provide texture, color and detail information which have high resolution and accord with human vision habits. By carrying out accurate pixel level alignment and fusion on the infrared image and the visible light image of the same scene, an enhanced image simultaneously containing heat distribution information and visual details can be generated, so that situation awareness capability and diagnosis accuracy are greatly improved. For example, the method can be applied to power inspection, can accurately position the specific position of the overheat fault point on equipment, and can simultaneously observe whether the assembly state and the heat distribution of a product are abnormal or not in industrial manufacture. Precise alignment of infrared and visible images is a very challenging task because the physical principles of the two imaging modes differ, resulting in significant differences in modality between the images. The method is characterized by comprising the steps of inconsistent gray scale characteristics (temperature is reflected by an infrared image, reflectivity is reflected by a visible light image), mismatching of texture details, non-correspondence of edge structures and the like. The existing alignment methods can be mainly divided into the following categories: The method based on manual characteristics is a traditional mainstream method. And extracting manually designed features with certain invariance to the model change on the infrared image and the visible light image respectively, then establishing a corresponding relation between feature points, and finally completing alignment by solving a space transformation model. However, it appears unstable on infrared images with poor gradient information and visible images with excessive texture. The extracted feature points may have insufficient quantity and poor repeatability, or the same physical position cannot be detected under different modes, so that the matching success rate is low. Similarity measure mutual information is a classical measure for multi-modal image registration, which measures the statistical dependency between the gray scale distributions of two images. However, the global statistical characteristics are calculated by mutual information, the spatial position information of the feature points is ignored, and when the scene content is complex or local deformation exists, local extremum is easily trapped, so that registration failure is caused. Some methods use CNN direct regression transformation parameters, and some learn a similarity measure for registration. However, deep learning methods typically require extensive, precisely registered infrared-visible image pairs for supervised training, while obtaining such data sets is costly and the generalization ability of the model may be degraded to new, unseen scenes. Based on the above, the invention designs an infrared and visible light image self-adaptive alignment system and method for industrial detection to solve the above problems. Disclosure of Invention Aiming at the defects existing in the prior art, the invention provides the infrared and visible light image self-adaptive alignment system and the method for the industrial detection, which are independent of a large amount of training data, can effectively overcome modal differences and take global robustness and local precision into account. In order to achieve the above purpose, the invention is realized by the following technical scheme: an infrared and visible light image self-adaptive alignment system for industrial detection, comprising: the multi-mode image acquisition module is used for acquiring an infrared image and a visible light image of the same scene; the image preprocessing module is used for receiving the original infrared image and visible light image and performing preprocessing operation; The feature extraction and alignment calculation module is used for extracting layered feature points of the image processed by the image preprocessing module, extracting thermal contour points of the infrared image and extracting structural edge points of the