CN-118710689-B - Visible light and infrared image registration method based on region similarity

CN118710689BCN 118710689 BCN118710689 BCN 118710689BCN-118710689-B

Abstract

The invention provides a visible light and infrared image registration method based on region similarity, which comprises the steps of rendering a synthetic data set by using a graph with obvious corner characteristics, training a key point detector on the synthetic data set, performing scaling and gray scale processing on a visible light and infrared registration image pair, inputting the key point detector to generate corresponding key point coordinates, performing random homography transformation on the infrared and visible light images with the key points, simulating a transformation relation between the image pair, generating grids by Delaunay triangulation, processing the topological relation of the point set and triangle grid generation, inputting the transformed image pair into a descriptor generation network, designing a region-based loss function, and training the registration relation between the grids. The method effectively solves the problem of inconsistent angular points in cross-mode image matching, realizes image registration with high precision and high anti-interference capability, and can be applied to the fields of remote sensing image processing, target tracking and the like.

Inventors

ZHANG HONG
WU BINGXUAN
YUAN DING
YANG YIFAN
SONG JIANBO

Assignees

北京航空航天大学

Dates

Publication Date: 20260512
Application Date: 20240620

Claims (3)

1. A method for registering visible light and infrared images based on region similarity, the method comprising the steps of: Step 1, rendering a basic pattern to obtain a combined image, recording corner coordinates in the combined image, and obtaining a specified number of combined images to form a synthetic data set; step 2, training a key point detector on the synthesized data set obtained in the step 1; Step 3, after the visible light image and the infrared image are subjected to scaling and gray scale processing, inputting the visible light image and the infrared image into the key point detector to generate corresponding key point coordinates; step 4, randomly selecting an infrared image with key point coordinates and a visible light image to perform random homography transformation; Step 5, performing Delaunay triangulation on the infrared image and the visible light image with the key point coordinates after the random homography transformation according to the key point coordinate positions; step 6, inputting the infrared image and the visible light image with the grids after Delaunay triangulation into a descriptor generating network model for model training; Step 7, generating a network model by using the trained descriptors for reasoning test to complete registration tasks of the unregistered visible light image and the infrared image, wherein, The step4 comprises the following steps: step 4.1, randomly cutting a rectangular block in the randomly selected infrared image or visible light image; Step 4.2, four vertex coordinates of the rectangular block are obtained; step 4.3, adding random disturbance in a specified range to the four vertex coordinates to obtain four new vertex coordinates; Step 4.4, calculating homography matrix according to four new vertex coordinates ; Step 4.5, according to homography matrix Performing homography transformation on the randomly selected infrared image or visible light image to obtain a new image; the step 6 comprises the following steps: step 6.1, establishing a grid corresponding relation between an infrared image with a grid and a visible light image; Step 6.2, determining descriptor vectors of corresponding points in the grids corresponding to the infrared image and the visible light image by using a Lagrangian multiplier method, and enabling the two descriptor vectors to be consistent; Step 6.3, traversing each grid, each pair of infrared images and visible light images, and generating network parameters through gradient back-transfer optimization descriptors; step 6.4, stopping after the maximum training round is reached, and obtaining a trained descriptor to generate a network model; the step 7 comprises the following steps: Step 7.1, performing scaling and gray scale processing on an unregistered image pair of a visible light image and an infrared image; step 7.2, detecting key points of the unregistered image pair by using a key point detector, and ensuring that the number of the key points is consistent; step 7.3, performing Delaunay triangulation according to the coordinates of the key points to generate a grid; step 7.4, generating a network model through the trained descriptors to predict the corresponding relation between grids of the unregistered image pair, and regressing the coordinates of the matched key point pair of the unregistered image pair and the matched confidence score; step 7.5, screening out confidence score higher than the threshold value through a screening algorithm And (3) calculating a homography matrix between the unregistered image pairs according to the matching relation of the dot pairs, and finishing homography conversion from infrared to visible light images or from visible light to infrared images.
2. The method for registering visible and infrared images based on regional similarity according to claim 1, wherein the step 1 comprises: Step 1.1, obtaining the angular point coordinates of each basic pattern The basic pattern comprises triangles, quadrilaterals, lines, cubes, chessboard and stars; Step 1.2, respectively carrying out random homography transformation on each basic pattern to obtain new angular point coordinates ; Step 1.3, randomly combining the homography transformed basic pattern onto an image with a specified resolution, and obtaining coordinates of each corner point on the combined image ; Step 1.4, randomly selecting coordinates of a plurality of corner points on each combined image as labels of key point positions; And step 1.5, repeatedly executing the steps 1.2-1.4 until the combination images with the specified number N are rendered, so as to form a synthetic data set.
3. The method for registering visible and infrared images based on regional similarity according to claim 1, wherein the step 2 comprises: step 2.1, dividing the combined image in the composite data set into a plurality of cells according to the range covered by the labels, and ensuring that each cell has at least one label; step 2.2, predicting a keypoint coordinate by a keypoint detector for each cell ; Step 2.3, according to the predicted key point coordinates And calculating a loss value to optimize network parameters, and if a plurality of labels exist in one cell, randomly selecting one label as the key point coordinate of the cell.

Description

Visible light and infrared image registration method based on region similarity Technical Field The invention belongs to the fields of night monitoring and reconnaissance, medical diagnosis, planet ground surface monitoring and the like, and particularly relates to a visible light and infrared image registration method based on regional similarity. Background The image matching technology has wide application in the fields of computer vision, remote sensing, robot navigation, medical image processing and the like. With the development of science and technology, the research and application of the image matching technology in various fields are deeper and deeper, and the requirements on the accuracy and speed of image matching are higher and higher. The traditional image matching method is mainly based on gray information of images, such as template matching, feature point matching and the like. However, these methods have certain limitations in processing different types of images. For example, in the matching of a visible light image and an infrared image, it is often difficult to achieve a satisfactory matching effect directly using a conventional image matching method due to the imaging mechanism and the illumination condition. In order to solve the problem of matching the visible light image and the infrared image, researchers have recently proposed some image matching methods for specific applications. These methods are mainly divided into two categories, feature-based matching methods and model-based matching methods. The feature-based matching method first extracts representative feature points, such as corner points, edge points, and the like, from the visible light image and the infrared image. Then, the matching relationship is determined by calculating a similarity measure (such as euclidean distance, mutual information, etc.) between the feature points. The method can improve the matching robustness to a certain extent, but is sensitive to geometric transformation and illumination change among images. The model-based matching method maps the visible light image into the coordinate system of the infrared image by establishing a transformation model between the visible light image and the infrared image. The key to this type of approach is the creation of the model and the solution of the parameters. Common transformation models include linear models, affine models, projection models, and the like. The model-based matching method has certain advantages in terms of processing geometric transformation and illumination change between images, but the complexity of the model and the solving process may result in large calculation amount. Although the existing image matching method can solve the problem of matching the visible light image and the infrared image to a certain extent, challenges in matching precision, calculation efficiency, robustness and the like still exist, and especially most of the existing methods do not solve the problem of inconsistent angular points existing in matching the visible light image and the infrared image. Disclosure of Invention Aiming at the problem that the corner points of the visible light and infrared image pairs are inconsistent, the invention provides a method for registering the visible light and infrared image based on region similarity, which adopts region matching to replace characteristic point pair matching, combines a Lagrange multiplier method, and searches for the optimal region matching relationship between the visible region and the infrared region by constructing a global optimization objective function and combining corresponding constraint conditions, thereby overcoming the problem of corner point offset and improving the robustness and reliability of image registration. In order to achieve the above purpose, the technical solution adopted by the present invention is: A method for registering visible light and infrared images based on region similarity comprises the following specific implementation steps: Step 1, rendering a basic pattern to obtain a combined image, recording corner coordinates in the combined image, and obtaining a specified number of combined images to form a synthetic data set; step 2, training a key point detector on the synthesized data set obtained in the step 1; Step 3, after the visible light image and the infrared image are subjected to scaling and gray scale processing, inputting the visible light image and the infrared image into the key point detector to generate corresponding key point coordinates; step 4, randomly selecting an infrared image with key point coordinates and a visible light image to perform random homography transformation; Step 5, performing Delaunay triangulation on the infrared image and the visible light image with the key point coordinates after the random homography transformation according to the key point coordinate positions; step 6, inputting the infrared image and the visible light image wit