CN-122023474-A - Automatic calibration double-light fusion image configuration method, device, equipment and medium

CN122023474ACN 122023474 ACN122023474 ACN 122023474ACN-122023474-A

Abstract

The invention relates to an automatic calibration double-light fusion image configuration method, a device, equipment and a medium, wherein the method comprises the steps of obtaining a visible light image and an infrared image aiming at the same road area; the method comprises the steps of respectively extracting features of a visible light image and an infrared image to obtain a matching point pair and a confidence coefficient between the visible light image and the infrared image, carrying out geometric consistency check and fitting on the matching point pair to obtain a homography matrix describing geometric relations between the visible light image and the infrared image, carrying out quality evaluation on the homography matrix to obtain a homography matrix and a quality index which pass the quality evaluation, and carrying out pixel level alignment and fusion according to the homography matrix and the quality index which pass the quality evaluation. According to the invention, through automatic feature extraction and matching, geometric consistency check and quality assessment, efficient calibration and accurate fusion of the double-light images are realized, the labor cost is obviously reduced, the calibration precision and the fused image quality are improved, and the robustness and the practicability of the system under a complex scene are enhanced.

Inventors

WU KEWEI
HE XIAOGANG
CHEN ZHIQIANG
YAN JINYU

Assignees

北京邮电大学

Dates

Publication Date: 20260512
Application Date: 20251219

Claims (10)

1. The automatic calibration double-light fusion image configuration method is characterized by comprising the following steps of: obtaining a visible light image and an infrared image aiming at the same road area, wherein the visible light image and the infrared image comprise the same scene and the same target; Respectively extracting features of the visible light image and the infrared image to obtain a matching point pair and confidence between the visible light image and the infrared image; according to the confidence coefficient of the matching point pair, performing geometric consistency check and fitting on the matching point pair to obtain a homography matrix describing the geometric relationship between the visible light image and the infrared image; And carrying out quality evaluation on the homography matrix to obtain a homography matrix and a quality index which pass the quality evaluation, and carrying out pixel level alignment and fusion on the visible light image and the infrared image to be processed according to the homography matrix and the quality index which pass the quality evaluation.
2. The method according to claim 1, wherein the feature extracting the visible light image and the infrared image respectively to obtain the matching point pair and the confidence coefficient between the visible light image and the infrared image includes: extracting a plurality of sparse feature points and descriptors of each sparse feature point from the visible light image and the infrared image respectively; and performing feature matching on the sparse feature points corresponding to the visible light image and the infrared image and the descriptors of each sparse feature point to obtain a matching point pair and confidence between the visible light image and the infrared image.
3. The method of claim 1, wherein said performing a geometric consistency check and fitting of said matching point pairs based on the confidence level of said matching point pairs to obtain a homography matrix describing the geometric relationship between said visible light image and said infrared image comprises: According to the confidence coefficient of the matching point pair, performing geometric consistency check on the matching point pair, and screening out internal points conforming to a geometric model; Fitting the internal points conforming to the geometric model to obtain the homography matrix.
4. A method according to any one of claims 1 to 3, further comprising: Determining quality indexes according to the homography matrix and the matching point pairs, wherein the quality indexes comprise reprojection errors, interior point rates and space coverage; the quality evaluation of the homography matrix comprises the following steps: And carrying out quality evaluation on the homography matrix according to the quality index.
5. The method of claim 4, wherein said evaluating the quality of the homography matrix based on the quality metrics comprises: judging whether the fitting precision of the homography matrix meets the requirement according to the reprojection error and a first set threshold; judging whether the reliability of the homography matrix meets the requirement or not according to the interior point rate and a second set threshold value; judging whether the distribution condition of the matching point pairs corresponding to the homography matrix meets the requirement or not according to the space coverage and a third set threshold; and determining the homography matrix with the fitting precision, the reliability and the distribution condition meeting the respective corresponding requirements as the homography matrix passing the quality evaluation.
6. An automatically calibrated dual-light fusion image configuration device, comprising: The acquisition module is used for acquiring a visible light image and an infrared image aiming at the same road area, wherein the visible light image and the infrared image comprise the same scene and the same target; The matching module is used for extracting the characteristics of the visible light image and the infrared image respectively to obtain a matching point pair and a confidence coefficient between the visible light image and the infrared image; The homography matrix preliminary determination module is used for carrying out geometric consistency check and fitting on the matching point pairs according to the confidence coefficient of the matching point pairs to obtain homography matrixes describing geometric relations between the visible light images and the infrared images; The homography matrix final determining module is used for carrying out quality evaluation on the homography matrix to obtain a homography matrix and a quality index which pass the quality evaluation, and carrying out pixel level alignment and fusion on the visible light image and the infrared image to be processed according to the homography matrix and the quality index which pass the quality evaluation.
7. The apparatus of claim 6, wherein the matching module is configured to, when performing feature extraction on the visible light image and the infrared image respectively to obtain a matching point pair and a confidence coefficient between the visible light image and the infrared image, specifically: extracting a plurality of sparse feature points and descriptors of each sparse feature point from the visible light image and the infrared image respectively; and performing feature matching on the sparse feature points corresponding to the visible light image and the infrared image and the descriptors of each sparse feature point to obtain a matching point pair and confidence between the visible light image and the infrared image.
8. The apparatus of claim 6, wherein the homography preliminary determination module is configured to, when performing geometric consistency check and fitting on the matching point pairs according to the confidence level of the matching point pairs, obtain a homography matrix describing a geometric relationship between the visible light image and the infrared image: According to the confidence coefficient of the matching point pair, performing geometric consistency check on the matching point pair, and screening out internal points conforming to a geometric model; Fitting the internal points conforming to the geometric model to obtain the homography matrix.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1-5 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1-5.

Description

Automatic calibration double-light fusion image configuration method, device, equipment and medium Technical Field The invention relates to the technical field of computer vision and intelligent transportation, in particular to an automatic calibration double-light fusion image configuration method, device, equipment and medium. Background The dual-light system (visible light VIS/infrared IR) needs to complete the cross-spectrum geometric pairs in engineering deployment, the existing calibration modes have the following technical problems: 1) The artificial target (chessboard/ChArUco/AprilTag heating plate) is obtained by off-line determination of internal and external participation homographies and fine adjustment after on-site installation. The process has the problems of high labor cost, limited field conditions, poor repeatability and difficult timely correction of the external parameter drift in long-term operation. 2) Classical feature method (SIFT/orb+ransac) is limited in cross-spectrum matchability and poor in low texture/backlight/thermal saturation scene stability. Specifically, visible light and infrared belong to cross-spectrum imaging, appearance distribution differences lead to weak matchability and high mismatching rate of classical manual features (such as SIFT/ORB) during cross-spectrum matching, and a large amount of dynamic prospects caused by traffic flows and pedestrians further interfere geometric estimation in traffic scenes to lead to unstable homography/mapping solution. In addition, the two prior arts have the following technical problems: Engineering pain points, repeated target distribution and manual operation are time-consuming when a large number of devices are deployed, external parameters are easy to drift after scene migration, and parameter expression lacks unified configuration for multiplexing and rollback. The method is highly dependent on manpower, and needs target distribution, framing, illumination control and multi-round shooting. The cross-spectrum robustness is insufficient, and the traditional features are sparse in matching and many in mismatching when the VIS/IR difference is large. On-line maintenance is difficult, an automatic updating mechanism is lacking after the external parameter drifts, and fusion tearing and ghost are difficult to inhibit in time. Lacking engineering configuration, parameters are difficult to migrate across devices, and rollback and traceability are poor. Disclosure of Invention The invention aims to solve at least one technical problem by providing an automatic calibration double-light fusion image configuration method, device, equipment and medium. In a first aspect, the present invention provides a method for automatically calibrating a dual-light fusion image, the method comprising: Obtaining a visible light image and an infrared image aiming at the same road area, wherein the visible light image and the infrared image comprise the same scene and the same target; respectively extracting features of the visible light image and the infrared image to obtain a matching point pair and a confidence coefficient between the visible light image and the infrared image; according to the confidence coefficient of the matching point pair, carrying out geometric consistency check and fitting on the matching point pair to obtain a homography matrix describing the geometric relationship between the visible light image and the infrared image; And carrying out quality evaluation on the homography matrix to obtain a homography matrix and a quality index which pass the quality evaluation, and carrying out pixel level alignment and fusion on the visible light image and the infrared image to be processed according to the homography matrix and the quality index which pass the quality evaluation. The method has the advantages that the visible light image and the infrared image of the same road area are obtained, the two images are respectively subjected to feature extraction to obtain the matching point pair and the confidence coefficient, the geometric consistency check and fitting are further carried out according to the confidence coefficient of the matching point pair to obtain the homography matrix describing the geometric relationship between the visible light image and the infrared image, and finally the homography matrix is subjected to quality evaluation to obtain the homography matrix and the quality index which pass through the quality evaluation, so that the pixel level alignment and fusion of the visible light image and the infrared image to be processed are realized. The method realizes automatic calibration of the double-light image, does not need manual target distribution and off-line calibration, obviously reduces labor cost and deployment difficulty, and improves calibration efficiency. Through feature extraction and confidence coefficient screening, the accuracy and the reliability of the matching point pairs are ensured by combining