CN-122023113-A - Large parallax image stitching method, device, equipment and storage medium

CN122023113ACN 122023113 ACN122023113 ACN 122023113ACN-122023113-A

Abstract

The invention discloses a large parallax image stitching method, a device, equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the steps of carrying out feature point matching on a feature point set to be spliced and a reference feature point set to obtain a feature matching point pair set, layering parallaxes of overlapping areas intersecting with a reference image in an image to be spliced according to parallax values of each feature matching point pair in the feature matching point pair set to obtain a plurality of parallax layers and matching feature point pair sets corresponding to the parallax layers, determining each geometric feature area according to the matching feature points in the matching feature point pair set of each parallax layer by using an area growth method, and carrying out fusion processing on the image to be spliced and the reference image according to a global homography matrix and a local homography matrix of each geometric feature area to obtain a target spliced image. By utilizing the technical scheme provided by the invention, the accuracy of splicing the large parallax scene images can be improved.

Inventors

JIANG PENG
XU XIAOPING
HAN YIPING
GAO YUAN
WANG QIAN
SONG NING
HAN DONG
CHEN CHENG
ZHANG DAMING
LU XINQIAN
LU JIACHENG

Assignees

中国人民解放军陆军装备部驻南京地区军事代表局驻南京地区第三军事代表室

Dates

Publication Date: 20260512
Application Date: 20251211

Claims (10)

1. A method for stitching large parallax images, the method comprising: Acquiring an image to be spliced, a reference image, a set of feature points to be spliced of the image to be spliced, a feature descriptor to be spliced of each feature point to be spliced, a set of reference feature points of the reference image and a reference feature descriptor of each reference feature point; Performing feature point matching on the feature point set to be spliced and the reference feature point set based on the feature descriptors to be spliced of each feature point to be spliced and the reference feature descriptors of each reference feature point to obtain a feature matching point pair set; layering parallaxes of overlapping areas intersecting with the reference image in the image to be spliced according to the parallax value of each feature matching point pair in the feature matching point pair set to obtain a plurality of parallax layers and a matching feature point pair set corresponding to each parallax layer; determining each geometric feature region by using a region growing method according to the matched feature points to be spliced in the matched feature point pair set of each parallax layer; and carrying out fusion processing on the image to be spliced and the reference image according to a global homography matrix and a local homography matrix of each geometrical characteristic area to obtain a target spliced image, wherein the global homography matrix represents a global space mapping relation from the image to be spliced to the reference image.
2. The large parallax image stitching method according to claim 1, wherein layering the parallaxes of the overlapping area intersecting with the reference image in the image to be stitched according to the parallax value of each feature matching point pair in the feature matching point pair set to obtain a plurality of parallax layers and a matching feature point pair set corresponding to each parallax layer, and the method comprises: Determining an overlapping area intersecting with the reference image in the image to be spliced; Determining the parallax value of each feature matching point pair; According to the parallax value of each feature matching point pair, determining kurtosis corresponding to the parallax value, wherein the kurtosis is a statistic for measuring the distribution feature of the parallax value; Determining the number of layers of parallax layering based on the kurtosis; and layering the parallaxes in the overlapped area by adopting a preset clustering method based on the number of layers of the parallaxes layering, so as to obtain a plurality of parallaxes layers and a matched characteristic point pair set corresponding to each parallaxes layer.
3. The large parallax image stitching method according to claim 2, wherein the determining an overlapping area intersecting the reference image in the image to be stitched includes: determining the global homography matrix according to the feature matching point pair set; Based on the global homography matrix, carrying out transformation processing on four corner points of the image to be spliced to obtain four transformation corner points of the image to be spliced, wherein the four corner points comprise an upper left corner point, an upper right corner point, a lower right corner point and a lower left corner point, and the four transformation corner points comprise an upper left transformation corner point, an upper right transformation corner point, a lower right transformation corner point and a lower left transformation corner point; determining canvas offset according to coordinates of the up-left transformation corner points of the images to be spliced; Based on the canvas offset, respectively counteracting the canvas offset for the four transformation corner points of the image to be spliced serving as the left image and the four corner points of the reference image serving as the right image to obtain four canvas corner points of the image to be spliced and four canvas corner points of the reference image converted into a canvas coordinate system; Determining the minimum bounding box covering the four canvas corner points of the image to be spliced as a first bounding box, and determining the minimum bounding box covering the four canvas corner points of the reference image as a second bounding box; Determining an intersection of the first bounding box and the second bounding box as a canvas overlapping area; determining the position of the canvas overlapping region in the image to be spliced as an overlapping region intersecting with the reference image in the image to be spliced; and determining the position of the canvas overlapping region in the reference image as an overlapping region which is intersected with the image to be spliced in the reference image.
4. The method for stitching large parallax images according to claim 1, wherein the determining each geometric feature region by using a region growing method according to the matching feature points to be stitched in the matching feature point pair set of each parallax layer includes: According to the coordinates of the matched characteristic points to be spliced in the matched characteristic point pair set of each parallax layer in the image to be spliced, clustering the matched characteristic points to be spliced in the matched characteristic point pair set to obtain a target number of characteristic point space groups of each parallax layer; According to the characteristic points in each characteristic point space group, determining each geometrical characteristic point set of each characteristic point space group; And determining each geometrical characteristic region by using the geometrical characteristic points in each geometrical characteristic point set as starting pixel points and utilizing a region growing method.
5. The method of large parallax image stitching according to claim 4, wherein the determining each geometric feature region by using a region growing method with the geometric feature point in each geometric feature point set as a starting pixel point includes: Traversing each geometrical feature point set; under the condition of traversing any geometrical feature point set, determining a current geometrical feature area of the current geometrical feature point set, and setting the current geometrical feature area as empty; Any geometric feature point is taken out from the current geometric feature point set and is put into the current geometric feature region, and the geometric feature point put into the current geometric feature region is taken as an initial pixel point; Determining adjacent pixel points of the initial pixel point, and adding the adjacent pixel points meeting preset conditions into the current geometric feature area; repeating the step of determining the adjacent pixel points of the starting pixel point by taking the adjacent pixel points added into the current geometric feature area as the starting pixel point, adding the adjacent pixel points meeting the preset condition into the current geometric feature area until the adjacent pixel points added into the current geometric feature area are taken as the starting pixel point until no pixel points are added into the current geometric feature area; And determining whether the current geometric feature point set is empty, and repeating the step of taking out any geometric feature point from the current geometric feature point set and putting the geometric feature point into the current geometric feature region until the current geometric feature point set is determined to be empty until the current geometric feature point set is empty under the condition that the current geometric feature point set is not empty, so as to obtain the current geometric feature region.
6. The method for stitching images with large parallax according to claim 1, wherein the fusing the images to be stitched and the reference image according to the global homography matrix and the local homography matrix of each geometrical feature area to obtain a target stitched image includes: Determining a local homography matrix of each geometrical characteristic area; Correcting the images to be spliced according to the global homography matrix and the local homography matrix of each geometrical characteristic area to obtain corrected images to be spliced; Performing color correction on the corrected image to be spliced according to the reference image to obtain a corrected image to be spliced; Determining a target suture line of the color correction image to be spliced and the reference image; and carrying out fusion processing on the image to be spliced and the reference image along the target suture line by a gradual-in gradual-out fusion method to obtain the target spliced image.
7. The method for stitching images with large parallax according to claim 6, wherein the correcting the images to be stitched according to the global homography matrix and the local homography matrix of each geometrical feature area to obtain corrected images to be stitched includes: according to the global homography matrix, performing transformation processing on the non-geometric feature area in the image to be spliced to obtain a global transformed image to be spliced; And carrying out local transformation processing on each geometrical characteristic region in the overlapped region in the global transformation image to be spliced according to the target homography matrix of each geometrical characteristic region to obtain the corrected image to be spliced, wherein the target homography matrix of each geometrical characteristic region is obtained by carrying out weighted fusion on the basis of the global homography matrix and the local homography matrix of each geometrical characteristic region.
8. A large parallax image stitching apparatus, characterized in that the apparatus comprises: the data acquisition module is used for acquiring an image to be spliced, a reference image, a set of feature points to be spliced of the image to be spliced, a feature descriptor to be spliced of each feature point to be spliced, a set of reference feature points of the reference image and a reference feature descriptor of each reference feature point; The feature point matching module is used for carrying out feature point matching on the feature point set to be spliced and the reference feature point set based on the feature descriptors to be spliced of each feature point to be spliced and the reference feature descriptors of each reference feature point to obtain a feature matching point pair set; The parallax layering module is used for layering parallaxes of overlapping areas intersecting the reference image in the image to be spliced according to the parallax value of each feature matching point pair in the feature matching point pair set to obtain a plurality of parallax layers and matching feature point pair sets corresponding to each parallax layer; the geometric feature region determining module is used for determining each geometric feature region by utilizing a region growing method according to the matched feature points to be spliced in the matched feature point pair set of each parallax layer; And the image fusion module is used for carrying out fusion processing on the image to be spliced and the reference image according to the global homography matrix and the local homography matrix of each geometrical characteristic area to obtain a target spliced image, wherein the global homography matrix represents the global space mapping relation from the image to be spliced to the reference image.
9. An electronic device comprising a processor and a memory having stored therein at least one instruction and at least one program, the at least one instruction and the at least one program loaded and executed by the processor to implement the large parallax image stitching method of any one of claims 1 to 7.
10. A computer storage medium having at least one instruction and at least one program stored therein, the at least one instruction and the at least one program loaded and executed by a processor to implement the large parallax image stitching method according to any one of claims 1 to 7.

Description

Large parallax image stitching method, device, equipment and storage medium Technical Field The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for stitching a large parallax image. Background Image stitching is an important task in the field of image processing, which aims to stitch acquired multiple images into a wide-viewing-angle, high-resolution panoramic image. With the development of information science and technology, image stitching methods are more and more mature, and most of the stitching methods can achieve better effects for small parallax image stitching, however, for large-viewing-angle image stitching, ghosting or obvious folds often occur in the stitching results. And the large parallax image is obvious in displacement difference of foreground, middle and background, so that the problems of failure of fitting a global homography matrix, splicing deformation, hard transition of an overlapping area and the like are easy to occur. Thus, there is a need to provide a more reliable solution. Disclosure of Invention The invention aims to overcome the defects in the prior art and provide a large parallax image splicing method, device, equipment and storage medium, which can improve the accuracy and reliability of large parallax scene image splicing. In order to achieve the above purpose, the invention is realized by adopting the following technical scheme: In one aspect, the present invention provides a large parallax image stitching method, which includes: Acquiring an image to be spliced, a reference image, a set of feature points to be spliced of the image to be spliced, a feature descriptor to be spliced of each feature point to be spliced, a set of reference feature points of the reference image and a reference feature descriptor of each reference feature point; Performing feature point matching on the feature point set to be spliced and the reference feature point set based on the feature descriptors to be spliced of each feature point to be spliced and the reference feature descriptors of each reference feature point to obtain a feature matching point pair set; layering parallaxes of overlapping areas intersecting with the reference image in the image to be spliced according to the parallax value of each feature matching point pair in the feature matching point pair set to obtain a plurality of parallax layers and a matching feature point pair set corresponding to each parallax layer; determining each geometric feature region by using a region growing method according to the matched feature points to be spliced in the matched feature point pair set of each parallax layer; and carrying out fusion processing on the image to be spliced and the reference image according to a global homography matrix and a local homography matrix of each geometrical characteristic area to obtain a target spliced image, wherein the global homography matrix represents a global space mapping relation from the image to be spliced to the reference image. In some possible embodiments, layering parallaxes of overlapping areas intersecting with the reference image in the image to be stitched according to the parallax value of each feature matching point pair in the feature matching point pair set, to obtain a plurality of parallax layers and matching feature point pair sets corresponding to each parallax layer, including: Determining an overlapping area intersecting with the reference image in the image to be spliced; Determining the parallax value of each feature matching point pair; According to the parallax value of each feature matching point pair, determining kurtosis corresponding to the parallax value, wherein the kurtosis is a statistic for measuring the distribution feature of the parallax value; Determining the number of layers of parallax layering based on the kurtosis; and layering the parallaxes in the overlapped area by adopting a preset clustering method based on the number of layers of the parallaxes layering, so as to obtain a plurality of parallaxes layers and a matched characteristic point pair set corresponding to each parallaxes layer. In some possible embodiments, the determining the overlapping area of the image to be stitched that intersects the reference image includes: determining the global homography matrix according to the feature matching point pair set; Based on the global homography matrix, carrying out transformation processing on four corner points of the image to be spliced to obtain four transformation corner points of the image to be spliced, wherein the four corner points comprise an upper left corner point, an upper right corner point, a lower right corner point and a lower left corner point, and the four transformation corner points comprise an upper left transformation corner point, an upper right transformation corner point, a lower right transformation corner point and a lower