CN-121982076-A - Air-space image alignment method, device, equipment and storage medium

CN121982076ACN 121982076 ACN121982076 ACN 121982076ACN-121982076-A

Abstract

The invention discloses an air-ground image alignment method which comprises the steps of carrying out air triangulation on an air image set and a ground image set, extracting key images from the air image set and the ground image set based on obtained air three results, inputting the key images into a deep learning model, outputting an air relative depth image and a ground relative depth image, converting the key images into an air absolute depth image and a ground absolute depth image, generating an air dense point cloud and a ground dense point cloud based on the absolute depth image, extracting and obtaining an air virtual plane primitive and a ground virtual plane primitive from the air dense point cloud, extracting texture features and geometric features of the air virtual plane primitive and the ground virtual plane primitive, screening out a preset number of plane primitive pairs, calculating conversion parameters based on the plane primitive pairs, and converting the air point cloud into a ground coordinate system according to the conversion parameters so as to realize alignment of the air image and the ground image.

Inventors

XU DONG
LIU YANG
ZHANG MING
YANG ZEXIN
LIU KEKE
Wu Yousi
LUO YUANYUAN
DU GUANGLONG
GAO ZHIGUO

Assignees

广州市城市规划勘测设计研究院有限公司
华南理工大学

Dates

Publication Date: 20260505
Application Date: 20251212

Claims (10)

1. A method for aligning space images, comprising: performing aerial triangulation on the aerial image set and the ground image set containing GPS position information to obtain aerial three results and ground three results; Extracting key images from the aviation image set and the ground image set based on the aviation air result and the ground air result respectively to obtain an aviation key image set and a ground key image set; Inputting each aviation key image and each ground key image into a depth learning model, outputting an aviation relative depth map and a ground relative depth map, converting the aviation relative depth map into an aviation absolute depth map, and converting the ground relative depth map into a ground absolute depth map; generating an aviation dense point cloud and a ground dense point cloud based on the aviation absolute depth map and the ground absolute depth map, and extracting aviation virtual plane primitives and ground virtual plane primitives from the aviation dense point cloud and the ground dense point cloud; extracting texture features and geometric features of the aviation virtual plane primitives and the ground virtual plane primitives, and screening out preset number of plane primitive pairs based on the texture features and the geometric features; And calculating conversion parameters based on the plane primitive pairs, and converting the aviation point cloud into a ground coordinate system according to the conversion parameters so as to realize the alignment of the aviation image and the ground image.
2. The air-to-ground image alignment method of claim 1, wherein the extracting key images from the aerial image set and the ground image set based on the aerial air three results and the ground air three results, respectively, to obtain an aerial key image set and a ground key image set, comprises: Acquiring aviation sparse point clouds and ground sparse point clouds from the aviation air three results and the ground air three results, and projecting the aviation sparse point clouds and the ground sparse point clouds to an xy plane to obtain projected aviation point sets and ground point sets; calculating an overlapping area of the aviation point set and the ground point set, covering a buffer area with a preset range in the overlapping area to obtain a buffer area, and uniformly sampling in the buffer area to obtain a sampling point set; and calculating the proportion of each aerial image to the visible points in the sampling point set, and reserving the images with the proportion larger than a preset proportion threshold as key images to obtain an aerial key image set and a ground key image set.
3. The air-to-ground image alignment method of claim 1, wherein said converting the aerial relative depth map to an aerial absolute depth map comprises: acquiring aviation sparse point clouds from the aviation air three results; calculating a depth true value and a pixel coordinate projected to an image coordinate of each point in the aviation sparse point cloud; determining a scaling scale factor and an offset based on the depth truth value and the pixel coordinates; and scaling and shifting the aviation relative depth map according to the scaling scale factors and the offset to obtain an aviation absolute depth map.
4. The air-to-ground image alignment method of claim 1, wherein the generating an aviation dense point cloud and a ground dense point cloud based on the aviation absolute depth map and the ground absolute depth map comprises: Performing back projection operation on each pixel in the aviation absolute depth map, and calculating 3D coordinates of the pixel in a 3D space to obtain aviation dense point cloud containing 3D coordinates of all the pixels; And carrying out back projection operation on each pixel in the ground absolute depth map, and calculating the 3D coordinates of the pixel in a 3D space to obtain a ground dense point cloud containing the 3D coordinates of all the pixels.
5. The air-to-ground image alignment method of claim 1, wherein the extracting texture features and geometric features of the aerial virtual plane primitive and the ground virtual plane primitive comprises: Respectively acquiring an aviation 3D point set and a ground 3D point set corresponding to the aviation virtual plane primitive and the ground virtual plane primitive; Calculating a first vertical distance from each 3D point in the aviation 3D point set to the aviation virtual plane primitive, and calculating a second vertical distance from each 3D point in the ground 3D point set to the ground virtual plane primitive; removing 3D points with the first vertical distance and the second vertical distance larger than a preset distance threshold value to obtain a first point set and a second point set after removing; Counting the color information of each point in the first point set and the second point set to obtain a first color histogram and a second color histogram which are respectively used as the texture characteristics of the aviation virtual plane primitive and the texture characteristics of the ground virtual plane primitive; calculating a first normal vector of the aviation virtual plane primitive as a geometric feature of the aviation virtual plane primitive; And calculating a second normal vector of the ground virtual plane primitive as a geometric feature of the ground virtual plane primitive.
6. The method of claim 1, wherein the screening out a predetermined number of planar primitive pairs based on the texture features and the geometric features comprises: Acquiring an initial plane primitive pair formed by all aviation virtual plane primitives and all ground virtual plane primitives; calculating the texture feature similarity and the geometric feature similarity of each initial plane primitive pair based on the texture features and the geometric features; calculating to obtain total similarity according to the texture feature similarity and the geometric feature similarity; And screening out all target plane primitive pairs with the total similarity larger than a preset similarity threshold.
7. The air-to-ground image alignment method of claim 1, further comprising: according to a preset weight setting rule, giving weight to each aviation virtual plane element and each ground virtual plane element; the weight setting rule is as follows: calculating the root mean square of the reprojection error of each aviation virtual plane primitive and each ground virtual plane primitive; And determining the weight based on a preset control parameter and the root mean square of the re-projection error, so that the weight is larger when the root mean square of the re-projection error is smaller, and the weight is smaller when the root mean square of the re-projection error is larger.
8. An air-ground image alignment apparatus, comprising: The aerial triangulation module is used for carrying out aerial triangulation on the aerial image set and the ground image set containing the GPS position information to obtain an aerial air three result and a ground air three result; the key image extraction module is used for respectively extracting key images from the aviation image set and the ground image set based on the aviation air three results and the ground air three results to obtain an aviation key image set and a ground key image set; The relative depth map extraction module is used for inputting each aviation key image and each ground key image into the depth learning model, outputting an aviation relative depth map and a ground relative depth map, converting the aviation relative depth map into an aviation absolute depth map, and converting the ground relative depth map into a ground absolute depth map; the virtual plane primitive generation module is used for generating aviation dense point cloud and ground dense point cloud based on the aviation absolute depth map and the ground absolute depth map, and extracting aviation virtual plane primitives and ground virtual plane primitives from the aviation dense point cloud and the ground dense point cloud; the plane primitive pair construction module is used for extracting texture features and geometric features of the aviation virtual plane primitive and the ground virtual plane primitive, and screening out preset number of plane primitive pairs based on the texture features and the geometric features; And the air-ground image alignment module is used for calculating conversion parameters based on the plane primitive pairs, and converting the aviation point cloud into a ground coordinate system according to the conversion parameters so as to realize the alignment of the aviation image and the ground image.
9. An electronic device, comprising: A memory for storing a computer program; a processor for executing the computer program; wherein the processor, when executing the computer program, implements the space-to-ground image alignment method according to any one of claims 1 to 7.
10. A computer readable storage medium, wherein the computer readable storage medium stores a computer program, which when executed implements the space-to-ground image alignment method according to any one of claims 1 to 7.

Description

Air-space image alignment method, device, equipment and storage medium Technical Field The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for air-to-ground image alignment. Background Along with the fact that the live-action three-dimensional construction enters a new high-quality development stage, the fine modeling becomes a core requirement in the fields of urban planning, disaster monitoring, digital twinning and the like, the aviation image and ground image collaborative modeling can achieve the double advantages of large-scale coverage and fine detail, and the method becomes a main technical path of the current live-action three-dimensional fine modeling. However, in the ground image acquisition link, the satellite positioning signal is limited to a complex scene environment, satellite positioning signals are easy to be blocked, so that the accuracy of external azimuth elements of the ground image is insufficient, and meanwhile, multipath effects further interfere with positioning results, so that point clouds generated by modeling of the ground image, a model and coordinate system references of aviation image modeling achievements have deviations, the point clouds and the model cannot be naturally aligned, the overall accuracy of real-scene three-dimensional modeling is severely restricted, and therefore, the high-accuracy alignment of the ground and the aviation images becomes a necessary premise for breaking through the bottleneck of fine modeling. The existing alignment method still has a certain limitation, and cannot meet the requirements of real-scene three-dimensional high-quality and fine modeling in terms of automation degree, robustness, precision and scene adaptability, so that an alignment technology capable of automatically generating high-reliability registration primitives and adapting to complex scenes is needed. Disclosure of Invention The embodiment of the invention provides a space-to-ground image alignment method, which can obviously improve the precision and the overall efficiency of space-to-ground image alignment. In a first aspect, an embodiment of the present invention provides a method for aligning air-ground images, including: performing aerial triangulation on the aerial image set and the ground image set containing GPS position information to obtain aerial three results and ground three results; Extracting key images from the aviation image set and the ground image set based on the aviation air result and the ground air result respectively to obtain an aviation key image set and a ground key image set; Inputting each aviation key image and each ground key image into a depth learning model, outputting an aviation relative depth map and a ground relative depth map, converting the aviation relative depth map into an aviation absolute depth map, and converting the ground relative depth map into a ground absolute depth map; generating an aviation dense point cloud and a ground dense point cloud based on the aviation absolute depth map and the ground absolute depth map, and extracting aviation virtual plane primitives and ground virtual plane primitives from the aviation dense point cloud and the ground dense point cloud; extracting texture features and geometric features of the aviation virtual plane primitives and the ground virtual plane primitives, and screening out preset number of plane primitive pairs based on the texture features and the geometric features; And calculating conversion parameters based on the plane primitive pairs, and converting the aviation point cloud into a ground coordinate system according to the conversion parameters so as to realize the alignment of the aviation image and the ground image. Further, the extracting key images from the aerial image set and the ground image set based on the aerial air three results and the ground air three results to obtain an aerial key image set and a ground key image set, respectively, includes: Acquiring aviation sparse point clouds and ground sparse point clouds from the aviation air three results and the ground air three results, and projecting the aviation sparse point clouds and the ground sparse point clouds to an xy plane to obtain projected aviation point sets and ground point sets; calculating an overlapping area of the aviation point set and the ground point set, covering a buffer area with a preset range in the overlapping area to obtain a buffer area, and uniformly sampling in the buffer area to obtain a sampling point set; and calculating the proportion of each aerial image to the visible points in the sampling point set, and reserving the images with the proportion larger than a preset proportion threshold as key images to obtain an aerial key image set and a ground key image set. Further, the converting the aviation relative depth map into an aviation absolute depth map includes: acquiring aviation