CN-116030194-B - Air-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance

CN116030194BCN 116030194 BCN116030194 BCN 116030194BCN-116030194-B

Abstract

The invention discloses an air-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance, which is used for realizing unified air-space three-calculation and live-action three-dimensional modeling by setting a unified target shooting scene, determining a target shooting area, collecting ground shooting visual angle images, collecting sky visual angle images, extracting and correcting EXIF information and the like in a mode of simultaneous collaborative investigation of an unmanned aerial vehicle and a mobile phone in the early investigation of live-action three-dimensional modeling, and assisting a ground acquisition photographer in avoiding invalid ground movable targets by utilizing a target detection algorithm, simplifying the redundant quantity of ground acquisition shooting sampling points, and simultaneously reducing the workload of the early field investigation and the calculation amount of later air-space three-calculation. The method can cooperate with the photo data of the unmanned aerial vehicle and the ground mobile phone to the unified air three modeling process under the conditions of low cost and high efficiency acquisition, and realize high-precision and multi-detail real-scene three-dimensional rapid modeling under the conditions of limited cost and limited sampling.

Inventors

YANG YING
XU FENG
YANG YUCHENG
XIN YI
LIU YUBO
NIU QIXIANG
Jin Yeningyan
LIU SIDI
HU YUEMING
Guo Taoyang
MAO LEI
LI CHAO
WEN HONGSEN
LIU ZHAO
Tang Shuoning

Assignees

中国建筑第五工程局有限公司

Dates

Publication Date: 20260505
Application Date: 20230206

Claims (10)

1. The air-ground cooperative live-action three-dimensional modeling optimization method based on target detection avoidance is characterized by comprising the following steps of: step S1, setting a target shooting scene, namely determining a shooting acquisition window, wherein the shooting acquisition window is a project requirement condition which needs to be met jointly by air and ground acquisition shooting; step S2, determining a target area to be shot, and dividing a shooting acquisition task into two parts, namely a ground shooting visual angle work in step S3 and an air shooting visual angle work in step S4 based on the target area; Setting a set of shooting rules for efficiently acquiring building environment information of the ground shooting view angle as targets, aiming at complementing dense obstacle areas and/or aerial shooting dead angles which are difficult to shoot in the air on the premise of safe operation, detecting movable targets which cause ground interference as much as possible through image processing software in portable mobile shooting equipment, judging the number of the movable targets in a shot image, screening out photos meeting the requirement of shooting basic conditions, and then executing step S5; step S4, working at an aerial shooting view angle, namely setting a set of shooting rules, taking environment information of the aerial shooting view angle as a target, and aiming at complementing the aerial view angle which is difficult to shoot on the ground on the premise of safe operation so as to complement aerial ground data required by air-ground collaborative air three calculation, and then executing step S6; s5, outputting and storing all the ground shooting data with the position information in the step S3; s6, outputting and storing all aerial shooting data with position information in the step S4; S7, integrating the shooting data of the ground view angle in the step S5 and the air view angle in the step S6, and integrating the acquired photos of the air and ground shooting view angles into an image group capable of simultaneously meeting the same air-space three operation by extracting EXIF information, POS information and correction POS information, so as to provide a unified data base for the air-ground collaborative air-space three operation; step S8, uniformly inputting the image groups of the air and the ground subjected to correction and unification in the step S7 into air triangular calculation; and S9, generating a real three-dimensional model of the building.
2. The air-ground collaborative live-action three-dimensional modeling optimization method based on object detection avoidance as claimed in claim 1, wherein step S3 further comprises: step S31, determining a ground basic shooting view angle, namely determining a main shooting object of the ground view angle to be shot and an interference object not to be shot and acquired, and determining a shooting view angle, a shooting height and a shooting focal length; Step S32, the portable mobile shooting equipment identifies movable targets in a scene through a target detection method, wherein the portable mobile shooting equipment invokes a trained model and a weight file based on a YOLO target detection algorithm to construct a target detection function, and the movable targets in the original scene targets are detected according to types by calculating a detection trunk of deduced sampling characteristics of YOLO, a hierarchical neck for widening a receptive field and a prediction head for multi-scale output; Step S33, identifying a movable target through the YOLO target detection algorithm, marking various movable target types, calculating position range parameters (x, y, w, h) and center point parameters (x+w/2, y+h/2) of the movable target, wherein (x, y, w, h) represents position and size parameters of an identification frame, (x, y) represents coordinate offset of an upper left corner point of the identification frame relative to an upper left corner point (0, 0) of a shot picture, x represents right offset, y represents downward offset, and w and h represent width and height of the identification frame, w represents a starting point right offset width measure with the upper left corner point (x, y) of the identification frame as a starting point, and h represents a starting point downward offset Gao Duliang; Step S34, drawing a target center point based on movable target center point parameters (x+w/2, y+h/2), and drawing a target identification frame through superposition of the center points, wherein the identification frame is jointly determined by upper left corner coordinate parameters (x, y) and lower right corner coordinate parameters (x+w, y+h), and the identification frame is used as a basis for assisting a photographer in avoiding movable targets in an environmental scene and judging shooting quality of acquired photos; Step S35, through the overlapped identification frames as a judging tool, a photographer of the portable mobile shooting equipment can quickly identify and understand an environment scene, and assist the photographer in judging whether the number of the interfered movable targets in the scene can meet the shooting condition requirement of step S31, if so, executing the next step S36, otherwise, indicating that the shooting condition requirement of step S31 cannot be met, and returning to step S31 to replace the shooting scene and/or the view angle; step S36, when the ground shooting meets the shooting condition requirement, shooting an image through the portable mobile shooting equipment; step S37, the photographer judges whether the shooting collection of the ground is finished completely, if yes, the step S5 is skipped, if not, after the shooting scene and/or the view angle are finished, the step S31 is returned to replace the view angle of the next scene.
3. The method for three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance according to claim 2, wherein the building is a dense building, the portable mobile shooting device is a mobile phone, an APP in the mobile phone is used for executing the step S31-step S37, and the YOLO object detection algorithm is replaced by an object detection algorithm with other data lightweight.
4. The air-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance according to claim 3, wherein APP in the mobile phone is crowdsourcing software.
5. The method for three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance according to claim 4, wherein step S33 further comprises step S331 of directly outputting the captured image as captured image sampling data without processing a static environment scene in which no movable object is detected, and taking the original image data as a data source of effective ground environment information in the air-three solution.
6. The method of three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance according to claim 5, wherein the subject photographed object in step S31 includes a building, a tablet, a shop sign and/or an eave, and the interfering object that does not require photographing acquisition includes a movable pedestrian, an automobile and/or an umbrella.
7. The three-dimensional modeling optimization method for the air-ground collaborative live-action based on the object detection avoidance as claimed in claim 1, wherein the step S1 includes that the shooting acquisition window includes a shooting time interval, a weather interval and a space interval: Determining the cooperative working time condition of the air-ground coordination, and ensuring that the sunlight angle shooting condition is controlled; Determining a cooperative working climate condition of the air-ground coordination, and ensuring that shooting illumination intensity and visibility shooting conditions are controlled; And determining the space interval, namely determining the condition of the space-ground cooperative joint working space range, and guaranteeing that the shooting condition is controlled by longitude, latitude and height limitation.
8. The method for three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance as claimed in claim 1, wherein the step S4 further comprises: Step 41, determining an aerial ground basic shooting view angle, namely determining a main shooting object of the ground view angle to be shot, and simultaneously determining a shooting lens holder view angle, a shooting height and a shooting focal length; Step S42, planning a flight route of the unmanned aerial vehicle in the air, and planning a shooting angle of the flight route and a cradle head of the unmanned aerial vehicle in the air according to a five-way flight, a cross flight and/or a surrounding flight mode by combining project shooting requirements; s43, shooting an unmanned aerial vehicle when a target waypoint is reached and the aerial shooting requirement is met; And step S44, judging whether the aerial shooting acquisition is finished completely according to a preset route, if so, jumping to step S6, and if not, indicating that the aerial shooting is not finished completely, and jumping to step S42 and continuing to fly to the next waypoint for shooting after the local waypoint shooting is finished.
9. The method for three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance as claimed in claim 1, wherein the step S7 further comprises: Step S71, extracting EXIF information of a ground photo, wherein the EXIF information comprises position information POS, a shooting lens model and lens focal length camera parameters; and S72, extracting the POS position information of the ground photo from the EXIF information, reading the longitude, latitude and height of the POS information of the ground photo, and correcting according to project requirements so as to ensure that the POS position information of the ground and the air are positioned under the same set of coordinate system and the same set of height reference.
10. The method for three-dimensional modeling optimization of air-ground collaborative live-action based on object detection avoidance according to claim 9, wherein the step S7 further comprises: Step S711, extracting EXIF information of aerial photos, wherein the EXIF information comprises position information POS, a holder shooting angle, a shooting lens model and lens focal length camera parameters; And S721, extracting POS position information of the aerial photo from the EXIF information, reading longitude, latitude and height of the POS information of the aerial ground photo, and correcting according to project requirements to ensure that the POS position information of the aerial and ground are positioned under the same set of coordinate system and the same set of height reference.

Description

Air-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance Technical Field The invention belongs to the field of urban and rural building three-dimensional modeling, and particularly relates to a space-ground collaborative live-action three-dimensional modeling optimization method based on target detection and avoidance. Background Along with the acceleration of the urban process, digital smart city technology is rapidly developed, and a general planar map cannot meet related requirements of various industries, so that compared with a traditional two-dimensional map, the three-dimensional data map has the advantages of reality and high precision. At present, aiming at special overhanging building forms, such as overhanging, eave, building of building forms, such as village in city, old district and other building dense urban areas, the traditional unmanned aerial vehicle flight shooting has flight safety limitation, unmanned aerial vehicle acquisition flight cost is high, even has safety risks such as a flight collision machine, and the problems of fuzzy space details, rough precision, model broken surfaces and the like of a three-dimensional model eave, overhanging and the like can be caused. Because the general urban architecture is dense and the interference targets are more, the ground shooting has many limitations such as limited ground path sampling points, interference of movable targets and the like, and the available shooting travel route of the ground is limited on the sampling points, the shooting sampling points cannot fully cover the sampling area, and certain dense redundant sampling is required on the limited sampling points to ensure the data quality, so that the problems of increased operation amount and the like are also brought. In the prior art, although a technology for three-dimensional modeling of a building by using a space-to-ground measurement mode is also available, the technical effect obtained by the technology is often improved only to a limited extent. Because the general urban architecture is dense and the interference targets are more, the ground shooting has many limitations such as limited ground path sampling points, interference of movable targets and the like, and the available shooting travel route of the ground is limited on the sampling points, the shooting sampling points cannot fully cover the sampling area, and certain dense redundant sampling is required on the limited sampling points to ensure the data quality, so that the problems of increased operation amount and the like are also brought. In the existing space-to-ground measurement technology, the used camera is always fixed around a building by 360 degrees, other moving interference targets do not exist around the building, the high-cost ground image acquisition cannot be carried out on urban buildings with more people, other dynamic interference targets except the building are easy to exist in photos of the urban buildings shot on the ground, and the image data acquired in the space cannot be directly fused, so that the calculated amount of data and a model in the fusion of the space-to-ground image data is large, the precision and details of three-dimensional modeling of the urban buildings are low, key detail loss is directly caused to important person viewing angles and key positions of the buildings, and the research and judgment of a designer on the problem of urban updating are very influenced. For example, chinese patent CN202111288002.1 discloses a method for locally and quickly updating based on live-action fine modeling of mobile phone images, but the method has a certain limitation, and the technical process is that the unmanned aerial vehicle air three results and the air three results shot by the mobile phone are fused in the later stage, which belongs to a three-dimensional model updating method for post-supplementary modification, and although the model improvement is realized, a great amount of manual calculation and alignment modification are still needed, the problem of lower manual repair efficiency is existed, the same image feature points from different unmanned aerial vehicles and mobile phone devices cannot participate in the same group of air three calculation, the images acquired by the mobile phone are not reasonably screened for redundant targets in urban building shooting, and the measurement errors and the air three calculation amount are increased. Based on the method, an air-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance needs to be designed. Disclosure of Invention First technical problem Based on the technical problems, the invention provides the space-ground collaborative live-action three-dimensional modeling optimization method based on target detection avoidance, which can cooperate with effective photo data shot by mobile equipment such as an