US-12624944-B2 - Autonomous UAS data acquisition with context awareness

US12624944B2US 12624944 B2US12624944 B2US 12624944B2US-12624944-B2

Abstract

A system and methods are provided for generating photogrammetry data acquisition plans of target structures, including: establishing a location and orientation of the target structure; identifying on the target structure predetermined target objects designated for data acquisition; correlating shapes, locations, and orientations of the target objects with the shape, location, and orientation of the target structure; and applying the shapes, locations, and orientations of the target structure and of the target objects, to generate a data acquisition plan for drone photogrammetry of the target structure and of the target objects.

Inventors

Tomer Daniel
Amir Jacob SHENHAV
Doron FINE
Hadas BAR-DAVID

Assignees

VHIVE TECH LTD

Dates

Publication Date: 20260512
Application Date: 20220823
Priority Date: 20210825

Claims (20)

1 . A computing system comprising at least one processor and non-transient memory storage communicatively coupled to the at least one processor and comprising computer-readable instructions that when executed by the at least one processor cause the computing system to implement steps of, after launch of a drone, capturing images taken by the drone in the direction of a target structure and, by image processing of the captured images: (i) distinguishing the target structure from other structures in a field of view of the drone and identifying a shape of the target structure; (ii) establishing a location of the target structure and an orientation of the target structure, (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; (iii) identifying on the target structure predetermined target objects designated for data acquisition; (iv) correlating the target structure and the target objects in 3D space by correlating their respective shapes, locations, and orientations; and (v) generating a data acquisition plan for drone photogrammetry of the target structure, wherein the data acquisition plan includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure and target objects.
2 . The computing system according to claim 1 , further comprising identifying obstacles among the other structures, estimating locations and elevations of the obstacles, and planning a flight path to the target structure avoiding the identified obstacles.
3 . The computing system according to claim 1 , further comprising estimating a distance from the drone to the target structure and planning a flight path to the target structure according to the estimated distance.
4 . The computing system according to claim 1 , further comprising identifying one or more obstacles in the vicinity of the target structure that may obstruct flight of a drone performing photogrammetry, and applying shapes, locations, and orientations of the one or more obstacles as additional parameters to generate the data acquisition plan for drone photogrammetry.
5 . The computing system according to claim 1 , further comprising estimating a height of the target structure, and applying the height as an additional parameter for generating the data acquisition plan.
6 . The computing system according to claim 5 , wherein the height estimation is achieved via stereoscopic and/or triangulation methods.
7 . The computing system according to claim 6 , wherein said stereoscopic and/or triangulation methods comprise feature-based computer vision algorithms for image matching, registration, camera calibration, warping, fundamental matrix calculation, disparity calculation, via structure from motion or structured light, and laser pointing methods.
8 . The computing system according to claim 5 , wherein the height estimation is achieved by flying the drone to a height where the target structure is viewed exactly on the horizon line, and setting the height estimation to the height of the drone.
9 . The computing system according to claim 1 , wherein identifying the shape of the target structure comprises identifying the shape by a set of thresholding and morphological operations and/or a feature-based, object-detection neural network algorithm.
10 . The computing system according to claim 1 , wherein establishing the location of the target structure further comprises flying the drone above the target structure and aligning the drone such that when taking a picture of the target structure, the target structure is in the center of the picture, and setting the location to coordinates of the drone projected to the ground.
11 . A method of photogrammetry comprising, after launch of a drone, capturing images taken by the drone in the direction of a target structure and, by image processing of the captured images, performing steps of: (i) distinguishing the target structure from other structures in a field of view of the drone and identifying a shape of the target structure; (ii) establishing a location of the target structure and an orientation of the target structure, (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; (iii) identifying on the target structure predetermined target objects designated for data acquisition; (iv) correlating the target structure and the target objects in 3D space by correlating their respective shapes, locations, and orientations; and (v) generating a data acquisition plan for drone photogrammetry of the target structure, wherein the data acquisition plan includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure and target objects.
12 . The method of claim 11 , further comprising identifying obstacles among the other structures, estimating locations and elevations of the obstacles, and planning a flight path to the target structure avoiding the identified obstacles.
13 . The method of claim 11 , further comprising estimating a distance from the drone to the target structure and planning a flight path to the target structure according to the estimated distance.
14 . The method of claim 11 , further comprising identifying one or more obstacles in the vicinity of the target structure that may obstruct flight of a drone performing photogrammetry, and applying shapes, locations, and orientations of the one or more obstacles as additional parameters to generate the data acquisition plan for drone photogrammetry.
15 . The method of claim 11 , further comprising estimating a height of the target structure, and applying the height as an additional parameter for generating the data acquisition plan.
16 . The method of claim 15 , wherein the height estimation is achieved via stereoscopic and/or triangulation methods.
17 . The method of claim 16 , wherein said stereoscopic and/or triangulation methods comprise feature-based computer vision algorithms for image matching, registration, camera calibration, warping, fundamental matrix calculation, disparity calculation, via structure from motion or structured light, and laser pointing methods.
18 . The method of claim 15 , wherein the height estimation is achieved by flying the drone to a height where the target structure is viewed exactly on the horizon line, and setting the height estimation to the height of the drone.
19 . The method of claim 11 , wherein identifying the shape of the target structure comprises identifying the shape by a set of thresholding and morphological operations and/or a feature-based, object-detection neural network algorithm.
20 . The method of claim 11 , wherein establishing the location of the target structure further comprises flying the drone above the target structure and aligning the drone such that when taking a picture of the target structure, the target structure is in the center of the picture, and setting the location to coordinates of the drone projected to the ground.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is a national phase entry of International Patent Application No. PCT/IL2022/050919, titled, “AUTONOMOUS UAS DATA ACQUISITION WITH CONTEXT AWARENESS,” filed Aug. 23, 2022, which claims the benefit under 35 U.S.C. § 119 (b) to Italian Patent Application No. IT102021000022310A, filed Aug. 25, 2021, the entire contents of which are hereby incorporated by reference. FIELD OF THE INVENTION The invention generally relates to built environment data acquisition, particularly by unmanned aerial systems. BACKGROUND Photogrammetry has developed into an important field for supporting the construction and maintenance of large structures of the built environment. Applications range from photogrammetry of permanent structures such as buildings, high power transmission lines, and antenna towers, to temporary structures, such as cranes, and remote structures, such as oil platforms. Based on photogrammetry image sets, 3D representations of the structures may be generated, which may then be used for analyzing the structures and their associated accessories. Analyses of 3D representations generated by photogrammetry may then be used for developing plans for construction and for maintenance, as well as navigation plans for related drone inspections. Unmanned aerial systems (UAS's), also referred to herein as drones, have thus far proven to be effective at inspecting built assets, especially in the construction, electricity, telecommunications, and transportation industries. However, developing navigation plans for photogrammetry by drones remains a time-consuming process. Data acquisition plans are generally defined as a set of waypoints, i.e., 3D coordinates, that specify all the points in space and associated orientations from which a drone should take images of a target structure. Improved processes for developing such photogrammetry data acquisition plans, as disclosed hereinbelow, may reduce costs and improve results of photogrammetry of built assets. SUMMARY Embodiments of the present invention provide a system and methods for generating drone data acquisition plans for photogrammetry of built structures. Embodiments include a computing system having at least one processor and non-transient memory storage communicatively coupled to the at least one processor. When the instructions are executed by the at least one processor, the computing system implements steps of: 1) capturing images taken by a drone flying in the direction of a target structure (after launch of the drone); 2) image processing of the captured images, to distinguish the target structure from other structures in a field of view of the drone and identifying a shape of the target structure; 3) establishing a location of the target structure and an orientation of the target structure, (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; 4) identifying on the target structure predetermined target objects designated for data acquisition; 5) correlating the target structure and the target objects in 3D space by correlating their respective shapes, locations, and orientations; and 6) generating a data acquisition plan for drone photogrammetry of the target structure, wherein the data acquisition plan includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure and target objects. Some embodiments may further include identifying obstacles among the other structures, estimating locations and elevations of the obstacles, and planning a flight path to the target structure avoiding the identified obstacles. The image processing may also include estimating a distance from the drone to the target structure and planning a flight path to the target structure according to the estimated distance. Some embodiments may also include a configuration for identifying one or more obstacles in the vicinity of the target structure that may obstruct flight of a drone performing photogrammetry, and applying shapes, locations, and orientations of the one or more obstacles as additional parameters to generate the data acquisition plan for drone photogrammetry. Some embodiments may further include estimating the height of the target structure, and applying the height as an additional parameter for generating the data acquisition plan. The height estimation may be performed by stereoscopic and/or triangulation methods. The stereoscopic and/or triangulation methods may include feature-based computer vision algorithms for image matching, registration, camera calibration, warping, fundamental matrix calculation, disparity calculation, v