Search

EP-4392931-B1 - AUTONOMOUS UAS DATA ACQUISITION WITH CONTEXT AWARENESS

EP4392931B1EP 4392931 B1EP4392931 B1EP 4392931B1EP-4392931-B1

Inventors

  • DANIEL, Tomer
  • SHENHAV, Amir Jacob
  • FINE, Doron
  • BAR-DAVID, Hadas

Dates

Publication Date
20260513
Application Date
20220823

Claims (11)

  1. A computing system (100) comprising at least one processor and non-transient memory storage communicatively coupled to the at least one processor and comprising computer-readable instructions that when executed by the at least one processor cause the computing system (100) to implement steps of, after launch of a drone (102), capturing images taken by the drone (102) in the direction of a target structure (110) and, by image processing of the captured images: i. distinguishing the target structure (110) from other structures in a field of view of the drone (102) and identifying a shape of the target structure (110); ii. establishing a location of the target structure (110) and an orientation of the target structure (110), (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure (110) is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure (110) is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; iii. identifying on the target structure (110) predetermined target objects (112) designated for data acquisition; iv. correlating the target structure (110) and the target objects (112) in 3D space by correlating their respective shapes, locations, and orientations; and v. generating a data acquisition plan (120) for drone photogrammetry of the target structure (110), wherein the data acquisition plan (120) includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure (110) and target objects (112).
  2. The computing system (100) according to claim 1, further comprising identifying obstacles (114) among the other structures, estimating locations and elevations of the obstacles (114), and planning a flight path to the target structure (110) avoiding the identified obstacles (114).
  3. The computing system (100) according to claims 1 or 2, further comprising estimating a distance from the drone (102) to the target structure (110) and planning a flight path to the target structure (110) according to the estimated distance.
  4. The computing system (100) according to any of claims 1-3, further comprising identifying one or more obstacles (116) in the vicinity of the target structure (110) that may obstruct flight of a drone (102) performing photogrammetry, and applying shapes, locations, and orientations of the one or more obstacles (116) as additional parameters to generate the data acquisition plan (120) for drone photogrammetry.
  5. The computing system (100) according to any of claims 1-4, further comprising estimating a height of the target structure (110), and applying the height as an additional parameter for generating the data acquisition plan (120).
  6. The computing system (100) according to claim 5, wherein the height estimation is achieved via stereoscopic and/or triangulation methods.
  7. The computing system (100) according to claim 6, wherein said stereoscopic and/or triangulation methods comprise feature-based computer vision algorithms for image matching, registration, camera calibration, warping, fundamental matrix calculation, disparity calculation, via structure from motion or structured light, and laser pointing methods.
  8. The computing system (100) according to claim 5, wherein the height estimation is achieved by flying the drone (102) to a height where the target structure (110) is viewed exactly on the horizon line, and setting the height estimation to the height of the drone (102).
  9. The computing system (100) according to any of claims 1-5, wherein identifying the shape of the target structure (110) comprises identifying the shape by a set of thresholding and morphological operations and/or a feature-based, object-detection neural network algorithm.
  10. The computing system (100) according to one of claims 1-5 or 9, wherein establishing the location of the target structure (110) further comprises flying the drone (102) above the target structure (110) and aligning the drone (102) such that when taking a picture of the target structure (110), the target structure (110) is in the center of the picture, and setting the location to coordinates of the drone (102) projected to the ground.
  11. A method of photogrammetry comprising, after launch of a drone (102), capturing images taken by the drone (102) in the direction of a target structure (110) and, by image processing of the captured images, performing steps of: i. distinguishing the target structure (110) from other structures in a field of view of the drone (102) and identifying a shape of the target structure (110); ii. establishing a location of the target structure (110) and an orientation of the target structure (110), (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure (110) is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure (110) is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; iii. identifying on the target structure (110) predetermined target objects (112) designated for data acquisition; iv. correlating the target structure (110) and the target objects (112) in 3D space by correlating their respective shapes, locations, and orientations; and v. generating a data acquisition plan (120) for drone photogrammetry of the target structure (110), wherein the data acquisition plan (120) includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure (110) and target objects (112).

Description

FIELD OF THE INVENTION The invention generally relates to built environment data acquisition, particular by unmanned aerial systems. BACKGROUND Photogrammetry has developed into an important field for supporting the construction and maintenance of large structures of the built environment. Applications range from photogrammetry of permanent structures such as buildings, high power transmission lines, and antenna towers, to temporary structures, such as cranes, and remote structures, such as oil platforms. Based on photogrammetry image sets, 3D representations of the structures may be generated, which may then be used for analyzing the structures and their associated accessories. Analyses of 3D representations generated by photogrammetry may then be used for developing plans for construction and for maintenance, as well as navigation plans for related drone inspections. Unmanned aerial systems (UAS's), also referred to herein as drones, have thus far proven to be effective at inspecting built assets, especially in the construction, electricity, telecommunications, and transportation industries. Data acquisition plans are generally defined as a set of waypoints, i.e., 3D coordinates, that specify all the points in space and associated orientations from which a drone should take images of a target structure. For example, US2018/0267561A1 describes methods for generating autonomous flight paths for unmanned aircraft based on target-specific variables, including the type of aircraft, capture routines, and desired data parameters. The system supports flight path control via a graphical user interface and enables automated definition of flight paths considering target structures' dimensions and features, which facilitates data collection around vertical structures. Additionally, CN112596071A describes a drone-based mapping system that leverages localization and mapping (SLAM) algorithms to enhance the accuracy of structural inspections. The system employs visual-inertial odometry (VIO) and integrates multi-sensor data to generate 3D maps of built structures while compensating for GPS-denied environments. Such approaches improve drone autonomy and inspection accuracy in built environments where obstacle avoidance and real-time path planning are needed. Despite these advances, developing navigation plans for photogrammetry by drones remains a time-consuming process. Improved processes for developing such photogrammetry data acquisition plans, as disclosed hereinbelow, may reduce costs and improve results of photogrammetry of built assets. SUMMARY OF THE INVENTION Embodiments of the present invention provide a system and methods for generating drone data acquisition plans for photogrammetry of built structures. Embodiments include a computing system having at least one processor and non-transient memory storage communicatively coupled to the at least one processor. When the instructions are executed by the at least one processor, the computing system implements steps of: 1) capturing images taken by a drone flying in the direction of a target structure (after launch of the drone); 2) image processing of the captured images, to distinguish the target structure from other structures in a field of view of the drone and identifying a shape of the target structure; 3) establishing a location of the target structure and an orientation of the target structure, (a) by identifying a center-of-tower location and by identifying guy tower anchors, when the target structure is tower-shaped, (b) by identifying a crane tip and a crane base, when the target structure is crane-shaped, and (c) by identifying coordinates of two or more corners of a predefined location of the structure, when the structure is not tower or crane-shaped; 4) identifying on the target structure predetermined target objects designated for data acquisition; 5) correlating the target structure and the target objects in 3D space by correlating their respective shapes, locations, and orientations; and 6) generating a data acquisition plan for drone photogrammetry of the target structure, wherein the data acquisition plan includes 3D coordinates of photogrammetry positions and orientations set at pre-defined distances and orientations from the correlated target structure and target objects. Some embodiments may further include identifying obstacles among the other structures, estimating locations and elevations of the obstacles, and planning a flight path to the target structure avoiding the identified obstacles. The image processing may also include estimating a distance from the drone to the target structure and planning a flight path to the target structure according to the estimated distance. Some embodiments may also include a configuration for identifying one or more obstacles in the vicinity of the target structure that may obstruct flight of a drone performing photogrammetry, and applying shapes, locations, and orientations of the one or more obstacles as