US-12626368-B2 - Image analysis for aerial images

US12626368B2US 12626368 B2US12626368 B2US 12626368B2US-12626368-B2

Abstract

Disclosed is a system comprising a data-processing system. The data-processing system comprises a data-storage component, a segmentation component and a projection component. The data-storage component is configured for providing an input orthophoto map of an area and an input digital elevation model of the area. The segmentation component is configured for performing a segmentation step, the segmentation step comprises generating at least one or a plurality of polygon(s) based on the input orthophoto map. Each polygon approximates a part of the input orthophoto map. The projection component is configured for performing a projection step and the projection step comprises projecting the polygon(s) on the input digital elevation model of the area. The projection component is further configured for performing a reference surface generation step, the reference surface generation step comprising generating a reference surface for each of at least some of the polygon(s). Further, a corresponding method and a corresponding computer-program product are disclosed.

Inventors

Aleksander BUCZKOWSKI
Michal MAZUR
Adam Wisniewski
Dariusz CIESLA

Assignees

AI CLEARING INC.

Dates

Publication Date: 20260512
Application Date: 20211116
Priority Date: 20201116

Claims (13)

1 . A system comprising a data-processing system, wherein the data-processing system comprises a data-storage component, wherein the data-storage component is configured for providing an input orthophoto map and an input digital elevation model of the area, wherein the data-processing system further comprises a segmentation component, wherein the segmentation component is configured for generating polygon(s) based on the input orthophoto map, each polygon approximating a part of the input orthophoto map, wherein the data-processing system comprises a projection component, wherein the projection component is configured for projecting the polygon(s) on the input digital elevation model of the area and for generating a reference surface for each of the at least some of the polygon(s), wherein the data-processing system comprises a pre-processing component, wherein the pre-processing component is configured for determining at least a component of a gradient of the input digital elevation model, and the segmentation component is configured for determining the parts of the input orthophoto map based at least on the input orthophoto map and the component(s) of the gradient of the input digital elevation model, and wherein the pre-processing component is configured for generating tiles of the input orthophoto map and the digital elevation model, and wherein the segmentation component is configured for processing at least some of the tiles individually.
2 . The system according to claim 1 , wherein the data-processing system comprises a volume determining component configured for determining a volume between a portion of the input digital elevation model and a portion of the reference surface for each reference surface.
3 . The system according to claim 2 , wherein the segmentation component and the projection component are configured for processing the first orthophoto map as input orthophoto map and the first digital elevation model as input digital elevation model, the segmentation component being configured for thus generating first polygon(s) and the projection component being configured for thus generating first reference surface(s); and the second orthophoto map as input orthophoto map and the second digital elevation model as input digital elevation model, the segmentation component being configured for thus generating second polygon(s) and the projection component being configured for thus generating second reference surface(s), wherein the volume determining component is configured for processing the first reference surface(s) and the first digital elevation model and thus generating first volume(s), and for processing the second reference surface(s) and the second digital elevation model and thus generating second volume(s), and wherein the volume determining component is configured for comparing at least some of the first and second volume(s).
4 . The system according to claim 3 , the volume determining component is configured for at least one of determining volume differences between at least some of the first and the second volume(s), and determining volumes that are present in only one of the first and the second volume(s).
5 . The system according to claim 1 , wherein the projection component is configured for processing elevation coordinates of the vertexes of the at least some polygon(s) projected to the input digital elevation model, wherein processing the elevation coordinates of the vertexes comprises generating a statistic measure of the elevation coordinates.
6 . The system according to claim 1 , wherein the segmentation component is configured for determining the parts of the orthophoto map by means of at least one convolutional neural network.
7 . The system according to claim 6 , wherein the segmentation component is configured for assigning different classes to different portions of the orthophoto map and for assigning portions comprising same classes to groups, wherein the data-processing system comprises a post-processing component, and wherein the post-processing component is configured for applying a conditional random fields algorithm to borders of the groups.
8 . The system according to claim 1 , wherein the data-storage component is further configured for providing design data, wherein the data-processing system further comprises an area-comparison component, wherein the area-comparison component is configured for at least one of comparing the polygon(s) and the design data, and generating reporting units based on the design data, wherein generating the reporting units comprises dividing at least one object represented by the design data into a plurality of reporting units spatially different from each other.
9 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of the data-processing system of claim 1 , cause the one or more processors to perform the steps for which the data-processing system is configured.
10 . A method, comprising providing an input orthophoto map of an area, providing an input digital elevation model of the area, performing a segmentation step, wherein the segmentation step comprises generating at least one or a plurality of polygon(s) based on the input orthophoto map, each polygon approximating a part of the input orthophoto map, performing a projection step, the projection step comprising projecting the polygon(s) on the input digital elevation model of the area, and a reference surface generation step, the reference surface generation step comprising generating a reference surface for each of at least some of the polygon(s), wherein the segmentation step comprises generating the polygon(s) based on the input orthophoto map and the input digital elevation model, wherein the semantic segmentation step comprises a pre-processing step, the pre-processing step comprising determining at least a component of a gradient of the input digital elevation model, and wherein the segmentation step comprises determining the parts of the input orthophoto map by means of at least one convolutional neural network based at least on the input orthophoto map and the component(s) of the gradient of the input digital elevation model.
11 . The method according to claim 10 , wherein the method further comprises a volume determining step, the volume determining step comprising for each reference surface determining a volume between a portion of the input digital elevation model and a portion of the reference surface, wherein the segmentation step comprises determining the parts of the orthophoto map by means of at least one convolutional neural network.
12 . The method according to claim 10 , wherein the pre-processing step comprises generating tiles of the input orthophoto map and the digital elevation model, wherein the segmentation step comprises assigning different classes to different portions of the orthophoto map by the at least one convolutional neural network, and wherein the method comprises processing at least some tiles individually by means of the at least one convolutional neural network, wherein the segmentation step comprises a post-processing step, wherein the post-processing step comprises applying a conditional random fields algorithm to borders of the groups.
13 . The method according to claim 10 , wherein the method comprises a data comparison step, wherein the data comparison step comprises at least one of comparing the polygon(s) and the design data, and generating reporting units based on the design data, wherein generating the reporting units comprises dividing at least one object represented by the design data into a plurality of reporting units spatially different from each other.

Description

RELATED APPLICATIONS The present application is a U.S. National Stage application under 35 USC 371 of PCT Application Serial No. PCT/US2021/059493, filed on 16 Nov. 2021, which claims priority to EP Patent Application Serial No. 20207918.2, filed on 16 Nov. 2020, and EP Patent Application Serial No. 20207919.0, filed on 16 Nov. 2020, the entirety of each of which is incorporated herein by reference. The present invention relates to the field of image analysis and particularly to the field of analysis of aerial images. The present invention further relates to determining types of surfaces, volumes and volume changes. The concept of analysing areas by means of aerial images is generally known. It is also applied to imagery generated by unmanned aerial vehicles. Aerial images can for example be used for analysing construction sites, e.g. the progress of the work can be monitored. Classically, the progress of construction sites as well as an adherence to plans, e.g. in terms of precise positions of structures etc, is monitored by land surveyors. Depending on the size of the construction site, the monitoring can only be performed at crucial points or at random, already due to the distances to cover, e.g. in case of highway construction sites. In some cases, not the whole site can be analysed. Further, errors during the process of surveying cannot be completely excluded. Since survey results on construction sites are inter alia used as condition for authorizing payments, there may be a need for revisable and fast survey. For large construction sites, surveyors usually inspect only small sample of data, extrapolating findings for the whole site. This process may be prone to errors—both human errors (like selecting bad sample data, which is convenient to measure, but not representative) and errors due to extrapolation. The analysis of the sites may be performed by cameras mounted to aerial vehicles, such as airplanes or drones. However, in this case, the resulting images need to be processed correspondingly. The further processing can for example be performed with computer-support. U.S. Pat. No. 10,339,663 B2 discloses systems and methods for generating georeference information with respect to aerial images. In particular, in one or more embodiments, systems and methods generate georeference information relating to aerial images captured without ground control points based on existing aerial images. For example, systems and methods can access a new set of aerial images without ground control points and utilize existing aerial images containing ground control points to generate a georeferenced representation corresponding to the features of the new set of aerial images. Similarly, systems and methods can access a new image without ground control points and utilize an existing georeferenced orthomap to produce a georeferenced orthomap corresponding to the features of the new image. One or more embodiments of the disclosed systems and methods permit users to obtain georeference information related to new images without the need to place ground control points or collect additional georeference information. U.S. Pat. No. 10,593,108 B2 discloses systems and methods for more efficiently and quickly utilizing digital aerial images to generate models of a site. In particular, in one or more embodiments, the disclosed systems and methods capture a plurality of digital aerial images of a site. Moreover, the disclosed systems and methods can cluster the plurality of digital aerial images based on a variety of factors, such as visual contents, capture position, or capture time of the digital aerial images. Moreover, the disclosed systems and methods can analyze the clusters independently (i.e., in parallel) to generate cluster models. Further, the disclosed systems and methods can merge the cluster models to generate a model of the site. U.S. Pat. No. 9,389,084 B2 is directed toward systems and methods for identifying changes to a target site based on aerial images of the target site. For example, systems and methods described herein generate representations of the target site based on aerial photographs provided by an unmanned aerial vehicle. In one or more embodiments, systems and method described herein identify differences between the generated representations in order to detect changes that have occurred at the target site. While the prior art approaches may be satisfactory in some regards, they have certain shortcomings and disadvantages. For example, objects must still be identified, e.g. on orthophoto maps or orthomosaics generated based on the aerial images. After objects are detected, analysis as regards to position, surface, volume or the like is performed based on the orthophoto map and/or digital surface models (DSMs) or digital terrain models (DTMs). It is therefore an object of the invention to overcome or at least alleviate the shortcomings and disadvantages of the prior art. More particularly, it is an object o