Search

EP-4737854-A1 - METHODS, SYSTEMS, AND COMPUTER PROGRAM PRODUCTS FOR GEOREFERENCING

EP4737854A1EP 4737854 A1EP4737854 A1EP 4737854A1EP-4737854-A1

Abstract

A computer-implemented georeferencing method, the method comprising: receiving sensor data representing a portion of the surface of the earth, wherein the sensor data is obtainable or obtained by a sensor included in or attached to a vehicle, preferably an aerial vehicle; segmenting the sensor data to generate a sensor data segmentation map comprising one or more segments and a set of labels, wherein each of the labels relates to a segment and is indicative of a type of the segment; determining a labelled reference segmentation map obtainable by segmenting georeferenced reference data; and comparing the sensor data segmentation map to the reference segmentation map to determine a georeference of the sensor data segmentation map.

Inventors

  • WENZEL, PATRICK
  • WHITE, Nick

Assignees

  • Helsing GmbH

Dates

Publication Date
20260506
Application Date
20241031

Claims (15)

  1. A computer-implemented georeferencing method, the method comprising: receiving (102) sensor data (300) representing a portion of the surface of the earth, wherein the sensor data (300) is obtainable or obtained by a sensor (510) included in or attached to a vehicle (404, 502, 602), preferably an aerial vehicle (404, 502); segmenting (106) the sensor data (300) to generate a sensor data segmentation map (302) comprising one or more segments and a set of labels, wherein each of the labels relates to a segment (308) and is indicative of a type of the segment (308); determining (118) a labelled reference segmentation map (314) obtainable by segmenting georeferenced reference data (310); and comparing (126) the sensor data segmentation map (302) to the reference segmentation map (314) to determine a georeference (322) of the sensor data segmentation map (302).
  2. The method of claim 1, wherein the type of the segment (308) is indicative of whether the segment (308) is indicative of an area that comprises man-made objects and/or vegetation.
  3. The method of any of the preceding claims, wherein the type of the segment (308) is indicative of a roughness of an area related to the segment (308).
  4. The method of any of the preceding claims, wherein the method further comprises determining (130) a pose of the vehicle (404, 502, 602) based on the comparison, in particular based on the determined georeference (322) of the sensor data segmentation map (302).
  5. The method of any of the preceding claims, wherein the sensor (510) comprises a range-finding system, preferably a sweeping range-finding system, more preferably a LIDAR sensor.
  6. The method of any of the preceding claims, wherein the sensor data (300) comprises one or more measurements by the sensor (510), wherein each measurement is indicative of a position of a location on the surface of the earth relative to the position of the vehicle (404, 502, 602), and wherein the method further comprises: determining (110) an absolute position for each respective location related to each measurement based on an estimated position of the vehicle (404, 502, 602), and segmenting the sensor data (300) based on the absolute position.
  7. The method of any of the preceding claims , further comprising: segmenting the sensor data (300) based on a or the absolute position of the portion, in particular a or the location, on the surface of the earth; wherein determining the absolute position comprises: receiving a first indication of an altitude of the vehicle (404, 502, 602) from a first altimeter (506, 606), preferably a barometer; and determining (112) an elevation of the portion or location based on the altitude of the vehicle (404, 502, 602) and, preferably, the sensor data.
  8. The method of claim 6 or claim 7, wherein segmenting the sensor data (300) comprises receiving a digital elevation map and/or a digital terrain map and determining the type of the segment (308) based on a corresponding elevation at a longitude and latitude based on coordinates of the location.
  9. The method of any of the preceding claims, wherein segmenting the sensor data (300) comprises processing (116) the measurements by a machine learning algorithm, preferably a deep neural network.
  10. The method of any of the preceding claims, further comprising: scaling (124) the labelled reference segmentation map (314) to a horizontal scale of the sensor data segmentation map (302); and/or scaling (124) the sensor data segmentation map (302) to a horizontal scale of the labelled reference segmentation map (314).
  11. The method of claim 10, wherein the scaling is executed prior to comparing the sensor data segmentation map (302) to the reference segmentation map (314), and wherein the scaling is based on a second indication of the altitude of the vehicle (404, 502, 602), preferably an altitude measurement by a second altimeter (508) operable to measure a distance of the vehicle (404, 502, 602) to the ground.
  12. The method of any of the preceding claims, wherein comparing the sensor data segmentation map (302) to the reference segmentation map (314) comprises adjusting (128) a relative translation and/or a rotation between the sensor data segmentation map (302) and the reference segmentation map (314).
  13. The method of any of the preceding claims, wherein the labelled reference segmentation map (314) is determined by: determining georeferenced reference data (310); and segmenting the reference data (310) to generate the reference segmentation map (314), wherein the reference segmentation map (314) comprises one or more reference segments (320), and a set of labels, wherein each of the labels relates to a type of a reference segment (308) and is indicative of a type of the segment (308).
  14. A system comprising one or more processors and one or more storage devices, wherein the system is configured to perform the computer-implemented method of any one of claims 1-13.
  15. A computer program product for loading into a memory of a computer, comprising instructions, that, when executed by a processor of the computer, cause the computer to execute a computer-implemented method of any of claims 1-13.

Description

Technical Field The present disclosure relates to systems, methods, and computer program products for Georeferencing. The disclosure is applicable in the field of computer vision, in particular visual self-positioning for vehicle navigation. Background A frequent problem in visual self-localization is relating a measured position of a location on the surface of the earth to a known georeference. This requires both reliable identification and precise determination of the location. A known solution includes capturing a picture of a portion of the surface of the earth by a camera and comparing the image to a georeferenced image. The reliability and accuracy of this method depend, however, on the quality of the taken image and on atmospheric conditions. Moreover, processing the images for comparison is computationally expensive. There is a need for systems and methods that overcome these shortcomings. Summary Disclosed and claimed herein are systems, methods, and devices for georeferencing. A first aspect of the present disclosure relates to a computer-implemented georeferencing method. The method comprises the following steps: receiving sensor data representing a portion of the surface of the earth, wherein the sensor data is obtainable or obtained by a sensor included in or attached to a vehicle, preferably an aerial vehicle;segmenting the sensor data to generate a sensor data segmentation map comprising one or more segments and a set of labels, wherein each of the labels relates to a segment and is indicative of a type of the segment;determining a labelled reference segmentation map obtainable by segmenting georeferenced reference data; andcomparing the sensor data segmentation map to the reference segmentation map to determine a georeference of the sensor data segmentation map. Comparing segmentation maps has the advantage that the results are more accurate, and the processing time is reduced, in particular in comparison to comparing the images directly or detecting and comparing key points. In particular, the results of the comparison are more robust with respect to changes in brightness and illumination. Each segmentation map may be seen a representation of the portion of the surface of the earth. Depending on the size of the segments, the segmentation map may introduce a simplification of the representation. In an embodiment, the type of the segment is indicative of whether the segment is indicative of an area that comprises man-made objects and/or vegetation. These two cases are distinguishable by distance measuring processes, such as the determination of a two- or three-dimensional surface by active or passive means, such as LIDAR and/or photogrammetry. They allow for an unambiguous characterization of the surface of the earth, and corresponding information is generally available from reference maps. Moreover, such segmentation maps change only very little, if at all, over the seasons since man-made objects, such as buildings, and vegetation do not appear or disappear in the course of a year. Changes that typically occur over the seasons, such as foliage and/or snow, have a less sizable effect on such segmentation maps much less than on photographs. In a further embodiment, the type of the segment is indicative of a roughness of an area related to the segment. A roughness of an area is determinable by a distance measuring process and accurately describes different kinds of surfaces. For example, a smooth surface, such as a street, the tarmac of an airport, or a water surface of a lake, can be distinguished from rough surface, such as a corn field. The reliability of the comparison is thereby improved. In another embodiment, the method further comprises determining a pose of the vehicle based on the comparison, in particular based on the determined georeference of the sensor data segmentation map. This allows application of the method for visual self-positioning. This is particularly advantageous, when navigating a terrestrial or aerial vehicle where the sensor data are typically recorded at low altitude. However, the georeferenced reference data may comprise satellite images recorded at a fixed and much higher altitude. The systematic differences between these images are better overcome by comparing the segmentation maps, rather than comparing the images. In yet another embodiment, the sensor comprises a range-finding system, preferably a sweeping range-finding system, more preferably a LIDAR sensor. Range-finding sensors allow determining a relative elevation profile. Active sensors, such as LIDAR or RADAR sensors, are preferred because they are operable day and night and in adverse weather conditions. Therefore, one LIDAR sensor mounted on a vehicle allows determination of the sensor data in a variety of conditions and for a variety of mission profiles. It is not necessary to install a plurality of different sensors on the vehicle that operate under different environmental conditions. However, passiv