Search

EP-4737853-A1 - 3D DATA STRUCTURE

EP4737853A1EP 4737853 A1EP4737853 A1EP 4737853A1EP-4737853-A1

Abstract

A computer implemented method of populating a data structure suitable for conducting a survey in an environment, comprising: determining that a spatial unit of a set of spatial units is associated with a semantic classification label representative of an inspection asset; receiving inspection criteria associated with the inspection asset, wherein the inspection criteria comprises a threshold condition associated with an inspection sensor; receiving inspection sensor data, from the inspection sensor, representing the inspection asset; applying the threshold condition to the inspection sensor data to generate inspection data; and modifying an inspection data content of the spatial unit based on the inspection data.

Inventors

  • Carrasco, Pep Lluis Negre
  • MAYNARD, Magnus

Assignees

  • Rovco Limited

Dates

Publication Date
20260506
Application Date
20240617

Claims (15)

  1. A computer implemented method of populating a data structure suitable for conducting a survey in an environment, the method comprising: determining that a spatial unit of a set of spatial units is associated with a semantic classification label representative of an inspection asset; receiving inspection criteria associated with the inspection asset, wherein the inspection criteria comprises a threshold condition associated with an inspection sensor; receiving inspection sensor data, from the inspection sensor, representing the inspection asset; applying the threshold condition to the inspection sensor data to generate inspection data; and, modifying an inspection data content of the spatial unit based on the inspection data.
  2. The method of claim 1, wherein the set of spatial units is each associated with a modifiable content, the modifiable content comprising inspection field comprising the inspection data.
  3. The method of any of claims 1 or 2, wherein the inspection further comprises a second threshold condition associated with the inspection sensor; the method further comprising: applying the second threshold condition to the inspection sensor data to generate second inspection data; and, modifying the inspection data content of the spatial unit based on the second inspection data.
  4. The method of any preceding claim, the method further comprising: receiving a second inspection criteria comprising a third threshold condition associated with a second inspection sensor; receiving second inspection sensor data, from the second inspection sensor, representing the inspection asset; applying the third threshold condition to the inspection sensor data to generate second inspection data; and, modifying the inspection data content of the spatial unit based on the second inspection data.
  5. The method of any preceding claim, wherein an inspection confidence value is associated with the inspection data, wherein the method only receives the second inspection sensor data if the inspection confidence value is greater than an inspection confidence threshold.
  6. The method of any preceding claim, the method further comprising: determining that a second spatial unit of the set of spatial units is associated with a second semantic classification label representative of a second inspection asset; receiving a third inspection criteria associated with the second inspection asset and the inspection sensor, wherein the third inspection criteria comprises a fourth threshold condition; receiving third inspection sensor data, from the inspection sensor, representing the second inspection asset; applying the fourth threshold condition to the third inspection sensor data to generate third inspection data; and, modifying an inspection field of the second spatial unit based on the third inspection data.
  7. The method of any preceding claim, wherein the threshold condition is one of: a defined blurriness factor of captured inspection sensor data from the inspection sensor; a defined number of captured inspection sensor data from the inspection sensor; a defined maximum range between the inspection asset and the inspection sensor; a defined minimum range between the inspection asset and the inspection sensor; a defined total number of received depth points per unit volume, wherein the inspection sensor is a sonar sensor; a defined measurement value of the inspection sensor; a binary determination that the inspection sensor is in contact with the inspection asset; or, a binary determination that a tool is in contact with the inspection asset.
  8. The method of any of claims 2 to 7, wherein the set of spatial units is further associated with an attribute defining an exclusive spatial volume in a 3D survey environment, wherein the modifiable content further comprises a measurement field comprising occupancy data associated with the attribute, and optionally wherein the inspection sensor is the occupancy sensor.
  9. The method of claim 8, the method further comprising: receiving occupancy sensor data, from an occupancy sensor, representing the survey environment; and, modifying the content of the measurement field of the spatial unit based on the occupancy sensor data, and optionally wherein the content of each measurement field of the set of spatial units is modified simultaneously based on the occupancy sensor data.
  10. The method of any of claims 8 to 9, wherein the occupancy sensor data represents two-dimensional image data, the method further comprising: applying a 2D to 3D transformation to the occupancy sensor data to generate 3D occupancy data; and, modifying the content of the measurement field of the spatial unit based on the 3D occupancy data.
  11. The method of any of claims 2 to 10, wherein the modifiable content further comprises a semantic label field comprising a semantic classification label, and optionally wherein the inspection sensor is the classification sensor.
  12. The method of claim 11, the method further comprising: determining a semantic classification label of the inspection asset based on classification sensor data, from a classification sensor; and, modifying the content of the semantic label field of the spatial unit based on the semantic classification label, and optionally, the method further comprising: determining that the semantic label field of the spatial unit is occupied by a previous semantic classification label; and, performing probabilistic based data-fusion to determine the content of the semantic label field based on the previous semantic classification label and the semantic classification label.
  13. The method of any of claims 11 to 12, wherein a semantic inference step determines the semantic classification label based on the occupancy sensor data, and optionally the semantic inference step also determines a confidence value associated with the semantic classification label.
  14. The method of any of claims 11 to 13, the method further comprising: wherein the classification sensor data represents two-dimensional image data; performing semantic segmentation on the classification sensor data to generate a 2D segmentation map; determining that the 2D segmentation map contains the inspection asset; applying a 2D to 3D transformation to the 2D segmentation map to generate a 3D segmentation map comprising one or more semantic classification labels; and, modifying the content of the semantic label field of the spatial unit based on the 3D segmentation map.
  15. A computing system for performing the method of claims 1 to 14, the computing system comprising: an occupancy sensor; an inspection sensor; a data processor configured to: perform an inspection planning algorithm to plan a path for a mobile vehicle to follow which fulfils an inspection criteria, wherein the path is planned via a 3D model of the environment comprising a set of spatial units; and, perform the method of any of claims 1 to 14.

Description

FIELD OF THE INVENTION The present invention is in the field of autonomous inspection algorithms. Specifically, in the field of 3D data structures for autonomous inspection algorithms. BACKGROUND OF THE INVENTION Semantic maps are a known 3D spatial data structure that divides a space into units and assigns one or more semantic classes to each unit. These data structures may be used for autonomous planning algorithms to guide the exploration and mapping of a survey environment. It is desirable for an autonomous vehicle (e.g., a robot) to not only perform mapping of a survey environment but also to perform an inspection of one or more assets without human intervention. Currently, inspection of assets is controlled manually with skilled personnel. This disclosure relates to data structures which store and manipulate spatial information for autonomous asset inspection. This disclosure also relates to mechanisms to populate the data structures. SUMMARY OF THE INVENTION A first aspect of the invention provides a computer implemented method of populating a data structure suitable for conducting a survey in an environment, comprising: determining that a spatial unit of a set of spatial units is associated with a semantic classification label representative of an inspection asset;receiving inspection criteria associated with the inspection asset, wherein the inspection criteria comprises a threshold condition associated with an inspection sensor;receiving inspection sensor data, from the inspection sensor, representing the inspection asset;applying the threshold condition to the inspection sensor data to generate inspection data; and,modifying an inspection data content of the spatial unit based on the inspection data. An advantage of at least the first aspect is that the method does not need to assume any target-specific requirement and can be agnostic to the inspection target. The data structure described in this first aspect allows for the definition of one or more semantic classification labels as inspection targets as well as the set of inspection sensors and their ranges, eliminating target-specific constraints. This enables an inspection planning algorithm using the data structure to discover, explore and inspect a desired asset (i.e., perform an autonomous inspection survey), in accordance with the inspection criteria. Optionally, the computer implemented method of populating a data structure suitable of the first aspect is for conducting a survey in a harsh environment, such as a subsea environment. Optionally, the set of spatial units is each associated with a modifiable content, the modifiable content comprising inspection field comprising the inspection data. Optionally, the inspection further comprises a second threshold condition (e.g., max range) associated with the inspection sensor; the method further comprising: applying the second threshold condition to the inspection sensor data to generate second inspection data; and optionally, modifying the inspection data content of the spatial unit based on the second inspection data. Optionally, the method further comprises: receiving a second inspection criteria (e.g., magnetometer probe) comprising a third threshold condition (e.g., probe measurement) associated with a second inspection sensor (e.g., probe); receiving second inspection sensor data (e.g., magnetism), from the second inspection sensor, representing the inspection asset; applying the third threshold condition to the inspection sensor data to generate second inspection data; and/or, modifying the inspection data content of the spatial unit based on the second inspection data. Optionally, an inspection confidence value (e.g., confidence of camera range) is associated with the inspection data (e.g., range). Optionally, the method only receives the second inspection sensor data (e.g., magnetism) if the inspection confidence value is greater than an inspection confidence threshold. Optionally, the method further comprises: determining that a second spatial unit of the set of spatial units is associated with a second semantic classification label representative of a second inspection asset (e.g., hull); receiving a third inspection criteria (e.g., range) associated with the second inspection asset and the inspection sensor (e.g. camera), wherein the third inspection criteria comprises a fourth threshold condition (e.g. min range); receiving third inspection sensor data (e.g., more range), from the inspection sensor, representing the second inspection asset; applying the fourth threshold condition to the third inspection sensor data to generate third inspection data; and, modifying an inspection field of the second spatial unit based on the third inspection data. Optionally, the threshold condition, second threshold condition, third threshold condition, and/or fourth threshold condition are each one of: (a) a defined blurriness factor of captured inspection sensor data from the inspection sensor;