EP-4736136-A1 - COMPUTER SYSTEM AND METHOD FOR AUTOMATED OBJECT DETECTION IN OPTICAL SATELLITE IMAGERY USING MACHINE LEARNING

EP4736136A1EP 4736136 A1EP4736136 A1EP 4736136A1EP-4736136-A1

Abstract

Systems, methods, and storage media for automatically locating and characterizing ships in electro-optical satellite imagery are provided. The method includes: extracting a ground sample distance from metadata of an electro-optical satellite image; selecting a ship detection model from a plurality of ship detection models using the extracted ground sample distance; converting the electro-optical satellite image into a standardized format; processing the standardized image using the selected ship detection model to obtain a ship detection output including a bounding box for each ship detection; for each ship detection in the ship detection output, geocoding pixel coordinates of the bounding box into geocoded coordinates; generating a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; and transmitting the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

Inventors

SCHNEIDER, ADAM
RUBIO, GONZALO
WIENS, Jeff
MATASCI, Giona

Assignees

MDA Systems Ltd.

Dates

Publication Date: 20260506
Application Date: 20240501

Claims (20)

1. A method of automatically locating and characterizing ships in electro-optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into geocoded coordinates comprising latitude and longitude coordinates; generating, using the at least one processor, a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; and transmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.
2. The method of claim 1 , further comprising: for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; wherein the ship detection report further includes the length and width of each ship detection in the ship detection output.
3. The method of claim 2, wherein the selected ship detection model is an oriented bounding box model, wherein the ship characterization module is further configured to estimate an orientation of the ship detection, and wherein the ship detection report further includes the orientation of each ship detection in the ship detection output.
4. The method of claim 2, further comprising: for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; and wherein the ship detection report further includes the predicted ship type for each ship detection in the ship detection output.
5. The method of claim 1 , wherein the electro-optical satellite image is in a GeoTIFF format with atmospheric compensation and orthorectification already applied.
6. The method of claim 1 , wherein the ground sample distance describes a distance on ground between centers of each pixel.
7. The method of claim 1 , wherein converting the electro-optical satellite image comprises processing the electro-optical satellite image to have only red, green, and blue colour channels.
8. The method of claim 7, wherein converting the electro-optical satellite image further comprises: where the electro-optical satellite image is panchromatic with only one channel, duplicating the one channel twice to obtain three identical channels, the three identical channels being the RGB channels; and where the electro-optical satellite image includes additional channels beyond the red, green, and blue channels, ignoring the additional channels.
9. The method of claim 1 , wherein the electro-optical satellite image comprises a plurality of tiles, and wherein converting the electro-optical satellite image comprises stitching the plurality of tiles into a single geotiff image file.
10. The method of claim 1 , further comprising performing, using the at least one processor, spatial resampling on the standardized image to the same spatial resolution that the selected ship detection model was trained on.
11. The method of claim 1 , further comprising generating, using the at least one processor, a plurality of equally sized image chips from the standardized image, wherein the image chips are generated with some overlap, and wherein processing the standardized image using the selected ship detection model comprises processing the plurality of equally sized image chips using the selected ship detection model.
12. The method of claim 1 , wherein the plurality of ship detection models include at least one horizontal bounding box (“HBB”) model and at least one oriented bounding box (“OBB”) model.
13. The method of claim 1 , wherein the plurality of ship detection models include at least one oriented bounding box model.
14. The method of claim 1 , wherein the plurality of ship detection models have been trained at different spatial resolutions, and wherein the selected ship detection model and the standardized image have the same spatial resolution, after the standardized image has been resampled to that spatial resolution.
15. The method of claim 1 , further comprising, for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into a geocoded detection comprising latitude and longitude coordinates, and wherein the latitude and longitude coordinates are included in the ship detection output.
16. The method of claim 15, further comprising filtering, using the at least one processor, the geocoded detections using an intersection over area (“loA”) metric to identify overlapping detections, determining that the overlapping detections have an loA meet an loA size threshold, and retaining only one of the overlapping detections with the largest area.
7. A computer system for automatically locating and characterizing ships in electro- optical satellite imagery, the system comprising: a communication interface device for: receiving an electro-optical satellite image comprising image metadata; and transmitting a ship detection report to a user computing device configured to display the ship detection report in a graphical user interface; a data storage device for storing: the electro-optical satellite image; and a plurality of ship detection machine learning models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates; at least one processor configured to: extract a ground sample distance from the image metadata; select a ship detection model from the plurality of ship detection models, wherein the ship detection model is selected using the extracted ground sample distance; convert the electro-optical satellite image into a standardized format for subsequent processing by the ship detection model to obtain a standardized image; process the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, extracting a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; generating the ship detection report, the ship report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output.
18. A non-transitory computer readable storage medium storing processor-executable instructions which, when executed by at least one processor, cause the at least one processor to perform a method of automatically locating and characterizing ships in electro-optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; generating, using the at least one processor, a ship detection report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output; and transmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.
19. A computer system comprising at least one processor configured to execute the method of any one of claims 1 -16.
20. A non-transitory computer readable medium storing processor-executable instructions which, when executed by at least one processor, cause the processor to perform the method of any one of claims 1 -16.

Description

COMPUTER SYSTEM AND METHOD FOR AUTOMATED OBJECT DETECTION IN OPTICAL SATELLITE IMAGERY USING MACHINE LEARNING Technical Field [0001] The following relates generally to earth observation and surveillance, and more particularly to systems and methods for ship detection using computer vision techniques. Introduction [0002] As satellite imagery becomes increasingly available, new approaches of obtaining intelligence from satellite imagery are desired. Obtaining intelligence can include the detection and tracking of objects in and across satellite images. One example application with particular value is the detection and tracking of ships, or dark vessels, across marine scenes captured in optical satellite images. Techniques are desired that enable automatic, fast, and accurate localization and classification of ships in optical satellite imagery of various resolutions. [0003] Accordingly, there is a need for an improved system and method for ship detection and processing and analysis of optical satellite imagery that overcomes at least some of the disadvantages of existing systems and methods. Summary [0004] A method of automatically locating and characterizing ships in electro- optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into geocoded coordinates comprising latitude and longitude coordinates; generating, using the at least one processor, a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; and transmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface. [0005] The method may further comprise: for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; wherein the ship detection report further includes the length and width of each ship detection in the ship detection output. [0006] The selected ship detection model may be an oriented bounding box model, the ship characterization module may be further configured to estimate an orientation of the ship detection, and the ship detection report may further include the orientation of each ship detection in the ship detection output. [0007] The method may further include: for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; and wherein the ship detection report further includes the predicted ship type for each ship detection in the ship detection output. [0008] The electro-optical satellite image may be in a GeoTIFF format with atmospheric compensation and orthorectification already applied. [0009] The ground sample distance may describe a distance on ground between centers of each pixel. [0010] Converting the electro-optical satellite image may include processing the electro-optical satellite image to have only red, green, and blue colour channels. [0011] Converting the electro-optical satellite image may further include: where the electro-optical satellite image is panchromatic with only one channel, duplicating the one channel twice to obtain three identical channels, the three identical channels being the RGB channels; and where the electro-optical satellite image includes additional channels beyond the red, green, and b