EP-4736121-A1 - POSE ESTIMATION OF AIRCRAFT

Abstract

The present invention relates to pose estimation of aircraft, in particular to pose estimation of aircraft using machine learning. According to an aspect of the present invention, there is provided a computer-implemented method for performing pose estimation of aircraft. The method comprises: obtaining an input image; and using a machine-learning aircraft pose estimation model to obtain one or more aircraft parameters associated with one or more aircraft in the input image.

Inventors

• BILTCLIFFE, Scott, Gilmour

Assignees

• BAE SYSTEMS plc

Dates

Publication Date

20260506

Application Date

20240619

Claims (20)

1. 1. A computer-implemented image processing method for performing pose estimation of aircraft, the method comprising: obtaining an input image; and using a machine-learning aircraft pose estimation model to obtain one or more aircraft parameters associated with one or more aircraft in the input image.
2. 2. The method according to claim 1 , wherein the input image is an image that may contain said one or more aircraft or may not contain any aircraft, and wherein an output of the aircraft pose estimation model is indicative of whether the image does contain said one or more aircraft, and in the event that the output of the aircraft pose estimation model indicates that the image does contain said one or more aircraft, said output is further indicative of said one or more aircraft parameters.
3. 3. The method according to claim 1 or claim 2, wherein said one or more aircraft parameters are indicative of one or more of: a range to a respective one of said one or more aircraft; an aircraft type of a respective one of said one or more aircraft; an orientation of a respective one of said one or more aircraft; and a position of a respective one of said one or more aircraft.
4. 4. The method according to claim 3, wherein the machine-learning aircraft pose estimation model comprises: a segmentation stage configured to output a mask defining which pixels in the input image are estimated to be part of said one or more aircraft and which pixels in the input image are estimated to be part of a background; and/or a position determining stage configured to determine the position of each of said one or more aircraft in the input image; and/or an orientation determining stage configured to determine the orientation of each of said one or more aircraft in the input image.
5. 5. The method according to claim 4, wherein the position determining stage is further configured to determine the range to each of said one or more aircraft.
6. 6. The method according to claim 5, wherein for each one of said one or more aircraft the position determining stage is configured to determine the range to said aircraft based on the mask outputted by the segmentation stage, by: determining a distance for each one of the plurality of pixels that are estimated to be part of said aircraft, based on the mask; and determining the range to said aircraft based on an average of the determined distances for each one of said plurality of pixels.
7. 7. The method according to claim 6, wherein the position determining stage is configured to determine the range to each of said one or more aircraft by assigning a same predetermined distance to all pixels in the mask that are estimated to be part of the background.
8. 8. The method according to any one of claims 4 to 7, wherein the orientation determining stage is configured to output the orientation of each of said one or more aircraft in the form of a quaternion.
9. 9. The method according to any one of claims 4 to 8, wherein the segmentation stage is configured to determine the aircraft type of a respective one of said one or more aircraft.
10. 10. The method according to any one of claims 3 to 9, wherein the machinelearning aircraft pose estimation model is configured to use centre voting to determine the position of each of said one or more aircraft.
11. 11. The method according to claim 10, wherein the machine-learning aircraft pose estimation model is configured to determine the position of each of said one or more aircraft in terms of x, y coordinates by: determining the x coordinate by counting a number of votes for each one of a plurality of vertical lines in the image, and determining the x coordinate based on a position along the x axis of the vertical line having the most votes among the plurality of vertical lines; and determining the y coordinate by counting a number of votes for each one of a plurality of horizontal lines in the image, and determining the y coordinate based on a position along the y axis of the horizontal line having the most votes among the plurality of horizontal lines.
12. 12. The method according to any one of the preceding claims, wherein obtaining the input image comprises: obtaining an initial image having a larger size than the input image; applying a shape recognition algorithm to determine one or more regions of interest, ROIs, in the initial image; and extracting part of the initial image containing one or more of said ROIs as the input image to be classified by the machine learning aircraft pose estimation model.
13. 13. The method according to any one of the preceding claims, wherein the machine-learning aircraft pose estimation model is configured to detect one or more visual cues associated with heavy manoeuvring by one of said one or more aircraft and to output an indication of whether any such visual cues were detected, wherein heavy manoeuvring involves a change in velocity and/or direction of travel that exceeds a threshold.
14. 14. The method according to claim 13, comprising: displaying a graphical indication of heavy manoeuvring on an onscreen display, in response to the machine learning aircraft pose estimation model outputting an indication that one or more such visual cues were detected.
15. 15. The method according to any one of the preceding claims, wherein the method is implemented by one or more computer systems onboard an aircraft, for detecting one or more other aircraft substantially in real-time.
16. 16. The method according to claim 15, wherein the input image comprises a part or a whole of an image captured by an imaging system onboard said aircraft in which the method is implemented.
17. 17. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of the preceding claims.
18. 18. A non-transitory computer-readable storage medium having stored thereon the computer program of claim 17.
19. 19. A system comprising: one or more processors; and computer-readable memory storing computer program instructions which, when executed by the one or more processors, cause the system to: obtain an input image; and use a machine-learning aircraft pose estimation model to obtain one or more aircraft parameters associated with one or more aircraft in the input image.
20. 20. The system according to claim 19, wherein the input image is an image that may contain said one or more aircraft or may not contain any aircraft, and wherein an output of the aircraft pose estimation model is indicative of whether the image does contain said one or more aircraft, and in the event that the output of the aircraft pose estimation model indicates that the image does contain said one or more aircraft, said output is further indicative of said one or more aircraft parameters.

Description

POSE ESTIMATION OF AIRCRAFT FIELD The present invention relates to pose estimation of aircraft, in particular using machine learning. BACKGROUND Automated methods for estimating the pose of aircraft, for example the position and orientation of an aircraft in an image, typically involve using shape recognition to detect specific parts of the aircraft such as the nose, wingtips and so on. However, such methods may not reliably determine the aircraft’s actual orientation, particularly in cases where the aircraft is further away from the camera and therefore only occupies a small portion of the image. SUMMARY According to a first aspect of the present invention, there is provided a computer-implemented image processing method for performing pose estimation of aircraft, the method comprising: obtaining an input image; and using a machine-learning aircraft pose estimation model to obtain one or more aircraft parameters associated with one or more aircraft in the input image. In some embodiments according to the first aspect, the input image is an image that may contain said one or more aircraft or may not contain any aircraft, wherein an output of the aircraft pose estimation model is indicative of whether the image does contain said one or more aircraft, and in the event that the output of the aircraft pose estimation model indicates that the image does contain said one or more aircraft, the image classification is further indicative of said one or more aircraft parameters. In some embodiments according to the first aspect, said one or more aircraft parameters are indicative of one or more of: a range to a respective one of said one or more aircraft; an aircraft type of a respective one of said one or more aircraft; an orientation of a respective one of said one or more aircraft; and a position of a respective one of said one or more aircraft. In some embodiments according to the first aspect, the machine-learning aircraft pose estimation model comprises: a segmentation stage configured to output a mask defining which pixels in the input image are estimated to be part of said one or more aircraft and which pixels in the input image are estimated to be part of a background; and/or a position determining stage configured to determine the position of each of said one or more aircraft in the input image; and/or an orientation determining stage configured to determine the orientation of each of said one or more aircraft in the input image. In some embodiments according to the first aspect, the position determining stage is further configured to determine the range to each of said one or more aircraft. In some embodiments according to the first aspect, for each one of said one or more aircraft the position determining stage is configured to determine the range to said aircraft based on the mask outputted by the segmentation stage, by: determining a distance for each one of the plurality of pixels that are estimated to be part of said aircraft, based on the mask; and determining the range to said aircraft based on an average of the determined distances for each one of said plurality of pixels. In some embodiments according to the first aspect, the position determining stage is configured to determine the range to each of said one or more aircraft by assigning a same predetermined distance to all pixels in the mask that are estimated to be part of the background. In some embodiments according to the first aspect, the orientation determining stage is configured to output the orientation of each of said one or more aircraft in the form of a quaternion. In some embodiments according to the first aspect, the segmentation stage is configured to determine the aircraft type of a respective one of said one or more aircraft. In some embodiments according to the first aspect, the machine-learning aircraft pose estimation model is configured to use centre voting to determine the position of each of said one or more aircraft. In some embodiments according to the first aspect, the machine-learning aircraft pose estimation model is configured to determine the position of each of said one or more aircraft in terms of x, y coordinates by: determining the x coordinate by counting a number of votes for each one of a plurality of vertical lines in the image, and determining the x coordinate based on a position along the x axis of the vertical line having the most votes among the plurality of vertical lines; and determining the y coordinate by counting a number of votes for each one of a plurality of horizontal lines in the image, and determining the y coordinate based on a position along the y axis of the horizontal line having the most votes among the plurality of horizontal lines. In some embodiments according to the first aspect, obtaining the input image comprises: obtaining an initial image having a larger size than the input image; applying a shape recognition algorithm to determine one or more regions of interest, ROIs, in the