Search

JP-2026074763-A - System, detection system for detecting foreign objects on runways, and method for the system

JP2026074763AJP 2026074763 AJP2026074763 AJP 2026074763AJP-2026074763-A

Abstract

[Problem] This relates to a system for detecting foreign objects on a runway. [Solution] This system comprises a thermal camera having a first field of view and configured to capture a thermal image of a target area on the runway, and a visible light camera having a second field of view and configured to capture a visible light image of a target area on the runway, wherein the first field of view overlaps with the second field of view. The system further comprises a processor and a memory that communicates with the processor and stores instructions that can be executed by the processor. The processor is configured to detect thermal object images in the thermal image and visible light object images in the visible light image, and to determine that a foreign object has been detected when a thermal object image and a visible light object image are detected in the thermal image and visible light object image, respectively. [Selection Diagram] Figure 1

Inventors

  • チュウ ロン－ジー デイビッド
  • チュウ ロン－キ フィービー

Assignees

  • チュウ キエン ミャオ デイビッド
  • チュウ ロン－キ フィービー

Dates

Publication Date
20260507
Application Date
20241021

Claims (20)

  1. A method for identifying foreign objects on a runway based on a thermal image of the foreign object, To capture a thermal image of the target area on the runway, To capture a visible light image of the target area on the runway, The detection of thermal object images within the thermal image and the detection of visible light object images within the visible light image, In the thermal image and the visible light object image, if the thermal image and the visible light object image are detected, it is determined that the foreign object has been detected. Identifying the visible light object image and classifying it into the object category of the foreign object, A method comprising: determining the relationship between the object category of the foreign object in the visible light image and the thermal object image of the foreign object; and, when the thermal object image of the foreign object is detected, identifying the foreign object by mapping the object category to the thermal object image based on the relationship.
  2. The method according to claim 1, wherein determining the foreign object includes generating at least one attribute of the foreign object in each of the thermal object image and the visible light object image, and comparing the at least one attribute of the foreign object in the thermal object image and the visible light object image, wherein the foreign object is detected if the at least one attribute of the foreign object in the thermal object image and the visible light object image are the same.
  3. The method according to claim 2, wherein the at least one attribute of the foreign object includes the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image.
  4. The method according to claim 3, wherein, if the distance between the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image is within the position parameter, then at least one attribute of the foreign object in the thermal object image and the visible light object image is the same.
  5. The method according to any one of claims 2 to 4, wherein the at least one attribute of the foreign object includes the size of the thermal object image and the visible light object image.
  6. The method according to claim 5, wherein if the difference between the size of the thermal object image in the thermal image and the size of the visible light object image in the visible light image is within the size parameter, then at least one attribute of the foreign object in the thermal object image and the visible light object image is the same.
  7. The method according to any one of claims 1 to 6, further comprising obtaining an enlarged thermal object image and an enlarged visible light object image when the foreign object is detected.
  8. This further includes identifying the object category of foreign objects in the thermal image, and identifying the object category is The thermal image is segmented into multiple thermal image regions, Assigning a feature vector to each of the aforementioned multiple thermal image regions, The process involves comparing the aforementioned feature vector with a plurality of reference feature vectors, wherein each of the plurality of reference feature vectors represents an object category. The method according to any one of claims 1 to 7, comprising identifying the reference feature vector and its object category that is closest to the feature vector.
  9. The method according to claim 8, wherein segmenting the thermal image includes labeling each pixel in the thermal image, and grouping the labeled pixels having the same characteristics into a plurality of groups to form the plurality of thermal image regions.
  10. Identifying the aforementioned object category The visible light image is segmented into multiple visible light image regions, Assigning a feature vector to each of the aforementioned multiple visible light image regions, The process involves comparing the aforementioned feature vector with a plurality of reference feature vectors, wherein each of the plurality of reference feature vectors represents an object category. The method according to any one of claims 1 to 11, comprising identifying the reference feature vector and its object category that is closest to the feature vector.
  11. The method according to claim 10, wherein segmenting the visible light image includes labeling each pixel in the visible light image, and grouping the labeled pixels having the same characteristics into a plurality of groups to form the plurality of visible light image regions.
  12. The method according to claim 10 or 11, further comprising training a thermal camera to detect the foreign object based on the visible light image from the visible light camera.
  13. The method according to any one of claims 1 to 12, further comprising generating a thermal profile model of the foreign object based on the aforementioned relationship and optimizing the detection configuration parameters of the thermal camera based on the thermal profile model.
  14. A detection system for identifying foreign objects on a runway based on thermal images of the foreign objects, A thermal camera having a first field of view and configured to capture a thermal image of a target area on the runway, A visible light camera having a second field of view and configured to capture a visible light image of the target area on the runway, wherein the first field of view overlaps with the second field of view, A processor that communicates with the thermal camera and the visible light camera, The system includes a memory that communicates with the processor and stores instructions that can be executed by the processor, The aforementioned processor, To detect the thermal object image within the thermal image, To detect a visible light object image within the aforementioned visible light image, In the thermal image and the visible light object image, if the thermal image and the visible light object image are detected, it is determined that the foreign object has been detected. Identifying the visible light object image and classifying it into the object category of the foreign object, The system is configured to determine the relationship between the object category of the foreign object in the visible light image and the thermal object image of the foreign object, A detection system that identifies a foreign object by mapping the object category to the thermal object image based on the relationship when a thermal object image of the foreign object is detected.
  15. The detection system according to claim 14, wherein, in order to determine the foreign object, the processor is configured to generate at least one attribute of the foreign object in each of the thermal object image and the visible light object image, and to compare the at least one attribute of the foreign object in the thermal object image and the visible light object image, and the foreign object is detected if the at least one attribute of the foreign object in the thermal object image and the visible light object image are the same.
  16. The detection system according to claim 15, wherein the at least one attribute of the foreign object includes the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image.
  17. The detection system according to claim 16, wherein if the distance between the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image is within the position parameter, then at least one attribute of the foreign object in the thermal object image and the visible light object image is the same.
  18. The detection system according to any one of claims 15 to 17, wherein the at least one attribute of the foreign object includes the size of the thermal object image and the visible light object image.
  19. The detection system according to claim 18, wherein if the difference between the size of the thermal object image in the thermal image and the size of the visible light object image in the visible light image is within the size parameter, then at least one attribute of the foreign object in the thermal object image and the visible light object image is the same.
  20. The detection system according to any one of claims 14 to 19, wherein the processor is further configured to zoom in on the thermal camera and the visible light camera to acquire an enlarged thermal image and an enlarged visible light image of the object when the foreign object is detected.

Description

This invention relates to a system, a detection system for detecting foreign objects on a runway, and a method for the system. Foreign object debris (FOD) on airport runways poses a danger to aircraft landings and takeoffs. To reliably detect FOD under normal clear weather conditions, FOD detection systems using visible light spectrum cameras are employed. Under normal clear weather conditions, such as when there is no fog, FOD detection systems can detect FOD with high accuracy by capturing and processing high-resolution images of the FOD. Examples of FOD include engine and aircraft parts, tools, construction waste, rubber materials, and natural substances. However, in adverse weather conditions, especially foggy conditions, the operation of the FOD detection system may be negatively affected and reduced. Because the FOD detection system operates only in the visible light spectrum, it may not be able to reliably detect FODs in foggy weather conditions, i.e., under poor visibility conditions. Under such conditions due to fog, visibility on the runway typically drops to less than 1 km, making it impossible to "see" the FOD. Visibility conditions are classified into several categories. For example, Cat II represents standard operations corresponding to runway visual range (RVR) in the range of 550 meters (1,800 feet) to 300 meters (1,000 feet). Cat IIIa represents precision instrument approach and landing operations with an RVR of 175 meters (600 feet) or more. Cat IIIb represents precision instrument approach and landing operations with an RVR of less than 175 meters (600 feet) and 50 meters (200 feet) or more. Cat IIIc represents precision instrument approach and landing operations with no RVR restrictions, including cases with zero visibility. Airport runway visibility varies depending on the airport's geographical location and is classified accordingly. Many FOD detection systems can detect FOD within an airport with Cat II visibility, but cannot be used at airports with Cat IIIa, Cat IIIb, or Cat IIIc visibility. Furthermore, FOD detection systems often generate false alarms or incorrect warnings. These false warnings can be caused by phenomena such as light reflection from artificial light sources, including nearby buildings or runway lights. When such artificial light reflects off the smooth surface of the runway, or off puddles or standing water on the runway surface, the FOD detection system may identify it as an FOD and issue a false warning. The number of such false warnings due to reflection often increases significantly after rainfall, when puddles or standing water form on the runway surface. While such reflections occur during the day, they are even more frequent at night, and during the early morning and evening hours. Therefore, it is crucial to provide a solution that can detect FODs even under poor visibility conditions such as bad weather, and prevent or minimize false FOD detections. Various embodiments provide a method for detecting foreign objects on a runway. This method comprises capturing a thermal image of a target area on the runway, capturing a visible light image of the target area on the runway, detecting thermal object images within the thermal image, detecting visible light object images within the visible light image, and determining that a foreign object has been detected when thermal object images and visible light object images are detected in the thermal image and visible light object images, respectively. In various embodiments, determining a foreign object may involve generating at least one attribute of the foreign object in both the thermal object image and the visible light object image, and comparing at least one attribute of the foreign object in the thermal object image and the visible light object image. A foreign object may be detected if at least one attribute of the foreign object in the thermal object image and the visible light object image is the same. In various embodiments, at least one attribute of the foreign object may include the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image. In various embodiments, if the distance between the position of the thermal object image in the thermal image and the position of the visible light object image in the visible light image is within the positional parameter, then at least one attribute of the foreign object in the thermal object image and the visible light object image may be the same. In various embodiments, at least one attribute of the foreign object may include the size of the thermal object image and the visible light object image. In various embodiments, if the difference between the size of the thermal object image in the thermal image and the size of the visible light object image in the visible light image is within the size parameter, then at least one attribute of the foreign object in the thermal object image and