Search

EP-4740454-A1 - AIRCRAFT, SYSTEM, AND METHOD

EP4740454A1EP 4740454 A1EP4740454 A1EP 4740454A1EP-4740454-A1

Abstract

An aircraft (3) comprising a camera array (8) for continuously monitoring the entire surface of the earth (2), wherein the camera array (8) comprises: a number of cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16- m, 17, 17- 1, 17-2, 17-3, 17-4, 17-m, 18), each visual axis (19, 20, 21, 22) of which is tilted with respect to the others; N camera modules (11, 12, 13, 14) with N ≥ 2, each camera module (11, 12, 13, 14) having M of the cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) with M ≥ 1, and each of the N camera modules (11, 12, 13, 14) having a processing unit (24, 25, 26) for processing the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18); a network (35), which connects the processing units (24, 25, 26) to one another; and a transmission unit (45) which is connected to the network (35) for data connection (10) of the camera array (8) to a ground station (9) of the aircraft (3), the transmission unit (45) being designed to transmit the image data processed by the processing units (24, 25, 26) to the ground station (9), and a data transmission rate (C1) of the network (35) being too low to transmit the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) and/or the image data processed by the processing units (24, 25, 26) in real time.

Inventors

  • PARSCH, Patrick
  • de Serpa Marques, Bernardo

Assignees

  • Rheinmetall Technical Publications GmbH

Dates

Publication Date
20260513
Application Date
20240627

Claims (15)

  1. 1. Aircraft (3) with a camera group (8) for continuous, comprehensive monitoring of the earth's surface (2), the camera group (8) comprising: several cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17'3, 17-4, 17-m, 18), the lines of sight (19, 20, 21, 22) of which are all arranged at an angle to one another, N camera modules (11, 12, 13, 14) with N > 2, wherein each camera module (11, 12, 13, 14) has M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16'4, 16-m, 17, 17'1, 17'2, 17'3, 17'4, 17-m, 18) with M > 1, and wherein each of the N camera modules (11, 12, 13, 14) has a processing unit (24, 25, 26) for processing the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18), a network (35) which connects the processing units (24, 25, 26) to one another, and a transmission unit (45) connected to the network (35) for data connection (10) of the camera group (8) to a base station (9) of the aircraft (3), wherein the transmission unit (45) is designed to transmit the processed image data of the processing units (24, 25, 26) to the base station (9), and wherein a data transmission rate (Cl) of the network (35) is too low to transmit the data from the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16- 4, 16-m, 17, 17-1, 17-2, 17'3, 17-4, 17-m, 18) and/or the processed image data of the processing units (24, 25, 26) in real time.
  2. 2. Aircraft according to claim 1, characterized in that each processing unit (24, 25, 26) has sufficient computing power to process the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) in real time.
  3. 3. Aircraft according to claim 1 or 2, characterized in that the data transmission rate (Cl) of the network (35) is greater than or equal to, preferably similar to, a data transmission rate (C2) between the transmission unit (45) and the base station (9).
  4. 4. Aircraft according to one of claims 1 - 3, characterized in that the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) is set up to identify an area of interest (56, 57) or several areas of interest (56, 57) of the earth's surface (2) based on the image data and to forward only a part of the image data depicting the area of interest (56, 57) or the areas of interest (56, 57) to the transmission unit (45).
  5. 5. Aircraft according to claim 4, characterized in that the transmission unit (45) is arranged to request the part of the image data which depicts (56, 57) the region or regions of interest (56, 57) from the processing units (24, 25, 26) of the N camera modules (11, 12, 13, 14).
  6. 6. Aircraft according to one of claims 1 - 5, characterized in that the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) is configured to combine the image data of the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17'3, 17-4, 17-m, 18) into a video mosaic (47).
  7. 7. Aircraft according to one of claims 1 - 6, characterized in that the transmission unit (45) is set up to combine the image data of the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17- 1, 17'2, 17'3, 17'4, 17-m, 18) and/or image data of the N camera modules (11, 12, 13, 14) to form a video mosaic (47).
  8. 8. Aircraft according to one of claims 1 - 7, characterized in that the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) has a camera interface (27), a video and data processor (28) and a memory (29).
  9. 9. Aircraft according to one of claims 1 - 8, characterized in that the network (35) has a plurality of switches (36, 37, 38), wherein each of the N camera modules (11, 12, 13, 14) is assigned a switch (36, 37, 38), and wherein the transmission unit (45) is connected to one of the switches (36, 37, 38).
  10. 10. Aircraft according to claim 9, characterized in that each of the N camera modules (11, 12, 13, 14) has one of the switches (36, 37, 38).
  11. 11. Aircraft according to one of claims 1 - 10, characterized by a fuselage (4), a wing (5) attached to the fuselage (4), and a drive (6) attached to the fuselage (4) or to the wing (5), wherein the camera group (8) is attached to the fuselage (4) and/or to the wing (5).
  12. 12. System (1) with X aircraft (3) according to one of claims 1 - 11 with X > 1 and the base station (9), wherein the transmission unit (45) transmits the processed image data of the processing units (24, 25, 26) to the base station (9).
  13. 13. Method for operating an aircraft (3) with a camera group (8) for continuous, comprehensive monitoring of the earth's surface (2), wherein the camera group (8) comprises a plurality of cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18), the viewing axes (19, 20, 21, 22) of which are all arranged inclined to one another, N camera modules (11, 12, 13, 14) with N > 2, wherein each camera module (11, 12, 13, 14) M of the cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17'3, 17-4, 17-m, 18) with M > 1, and wherein each of the N camera modules (11, 12, 13, 14) has a processing unit (24, 25, 26), a network (35) which connects the processing units (24, 25, 26) to one another, and a transmission unit (45) connected to the network (35) for data connection (10) of the camera group (8) to a base station (9) of the aircraft (3), the method comprising the following steps: a) processing (Sl) from the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) using the processing units (24, 25, 26) of the N camera modules (11, 12, 13, 14), and b) transmitting (S2) the processed image data of the processing units (24, 25, 26) to the base station (9) using the transmission unit (45), wherein a data transmission rate (Cl) of the network (35) is too small to transmit the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17'3, 17-4, 17-m, 18) and/or the processed image data of the processing units (24, 25, 26) in real time.
  14. 14. The method according to claim 13, characterized in that the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) identifies one or more regions of interest (56, 57) of the earth's surface (2) based on the image data.
  15. 15. Method according to claim 14, characterized in that only a part of the image data depicting the region of interest (56, 57) or the regions of interest (56, 57) is passed on to the transmission unit (45).

Description

AIRCRAFT, SYSTEM AND PROCESS The present invention relates to an aircraft, a system with such an aircraft and a method for operating such an aircraft. For continuous, comprehensive monitoring of the earth's surface, particularly for so-called WARS applications (Wide-Area Persistent Surveillance), manned or unmanned aerial vehicles (UAVs) with so-called cluster cameras (Cluster Cameras) can be used. A cluster camera of this type comprises several cameras for capturing image data. The image data from the cameras is transmitted to a ground-based base station. The base station is suitable for evaluating the image data. According to internal company findings, processing units of such cluster cameras have a classic Von Neumann computer architecture. Individual single or multiple components of a computer are linked together using a central bus system so that data exchange is possible. The well-known problem with this computer architecture is that a computer with this type of architecture is only able to handle several resource-intensive tasks simultaneously to a certain extent due to the so-called "Von Neumann bottleneck". In the case of a cluster camera with a large number of cameras, however, very large amounts of data must be moved from different sources and processed using simple computing operations. The bus system therefore quickly becomes a bottleneck, especially when several elementary functions such as processing, storage, analysis or the like are used simultaneously. This problem This severely hampers the scalability of cluster cameras, as although the computing power of individual components can be very high, the data exchange between individual computing components is limited. In addition, there is a trade-off or conflict of objectives with such cluster cameras for aircraft, which is generally the basis of airborne WARS applications. Very high resolutions and computing power are required to carry out WARS. At the same time, space, weight and data transmission in aircraft, helicopters and drones are very different and very limited. This means, for example, that an increase in computing power can lead to a reduction in the range and/or the operational duration of such an aircraft. Against this background, one object of the present invention is to provide an improved aircraft. Accordingly, an aircraft with a camera group for continuous, comprehensive monitoring of the earth's surface is proposed. The camera group has a plurality of cameras, the lines of sight of which are all arranged at an angle to one another, N camera modules with N > 2, each camera module having M cameras with M > 1, and each of the N camera modules having a processing unit for processing image data provided by the M cameras, a network that connects the processing units to one another, and a transmission unit connected to the network for data connection of the camera group to a base station of the aircraft. The transmission unit is designed to transmit the processed image data of the processing units to the base station, wherein a data transmission rate of the network is too low to transmit the image data provided by the M cameras and/or the processed image data of the processing units in real time. Because each camera module has its own processing unit, a decentralized or distributed computer architecture can be implemented. In particular, by exploiting the physical limits of the cameras, the task of image processing in live operation of the camera group can be mathematically separated. The previously explained problem of the Von Neumann bottleneck can no longer occur. The camera group can be constructed from so-called OTS products (Off the Shelf), in particular from so-called COTS products (Commercial off the Shelf). This enables the camera group to be manufactured cost-effectively. Furthermore, this also enables any scalability, high flexibility and rapid further development, for example with regard to extended functions. All cameras preferably form a rigid mechanism of the camera group. Due to the rigid mechanism, the camera group advantageously only needs to be calibrated once. The aircraft can be manned or unmanned. The aircraft is preferably a drone, in particular a reconnaissance drone. The terms "aircraft" and "drone" can therefore be interchanged as desired in this case. The aircraft can be used for both civil and military purposes. The aircraft can be an airplane, in particular a fixed-wing aircraft. However, the aircraft can also be a rotary-wing aircraft or helicopter. In particular, the aircraft is suitable for WAPS applications or AW APS applications (EnglJ Airborne Wide-Area Persistent Surveillance). The earth's surface can be a water surface. Accordingly, the terms "earth's surface" and "water surface" can be freely interchanged. In particular, the earth's surface can be the water surface of a sea or an inland body of water. The earth's surface can also be a another object that is far away and therefore appears com