Search

US-12626603-B2 - System and method to integrate vehicle information for effective coordinated missions

US12626603B2US 12626603 B2US12626603 B2US 12626603B2US-12626603-B2

Abstract

A method and system to integrate information in a coordinated mission of a group of user vehicles is provided. The method comprises storing a call sign in a database for each user vehicle, among the group of user vehicles, in the coordinated mission; and identifying a speaker vehicle among the user vehicles by comparing a call sign from parts of speech (POS) with the stored call signs. The method further comprises highlighting the identified speaker vehicle in a common display of each user vehicle; and determining whether the POS from the speaker vehicle contains a reference to a point of interest (POI). The method generates a graphical symbol for the POI when the POS from the speaker vehicle contains a reference to the POI, and tags the graphical symbol at a location of the POI in the common display of each user vehicle.

Inventors

  • Gobinathan Baladhandapani
  • Sunit Kumar Saxena
  • Vignesh K
  • Sivakumar Kanagarajan
  • Karthikeyan Mariappan

Assignees

  • HONEYWELL INTERNATIONAL INC.

Dates

Publication Date
20260512
Application Date
20230914

Claims (20)

  1. 1 . A method comprising: storing a plurality of aviation call signs in a call sign database, each of the aviation call signs identifying a respective user aerial vehicle among a group of user aerial vehicles in a coordinated mission; identifying a speaker vehicle among the user aerial vehicles by comparing the aviation call sign from parts of speech (POS) from the speaker vehicle with the stored aviation call signs; highlighting the identified speaker vehicle in a common display of each user aerial vehicle; determining whether the POS from the speaker vehicle contains a reference to a point of interest (POI); generating a graphical symbol for the POI when the POS from the speaker vehicle contains a reference to the POI; and tagging the graphical symbol at a location of the POI in the common display of each user aerial vehicle.
  2. 2 . The method of claim 1 , wherein when the graphical symbol is tagged at the location of the POI, identifying at least one nearby ground feature that is visible at a higher altitude, and tagging the nearby ground feature in the common display of each user aerial vehicle to aid in locating the POI.
  3. 3 . The method of claim 1 , further comprising: sharing a video feed of the speaker vehicle with other user aerial vehicles in the group via the common display in each user aerial vehicle.
  4. 4 . The method of claim 1 , wherein the user aerial vehicles comprise a fleet of aerial vehicles.
  5. 5 . The method of claim 1 , wherein the user aerial vehicles comprise manned aircraft, unmanned aircraft, helicopters, or urban air mobility vehicles.
  6. 6 . A system to integrate information in a coordinated mission of a plurality of user aerial vehicles, the system comprising: a communication unit that hosts a communication radio, and a user audio module; a call sign database configured to store a plurality of aviation call signs identifying respective user aerial vehicles among the plurality of user aerial vehicles in the coordinated mission; at least one processor in operative communication with the communication unit, the at least one processor hosting a plurality of processing modules comprising: a speech-to-text converter module configured to receive audio signals from the communication radio and the user audio module, and convert the audio signals to corresponding text messages; a keyword spotter and sentence segmentor module configured to receive the corresponding text messages from the speech-to-text converter module, the keyword spotter and sentence segmentor module operative to analyze the received text messages for selected words and to segment the text to provide parts of speech (POS) that include an aviation call sign and/or voice data; a call sign extractor module configured to receive the segmented text from the keyword spotter and sentence segmentor module, the call sign extractor module operative to identify the aviation call sign from the segmented text and produce corresponding call sign data; and a speaker identification module configured to receive the call sign data and/or voice data from the call sign extractor module, the speaker identification module operative to identify a speaker vehicle among the user aerial vehicles by comparing the call sign data with the stored aviation call signs from the call sign database; one or more onboard user vehicle systems in operative communication with the speaker identification module, the one or more onboard user vehicle systems configured to send vehicle state data to the speaker identification module; a common display onboard each of the user aerial vehicles and in operative communication with the speaker identification module, the common display configured to receive identifying information for the speaker vehicle from the speaker identification module; and a graphical symbol database in operative communication with the common display onboard each of the user aerial vehicles.
  7. 7 . The system of claim 6 , further comprising a terrain/geographical reference database in operative communication with the speaker identification module and the common display.
  8. 8 . The system of claim 6 , further comprising a search light controller configured to send control signals to a search light feature extractor.
  9. 9 . The system of claim 8 , wherein the common display is configured to receive search light data from the search light feature extractor.
  10. 10 . The system of claim 6 , wherein the user aerial vehicles comprise a fleet of aerial vehicles.
  11. 11 . The system of claim 6 , wherein the user aerial vehicles comprise manned aircraft, unmanned aircraft, helicopters, or urban air mobility vehicles.
  12. 12 . The system of claim 6 , wherein the speech-to-text converter module comprises an air traffic control (ATC) transcription module.
  13. 13 . The system of claim 6 , wherein the audio signals comprise one or more of ATC clearance audio, pilot voice commands, automatic terminal information service (ATIS) messages, airport operation center (AOC) messages, aerodrome flight information service (AFIS) messages, or terminal weather information for pilots (TWIP) messages.
  14. 14 . The system of claim 6 , wherein the onboard user vehicle systems comprise one or more of a flight management system, a traffic collision avoidance system (TCAS), a fuel monitoring system, electrical systems, a flight computer, a weather monitoring system, engine systems, or a transponder.
  15. 15 . The system of claim 6 , wherein the speaker vehicle is graphically highlighted in the common display of each user aerial vehicle.
  16. 16 . The system of claim 6 , wherein when a user aerial vehicle among the user aerial vehicles is in an emergency situation, the common display in each user aerial vehicle is configured to show a list of emergency parameters for the user aerial vehicle in the emergency situation.
  17. 17 . The method of claim 1 , further comprising: sending control signals to a search light feature extractor; and receiving search light data from the search light feature extractor, which uses a search light and camera feed to identify the POI.
  18. 18 . The method of claim 1 , further comprising: mapping the aviation call sign with transponder data for the speaker vehicle.
  19. 19 . The method of claim 18 , wherein the transponder data includes automatic dependent surveillance-broadcast (ADS-B) data for the speaker vehicle.
  20. 20 . The system of claim 6 , wherein in response to determining that POS from the speaker vehicle contains a reference to a point of interest (POI), generating a graphical symbol on the common display in each user aerial vehicle, and tagging the graphical symbol at a location of the POI in the common display.

Description

BACKGROUND Search and rescue (SAR) operations play an important role in reducing devastating impacts caused by disasters. Aerial vehicles are often deployed during missions to analyze a region and obtain the necessary information before proceeding to ground. In coordinated missions such as SAR, or formation flying, information sharing is challenging due to various reasons. For example, communications are done through radio telephony (RT), which is a broadcast type of communication where only one speaker can talk at a given time while others listen. When there are a number of vehicles in mission, it is difficult for all vehicle operators to explain their view/details when needed. Further, when the vehicle operators want to share information, this consumes more RT time, which in turn creates RT congestion. There are also additional problems specific to formation flying. For example, when an aircraft in the formation has an issue, the other aircraft in mission should be notified immediately as these aircraft are typically flying with minimum separation. Currently, a pilot having the aircraft issue must inform the other aircraft through RT, which might not be available. SUMMARY A method and system to integrate information in a coordinated mission of a group of user vehicles is described herein. The method comprises storing a call sign in a database for each user vehicle, among the group of user vehicles, in the coordinated mission; and identifying a speaker vehicle among the user vehicles by comparing a call sign from parts of speech (POS) with the stored call signs. The method further comprises highlighting the identified speaker vehicle in a common display of each user vehicle; and determining whether the POS from the speaker vehicle contains a reference to a point of interest (POI). The method generates a graphical symbol for the POI when the POS from the speaker vehicle contains a reference to the POI, and tags the graphical symbol at a location of the POI in the common display of each user vehicle. BRIEF DESCRIPTION OF THE DRAWINGS Features of the present invention will become apparent to those skilled in the art from the following description with reference to the drawings. Understanding that the drawings depict only typical embodiments and are not therefore to be considered limiting in scope, the invention will be described with additional specificity and detail through the use of the accompanying drawings, in which: FIG. 1 is a flow diagram for a method to integrate information for a coordinated mission of user vehicles, according to one implementation; FIG. 2 is a block diagram of a system to integrate information in a coordinated mission for a fleet of aerial vehicles, according to one embodiment; FIGS. 3A and 3B are example screenshots of a common display for a user aircraft on a mission with other aircraft; and FIG. 4 is an example screenshot of a common display for a user aircraft on a mission with other aircraft, showing a list of emergency parameters for one of the aircraft that is in an emergency situation. DETAILED DESCRIPTION In the following detailed description, embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It is to be understood that other embodiments may be utilized without departing from the scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense. A method and system to integrate vehicle systems for effective coordinated missions are described herein. In general, the present method and system are operative to identify a speaker vehicle among user vehicles in a mission, obtain a location of the speaker vehicle, and highlight features on a common display in each user vehicle. The method and system can also enable the speaker vehicle to share its camera feed or captured point of interest (POI) with the other user vehicles through the common display in each user vehicle. The user vehicles in the mission can include various combinations of manned or unmanned vehicles, such as airplanes, helicopters, urban air mobility (UAM) vehicles, and the like. The present method and system increase a coordinated mission's effectiveness by automatic identification of a particular user vehicle, such as a speaker aircraft, based on a call sign annunciation and mapping the call sign with the transponder data for the particular user vehicle, such as automatic dependent surveillance-broadcast (ADS-B) data for the speaker aircraft. For example, once the transponder data is derived through call sign filtering, the transponder data is mapped, along with data extracted from clearances, into various avionics systems. After successfully identifying the particular user vehicle, the present method and system directs the common display of each user vehicle in the mission to show various unique features. The common display implementation used in the present method and system can provide various features, in