EP4210021A1 - Systèmes et procédés pour ajouter des données pertinentes à des messages de transcription atc - Google Patents
Systèmes et procédés pour ajouter des données pertinentes à des messages de transcription atc Download PDFInfo
- Publication number
- EP4210021A1 EP4210021A1 EP22214302.6A EP22214302A EP4210021A1 EP 4210021 A1 EP4210021 A1 EP 4210021A1 EP 22214302 A EP22214302 A EP 22214302A EP 4210021 A1 EP4210021 A1 EP 4210021A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- atis
- data
- atc
- text
- flight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013518 transcription Methods 0.000 title claims abstract description 66
- 230000035897 transcription Effects 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000000007 visual effect Effects 0.000 claims abstract description 28
- 238000004458 analytical method Methods 0.000 claims abstract description 13
- 238000004891 communication Methods 0.000 claims description 16
- 230000008859 change Effects 0.000 claims description 10
- 238000013459 approach Methods 0.000 claims description 7
- 238000009877 rendering Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 description 51
- 230000006870 function Effects 0.000 description 12
- 238000007726 management method Methods 0.000 description 10
- ALRKEASEQOCKTJ-UHFFFAOYSA-N 2-[4,7-bis(2-amino-2-oxoethyl)-1,4,7-triazonan-1-yl]acetamide Chemical compound NC(=O)CN1CCN(CC(N)=O)CCN(CC(N)=O)CC1 ALRKEASEQOCKTJ-UHFFFAOYSA-N 0.000 description 5
- 238000013500 data storage Methods 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000003416 augmentation Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010006 flight Effects 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0004—Transmission of traffic-related information to or from an aircraft
- G08G5/0013—Transmission of traffic-related information to or from an aircraft with a ground station
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0017—Arrangements for implementing traffic-related aircraft activities, e.g. arrangements for generating, displaying, acquiring or managing traffic information
- G08G5/0021—Arrangements for implementing traffic-related aircraft activities, e.g. arrangements for generating, displaying, acquiring or managing traffic information located in the aircraft
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/003—Flight plan management
- G08G5/0039—Modification of a flight plan
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0073—Surveillance aids
- G08G5/0091—Surveillance aids for monitoring atmospheric conditions
Definitions
- the subject matter described herein relates to the automated transcription of air traffic control (ATC) messages and augmenting the ATC messages with further relevant information.
- ATC air traffic control
- Air traffic control (ATC) systems may utilize voice communications to transmit various messages between air traffic controllers and pilots.
- the messages that are transmitted between air traffic controllers and pilots include, among other things, air traffic control clearances, various advisories, and aviation weather service, which help ensure coordination of aircraft movement and appropriate aircraft separation.
- Broadcast messages from/to ATC can be transcribed and displayed on a pilot window of an Electronic Flight Bag (application).
- a voice to text transcription application transcribes the broadcast messages.
- the transcription application provides direct translation of conversation messages between crew and a ground ATC member.
- ATIS Automatic Terminal Information Service
- ATIS broadcasts contain relevant information, such as current weather information, active runways, winds, available approaches, QNH (the pressure set on the subscale of the altimeter so that the instrument indicates its height above sea level), and any other information supporting pilots.
- Other relevant reports include Notice to Air Missions (NOTAMs) and Meteorological Terminal Air Reports (METARs).
- NOTAMs Notice to Air Missions
- METARs Meteorological Terminal Air Reports
- Flight crew study the various services, such as ATIS and NOTAMs, during the preflight and also listen to these reports before beginning an approach to a destination aerodrome. Information from such reports may impact an active flight plan and any changes to the active flight plan requested by ATC. Simultaneously listening to the reports and taking into account ATC requests can be taxing for a pilot during an already high workload flight phase and could lead to inaccuracies or errors during takeoff and landing.
- a method of converting voice to text messages in an aircraft.
- the method includes transcribing, using a first voice to text application, voice messages between a member of the flight crew and Air Traffic Control (ATC) to provide ATC text messages, transcribing, using a second voice to text application, a voice-automatic terminal information service report (voice-ATIS) to provide an ATIS text report, determining, via a processor, flight context data based at least on an analysis of the ATC text messages, determining, via the processor, relevant ATIS data from the ATIS text report using the flight context data, and rendering, via the processor, a visual User Interface (UI) including at least some of the ATC text messages and at least some of the relevant ATIS data on the same ATC transcription page.
- ATC Air Traffic Control
- UI visual User Interface
- the visual UI includes a selectable ATIS element that, when selected, causes rending of an ATIS transcription page including the ATIS text report.
- the method includes receiving, via the processor, an active flight plan from a Flight Management System (FMS) of the aircraft, and determining, via the processor, the flight context data based at least on an analysis of the ATC text messages and the active flight plan.
- FMS Flight Management System
- the method includes receiving, via the processor, an active flight plan from a Flight Management System (FMS) of the aircraft, comparing the relevant ATIS data to corresponding data included in the active flight, and highlighting part of the ATIS text report based on the comparison.
- FMS Flight Management System
- the method includes receiving, via the processor, an active flight plan from the Flight Management System (FMS) of the aircraft, and determining, via the processor, update data representing a recommended update to the active flight plan based on a comparison between the relevant ATIS data and the active flight plan.
- FMS Flight Management System
- the visual UI including the ATC transcription page includes a selectable update element that, when selected, causes the update data to be submitted to the FMS of the aircraft.
- the ATIS text report includes one or more of: runway condition information, runway in use information, surface wind information, weather information, visibility information, cloud height information, approach type information, and holding information.
- the ATC text messages include a description of an update to the active flight plan and the relevant ATIS data is determined based on ATIS data associated with the update to the active flight plan.
- the update to the active flight plan includes a change in arrival or departure runway, which constitutes a new arrival or departure runway
- the ATIS data associated with the update to the active flight plan includes one or more of the following for the new arrival or departure runway: runway condition information and runway wind information.
- the visual UI including the ATC transcription page includes a selectable update element that, when selected, causes the update to be submitted to the FMS of the aircraft.
- the first and second voice to text applications are the same.
- determining relevant ATIS data from the ATIS text report using the flight context data includes determining that ATIS data is relevant based at least in part on proximity of aircraft to a location referenced in the ATIS text report.
- the method comprises transcribing one or more other reports including Aviation Routine Weather Reports (METARs) and/or Aviation Selected Special Weather Reports (SPECIs) and/or Notice to Air Missions (NOTAMs) and determining, via the processor, relevant data from the one or more other reports using the flight context data; and rendering, via the processor, a visual User Interface (UI) including at least some of the ATC text messages and at least some of the relevant data on the same ATC transcription page.
- METARs Aviation Routine Weather Reports
- SPECIs Aviation Selected Special Weather Reports
- NOTAMs Notice to Air Missions
- the visual UI depicts the ATC text messages and the relevant ATIS data on the same ATC transcription page in a visually differentiated manner.
- a system for converting voice to text messages in an aircraft includes a receiver and transmitter for receiving and transmitting voice messages between a member of the flight crew and Air Traffic Control (ATC), a display device, and a processor in operable communication with the receiver and transmitter and the display device.
- the processor is configured to execute program instructions.
- the program instructions are configured to cause the processor to: transcribe the voice messages between the member of the flight crew and the ATC to provide ATC text messages, transcribe a voice-automatic terminal information service report (voice-ATIS) to provide an ATIS text report, determine flight context data based at least on an analysis of the ATC text messages, determine relevant ATIS data from the ATIS text report using the flight context data, and render a visual User Interface (UI) on the display device including at least some of the ATC text messages and at least some of the relevant ATIS data on the same ATC transcription page.
- voice-ATIS voice-automatic terminal information service report
- UI visual User Interface
- the program instructions are configured to cause the processor to: receive an active flight plan from a Flight Management System (FMS) of the aircraft and determine the flight context data based at least on an analysis of the ATC text messages and the active flight plan.
- FMS Flight Management System
- the program instructions are configured to cause the processor to: receive an active flight plan from the Flight Management System (FMS) of the aircraft, and determine update data representing a recommended update to the active flight plan based on a comparison between the relevant ATIS data and the active flight plan.
- FMS Flight Management System
- the visual UI including the ATC transcription page includes a selectable update element that, when selected, causes the update data to be submitted to the FMS of the aircraft.
- determining relevant ATIS data from the ATIS text report using the flight context data includes determining that ATIS data is relevant based at least in part on proximity of aircraft to a location referenced in the ATIS text report.
- the ATC text messages include a description of an update to an active flight plan and the relevant ATIS data is determined based on ATIS data associated with the update to the active flight plan.
- Systems and methods described herein provide pilot and ATC conversation transcription in a user interface that allows a text version of the conversation between ATC and the pilot to be viewed in a single display window.
- the user interface is augmented with one or more of the following features: an interface to ATIS data (and other report sources such as METAR and NOTAM); the ATIS data is presented in an organized and uncluttered format; ATIS data that is relevant to a current flight context is differentiated; the relevant ATIS data is populated in the ATC transcription window in some instances such as when there is a change of clearance information; and a selectable GUI (Graphical User Interface) element is provided to submit any updates to the flight plan as a result of the relevant ATIS data and analysis of the ATC transcription.
- GUI Graphic User Interface
- An ATC transcription application that also tunes to an ATIS (or other report source) frequency and transcribes the ATC application and also the ATIS report (voice to text).
- a data extractor that analyzes the ATIS data (or other report) based on current flight context to determine relevant ATIS data.
- the flight context is derived from the ATC conversation in its transcribed form and also from an active flight plan.
- a data comparator compares the relevant data with corresponding data in the active flight plan to determine any changes. The changes to the active flight plan can be highlighted in a text report window as compared to text that is not determined to be relevant and changed.
- the application may receive the active flight plan through a cFMS (connected Flight Management System) or another open world system.
- a decoder interprets the conversation data between the pilot and ATC and automatically populates part of the ATIS data in the ATC transcription window.
- a selectable element GUI (Graphical User Interface)
- GUI Graphic User Interface
- the decoder works in conjunction with the FMS to update the active flight plan.
- FIG. 1 depicts an exemplary embodiment of a system 100 for adding relevant data to ATC transcription messages.
- the system 100 is associated with an aircraft 10.
- the illustrated system 100 includes a processing system 160 configured to execute a voice to text transcription application 102 coupled to a data extractor 104, a data comparator 106, a flight context analyzer 144 and a display generator 108.
- the voice to text transcription application 102 receives ATC broadcast messages 124, 125 sent between ATC 114 and a member of the flight crew in voice form.
- the system 100 further includes a display device 110 for rendering an ATC transcription window 206 (see FIGS. 2 and 3 ) with displayed ATC text messages 155, 156 that have been transcribed by the voice to text transcription application 102.
- voice reports 164 are transcribed by the voice to text transcription application 102 to generate a text report 162, which can be displayed as shown in FIGS. 3 and 4 .
- the text report 162 is filtered by the processing system 160 as to relevancy to an active flight plan, current flight context and/or the ATC text messages 154 to provide relevant report data 130. At least part of the relevant report data 130 is displayed in the ATC transcription window 206 in the form of displayed relevant report data 166.
- the system 100 further includes an avionics service system 115 that provides flight plan data 158 from the FMS 118 describing an active flight plan and the current status of the aircraft 10 along the flight plan.
- the system 100 is implemented, at least in part, by a processing system 506 as described in further detail below with respect to FIG. 7 .
- the processing system 506 executes programming instructions to analyze ATC text messages 154 based on flight context and to extract relevant report data 130 based at least partly on the analysis and further to present displayed ATC text messages 155, 156 in association with displayed relevant report data 166 in the ATC transcription window 206 (see FIGS. 2 and 3 ).
- FIG. 1 is a simplified representation of the system 100 associated with an aircraft 10 for purposes of explanation and is not intended to limit the subject matter in any way.
- the system 100 onboard the aircraft 10 may include any number of different onboard systems configured to support operation of the aircraft 10, and the subject matter described herein is not limited to any particular type or number of onboard systems.
- the ATC 114 is a service provided for the purpose of: preventing collisions between aircraft and on the maneuvering area between aircraft and obstructions; and expediting and maintaining an orderly flow of air traffic.
- the aircraft 10 submits a flight plan to ATC 114.
- the path of the aircraft 10 may have to be changed from the original flight plan, e.g. to move around bad weather or avoid a congested sector.
- the pilot may request a change to the flight plan by submitting such a request to the ATC 114.
- the changes may include changes in flight parameters like heading, speed and/or altitude and also changes in target runway and other landing information.
- the system 100 of FIG. 1 includes a receiver 140 and a transmitter 142.
- the receiver 140 receives FROM ATC broadcast messages 125 over a voice based radio channel.
- the receiver further receives voice reports 164 over another voice based radio channel.
- the voice reports 164 may be ATIS reports, METAR reports, NOTAM reports, Pilot Reports (PIREP), and the like.
- the transmitter 142 broadcasts TO ATC broadcast messages 124 over the voice based radio channel.
- the TO ATC broadcast messages 125 are initiated by a member of the flight crew speaking into a microphone 150, which may be part of a headset.
- the received FROM ATC broadcast messages 124 may be heard by a member of the flight crew through a speaker 151, which may be part of the headset.
- the ATC broadcast messages 124, 125 (which include both received and transmitted ATC broadcast messages 124, 125 by/from the aircraft 10) and the voice reports 164 can be passed to the voice to text transcription application 102 for conversion from a voice format to a text format.
- the voice to text transcription application 102 can use any suitable technique for performing conversion from voice to text.
- the voice to text transcription application 102 may use artificial intelligence (e.g. a recurrent neural network) that transcribes analog or digital aviation audio into text in near-real time.
- the artificial intelligence application is trained with an aviation specific dataset allowing the voice to text transcription application 102 to perform better than general natural language processing applications, which can provide inaccurate transcriptions due to cadence, noise, and vocabulary of the aviation industry.
- separate voice to text transcription applications 102 may be provided for the voice reports 164 and the ATC broadcast messages 124, 125 so that the respective voice to text transcription applications 102 can be optimally configured for the different lexicons and voice styles of the voice reports 164 and the ATC broadcast messages 124, 125.
- the voice to text transcription application 102 outputs ATC text messages 154 and a text report 162 (or reports) that are provided to the display generator 108 for display on the display device 110.
- the receiver 140 receives FROM ATC broadcast messages 124 that are relevant to the ownship aircraft 10 and also other aircraft.
- the processing system 160 is configured to sort messages directed to the ownship aircraft 10 from those directed to other aircraft so that the display generator 108 exclusively displays ATC text messages 154 that are relevant to the ownship aircraft 10.
- the processing system 160 can perform this sorting processing by comparing an identifier of the ownship aircraft 10 with an identifier that is used by ATC 114 (and transcribed by the voice to text transcription application 102) with respect to each ATC text message 154.
- the avionics service system 115 includes a Flight Management System (FMS) 118, which is an on-board multi-purpose navigation, performance, and aircraft operations computer.
- FMS Flight Management System
- a primary function of the FMS 118 is in-flight management of a flight plan.
- sensors such as Global Positioning System (GPS) and Inertial Navigation System (INS) often backed up by radio navigation
- GPS Global Positioning System
- INS Inertial Navigation System
- the FMS 118 sends the flight plan for display to an Electronic Flight Instrument System (EFIS), a Navigation Display (ND), and/or a Multifunction Display (MFD).
- EFIS Electronic Flight Instrument System
- ND Navigation Display
- MFD Multifunction Display
- a flight plan contains the following information as is relevant to the flight, by way of example: aircraft identification, flight rules and type of flight, number and type(s) of aircraft and wake turbulence category, equipment, departure aerodrome, estimated off-block time, cruising speed, cruising level, route to be followed, destination aerodrome and total estimated elapsed time, alternate destination aerodrome, fuel endurance, total number of persons on board, emergency and survival equipment, and other information.
- the flight context analyzer 144 receives active flight plan data 158 embodying the flight plan from the avionics service system 115, particularly from the FMS 118.
- the flight plan data 158 may be provided through a cloud service 116 connected to the FMS 118.
- Other interfaces e.g. an Application Programming Interface
- the processing system 160 and the display device 110 is provided as part of an electronic flight bag (EFB) device that wirelessly connects to the FMS 118.
- the flight plan data 158 further includes a current location of the aircraft 10 along the active flight plan.
- the flight context analyzer 144 further receives the ATC text messages 154 to determine flight context from the communications between the flight crew and the ATC 114.
- landing or departure information e.g. runways, route to be taken, instrument vectoring, aerodrome conditions
- flight context the proximity of the aircraft 10 to a destination or departure runway, or its progress along the flight plan towards the destination runway relative to certain threshold markers, is factored into the flight context.
- the flight context is embodied into output flight context data 170.
- the data extractor 104 receives the text report 162 and the flight context data 170.
- the data extractor 104 isolates data in the text report 162 that is relevant to the flight context based on the flight context data 170. For example, report data relevant to a destination runway set in the flight plan is extracted or requested changes (or other clearance information) to the destination runway according to the ATC text messages 154 described in the flight context data 170 are extracted.
- a location of the aircraft along the flight plan may also be determinative on relevancy of the information in the text report 162. For example, runway conditions described in the text report 162 may not become relevant until the final approach stage is reached.
- the data extractor 104 organizes the text report 162 in a table structure that groups together runway specific information.
- the flight context data 170 may include runway information based on the active flight plan or the ATC text messages 154 (or other key field information) and the table structure can be looked up based thereon in order to extract the relevant report data 130.
- the data extractor 104 acts as a filter of the text report 162 to extract relevant parts of the data contained in the text report 162 based on the flight context described in the flight context data 170.
- the data extractor 104 outputs the extracted relevant data as relevant report data 130.
- the data comparator 106 receives the relevant report data 130 and compares information contained therein with corresponding information contained in the active flight plan data 158. For example, runway conditions, weather information, destination runway and other data items are compared.
- the data comparator 106 outputs a comparison result 172 identifying changed data in the text report 162 as compared to the corresponding information in the active flight plan data 158. In this way, the display generator is able to highlight the changed data when displaying the text report, as will be described with respect to FIGS. 4 and 5 .
- the processing system 160 includes a decoder 174 that receives the comparison result 172 and determines which, if any, of the changed information should be submitted to the FMS 118 as part of a flight plan update. For example, changed wind information, changed destination runway information, changed runway conditions, etc. may need to be submitted to the FMS 118.
- the decoder 174 assesses which of the relevant report data 130 that has changed (according to the comparison result 172) should be recommended as a flight plan update according to predetermined rules.
- the decoder 174 thus outputs recommendation data 176 representing a recommendation to update the flight plan.
- the processing system 160 includes a display generator 108 for generating an ATC transcription window 206 for the display device 110 as exemplified in FIGS. 3 and 4 and a displayed text report 302 as exemplified in FIGS. 4 and 5 .
- the ATC transcription window 206 includes displayed ATC text messages 155, 156 according to the ATC text messages 154 from the voice to text transcription application 102.
- the ATC transcription window 206 further includes displayed relevant report data 166, which is generated based on the text report 162 and may focus on changed data according to the comparison result 172. Further, the displayed relevant report data 166 may display report data that should be submitted to the FMS 118 as an update to the flight plan according to the recommendation data 176.
- the display generator 108 associates a selectable update element 208 with the displayed relevant report data 166 that gives a flight crew member the option to accept the recommended update to the flight plan.
- update data 178 is sent to the FMS 118 from the processing system 160 and the FMS incorporates the update data 178 into the active flight plan.
- the displayed text report 302 includes the text report 162 and may also highlight changed parts of the text report 162 as compared to that submitted into the FMS 118 as part of the process of setting up the flight plan (e.g. prior to departure). The highlighted changes are according to the comparison result 172.
- An ATIS report may include one or more of the following data items: name of aerodrome, arrival and/or departure indicator, contract type, if communication is via D-ATIS, designator, time of observation, if appropriate, type of approach(es) to be expected, the runway(s) in use, status of arresting system constituting a potential hazard, if any, significant runway surface conditions and, if appropriate, braking action, holding delay, if appropriate, transition level, if applicable, other essential operational information, surface wind direction (in degrees magnetic) and speed, including significant variations and, if surface wind sensors related specifically to the sections of runway(s) in use are available and the information is required by operators, the indication of the runway and the section of the runway to which the information refers, visibility and, when applicable, RVR, present weather, cloud below 1 500 m (5 000 ft) or below the highest minimum sector altitude, whichever is greater; cumulonimbus; if the sky is obscured, vertical visibility when available, air temperature, dew point temperature, altimeter setting(
- information relating to a runway that is not a target for the aircraft 10 according to its flight plan is not relevant.
- data already known to the pilot according to the flight plan data 158 may be considered not relevant.
- information may only be relevant at certain stages along the flight plan (e.g. cloud information may not relevant once the cloud ceiling has been passed).
- the data extractor 104 and the data comparator 106 work together to sort part of relevant report data 130 to be displayed in the ATC transcription window 206 to provide contextualized report updates as part of the ATC transcription. Further, the conversation between the ATC and the flight crew may alter the parts of the text report 162 that are considered relevant (e.g. when a runway change is being requested by the ATC 114).
- the processing system 160 may include one or more of the following data items: location, date/time of report, surface wind, visibility, runway visual range, weather type, cloud information, temperature, QNH, wind shear, runway state, and trend.
- NOTAMs include one or more of the following data items: hazards (such as air shows, parachute jumps and glider or micro-light flying), flights by important people (such as heads of state), closed runways or taxiways, unserviceable navigational instruments, airspace restrictions, unserviceable lights on tall obstructions, and temporary obstacles.
- the processing system 160 generates a visual UI 200 for ATC text messages as shown in FIG. 2 using information contained within a text report that is detailed with respect to the visual UI 300 for text reports as shown in FIG. 4 .
- the voice to text transcription application 102 transcribes ATC broadcast messages 124, 125 and the voice reports 164.
- the display generator 108 outputs displayed ATC text messages 156 from the ATC and displayed ATC text messages 155 from the flight crew in the ATC transcription window 206. In the same ATC transcription window 206, the display generator 108 outputs displayed relevant report data 166 based on the text report 162.
- the flight context analyzer 144 analyzes the ATC text messages 154 and the active flight plan data 158 to produce flight context data 170 defining a flight context for use in determining relevancy of data contained in the text report 162.
- the flight context includes a "request change of RWY 27L" message, which is interpreted by the flight context analyzer 144 as being a relevant data item for filtering the text report 162.
- the data extractor 104 filters the ATIS report based on the flight context data 170, specifically based on the requested runway change to runway 27L. Referring to FIG. 4 , the data extractor 104 finds the relevant report data 130 including wind and surface conditions information for that runway.
- the text report 162 may be an ATIS report, a METAR report and/or a NOTAM report.
- the data comparator 106 compares this information with that already entered into the FMS 118 as defined by the active flight plan data 158. Since the FMS 118 has been entered with information for a different runway, the comparison result 172 will indicate all of the information about runway 27L as being new information. Based on the comparison result 172, the display generator 108 highlights the new information as shown by the highlighted parts 304 of the displayed text report 302. The decoder 174 receives the comparison result 172 and assesses whether changes should be submitted to the FMS 118.
- the decoder 174 provides recommendation data 176 that describes relevant report data 130 from the text report 162 that has been determined to be suitable for an update submission to the FMS 118.
- the part (or whole) of the relevant report data 130 is detailed in the displayed relevant report data 166 along with the selectable update element 208 and a cancel update element 210.
- update data 178 carrying the new, relevant, information from the text report 162 is sent to the FMS 118 or other sub-system of the aircraft.
- the update data 178 includes a request to change to runway 27L, the runway surface conditions (3/5/8) and runway surface wind conditions (100/08 KTS).
- the visual UI 200 for ATC text messages includes selectable report elements 214 allowing the pilot to view the text reports 162 including an ATIS text report, a METAR text report and a NOTAM text report.
- selectable report elements 214 allowing the pilot to view the text reports 162 including an ATIS text report, a METAR text report and a NOTAM text report.
- the ATIS selectable report element 214 is selected, the ATIS report of FIG. 4 is displayed including the highlighted parts 304 to show the changed information that is relevant to the flight context.
- the visual UI 200 includes displayed ATC text messages 156 from the flight crew and displayed ATC text messages 156 from the ATC 114.
- the displayed ATC text messages 155, 156 have been generated by the voice to text transcription application 102, as described previously.
- the runway is not being changed and matches that of the active flight plan data 158.
- the text report 162 from the ATIS report indicates that runway conditions and wind conditions have changed.
- the flight context data 170 identifies runway 09L as being of relevance to further processing based on the runway of the active flight plan data 158 and an analysis of the ATC text messages 154 that do not include any requests to change runway.
- the data extractor 104 filters the text report 162 to find information relating to the identified runway 09L.
- the data extractor 104 extracts relevant report data 130 including information on wind and surface conditions for runway 09L.
- the data comparator determines that this information is different from that entered into the flight plan included in the active flight plan data 158 and thus identifies the new flight plan information.
- the decoder 174 works with the display generator 108 to display the changed wind and runway information along with a selectable update element 208 that initiates submission of the changed information to the FMS 118. Further, the display generator 108 highlights the changed, relevant information in the displayed text report 302.
- FIG. 5 another exemplary visual UI 300 for text report is shown, which includes a displayed text report 302.
- the text report of FIG. 5 is a METAR report that has been transcribed by the voice to text transcription application 102.
- the METAR report of FIG. 5 may be analyzed by the processing system 160 to extract relevant information to a particular flight plan and incorporate that information into an ATC transcription window 206.
- an active flight plan, or change in active flight plan requested with the ATC 114, 114 including runway 28 may result in the visual range of 2400 feet being included in the ATC transcription window 206 along with a selectable update element 208 to submit the new information to the FMS 118.
- FIG. 6 A flow chart of an exemplary method 400 of adding relevant data to ATC transcription messages is provided in FIG. 6 .
- the method 400 is computer implemented by the processing system 506 of FIG. 5 or by the various modules of the processing system 160 of FIG. 1 .
- the method 400 includes step 410 of receiving voice reports 164 (e.g. ATIS, METAR and/or NOTAM reports) and ATC broadcast messages 124, 125 between the flight crew and the ATC.
- voice reports 164 e.g. ATIS, METAR and/or NOTAM reports
- ATC broadcast messages 124, 125 between the flight crew and the ATC.
- the processing system 160 transcribes the voice reports 164 and the ATC broadcast messages 124, 125 to provide one or more text reports 162 and ATC text messages 154.
- the transcription can take place using the same voice to text transcription application 102 or different voice to text transcription applications 102.
- relevant report data 130 is extracted from the text report 162 based on flight context.
- the flight context can include location of the aircraft 10 including flight phase and proximity to a take-off or landing runway.
- the flight context can be derived from an analysis of the ATC text messages 154, particularly any clearance information.
- the flight context can be derived from a currently active flight plan as described in active flight plan data 158.
- an ATC transcription window 206 is generated by the processing system 160, which includes the ATC text messages 154 directed to/from the ownship aircraft 10.
- the ATC transcription window 206 is augmented so as to include at least some of the relevant report data 130.
- the ATC transcription window may include a UI element, namely the selectable update element 208, that can be selected to update the flight plan (by submission to the FMS 118), in step 460, with the relevant report data 130 when selected by the flight crew.
- the text report 162 that has been transcribed by the processing system 160 is displayed in another window with highlighted parts 304 showing data items that have been determined to be relevant to flight context (e.g. based on an active flight plan and ATC conversation) and which are different from corresponding data items in the active flight plan.
- FIG. 5 depicts an exemplary embodiment of an aircraft system 500 suitable for implementing the system 100 of FIG. 1 .
- the illustrated aircraft system 500 (corresponding to system 100 of FIG. 1 ) includes, without limitation, a display device 502 (corresponding to display device 110 of FIG. 1 ), one or more user input devices 504, a processing system 506 (for implementing functions and methods described herein with respect to system 100), a communications system 510, a navigation system 512, a flight management system (FMS) 514 (corresponding to the FMS 118 of FIG. 1 ), one or more avionics systems 516, and a data storage element 518 (suitably configured to support operation of the system 500).
- FMS flight management system
- the display device 502 is realized as an electronic display capable of graphically displaying flight information or other data associated with operation of the aircraft 520 under control of the display system 508 and/or processing system 506.
- the display device 502 is coupled to the display system 508 and the processing system 506, wherein the processing system 506 and the display system 508 are cooperatively configured to display, render, or otherwise convey one or more graphical representations or images associated with operation of the aircraft 520 on the display device 502.
- the user input device 504 is coupled to the processing system 506, and the user input device 504 and the processing system 506 are cooperatively configured to allow a user (e.g., a pilot, co-pilot, or crew member) to interact with the display device 502 and/or other elements of the system 500, as described herein.
- a user e.g., a pilot, co-pilot, or crew member
- the user input device(s) 504 may be realized as a keypad, touchpad, keyboard, mouse, touch panel (or touchscreen), joystick, knob, line select key or another suitable device adapted to receive input from a user.
- the user input device 504 includes or is realized as an audio input device, such as a microphone, audio transducer, audio sensor, or the like, that is adapted to allow a user to provide audio input to the system 500 in a "hands free” manner without requiring the user to move his or her hands, eyes and/or head to interact with the system 500.
- an audio input device such as a microphone, audio transducer, audio sensor, or the like
- the processing system 506 generally represents the hardware, software, and/or firmware components configured to facilitate communications and/or interaction between the elements of the aircraft system 500 and perform additional tasks and/or functions to support the system 100 of FIG. 1 during operation of the aircraft system 500, as described herein.
- the processing system 506 may be implemented or realized with a general-purpose processor, a content addressable memory, a digital signal processor, an application specific integrated circuit, a field programmable gate array, any suitable programmable logic device, discrete gate or transistor logic, processing core, discrete hardware components, or any combination thereof, designed to perform the functions described herein.
- the processing system 506 may also be implemented as a combination of computing devices, e.g., a plurality of processing cores, a combination of a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other such configuration.
- the processing system 506 includes processing logic that may be configured to carry out the functions, techniques, and processing tasks associated with the operation of the aircraft system 500, as described herein.
- the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in firmware, in a software module executed by the processing system 506, or in any practical combination thereof.
- the processing system 506 includes or otherwise accesses a data storage element 518 (or memory), which may be realized as any sort of non-transitory short- or long-term storage media capable of storing programming instructions for execution by the processing system 506.
- the code or other computer-executable programming instructions when read and executed by the processing system 506, cause the processing system 506 to support or otherwise perform certain tasks, operations, and/or functions described herein in the context of the flight rules alerts.
- the data storage element 518 may be physically realized using RAM memory, ROM memory, flash memory, registers, a hard disk, or another suitable data storage medium known in the art or any suitable combination thereof.
- the display system 508 generally represents the hardware, software, and/or firmware components configured to control the display and/or rendering of one or more navigational maps and/or other displays pertaining to operation of the aircraft 520 and/or onboard systems 510, 512, 514, 516 on the display device 502.
- the display system 508 may access or include one or more databases suitably configured to support operations of the display system 508, such as, for example, a terrain database, an obstacle database, a navigational database, a geopolitical database, a terminal airspace database, a special use airspace database, or other information for rendering and/or displaying navigational maps and/or other content on the display device 502.
- the processing system 506 is coupled to the navigation system 512, which is configured to provide real-time navigational data and/or information regarding operation of the aircraft 520.
- the navigation system 512 may be realized as a global navigation satellite system (e.g., a global positioning system (GPS), a ground-based augmentation system (GBAS), a satellite-based augmentation system (SBAS), and/or the like), inertial reference system (IRS), or a radio-based navigation system (e.g., VHF omni-directional radio range (VOR) or long range aid to navigation (LORAN)), and may include one or more navigational radios or other sensors suitably configured to support operation of the navigation system 512, as will be appreciated in the art.
- GPS global positioning system
- GBAS ground-based augmentation system
- SBAS satellite-based augmentation system
- RATS inertial reference system
- radio-based navigation system e.g., VHF omni-directional radio range (VOR) or long range aid to navigation (LORAN)
- the navigation system 512 is capable of obtaining and/or determining the instantaneous position of the aircraft 520, that is, the current (or instantaneous) location of the aircraft 520 (e.g., the current latitude and longitude) and the current (or instantaneous) altitude or above ground level for the aircraft 520.
- the navigation system 512 is also capable of obtaining or otherwise determining the heading of the aircraft 520 (i.e., the direction the aircraft is traveling in relative to some reference).
- the processing system 506 is also coupled to the communications system 510, which is configured to support communications to and/or from the aircraft 520.
- the communications system 510 may support communications between the aircraft 520 and air traffic control or another suitable command center or ground location.
- the communications system 510 may be realized using a radio communication system and/or another suitable data link system.
- the processing system 506 is also coupled to the FMS 514, which is coupled to the navigation system 512, the communications system 510, and one or more additional avionics systems 516 to support navigation, flight planning, and other aircraft control functions in a conventional manner, as well as to provide real-time data and/or information regarding the operational status of the aircraft 520 to the processing system 506.
- FIG. 5 depicts a single avionics system 516, in practice, the aircraft system 500 and/or aircraft 520 will likely include numerous avionics systems for obtaining and/or providing real-time flight-related information that may be displayed on the display device 502 or otherwise provided to a user (e.g., a pilot, a co-pilot, or crew member).
- aircraft system 500 and/or aircraft 520 will likely include one or more of the following avionics systems suitably configured to support operation of the aircraft 520: a weather system, an air traffic management system, a radar system, a traffic avoidance system, an autopilot system, an autothrust system, a flight control system, hydraulics systems, pneumatics systems, environmental systems, aircraft systems, engine systems, trim systems, lighting systems, crew alerting systems, electronic checklist systems, an electronic flight bag and/or another suitable avionics system.
- avionics systems suitably configured to support operation of the aircraft 520: a weather system, an air traffic management system, a radar system, a traffic avoidance system, an autopilot system, an autothrust system, a flight control system, hydraulics systems, pneumatics systems, environmental systems, aircraft systems, engine systems, trim systems, lighting systems, crew alerting systems, electronic checklist systems, an electronic flight bag and/or another suitable avionics system.
- the processing system 506 may obtain information pertaining to the current location and/or altitude of the aircraft 520 and/or other operational information characterizing or otherwise describing the current operational context or status of the aircraft 520 from one or more of the onboard systems 508, 510, 512, 514, 516.
- FIG. 5 is a simplified representation of the aircraft system 500 for purposes of explanation and ease of description, and FIG. 5 is not intended to limit the application or scope of the subject matter described herein in any way.
- FIG. 10 shows the various elements of the system 500 being located onboard the aircraft 520 (e.g., in the cockpit), in practice, one or more of the elements of the system 500 may be located outside the aircraft 520 (e.g., on the ground as part of an air traffic control center or another command center) and communicatively coupled to the remaining elements of the aircraft system 500 (e.g., via a data link and/or communications system 510).
- the data storage element 518 may be located outside the aircraft 520 and communicatively coupled to the processing system 506 via a data link and/or communications system 510.
- practical embodiments of the aircraft system 500 and/or aircraft 520 will include numerous other devices and components for providing additional functions and features, as will be appreciated in the art.
- FIG. 5 shows a single display device 502, in practice, additional display devices may be present onboard the aircraft 520.
- features and/or functionality of processing system 506 described herein can be implemented by or otherwise integrated with the features and/or functionality provided by the FMS 514. In other words, some embodiments may integrate the processing system 506 with the FMS 514.
- various aspects of the subject matter described herein may be implemented by or at an electronic flight bag (EFB) or similar electronic device that is communicatively coupled to the processing system 506 and/or the FMS 514.
- EFB electronic flight bag
- embodiments of the subject matter described herein can be stored on, encoded on, or otherwise embodied by any suitable non-transitory computer-readable medium as computer-executable instructions or data stored thereon that, when executed (e.g., by a processing system), facilitate the processes described above.
- Coupled means that one element/node/feature is directly or indirectly joined to (or directly or indirectly communicates with) another element/node/feature, and not necessarily mechanically.
- drawings may depict one exemplary arrangement of elements directly connected to one another, additional intervening elements, devices, features, or components may be present in an embodiment of the depicted subject matter.
- certain terminology may also be used herein for the purpose of reference only, and thus are not intended to be limiting.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Life Sciences & Earth Sciences (AREA)
- Atmospheric Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Traffic Control Systems (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IN202211000754 | 2022-01-06 | ||
US17/655,285 US12020578B2 (en) | 2022-01-06 | 2022-03-17 | Systems and methods for adding relevant data to ATC transcription messages |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4210021A1 true EP4210021A1 (fr) | 2023-07-12 |
Family
ID=84537679
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP22214302.6A Pending EP4210021A1 (fr) | 2022-01-06 | 2022-12-16 | Systèmes et procédés pour ajouter des données pertinentes à des messages de transcription atc |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP4210021A1 (fr) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130346081A1 (en) * | 2012-06-11 | 2013-12-26 | Airbus (Sas) | Device for aiding communication in the aeronautical domain |
EP3474256A1 (fr) * | 2017-09-26 | 2019-04-24 | Honeywell International Inc. | Systèmes et procédés de présentation d'une visualisation de la chronologie intuitive via un écran de vol primaire (pfd) d'avionique |
US20210233411A1 (en) * | 2020-01-27 | 2021-07-29 | Honeywell International Inc. | Aircraft speech recognition systems and methods |
-
2022
- 2022-12-16 EP EP22214302.6A patent/EP4210021A1/fr active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130346081A1 (en) * | 2012-06-11 | 2013-12-26 | Airbus (Sas) | Device for aiding communication in the aeronautical domain |
EP3474256A1 (fr) * | 2017-09-26 | 2019-04-24 | Honeywell International Inc. | Systèmes et procédés de présentation d'une visualisation de la chronologie intuitive via un écran de vol primaire (pfd) d'avionique |
US20210233411A1 (en) * | 2020-01-27 | 2021-07-29 | Honeywell International Inc. | Aircraft speech recognition systems and methods |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2902991B1 (fr) | Procédés et systèmes pour l'utilisation de commandes vocales à bord d'un avion | |
US8838294B2 (en) | Methods and systems for briefing aircraft procedures | |
EP3321916A2 (fr) | Systèmes et procédés d'étiquetage contextuel de données sur un affichage de véhicule | |
US20210233411A1 (en) | Aircraft speech recognition systems and methods | |
CN108630019B (zh) | 用于渲染供atc条件许可指令使用的飞机驾驶舱显示的***和方法 | |
US11973570B2 (en) | Methods and systems for assisting radio frequency selection | |
US20220343897A1 (en) | Adaptive speech recognition methods and systems | |
US20230392954A1 (en) | Vehicle systems and related message prioritization methods | |
US11908330B2 (en) | Systems and methods for analyzing air traffic control messages and generating associated flight performance parameters | |
US12020578B2 (en) | Systems and methods for adding relevant data to ATC transcription messages | |
EP3889947A1 (fr) | Système et procédé d'assistance au pilote à l'aide de la lecture d'autorisation | |
US20230074014A1 (en) | Systems and methods for transcribing and analyzing broadcast messages in an aircraft | |
EP4095853B1 (fr) | Procédés et systèmes de reconnaissance vocale dynamique avec performance configurable par l'utilisateur | |
US11688390B2 (en) | Dynamic speech recognition methods and systems with user-configurable performance | |
EP4210021A1 (fr) | Systèmes et procédés pour ajouter des données pertinentes à des messages de transcription atc | |
EP3855428B1 (fr) | Systèmes et procédés de reconnaissance de la parole dans un aéronef | |
US20220406194A1 (en) | Contextual transcription augmentation methods and systems | |
EP4145421A1 (fr) | Systèmes et procédés de transcription et d'analyse de messages diffusés dans un aéronef | |
US20230368682A1 (en) | Transcription systems and related supplementation methods | |
US11955012B2 (en) | Transcription systems and message fusion methods | |
EP4152294A1 (fr) | Systèmes et procédés pour analyser des messages de contrôle de trafic aérien et analyser des paramètres de performance de vol associés | |
EP4210047A1 (fr) | Procédés et systèmes de reconnaissance vocale modifiables contextuels | |
EP4276794A1 (fr) | Systèmes de transcription et procédés de supplémentation associés | |
EP4290498A1 (fr) | Systèmes de véhicule et procédés de priorisation de messages associés | |
US20230215431A1 (en) | Contextual editable speech recognition methods and systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20231220 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |