US20190130783A1 - Vr emulator using galvanic vestibular stimulation devices - Google Patents
Vr emulator using galvanic vestibular stimulation devices Download PDFInfo
- Publication number
- US20190130783A1 US20190130783A1 US15/936,915 US201815936915A US2019130783A1 US 20190130783 A1 US20190130783 A1 US 20190130783A1 US 201815936915 A US201815936915 A US 201815936915A US 2019130783 A1 US2019130783 A1 US 2019130783A1
- Authority
- US
- United States
- Prior art keywords
- user
- aircraft
- flight
- slave
- emulator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/30—Simulation of view from aircraft
- G09B9/307—Simulation of view from aircraft by helmet-mounted projector or display
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/085—Special purpose teaching, e.g. alighting on water, aerial photography
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/12—Motion systems for aircraft simulators
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/24—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer including display or recording of simulated flight path
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
Definitions
- FIG. 1 is a side view of a virtual reality (“VR”) flight emulator according to this disclosure.
- VR virtual reality
- FIG. 2 is a diagram of an embodiment of a pilot control interface according to this disclosure.
- FIG. 3 is a schematic diagram of a virtual reality (“VR”) flight emulator system according to this disclosure.
- VR virtual reality
- FIG. 4 is a schematic diagram of a virtual reality (“VR”) flight teleport system according to this disclosure.
- VR virtual reality
- FIG. 5 is a flowchart of a method of operating a virtual reality (“VR”) flight emulator according to this disclosure.
- VR virtual reality
- FIG. 6 is a schematic diagram of a general-purpose processor (e.g. electronic controller or computer) system suitable for implementing the embodiments of this disclosure.
- a general-purpose processor e.g. electronic controller or computer
- Flight emulator 100 generally comprises a virtual reality (“VR”) system that simulates control, operation, and response of a vehicle, which may be a virtual vehicle or an actual remote vehicle (e.g., helicopter, tiltrotor, vertical take-off and landing (VTOL) aircraft, etc.), and allows a user 110 to provide inputs via a pilot control interface 102 to control, operate, and/or otherwise manipulate the vehicle.
- VR virtual reality
- VTOL vertical take-off and landing
- flight emulator 100 may also be applicable to other mobile equipment and vehicle types, including those found in automotive, construction, and marine applications.
- Flight emulator 100 comprises a pilot control interface 102 , a seat 104 connected to a base 106 (or other support system for seat 104 ), and a head-mounted display 108 worn by a user 110 that comprises a plurality of sensors, emitters, and/or reflectors 109 used to detect head movement of the user 110 .
- flight emulator 100 may also comprise an external display 112 that replicates images shown by the head-mounted display 108 .
- flight emulator 100 also comprises a plurality of galvanic vestibular stimulation (GVS) devices 117 .
- the GVS devices 117 may be attached to or carried by the head-mounted display 108 .
- the GVS devices 117 , the head-mounted display 108 , and/or the plurality of sensors, emitters, and/or reflectors 109 may be incorporated into a helmet worn by a user 110 .
- Pilot control interface 102 generally comprises physical and/or virtual flight control input hardware (e.g. flight input controls) configured to receive inputs and/or commands from user 110 to control operation of the vehicle and make the vehicle respond. Pilot control interface 102 also comprises vehicle and environmental virtual reality (“VR”) software 103 configured to respond to the inputs and/or commands from the user 110 and simulate operational response of the vehicle in response to the inputs and/or commands from the user 110 .
- the pilot control interface 102 is also configured to simulate operation of the vehicle through operation of the GVS devices 117 and selectively displaying and/or adjusting graphical imagery shown to a user 110 on the head-mounted display 108 .
- the pilot control interface 102 may be configured for a vehicle controlled by the flight emulator 100 or may comprise a generic system comprising physical and/or virtual hardware components or input controls common to all aircraft or other vehicles. Additionally, as will be discussed later herein, the pilot control interface 102 may also be configured to control the vehicle associated with the flight emulator 100 as well as a remote or virtual slave vehicle upon virtually teleporting into the slave vehicle.
- the GVS devices 117 generally comprise small electrodes that are disposed behind each of a left and right ear of a user 110 and on each of a front and back of the head of user 110 .
- GVS devices 117 operate by passing a current between GVS devices 117 to trick the user 110 to feel a motion, orientation, and/or force applied to him/her.
- a current may be passed between GVS devices 117 to make a user 110 think he/she is experiencing a turning motion, tilting motion, banked orientation, and/or acceleration and deceleration of an aircraft.
- GVS devices 117 By utilizing GVS devices 117 , passing current between the GVS devices 117 tricks a user to feel the motion of, orientation of, and/or forces acting on a aircraft, such as changes in pitch, roll, and yaw. Accordingly, when a user 110 has virtually teleported into a slave vehicle using the flight emulator 100 , the motion of, orientation of, and/or forces acting on the slave vehicle as a result of operational parameters (e.g. speed, turning), environmental factors (e.g. wind gusts), and/or inputs of the user 110 through the pilot control interface 102 are emulated on or replicated to the user 110 through selective operation of the GVS devices 117 .
- operational parameters e.g. speed, turning
- environmental factors e.g. wind gusts
- the head-mounted display 108 comprises a virtual reality (“VR”) display that is worn on the head of a user 110 .
- the head-mounted display 108 displays imagery (e.g., simulated environment) visible to the user 110 that simulates the imagery of the environment.
- the head-mounted display 108 is also configured to display portions of the pilot control interface 102 when the pilot control interface 102 is in the line of sight of user 110 .
- the head-mounted display 108 comprises a plurality of sensors, emitters, and/or reflectors 109 used to detect and track head movement of the user 110 .
- the sensors, emitters, and/or reflectors 109 and the head-mounted display 108 are each coupled to components and/or software 103 of the pilot control interface 102 , such that the imagery displayed to a user 110 on the head-mounted display 108 changes in response to head movement of the user 110 .
- the head-mounted display 108 may not display imagery.
- the head-mounted display 108 may show a virtual representation of a state of a remote fleet of vehicles.
- flight emulator 100 provides a user 110 the capability to operate a remote or virtual slave vehicle while in the flight emulator 100 .
- the flight emulator 100 may be a standalone stationary flight emulator 100 .
- the flight emulator 100 may be installed in a vehicle, such as an aircraft.
- inputs provided by user 110 through the pilot control interface 102 cause the slave vehicle to respond and further causes operation of the GVS devices 117 to simulate the effect of the motion of, orientation of, and/or forces acting on the slave vehicle on the user 110 .
- the head-mounted display 108 may continuously adjust the imagery displayed to the user 110 to reflect what an operator of the slave vehicle would see in response to the motion detected by the sensors, emitters, and/or reflectors 109 .
- the imagery displayed to the user 110 may be passed between GVS devices 117 to cause the user 110 to feel the bank of the turn as would be experienced by the aircraft, while also adjusting the imagery displayed to the user 110 on the head-mounted display 108 to simulate the orientation of the aircraft.
- current may be passed between GVS devices 117 on the front and rear of the head of user 110 to cause the user 110 to feel the pitch of the turn as experienced by the aircraft.
- pilot control interface 102 comprises an instrument panel 118 comprising a plurality of touchscreen displays 120 , warning and/or notification lights 122 , mechanical and/or electrical gauges 124 , mechanical and/or electrical switches 126 , and/or other instrumentation 128 necessary to communicate and/or adjust operational parameters of the virtual or vehicle carrying the flight emulator 100 in addition to a remote vehicle.
- the pilot control interface 102 may comprise a virtual instrument panel 118 of a remote vehicle.
- pilot control interface 102 may comprise a blank virtual display or multiple displays that populate with the appropriate gauges when the head mounted display 108 is worn by a user 110 .
- Pilot control interface 102 may also comprise cyclic control 130 , collective control 132 , and/or pedals 134 for selectively manipulating operation of the vehicle.
- pilot control interface 102 may also comprise co-pilot cyclic control 130 ′, co-pilot collective control 132 ′, and/or co-pilot pedals 134 ′.
- one or more touchscreen displays 120 may display a list of all vehicles connected through a network. Additionally, one or more touchscreen displays 120 may selectively alter their respective displays depending on the type of vehicle being controlled. As such, a single pilot control interface 102 or flight emulator 100 may be configured to emulate multiple vehicles.
- Instrument panel 118 may generally comprise any combination of virtual, physical, simulated, and/or actual components 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ that allow a user 110 control over the present vehicle and a virtual or remote vehicle.
- instrument panel 118 may comprise a mock instrument panel that allows physical interaction by a user 110 with one or more of the physical components 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ that would be present in an actual vehicle.
- the head-mounted display 108 may be at least partially augmented or transparent to allow a user 110 to see and interact with one or more components 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′.
- one or more of the components 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ on the instrument panel 118 may comprise a non-functioning “dummy” component.
- the instrument panel 118 may comprise a plurality of sensors, emitters, reflectors, and/or cameras used to anticipate or detect movement (e.g., arm motion, head motion) of a user 110 towards a component 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ to anticipate a command in order to reduce latency in the pilot control interface 102 .
- one or more touchscreen displays 120 may change its display depending on whether the flight emulator 100 is being used to control a master vehicle or a remote vehicle.
- flight emulator system 150 comprises a plurality of flight emulators 100 selectively connected in communication through network 152 , which may be wired, wireless, or any combination thereof. It will be appreciated that flight emulator system 150 may comprise any number of flight emulators 100 . Additionally, flight emulator system 150 also comprises a master flight emulator 154 . Master flight emulator 154 comprises a flight emulator 100 selectively connected in communication with each of the plurality of slave flight emulators 100 a 1-n through network 152 .
- the flight emulator 100 of the master flight emulator 154 comprises the capability to selectively virtually teleport user 110 of the master flight emulator 154 into a slave flight emulator 100 a 1-n to assume control of the pilot control interface 102 of any of the slave flight emulators 100 a 1-n connected through the network 152 .
- this capability may be encoded in the software 103 and/or made possible through components of the pilot control interface 102 of the master flight emulator 154 .
- this capability may be present in each of the flight emulators 100 but overridden or disabled in each of the slave flight emulators 100 a 1-n by the master flight emulator 154 .
- the flight emulators 100 may be flown in the same simulated environment. This allows for multi-vehicle training that may require operating a mix of various aircraft simultaneously to build teaming protocol and flight patterns.
- a user 110 e.g., instructor
- the master flight emulator 154 may not be simulating a vehicle actively participating in a training exercise on the network 152 .
- an instructor in the master flight emulator 154 may virtually teleport into a slave flight emulator 100 a 1-n without taking over control of the pilot control interface 102 .
- a passive teleport This is referred to as a passive teleport.
- the instructor in the master flight emulator 154 perceives the simulated environment of the slave flight emulator 100 a 1-n that the instructor in the master flight emulator 154 has virtually teleported into through the head-mounted display 108 associated with the master flight emulator 154 .
- the head-mounted display 108 worn by the instructor in the master flight emulator 154 may replicate imagery seen by the user in the slave flight emulator 100 a 1-n .
- the head-mounted display 108 worn by the instructor in the master flight emulator 154 may operate independently and track head movement of the instructor via sensors, emitters, and/or reflectors 109 .
- any motion of, orientation of, and/or forces acting on the master aircraft controlled by the master flight emulator 154 are minimized or preferably cancelled via the GVS devices 117 .
- the GVS devices 117 may be further operated such that the instructor in the master flight emulator 154 experiences the motion of, orientation of, and/or forces acting on the vehicle controlled by the slave flight emulator 100 a 1-n into which the instructor in the master flight emulator 154 has virtually teleported.
- the GVS devices 117 of the master flight emulator 154 may be simply operated to impart motion of, orientation of, and/or forces acting on the vehicle controlled by the slave flight emulator 100 a 1-n . In both scenarios, this allows the instructor in the master flight emulator 154 to “feel” only the motion of, orientation of, and/or forces acting on the slave flight emulator 100 a 1-n as if the instructor were in the slave flight emulator 100 a 1-n .
- the instructor may receive feedback through the pilot control interface 102 and instrument panel 118 of the master flight emulator 154 in response to a user 110 in the slave flight emulator 100 a 1-n providing inputs through the pilot control interface 102 of the slave flight emulator 100 a 1-n .
- the instructor preferably feels the amount of force or movement a user 110 may be using to manipulate the cyclic control 130 , collective control 132 , and pedals 134 of the slave flight emulator 100 a 1-n .
- the instructor further may be able to observe readings on the gauges 124 and position of switches 126 of the slave flight emulator 100 a 1-n through emulation of the pilot control interface 102 of the slave flight emulator 100 a 1-n through the pilot control interface 102 of the master flight emulator 154 . This allows the instructor to fully observe operation of the slave flight emulator 100 a 1-n as if the instructor were actually in the slave flight emulator 100 a 1-n .
- the instructor in the master flight emulator 154 may selectively assume control of the pilot control interface 102 of a slave flight emulator 100 a 1-n .
- This is referred to as an active teleport.
- the instructor in the master flight emulator 154 may provide inputs through the pilot control interface 102 of the master flight emulator 154 that are imparted into the slave flight emulator 100 a 1-n .
- the pilot control interface 102 of the slave flight emulator 100 a 1-n may be locked out, such that a user 110 may not provide inputs.
- the components of the pilot control interface 102 of the slave flight emulator 100 a 1-n may be driven by and move in accordance with inputs made by the instructor through the pilot control interface 102 of the master flight emulator 154 .
- This allows the instructor in the master flight emulator 154 to operate the slave flight emulator 100 a 1-n , while the user 110 in the slave flight emulator 100 a 1-n receives real-time feedback and “feels” the response of the simulated vehicle associated with the slave flight emulator 100 a 1-n in response to the inputs made by the instructor in the master flight emulator 154 .
- the head-mounted display 108 worn by the instructor in the master flight emulator 154 replicates imagery from the slave flight emulator 100 a 1-n .
- the instructor in the master flight emulator 154 may selectively observe gauges, instruments, and controls in the slave flight emulator 100 a 1-n that may not be replicated by the pilot control interface 102 of the master flight emulator 154 .
- the head-mounted display 108 worn by the instructor in the master flight emulator 154 is used to track head movement of the instructor via the sensors, emitters, and/or reflectors 109 and the imagery displayed to the instructor on the head-mounted display 108 is responsively adjusted, thereby allowing the instructor to selectively observe the environment of the slave flight emulator 100 a 1-n .
- feedback through the pilot control interface 102 , the head-mounted display 108 , and the GVS devices 117 of the master flight emulator 154 provides real-time emulation capability that allows the instructor to fully observe and control operation of the slave flight emulator 100 a 1-n n when the instructor is virtually teleported as if the instructor of the master flight emulator 154 were actually in the slave flight emulator 100 a 1-n .
- inputs made by the instructor that impart motion, orientation, and/or forces into the vehicle associated with the slave flight emulator 100 a 1-n are emulated or replicated to the user 110 of the slave flight emulator 100 a 1-n through selective operation of the GVS devices 117 of the slave flight emulator 100 a 1-n .
- This also allows the user 110 of the slave flight emulator 100 a 1-n to associate inputs made by the instructor and associated response of the simulated vehicle.
- the instructor may virtually teleport into the slave flight emulator 100 a 1-n of the user 110 and provide the correct cyclic control 130 input necessary to make a smooth, wide-radius turn or correct collective control 132 input necessary for a smooth takeoff or landing.
- this real-time feedback to the user 110 of the slave flight emulator 100 a 1-n may help a user 110 more effectively learn the correct inputs to properly operate a vehicle associated with a slave flight emulator 100 a 1-n .
- Flight teleport system 150 is also capable of utilizing the control loading system (CLS) of a flight emulator 100 to selectively record and playback of a training exercise or flight mission, thereby allowing a user 110 to receive feedback through the pilot control interface 102 and “feel” the amount and direction of control inputs necessary to expertly pilot the simulated vehicle without having a training instructor onboard. Additionally, playback of a training exercise or flight mission also imparts simulated motion, orientation, and/or forces to a user 110 by selectively operating the GVS devices 117 of each flight emulator 100 , thereby allowing a user 110 to associate inputs with various responses of the simulated vehicle while also providing visual imagery through the head-mounted display 108 of each flight emulator 100 .
- CCS control loading system
- Flight teleport system 200 comprises a master aircraft 202 and at least one remote aircraft 204 connected in communication through a wireless communication network 208 .
- Flight teleport system 200 may comprise any number of remote aircraft 204 a 1-n .
- Master aircraft 202 comprises a flight emulator 100 that may be in place of a pilot's seat, a co-pilot's seat, or in another location of the master aircraft 202
- each of the plurality of remote aircraft 204 comprise a flight control system 206 as known in the art for operating the associated remote aircraft 204 .
- some or all of the remote aircraft 204 may also comprise a flight emulator 100 .
- Master aircraft 202 may be substantially similar to any of the remote aircraft 204 .
- flight emulator 100 of the master aircraft 202 comprises the ability to selectively virtually teleport user 110 into a remote aircraft 204 to assume control of any of the so-called remote “slave” aircraft 204 on the network 208 via the flight emulator 100 in the master aircraft 202 .
- This effectively allows a user 110 of the master aircraft 202 which may be a pilot, co-pilot, or other crew member of the master aircraft 202 , to virtually teleport into and assume control of a fleet of remote aircraft 204 connected through the network 208 .
- this capability may be encoded in the software 103 and/or made possible through components of the pilot control interface 102 of the flight emulator 100 of the master aircraft 202 .
- the flight control system 206 of any of the remote aircraft 204 may comprise components or software necessary to allow the user 110 of the flight emulator 100 of the master aircraft 202 to assume control of its flight control system 206 and/or any of its other control systems (e.g., landing gear, weaponry).
- the user 110 of the flight emulator 100 of the master aircraft 202 is virtually teleported into a remote aircraft 204
- user 110 of the flight emulator 100 assumes at least partial control of the remote aircraft 204 through the network 208 .
- the flight control system 206 of the remote aircraft 204 may be locked out or disabled, such that a pilot of the remote aircraft 204 may not provide inputs to alter operation of the remote aircraft 204 .
- GVS devices 117 of the flight emulator 100 in the master aircraft 202 may be selectively operated to minimize or preferably cancel the motion of, orientation of, and/or forces acting on the master aircraft 202 .
- the GVS devices 117 may be further operated to impart the simulated motion of, orientation of, and/or forces acting on the remote aircraft 204 . This allows the pilot in the master aircraft 202 to “feel” the motion of, orientation of, and/or forces acting on the slave aircraft 204 .
- a pilot of the master aircraft 202 may place the master aircraft 202 on a predetermined and/or automated flight plan such as a straight flight path, a large radius turn, or a flat hover, then virtually teleport into a remote aircraft 204 .
- a predetermined and/or automated flight plan such as a straight flight path, a large radius turn, or a flat hover
- the motion of, orientation of, and/or forces acting on the master aircraft 202 are minimized or preferably cancelled by passing a current between selective GVS devices 117 of the flight emulator 100 in the master aircraft 202 , and the GVS devices 117 may be operated such that the user 110 experiences the motion of, orientation of, and/or forces acting on the remote aircraft 204 .
- This may help a pilot in the master aircraft 202 effectively operate the remote aircraft 204 or diagnose aircraft performance issues affecting the flight of the remote aircraft 204 .
- the user 110 in the master aircraft 202 also receives feedback through the pilot control interface 102 and instrument panel 118 of the flight emulator 100 of the master aircraft 202 .
- the user 110 may receive feedback via the cyclic control 130 , collective control 132 , and pedals 134 from the flight control system 206 from the remote aircraft 204 .
- User 110 further may be able to observe readings on the gauges 124 and position of switches 126 of the flight emulator 100 through emulation of the flight control system 206 of the remote aircraft 204 on the pilot control interface 102 .
- the head-mounted display 108 worn by the user 110 in the master aircraft 202 may replicate imagery from the remote aircraft 204 .
- the user 110 in the master aircraft 202 may selectively observe gauges, instruments, and controls in the remote aircraft 204 that may not be replicated by the pilot control interface 102 when virtually teleported.
- the head-mounted display 108 worn by the user 110 in the master aircraft 202 is used to detect head movement of the pilot via the sensors, emitters, and/or reflectors 109 and responsively adjusts the imagery displayed to the user 110 on the head-mounted display 108 , thereby allowing the user 110 in the master aircraft 202 to selectively observe the environment of the remote aircraft 204 .
- feedback through the pilot control interface 102 , the head-mounted display 108 , and the GVS devices 117 of the flight emulator 100 in the master aircraft 202 provides real-time emulation capability that allows the pilot of the master aircraft 202 to fully observe and control operation of the remote aircraft 204 when virtually teleported as if the pilot of the master aircraft 202 were actually in the remote aircraft 204 .
- a piloted master aircraft 202 and a plurality of unmanned remote aircraft 204 may be flown in an attack and/or utility configuration simultaneously on a mission.
- a pilot or co-pilot of the master aircraft 202 may place the master aircraft 202 on a predetermined and/or automated flight plan, while virtually teleporting into an attack remote aircraft 204 .
- the pilot or co-pilot of the master aircraft 202 is able to see through the head-mounted display 108 the environment surrounding the remote aircraft 204 .
- the pilot of the master aircraft 202 may then assume control of the flight control system 206 of the attack remote aircraft 204 through the flight emulator 100 in the master aircraft 202 to perform a strike and/or other operation, then regain command of his/her physical master aircraft 202 .
- the pilot or co-pilot of the master aircraft 202 may then virtually teleport into another remote aircraft 204 to perform specific operations before ending use of the flight emulator 100 in the master aircraft 202 and returning to operating the master aircraft 202 .
- latency of communication e.g., control signals, feedback, etc.
- the network 208 may allow communication and/or control without the use of satellites.
- the network 208 may use encrypted control inputs between a master aircraft 202 and a remote aircraft 204 to prevent the pilot control interface 102 and/or flight control system 206 of an aircraft 202 or remote aircraft 204 from being hacked or overridden.
- the virtually teleporting capabilities of the flight emulator 100 in the master aircraft 202 may be present in one or more of the remote aircraft 204 connected through network 208 , so that multiple users 110 may virtually teleport into any of the other remote aircraft 204 on the network 208 .
- the network 208 may allow multiple teleporting users 110 simultaneously. In some embodiments, this may allow a user 110 of one remote aircraft 204 to overtake the controls of another remote aircraft 204 having an incapacitated pilot.
- providing the virtually teleporting capabilities of the flight emulator 100 provides benefits such as increased situational awareness, increased accuracy, and/or quicker decision making during flight missions.
- Method 400 begins at block 402 by providing a flight emulator 100 comprising a pilot control interface 102 and a plurality of GVS devices 117 .
- the flight emulator 100 may be a master flight emulator 154 or a master aircraft 202 .
- Method 400 continues at block 404 by virtually teleporting a user 110 of the flight emulator 100 into a slave vehicle.
- the slave vehicle may be a virtual vehicle controlled by a slave flight emulator 100 a 1-n or a remote aircraft 204 a 1-n connected via a network 152 , 208 .
- flight emulator 100 may be selectively configured to emulate the motion of, orientation of, and/or forces acting on the slave vehicle via selective operation of the GVS devices 117 in the flight emulator 100 to simulate the motion of, orientation of, and/or forces acting on the slave vehicle. This is further accomplished via adjustment, movement, selection, or change of display of components 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ of the pilot control interface 102 . Further, it will be appreciated that this is accomplished via continuously replicating visual imagery through the head-mounted display 108 depending on the head movement and/or orientation of the user 100 of the flight emulator 100 .
- Method 400 concludes at block 406 by the user 110 of the flight emulator 100 providing an input to operate the slave vehicle. This may be accomplished via the user 110 in a master flight emulator 154 or a pilot in a master aircraft 202 providing an input via a component 118 , 120 , 122 , 124 , 126 , 128 , 130 , 130 ′, 132 , 132 ′, 134 , 134 ′ of the pilot control interface 102 that causes a slave flight emulator 100 a 1-n , or the flight control system 206 of a remote aircraft 204 to adjust, alter, change, or select a mode or parameter of operation of the remote aircraft 204 . Further, inputs that result in changes in motion of, orientation of, and/or forces acting on the slave aircraft are imparted to the user 110 of the flight emulator 100 via the GVS devices 117 in the flight emulator 100 .
- FIG. 6 a schematic diagram of a general-purpose processor (e.g. electronic controller or computer) system 500 suitable for implementing the embodiments of this disclosure is shown.
- System 500 that includes a processing component 510 suitable for implementing one or more embodiments disclosed herein.
- each of the above-described flight emulators 100 , pilot control interfaces 102 , instrument panels 118 , flight emulator systems 150 , and/or flight teleport systems 200 may comprise one or more systems 500 .
- the system 500 might include network connectivity devices 520 , random access memory (RAM) 530 , read only memory (ROM) 540 , secondary storage 550 , and input/output (I/O) devices 560 . In some cases, some of these components may not be present or may be combined in various combinations with one another or with other components not shown. These components might be located in a single physical entity or in more than one physical entity. Any actions described herein as being taken by the processor 510 might be taken by the processor 510 alone or by the processor 510 in conjunction with one or more components shown or not shown in the system 500 . It will be appreciated that the data described herein can be stored in memory and/or in one or more databases.
- the processor 510 executes instructions, codes, computer programs, or scripts that it might access from the network connectivity devices 520 , RAM 530 , ROM 540 , or secondary storage 550 (which might include various disk-based systems such as hard disk, floppy disk, optical disk, or other drive). While only one processor 510 is shown, multiple processors may be present. Thus, while instructions may be discussed as being executed by processor 510 , the instructions may be executed simultaneously, serially, or otherwise by one or multiple processors 510 .
- the processor 510 may be implemented as one or more CPU chips and/or application specific integrated chips (ASICs).
- ASICs application specific integrated chips
- the network connectivity devices 520 may take the form of modems, modem banks, Ethernet devices, universal serial bus (USB) interface devices, serial interfaces, token ring devices, fiber distributed data interface (FDDI) devices, wireless local area network (WLAN) devices, radio transceiver devices such as code division multiple access (CDMA) devices, global system for mobile communications (GSM) radio transceiver devices, worldwide interoperability for microwave access (WiMAX) devices, and/or other well-known devices for connecting to networks.
- These network connectivity devices 520 may enable the processor 510 to communicate with the Internet or one or more telecommunications networks or other networks from which the processor 510 might receive information or to which the processor 510 might output information.
- the network connectivity devices 520 might also include one or more transceiver components 525 capable of transmitting and/or receiving data wirelessly in the form of electromagnetic waves, such as radio frequency signals or microwave frequency signals. Alternatively, the data may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media such as optical fiber, or in other media.
- the transceiver component 525 might include separate receiving and transmitting units or a single transceiver.
- Information transmitted or received by the transceiver 525 may include data that has been processed by the processor 510 or instructions that are to be executed by processor 510 . Such information may be received from and outputted to a network in the form, for example, of a computer data baseband signal or signal embodied in a carrier wave.
- the data may be ordered according to different sequences as may be desirable for either processing or generating the data or transmitting or receiving the data.
- the baseband signal, the signal embedded in the carrier wave, or other types of signals currently used or hereafter developed may be referred to as the transmission medium and may be generated according to several methods well known to one skilled in the art.
- the RAM 530 might be used to store volatile data and perhaps to store instructions that are executed by the processor 510 .
- the ROM 540 is a non-volatile memory device that typically has a smaller memory capacity than the memory capacity of the secondary storage 550 .
- ROM 540 might be used to store instructions and perhaps data that are read during execution of the instructions. Access to both RAM 530 and ROM 540 is typically faster than to secondary storage 550 .
- the secondary storage 550 is typically comprised of one or more disk drives or tape drives and might be used for non-volatile storage of data or as an over-flow data storage device if RAM 530 is not large enough to hold all working data. Secondary storage 550 may be used to store programs or instructions that are loaded into RAM 530 when such programs are selected for execution or information is needed.
- the I/O devices 560 may include liquid crystal displays (LCDs), touchscreen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, printers, video monitors, transducers, sensors, or other well-known input or output devices.
- the transceiver 525 might be considered to be a component of the I/O devices 560 instead of or in addition to being a component of the network connectivity devices 520 .
- Some or all of the I/O devices 560 may be substantially similar to various components disclosed herein and/or may be components of the above-described flight emulators 100 , pilot control interfaces 102 , instrument panels 118 , flight emulator systems 150 , and/or flight teleport systems 200 .
- R 1 a numerical range with a lower limit, R 1 , and an upper limit, R u , any number falling within the range is specifically disclosed.
- R R 1 +k*(R u ⁇ R 1 ), wherein k is a variable ranging from 1 percent to 100 percent with a 1 percent increment, i.e., k is 1 percent, 2 percent, 3 percent, 4 percent, 5 percent, . . . 50 percent, 51 percent, 52 percent, . . . , 95 percent, 96 percent, 95 percent, 98 percent, 99 percent, or 100 percent.
- any numerical range defined by two R numbers as defined in the above is also specifically disclosed.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
- Selective Calling Equipment (AREA)
- Toys (AREA)
Abstract
Systems and methods include providing a virtual reality (“VR”) flight emulator that simulates control, operation, and response of a vehicle. The flight emulator includes a control interface and a head-mounted display worn by a user. The user of the flight emulator can virtually teleport into a slave virtual or remote aircraft in order to assume control of the slave virtual or remote aircraft. Motion of, orientation of, and/or forces acting on the slave virtual or remote aircraft are imparted to a user through a plurality of galvanic vestibular stimulation (GVS) devices when the user of the flight emulator is virtually teleported into the slave virtual or remote aircraft. The flight emulator can be connected through a communication network to a plurality of other flight emulators. The flight emulator can also be installed in an aircraft and connected through the communication network to a plurality of other remote aircraft.
Description
- This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 62/581,012 filed on Nov. 2, 2017, by Jeffrey Paul Nissen, et al., titled “VR Emulator,” the disclosure of which is hereby incorporated by reference in its entirety.
- Not applicable.
- Military, commercial, and other operations generally involve a fleet of different vehicles. Operators of many of these vehicles often require extensive training in order to ensure safe and effective operation of their respective vehicles. However, training pilots of aircraft presents unique training challenges and safety concerns not traditionally present in training operators of ground vehicles, since physical flight of the aircraft requires the aircraft to be airborne. Additionally, some of these vehicles may include the capability of being operated and/or controlled remotely. However, operating and/or controlling a vehicle remotely presents many other challenges, including signal, communication, and/or control latency due to large distances between a control center and a remote vehicle, delays in receiving accurate and/or real-time feedback, and/or other deficiencies that materially affect critical decision making during an operation.
-
FIG. 1 is a side view of a virtual reality (“VR”) flight emulator according to this disclosure. -
FIG. 2 is a diagram of an embodiment of a pilot control interface according to this disclosure. -
FIG. 3 is a schematic diagram of a virtual reality (“VR”) flight emulator system according to this disclosure. -
FIG. 4 is a schematic diagram of a virtual reality (“VR”) flight teleport system according to this disclosure. -
FIG. 5 is a flowchart of a method of operating a virtual reality (“VR”) flight emulator according to this disclosure. -
FIG. 6 is a schematic diagram of a general-purpose processor (e.g. electronic controller or computer) system suitable for implementing the embodiments of this disclosure. - In this disclosure, reference may be made to the spatial relationships between various components and to the spatial orientation of various aspects of components as the devices are depicted in the attached drawings. However, as will be recognized by those skilled in the art after a complete reading of this disclosure, the devices, members, apparatuses, etc. described herein may be positioned in any desired orientation. Thus, the use of terms such as “above,” “below,” “upper,” “lower,” or other like terms to describe a spatial relationship between various components or to describe the spatial orientation of aspects of such components should be understood to describe a relative relationship between the components or a spatial orientation of aspects of such components, respectively, as the device described herein may be oriented in any desired direction.
- Referring now to
FIG. 1 , a simplified side view of a virtual reality (“VR”)flight emulator 100 is shown according to this disclosure.Flight emulator 100 generally comprises a virtual reality (“VR”) system that simulates control, operation, and response of a vehicle, which may be a virtual vehicle or an actual remote vehicle (e.g., helicopter, tiltrotor, vertical take-off and landing (VTOL) aircraft, etc.), and allows auser 110 to provide inputs via apilot control interface 102 to control, operate, and/or otherwise manipulate the vehicle. Further, it will be appreciated that whileflight emulator 100 is described in the context of an aircraft,flight emulator 100 may also be applicable to other mobile equipment and vehicle types, including those found in automotive, construction, and marine applications.Flight emulator 100 comprises apilot control interface 102, aseat 104 connected to a base 106 (or other support system for seat 104), and a head-mounteddisplay 108 worn by auser 110 that comprises a plurality of sensors, emitters, and/orreflectors 109 used to detect head movement of theuser 110. In some embodiments,flight emulator 100 may also comprise anexternal display 112 that replicates images shown by the head-mounteddisplay 108. Additionally,flight emulator 100 also comprises a plurality of galvanic vestibular stimulation (GVS)devices 117. In some embodiments, theGVS devices 117 may be attached to or carried by the head-mounteddisplay 108. However, in other embodiments, theGVS devices 117, the head-mounteddisplay 108, and/or the plurality of sensors, emitters, and/orreflectors 109 may be incorporated into a helmet worn by auser 110. -
Pilot control interface 102 generally comprises physical and/or virtual flight control input hardware (e.g. flight input controls) configured to receive inputs and/or commands fromuser 110 to control operation of the vehicle and make the vehicle respond.Pilot control interface 102 also comprises vehicle and environmental virtual reality (“VR”)software 103 configured to respond to the inputs and/or commands from theuser 110 and simulate operational response of the vehicle in response to the inputs and/or commands from theuser 110. Thepilot control interface 102 is also configured to simulate operation of the vehicle through operation of theGVS devices 117 and selectively displaying and/or adjusting graphical imagery shown to auser 110 on the head-mounteddisplay 108. Thepilot control interface 102 may be configured for a vehicle controlled by theflight emulator 100 or may comprise a generic system comprising physical and/or virtual hardware components or input controls common to all aircraft or other vehicles. Additionally, as will be discussed later herein, thepilot control interface 102 may also be configured to control the vehicle associated with theflight emulator 100 as well as a remote or virtual slave vehicle upon virtually teleporting into the slave vehicle. - The
GVS devices 117 generally comprise small electrodes that are disposed behind each of a left and right ear of auser 110 and on each of a front and back of the head ofuser 110.GVS devices 117 operate by passing a current betweenGVS devices 117 to trick theuser 110 to feel a motion, orientation, and/or force applied to him/her. For example, a current may be passed betweenGVS devices 117 to make auser 110 think he/she is experiencing a turning motion, tilting motion, banked orientation, and/or acceleration and deceleration of an aircraft. By utilizingGVS devices 117, passing current between theGVS devices 117 tricks a user to feel the motion of, orientation of, and/or forces acting on a aircraft, such as changes in pitch, roll, and yaw. Accordingly, when auser 110 has virtually teleported into a slave vehicle using theflight emulator 100, the motion of, orientation of, and/or forces acting on the slave vehicle as a result of operational parameters (e.g. speed, turning), environmental factors (e.g. wind gusts), and/or inputs of theuser 110 through thepilot control interface 102 are emulated on or replicated to theuser 110 through selective operation of theGVS devices 117. - The head-mounted
display 108 comprises a virtual reality (“VR”) display that is worn on the head of auser 110. The head-mounteddisplay 108 displays imagery (e.g., simulated environment) visible to theuser 110 that simulates the imagery of the environment. The head-mounteddisplay 108 is also configured to display portions of thepilot control interface 102 when thepilot control interface 102 is in the line of sight ofuser 110. The head-mounteddisplay 108 comprises a plurality of sensors, emitters, and/orreflectors 109 used to detect and track head movement of theuser 110. The sensors, emitters, and/orreflectors 109 and the head-mounteddisplay 108 are each coupled to components and/orsoftware 103 of thepilot control interface 102, such that the imagery displayed to auser 110 on the head-mounteddisplay 108 changes in response to head movement of theuser 110. For example, whenuser 110 looks forward,user 110 may see through the windshield of a cockpit of a vehicle and portions of thepilot control interface 102, and whenuser 110 looks to the left,user 110 may see a co-pilot or leftward portions of thepilot control interface 102. Additionally, whenuser 110 is not virtually teleported into a remote vehicle, the head-mounteddisplay 108 may not display imagery. Furthermore, whenuser 110 is not virtually teleported into a remote vehicle, the head-mounteddisplay 108 may show a virtual representation of a state of a remote fleet of vehicles. - In operation,
flight emulator 100 provides auser 110 the capability to operate a remote or virtual slave vehicle while in theflight emulator 100. In some embodiments, theflight emulator 100 may be a standalonestationary flight emulator 100. However, in other embodiments, theflight emulator 100 may be installed in a vehicle, such as an aircraft. Whenuser 110 is virtually teleported into the slave vehicle, inputs provided byuser 110 through thepilot control interface 102 cause the slave vehicle to respond and further causes operation of theGVS devices 117 to simulate the effect of the motion of, orientation of, and/or forces acting on the slave vehicle on theuser 110. Additionally, the head-mounteddisplay 108 may continuously adjust the imagery displayed to theuser 110 to reflect what an operator of the slave vehicle would see in response to the motion detected by the sensors, emitters, and/orreflectors 109. For example, ifuser 110 provides an input to turn an aircraft to the left, current may be passed betweenGVS devices 117 to cause theuser 110 to feel the bank of the turn as would be experienced by the aircraft, while also adjusting the imagery displayed to theuser 110 on the head-mounteddisplay 108 to simulate the orientation of the aircraft. In a further example, ifuser 110 pitches the aircraft forward, current may be passed betweenGVS devices 117 on the front and rear of the head ofuser 110 to cause theuser 110 to feel the pitch of the turn as experienced by the aircraft. - Referring now to
FIG. 2 , a diagram of apilot control interface 102 is shown according to this disclosure. In the embodiment shown,pilot control interface 102 comprises aninstrument panel 118 comprising a plurality oftouchscreen displays 120, warning and/ornotification lights 122, mechanical and/orelectrical gauges 124, mechanical and/orelectrical switches 126, and/orother instrumentation 128 necessary to communicate and/or adjust operational parameters of the virtual or vehicle carrying theflight emulator 100 in addition to a remote vehicle. However, in some embodiments, thepilot control interface 102 may comprise avirtual instrument panel 118 of a remote vehicle. Further, in some embodiments, thepilot control interface 102 may comprise a blank virtual display or multiple displays that populate with the appropriate gauges when the head mounteddisplay 108 is worn by auser 110.Pilot control interface 102 may also comprisecyclic control 130,collective control 132, and/orpedals 134 for selectively manipulating operation of the vehicle. Additionally, in some embodiments,pilot control interface 102 may also comprise co-pilotcyclic control 130′, co-pilotcollective control 132′, and/orco-pilot pedals 134′. In some embodiments, one ormore touchscreen displays 120 may display a list of all vehicles connected through a network. Additionally, one or more touchscreen displays 120 may selectively alter their respective displays depending on the type of vehicle being controlled. As such, a singlepilot control interface 102 orflight emulator 100 may be configured to emulate multiple vehicles. -
Instrument panel 118 may generally comprise any combination of virtual, physical, simulated, and/oractual components user 110 control over the present vehicle and a virtual or remote vehicle. In some embodiments,instrument panel 118 may comprise a mock instrument panel that allows physical interaction by auser 110 with one or more of thephysical components display 108 may be at least partially augmented or transparent to allow auser 110 to see and interact with one ormore components components instrument panel 118 may comprise a non-functioning “dummy” component. Additionally, theinstrument panel 118 may comprise a plurality of sensors, emitters, reflectors, and/or cameras used to anticipate or detect movement (e.g., arm motion, head motion) of auser 110 towards acomponent pilot control interface 102. Further, one or more touchscreen displays 120 may change its display depending on whether theflight emulator 100 is being used to control a master vehicle or a remote vehicle. - Referring now to
FIG. 3 , a schematic diagram of a virtual reality (“VR”)flight emulator system 150 is shown according to this disclosure. In the embodiment shown,flight emulator system 150 comprises a plurality offlight emulators 100 selectively connected in communication throughnetwork 152, which may be wired, wireless, or any combination thereof. It will be appreciated thatflight emulator system 150 may comprise any number offlight emulators 100. Additionally,flight emulator system 150 also comprises amaster flight emulator 154.Master flight emulator 154 comprises aflight emulator 100 selectively connected in communication with each of the plurality ofslave flight emulators 100 a 1-n throughnetwork 152. However, theflight emulator 100 of themaster flight emulator 154 comprises the capability to selectively virtually teleportuser 110 of themaster flight emulator 154 into aslave flight emulator 100 a 1-n to assume control of thepilot control interface 102 of any of theslave flight emulators 100 a 1-n connected through thenetwork 152. This effectively allows theuser 110 of themaster flight emulator 154 to control the vehicle associated with theslave flight emulator 100 a 1-n into which theuser 110 of themaster flight emulator 154 has virtually teleported. In some embodiments, this capability may be encoded in thesoftware 103 and/or made possible through components of thepilot control interface 102 of themaster flight emulator 154. However, in other embodiments, this capability may be present in each of theflight emulators 100 but overridden or disabled in each of theslave flight emulators 100 a 1-n by themaster flight emulator 154. - By connecting
multiple flight emulators 100 throughnetwork 152, theflight emulators 100 may be flown in the same simulated environment. This allows for multi-vehicle training that may require operating a mix of various aircraft simultaneously to build teaming protocol and flight patterns. As such, a user 110 (e.g., instructor) in themaster flight emulator 154 may monitor each of theflight emulators 100 operated by a user 110 (e.g., student or training participant). In some embodiments, themaster flight emulator 154 may not be simulating a vehicle actively participating in a training exercise on thenetwork 152. Thus, an instructor in themaster flight emulator 154 may virtually teleport into aslave flight emulator 100 a 1-n without taking over control of thepilot control interface 102. This is referred to as a passive teleport. In a passive teleport, the instructor in themaster flight emulator 154 perceives the simulated environment of theslave flight emulator 100 a 1-n that the instructor in themaster flight emulator 154 has virtually teleported into through the head-mounteddisplay 108 associated with themaster flight emulator 154. In some embodiments, the head-mounteddisplay 108 worn by the instructor in themaster flight emulator 154 may replicate imagery seen by the user in theslave flight emulator 100 a 1-n. However, in some embodiments, the head-mounteddisplay 108 worn by the instructor in themaster flight emulator 154 may operate independently and track head movement of the instructor via sensors, emitters, and/orreflectors 109. - When the instructor has virtually teleported into a
slave flight emulator 100 a 1-n, any motion of, orientation of, and/or forces acting on the master aircraft controlled by themaster flight emulator 154 are minimized or preferably cancelled via theGVS devices 117. TheGVS devices 117 may be further operated such that the instructor in themaster flight emulator 154 experiences the motion of, orientation of, and/or forces acting on the vehicle controlled by theslave flight emulator 100 a 1-n into which the instructor in themaster flight emulator 154 has virtually teleported. However, if themaster flight emulator 154 is not simulating a vehicle actively participating in a training exercise, theGVS devices 117 of themaster flight emulator 154 may be simply operated to impart motion of, orientation of, and/or forces acting on the vehicle controlled by theslave flight emulator 100 a 1-n. In both scenarios, this allows the instructor in themaster flight emulator 154 to “feel” only the motion of, orientation of, and/or forces acting on theslave flight emulator 100 a 1-n as if the instructor were in theslave flight emulator 100 a 1-n. - Still further, once virtually teleported into a
slave flight emulator 100 a 1-n, the instructor may receive feedback through thepilot control interface 102 andinstrument panel 118 of themaster flight emulator 154 in response to auser 110 in theslave flight emulator 100 a 1-n providing inputs through thepilot control interface 102 of theslave flight emulator 100 a 1-n. Thus, for example, the instructor preferably feels the amount of force or movement auser 110 may be using to manipulate thecyclic control 130,collective control 132, andpedals 134 of theslave flight emulator 100 a 1-n. The instructor further may be able to observe readings on thegauges 124 and position ofswitches 126 of theslave flight emulator 100 a 1-n through emulation of thepilot control interface 102 of theslave flight emulator 100 a 1-n through thepilot control interface 102 of themaster flight emulator 154. This allows the instructor to fully observe operation of theslave flight emulator 100 a 1-n as if the instructor were actually in theslave flight emulator 100 a 1-n. - By virtually teleporting the instructor in the
master flight emulator 154 into aslave flight emulator 100 a 1-n, the instructor in themaster flight emulator 154 may selectively assume control of thepilot control interface 102 of aslave flight emulator 100 a 1-n. This is referred to as an active teleport. In the active teleport, the instructor in themaster flight emulator 154 may provide inputs through thepilot control interface 102 of themaster flight emulator 154 that are imparted into theslave flight emulator 100 a 1-n. In some embodiments, thepilot control interface 102 of theslave flight emulator 100 a 1-n may be locked out, such that auser 110 may not provide inputs. However, the components of thepilot control interface 102 of theslave flight emulator 100 a 1-n may be driven by and move in accordance with inputs made by the instructor through thepilot control interface 102 of themaster flight emulator 154. This allows the instructor in themaster flight emulator 154 to operate theslave flight emulator 100 a 1-n, while theuser 110 in theslave flight emulator 100 a 1-n receives real-time feedback and “feels” the response of the simulated vehicle associated with theslave flight emulator 100 a 1-n in response to the inputs made by the instructor in themaster flight emulator 154. - Additionally, in the active teleport, the head-mounted
display 108 worn by the instructor in themaster flight emulator 154 replicates imagery from theslave flight emulator 100 a 1-n. Thus, when virtually teleported, the instructor in themaster flight emulator 154 may selectively observe gauges, instruments, and controls in theslave flight emulator 100 a 1-n that may not be replicated by thepilot control interface 102 of themaster flight emulator 154. Additionally, the head-mounteddisplay 108 worn by the instructor in themaster flight emulator 154 is used to track head movement of the instructor via the sensors, emitters, and/orreflectors 109 and the imagery displayed to the instructor on the head-mounteddisplay 108 is responsively adjusted, thereby allowing the instructor to selectively observe the environment of theslave flight emulator 100 a 1-n. Accordingly, feedback through thepilot control interface 102, the head-mounteddisplay 108, and theGVS devices 117 of themaster flight emulator 154 provides real-time emulation capability that allows the instructor to fully observe and control operation of theslave flight emulator 100 a 1-n n when the instructor is virtually teleported as if the instructor of themaster flight emulator 154 were actually in theslave flight emulator 100 a 1-n. - Furthermore, in the active teleport, inputs made by the instructor that impart motion, orientation, and/or forces into the vehicle associated with the
slave flight emulator 100 a 1-n are emulated or replicated to theuser 110 of theslave flight emulator 100 a 1-n through selective operation of theGVS devices 117 of theslave flight emulator 100 a 1-n. This also allows theuser 110 of theslave flight emulator 100 a 1-n to associate inputs made by the instructor and associated response of the simulated vehicle. For example, where auser 110 is providing too much input on acyclic control 130, the instructor may virtually teleport into theslave flight emulator 100 a 1-n of theuser 110 and provide the correctcyclic control 130 input necessary to make a smooth, wide-radius turn or correctcollective control 132 input necessary for a smooth takeoff or landing. As such, this real-time feedback to theuser 110 of theslave flight emulator 100 a 1-n may help auser 110 more effectively learn the correct inputs to properly operate a vehicle associated with aslave flight emulator 100 a 1-n. -
Flight teleport system 150 is also capable of utilizing the control loading system (CLS) of aflight emulator 100 to selectively record and playback of a training exercise or flight mission, thereby allowing auser 110 to receive feedback through thepilot control interface 102 and “feel” the amount and direction of control inputs necessary to expertly pilot the simulated vehicle without having a training instructor onboard. Additionally, playback of a training exercise or flight mission also imparts simulated motion, orientation, and/or forces to auser 110 by selectively operating theGVS devices 117 of eachflight emulator 100, thereby allowing auser 110 to associate inputs with various responses of the simulated vehicle while also providing visual imagery through the head-mounteddisplay 108 of eachflight emulator 100. Playback of a training exercise or flight mission recorded from previous training exercises or simulated pre-programmed missions which provide correct inputs in response to various changes in operational parameters or environmental factors. Further, sincemultiple flight emulators 100 are connected throughnetwork 152, playback may controlmultiple flight emulators 100 simultaneously, allowingseveral users 110 to experience feedback from a single simulated vehicle or a fleet of various simulated vehicles. - Referring now to
FIG. 4 , a schematic diagram of a virtual reality (“VR”)flight teleport system 200 is shown according to this disclosure.Flight teleport system 200 comprises amaster aircraft 202 and at least one remote aircraft 204 connected in communication through awireless communication network 208. However, it will be appreciated thatflight teleport system 200 may comprise any number ofremote aircraft 204 a 1-n.Master aircraft 202 comprises aflight emulator 100 that may be in place of a pilot's seat, a co-pilot's seat, or in another location of themaster aircraft 202, while each of the plurality of remote aircraft 204 comprise aflight control system 206 as known in the art for operating the associated remote aircraft 204. Additionally, in some embodiments, some or all of the remote aircraft 204 may also comprise aflight emulator 100.Master aircraft 202 may be substantially similar to any of the remote aircraft 204. However,flight emulator 100 of themaster aircraft 202 comprises the ability to selectively virtually teleportuser 110 into a remote aircraft 204 to assume control of any of the so-called remote “slave” aircraft 204 on thenetwork 208 via theflight emulator 100 in themaster aircraft 202. This effectively allows auser 110 of themaster aircraft 202, which may be a pilot, co-pilot, or other crew member of themaster aircraft 202, to virtually teleport into and assume control of a fleet of remote aircraft 204 connected through thenetwork 208. - In some embodiments, this capability may be encoded in the
software 103 and/or made possible through components of thepilot control interface 102 of theflight emulator 100 of themaster aircraft 202. Further, in some embodiments, it will be appreciated that theflight control system 206 of any of the remote aircraft 204 may comprise components or software necessary to allow theuser 110 of theflight emulator 100 of themaster aircraft 202 to assume control of itsflight control system 206 and/or any of its other control systems (e.g., landing gear, weaponry). As such, when theuser 110 of theflight emulator 100 of themaster aircraft 202 is virtually teleported into a remote aircraft 204,user 110 of theflight emulator 100 assumes at least partial control of the remote aircraft 204 through thenetwork 208. When a remote aircraft 204 is virtually teleported into by auser 110, theflight control system 206 of the remote aircraft 204 may be locked out or disabled, such that a pilot of the remote aircraft 204 may not provide inputs to alter operation of the remote aircraft 204. - When a
user 110 of themaster aircraft 202 virtually teleports into another aircraft 204,GVS devices 117 of theflight emulator 100 in themaster aircraft 202 may be selectively operated to minimize or preferably cancel the motion of, orientation of, and/or forces acting on themaster aircraft 202. TheGVS devices 117 may be further operated to impart the simulated motion of, orientation of, and/or forces acting on the remote aircraft 204. This allows the pilot in themaster aircraft 202 to “feel” the motion of, orientation of, and/or forces acting on the slave aircraft 204. For example, a pilot of themaster aircraft 202 may place themaster aircraft 202 on a predetermined and/or automated flight plan such as a straight flight path, a large radius turn, or a flat hover, then virtually teleport into a remote aircraft 204. Whenuser 110 is virtually teleported into the remote aircraft 204, the motion of, orientation of, and/or forces acting on themaster aircraft 202 are minimized or preferably cancelled by passing a current betweenselective GVS devices 117 of theflight emulator 100 in themaster aircraft 202, and theGVS devices 117 may be operated such that theuser 110 experiences the motion of, orientation of, and/or forces acting on the remote aircraft 204. This may help a pilot in themaster aircraft 202 effectively operate the remote aircraft 204 or diagnose aircraft performance issues affecting the flight of the remote aircraft 204. - Furthermore, once virtually teleported into a remote aircraft 204, the
user 110 in themaster aircraft 202 also receives feedback through thepilot control interface 102 andinstrument panel 118 of theflight emulator 100 of themaster aircraft 202. Thus, for example, theuser 110 may receive feedback via thecyclic control 130,collective control 132, andpedals 134 from theflight control system 206 from the remote aircraft 204.User 110 further may be able to observe readings on thegauges 124 and position ofswitches 126 of theflight emulator 100 through emulation of theflight control system 206 of the remote aircraft 204 on thepilot control interface 102. Additionally, the head-mounteddisplay 108 worn by theuser 110 in themaster aircraft 202 may replicate imagery from the remote aircraft 204. Thus, theuser 110 in themaster aircraft 202 may selectively observe gauges, instruments, and controls in the remote aircraft 204 that may not be replicated by thepilot control interface 102 when virtually teleported. Additionally, the head-mounteddisplay 108 worn by theuser 110 in themaster aircraft 202 is used to detect head movement of the pilot via the sensors, emitters, and/orreflectors 109 and responsively adjusts the imagery displayed to theuser 110 on the head-mounteddisplay 108, thereby allowing theuser 110 in themaster aircraft 202 to selectively observe the environment of the remote aircraft 204. Accordingly, feedback through thepilot control interface 102, the head-mounteddisplay 108, and theGVS devices 117 of theflight emulator 100 in themaster aircraft 202 provides real-time emulation capability that allows the pilot of themaster aircraft 202 to fully observe and control operation of the remote aircraft 204 when virtually teleported as if the pilot of themaster aircraft 202 were actually in the remote aircraft 204. - As an example, a piloted
master aircraft 202 and a plurality of unmanned remote aircraft 204 may be flown in an attack and/or utility configuration simultaneously on a mission. A pilot or co-pilot of themaster aircraft 202 may place themaster aircraft 202 on a predetermined and/or automated flight plan, while virtually teleporting into an attack remote aircraft 204. When virtually teleported into the remote aircraft 204, the pilot or co-pilot of themaster aircraft 202 is able to see through the head-mounteddisplay 108 the environment surrounding the remote aircraft 204. The pilot of themaster aircraft 202 may then assume control of theflight control system 206 of the attack remote aircraft 204 through theflight emulator 100 in themaster aircraft 202 to perform a strike and/or other operation, then regain command of his/herphysical master aircraft 202. The pilot or co-pilot of themaster aircraft 202 may then virtually teleport into another remote aircraft 204 to perform specific operations before ending use of theflight emulator 100 in themaster aircraft 202 and returning to operating themaster aircraft 202. Thus, in this example, there is always a “pilot in command” when a critical decision, such as an offensive strike, is made, and the “pilot in command” of the remote aircraft 204 is the virtually teleported pilot or co-pilot in themaster aircraft 202. - By virtually teleporting a
user 110 from themaster aircraft 202 into a nearby remote aircraft 204 vianetwork 208, latency of communication (e.g., control signals, feedback, etc.) between themaster aircraft 202 and the remote aircraft 204 may be significantly reduced when compared to a distant ground-based control station. Additionally, thenetwork 208 may allow communication and/or control without the use of satellites. Furthermore, thenetwork 208 may use encrypted control inputs between amaster aircraft 202 and a remote aircraft 204 to prevent thepilot control interface 102 and/orflight control system 206 of anaircraft 202 or remote aircraft 204 from being hacked or overridden. Additionally, in some embodiments, it will be appreciated that the virtually teleporting capabilities of theflight emulator 100 in themaster aircraft 202 may be present in one or more of the remote aircraft 204 connected throughnetwork 208, so thatmultiple users 110 may virtually teleport into any of the other remote aircraft 204 on thenetwork 208. As such, thenetwork 208 may allow multiple teleportingusers 110 simultaneously. In some embodiments, this may allow auser 110 of one remote aircraft 204 to overtake the controls of another remote aircraft 204 having an incapacitated pilot. As such, providing the virtually teleporting capabilities of theflight emulator 100 provides benefits such as increased situational awareness, increased accuracy, and/or quicker decision making during flight missions. - Referring now to
FIG. 5 , a flowchart of amethod 400 of operating a virtual reality (“VR”)flight emulator 100 is shown according to this disclosure.Method 400 begins atblock 402 by providing aflight emulator 100 comprising apilot control interface 102 and a plurality ofGVS devices 117. In some embodiments, theflight emulator 100 may be amaster flight emulator 154 or amaster aircraft 202.Method 400 continues atblock 404 by virtually teleporting auser 110 of theflight emulator 100 into a slave vehicle. The slave vehicle may be a virtual vehicle controlled by aslave flight emulator 100 a 1-n or aremote aircraft 204 a 1-n connected via anetwork user 110 is virtually teleported into the slave vehicle,flight emulator 100 may be selectively configured to emulate the motion of, orientation of, and/or forces acting on the slave vehicle via selective operation of theGVS devices 117 in theflight emulator 100 to simulate the motion of, orientation of, and/or forces acting on the slave vehicle. This is further accomplished via adjustment, movement, selection, or change of display ofcomponents pilot control interface 102. Further, it will be appreciated that this is accomplished via continuously replicating visual imagery through the head-mounteddisplay 108 depending on the head movement and/or orientation of theuser 100 of theflight emulator 100.Method 400 concludes atblock 406 by theuser 110 of theflight emulator 100 providing an input to operate the slave vehicle. This may be accomplished via theuser 110 in amaster flight emulator 154 or a pilot in amaster aircraft 202 providing an input via acomponent pilot control interface 102 that causes aslave flight emulator 100 a 1-n, or theflight control system 206 of a remote aircraft 204 to adjust, alter, change, or select a mode or parameter of operation of the remote aircraft 204. Further, inputs that result in changes in motion of, orientation of, and/or forces acting on the slave aircraft are imparted to theuser 110 of theflight emulator 100 via theGVS devices 117 in theflight emulator 100. - Referring now to
FIG. 6 , a schematic diagram of a general-purpose processor (e.g. electronic controller or computer)system 500 suitable for implementing the embodiments of this disclosure is shown.System 500 that includes aprocessing component 510 suitable for implementing one or more embodiments disclosed herein. Particularly, each of the above-describedflight emulators 100, pilot control interfaces 102,instrument panels 118,flight emulator systems 150, and/orflight teleport systems 200 may comprise one ormore systems 500. In addition to the processor 510 (which may be referred to as a central processor unit or CPU), thesystem 500 might includenetwork connectivity devices 520, random access memory (RAM) 530, read only memory (ROM) 540,secondary storage 550, and input/output (I/O)devices 560. In some cases, some of these components may not be present or may be combined in various combinations with one another or with other components not shown. These components might be located in a single physical entity or in more than one physical entity. Any actions described herein as being taken by theprocessor 510 might be taken by theprocessor 510 alone or by theprocessor 510 in conjunction with one or more components shown or not shown in thesystem 500. It will be appreciated that the data described herein can be stored in memory and/or in one or more databases. - The
processor 510 executes instructions, codes, computer programs, or scripts that it might access from thenetwork connectivity devices 520,RAM 530,ROM 540, or secondary storage 550 (which might include various disk-based systems such as hard disk, floppy disk, optical disk, or other drive). While only oneprocessor 510 is shown, multiple processors may be present. Thus, while instructions may be discussed as being executed byprocessor 510, the instructions may be executed simultaneously, serially, or otherwise by one ormultiple processors 510. Theprocessor 510 may be implemented as one or more CPU chips and/or application specific integrated chips (ASICs). - The
network connectivity devices 520 may take the form of modems, modem banks, Ethernet devices, universal serial bus (USB) interface devices, serial interfaces, token ring devices, fiber distributed data interface (FDDI) devices, wireless local area network (WLAN) devices, radio transceiver devices such as code division multiple access (CDMA) devices, global system for mobile communications (GSM) radio transceiver devices, worldwide interoperability for microwave access (WiMAX) devices, and/or other well-known devices for connecting to networks. Thesenetwork connectivity devices 520 may enable theprocessor 510 to communicate with the Internet or one or more telecommunications networks or other networks from which theprocessor 510 might receive information or to which theprocessor 510 might output information. - The
network connectivity devices 520 might also include one ormore transceiver components 525 capable of transmitting and/or receiving data wirelessly in the form of electromagnetic waves, such as radio frequency signals or microwave frequency signals. Alternatively, the data may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media such as optical fiber, or in other media. Thetransceiver component 525 might include separate receiving and transmitting units or a single transceiver. Information transmitted or received by thetransceiver 525 may include data that has been processed by theprocessor 510 or instructions that are to be executed byprocessor 510. Such information may be received from and outputted to a network in the form, for example, of a computer data baseband signal or signal embodied in a carrier wave. The data may be ordered according to different sequences as may be desirable for either processing or generating the data or transmitting or receiving the data. The baseband signal, the signal embedded in the carrier wave, or other types of signals currently used or hereafter developed may be referred to as the transmission medium and may be generated according to several methods well known to one skilled in the art. - The
RAM 530 might be used to store volatile data and perhaps to store instructions that are executed by theprocessor 510. TheROM 540 is a non-volatile memory device that typically has a smaller memory capacity than the memory capacity of thesecondary storage 550.ROM 540 might be used to store instructions and perhaps data that are read during execution of the instructions. Access to bothRAM 530 andROM 540 is typically faster than tosecondary storage 550. Thesecondary storage 550 is typically comprised of one or more disk drives or tape drives and might be used for non-volatile storage of data or as an over-flow data storage device ifRAM 530 is not large enough to hold all working data.Secondary storage 550 may be used to store programs or instructions that are loaded intoRAM 530 when such programs are selected for execution or information is needed. - The I/
O devices 560 may include liquid crystal displays (LCDs), touchscreen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, printers, video monitors, transducers, sensors, or other well-known input or output devices. Also, thetransceiver 525 might be considered to be a component of the I/O devices 560 instead of or in addition to being a component of thenetwork connectivity devices 520. Some or all of the I/O devices 560 may be substantially similar to various components disclosed herein and/or may be components of the above-describedflight emulators 100, pilot control interfaces 102,instrument panels 118,flight emulator systems 150, and/orflight teleport systems 200. - At least one embodiment is disclosed, and variations, combinations, and/or modifications of the embodiment(s) and/or features of the embodiment(s) made by a person having ordinary skill in the art are within the scope of this disclosure. Alternative embodiments that result from combining, integrating, and/or omitting features of the embodiment(s) are also within the scope of this disclosure. Where numerical ranges or limitations are expressly stated, such express ranges or limitations should be understood to include iterative ranges or limitations of like magnitude falling within the expressly stated ranges or limitations (e.g., from about 1 to about 10 includes, 2, 3, 4, etc.; greater than 0.10 includes 0.11, 0.12, 0.13, etc.). For example, whenever a numerical range with a lower limit, R1, and an upper limit, Ru, is disclosed, any number falling within the range is specifically disclosed. In particular, the following numbers within the range are specifically disclosed: R=R1+k*(Ru−R1), wherein k is a variable ranging from 1 percent to 100 percent with a 1 percent increment, i.e., k is 1 percent, 2 percent, 3 percent, 4 percent, 5 percent, . . . 50 percent, 51 percent, 52 percent, . . . , 95 percent, 96 percent, 95 percent, 98 percent, 99 percent, or 100 percent. Moreover, any numerical range defined by two R numbers as defined in the above is also specifically disclosed.
- Use of the term “optionally” with respect to any element of a claim means that the element is required, or alternatively, the element is not required, both alternatives being within the scope of the claim. Use of broader terms such as comprises, includes, and having should be understood to provide support for narrower terms such as consisting of, consisting essentially of, and comprised substantially of Accordingly, the scope of protection is not limited by the description set out above but is defined by the claims that follow, that scope including all equivalents of the subject matter of the claims. Each and every claim is incorporated as further disclosure into the specification and the claims are embodiment(s) of the present invention. Also, the phrases “at least one of A, B, and C” and “A and/or B and/or C” should each be interpreted to include only A, only B, only C, or any combination of A, B, and C.
Claims (20)
1. A virtual reality (“VR”) flight emulator, comprising:
a pilot control interface; and
a plurality of galvanic vestibular stimulation (GVS) devices worn by a user of the flight emulator;
wherein the flight emulator is configured to allow the user to virtually teleport through a network into at least one of a virtual or remote aircraft connected to the flight emulator.
2. The flight emulator of claim 1 , wherein the pilot control interface is configured to allow the user to control operation of the virtual or remote vehicle when the user is virtually teleported into the virtual or remote aircraft.
3. The flight emulator of claim 2 , wherein the GVS devices are located behind each of a left and a right ear and on each of a front and back of a head of the user.
4. The flight emulator of claim 3 , wherein the GVS devices are configured to selectively operate to cause user to feel motion of, orientation of, or forces acting on the virtual or remote aircraft when the user is virtually teleported into the virtual or remote aircraft.
5. The flight emulator of claim 4 , wherein the GVS devices simulate changes in pitch, roll, and yaw of the virtual or remote aircraft to the user when the user is virtually teleported into the virtual or remote aircraft in response to an input made by the user via the pilot control interface to control the virtual or remote vehicle.
6. The flight emulator of claim 5 , wherein the flight emulator comprises a head-mounted display worn by the user, and wherein the head-mounted display is configured to replicate visual imagery related to the virtual or remote vehicle and further configured to continuously adjust the imagery in response to head movement of the user tracked by a plurality of sensors, emitters, or reflectors.
7. The flight emulator of claim 2 , wherein the virtual aircraft is controlled by a slave flight emulator, and wherein the flight emulator is configured to control the slave flight emulator when the user is virtually teleported into the virtual aircraft.
8. The flight emulator of claim 2 , wherein the flight emulator is carried by a master aircraft and configured to control the remote aircraft when the user is virtually teleported into the remote aircraft.
9. A virtual reality (“VR”) flight system, comprising:
a master flight emulator comprising a pilot control interface and a plurality of galvanic vestibular stimulation (GVS) devices worn by a user of the master flight emulator; and
at least one virtual or remote slave aircraft connected in communication through a network to the master flight emulator;
wherein the master flight emulator is configured to allow the user to virtually teleport into the virtual or remote aircraft.
10. The flight system of claim 9 , wherein the pilot control interface is configured to allow the user to control operation of the slave aircraft when the user is virtually teleported into the slave aircraft.
11. The flight system of claim 10 , wherein the GVS devices are configured to selectively pass current between the GVS devices that cause user to feel motion of, orientation of, or forces acting on the slave aircraft when the user is virtually teleported into the slave aircraft.
12. The flight system of claim 11 , wherein the GVS devices are configured to minimize motion of, orientation of, or forces acting on an aircraft associated with the master flight emulator and simulate only the motion of, orientation of, or forces acting on the slave aircraft when the user is virtually teleported into the slave aircraft.
13. The flight system of claim 12 , wherein the GVS devices impart changes in pitch, roll, and yaw of the slave aircraft to the user when the user is virtually teleported into the slave aircraft in response to an input made by the user via the pilot control interface to control the slave vehicle.
14. The flight system of claim 13 , wherein the master flight emulator comprises a head-mounted display worn by the user, and wherein the head-mounted display is configured to replicate visual imagery related to the slave vehicle and further configured to continuously adjust the imagery in response to head movement of the user detected by a plurality of sensors, emitters, or reflectors coupled to the head-mounted display.
15. The flight system of claim 10 , wherein the slave virtual aircraft is controlled by a slave flight emulator, and wherein the master flight emulator is configured to control the slave flight emulator when the user is virtually teleported into the slave virtual aircraft.
16. The flight system of claim 10 , wherein the master flight emulator is carried by a master aircraft and configured to control the slave remote aircraft when the user is virtually teleported into the slave remote aircraft.
17. A method of operating a virtual reality (“VR”) flight system, comprising:
providing a master flight emulator comprising a pilot control interface and a plurality of galvanic vestibular stimulation (GVS) devices worn by a user of the master flight emulator;
virtually teleporting the user into a virtual or remote slave aircraft connected in communication to the master aircraft; and
providing an input via the pilot control interface to operate the slave aircraft.
18. The method of claim 17 , further comprising: operating the slave aircraft in response to an input made by the user into the pilot control interface when the user is virtually teleported into the slave aircraft.
19. The method of claim 18 , further comprising: selectively operating the GVS devices to minimize motion of, orientation of, or forces acting on an aircraft associated with the master flight emulator and impart only the motion of, orientation of, or forces acting on the slave aircraft when the user is virtually teleported into the slave aircraft.
20. The method of claim 17 , further comprising: replicating visual imagery related to the slave aircraft on a head-mounted display worn by the user; and continuously adjusting the imagery in response to head movement of the user.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/936,915 US20190130783A1 (en) | 2017-11-02 | 2018-03-27 | Vr emulator using galvanic vestibular stimulation devices |
EP18203830.7A EP3547287B1 (en) | 2018-03-27 | 2018-10-31 | Vr emulator using galvanic vestibular stimulation devices |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762581012P | 2017-11-02 | 2017-11-02 | |
US15/936,915 US20190130783A1 (en) | 2017-11-02 | 2018-03-27 | Vr emulator using galvanic vestibular stimulation devices |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190130783A1 true US20190130783A1 (en) | 2019-05-02 |
Family
ID=66243156
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/928,644 Active 2039-06-05 US11043138B2 (en) | 2017-11-02 | 2018-03-22 | VR emulator |
US15/936,894 Active 2039-02-05 US11183077B2 (en) | 2017-11-02 | 2018-03-27 | VR emulator aboard aircraft |
US15/936,915 Abandoned US20190130783A1 (en) | 2017-11-02 | 2018-03-27 | Vr emulator using galvanic vestibular stimulation devices |
US17/354,999 Active 2038-05-05 US11587456B2 (en) | 2017-11-02 | 2021-06-22 | VR emulator |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/928,644 Active 2039-06-05 US11043138B2 (en) | 2017-11-02 | 2018-03-22 | VR emulator |
US15/936,894 Active 2039-02-05 US11183077B2 (en) | 2017-11-02 | 2018-03-27 | VR emulator aboard aircraft |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/354,999 Active 2038-05-05 US11587456B2 (en) | 2017-11-02 | 2021-06-22 | VR emulator |
Country Status (1)
Country | Link |
---|---|
US (4) | US11043138B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AT523558A1 (en) * | 2020-03-05 | 2021-09-15 | Nekonata Xr Tech Gmbh | Method for representing an environment by means of a display unit arranged on a person and visible to the person |
US11164476B2 (en) * | 2018-05-29 | 2021-11-02 | Cse Software Inc. | Heavy equipment simulation system and methods of operating same |
US11262749B2 (en) * | 2019-12-23 | 2022-03-01 | Northrop Grumman Systems Corporation | Vehicle control system |
US11442472B1 (en) * | 2021-08-19 | 2022-09-13 | Beta Air, Llc | System and method for automated flight plan reporting in an electric aircraft |
RU2793663C1 (en) * | 2022-09-28 | 2023-04-04 | Федеральное автономное учреждение "Центральный аэрогидродинамический институт имени профессора Н.Е. Жуковского" (ФАУ "ЦАГИ") | Method for reproducing flight conditions using virtual reality technology on a desktop flight stand for ergonomic research in the information and control field of the flight cockpit |
US11688297B2 (en) * | 2018-11-19 | 2023-06-27 | The Boeing Company | Virtual reality with virtualization in trainers and test environments |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
IL276453B2 (en) * | 2018-02-02 | 2024-02-01 | S Horowitz & Co | Virtual reality based pilot training system |
IT201900016859A1 (en) * | 2019-09-20 | 2021-03-20 | Leonardo Spa | SYSTEM AND METHOD OF EVALUATING THE TRAINING OF AN OPERATOR IN A REAL-TIME TRAINING MISSION |
CN111028608A (en) * | 2019-12-05 | 2020-04-17 | 珠海超凡视界科技有限公司 | Subject two independent training system and method based on virtual reality |
CN113947969A (en) * | 2020-07-15 | 2022-01-18 | 北京易智时代数字科技有限公司 | Flight special situation disposal experience system and operation method |
US20220398936A1 (en) * | 2021-06-15 | 2022-12-15 | Richard Parker | Aircraft training aid systems and processes |
US11993363B2 (en) * | 2022-04-30 | 2024-05-28 | Beta Air, Llc | Methods and apparatuses for a portable cockpit seat |
US11801944B1 (en) | 2022-08-09 | 2023-10-31 | Beta Air, Llc | System and method for monitoring and mitigating pilot actions in an electric aircraft |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140127666A1 (en) * | 2009-03-05 | 2014-05-08 | Vivonics, Inc. | Galvanic vestibular stimulation system and method of use for simulation, directional cueing, and alleviating motion-related sickness |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5865624A (en) | 1995-11-09 | 1999-02-02 | Hayashigawa; Larry | Reactive ride simulator apparatus and method |
US5762612A (en) | 1997-02-28 | 1998-06-09 | Campbell; Craig | Multimodal stimulation in virtual environments |
US7099752B1 (en) | 2003-10-27 | 2006-08-29 | Leslie Jae Lenell | Safelander |
TWI245508B (en) | 2004-04-19 | 2005-12-11 | Lai Yin Liang | Share-memory networked motion simulation system |
US7653883B2 (en) | 2004-07-30 | 2010-01-26 | Apple Inc. | Proximity detector in handheld device |
US20100302359A1 (en) | 2009-06-01 | 2010-12-02 | Honeywell International Inc. | Unmanned Aerial Vehicle Communication |
US8773330B2 (en) | 2009-06-25 | 2014-07-08 | The Boeing Company | Method and apparatus for a virtual mission control station |
US8755965B1 (en) | 2011-03-25 | 2014-06-17 | Frank McClintic | Unmanned vehicle simulator based control methods and apparatus |
ITVR20130046U1 (en) | 2013-10-25 | 2015-04-26 | Giovanni Nicola Lopreiato | TRANSPORTABLE FLIGHT SIMULATOR |
US9335764B2 (en) | 2014-05-27 | 2016-05-10 | Recreational Drone Event Systems, Llc | Virtual and augmented reality cockpit and operational control systems |
US10529248B2 (en) * | 2014-06-19 | 2020-01-07 | Embraer S.A. | Aircraft pilot training system, method and apparatus for theory, practice and evaluation |
-
2018
- 2018-03-22 US US15/928,644 patent/US11043138B2/en active Active
- 2018-03-27 US US15/936,894 patent/US11183077B2/en active Active
- 2018-03-27 US US15/936,915 patent/US20190130783A1/en not_active Abandoned
-
2021
- 2021-06-22 US US17/354,999 patent/US11587456B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140127666A1 (en) * | 2009-03-05 | 2014-05-08 | Vivonics, Inc. | Galvanic vestibular stimulation system and method of use for simulation, directional cueing, and alleviating motion-related sickness |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11164476B2 (en) * | 2018-05-29 | 2021-11-02 | Cse Software Inc. | Heavy equipment simulation system and methods of operating same |
US11527173B2 (en) * | 2018-05-29 | 2022-12-13 | Cse Software Inc. | Heavy equipment simulation system and methods of operating same |
US11688297B2 (en) * | 2018-11-19 | 2023-06-27 | The Boeing Company | Virtual reality with virtualization in trainers and test environments |
US11262749B2 (en) * | 2019-12-23 | 2022-03-01 | Northrop Grumman Systems Corporation | Vehicle control system |
AT523558A1 (en) * | 2020-03-05 | 2021-09-15 | Nekonata Xr Tech Gmbh | Method for representing an environment by means of a display unit arranged on a person and visible to the person |
AT523558B1 (en) * | 2020-03-05 | 2023-07-15 | Nekonata Xr Tech Gmbh | Method for displaying an environment by means of a display unit arranged on a person and visible to the person |
US11442472B1 (en) * | 2021-08-19 | 2022-09-13 | Beta Air, Llc | System and method for automated flight plan reporting in an electric aircraft |
RU2793663C1 (en) * | 2022-09-28 | 2023-04-04 | Федеральное автономное учреждение "Центральный аэрогидродинамический институт имени профессора Н.Е. Жуковского" (ФАУ "ЦАГИ") | Method for reproducing flight conditions using virtual reality technology on a desktop flight stand for ergonomic research in the information and control field of the flight cockpit |
Also Published As
Publication number | Publication date |
---|---|
US20190130782A1 (en) | 2019-05-02 |
US11043138B2 (en) | 2021-06-22 |
US11183077B2 (en) | 2021-11-23 |
US20190130781A1 (en) | 2019-05-02 |
US11587456B2 (en) | 2023-02-21 |
US20210343178A1 (en) | 2021-11-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20190130783A1 (en) | Vr emulator using galvanic vestibular stimulation devices | |
US9984586B2 (en) | Method and device to improve the flying abilities of the airborne devices operator | |
EP3543986A1 (en) | Vr emulator | |
EP3547287B1 (en) | Vr emulator using galvanic vestibular stimulation devices | |
US8755965B1 (en) | Unmanned vehicle simulator based control methods and apparatus | |
CN102356417A (en) | Teleoperation method and human robot interface for remote control of machine by human operator | |
CN113496635B (en) | Flight simulator and flight training simulation method | |
CA2890355C (en) | Method and device for the combined simulation and control of remote-controlled vehicles | |
KR20100007060A (en) | Instructor operating system | |
KR101437747B1 (en) | Apparatus and method for image processing | |
CN113706969A (en) | Virtual-real combined helicopter cabin system | |
CN105590496A (en) | Portable unmanned helicopter flight operator simulation training system | |
CN111857177B (en) | Method, device, equipment and medium for generating remote control target instruction | |
EP3547060B1 (en) | Virtual reality emulator aboard aircraft | |
KR101483106B1 (en) | Flight Simulator having a Displaying Point Changing Function of Display Video Picture and Controlling Method for the sames | |
Repperger et al. | Effects of haptic feedback and turbulence on landing performance using an immersive cave automatic virtual environment (CAVE) | |
Klyde et al. | A mixed reality simulation tool for in-flight evaluations | |
Chakraborty et al. | Piloted Simulation-Based Assessment of Simplified Vehicle Operations for Urban Air Mobility | |
KR102350576B1 (en) | Method and device for simulating spatial disorientation of pilot during flight | |
CN108100297A (en) | A kind of automatic mission phase test system of aircraft | |
Milam et al. | Spatial Audio Cueing Aids Pilot Navigation during Simulated Helicopter Flight in Degraded Visual Environments | |
McGrath et al. | The role of intelligent software in spatial awareness displays | |
KR102425452B1 (en) | A method for training coriolis illusion based on spatial disorientation training system | |
Perfect et al. | Investigation of Personal Aerial Vehicle Handling Qualities Requirements for Harsh Environmental Conditions | |
Underhill et al. | Helmet-mounted display technology on the VISTA NF-16D |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BELL HELICOPTER TEXTRON INC., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NISSEN, JEFFREY PAUL;MCCALL, DANIEL BRENT;RICCOBONO, JOSHUA PETER;SIGNING DATES FROM 20180320 TO 20180326;REEL/FRAME:045384/0648 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |