WO2017089345A1

WO2017089345A1 - Video game system utilizing biofeedback for improving player's pulse oximeter parameter values

Info

Publication number: WO2017089345A1
Application number: PCT/EP2016/078429
Authority: WO
Inventors: John Cronin; Michael D' ANDREA
Original assignee: Koninklijke Philips N.V.
Priority date: 2015-11-24
Filing date: 2016-11-22
Publication date: 2017-06-01

Abstract

A method for utilizing biofeedback information in a video game system comprising: acquiring pulse oximeter data using a pulse oximeter; acquiring environmental context data using environmental sensors; retrieving a player's medical record from a database; transmitting the acquired pulse oximeter data, the acquired environmental context data and the retrieved player's medical record to the video game system; adjusting a target 5 pulse oximeter parameter range based on the transmitted environmental context data and the transmitted player's medical record; and adjusting a gameplay of the video game system based on the transmitted pulse oximeter data in order for the player to reach a pulse oximeter parameter value that lies within the target pulse oximeter parameter range.

Description

Video game system utilizing biofeedback for improving player's pulse oximeter parameter values

BACKGROUND OF THE INVENTION

Patients recovering from illness are sometimes asked by healthcare providers to engage in some type of exercise regimen to help speed up their recovery. However, without active monitoring by health care providers, patients do not always follow the recommended exercise regimen or they do not perform it properly, if they perform it at all.

An innovative way to encourage patients to perform recommended exercise as part of a physical therapy regimen is to incorporate the exercise regimen in a video game. Video games designed for fitness have gained popularity over the years. Current video game systems already incorporate physiological sensors for inputting physiological data into video game systems to enable biofeedback training. Biofeedback has been used to monitor and regulate health parameters. By exploiting biofeedback in video games, patients are motivated to not only engage in the prescribed exercise therapy but to also strive to reach one or more recommended health-related goals.

The article entitled "Improving Health Outcomes for the Elderly an Analytic Framework" by E. Lawrence et al. discloses developing training routines and exercises tailored to the user's needs. The reference discloses integrating interactive gaming technologies into a remote healthcare system that comprises a pulse oximeter and

environmental sensor motes.

Korean patent number 1020150105340 discloses a gaming device comprising a head mounted display that includes a sensor subsystem. The sensor subsystem provides pulse oximeter data and physical environment data. The acquired sensor data are used to select safety rules to be applied to the video game.

SUMMARY OF THE CLAIMED INVENTION

The present invention relates to a method for utilizing biofeedback

information in a video game system. Pulse oximeter data and environmental context data are acquired using a pulse oximeter and environmental sensors, respectively, and the player's medical record is retrieved from a database. The pulse oximeter data, environmental context data and player's medical record are then transmitted to the video game system. Based on the transmitted environmental context data and the transmitted player's medical record, a target pulse oximeter parameter range is adjusted. A video game system's gameplay is also adjusted based on the transmitted pulse oximeter data to allow the player to reach a pulse oximeter parameter value that lies within the target pulse oximeter parameter range.

The present invention also relates to a system comprising a pulse oximeter for acquiring pulse oximeter data, environmental sensors for acquiring environmental context data, at least one database for storing a player's medical record, and a video game system for receiving and utilizing the acquired pulse oximeter data, environmental context data, and player's medical record to adjust the video game system's gameplay.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated herein to illustrate embodiments of the invention. Along with the description, they also serve to explain the principle of the invention. In the drawings:

FIG. 1 illustrates a block diagram of a system for utilizing biofeedback information in a video game system according to a preferred embodiment of the present invention.

FIG. 2 is a flowchart according to a preferred embodiment of the present invention.

FIG. 3 illustrates a block diagram of an embodiment of the system of the present invention.

FIG. 4 is a flowchart describing a software for adjusting a target pulse oximeter parameter range.

FIG. 5 is a flowchart describing a software for adjusting a video game's gameplay.

DETAILED DESCRIPTION

The following are definitions of terms as used in the various embodiments of the present invention.

The term "pulse oximetry data" as used herein refers to data relating to a person's oxygen saturation including Sp0₂ level, pulse rate, and perfusion index, among others. The term "environmental context data' as used herein refers to data relating to the person's surroundings including light level (e.g., sunlight and/or artificial light), weather, ambient temperature, humidity, wind, air quality, and atmospheric conditions, among others.

The term "medical record" as used herein refers to a documentation of a patient's medical and health care related data. Medical records include medical history (such as surgical history, obstetric history, habits, etc.), family history, allergies, medical prescriptions, and recent physiological test results, among others.

The term "video game system" as used herein refers to an electronic system used to play video games or otherwise engage in an interactive activity. Examples of a video game system are personal computers, video game consoles, electronic equipment, and hand held devices, among others.

The term "biofeedback" as used herein refers to a system or process that enables a person to improve physiological health or performance by suggesting changes or adjustments to the person's activities.

The term "gameplay" as used herein refers to the specific way in which a person interact with a videogame. "Gameplay" as used herein generally relates to a pattern defined through game rules, difficulty levels, challenges, and goals, among others.

The term "database" as used herein refers to a collection of data and information organized in such a way as to allow the data and information to be stored, retrieved, updated, and manipulated and to allow them to be presented into one or more formats such as in table form or to be grouped into text, numbers, images, and audio data. The database typically resides in computer memory that includes various types of volatile and non-volatile computer memory. "Database" as used herein also refers to conventional databases that may reside locally or that may be accessed from a remote location, e.g., remote network servers. The term "database" as used herein may also refer to a segment or portion of a larger database, which in this case forms a type of database within a database. Memory wherein the database resides may include high-speed random access memory or non- volatile memory such as magnetic disk storage devices, optical storage devices, and flash memory. Memory where the database resides may also comprise one or more software for processing and organizing data received by and stored into the database.

The present invention relates to a method for utilizing biofeedback information in a video game system comprising: acquiring pulse oximeter data using a pulse oximeter; acquiring environmental context data using environmental sensors; retrieving a player's medical record from a database; transmitting the acquired pulse oximeter data, environmental context data and player's medical record to the video game system; adjusting a target pulse oximeter parameter range based on the transmitted environmental context data and the transmitted player's medical record; and adjusting the video game system's gameplay based on the transmitted pulse oximeter data to enable the player to reach a pulse oximeter parameter value that lies within the target pulse oximeter parameter range.

The present invention also relates to a system for utilizing biofeedback information in a video game system comprising: a pulse oximeter for acquiring pulse oximeter data; environmental sensors for acquiring environmental context data; at least one database for storing a player's medical record; and a video game system for receiving and utilizing the acquired pulse oximeter data, environmental context data and player's medical record to adjust the video game system's gameplay.

FIG. 1 illustrates a preferred embodiment of a system for utilizing biofeedback information in a video game system. As shown in the figure, a pulse oximeter 100 for acquiring pulse oximeter data, environmental sensor 102 for acquiring environmental context data, and database 104 for storing medical records are connected to a video game system 106. In one embodiment, pulse oximeter 100 and environmental sensor 102 are stand-alone devices in communication with the video game system 106 via a communications module. The database 104 may be remotely accessed by the video game system 106 via the cloud/Internet. In another embodiment, the pulse oximeter 100, environmental sensor 102, and database 104 are integrated into the video game system 106.

FIG. 2 illustrates a preferred method of the present invention. Pulse oximeter data is acquired using a pulse oximeter 100 (step 200). The pulse oximeter comprises an electronic processor and a couple of small light emitting diodes (LEDs) facing a photodiode through a translucent portion of a patient's body, typically a fingertip or an earlobe. One of the LEDs emits light in the red portion of the visible region of the electromagnetic spectrum (red LED) while the other emits in the infrared region. The amount of light absorbed at these two wavelengths differs significantly between oxygen rich blood and blood deficient in oxygen. Oxygenated hemoglobin absorbs more infrared light and allows more red light to pass through. On the other hand, deoxygenated hemoglobin allows more infrared light to pass through and absorbs more red light. Oxy- hemoglobin and its deoxygenated form have significantly different absorption pattern.

In operation, the LEDs alternately turn on and off, and then both off approximately for a predetermined period of time. This allows the light sensor, for example, a photodiode, to respond to the red and infrared light separately and also to correct for the light detected due to ambient light (measured when both LEDs are off; used as baseline or reference signal). The amount of light that is transmitted (that is, not absorbed) is measured, and separate normalized signals are produced for each wavelength. These signals due to transmitted light vary with time because the amount of arterial blood that is present increases with each heartbeat. By subtracting the minimum transmitted light from the peak transmitted light in each wavelength, the effects of other tissues and materials (e.g., venous blood, skin, bone, muscle, fat, including nail polish) can be corrected for because they normally absorb a constant amount of light over a period of time. The ratio of the measured red light to the measured infrared light is calculated by the processor. This ratio, which represents the ratio of oxygenated hemoglobin to deoxygenated hemoglobin, is then converted to a Sp0₂ reading by the processor.

In the embodiment shown in FIG. 2, simultaneous to acquiring pulse oximetry data (step 200), environmental context data is acquired using environmental sensor 102 (step 202). Also, a player's medical record is retrieved from a database 104 (step 204). Then, the acquired pulse oximeter data, environmental context data and the retrieved player's medical record are transmitted to the video game system 106 (step 206). Subsequently, a target pulse oximeter parameter range is adjusted based on the transmitted environmental context data and the transmitted player's medical record (step 208). A video game system's 106 gameplay is adjusted based on the transmitted pulse oximeter data in order for the player to attain a pulse oximeter parameter value that lies within the target pulse oximeter parameter range (step 210).

FIG. 3 illustrates another embodiment of the present invention. The system comprises a video game system 300 for executing a video game. Preferably, the video game is an interactive game focusing on fitness and exercise. The video game system 300 is in communication with a pulse oximeter 302, which is attached to a player 304, and an environmental sensor 306 installed in a room. The video game system 300 is also in connected to an electronic medical record network 308 via the internet 310. The electronic medical record network 308 contains a database 312 wherein the player's electronic medical record 314 is stored. Acquired pulse oximeter data, environmental sensor data and retrieved electronic medical record 314 are transmitted to and stored in the memory of the video game system 300.

The video game system 300 further comprises a target adjustment software 316 for adjusting a target pulse oximeter parameter range, biofeedback software 318 for adjusting a gameplay of the video game, a biofeedback game 320, and a library of biofeedback exercise videos 322. The biofeedback game 320 is an electronic game that provides the player 304 exercises to perform in order to improve the player's pulse oximeter parameter values. The biofeedback exercises are demonstrated to the player 304 by displaying a video retrieved from the library of biofeedback exercise videos 322. Preferably, the biofeedback game 320 and the library of biofeedback exercise videos 322 are

downloaded via the internet 310 and are stored in the memory of the video game system 300. Alternatively, the biofeedback game 320 and the library of biofeedback exercise videos 322 are streamed via the internet 310.

In one embodiment, the biofeedback game 320 is presented as a side or preliminary game that the player 304 must complete before the player can proceed with the actual video game. For example, to increase the Sp0₂ level of the player 304, the video game is interrupted and the player 304 is prompted to perform a breathing exercise. Once the player's Sp0₂ level has risen to a certain level, the video game resumes.

FIG. 4 illustrates target adjustment software 316 for adjusting a target pulse oximeter parameter range. The software is initiated once the video game system 300 receives the environmental sensor data and the player's electronic medical record 314 (step 400). The environmental sensor data and player's electronic medical record 314 are stored in the video game system's memory. The video game system 300 then determines if the player's electronic medical record 314 is stored within its memory (step 402). If so, the video game system 300 accesses the player's electronic medical record 314. From the electronic medical record 314, the video game system 300 determines if there are any information about the player 304 that may affect the player's pulse oximeter parameters (step 404). If any relevant information is found, the target pulse oximeter parameter range is adjusted based on the acquired relevant information (step 406). For example, the player's electronic medical record 314 discloses that player 304 is a heavy smoker. Smoking increases the amount of carboxyhemoglobin (HbCO) in the blood. HbCO is hemoglobin which is incapable of carrying oxygen. The presence of large amount of HbCO in the blood leads to a lower-than- normal Sp0₂ level. Thus, in this case, the video game system 300 will detect an Sp0₂ level of the player 304 that lies below the target Sp0₂ level range. To compensate for the player's lower-than-normal Sp0₂ level, the video game system 300 lowers the target Sp0₂ level range.

Once the target pulse oximeter parameter range has been adjusted (step 406) or if the player's electronic medical record 314 is not available (step 402), the video game system 300 proceeds to determine if there are available environmental data (step 408). If environmental data are unavailable, the process ends. Otherwise, the video game system 300 determines if the environmental parameters will affect the player's one or more pulse oximeter parameters (step 410). If the environmental parameters are determined to have no effect, the process terminates. Else, the target pulse oximeter parameter range is adjusted based on the one or more environmental parameters (step 412). For example a temperature sensor installed in the room measures the temperature to be 50°F. At this temperature, the player 304 may experience local hyperthermia in the fingers, which causes the arteries to constrict in order to reduce heat dissipation. The constriction of arteries results in lower oxygen levels in the blood thus lowering the Sp0₂ level. To compensate for the Sp0₂ level decrease, the video game system 300 lowers the target Sp0₂ level range. After adjusting the target pulse oximeter parameter range (step 412), the process ends.

In one embodiment, the adjustments used to modify the target pulse oximeter parameter range are obtained from a target pulse oximeter parameter range database. The target pulse oximeter parameter range database is stored preferably in the video game system 300. Alternatively, the target pulse oximeter parameter range database is accessed through the internet 310. In another embodiment, the adjustments made on the target pulse oximeter parameter range are generated using an algorithm preferably included in the video game system.

FIG. 5 illustrates the biofeedback software 318 for adjusting a video game's gameplay. The software is initiated once the player 304 starts the video game. The adjusted target pulse oximeter parameter range determined by the target adjustment software 316 is retrieved (step 500). Pulse oximetry data are continuously acquired and transmitted to the video game system 300. The video game system 300 determines if the pulse oximeter parameter value is within the target oximeter parameter range (step 502). If not, a

biofeedback game 320 is initiated (step 504). The biofeedback game 320 inserts a game object within the video game. Once the player 304 encounters the game object, a video is displayed on the display of the video game system 300 to demonstrate an exercise for controlling the player's one or more pulse oximeter parameters (step 506). The exercise video is obtained from the library of biofeedback exercise videos 322. After a system-predefined time, if the pulse oximeter parameter value is still not within the target pulse oximeter parameter range (step 508), an alternative exercise video is displayed (step 510). If after a second system-predefined time the pulse oximeter parameter value is still not within the target oximeter parameter range (step 512), the biofeedback game 320 restarts (step 514) allowing player 304 to perform the displayed exercises again. In one embodiment, the video game system 300 detects through a video capturing device if player 304 performs the displayed exercise. The video capturing device captures the motion of player 304 and compares this with the displayed video.

After replaying the biofeedback game 320, a healthcare provider-suggested action is delivered to player 304 via the video game system 300 (step 518) if the pulse oximeter parameter value is still not within the target pulse oximeter parameter range (step 516). The healthcare provider-suggested action is delivered in the form of at least one of audio data, video data, and text data. Preferably, the healthcare provider suggested action is determined using the player's electronic medical record 314. Alternatively, the healthcare provider suggested action is obtained from a healthcare provider database accessed via the internet 310. Examples of healthcare provider suggested actions are shutting off the game, contacting a caregiver, and utilizing oxygen tanks, among others.

If the one or more pulse oximeter parameter values lie within the target pulse oximeter parameter range (step 502, 508, 512, and 516), player 304 returns to the original gameplay of the video game. The software runs continuously until player 304 terminates the video game.

In one embodiment, the system does not include a biofeedback game 320 and a library of biofeedback exercise videos 322. In this case, the biofeedback software 318 preferably adjusts the game's difficulty as a response to the transmitted pulse oximeter parameter value that were not within the target pulse oximeter parameter range.

In an exemplary embodiment, the player was advised by a doctor that during exercise, the player's Sp0₂ level must not drop more than 2% from normal Sp0₂ level. The player initiates a fitness video game using a video game console. Upon initiation of the fitness video game, the video game console retrieves the player's medical records from the doctor's database via a cloud network. Based on the player's medical record, the video game console determines that the player is a heavy smoker. The video game console then accesses a target Sp0₂ level range database via the cloud network. From the target Sp0₂ level range database, it is found that for a heavy smoker, the normal Sp0₂ level is 94%, therefore the target Sp0₂ level range is set to 92%- 100%. Thus, the video game console sets the target Sp0₂ level range to 92%- 100%.

After retrieving the player's medical records, the video game console receives data regarding the ambient temperature of the player's environment via a temperature sensor. The room's temperature is measured to be at 55 °F. To compensate for the cold temperature's effect on the Sp0₂ level parameter, the video game console adjusts the target Sp0₂ level range, via an algorithm, to 91%-99%.

During fitness video game, the player's Sp0₂ level is constantly monitored using a pulse oximeter. After the player performs the exercise shown in the fitness video game, the player's Sp0₂ level dropped to 90%. Thus, the fitness video game's gameplay is interrupted to alert the player that the Sp0₂ level is now below the target Sp0₂ level range of 91%-99%. A video is then presented to the player to demonstrate a breathing exercise for increasing Sp0₂ level. After 2 minutes, the player's measured Sp0₂ level remains at 90%>. This triggers the presentation of a second video to the player to demonstrate a second breathing exercise. After another 2 minutes, the player's Sp0₂ level is still at 90%. The video game console thus loops back to the first video. After yet another 2 minutes, the player's Sp0₂ level remains unchanged. This time, the video game console accesses the retrieved player's medical history, which includes a doctor's suggestion in response to the above situation. In this case, the doctor suggests that the player breathe oxygen from an oxygen tank. The video game console relays the doctor's suggestion to the player in the form of a text message. Afterwards, the video game console terminates the video game.

In accordance with the various embodiments of the present invention, the pulse oximeter 100 is preferably a portable pulse oximeter device adapted to be worn on a patient's finger and adapted to measure the oxygen saturation of the patient. Alternatively, the pulse oximeter 100 is adapted to be worn on a body part other than the patient's finger. In another embodiment, the pulse oximeter 100 is integrated with a peripheral device of the video game system 106 such as a game controller and head-mounted display.

In accordance with the various embodiments of the present invention, the environmental sensor 102 is preferably a stand-alone device that measures at least one of atmospheric pressure, altitude, humidity, and temperature.

In accordance with the various embodiments of the present invention, the video game system 106 comprises a processer, a memory, a communications module, and peripheral devices for user input. The video game system 106 is able to execute a computer game program. The computer game program may be stored in memory storing devices such as a CD or may be downloaded from the internet or cloud.

The communications module may include any transmitter or receiver used for WiFi, Bluetooth, infrared, NFC, radio frequency, cellular communication, visible light communication, Li-Fi, WiMax, ZigBee, fiber optic and other forms of wireless communication devices. Alternatively, the communications module may be a physical channel such as a USB cable or other wired forms of communication.

The present invention is not intended to be restricted to the several exemplary embodiments of the invention described above. Other variations that may be envisioned by those skilled in the art are intended to fall within the disclosure.

Claims

CLAIMS:

1. A method for utilizing biofeedback information in a video game system, the method comprising:

monitoring pulse oximeter data using a pulse oximeter;

monitoring environmental context data using environmental sensors;

retrieving a player's medical record from a database;

transmitting the pulse oximeter data, the environmental context data, and the retrieved player's medical record to the video game system;

adjusting a target pulse oximeter parameter range based on the transmitted environmental context data and the transmitted player's medical record; and

adjusting a gameplay of the video game system based on the transmitted pulse oximeter data in order for the player to reach a pulse oximeter parameter value that lies within the target pulse oximeter parameter range.

2. The method of claim 1 , further comprising displaying at least one video via the video game system that demonstrates an exercise for adjusting one or more pulse oximeter parameter values.

3. The method of claim 2, wherein playing the at least one video via the video game system comprises selecting the at least one video from a library of videos.

4. The method of claim 3, wherein playing the at least one video via the video game system comprises streaming the at least one video from the library of videos.

5. The method of claim 1 , further comprising providing to the player via the video game system a healthcare provider-defined action when the transmitted pulse oximeter value lies outside the target pulse oximeter parameter range.

6. The method of claim 5, further comprising interrupting a video game to provide the healthcare provider-defined action and resuming the video game when a subsequent pulse oximeter value lies within the target pulse oximeter parameter range.

7. The method of claim 6, further comprising selecting the healthcare provider- defined action from a healthcare provider database.

8. The method of claim 1, wherein adjusting a gameplay of the video game system comprises adjusting a difficulty level.

9. A system for utilizing biofeedback information in a video game system, the system comprising:

a pulse oximeter that monitors pulse oximeter data;

one or more environmental sensors that monitors environmental context data; a database that stores a player's medical record; and

a video game system that adjusts gameplay based on the pulse oximeter data, the acquired environmental context data, and the retrieved player's medical record.

10. The system of claim 9, wherein the video game system displays at least one video that demonstrates an exercise for adjusting one or more pulse oximeter parameter values.

11. The system of claim 10, wherein the at least one video is selected from a library of videos.

12. The system of claim 11 , wherein the at least one video is streamed from the library of videos.

13. The system of claim 9, wherein the video game system provides the player with a healthcare provider-defined action when the transmitted pulse oximeter value lies outside the target pulse oximeter parameter range.

14. The system of claim 13, wherein the video game system interrupts a video game to provide the healthcare provider-defined action and resumes the video game when a subsequent pulse oximeter value lies within the target pulse oximeter parameter range.

15. The system of claim 14, wherein the healthcare provider-defined action is selected from a healthcare provider database.

16. The system of claim 9, wherein the video game system adjusts gameplay by adjusting a difficulty level of a video game.

17. A non-transitory computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for utilizing biofeedback information in a video game system, the method comprising:

monitoring pulse oximeter data using a pulse oximeter;

monitoring environmental context data using environmental sensors;

retrieving a player's medical record from a database;