CN111833682A - Virtual physical examination teaching method and device based on VR technology - Google Patents
Virtual physical examination teaching method and device based on VR technology Download PDFInfo
- Publication number
- CN111833682A CN111833682A CN202010640933.2A CN202010640933A CN111833682A CN 111833682 A CN111833682 A CN 111833682A CN 202010640933 A CN202010640933 A CN 202010640933A CN 111833682 A CN111833682 A CN 111833682A
- Authority
- CN
- China
- Prior art keywords
- virtual
- physical examination
- patient
- trainer
- voice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000005516 engineering process Methods 0.000 title claims abstract description 27
- 230000006399 behavior Effects 0.000 claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000012544 monitoring process Methods 0.000 claims abstract description 15
- 210000003414 extremity Anatomy 0.000 claims description 26
- 230000009471 action Effects 0.000 claims description 19
- 238000011156 evaluation Methods 0.000 claims description 16
- 230000001575 pathological effect Effects 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 6
- 230000006461 physiological response Effects 0.000 claims description 6
- 208000004998 Abdominal Pain Diseases 0.000 claims description 4
- 206010011224 Cough Diseases 0.000 claims description 4
- 206010017577 Gait disturbance Diseases 0.000 claims description 4
- 208000002193 Pain Diseases 0.000 claims description 4
- 210000003127 knee Anatomy 0.000 claims description 4
- 210000002414 leg Anatomy 0.000 claims description 3
- 230000036544 posture Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000007596 consolidation process Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241001456553 Chanodichthys dabryi Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000002555 auscultation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036461 convulsion Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002559 palpation Methods 0.000 description 1
- 238000009527 percussion Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The application relates to a virtual physical examination teaching method and device based on VR technology, wherein the virtual physical examination teaching method based on VR technology comprises the following steps: constructing a virtual scene and a virtual patient; displaying the virtual environment and the virtual patient to the trainer through VR helmet equipment, and interacting with the trainer; identifying the physical examination operation behavior of the trainer through a limb and gesture identification module; recognizing the voice information of the trainer through voice recognition equipment; controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information; and monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, and counting and evaluating the physical examination operation behaviors and the voice information to give training results and guidance suggestions.
Description
Technical Field
The application relates to the relevant field of medical education, in particular to a virtual physical examination teaching method and device based on VR technology.
Background
The physical examination is an important means for the doctor to clinically diagnose the patient, and is a skill that each doctor must be skilled before becoming a doctor. Repeated training and consolidation is required throughout the medical learning process. How to form an effective automatic teaching solution under the condition of consuming teaching resources and instructor time as little as possible is a difficult problem of medical education.
The existing training method is the SP (standardized patient) protocol, namely: selecting qualified personnel, performing standardization and systematization training, and simulating clinical patients by means of performance, make-up, props and the like. The trainee performs simulated training through such simulated clinical patients. The disadvantages are that: training the SP requires a significant investment of capital and time, and training is costly. Moreover, because the simulated patient has a large difference with the real patient, the simulated patient cannot embody all pathological characteristics through performance and makeup, and the reality is not high.
Disclosure of Invention
In order to overcome the problems in the related art at least to a certain extent, the application provides a virtual physical examination teaching method and device based on VR technology.
According to a first aspect of the embodiments of the present application, a virtual physical examination teaching method based on VR technology is provided, which includes:
constructing a virtual scene and a virtual patient;
displaying the virtual environment and the virtual patient to the trainer through VR helmet equipment, and interacting with the trainer;
identifying the physical examination operation behavior of the trainer through a limb and gesture identification module;
recognizing the voice information of the trainer through voice recognition equipment;
controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information;
and monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, and counting and evaluating the physical examination operation behaviors and the voice information to give training results and guidance suggestions.
Optionally, constructing the virtual scene includes:
the ward where the physical examination is located, medical equipment, furnishing appliances and physical examination instruments are constructed.
Optionally, constructing the virtual patient comprises:
constructing patient models, patient actions, and patient speech
The patient model comprises three parts, namely a grid, a map and a material, and is used for expressing the appearance of a real patient;
the patient action is used for representing pathological characteristics and physiological response of the patient; wherein, the pathological features include: such as abdominal pain, hunchback, limping and body position during physical examination;
the physiological response comprises: such as knee jerking, cough, groaning, pain, etc.;
the patient voice is used for simulating the phonation of the patient; wherein the medical characteristics of the patient's voice are adjusted based on different case data.
Optionally, the VR device is an htc vive pro VR device, and is configured to display a virtual scene and a virtual patient, and to display feedback information of all interaction results during the physical examination process, and to position and synchronize data such as a real position and an orientation of a trainer in real time.
Optionally, the limb and gesture recognition device: the system is used for detecting and identifying the limb actions and the gestures of the trainer, and undertakes the synchronization of the limb actions and the hand actions of the trainer and the postures of the virtual trainer;
optionally, the limb and gesture recognition device includes: a kinect limb gesture recognition device, and a lepemotion gesture motion recognition device.
Optionally, the speech recognition device: the system is responsible for translating words spoken by a trainer into characters, converting the characters into instruction entries and driving a virtual patient to make corresponding feedback;
wherein the feedback driving the virtual patient comprises: motion feedback and/or voice feedback.
Optionally, the method further includes:
monitoring the physical examination operation behavior of the trainer to determine whether the voice content meets the flow specification and professional specification of physical examination; and corresponding prompts are made for the physical examination operation behaviors and the voice contents which do not meet the specifications.
Based on the second aspect of this application, provide a virtual physical examination teaching equipment based on VR technique, include:
the construction module is used for constructing a system to construct a virtual scene and a virtual patient;
the display module is used for displaying the virtual environment and the virtual patient to the trainer through the VR helmet equipment and interacting with the trainer;
the recognition module is used for recognizing the module through limbs and gestures: identifying physical examination operation behaviors of a trainer, and identifying voice information of the trainer through voice identification equipment;
the feedback module is used for controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information;
and the evaluation module is used for monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, counting and evaluating the physical examination operation behaviors and the voice information and giving training results and guidance suggestions.
According to a third aspect of the embodiments of the present application, there is provided a storage medium, in which a preset program is stored, and when the preset program is executed, the virtual physical examination teaching method and apparatus based on VR technology according to any one of claims 1 to 8 are implemented.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
in the method provided by the embodiment of the application, a virtual scene and a virtual patient are constructed; displaying the virtual environment and the virtual patient to the trainer through VR helmet equipment, and interacting with the trainer; identifying the physical examination operation behavior of the trainer through a limb and gesture identification module; recognizing the voice information of the trainer through voice recognition equipment; controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information; and monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, and counting and evaluating the physical examination operation behaviors and the voice information to give training results and guidance suggestions. By the method, in the virtual physical examination teaching, a trainer can complete the same interactive operation and doctor-patient communication conversation as the real physical examination on the virtual patient model, and can intelligently detect the process and result data in the physical examination training. By the invention, a trainer can repeatedly drill various physical exams in a high-fidelity operation simulation environment, a real physical examination patient model is not needed, the resource cost of physical examination training is effectively reduced, the on-site supervision and evaluation of a guide can be separated, and the efficiency of physical examination teaching training is greatly improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart illustrating a virtual physical exam teaching method based on VR technology according to an exemplary embodiment.
Fig. 2 is a block diagram illustrating a virtual physical exam teaching device based on VR technology in accordance with another exemplary embodiment.
Fig. 3 is a flowchart illustrating a virtual charter teaching method based on VR technology according to another exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
First, an application scenario of the embodiment of the present invention will be described, and the physical examination is an important means for a doctor to clinically diagnose and treat a patient, and is a skill that each doctor must be skilled before becoming a doctor. Repeated training and consolidation is required throughout the medical learning process. How to form an effective automatic teaching solution under the condition of consuming teaching resources and instructor time as little as possible is a difficult problem of medical education. There are generally two types of existing training methods:
SP (standardized patient) protocol:
selecting qualified personnel, performing standardization and systematization training, and simulating clinical patients by means of performance, make-up, props and the like. The trainee performs simulated training through such simulated clinical patients. The disadvantages are that: training the SP requires a significant investment of capital and time, and training is costly. Moreover, because the simulated patient and the real patient have a large difference, all pathological characteristics cannot be reflected through performance and makeup, and the reality is not high.
Virtual patient protocol:
according to the scheme, a planar or three-dimensional patient model is displayed on a screen through a three-dimensional modeling technology, then optional physical examination operations are displayed on the screen, a user can click a certain part of the patient model through a mouse or a touch screen, the system provides an operation list containing operations of palpation, percussion, visual diagnosis, auscultation and the like of the part according to the part clicked by the user, and the user selects options from the operation list to obtain physical examination results. The defects are obvious, the operation interaction is weak, language communication and interaction are lacked, the immersion is not high, evaluation means for interactive operation and communication are lacked, the physical examination process training device is more suitable for physical examination process training, and the teaching purpose cannot be effectively achieved. In order to solve the above problems, the present invention provides a virtual physical examination teaching method and device based on VR technology.
Fig. 1 is a flowchart illustrating a virtual physical exam teaching method based on VR technology according to an exemplary embodiment. As shown in fig. 1, the virtual physical examination teaching method based on VR technology includes the following steps.
In step S11, a virtual scene and a virtual patient are constructed;
in step S12, displaying the virtual environment and the virtual patient to the trainer through the VR headset device, interacting with the trainer;
in step S13, the physical examination operation behavior of the trainer is recognized through the limb and gesture recognition module;
in step S14, recognizing the voice information of the trainer by the voice recognition device;
in step S15, controlling the virtual patient to perform feedback to the trainer according to the physical examination operation behavior and the voice information;
in step S16, whether the physical examination operation behavior and the voice information of the trainer meet the specifications or not is monitored in real time, and statistics and evaluation are performed on the physical examination operation behavior and the voice information to give a training result and guidance suggestion.
By the method, in the virtual physical examination teaching, a trainer can complete the same interactive operation and doctor-patient communication conversation as the real physical examination on the virtual patient model, and can intelligently detect the process and result data in the physical examination training. By the invention, a trainer can repeatedly drill various physical exams in a high-fidelity operation simulation environment, a real physical examination patient model is not needed, the resource cost of physical examination training is effectively reduced, the on-site supervision and evaluation of a guide can be separated, and the efficiency of physical examination teaching training is greatly improved.
Specifically, the constructing of the virtual scene includes:
the ward where the physical examination is located, medical equipment, furnishing appliances and physical examination instruments are constructed.
A virtual scene module: the virtual world is used for constructing physical examination, such as wards where physical examination is located, medical equipment, furnishings and instruments, physical examination instruments and the like.
The content of the part is mainly finished by prefabricating in three-dimensional modeling software such as 3DMAX, MAYA and the like, and is loaded into the device according to the preset target of the current training when the training is started.
Constructing a virtual patient includes: and constructing a patient model, constructing patient actions and constructing patient voice. The patient model comprises three parts, namely a grid, a map and a material, and is used for expressing the appearance of a real patient; the patient action is used for representing pathological characteristics and physiological response of the patient; wherein, the pathological features include: abdominal pain, hunchback, limping legs, and body position during physical examination; the physiological response comprises: knee jerk, cough, groaning, pain, etc.; the patient voice is used for simulating the phonation of the patient; wherein the medical characteristics of the patient's voice are adjusted based on different case data.
Namely: a virtual patient module: as the main subject of physical examination operations. The device has certain pathological characteristics, is used for simulating real clinical patients, and can simulate the actions, the voice, the expression and the like of the patients in the physical examination process. The virtual patient module mainly comprises: the patient model, the patient action and the patient voice. The patient model comprises three parts of a grid, a map and a material. To represent the appearance of a real patient. The actions of the patient are mainly used for reflecting the pathological features of the patient (such as abdominal pain, humpback, limping legs and the like), the body position in the body examination process, physiological reactions (such as knee jump, cough, groan, pain and the like). The voice of the patient is mainly used for simulating the voice of the patient, and the voice characteristics of the patient are different according to different case data, such as the old, children, men and women, and the like.
The virtual trainer: the current trainer bears objects in the virtual world. The method is mainly used for synchronizing the position and orientation data of the VR helmet equipment and the trainer state data recognized by the limb gesture recognition module. Through virtual trainers, virtual scenes, virtual patients and other virtual contents, a more real physical examination training teaching situation can be created.
Further, the VR device: the system is used for displaying the virtual scene and the virtual patient, and displaying the feedback information of all interaction results in the physical examination process. And positioning and synchronizing data such as real position, orientation and the like to a trainer in real time. VR devices are not limited to a particular product and model. In this embodiment, htc vive pro is used as VR equipment.
Specifically, the limb and gesture recognition device: the system is used for detecting and identifying the limb actions and the gestures of the trainer, and undertakes the synchronization of the limb actions and the hand actions of the trainer and the postures of the virtual trainer;
the limb and gesture recognition apparatus includes: a kinect limb gesture recognition device, and a lepemotion gesture motion recognition device. The speech recognition device: the system is responsible for translating words spoken by a trainer into characters, converting the characters into instruction entries and driving a virtual patient to make corresponding feedback;
wherein the feedback driving the virtual patient comprises: motion feedback and/or voice feedback.
Further, the virtual physical examination teaching method based on the VR technology further includes:
monitoring the physical examination operation behavior of the trainer, and judging whether the language content meets the flow specification and the professional specification of the physical examination; and corresponding prompts are made for physical examination operation behaviors and medical use which do not meet the standards.
Specifically, the step can be completed by a physical examination monitoring module and a physical examination evaluation module, wherein the physical examination monitoring module is used for monitoring the system state, such as physical examination operation behavior of a trainer, whether language content meets the flow specification and professional specification of physical examination, and correspondingly prompting operations which do not meet the specification.
The body-checking action is judged according to the recognition result of the limbs and the gestures, and the part operated on the virtual patient is further positioned through the collision detection between the virtual trainer and the virtual patient.
A physical examination evaluation module: recording and counting the detection result of the physical examination monitoring module and the physical examination operation behavior of the trainer, and giving detailed physical examination training statistics and detail analysis according to the evaluation rule of the current physical examination. According to the evaluation result, virtual instructor instruction opinions are given, so that a trainer can know the existing problems and how to improve, and the training result can be better achieved.
FIG. 2 is a block diagram illustrating a standardized patient-based interview training and evaluation device according to another exemplary embodiment. Referring to fig. 2, the present application provides a standardized patient-based interview training and evaluation device, comprising:
a construction module 21 for constructing a system, a virtual scene and a virtual patient;
the specific building module can refer to the relevant expressions in the virtual scene module, the virtual patient module and the virtual trainer in the above embodiment.
A display module 22 for displaying the virtual environment and the virtual patient to the trainer via the VR headset device;
in particular, the specific content of the display module may refer to the relevant representation of the VR device.
A recognition module 23, configured to recognize, via the limb and gesture recognition module: identifying physical examination operation behaviors of a trainer, and identifying voice information of the trainer through voice identification equipment;
a feedback module 24, configured to control the virtual patient to perform feedback to the trainer according to the physical examination operation behavior and the voice information;
and the evaluation module 25 is used for monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, counting and evaluating the physical examination operation behaviors and the voice information, and giving out training results and guidance suggestions.
The specific evaluation module refers to the related expressions of the physical examination monitoring module and the physical examination evaluation module.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Further, the present application provides a storage medium, wherein a preset program is stored in the storage medium, and when the preset program is executed, the method for standardized patient training based on speech recognition technology according to any embodiment of the present application is implemented.
Fig. 3 is a flowchart illustrating a virtual charter teaching method based on VR technology according to another exemplary embodiment. The virtual physical examination teaching method based on VR technology provided in this application is further described with reference to fig. 3.
In step S301, a virtual scene and a virtual patient are constructed.
In step S302, the virtual physical examination environment and the patient are displayed by the VR headset, the position orientation of the trainer is synchronized, and the like.
In step S303, the preparation work of virtual instruments, medicines, etc. before physical examination is completed by recognizing the physical examination operation behavior of the trainer through the body and gesture recognition device.
In step S304, the virtual patient understands the question of the trainer and makes a corresponding voice feedback according to the voice recognition module, thereby completing the doctor-patient communication training before physical examination.
In step S305, the doctor physical examination command is synchronized by the voice recognition module, and the virtual patient completes the coordination required by the physical examination.
In step S306, the recognition results of the limb and the gesture recognition device are synchronized to the virtual training object, and the virtual training object is used to complete the operation interaction with the virtual patient and the virtual instrument.
In step S307, the physical examination operation of the trainee, the voice command and the doctor-patient communication content are monitored in real time.
In step S308, statistics and evaluation are performed on the operation result and the voice communication content, and a training result and guidance suggestion are given.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A virtual physical examination teaching method based on VR technique is characterized by comprising the following steps:
constructing a virtual scene and a virtual patient;
displaying the virtual environment and the virtual patient to the trainer through VR helmet equipment, and interacting with the trainer;
identifying the physical examination operation behavior of the trainer through a limb and gesture identification module;
recognizing the voice information of the trainer through voice recognition equipment;
controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information;
and monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, and counting and evaluating the physical examination operation behaviors and the voice information to give training results and guidance suggestions.
2. The VR technology based virtual physical exam teaching method of claim 1, wherein constructing a virtual scene comprises:
the ward where the physical examination is located, medical equipment, furnishing appliances and physical examination instruments are constructed.
3. The VR technology based virtual physical exam teaching method of claim 1, wherein constructing a virtual patient comprises:
constructing a patient model, constructing patient actions and constructing patient voices;
the patient model comprises three parts, namely a grid, a map and a material, and is used for expressing the appearance of a real patient;
the patient action is used for representing pathological characteristics and physiological response of the patient; wherein, the pathological features include: abdominal pain, hunchback, limping legs, and body position during physical examination;
the physiological response comprises: knee jump, cough, groaning, and pain;
the patient voice is used for simulating the phonation of the patient; wherein the medical characteristics of the patient's voice are adjusted based on different case data.
4. The VR technology based virtual physical education method of claim 1, wherein the VR device is an htc vive pro VR device for displaying virtual scenes and virtual patients, and feedback information display of all interaction results during physical examination, locating and synchronizing real position and orientation data to trainers in real time.
5. The VR technology based virtual physical examination teaching method of claim 1, wherein the limb and gesture recognition device: the system is used for detecting and identifying the limb actions and the gestures of the trainer, and undertakes the synchronization of the limb actions and the hand actions of the trainer and the postures of the virtual trainer;
the virtual training object is a bearing object of a current trainer in the virtual world; the virtual physical examination training teaching method is mainly used for synchronizing the position and orientation data of VR equipment and the state data of trainers identified by limb and gesture recognition equipment, and creates a real physical examination training teaching situation through virtual trainers, virtual scenes and virtual patients.
6. The VR technology based virtual physical examination teaching method of claim 1, wherein the limb and gesture recognition device comprises: a kinect limb gesture recognition device, and a lepemotion gesture motion recognition device.
7. The VR technology based virtual physical exam teaching method of claim 1, wherein the speech recognition device: the system is responsible for translating words spoken by a trainer into characters, converting the characters into instruction entries and driving a virtual patient to make corresponding feedback;
wherein the feedback of the virtual patient comprises: motion feedback and/or voice feedback.
8. The VR technology based virtual physical exam teaching method of claim 1, further comprising:
monitoring the physical examination operation behavior of the trainer, and judging whether the language content meets the flow specification and the professional specification of the physical examination; and corresponding prompts are made for physical examination operation behaviors and medical use which do not meet the standards.
9. The utility model provides a virtual physical examination teaching equipment based on VR technique which characterized in that includes:
the building module is used for building a virtual scene and a virtual patient;
the display module is used for displaying a virtual environment and a virtual patient to a trainer through VR helmet equipment, and the display module is used for displaying the virtual environment and the virtual patient to the trainer through limbs and a gesture recognition module: identifying physical examination operation behaviors of a trainer, and identifying voice information of the trainer through voice identification equipment;
the feedback module is used for controlling the virtual patient to feed back to a trainer according to the physical examination operation behavior and the voice information;
and the evaluation module is used for monitoring whether the physical examination operation behaviors and the voice information of the trainer meet the standards in real time, counting and evaluating the physical examination operation behaviors and the voice information and giving training results and guidance suggestions.
10. A storage medium having a predetermined program stored therein, wherein the predetermined program is executed to implement the standardized patient training method based on the voice recognition technology according to any one of claims 1 to 8.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010494347 | 2020-06-03 | ||
CN2020104943471 | 2020-06-03 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111833682A true CN111833682A (en) | 2020-10-27 |
Family
ID=72900181
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010640933.2A Pending CN111833682A (en) | 2020-06-03 | 2020-07-06 | Virtual physical examination teaching method and device based on VR technology |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111833682A (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113205720A (en) * | 2021-06-03 | 2021-08-03 | 深圳妙创医学技术有限公司 | Multi-terminal situational training system |
CN113344335A (en) * | 2021-05-11 | 2021-09-03 | 深圳市医养护教育服务有限公司 | Health assessment teaching system and method |
CN117238188A (en) * | 2023-11-13 | 2023-12-15 | 成都泰盟软件有限公司 | AR equipment-based doctor-patient communication training method, device, equipment and medium |
CN117541444A (en) * | 2023-12-04 | 2024-02-09 | 新励成教育科技股份有限公司 | Interactive virtual reality talent expression training method, device, equipment and medium |
CN117541445A (en) * | 2023-12-11 | 2024-02-09 | 新励成教育科技股份有限公司 | Talent training method, system, equipment and medium for virtual environment interaction |
CN117854341A (en) * | 2023-12-07 | 2024-04-09 | 广东工业大学 | General rule-based doctor-patient communication training assessment method based on virtual reality technology |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106295198A (en) * | 2016-08-16 | 2017-01-04 | 南京美卡数字科技有限公司 | Standard patient system and method based on VR virtual reality technology |
CN107067862A (en) * | 2017-03-31 | 2017-08-18 | 张小来 | A kind of virtual emulation hospital care teaching training system and its construction method |
CN107705826A (en) * | 2017-09-30 | 2018-02-16 | 深圳前海卓岳科技发展有限公司 | A kind of virtual clinic system based on VR |
US20180052512A1 (en) * | 2016-08-16 | 2018-02-22 | Thomas J. Overly | Behavioral rehearsal system and supporting software |
CN109147529A (en) * | 2018-09-29 | 2019-01-04 | 上海嘉奕医学科技有限公司 | The standard patient simulation system of achievable humane examination based on artificial intelligence |
CN109285225A (en) * | 2018-10-15 | 2019-01-29 | 东北大学 | A kind of method for building up of the virtual reality auxiliary operation based on medical image |
CN110580101A (en) * | 2019-09-19 | 2019-12-17 | 中国医学科学院北京协和医院 | Interactive medical system based on virtual reality |
CN110728858A (en) * | 2019-11-20 | 2020-01-24 | 拓普虚拟现实技术研究院(江苏)有限公司 | Post simulation training system based on VR technology |
-
2020
- 2020-07-06 CN CN202010640933.2A patent/CN111833682A/en active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106295198A (en) * | 2016-08-16 | 2017-01-04 | 南京美卡数字科技有限公司 | Standard patient system and method based on VR virtual reality technology |
US20180052512A1 (en) * | 2016-08-16 | 2018-02-22 | Thomas J. Overly | Behavioral rehearsal system and supporting software |
CN107067862A (en) * | 2017-03-31 | 2017-08-18 | 张小来 | A kind of virtual emulation hospital care teaching training system and its construction method |
CN107705826A (en) * | 2017-09-30 | 2018-02-16 | 深圳前海卓岳科技发展有限公司 | A kind of virtual clinic system based on VR |
CN109147529A (en) * | 2018-09-29 | 2019-01-04 | 上海嘉奕医学科技有限公司 | The standard patient simulation system of achievable humane examination based on artificial intelligence |
CN109285225A (en) * | 2018-10-15 | 2019-01-29 | 东北大学 | A kind of method for building up of the virtual reality auxiliary operation based on medical image |
CN110580101A (en) * | 2019-09-19 | 2019-12-17 | 中国医学科学院北京协和医院 | Interactive medical system based on virtual reality |
CN110728858A (en) * | 2019-11-20 | 2020-01-24 | 拓普虚拟现实技术研究院(江苏)有限公司 | Post simulation training system based on VR technology |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113344335A (en) * | 2021-05-11 | 2021-09-03 | 深圳市医养护教育服务有限公司 | Health assessment teaching system and method |
CN113205720A (en) * | 2021-06-03 | 2021-08-03 | 深圳妙创医学技术有限公司 | Multi-terminal situational training system |
CN117238188A (en) * | 2023-11-13 | 2023-12-15 | 成都泰盟软件有限公司 | AR equipment-based doctor-patient communication training method, device, equipment and medium |
CN117541444A (en) * | 2023-12-04 | 2024-02-09 | 新励成教育科技股份有限公司 | Interactive virtual reality talent expression training method, device, equipment and medium |
CN117541444B (en) * | 2023-12-04 | 2024-03-29 | 新励成教育科技股份有限公司 | Interactive virtual reality talent expression training method, device, equipment and medium |
CN117854341A (en) * | 2023-12-07 | 2024-04-09 | 广东工业大学 | General rule-based doctor-patient communication training assessment method based on virtual reality technology |
CN117541445A (en) * | 2023-12-11 | 2024-02-09 | 新励成教育科技股份有限公司 | Talent training method, system, equipment and medium for virtual environment interaction |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111833682A (en) | Virtual physical examination teaching method and device based on VR technology | |
Rhienmora et al. | Intelligent dental training simulator with objective skill assessment and feedback | |
CN104271066B (en) | Mixed image with the control without hand/scene reproduction device | |
US11410564B2 (en) | System and method for creating immersive interactive application | |
CN107527542B (en) | Percussion training system based on motion capture | |
US20080160488A1 (en) | Trainee-as-mentor education and training system and method | |
US20080050711A1 (en) | Modulating Computer System Useful for Enhancing Learning | |
CN111861161B (en) | Question training evaluation method and device based on standardized patient | |
Ferracani et al. | Natural and virtual environments for the training of emergency medicine personnel | |
CN101739879A (en) | Real-time pose capture-based surgery simulation system, method and graph rendering module | |
CN111834019A (en) | Standardized patient training method and device based on voice recognition technology | |
Forrest et al. | Healthcare simulation at a glance | |
US20080137877A1 (en) | Subject actuated system and method for simulating normal and abnormal medical conditions | |
CN116778771A (en) | First aid training and checking system | |
CN107633724B (en) | Auscultation training system based on motion capture | |
Ochs et al. | A semi-autonomous system for creating a human-machine interaction corpus in virtual reality: Application to the ACORFORMed system for training doctors to break bad news | |
Coles et al. | Haptic palpation for the femoral pulse in virtual interventional radiology | |
Deutsch et al. | Formative evaluation and preliminary findings of a virtual reality telerehabilitation system for the lower extremity | |
Lai et al. | Improving the skills training by mixed reality simulation learning: A pilot case study of nasogastric tube care | |
Tai et al. | Development of haptic-enabled virtual reality simulator for video-assisted thoracoscopic right upper lobectomy | |
CN101840640B (en) | Interactive voice response system and method | |
Gilardi et al. | Nursing XR–A VR application to teach decision making to student nurses | |
Ndhaief et al. | Towards the Digitalization of the Healthcare Learning Process using 3D experience | |
RU2799123C1 (en) | Method of learning using interaction with physical objects in virtual reality | |
CN117789563B (en) | AI and MR technology based virtual-real fusion simulation medical teaching method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201027 |