WO2023050522A1 - 交互方法、终端设备及可读存储介质 - Google Patents
交互方法、终端设备及可读存储介质 Download PDFInfo
- Publication number
- WO2023050522A1 WO2023050522A1 PCT/CN2021/128069 CN2021128069W WO2023050522A1 WO 2023050522 A1 WO2023050522 A1 WO 2023050522A1 CN 2021128069 W CN2021128069 W CN 2021128069W WO 2023050522 A1 WO2023050522 A1 WO 2023050522A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- terminal
- electrical signal
- parameters
- myoelectric
- user
- Prior art date
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 156
- 238000000034 method Methods 0.000 title claims abstract description 80
- 230000009471 action Effects 0.000 claims abstract description 53
- 230000003183 myoelectrical effect Effects 0.000 claims description 92
- 230000002452 interceptive effect Effects 0.000 claims description 31
- 238000002567 electromyography Methods 0.000 claims description 23
- 230000008921 facial expression Effects 0.000 claims description 9
- 230000000694 effects Effects 0.000 abstract description 22
- 230000008569 process Effects 0.000 abstract description 20
- 230000014509 gene expression Effects 0.000 abstract description 6
- 230000035807 sensation Effects 0.000 abstract description 2
- 230000000875 corresponding effect Effects 0.000 abstract 2
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 239000011521 glass Substances 0.000 description 4
- 230000015541 sensory perception of touch Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 210000003414 extremity Anatomy 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 208000029549 Muscle injury Diseases 0.000 description 2
- 210000002683 foot Anatomy 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 210000003205 muscle Anatomy 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000002232 neuromuscular Effects 0.000 description 2
- 230000036982 action potential Effects 0.000 description 1
- 210000003423 ankle Anatomy 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 210000003169 central nervous system Anatomy 0.000 description 1
- 230000002301 combined effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000002161 motor neuron Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 210000000278 spinal cord Anatomy 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
Definitions
- the present application relates to the field of virtual/augmented reality interaction technology, in particular to an interaction method, terminal equipment and a readable storage medium.
- the main purpose of this application is to provide an interaction method, terminal equipment and readable storage medium, aiming to solve the different purposes of remote interaction through voice interaction only through communication, the interaction method is single and the interaction effect is poor The problem.
- the present application provides an interaction method, the interaction method is applied to the first terminal, and the interaction method includes:
- the first terminal receives the electrical signal parameter sent by the second terminal, and the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal;
- the step of the first terminal receiving the electrical signal parameters sent by the second terminal includes:
- the first terminal receives the sent encrypted data
- adjusting the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyographic signal is transmitted to the user of the first terminal includes:
- the myoelectric signal includes at least one of a facial expression signal and a body signal.
- the present application also provides an interaction method, the interaction method is applied to the second terminal, and the interaction method includes:
- the second terminal acquires the myoelectric signal collected by the myoelectric sensor
- the step of sending the electrical signal parameters to the first terminal includes:
- the electrical signal parameters are sent to the first terminal, so that the first terminal adjusts the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the myoelectric signal is transmitted
- the steps for the user of said first terminal include:
- the electrodes connected to the user of the first terminal are communicated to the user of the first terminal.
- the interaction device includes:
- the first sending module is used for the first terminal to receive the electrical signal parameter sent by the second terminal, and the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal;
- An output module configured to adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that the user connected to the electrodes restores the action corresponding to the myoelectric signal;
- the first acquisition module is used for the second terminal to acquire the myoelectric signal collected by the myoelectric sensor
- a second acquisition module configured to acquire electrical signal parameters corresponding to the myoelectric signal
- the second sending module is configured to send the electrical signal parameters to the first terminal, so that the first terminal can adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that the The user connected to the electrodes restores the action corresponding to the myoelectric signal.
- the present application also provides a terminal device, the terminal device includes: a memory, a processor, and an interactive program stored in the memory and operable on the processor, the interactive program When executed by the processor, each step of the above-mentioned interaction method is realized.
- the present application also provides a readable storage medium, on which an interactive program is stored, and when the interactive program is executed by the processor, each of the above-mentioned interactive methods can be realized. step.
- the first terminal receives the electrical signal parameters sent by the second terminal, based on the electrical signal parameters are converted from the myoelectric signals collected by the myoelectric sensor in the second terminal equipment, the first A terminal adjusts the working parameters of the first terminal according to the electrical signal parameters, and transmits the action corresponding to the electromyographic signal to the user of the first terminal, so that the user can truly perceive the facial expression or body movement of the other party during the remote interaction process, improving the interaction effect and Make the interaction mode more diverse.
- FIG. 1 is a schematic structural diagram of a terminal device involved in various embodiments of the interaction method of the present application
- FIG. 2 is a schematic flowchart of a first embodiment in which an interaction method of the present application is applied to a first terminal;
- FIG. 3 is a schematic flowchart of a first embodiment in which an interaction method of the present application is applied to a second terminal;
- FIG. 4 is a schematic flowchart of a second embodiment of the interaction method of the present application.
- FIG. 5 is a schematic flowchart of a third embodiment of the interaction method of the present application.
- FIG. 6 is a schematic diagram of the first module of the interaction device of the present application.
- FIG. 7 is a schematic diagram of a second module of the interaction device of the present application.
- FIG. 1 is a schematic structural diagram of a terminal device involved in various embodiments of the interaction method of the present application.
- the terminal device may be implemented in various forms.
- the terminal device in this application may include a terminal such as a mobile phone, a tablet computer, a notebook computer, and a handheld computer; it may also be a wearable device such as VR glasses and AR glasses.
- the terminal device may include: a memory 101 , a processor 102 and a communication unit 103 .
- a memory 101 a processor 102 and a communication unit 103 .
- the processor 102 is the control center of the terminal device, and the processor 102 executes the interaction program stored in the memory 101 to implement the steps of the embodiments of the interaction method of the present application.
- the communication unit 103 establishes data communication with other terminal devices through a network protocol (the data communication may be IP communication or Bluetooth channel), so as to realize data transmission with other terminal devices.
- the terminal device when the terminal device is a first terminal, When the other terminal device is the second terminal, the first terminal receives the electrical signal parameters sent by the second terminal, or, when the terminal device is the second terminal and other terminal devices are the first terminal, the second terminal obtains the parameters collected by the myoelectric sensor. Myoelectric signal; acquiring electrical signal parameters corresponding to the myoelectric signal; sending the electrical signal parameters to the first terminal.
- the terminal device may further include a display unit 104, the display unit 104 includes a display panel, and the display panel may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), etc. , which is used to output and display the interface browsed by the user.
- a liquid crystal display Liquid Crystal Display, LCD
- OLED Organic Light-Emitting Diode
- the terminal device may be a first terminal or a second terminal.
- the terminal device includes a first terminal and a second terminal.
- Data transmission and communication can be performed between the first terminal and the second terminal through the communication unit, so as to realize remote interaction between the first terminal and the second terminal.
- the terminal device when the terminal device is the first terminal, the first terminal is electrically connected to electrodes, wherein the electrodes are connected to the user in contact, so that the first terminal adjusts the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so as to The myoelectric signal corresponding to the electrical signal parameter is transmitted to the user through the electrode, and the user connected to the electrode can feel the action corresponding to the myoelectric signal through nerve transmission; when the terminal device is the second terminal, the second terminal can obtain the data collected by the myoelectric sensor EMG signal.
- the terminal device is the first terminal
- the following steps are implemented when the interactive program in the memory 101 of the first terminal is executed by the processor 102:
- the first terminal receives the electrical signal parameter sent by the second terminal, and the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal;
- the working parameters of the first terminal may include electrical signal parameters, mechanical parameters and other parameters corresponding to various devices that can transmit the action corresponding to the myoelectric signal to the user of the first terminal.
- adjust the vibration signal parameters of the vibration motor corresponding to the electrodes of the first terminal according to the electrical signal parameters When the terminal is a user, adjust the vibration signal parameters of the vibration motor corresponding to the electrodes of the first terminal according to the electrical signal parameters.
- the terminal device When the terminal device is the second terminal, the following steps are implemented when the interactive program in the memory 101 of the second terminal is executed by the processor 102:
- the second terminal acquires the myoelectric signal collected by the myoelectric sensor
- this interaction method can be applied to various remote interaction application scenarios, such as remote conferences, remote medical consultations, and remote assistance guidance.
- the application scenario of remote interaction may be a remote conference scenario.
- participants can wear AR glasses for remote conferences, so that the images seen by the participants on the AR glasses can be transmitted in real time to the remote terminal control system located outside the conference site, so that the far-end Participants can share conference content synchronously.
- the holder of the first terminal is the first participant
- the holder of the second terminal is the second participant.
- the first terminal receives the electrical signal parameter sent by the second terminal, based on the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal device, the first terminal adjusts the working parameter of the first terminal according to the electrical signal parameter, so as to The action corresponding to the electromyography signal is transmitted to the user of the first terminal, so that the user can truly perceive the facial expression or body movement of the other party during the remote interaction process, and improve the interaction effect of the remote conference.
- the application scenario of remote interaction may be a remote assistance and guidance scenario.
- the holder of the first terminal is the assisted person of the remote assistance
- the holder of the second terminal is the auxiliary guide of the remote assistance.
- the steps of the interaction method of the first terminal can also be applied to the second terminal
- the steps of the interaction method of the second terminal can be applied to the first terminal, so as to adjust the electrical signal of the first terminal according to the electrical signal parameters at the first terminal.
- the working parameters of the connected electrodes so that after the user connected to the electrodes restores the action corresponding to the myoelectric signal, so that the holder of the first terminal can realize the first terminal to acquire muscle
- the second terminal receives the electrical signal parameters sent by the first terminal, and the electrical signal parameters are obtained through conversion of the myoelectric signals collected by the myoelectric sensor in the first terminal; according to the electrical signal parameters, the parameters of the electrodes electrically connected to the second terminal are adjusted.
- the holder of the first terminal is a doctor who visits a doctor remotely
- the holder of the second terminal is a patient who visits a doctor remotely.
- the doctor can judge the degree of muscle damage of the patient by instructing the patient to move his arms, legs and feet, such as raising the arm, kicking, squatting, and moving the ankle.
- the second terminal can obtain the electromyographic signal collected by the electromechanical signal when the patient is active, obtain the electrical signal parameters corresponding to the electromyographic signal, and send the electrical signal parameters to the first terminal; correspondingly, The first terminal receives the electrical signal parameters sent by the second terminal, and adjusts the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the myoelectric signal is transmitted to the user of the first terminal, that is, the doctor, so that the doctor can Real perception and experience of the activities of the patients during activities, so as to better guide the patients to seek medical treatment and improve the effect of remote medical consultation.
- the present application provides an interaction method, please refer to FIG. 2 , which is a schematic flowchart of a first embodiment in which the interaction method of the present application is applied to a first terminal.
- the interactive method includes the following steps:
- Step S10 the first terminal receives the electrical signal parameter sent by the second terminal, and the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal;
- Step S20 adjusting the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyographic signal is transmitted to the user of the first terminal.
- the electrical signal parameters are obtained through conversion of the myoelectric signal collected by the myoelectric sensor in the second terminal.
- Electrical signal parameters include current and/or voltage.
- the electromyographic signal conversion to obtain electrical signal parameters can be based on the corresponding relationship between the electromyographic signal and the electrical signal parameters.
- the electromyographic signal corresponding to the electromyographic signal can be obtained Signal parameters, this step is not limited in this embodiment.
- the myoelectric signal may be at least one of facial expression signals and body movement signals.
- the myoelectric sensor can be worn on the user's limb or in contact with the user's face or limb, so as to detect the myoelectric signal accompanying the user's limb movement.
- Electromyography originates from motor neurons in the spinal cord as part of the central nervous system. It is the sum of action potentials emitted by many motor units that the electrodes touch, reflecting the activity and functional state of neuromuscular.
- surface electromyography is a signal generated by the combined effect of superficial muscle electrical signals and nerve trunk electrical activities on the skin surface, which contains rich neuromuscular activity information.
- the electrode or the mechanical structure can be electrically connected to the first terminal, and when the electrode is in contact with the user's skin, the action corresponding to the myoelectric signal can be transmitted to the user of the first terminal, or the electrical The signal is converted into a corresponding mechanical signal, for example, an electrical signal is converted into a vibration signal of a vibration motor.
- the action corresponding to the myoelectric signal can be transmitted to the user of the first terminal.
- adjusting the working parameters of the first terminal according to the electrical signal parameters includes adjusting the working parameters of the electrodes or mechanical structures electrically connected to the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyographic signal passes through the user of the first terminal.
- the connected electrodes or mechanical structures are communicated to the user of the first terminal.
- step S20 includes:
- the working parameter of the electrode refers to the current passing through the electrode, and/or, the voltage applied to the electrode. Adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, that is, adjust the working parameters of the electrodes electrically connected to the first terminal to the electrical signal parameters, and control the output of the electrodes according to the electrical signal parameters, so that the electrodes follow the electrical parameters.
- the signal parameters are run, and then the working parameters of the electrodes electrically connected to the first terminal are adjusted according to the electrical signal parameters, so that the electromyographic signals corresponding to the electrical signal parameters are input to the user connected to the electrodes through the electrodes, so that the electromyographic signals correspond to
- the action of the first terminal is transmitted to the user of the first terminal through the electrodes connected to the user of the first terminal, so that the action corresponding to the myoelectric signal is transmitted to the user of the first terminal.
- step S10 includes:
- the first terminal receives the encrypted data sent by the second terminal;
- the second terminal encrypts the electrical signal parameters to obtain encrypted data; sends the encrypted data to the first terminal, receives the encrypted data sent by the second terminal at the first terminal, and decrypts the encrypted data to obtain
- the electrical signal parameters are obtained to improve the security of data transmission between the first terminal and the second terminal.
- the first terminal receives the electrical signal parameters sent by the second terminal, based on the electrical signal parameters are converted from the myoelectric signals collected by the myoelectric sensor in the second terminal device, and the first terminal receives the electrical signal parameters according to the electrical signal parameters.
- parameters to adjust the working parameters of the first terminal so that the action corresponding to the EMG signal is transmitted to the user of the first terminal, so that the user can truly perceive the facial expression or body movement of the other party during the remote interaction process, improve the interaction effect and make the interaction
- the methods are more diverse.
- FIG. 3 is a schematic flowchart of a first embodiment in which the interaction method of the present application is applied to the second terminal.
- the interactive method includes the following steps:
- Step S30 the second terminal acquires the myoelectric signal collected by the myoelectric sensor
- Step S40 acquiring electrical signal parameters corresponding to the electromyography signal
- Step S50 sending the electrical signal parameters to the first terminal, so that the first terminal can adjust the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the myoelectric signal is transmitted to A user of the first terminal.
- the second terminal obtains the myoelectric signal collected by the myoelectric sensor, and the myoelectric sensor can be directly connected to the second terminal, so as to obtain the myoelectric signal collected by the myoelectric sensor; other terminal devices, so that after obtaining the myoelectric signals collected by the myoelectric sensors through other terminal devices, the electrical signal parameters corresponding to the myoelectric signals are sent to the second terminal, which is not limited in this embodiment.
- the electrical signal parameters include current and/or voltage. Obtaining the electrical signal parameters corresponding to the electromyographic signal can be based on the correspondence between the electromyographic signal and the electrical signal parameters. On the premise of obtaining the electromyographic signal, the electromyographic signal can be obtained Electrical signal parameters corresponding to electromyographic signals.
- the first terminal adjusts the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that the facial expressions or body movements of the other party can be used to facilitate the user to truly perceive the remote interaction process.
- the other party's facial expressions or body movements improve the interaction effect and make the interaction methods more diverse.
- the method of sending the electrical signal parameters to the first terminal may be a wireless communication transmission method, such as sending the electrical signal parameters to the first terminal based on WIFI or BT or a cellular network, so as to adapt to the remote interaction scenario, or
- the wired transmission mode is adopted, and there is no limitation on this.
- the second terminal obtains the myoelectric signal collected by the myoelectric sensor, and sends the electrical signal parameter corresponding to the myoelectric signal to the first terminal, and the first terminal receives the electrical signal parameter sent by the second terminal, and according to the electrical signal parameter Adjust the working parameters of the first terminal, so that the action corresponding to the electromyographic signal is transmitted to the user of the first terminal.
- the myoelectric sensor acquires the myoelectric signal of the target site from the second terminal, and after sending the electrical signal parameter corresponding to the myoelectric signal to the first terminal, correspondingly, the first terminal receives the electrical signal parameter sent by the second terminal Adjusting the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyographic signal is transmitted to the user of the first terminal at a site corresponding to the target site.
- step S50 includes:
- EMG signals are a key type of biological signal.
- EMG signals can be The corresponding electrical signal parameters are encrypted to obtain encrypted data, and then the encrypted data is sent to the first terminal.
- the encryption algorithm for encrypting the electrical signal parameters corresponding to the EMG signal may be a symmetric encryption algorithm or an asymmetric encryption algorithm, which is not limited in this embodiment.
- step S50 includes:
- the electrodes connected to the user of the first terminal are communicated to the user of the first terminal.
- the second terminal obtains the myoelectric signal collected by the myoelectric sensor, and sends the electrical signal parameters corresponding to the myoelectric signal to the first terminal, so that the first terminal can adjust the first terminal according to the electrical signal parameters.
- the working parameters of the terminal transmit the action corresponding to the EMG signal to the user of the first terminal, so that the interacting user can truly feel the other party's expression or body movement during the remote interaction process, which improves the interaction effect and makes the interaction mode more diverse.
- step S20 includes:
- Step S21 receiving the interaction information sent by the second terminal
- Step S22 outputting the interaction information, and adjusting the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyography signal is transmitted to the user of the first terminal.
- the interaction information includes at least one of voice interaction information and image interaction information.
- the voice interaction information refers to voice information obtained during remote interaction.
- the voice interaction information is collected through the microphone in the second terminal or the microphone connected to the second terminal.
- the image interaction information refers to the information obtained during the remote interaction based on the annotations and annotations of the operation image to realize the interaction between the two parties.
- the operation image refers to an operation screen shared and interacted by both interacting parties when performing remote interaction.
- the first terminal receives the interaction information sent by the second terminal, through the interaction information, so that the user can learn the interaction information auditorily and/or visually, and at the same time adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that The user connected to the electrode restores the action corresponding to the EMG signal, which can facilitate the user to truly perceive the tactile sensations such as the expression or body movements of the other party during the remote interaction process, and improve the remote interaction effect.
- step S21 includes:
- step S21 is executed.
- the first terminal and the second terminal adopt AR product equipment, so that the holder wearing the first terminal and the holder of the second terminal can share the same operation screen collaboratively,
- the first terminal sends a remote interaction request to the second terminal, it acquires the current operation image to be interactively shared in real time, and sends the operation image to the second terminal, so as to share the same operation screen with the first terminal and the second terminal in real time, which is convenient
- the holder of the second terminal realizes the remote interaction based on the operation screen, and at the same time improves the interaction effect of the remote interaction, and then performs step S21. In addition, it enriches the application scenarios and application experience of AR product equipment.
- acquiring the operation image currently to be interactively shared in real time may acquire the operation image currently to be interactively shared in real time through a camera device disposed on the first terminal.
- the sending of the remote interaction request from the first terminal to the second terminal may be triggered by the first terminal receiving a remote interaction instruction or detecting a touch operation corresponding to the remote interaction, so as to trigger the first terminal to send the remote interaction request to the second terminal.
- the technical solution disclosed in this embodiment while outputting the received interaction information sent by the second terminal, such as voice interaction information and/or image interaction information, it is convenient for the user to hear and/or visually learn the action information of the remote interaction , adjust the working parameters of the first terminal according to the electrical signal parameters, so that the action corresponding to the electromyographic signal is transmitted to the user of the first terminal, which can facilitate the user to truly perceive the facial expression or body movement of the other party during the remote interaction process, thus facilitating the user Comprehensively learn the tactile senses such as the expression or body movements of the other party during the remote interaction process from the auditory, visual and tactile senses, so as to improve the effect of remote interaction.
- the tactile senses such as the expression or body movements of the other party during the remote interaction process from the auditory, visual and tactile senses, so as to improve the effect of remote interaction.
- step S50 includes:
- Step S51 acquiring interaction information
- Step S52 sending the interaction information and the electrical signal parameters to the first terminal, so as to output the interaction information through the first terminal, and adjust the working parameters of the first terminal according to the electrical signal parameters, and transmitting the action corresponding to the electromyography signal to the user of the first terminal.
- the interaction information includes at least one of voice interaction information and image interaction information.
- the voice interaction information refers to voice information obtained during remote interaction.
- the voice interaction information is collected through a microphone in the second terminal or a microphone connected to the second terminal.
- the image interaction information refers to the information obtained during the remote interaction based on the annotations and annotations of the operation image to realize the interaction between the two parties.
- step S51 includes:
- the interaction information includes at least one of voice interaction information and image interaction information
- the image interaction information is based on the The above operation image is obtained by receiving the annotation instruction.
- the first terminal and the second terminal adopt AR product equipment, so that the holder wearing the first terminal and the holder of the second terminal can cooperate and share the same operation screen.
- the operation image currently to be interactively shared is acquired in real time, and the operation image is sent to the second terminal, and the second terminal receives the remote interaction request and the operation image sent by the first terminal,
- the second terminal and the first terminal share the same operation screen, that is, the operation image, and obtain interactive information based on the same operation screen, so that the holder of the first terminal and the holder of the second terminal can hear, Visual and tactile communication to achieve more accurate remote interaction.
- the image interaction information is obtained based on the operation image receiving an annotation instruction, and the operation image received by the second terminal from the first terminal can be realized by the holder of the second terminal through HoloLens to remotely interact with the wearer on the operation Images are annotated with annotations.
- HoloLens is a MR head-mounted display, that is, a mixed reality head-mounted display.
- the second terminal acquires interactive information such as voice interactive information and/or image interactive information, and simultaneously sends the interactive information and electrical signal parameters to the first terminal, so that the interactive information can be output through the first terminal. It is convenient for users to learn the action information of remote interaction from hearing and/or vision, and adjust the working parameters of the first terminal according to the electrical signal parameters, so that the user of the first terminal can feel and restore the action corresponding to the electromyography signal, which is convenient for interactive users It can truly feel the tactile sense of the other party's expressions or body movements during the remote interaction process, improving the interaction effect and making the interaction methods more diverse.
- FIG. 6 is a schematic diagram of the first module of the interaction device of the present application.
- the present application also proposes an interaction device 100, which includes:
- the first sending module 110 is used for the first terminal to receive the electrical signal parameter sent by the second terminal, and the electrical signal parameter is obtained by converting the myoelectric signal collected by the myoelectric sensor in the second terminal;
- the output module 120 is configured to adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that the user connected to the electrodes restores the action corresponding to the electromyographic signals.
- FIG. 7 is a schematic diagram of a second module of the interaction device of the present application.
- the present application also proposes an interaction device 200, which includes:
- the first acquiring module 210 is used for the second terminal to acquire the myoelectric signal collected by the myoelectric sensor;
- the second acquisition module 220 is configured to acquire electrical signal parameters corresponding to the electromyography signal
- the second sending module 230 is configured to send the electrical signal parameters to the first terminal, so that the first terminal can adjust the working parameters of the electrodes electrically connected to the first terminal according to the electrical signal parameters, so that the The user connected to the electrodes restores the action corresponding to the myoelectric signal.
- the present application also proposes a terminal device.
- the terminal device includes: a memory, a processor, and an interactive program stored in the memory and operable on the processor.
- an interactive program stored in the memory and operable on the processor.
- the present application also proposes a readable storage medium, on which an interactive program is stored, and when the interactive program is executed by a processor, the steps of the interactive method described in any of the above embodiments are implemented.
- the embodiments of the terminal device and the readable storage medium provided in this application include all the technical features of the above-mentioned embodiments of the interaction method. Let me repeat.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Animal Behavior & Ethology (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Dermatology (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Neurosurgery (AREA)
- Human Computer Interaction (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
本申请公开了一种交互方法，该方法包括：第一终端接收第二终端发送的电信号参数，电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；根据电信号参数调整第一终端的工作参数，以使肌电信号对应的动作传递给第一终端的用户。本申请还提供一种终端设备及可读存储介质。本申请的交互方法通过第一终端接收第二终端发送的电信号参数，基于电信号参数是第二终端设备中肌电传感器采集的肌电信号转换得到，第一终端根据电信号参数调整第一终端的工作参数，以使得第一终端的用户可感受并还原肌电信号对应的动作，便于交互的用户能真实感受远程交互过程中对方的表情或肢体动作等触觉，提升交互效果且使得交互方式更加多样化。
Description
本申请要求于2021年9月28日提交中国专利局、申请号为202111149091.1、发明名称为“交互方法、终端设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本申请涉及虚拟/增强现实交互技术领域，尤其涉及一种交互方法、终端设备及可读存储介质。
目前，在进行远程交互如进行远程会议、远程就诊或者远程辅助等场景时，仅仅通过语音交互方式以沟通交流，达到远程交互的不同目的，交互方式单一且交互效果差，举例来说，在远程会议交互时，参会人员可远程连线，然而，如果交互双方只能通过语言进行沟通，则使交互过程的体验形式单一，交互效果较差。
上述内容仅用于辅助理解本申请的技术方案，并不代表承认上述内容是现有技术。
发明内容
本申请的主要目的在于提供一种交互方法、终端设备及可读存储介质，旨在解决在进行远程交互时仅仅通过语音交互方式以沟通交流达到远程交互的不同目的，交互方式单一且交互效果差的问题。
为实现上述目的，本申请提供一种交互方法，交互方法应用于第一终端，所述交互方法包括：
第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；
根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
可选地，第一终端接收第二终端发送的电信号参数的步骤包括：
所述第一终端接收所述发送的加密数据；
对所述加密数据进行解密，以得到所述电信号参数。
可选地，根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户的步骤包括：
根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给第一终端的用户。
可选地，肌电信号包括面部表情信号以及肢体信号中的至少一种。
为实现上述目的，本申请还提供一种交互方法，交互方法应用于第二终端，所述交互方法包括：
第二终端获取肌电传感器采集的肌电信号；
获取所述肌电信号对应的电信号参数；
将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户。
可选地，将所述电信号参数发送至第一终端的步骤包括：
对所述电信号参数进行加密，得到加密数据；
将所述加密数据发送至所述第一终端。
可选地，将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户的步骤包括：
将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给所述第一终端的用户。
此外，为实现上述目的，本申请还提供一种交互装置，所述交互装置包括：
第一发送模块，用于第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；
输出模块，用于根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作；或者，
第一获取模块，用于第二终端获取肌电传感器采集的肌电信号；
第二获取模块，用于获取所述肌电信号对应的电信号参数；
第二发送模块，用于将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作。
此外，为实现上述目的，本申请还提供一种终端设备，所述终端设备包括：存储器、处理器以及存储在所述存储器里并可在所述处理器上运行的交互程序，所述交互程序被所述处理器执行时实现如以上所述交互方法的各个步骤。
此外，为实现上述目的，本申请还提供一种可读存储介质，所述可读存储介质上存储有交互程序，所述交互程序被所述处理器执行时实现如以上所述交互方法的各个步骤。
本申请提出的交互方法、终端设备及可读存储介质，第一终端接收第二终端发送的电信号参数，基于电信号参数是第二终端设备中肌电传感器采集的肌电信号转换得到，第一终端根据电信号参数调整第一终端的工作参数，将肌电信号对应的动作传递给第一终端的用户，便于用户能够真实感知远程交互过程中对方的面部表情或者肢体动作，提升交互效果且使得交互方式更加多样化。
图1为本申请的交互方法各个实施例涉及的终端设备的结构示意图；
图2为本申请的交互方法应用于第一终端的第一实施例的流程示意图；
图3为本申请的交互方法应用于第二终端的第一实施例的流程示意图；
图4为本申请的交互方法第二实施例的流程示意图；
图5为本申请的交互方法第三实施例的流程示意图；
图6为本申请的交互装置的第一模块示意图；
图7为本申请的交互装置的第二模块示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在后续的描述中，使用用于表示元件的诸如“模块”、“部件”或者“单元”的后缀仅为了有利于本申请的说明，其本身没有特定的意义。因此，“模块”、“部件”或者“单元”可以混合地使用。
请参考图1，图1为本申请的交互方法各个实施例涉及的终端设备的结构示意图。其中，终端设备可以是以各种形式来实施。例如，本申请中的终端设备可以是包括诸如手机、平板电脑、笔记本电脑以及掌上电脑等终端；也可以是可穿戴设备，如VR眼镜以及AR眼镜等。
如图1所示，该终端设备可以包括：存储器101、处理器102以及通信单元103。本领域技术人员可以理解，图1示出的终端设备的结构框图并不构成对终端设备的限定，终端设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。其中，存储器101中存储有操作***以及交互程序。处理器102是终端设备的控制中心，处理器102执行存储在存储器101内的交互程序，以实现本申请的交互方法各实施例的步骤。通信单元103通过网络协议与其他终端设备建立数据通信(该数据通信可为IP通信或者蓝牙通道)，以实现与其他终端设备之间进行数据传输，举例来说，在终端设备为第一终端，其他终端设备为第二终端时，第一终端接收第二终端发送的电信号参数，或者，在终端设备为第二终端，其他终端设备为第一终端时，第二终端获取肌电传感器采集的肌电信号；获取肌电信号对应的电信号参数；将电信号参数发送至第一终端。
可选地，终端设备还可包括显示单元104，显示单元104包括显示面板，可采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板，用于输出显示用户浏览的界面。
可选地，终端设备可以是第一终端，也可以是第二终端。
可选地，终端设备包括第一终端以及第二终端。第一终端与第二终端之间可通过通信单元进行数据传输通信，以实现第一终端与第二终端之间的远程交互。
可选地，终端设备为第一终端时，第一终端电连接有电极，其中，电极与用户接触连接，以在第一终端根据电信号参数调整第一终端电连接的电极的工作参数，以通过电极将电信号参数对应的肌电信号传输至用户，通过神经传递以使电极连接的用户感受肌电信号对应的动作；终端设备为第二终端时，第二终端可获取肌电传感器采集的肌电信号。
需要说明的是，终端设备为第一终端时，第一终端的存储器101中的交互程序被处理器102执行时实现如下步骤：
第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；
根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
第一终端的工作参数可以包括电信号参数、机械参数等各种可以实现将所述肌电信号对应的动作传递给第一终端的用户的装置对应的参数，例如当通过电极将所述肌电信号对应的动作传递给第一终端的用户时，根据所述电信号参数调整所述第一终端的电极对应的电信号参数；当通过振动马达将所述肌电信号对应的动作传递给第一终端的用户时，根据所述电信号参数调整所述第一终端的电极对应的振动马达的振动信号参数。
终端设备为第二终端时，第二终端的存储器101中的交互程序被处理器102执行时实现如下步骤：
第二终端获取肌电传感器采集的肌电信号；
获取所述肌电信号对应的电信号参数；
将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信 号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户。
在实际应用过程中，该交互方法可应用于各种远程交互的应用场景中，应用场景如远程会议、远程就诊以及远程辅助指导等。
一示例性地，远程交互的应用场景可以是远程会议场景。在远程会议场景中，与会人员可通过佩戴AR眼镜以进行远程会议，以实现与会人员在AR眼镜上看到的图像能够实时传送给位于会议现场之外的远程终端控制***，以使得远端的与会人员可同步共享会议内容，此外，在远程会议过程中，第一终端的持有人为第一与会人员，第二终端的持有人为第二与会人员。第一终端接收第二终端发送的电信号参数，基于电信号参数是第二终端设备中肌电传感器采集的肌电信号转换得到，第一终端根据电信号参数调整第一终端的工作参数，以使得肌电信号对应的动作传递给第一终端的用户，便于用户能够真实感知远程交互过程中对方的面部表情或者肢体动作，提升远程会议的交互效果。
一示例性地，远程交互的应用场景可以是远程辅助指导场景。第一终端的持有人为远程协助的被辅助人，第二终端的持有人为远程协助的辅助指导人。可选地，第一终端的交互方法的步骤也可应用于第二终端，而第二终端的交互方法的步骤可应用于第一终端，以在第一终端根据电信号参数调整第一终端电连接的电极的工作参数，以使电极连接的用户还原肌电信号对应的动作后，以使得第一终端的持有人在获知实际操作的操作触感后进行实操时，实现第一终端获取肌电传感器采集的肌电信号；获取肌电信号对应的电信号参数；将电信号参数发送至第二终端。相对应地，第二终端接收第一终端发送的电信号参数，电信号参数通过第一终端中的肌电传感器采集的肌电信号转换获得；根据电信号参数调整第二终端电连接的电极的工作参数，以使电极连接的用户还原肌电信号对应的动作，以使得第二终端的持有人通过感受第一终端持有人实操的操作动作，在第一终端持有人的操作动作存在偏差时，以进一步进行指导纠正，便于远程协助的双方进行充分沟通交流，进一步提升辅助效果。
一示例性地，在交互方法应用于远程就诊场景时，第一终端的持有人为远程就诊的就诊医生，第二终端的持有人为远程就诊的就诊患者。对于就诊 患者存在肌肉损伤，导致无法手臂、腿以及脚等无法灵活活动时，就诊患者可通过远程就诊方式进行问诊以获知就诊指导。其中，远程就诊过程中，就诊医生可通过指导就诊患者活动手臂、腿以及脚如抬起手臂、踢腿、下蹲以及活动脚踝等方式，以判断就诊患者的肌肉损伤的程度，为便于清楚获知就诊患者的实际活动动作，可第二终端获取机电信号采集的就诊患者活动时的肌电信号，获取肌电信号对应的电信号参数，并将电信号参数发送至第一终端；相对应地，第一终端接收第二终端发送的电信号参数，根据电信号参数调整第一终端的工作参数，以使肌电信号对应的动作传递给第一终端的用户也即就诊医生，以使得就诊医生可真实感知并体会就诊患者进行活动时的活动动作，以更好地指导就诊患者进行就医，提高远程就诊的就诊效果。
基于上述终端设备的结构框图，提出本申请的交互方法的各个实施例。
在第一实施例中，本申请提供一种交互方法，请参考图2，图2为本申请的交互方法应用于第一终端的第一实施例的流程示意图。在该实施例中，交互方法包括以下步骤：
步骤S10，第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；
步骤S20，根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得。电信号参数包括电流和/或电压。可选地，肌电信号转换获得电信号参数，可基于肌电信号与电信号参数之间的对应关系，在获得肌电信号的前提上，基于该肌电信号获取与肌电信号对应的电信号参数，本实施例对此步骤不做限定。
可选地，肌电信号可以是面部表情信号以及肢体动作信号中的至少一种。
需要说明的是，肌电传感器可佩戴于使用者的肢体或与使用者的面部或肢体接触，以用于检测伴随使用者肢体动作的肌电信号。肌电信号(Electromyography,EMG)发源于作为中枢神经一部分脊髓中的运动神经元，是电极所接触到的许多运动单元发放的动作电位的总和，反映了神经肌肉的活动、功能状态。其中，表面肌电信号(surface electromyography，sEMG)，是一种浅层肌肉电信号与神经干上电活动在皮肤表面的综合效应产生的信号， 其中包含有丰富神经肌肉的活动信息。
可选地，在实际应用过程中，可将电极或者机械结构与第一终端电连接，电极与用户的皮肤接触时，可以将肌电信号对应的动作传递给第一终端的用户，或者将电信号转换为相应的机械信号，例如将电信号转换为振动马达的振动信号，当第一终端的用户与振动马达接触时，可以将肌电信号对应的动作传递给第一终端的用户。步骤S20中根据电信号参数调整第一终端的工作参数包括根据电信号参数调整与第一终端电连接的电极或者机械结构的工作参数，以使肌电信号对应的动作通过与第一终端的用户连接的电极或者机械结构传递给第一终端的用户。
作为一种可选的实施方式，步骤S20包括：
根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给第一终端的用户。
电极的工作参数指的是通过电极的电流，和/或，加载在电极的电压。根据电信号参数调整第一终端电连接的电极的工作参数，也即，调整第一终端电连接的电极的工作参数至电信号参数，可根据电信号参数控制电极的输出，以使得电极按照电信号参数运行，进而根据电信号参数调整第一终端电连接的电极的工作参数后，以通过电极将电信号参数对应的肌电信号输入至与电极连接的用户，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给第一终端的用户，进而使得所述肌电信号对应的动作传递给第一终端的用户。
作为一种可选的实施方式，步骤S10包括：
第一终端接收第二终端发送的加密数据；
对所述加密数据进行解密，以得到所述电信号参数。
基于安全角度考虑，对应于第二终端对电信号参数进行加密，得到加密数据；将加密数据发送至第一终端，在第一终端接收第二终端发送的加密数据，对加密数据进行解密，以得到电信号参数，以提高第一终端与第二终端之间数据传输的安全性。
在本实施例公开的技术方案中，第一终端接收第二终端发送的电信号参数，基于电信号参数是第二终端设备中肌电传感器采集的肌电信号转换得到， 第一终端根据电信号参数调整第一终端的工作参数，以使得所述肌电信号对应的动作传递给第一终端的用户，便于用户能够真实感知远程交互过程中对方的面部表情或者肢体动作，提升交互效果且使得交互方式更加多样化。
相对应地，请参考图3，图3为本申请的交互方法应用于第二终端的第一实施例的流程示意图。在该实施例中，交互方法包括以下步骤：
步骤S30，第二终端获取肌电传感器采集的肌电信号；
步骤S40，获取所述肌电信号对应的电信号参数；
步骤S50，将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
第二终端获取肌电传感器采集的肌电信号，可将肌电传感器直接连接至第二终端，进而实现获取肌电传感器采集的肌电信号；也可将肌电传感器连接至除了第二终端外的其他终端设备，以在通过其他终端设备获取肌电传感器采集的肌电信号后，将该肌电信号对应的电信号参数发送至第二终端，本实施例对此不做限定。
电信号参数包括电流和/或电压，获取肌电信号对应的电信号参数，可基于肌电信号与电信号参数之间的对应关系，在获得肌电信号的前提上，基于该肌电信号获取与肌电信号对应的电信号参数。
将电信号参数发送至第一终端，以使得第一终端根据电信号参数调整第一终端电连接的电极的工作参数，以使对方的面部表情或者肢体动作，便于用户能够真实感知远程交互过程中对方的面部表情或者肢体动作，提升交互效果且使得交互方式更加多样化。
可选地，将电信号参数发送至第一终端的方式可采用无线通信传输方式，如基于WIFI或者BT或者蜂窝网络实现将电信号参数发送至第一终端，以适应于远程交互场景，也可采用有线传输方式，对此不做限定。
可以理解的是，第二终端获取肌电传感器采集的肌电信号，将肌电信号对应的电信号参数发送至第一终端，第一终端接收第二终端发送的电信号参数，根据电信号参数调整第一终端的工作参数，以使肌电信号对应的动作传递给第一终端的用户。从第二终端获取肌电传感器采集目标部位的肌电信号，将该肌电信号对应的电信号参数发送至第一终端后，相对应地，第一终端接 收到第二终端发送的电信号参数，根据电信号参数调整第一终端的工作参数，以使肌电信号对应的动作传递给第一终端的用户的与目标部位相对应的部位上。
作为一种可选的实施方式，步骤S50包括：
对所述电信号参数进行加密，得到加密数据；
将所述加密数据发送至所述第一终端。
基于安全角度考虑，肌电信号是关键的一类生物信号，为确保肌电信号对应的电信号参数能够安全且准确地传输至第一终端，以防止数据泄露或恶意篡改，可对肌电信号对应的电信号参数进行加密，得到加密数据，进而将加密数据发送至第一终端。
可选地，对肌电信号对应的电信号参数进行加密的加密算法可以是对称加密算法，也可以是非对称加密算法，本实施例对此不做限定。
作为一种可选的实施方式，步骤S50包括：
将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给所述第一终端的用户。
在本实施例公开的技术方案中，第二终端获取肌电传感器采集的肌电信号，将肌电信号对应的电信号参数发送至第一终端，以通过第一终端根据电信号参数调整第一终端的工作参数，将肌电信号对应的动作传递给第一终端的用户，便于交互的用户能真实感受远程交互过程中对方的表情或肢体动作等触觉，提升交互效果且使得交互方式更加多样化。
在基于第一实施例的基础上提出的第二实施例中，请参考图4，图4为本申请的交互方法第二实施例的流程示意图。在该实施例中，步骤S20包括：
步骤S21，接收所述第二终端发送的交互信息；
步骤S22，输出所述交互信息，且根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
交互信息包括语音交互信息以及图像交互信息中的至少一个。其中，语音交互信息是指远程交互时获取到的语音信息。语音交互信息通过第二终端 中的麦克风或者第二终端连接的麦克风采集得到。图像交互信息是指远程交互时获取到的基于操作图像的注释批注实现双方交互的信息。其中，操作图像是指交互双方进行远程交互时所共享交互的的操作画面。
第一终端接收第二终端发送的交互信息，通过交互信息，以便于用户从听觉和/或视觉上获知交互信息的同时，根据电信号参数调整第一终端电连接的电极的工作参数，以使电极连接的用户还原肌电信号对应的动作，可便于用户能够真实感知远程交互过程中对方的表情或肢体动作等触觉，提升远程交互效果。
作为一种可选的实施方式，步骤S21之前包括：
向所述第二终端发送远程交互请求后，实时获取当前待交互共享的操作图像；
将所述操作图像发送至第二终端，执行步骤S21。
在实际应用过程中，为更好实现远程交互，第一终端以及第二终端采用AR产品设备，以使得佩戴第一终端的持有人以及第二终端的持有人可协同共享同一操作画面，如第一终端向第二终端发送远程交互请求后，实时获取当前待交互共享的操作图像，将操作图像发送至第二终端，以实现与第一终端与第二终端实时共享同一操作画面，便于第二终端的持有人基于操作画面实现远程交互的同时，提升远程交互的交互效果，进而执行步骤S21。此外，丰富了AR产品设备的应用场景以及应用体验。
可选地，实时获取当前待交互共享的操作图像可通过设置于第一终端的摄像装置实时获取当前待交互共享的操作图像。
可选地，第一终端向第二终端发送远程交互请求可通过第一终端接收到远程交互指令或者检测到远程交互对应的触摸操作，以触发第一终端向第二终端发送远程交互请求。
在本实施例公开的技术方案中，输出接收到的第二终端发送的交互信息如语音交互信息和/或图像交互信息的同时，可便于用户从听觉和/或视觉上获知远程交互的动作信息，根据电信号参数调整第一终端的工作参数，以使肌电信号对应的动作传递给第一终端的用户，可便于用户能够真实感知远程交互过程中对方的面部表情或者肢体动作，从而便于用户从听觉、视觉以及触觉上综合获知远程交互过程中对方的表情或肢体动作等触觉，提升远程交互 的效果。
在基于上述任意一个实施例提出的第三实施例中，请参考图5，图5为本申请的交互方法第三实施例的流程示意图。在该实施例中，步骤S50包括：
步骤S51，获取交互信息；
步骤S52，将所述交互信息以及所述电信号参数发送至第一终端，以通过所述第一终端输出所述交互信息，且根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户。
交互信息包括语音交互信息以及图像交互信息中的至少一个。其中，语音交互信息是指远程交互时获取到的语音信息。语音交互信息通过第二终端中的麦克风或者第二终端连接的麦克风采集得到。图像交互信息是指远程交互时获取到的基于操作图像的注释批注实现双方交互的信息。
将交互信息以及电信号参数同时发送至第一终端，以通过第一终端输出交互信息时，便于用户从听觉和/或视觉上获知交互信息的同时，根据电信号参数调整第一终端的工作参数，以使得第一终端的用户可感受并还原肌电信号对应的动作，便于交互的用户能真实感受远程交互过程中对方的表情或肢体动作等触觉，提升交互效果且使得交互方式更加多样化。
作为一种可选的实施方式，步骤S51包括：
接收所述第一终端发送的远程交互请求以及操作图像，在远程交互过程中获取交互信息，其中，所述交互信息包括语音交互信息以及图像交互信息中的至少一个，所述图像交互信息基于所述操作图像接收到注释批注指令得到。
在实际应用过程中，为更好实现远程交互，第一终端以及第二终端采用AR产品设备，以使得佩戴第一终端的持有人以及第二终端的持有人可协同共享同一操作画面。对应于第一终端向第二终端发送远程交互请求后，实时获取当前待交互共享的操作图像，将操作图像发送至第二终端，第二终端接收第一终端发送的远程交互请求以及操作图像，在协助操作过程中第二终端与第一终端通过共享同一操作画面也即操作图像，基于同一操作画面获取交互信息，以使得第一终端的持有人以及第二终端的持有人通过听觉、视觉以及触觉上的相互沟通交流，以达到更加准确的远程交互的目的。
可选地，图像交互信息基于操作图像接收到注释批注指令得到，可在第二终端从第一终端接收到的操作图像，通过HoloLens实现第二终端的持有人进行远程交互的佩戴者对操作图像进行注释批注。其中，HoloLens是一种MR头显也即混合现实头戴式显示器。
在本实施例公开的技术方案中，第二终端获取交互信息如语音交互信息和/或图像交互信息，将交互信息以及电信号参数同时发送至第一终端，以通过第一终端输出交互信息可便于用户从听觉和/或视觉上获知远程交互的动作信息，根据电信号参数调整第一终端的工作参数，以使得第一终端的用户可感受并还原肌电信号对应的动作，便于交互的用户能真实感受远程交互过程中对方的表情或肢体动作等触觉，提升交互效果且使得交互方式更加多样化。
请参考图6，图6为本申请的交互装置的第一模块示意图。本申请还提出一种交互装置100，所述交互装置100包括：
第一发送模块110，用于第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；
输出模块120，用于根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作。
或者，请参考图7，图7为本申请的交互装置的第二模块示意图。本申请还提出一种交互装置200，所述交互装置200包括：
第一获取模块210，用于第二终端获取肌电传感器采集的肌电信号；
第二获取模块220，用于获取所述肌电信号对应的电信号参数；
第二发送模块230，用于将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作。
本申请还提出一种终端设备，所述终端设备包括：包括存储器、处理器以及存储在存储器里并可在处理器上运行的交互程序，交互程序被处理器执行时实现上述任一实施例中的交互方法的步骤。
本申请还提出一种可读存储介质，该可读存储介质上存储有交互程序，所述交互程序被处理器执行时实现如以上任一实施例所述的交互方法的步骤。
在本申请提供的终端设备和可读存储介质的实施例中，包含了上述交互方法各实施例的全部技术特征，说明书拓展和解释内容与上述交互方法的各实施例基本相同，在此不做再赘述。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。
Claims (10)
- 一种交互方法，其特征在于，所述交互方法应用于第一终端，所述交互方法包括：第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户。
- 如权利要求1所述的交互方法，其特征在于，所述第一终端接收第二终端发送的电信号参数的步骤包括：所述第一终端接收所述第二终端发送的加密数据；对所述加密数据进行解密，以得到所述电信号参数。
- 如权利要求1所述的交互方法，其特征在于，所述根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给第一终端的用户的步骤包括：根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给第一终端的用户。
- 如权利要求1所述的交互方法，其特征在于，所述肌电信号包括面部表情信号以及肢体信号中的至少一种。
- 一种交互方法，其特征在于，所述交互方法应用于第二终端，所述交互方法包括：第二终端获取肌电传感器采集的肌电信号；获取所述肌电信号对应的电信号参数；将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户。
- 如权利要求5所述的交互方法，其特征在于，所述将所述电信号参数发送至第一终端的步骤包括：对所述电信号参数进行加密，得到加密数据；将所述加密数据发送至所述第一终端。
- 如权利要求5所述的交互方法，其特征在于，所述将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端的工作参数，以使所述肌电信号对应的动作传递给所述第一终端的用户的步骤包括：将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述肌电信号对应的动作通过与所述第一终端的用户连接的所述电极传递给所述第一终端的用户。
- 一种交互装置，其特征在于，所述交互装置包括：第一发送模块，用于第一终端接收第二终端发送的电信号参数，所述电信号参数通过第二终端中的肌电传感器采集的肌电信号转换获得；输出模块，用于根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作；或者，第一获取模块，用于第二终端获取肌电传感器采集的肌电信号；第二获取模块，用于获取所述肌电信号对应的电信号参数；第二发送模块，用于将所述电信号参数发送至第一终端，以通过所述第一终端根据所述电信号参数调整所述第一终端电连接的电极的工作参数，以使所述电极连接的用户还原所述肌电信号对应的动作。
- 一种终端设备，其特征在于，所述终端设备包括：存储器、处理器以及存储在所述存储器里并可在所述处理器上运行的交互程序，所述交互程序被所述处理器执行时实现如权利要求1-7任一项交互方法的步骤。
- 一种可读存储介质，其特征在于，所述可读存储介质上存储有交互程序，所述交互程序被处理器执行时实现如权利要求1-7任一项交互方法的步骤。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111149091.1A CN113791692A (zh) | 2021-09-28 | 2021-09-28 | 交互方法、终端设备及可读存储介质 |
| CN202111149091.1 | 2021-09-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023050522A1 true WO2023050522A1 (zh) | 2023-04-06 |
Family
ID=78877501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/128069 WO2023050522A1 (zh) | 2021-09-28 | 2021-11-02 | 交互方法、终端设备及可读存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113791692A (zh) |
| WO (1) | WO2023050522A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115114962A (zh) * | 2022-07-19 | 2022-09-27 | 歌尔股份有限公司 | 基于表面肌电信号的控制方法、装置及可穿戴设备 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108549153A (zh) * | 2018-06-01 | 2018-09-18 | 烟台市安特洛普网络科技有限公司 | 一种基于面目表情识别的智能vr眼镜 |
| WO2018205424A1 (zh) * | 2017-05-09 | 2018-11-15 | 深圳市科迈爱康科技有限公司 | 基于肌电的生物识别方法、终端及计算机可读存储介质 |
| CN111857326A (zh) * | 2019-04-25 | 2020-10-30 | 北京小米移动软件有限公司 | 信号控制方法及装置 |
| CN113141517A (zh) * | 2021-04-09 | 2021-07-20 | 桂林电子科技大学 | 第一终端、第二终端、远程操作控制方法和可读存储介质 |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6440067B1 (en) * | 2000-02-28 | 2002-08-27 | Altec, Inc. | System and method for remotely monitoring functional activities |
| TWM288176U (en) * | 2005-02-04 | 2006-03-01 | Chang-An Jou | Put-on physiological signal detection and transmission/receipt device and system |
| KR20080028084A (ko) * | 2006-09-26 | 2008-03-31 | 삼성전자주식회사 | 휴대 단말기를 이용하는 근전도 기반의 의사 전달 장치 및방법 |
| CN101995947B (zh) * | 2009-08-21 | 2012-07-04 | 香港中文大学 | 通过网络感知用户动作的***和方法 |
| CN102949783B (zh) * | 2011-08-26 | 2015-03-25 | 云林科技大学 | 回馈控制的穿戴式上肢电刺激装置 |
| CN104107134B (zh) * | 2013-12-10 | 2017-08-01 | 中山大学 | 基于肌电反馈的上肢训练方法及*** |
| WO2016026100A1 (zh) * | 2014-08-20 | 2016-02-25 | 华为技术有限公司 | 一种肌电信号采集装置 |
| CN110444189B (zh) * | 2019-06-18 | 2022-01-04 | 中国人民解放军军事科学院国防科技创新研究院 | 一种缄默通信方法、***和存储介质 |
| CN111475024B (zh) * | 2019-12-25 | 2024-04-16 | 山东中科先进技术有限公司 | 一种人体动作捕捉***及方法 |
-
2021
- 2021-09-28 CN CN202111149091.1A patent/CN113791692A/zh active Pending
- 2021-11-02 WO PCT/CN2021/128069 patent/WO2023050522A1/zh unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018205424A1 (zh) * | 2017-05-09 | 2018-11-15 | 深圳市科迈爱康科技有限公司 | 基于肌电的生物识别方法、终端及计算机可读存储介质 |
| CN108549153A (zh) * | 2018-06-01 | 2018-09-18 | 烟台市安特洛普网络科技有限公司 | 一种基于面目表情识别的智能vr眼镜 |
| CN111857326A (zh) * | 2019-04-25 | 2020-10-30 | 北京小米移动软件有限公司 | 信号控制方法及装置 |
| CN113141517A (zh) * | 2021-04-09 | 2021-07-20 | 桂林电子科技大学 | 第一终端、第二终端、远程操作控制方法和可读存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113791692A (zh) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11071597B2 (en) | Telemedicine for orthopedic treatment | |
| Lee et al. | A novel method of surgical instruction: international telementoring | |
| Simone et al. | A low cost instrumented glove for extended monitoring and functional hand assessment | |
| Luo et al. | Integration of augmented reality and assistive devices for post-stroke hand opening rehabilitation | |
| Markovic et al. | GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses | |
| US9901740B2 (en) | Clinician programming system and method | |
| Kenyon et al. | Considerations for the future development of virtual technology as a rehabilitation tool | |
| WO2022262495A1 (zh) | 家用医疗设备的控制方法及相关装置 | |
| KR20180045278A (ko) | 생체신호연동 가상현실 인지재활 시스템 | |
| Luo et al. | An augmented reality training environment for post-stroke finger extension rehabilitation | |
| WO2023050522A1 (zh) | 交互方法、终端设备及可读存储介质 | |
| CN109288540A (zh) | 一种具有触觉反馈的远程超声诊断*** | |
| CN105630145A (zh) | 虚拟感觉的方法和装置以及使用其的眼镜或头盔 | |
| CN110717344A (zh) | 基于智能可穿戴设备的辅助交流*** | |
| Orejuela-Zapata et al. | Self-help devices for quadriplegic population: A systematic literature review | |
| CN110613459A (zh) | 一种基于共享云计算平台的耳鸣耳聋检测验配及治疗*** | |
| CN109875505A (zh) | 一种基于虚拟现实技术的一体化手术室远程会诊方法 | |
| Athanasiou et al. | Wireless Brain‐Robot Interface: User Perception and Performance Assessment of Spinal Cord Injury Patients | |
| KR101798817B1 (ko) | 이동식 무선보안 화상 의료 서비스 장치 | |
| WO2023103740A1 (zh) | 画面显示控制方法、设备、远程会诊***及存储介质 | |
| US20170286617A1 (en) | Holomedicine: Methodology, Process, Practice and Delivery | |
| WO2023226636A1 (zh) | 控制器、植入式神经刺激***及计算机可读存储介质 | |
| US20210069059A1 (en) | Electroacupuncture Treatment System and Data Optimization Method Thereof | |
| WO2023000787A1 (zh) | 视频处理方法、装置、电子设备及计算机可读存储介质 | |
| KR102142535B1 (ko) | 거울신경세포의 활성화를 극대화하는 기능성 전기 자극과 뇌-컴퓨터 인터페이스(bci) 기술을 결합한 게임 방식의 재활 시스템 및 그 제어방법 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21959086 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |