CN108056865A - A kind of multi-modal wheelchair brain control system and method based on cloud platform - Google Patents
A kind of multi-modal wheelchair brain control system and method based on cloud platform Download PDFInfo
- Publication number
- CN108056865A CN108056865A CN201711251132.1A CN201711251132A CN108056865A CN 108056865 A CN108056865 A CN 108056865A CN 201711251132 A CN201711251132 A CN 201711251132A CN 108056865 A CN108056865 A CN 108056865A
- Authority
- CN
- China
- Prior art keywords
- wheelchair
- control
- fpga processor
- brain
- current potential
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/04—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs motor-driven
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/12—Remote controls
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/18—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering by patient's head, eyes, facial muscles or voice
Landscapes
- Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Dermatology (AREA)
- Neurosurgery (AREA)
- Neurology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a kind of multi-modal wheelchair brain control system and methods based on cloud platform, realize the parallel control to wheelchair comprising three kinds of mode altogether：The first mode is movement imagination control, and second of mode is stable state vision inducting control, the third mode is voice control.In terms of the monitoring of cloud platform, the eeg data of user and the operation conditions of wheelchair are monitored by independently building MYSQL servers.FPGA carries gsm module by serial ports, and data are uploaded to server by the form of 4G networks or WIFI, when user logs in the webpage using JSP+HTML+CSS independent developments, it is possible to long-range monitoring wheelchair and user's situation.
Description
Technical field
The present invention relates to wheelchair technical fields, particularly a kind of multi-modal wheelchair brain control system and side based on cloud platform
Method.
Background technology
Human-computer interaction is one of current important research topic, and the virtual reality technology particularly currently rapidly developed makes
Human-computer interaction technology has obtained great promotion, is increasingly paid close attention in fields such as medical science of recovery therapy, industry, military affairs be subject to the mankind.
Brain-computer interface (Brain-computer Interface, BCI) is using people as interaction center, significantly improves people
The efficiency of machine interaction.E.E.G control belongs to a kind of intellectualized technology, and the electric signal sent by brain utilizes brain-computer interface skill
Art realizes the control to external object.Evoked ptential is that nervous system receives specific activities caused by inside and outside boundary's stimulation, external
Visual stimulus and generated on pathways for vision can the cerebral cortex of people generate and the electric signal that is measured on scalp.
Different stimulations is given, is then incorporated in the different Evoked ptentials of vision current potential generation and the one-to-one relationship stimulated, it can
To be inferred to the stimulus modality of subject, and then send control command.
Common push-chairs in the market promotes its wheel to make or completes various actions in the case where other people help by people,
It is not high that comfort level is applicable in the mankind；Secondly, electric wheelchair, by button, hand lever or sound control wheelchair complete various move
Make, but for the patient of the diseases such as headstroke, amyotrophic lateral sclerosis, this wheelchair can bring inconvenience.
It would therefore be highly desirable to develop a kind of multi-modal wheelchair brain control system and method based on cloud platform.
The content of the invention
The invention aims to provide a kind of multi-modal wheelchair brain control system and method based on cloud platform, can pass through
Brain electric control, voice control can be with overturning-preventing, remote monitoring, and suitable for various people, patient need not move, and need not fiber crops
It is tired of that other people help, the defects of completing various instructions by E.E.G controling wheelchair, compensate for wheelchair currently on the market.
In order to achieve the above objectives, the present invention is implemented according to following technical scheme：
A kind of multi-modal wheelchair brain control system based on cloud platform, including visual stimulator, Neurosky E.E.Gs chip, language
Sound acquisition module, FPGA processor, MYSQL servers, Bluetooth chip, DAC chip, mobile terminal, wheelchair drive control device；
There are five occipital lobe area, top area, the lateral lobe areas for being respectively used to acquisition human brain for the Neurosky E.E.Gs chip connection
And the electrode of the current potential of the EEG signals in ear-lobe area, the electrode in occipital lobe area are used to gather the vision inducting brain telecommunications of human brain generation
Number current potential, the electrode in lateral lobe area is used to gather the current potential of the movement imagination EEG signals of human brain, and Neurosky E.E.G chips lead to
It crosses Bluetooth chip to be connected with FPGA processor, the current potential of the vision induced EEG signals of acquisition and movement is imagined into EEG signals
Current potential is sent to FPGA processor；
The visual stimulator is used to that stimulus signal to be transferred to human brain by human eye, and human brain generates opposite with stimulus signal
The vision induced EEG signals answered；
The output terminal of the voice acquisition module is connected by bus with FPGA processor, for receive voice signal and to
FPGA processor sends the control instruction to wheelchair；
The FPGA processor is connected with MYSQL servers by gsm module and communicated, and the mobile terminal passes through
WiFi or 4G networks are connected with MYSQL servers and communicate；The FPGA processor lures for analyzing and processing the vision of acquisition
The signal that the current potential and voice acquisition module of the current potential and movement imagination EEG signals of sending out EEG signals gather, and converted
For the control instruction to wheelchair, and the running state information of wheelchair drive control device is monitored, by the operation data sending of monitoring extremely
MYSQL servers；
The wheelchair drive control device is connected by DAC chip and Bluetooth chip with FPGA processor successively, for receiving
FPGA processor is analyzed and treated converts control voltage of the output to wheelchair to the control instruction of wheelchair by DAC chip,
The wheelchair drive control device connects wheelchair drive mechanism, and the control voltage and then controling wheelchair for receiving to wheelchair drive dress
Put action.
Further, in the present invention, the mobile terminal is mobile phone or tablet computer.
In addition, the present invention also provides a kind of methods of the multi-modal wheelchair brain control based on cloud platform, three kinds of mode are included
Realize the control to wheelchair, it is specific as follows：
The first mode is movement imagination control：The left and right lateral lobe area of human brain is carried out using Neurosky E.E.Gs chip
The acquisition of the current potential of the movement imagination EEG signals of human brain, main acquisition target is the μ rhythm ripple of 8HZ-12HZ, passes through bluetooth core
Piece is transferred to FPGA processor, and when human body carries out left leg movement imagination, FPGA processor takes short term Fourier transform
The MU rhythm and pace of moving things energy magnitudes in the lateral lobe area of SFFT algorithms extraction human brain, are swashed by fixed energized μ rhythm energy magnitude
Upper limit a is encouraged, lower limit b when unexcited, FPGA processor judges the energy magnitude M of extraction using SVM algorithm, if
Result judgement is then A classes by a≤M≤b or M≤b, i.e., user carries out unconscious imagination or this moment without carrying out left leg fortune
Dynamic imagination, FPGA processor will not send control instruction to wheelchair drive control device；If M >=a, it is determined as B classes, works as left side
Leaf area current potential is in A classes, and lobus lateralis dexter area current potential is in B classes, then judges that user is carrying out left leg movement imagination consciously,
FPGA processor carries out start-up and shut-down control to wheelchair；
Second of mode is stable state vision inducting control：It is stared at by human eye regarding visual stimulator, will be stimulated and believed by human eye
Number Human brain occipital lobes are transferred to, after the working process of brain, Neurosky E.E.Gs chip is gathered in human brain stimulus signal
Occipital lobe area generates the current potential with stimulus signal corresponding vision induced EEG signals in frequency, is transferred to by Bluetooth chip
On FPGA processor, short term Fourier transform SFFT is carried out to the current potential of vision induced EEG signals by FPGA processor,
The frequency values corresponding to the highest Frequency point of amplitude are exactly the frequency stared at corresponding to the visual stimulus block regarded in a frequency domain, will frequency
After rate value is converted to the control instruction of wheel chair sport, then control instruction is converted into corresponding level signal, level signal warp
The control voltage of wheelchair can directly be exported by crossing DAC conversions, wheelchair drive control device receive to the control voltage of wheelchair and then
Controling wheelchair driving device acts；
The third mode is speech modality control：It is sent by voice acquisition module and FPGA processor is sent to wheelchair
Control instruction, FPGA processor carries out sub-frame processing using 20ms time windows to voice signal, using HMM algorithms, to each
In time window at different moments between shape, state transition probability observed, and obtains the observation vector in each time window
{O1, O2,.....ON, using DNN neutral nets for these observation vectors be identified, draw in each time window to wheel
The control instruction of chair, and take two-stage triggering mechanism, i.e. first-level instruction and the recognition result that two level instructs is all correct, can just touch
Hair control control instruction；
In three kinds of Model controls, by MYSQL servers to the eeg data of wheelchair user and the operation shape of wheelchair
Condition is monitored, and FPGA processor carries gsm module by serial ports, is uploaded to data by the form of 4G networks or WIFI
Server, user log in the webpage using JSP+HTML+CSS independent developments, carry out long-range monitoring wheelchair and user's shape
Condition.
Wherein, the method that the control instruction of wheelchair is analyzed and handled is by the FPGA processor：Pass through and configure
The IP kernel of the fast Fourier transform fft algorithm of FPGA processor realizes signal analysis and processing, wherein Transform length
It is arranged at 512 points, IP kernel schematic diagram, CLK is the system clock of FFT IP Core, and reset_n is high level reset signal,
sink_real[17:0] it is input data real part, souurce exports for data；FPGA processor is by DAC chip conversion output
The method for controlling voltage to wheelchair is：To the control instruction of wheelchair after by FPGA processor, FPGA processor will frequency
After rate value is converted into " 3 ", " 4 ", " 5 ", " 7 " hexadecimal number, using DAC chip according to the operation voltage of wheelchair, calculate with
Corresponding digital quantization relation：Vo (DAC A | B | C | D)=REF*CODE/256* (1+RNG bit value), wherein, Vo
For the control voltage to wheelchair, REF is the output reference voltage of DAC, and CODE is the digital quantity of output voltage values, and RNG is voltage
Doubling mode, RNG represents to close for 0 exports doubling mode, and RNG represents to open output voltage doubling mode for 1.
In addition, wheelchair can obtain angle to calculate the inclined degree of wheelchair, angle using gyroscope MPU6050 in the present invention
Spend calculation formula：
ACCE_X=(GetData (ACCEL_XOUT_H))；
ACCE_Y=(GetData (ACCEL_YOUT_H))；
X=(float) ACCE_X/4096.0；
Y=(float) ACCE_Y/4096.0；
AngleAx=atan2 (x, y) * 180/3.14；
Wherein, the angular speed for the X-direction that ACCE_X representatives are obtained from gyroscope, ACCE_Y representatives are obtained from gyroscope
The angular speed of the Y direction taken, 4096 be the conversion coefficient that datasheet is provided, and what angleAx was represented is the X-axis of wheelchair, Y
The angle-data that axis obtains, according to the X-axis of the wheelchair being calculated, the angle-data that Y-axis obtains judges to take turns FPGA processor
Chair is alarmed after having the trend of rollover, and is transferred to mobile terminal, so as to effectively prevent wheelchair from turning on one's side.
Compared with prior art, present invention incorporates vision induced Human brain occipital lobes frequency resonance brain wave extraction with
The extraction of movement imagination spontaneous brain electricity ripple, and control encoding mechanism is established with this, realize the control to wheelchair start and stop and the direction of motion
System.
On the basis of brain electric control, voice control is added, makes wheelchair control more multi-modalization, improves wheelchair fortune
Capable convenience and high efficiency.
FPGA is chosen as processor controling wheelchair, greatly improves the speed and precision of algorithm process.
Whether remote monitoring wheelchair state can constantly detect wheelchair and work normally and user's physiological status.
With anti-rollover function, protection user's safety.
Description of the drawings
Fig. 1 is the multi-modal wheelchair brain control system the general frame based on cloud platform of the present invention.
Fig. 2 is the vision induced interface of visual stimulator of the present invention.
Fig. 3 is the Neurosky chip evoked brain potential acquisition electrode point diagrams of the present invention.
Fig. 4 is the IP kernel of the FFT of FPGA processor.
Specific embodiment
With reference to specific embodiment, the invention will be further described, in the illustrative examples and explanation of the invention
For explaining the present invention, but it is not as a limitation of the invention.
As shown in Figure 1, a kind of multi-modal wheelchair brain control system based on cloud platform of the present embodiment, including visual stimulus
Device, Neurosky E.E.Gs chip, voice acquisition module, FPGA processor, MYSQL servers, Bluetooth chip, DAC chip, movement
Terminal, wheelchair drive control device；Wherein, FPGA processor is EP4CE6E22C8N models, and Bluetooth chip uses HL-MD08R-
C2A models, DAC chip are TLC5620 models；
There are five occipital lobe area, top area, the lateral lobe areas for being respectively used to acquisition human brain for the Neurosky E.E.Gs chip connection
And the electrode of the current potential of the EEG signals in ear-lobe area, the electrode in occipital lobe area are used to gather the vision inducting brain telecommunications of human brain generation
Number current potential, the electrode in lateral lobe area is used to gather the current potential of the movement imagination EEG signals of human brain, and Neurosky E.E.G chips lead to
It crosses Bluetooth chip to be connected with FPGA processor, the current potential of the vision induced EEG signals of acquisition and movement is imagined into EEG signals
Current potential is sent to FPGA processor；
The visual stimulator is used to that stimulus signal to be transferred to human brain by human eye, and human brain generates opposite with stimulus signal
The vision induced EEG signals answered；
The output terminal of the voice acquisition module is connected by bus with FPGA processor, for receive voice signal and to
FPGA processor sends the control instruction to wheelchair；
The FPGA processor is connected with MYSQL servers by gsm module and communicated, and the mobile terminal passes through
WiFi or 4G networks are connected with MYSQL servers and communicate, and mobile terminal is mobile phone or tablet computer；The FPGA processor
For analyzing and processing the current potential and voice collecting of the current potential of the vision induced EEG signals of acquisition and movement imagination EEG signals
The signal of module acquisition, and the control instruction to wheelchair is translated into, and monitor the operating status letter of wheelchair drive control device
Breath, by the operation data sending of monitoring to MYSQL servers；
The wheelchair drive control device is connected by DAC chip and Bluetooth chip with FPGA processor successively, for receiving
FPGA processor is analyzed and treated converts control voltage of the output to wheelchair to the control instruction of wheelchair by DAC chip,
The wheelchair drive control device connects wheelchair drive mechanism, and the control voltage and then controling wheelchair for receiving to wheelchair drive dress
Put action.
In addition, the present embodiment additionally provides a kind of method of the multi-modal wheelchair brain control based on cloud platform, three kinds of moulds are included
State realizes the parallel control to wheelchair, specific as follows：
The first mode is movement imagination control：Five electrodes are connected using the Neurosky chips of Shen Nian companies of the U.S.,
Specifically as shown in figure 3, wherein OZ is the current potential for gathering vision induced EEG signals, and C3 and C4 are acquisition movement imagination brain telecommunications
Number current potential, the current potential of the movement imagination EEG signals of human brain is gathered by electrode C3 and C4 respectively, main acquisition target is
The μ rhythm ripple of 8HZ-12HZ, and pass through Bluetooth chip and be sent to FPGA processor；The fast Fourier of FPGA processor can be configured
The IP kernel of fft algorithm is converted, specifically as indicated at 4, realizes signal analysis and processing, wherein Transform length are arranged to
512 points, IP kernel schematic diagram, CLK be FFT IP Core system clock, reset_n be high level reset signal, sink_real
[17:0] it is input data real part, souurce exports for data, and when human body carries out left leg movement imagination, FPGA processor is adopted
Take the MU rhythm and pace of moving things energy magnitudes in the lateral lobe area of short term Fourier transform SFFT algorithms extraction human brain：
Pass through fixed energized μ rhythm energy
Magnitude excitation upper limit a, lower limit b when unexcited are measured, FPGA processor carries out the energy magnitude M of extraction using SVM algorithm
Judge, if a≤M≤b or M≤b, by result judgement be A classes, i.e., user carry out this moment it is unconscious imagine or not into
The left leg movement imagination of row, FPGA processor will not send control instruction to wheelchair drive control device；If M >=a is determined as B
Class, when lobus lateralis sinister area current potential is in A classes, lobus lateralis dexter area current potential is in B classes, then judges that user is carrying out left leg fortune consciously
Dynamic imagination, FPGA processor carry out start-up and shut-down control to wheelchair；
Second of mode is stable state vision inducting control：As shown in Fig. 2, write APP in visual stimulator, interface by
The stimulation block composition of 7.4Hz, 8.8Hz, 11.5Hz, 13.6Hz different frequency flicker, respectively positioned at four positions at interface, each
The operational order that block corresponds to a wheelchair is stimulated to advance, retreat, turn left, turn right, is stared at regarding visual stimulator, passed through by human eye
Stimulus signal is transferred to Human brain occipital lobes by human eye, stimulus signal after the working process of brain, as shown in figure 3,
The occipital lobe area that Neurosky E.E.Gs chip gathers human brain by electrode OZ generates and stimulus signal corresponding vision in frequency
The current potential of evoked brain potential signal is transferred to FPGA processor by Bluetooth chip, by FPGA processor to vision inducting brain telecommunications
Number current potential carry out short term Fourier transform SFFT：
The frequency values corresponding to the highest Frequency point of amplitude are exactly the frequency stared at corresponding to the visual stimulus block regarded in a frequency domain, will frequency
After rate value is converted to the control instruction of wheel chair sport, FPGA processor by frequency values be converted into " 3 ", " 4 ", " 5 ", " 7 " 16 into
After number processed, using DAC chip according to the operation voltage of wheelchair, corresponding digital quantization relation is calculated：Vo(DAC A|B|
C | D)=REF*CODE/256* (1+RNG bit value), wherein, Vo is the control voltage to wheelchair, and REF is the output of DAC
Reference voltage, CODE are the digital quantity of output voltage values, and RNG is voltage doubling mode, and RNG represents to close output multiplication mould for 0
Formula, RNG represent to open output voltage doubling mode for 1；Level signal is by DAC chip conversion output to the control electricity of wheelchair
The behaviour such as pressure, advance, retrogressing, left-hand rotation, the right-hand rotation that wheelchair drive control device receives the control voltage to wheelchair and then carrys out controling wheelchair
Make；
The third mode is voice control：The control sent to FPGA processor to wheelchair is sent by voice acquisition module
Instruction, FPGA processor carries out sub-frame processing using 20ms time windows to voice signal, using HMM algorithms, to each time
In window at different moments between shape, state transition probability observed, and obtains the observation vector { O in each time window1,
O2,.....ON, be identified for these observation vectors using DNN neutral nets, draw in each time window to wheelchair
Control instruction, and take two-stage triggering mechanism, i.e. first-level instruction and the recognition result that two level instructs is all correct, can just trigger control
Control instruction processed, table specific as follows：
| Phonetic entry | The control instruction of conversion | The action that FPGA is performed |
| lun yi | 0x00 | Wait two level instruction |
| qian jin | 0x01 | Wheelchair is driven to advance |
| hou tui | 0x02 | Drive wheelchair rollback |
| zuo zhuan | 0x03 | Wheelchair is driven to turn left |
| you zhuan | 0x04 | Wheelchair is driven to turn right |
In three kinds of Model controls, by MYSQL servers to the eeg data of wheelchair user and the operation shape of wheelchair
Condition is monitored, and FPGA processor carries gsm module by serial ports, is uploaded to data by the form of 4G networks or WIFI
Server, user log in the webpage using JSP+HTML+CSS independent developments, carry out long-range monitoring wheelchair and user's shape
Condition.
In addition, wheelchair can obtain angle to calculate the inclined degree of wheelchair using gyroscope MPU6050, angle calculation is public
Formula：
ACCE_X=(GetData (ACCEL_XOUT_H))；
ACCE_Y=(GetData (ACCEL_YOUT_H))；
X=(float) ACCE_X/4096.0；
Y=(float) ACCE_Y/4096.0；
AngleAx=atan2 (x, y) * 180/3.14；
Wherein, the angular speed for the X-direction that ACCE_X representatives are obtained from gyroscope, ACCE_Y representatives are obtained from gyroscope
The angular speed of the Y direction taken, 4096 be the conversion coefficient that datasheet is provided, and what angleAx was represented is the X-axis of wheelchair, Y
The angle-data that axis obtains, according to the X-axis of the wheelchair being calculated, the angle-data that Y-axis obtains judges to take turns FPGA processor
Chair is alarmed after having the trend of rollover, and is transferred to mobile terminal so as to effectively prevent wheelchair from turning on one's side.
Technical scheme is not limited to the limitation of above-mentioned specific embodiment, and every technique according to the invention scheme is done
The technology deformation gone out, each falls within protection scope of the present invention.
Claims (4)
1. a kind of multi-modal wheelchair brain control system based on cloud platform, which is characterized in that including visual stimulator, Neurosky brains
Ripple chip, voice acquisition module, FPGA processor, MYSQL servers, Bluetooth chip, DAC chip, mobile terminal, wheelchair driving
Controller；
Neurosky E.E.Gs chip connection there are five be respectively used to the occipital lobe area of acquisition human brain, top area, lateral lobe area and
The electrode of the current potential of the EEG signals in ear-lobe area, the electrode in occipital lobe area are used to gather the vision induced EEG signals of human brain generation
Current potential, the electrode in lateral lobe area are used to gather the current potential of the movement imagination EEG signals of human brain, and Neurosky E.E.G chips pass through indigo plant
Tooth chip is connected with FPGA processor, by the current potential of the vision induced EEG signals of acquisition and the current potential of movement imagination EEG signals
It is sent to FPGA processor；
The visual stimulator is used to that stimulus signal to be transferred to human brain by human eye, and human brain generates corresponding with stimulus signal
Vision induced EEG signals；
The output terminal of the voice acquisition module is connected by bus with FPGA processor, for receiving voice signal and to FPGA
Processor sends the control instruction to wheelchair；
The FPGA processor is connected with MYSQL servers by gsm module and communicated, the mobile terminal by WiFi or
4G networks are connected with MYSQL servers and communicate；The FPGA processor is electric for analyzing and processing the vision inducting brain of acquisition
The current potential and the current potential of movement imagination EEG signals and the signal of voice acquisition module acquisition of signal, and be translated into wheel
The control instruction of chair, and the running state information of wheelchair drive control device is monitored, the operation data sending of monitoring to MYSQL is taken
Business device；
The wheelchair drive control device is connected by DAC chip and Bluetooth chip with FPGA processor successively, for receiving FPGA
Processor is analyzed and treated converts control voltage of the output to wheelchair to the control instruction of wheelchair by DAC chip, described
Wheelchair drive control device connects wheelchair drive mechanism, and the control voltage and then controling wheelchair driving device of wheelchair are moved for receiving
Make.
2. the multi-modal wheelchair brain control system according to claim 1 based on cloud platform, it is characterised in that：It is described mobile whole
It holds as mobile phone or tablet computer.
A kind of 3. method of the multi-modal wheelchair brain control based on cloud platform as described in claim 1, which is characterized in that include three
Kind mode realizes the control to wheelchair, specific as follows：
The first mode is movement imagination control：Human brain is carried out to the left and right lateral lobe area of human brain using Neurosky E.E.Gs chip
Movement imagination EEG signals current potential acquisition, main acquisition target be 8HZ-12HZ μ rhythm ripple, passed by Bluetooth chip
FPGA processor is defeated by, when human body carries out left leg movement imagination, FPGA processor takes short term Fourier transform SFFT
The MU rhythm and pace of moving things energy magnitudes in the lateral lobe area of algorithm extraction human brain, by fixed energized μ rhythm energy magnitude excitation
A is limited, lower limit b when unexcited, FPGA processor judges the energy magnitude M of extraction using SVM algorithm, if a≤M
Result judgement is then A classes by≤b or M≤b, i.e., user carries out unconscious imagination or thinks without carrying out left leg movement this moment
Picture, FPGA processor will not send control instruction to wheelchair drive control device；If M >=a, it is determined as B classes, when lobus lateralis sinister area
Current potential is in A classes, and lobus lateralis dexter area current potential is in B classes, then judges that user is carrying out left leg movement consciously and imagining, at FPGA
It manages device and start-up and shut-down control is carried out to wheelchair；
Second of mode is stable state vision inducting control：It is stared at regarding visual stimulator by human eye, is passed stimulus signal by human eye
Human brain occipital lobes are defeated by, for stimulus signal after the working process of brain, Neurosky E.E.Gs chip gathers the occipital lobe in human brain
Area generates the current potential with stimulus signal corresponding vision induced EEG signals in frequency, and FPGA is transferred to by Bluetooth chip
On processor, short term Fourier transform SFFT is carried out to the current potential of vision induced EEG signals by FPGA processor, in frequency
Frequency values in domain corresponding to the highest Frequency point of amplitude are exactly the frequency stared at corresponding to the visual stimulus block regarded, by frequency values
After the control instruction for being converted to wheel chair sport, then control instruction is converted into corresponding level signal, which passes through DAC
Conversion can directly export the control voltage of wheelchair, and wheelchair drive control device receives the control voltage and then control wheel to wheelchair
Chair driving device acts；
The third mode is voice control：The control sent FPGA processor to wheelchair is sent by voice acquisition module to refer to
Order, FPGA processor carries out sub-frame processing using 20ms time windows to voice signal, using HMM algorithms, to each time window
In at different moments between shape, state transition probability observed, and obtains the observation vector { O in each time window1,
O2,.....ON, be identified for these observation vectors using DNN neutral nets, draw in each time window to wheelchair
Control instruction, and take two-stage triggering mechanism, i.e. first-level instruction and the recognition result that two level instructs is all correct, can just trigger control
Control instruction processed；
In three kinds of Model controls, by MYSQL servers to the eeg data of wheelchair user and the operation conditions of wheelchair into
Row monitoring, FPGA processor carry gsm module by serial ports, data are uploaded to service by the form of 4G networks or WIFI
Device, user log in the webpage using JSP+HTML+CSS independent developments, carry out long-range monitoring wheelchair and user's situation.
4. the method for the multi-modal wheelchair brain control according to claim 3 based on cloud platform, it is characterised in that：The FPGA
The method that the control instruction of wheelchair is analyzed and handled is by processor：Become by the fast Fourier for configuring FPGA processor
The IP kernel of fft algorithm is changed, realizes signal analysis and processing, wherein Transform length are arranged at 512 points, IP kernel principle
Figure, CLK be FFT IP Core system clock, reset_n be high level reset signal, sink_real [17:0] it is input number
Factually portion, souurce export for data；FPGA processor is by DAC chip conversion output to the method for the control voltage of wheelchair
For：To the control instruction of wheelchair after by FPGA processor, FPGA processor by frequency values be converted into " 3 ", " 4 ", " 5 ",
After " 7 " hexadecimal number, using DAC chip according to the operation voltage of wheelchair, corresponding digital quantization relation is calculated：Vo
(DAC A | B | C | D)=REF*CODE/256* (1+RNG bit value), wherein, Vo is the control voltage to wheelchair, and REF is
The output reference voltage of DAC, CODE are the digital quantity of output voltage values, and RNG is voltage doubling mode, and RNG represents to close defeated for 0
Go out doubling mode, RNG represents to open output voltage doubling mode for 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711251132.1A CN108056865A (en) | 2017-12-01 | 2017-12-01 | A kind of multi-modal wheelchair brain control system and method based on cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711251132.1A CN108056865A (en) | 2017-12-01 | 2017-12-01 | A kind of multi-modal wheelchair brain control system and method based on cloud platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108056865A true CN108056865A (en) | 2018-05-22 |
Family
ID=62135876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711251132.1A Pending CN108056865A (en) | 2017-12-01 | 2017-12-01 | A kind of multi-modal wheelchair brain control system and method based on cloud platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108056865A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109015635A (en) * | 2018-08-08 | 2018-12-18 | 西安科技大学 | A kind of service robot control method based on brain-machine interaction |
| CN109376843A (en) * | 2018-10-12 | 2019-02-22 | 山东师范大学 | EEG signals rapid classification method, implementation method and device based on FPGA |
| CN109800626A (en) * | 2018-11-28 | 2019-05-24 | 东莞理工学院 | A kind of acquiring brain waves analysis system and electric platforms control system and control method |
| CN110413116A (en) * | 2019-07-24 | 2019-11-05 | 西安交通大学 | A kind of Steady State Visual Evoked Potential brain-computer interface design method based on FPGA |
| CN110658742A (en) * | 2019-09-05 | 2020-01-07 | 四川省康复辅具技术服务中心 | Multi-mode cooperative control wheelchair control system and method |
| CN110812019A (en) * | 2019-11-06 | 2020-02-21 | 四川省康复辅具技术服务中心 | Intelligent wheelchair based on multi-mode control |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101897640A (en) * | 2010-08-10 | 2010-12-01 | 北京师范大学 | Novel movement imagery electroencephalogram control-based intelligent wheelchair system |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN103099693A (en) * | 2013-02-07 | 2013-05-15 | 王禹 | Handicapped-helping control system based on electroencephalogram/voice instructions |
| CN104078039A (en) * | 2013-03-27 | 2014-10-01 | 广东工业大学 | Voice recognition system of domestic service robot on basis of hidden Markov model |
| CN203898306U (en) * | 2014-06-06 | 2014-10-29 | 上海理工大学 | Brain wave collecting device based on wireless transmission |
| CN105011932A (en) * | 2015-08-11 | 2015-11-04 | 西安科技大学 | Fatigue driving electroencephalogram monitoring method based on degree of meditation and degree of concentration |
| CN105105773A (en) * | 2015-08-11 | 2015-12-02 | 西安科技大学 | Fatigue driving electroencephalogram monitoring method based on multi-threshold comparison |
| CN105787471A (en) * | 2016-03-25 | 2016-07-20 | 南京邮电大学 | Gesture identification method applied to control of mobile service robot for elder and disabled |
| CN106038098A (en) * | 2016-06-24 | 2016-10-26 | 张学海 | Computer processor based electric wheelchair control system and electric wheelchair |
| CN106199639A (en) * | 2016-06-24 | 2016-12-07 | 张学海 | The long-range positioning system (Loran) of wheelchair and wheelchair thereof |
| CN205942354U (en) * | 2016-08-19 | 2017-02-08 | 陈子杰 | Intelligent control system |
-
2017
- 2017-12-01 CN CN201711251132.1A patent/CN108056865A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101897640A (en) * | 2010-08-10 | 2010-12-01 | 北京师范大学 | Novel movement imagery electroencephalogram control-based intelligent wheelchair system |
| CN102309380A (en) * | 2011-09-13 | 2012-01-11 | 华南理工大学 | Intelligent wheelchair based on multimode brain-machine interface |
| CN103099693A (en) * | 2013-02-07 | 2013-05-15 | 王禹 | Handicapped-helping control system based on electroencephalogram/voice instructions |
| CN104078039A (en) * | 2013-03-27 | 2014-10-01 | 广东工业大学 | Voice recognition system of domestic service robot on basis of hidden Markov model |
| CN203898306U (en) * | 2014-06-06 | 2014-10-29 | 上海理工大学 | Brain wave collecting device based on wireless transmission |
| CN105011932A (en) * | 2015-08-11 | 2015-11-04 | 西安科技大学 | Fatigue driving electroencephalogram monitoring method based on degree of meditation and degree of concentration |
| CN105105773A (en) * | 2015-08-11 | 2015-12-02 | 西安科技大学 | Fatigue driving electroencephalogram monitoring method based on multi-threshold comparison |
| CN105787471A (en) * | 2016-03-25 | 2016-07-20 | 南京邮电大学 | Gesture identification method applied to control of mobile service robot for elder and disabled |
| CN106038098A (en) * | 2016-06-24 | 2016-10-26 | 张学海 | Computer processor based electric wheelchair control system and electric wheelchair |
| CN106199639A (en) * | 2016-06-24 | 2016-12-07 | 张学海 | The long-range positioning system (Loran) of wheelchair and wheelchair thereof |
| CN205942354U (en) * | 2016-08-19 | 2017-02-08 | 陈子杰 | Intelligent control system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109015635A (en) * | 2018-08-08 | 2018-12-18 | 西安科技大学 | A kind of service robot control method based on brain-machine interaction |
| CN109376843A (en) * | 2018-10-12 | 2019-02-22 | 山东师范大学 | EEG signals rapid classification method, implementation method and device based on FPGA |
| CN109376843B (en) * | 2018-10-12 | 2021-01-08 | 山东师范大学 | FPGA-based electroencephalogram signal rapid classification method, implementation method and device |
| CN109800626A (en) * | 2018-11-28 | 2019-05-24 | 东莞理工学院 | A kind of acquiring brain waves analysis system and electric platforms control system and control method |
| CN110413116A (en) * | 2019-07-24 | 2019-11-05 | 西安交通大学 | A kind of Steady State Visual Evoked Potential brain-computer interface design method based on FPGA |
| CN110658742A (en) * | 2019-09-05 | 2020-01-07 | 四川省康复辅具技术服务中心 | Multi-mode cooperative control wheelchair control system and method |
| CN110812019A (en) * | 2019-11-06 | 2020-02-21 | 四川省康复辅具技术服务中心 | Intelligent wheelchair based on multi-mode control |
| CN110812019B (en) * | 2019-11-06 | 2021-06-04 | 四川省康复辅具技术服务中心 | Intelligent wheelchair based on multi-mode control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108056865A (en) | A kind of multi-modal wheelchair brain control system and method based on cloud platform | |
| Hossain et al. | Hardware and software implementation of real time electrooculogram (EOG) acquisition system to control computer cursor with eyeball movement | |
| CN109366508A (en) | A kind of advanced machine arm control system and its implementation based on BCI | |
| CN107315478A (en) | A kind of Mental imagery upper limbs intelligent rehabilitation robot system and its training method | |
| CN107463249A (en) | Show the brain machine interface system and control method of VEP based on VR heads | |
| CN110993056A (en) | Hybrid active rehabilitation method and device based on mirror image neurons and brain-computer interface | |
| CN108721048A (en) | Rehabilitation training control method, computer readable storage medium and terminal | |
| CN107961120A (en) | A kind of intelligent wheelchair control system based on brain electric control | |
| CN113398470A (en) | Nerve regulation and control system capable of enhancing sleep | |
| CN106267557A (en) | A kind of brain control based on wavelet transformation and support vector machine identification actively upper limb medical rehabilitation training system | |
| Wang et al. | Study on a portable electrode used to detect the fatigue of tower crane drivers in real construction environment | |
| CN114652532B (en) | Multifunctional brain-controlled wheelchair system based on SSVEP and attention detection | |
| Jeong et al. | Design of a brain-controlled robot arm system based on upper-limb movement imagery | |
| Shi et al. | A Novel Human‐Machine Collaboration Model of an Ankle Joint Rehabilitation Robot Driven by EEG Signals | |
| Su et al. | Intelligent wheelchair control system based on BCI and the image display of EEG | |
| CN110688013A (en) | English keyboard spelling system and method based on SSVEP | |
| Liu et al. | Detection of lower-limb movement intention from EEG signals | |
| CN206147520U (en) | A data acquisition device that is used for based on brain -computer interface control virtual reality that motion is imagined and P300 combines together | |
| Chu et al. | Robot-assisted rehabilitation system based on SSVEP brain-computer interface for upper extremity | |
| CN112370259A (en) | Control system of brain-controlled wheelchair based on SSVEP | |
| CN115444717B (en) | Limb function rehabilitation training method and system based on brain-computer interface | |
| CN110166645A (en) | A kind of call service system based on electro-ocular signal control | |
| LaRocco et al. | Optimizing computer–brain Interface parameters for non-invasive brain-to-brain Interface | |
| Winod et al. | Towards a Brain-Computer Interface based control for next generation electric wheelchairs | |
| CN106371590A (en) | High-performance motor imagery online brain-computer interface system based on OpenVIBE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180522 |