CN105807924A - Flexible electronic skin based interactive intelligent translation system and method - Google Patents
Flexible electronic skin based interactive intelligent translation system and method Download PDFInfo
- Publication number
- CN105807924A CN105807924A CN201610128250.2A CN201610128250A CN105807924A CN 105807924 A CN105807924 A CN 105807924A CN 201610128250 A CN201610128250 A CN 201610128250A CN 105807924 A CN105807924 A CN 105807924A
- Authority
- CN
- China
- Prior art keywords
- unit
- translation
- lip
- flexible electronic
- english
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08K—Use of inorganic or non-macromolecular organic substances as compounding ingredients
- C08K3/00—Use of inorganic substances as compounding ingredients
- C08K3/02—Elements
- C08K3/04—Carbon
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08K—Use of inorganic or non-macromolecular organic substances as compounding ingredients
- C08K3/00—Use of inorganic substances as compounding ingredients
- C08K3/02—Elements
- C08K3/08—Metals
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08K—Use of inorganic or non-macromolecular organic substances as compounding ingredients
- C08K7/00—Use of ingredients characterised by shape
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/24—Speech recognition using non-acoustical features
- G10L15/25—Speech recognition using non-acoustical features using position of the lips, movement of the lips or face analysis
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/34—Adaptation of a single recogniser for parallel processing, e.g. by use of multiple processors or cloud computing
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08J—WORKING-UP; GENERAL PROCESSES OF COMPOUNDING; AFTER-TREATMENT NOT COVERED BY SUBCLASSES C08B, C08C, C08F, C08G or C08H
- C08J2383/00—Characterised by the use of macromolecular compounds obtained by reactions forming in the main chain of the macromolecule a linkage containing silicon with or without sulfur, nitrogen, oxygen, or carbon only; Derivatives of such polymers
- C08J2383/04—Polysiloxanes
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08K—Use of inorganic or non-macromolecular organic substances as compounding ingredients
- C08K3/00—Use of inorganic substances as compounding ingredients
- C08K3/02—Elements
- C08K3/08—Metals
- C08K2003/085—Copper
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08K—Use of inorganic or non-macromolecular organic substances as compounding ingredients
- C08K2201/00—Specific properties of additives
- C08K2201/011—Nanostructured additives
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08L—COMPOSITIONS OF MACROMOLECULAR COMPOUNDS
- C08L2203/00—Applications
- C08L2203/16—Applications used for films
-
- C—CHEMISTRY; METALLURGY
- C08—ORGANIC MACROMOLECULAR COMPOUNDS; THEIR PREPARATION OR CHEMICAL WORKING-UP; COMPOSITIONS BASED THEREON
- C08L—COMPOSITIONS OF MACROMOLECULAR COMPOUNDS
- C08L2203/00—Applications
- C08L2203/20—Applications use in electrical or conductive gadgets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Abstract
The invention discloses a flexible electronic skin based interactive intelligent translation system and method, and particularly relates to an interactive intelligent translation apparatus combining flexible electronic skin with a Bluetooth earphone. When a user wears the apparatus and reads an English word or sentence at will, the English word or sentence can be automatically translated into Chinese and a result is read through the Bluetooth earphone, and grammar information such as use of word groups, collocation of the word groups or the like is displayed in a display screen in real time. Therefore, great convenience is provided for English learning, and translation and use of the word can be obtained in real time through the apparatus. Moreover, the apparatus is more helpful for foreign tourists, and English can be translated into Chinese only by reading English which is not understood, so that the problem of language barriers encountered by the foreign tourists is solved. The apparatus as a novel interactive translation device is simple in structure, light in weight, high in reliability, convenient to carry, low in cost, high in practicality and favorable for industrialization.
Description
Technical field
The present invention relates to the fields such as sensor, signal processing technology and radio sensing network, particularly relate to a kind of interactive intelligent translation system and method based on flexible electronic skin, belong to intelligent sound identification field.
Background technology
Intelligence wearable device is as a kind of novel wearable device, and its research temperature constantly promotes, and its application is also more and more extensive.Wearable device is not only a kind of hardware device, realizes powerful function alternately by software support and data interaction, high in the clouds especially, and our life, perception will be brought very big transformation by wearable device.
English study is very big problem for domestic child, is also the thing of the most headache of the domestic head of a family.In the study of English, word and grammer are again the most key parts.Traditional translation technology and equipment have very big drawback, in-convenience in use, it is necessary to word, a word input carry out translating and grammer identification, consume the time and efforts studied English in a large number, and learning effect is bad.It is proposed that this intelligent apparatus can easily by reading a word, the real-time translation obtaining word and about syntactic informations such as the usage of word and collocation, not only convenient and swift but also do not expend the energy of student, and substantially increase the efficiency studied English.
Along with going deep into of economic globalization, journey abroad, the activity such as be on home leave or go on business are more and more frequent, family's travelling or various free walker are very universal, but there is the problem that considerable people can run into language obstacle in airport, hotel, park or other stroke, such as some Sign Boards of None-identified (such as, traffic marking board, menu, route map etc.) information, and carry out being manually entered the very inconvenient and poor real of translation by mobile phone or computer.If by present device, as long as what directly those Sign Boards of reading just can be real-time is translated, thus dramatically reducing the worry of language obstacle.It addition, various international meetings are more and more general, even if there being Interpreter Officer to help translation, but yet suffer from significantly translating hysteresis quality.Apparatus of the present invention can well solve this problem, for purposes such as various international conferences, Chines-foreign academic exchanges meeting, business negotiations, has significantly high practical value.
Intelligent translation apparatus popular at present is all based on shooting technology and catches the shape of the mouth as one speaks, then passes through software and identifies word and translate.This method with it is proposed that directly measure speak time the vibration of lip surrounding the signal of telecommunication compared with identifying the method for lip reading, not only to realize difficulty big, complicated operation, complex operation step, equipment needed thereby (video camera etc.) cost is high, it has not been convenient to carries, is unfavorable for large-scale industrialization.
Summary of the invention
It is an object of the invention to propose a kind of interactive intelligent translation system and method based on flexible electronic skin.Overcome shortcoming, the efficiency being greatly improved translation and the efficiency studied English such as existing translating equipment complicated operation, poor, the poor real of portability.
For reaching this purpose, the present invention by the following technical solutions: interactive intelligent translation system and method that a kind of flexible electronic skin combines with bluetooth earphone, including lip reading signal gathering unit, character recognition unit, translation unit, report unit；
Described lip reading signal gathering unit includes flexible electronic skin, the change produced when being used for the motion gathering lip, and exports as electronic signals；
Described character recognition unit is for comparing the signal of lip reading signal gathering unit collection with the data of storage in its data base, it is achieved character recognition；
Described translation unit is to be input in the translation software such as CAJ by the result of character recognition, and the language collected is translated into the language wanting to obtain.And in electronic translation software, obtaining the grammer of the usage about this word or collocation, the display in real time for lower unit is prepared；
Described report unit is: the translation result in high in the clouds can be delivered in Bluetooth earphone device by blue-tooth device is counter, and what bluetooth earphone can be real-time reports out by earphone by the translation result that transmission obtains.Can also pass through that blue-tooth device is counter deliver on mobile phone, by the real-time report of the speaker of mobile phone out.
Further, described system also includes display unit, described display unit is: in high in the clouds electronic translation software obtain the syntactic information such as the usage about this word and collocation can by the blue-tooth device display that to be sent on LED display real-time, can certainly being sent on mobile phone A PP either directly through bluetooth, real-time display is about the syntactic information of this phrase.
Further, described system also includes feedback reminding unit, and described feedback reminding unit is to realize the result according to this identification by 3 LED to remind operation next time, realizes by algorithm is write recognizer.
Further, described flexible electronic skin prepares by the following method: Cu nano wire-graphene film cuts the strip of 2cm × 0.5cm, two ends elargol is stained with copper cash, embed in polydimethylsiloxane (PDMS) again, it is placed in 2h in 70 DEG C of air dry ovens, makes flexible electronic skin.The preparation method of described Cu nano wire-graphene film is as follows:
(1) in 20-25mL ethylene glycol solution, 20-42mgCu nano wire is added, 48-54mg ascorbic acid is added after being uniformly dispersed, 3-3.8mL graphene oxide is added after being uniformly dispersed, move in hydrothermal reaction kettle, it is placed in 120-160 DEG C of air dry oven and reacts 4-6h, it is cooled to room temperature, just obtains Cu nano wire-Graphene composite aquogel.
(2) Cu prepared nano wire-Graphene composite aquogel is placed in 0.5wt% hydrazine hydrate solution dialysis 16h, further take out to pour in 150mL deionized water and be uniformly dispersed, obtain suspension, then with core defecator sucking filtration, just obtain Cu nano wire-graphene film.
Further, described data base includes English alphabet data base and International Phonetic Symbols data base；Set up by the following method: flexible electronic skin is sticked in lip surrounding, when 26 English alphabets or 48 International Phonetic Symbols are read aloud in collection, the strain that lip motion produces, and it is stored in character recognition unit as electronic signals；Each alphabetical or each phonetic symbol has a characteristic of correspondence waveform；
The recognition methods of a kind of described system characters recognition unit, comprises the following steps:
(1) flexible electronic skin is sticked in lip surrounding, the strain produced when gathering lip motion, and it is sent to character recognition unit as electronic signals, each phonetic symbol or letter produce a signal waveform；
(2) character recognition unit utilizes data base, by artificial neural network recognizer, each waveform in the signal of telecommunication is identified, and identifies letter or the phonetic symbol of each corresponding wave band；
(3) recognition result is translated by translation unit；
(4) translation result carries out voice broadcast by reporting unit.
Further, described recognition result is: identify that the letter of each wave band obtained is according to sequencing superposition.
Described a kind of interactive intelligent translation system and method based on flexible electronic skin, native system has very strong interactive.When flexible sensor is worn on mouth surrounding by the other side, when oneself has on bluetooth earphone, the content that can directly the other side be spoken is translated as the language that can understand from the language do not understood, for purposes such as various international conferences, Chines-foreign academic exchanges meeting and business negotiations.Can without band Interpreter Officer, convenient and efficiently.
Further, native system can also realize two people saying different language and carry out real-time exchange, is attached to the corners of the mouth of object A by flexible skin sensor, and object B wears bluetooth earphone, hears the object A voice translated after speaking.Simultaneously object B can input him on the touch screen plate of display screen and wants to reply, and information can be transmitted high in the clouds by the blue-tooth device on touch screen plate, and high in the clouds is being transmitted back on bluetooth earphone after realizing translation, thus can realize real-time dialogue.
The advantage of present system is in that: this intelligent apparatus being identified based on the signal of telecommunication measuring lip vibration and then translating is easy to carry, and equipment is simple, and cost is low, and volume is little, and real-time is good, and is beneficial to industrialization, has good application prospect.Finally the terminal of this identification system is made the form of mobile phone A PP, in that context it may be convenient to by real-time the seeing the result of translation and read aloud the result of translation of this software, thus improving recognition efficiency.When speaking with the shooting of existing video camera lip photo and with compared with the technology of image-recognizing method identification lip reading, this little based on measuring the lip vibration intelligent apparatus volume that is identified of the signal of telecommunication, easy to carry, equipment is simple, cost is low, real-time is good, and is beneficial to industrialization, has good application prospect.
Accompanying drawing explanation
Be more fully described the exemplary embodiment of the present invention by referring to accompanying drawing, the above and other aspect of the present invention and advantage will become more easily clear, in the accompanying drawings:
Fig. 1 is the structural representation of a kind of wearable device for intelligent translation that the specific embodiment of the invention provides；
Fig. 2 is that apparatus of the present invention carry out high in the clouds identification and translation by bluetooth and realize the structural representation that real-time bluetooth earphone is reported and display screen shows；
Flexible skin sensor is attached to lip surrounding to survey the simulation design sketch of lip vibration during sounding by Fig. 3；
When Fig. 4 is to read English alphabet " R " and " S ", measure the signal of telecommunication being attached to lip surrounding flexible skin sensor with Keithley 2400 table；
When Fig. 5 is with different 3 times English phrase " nanomaterials " of tone liaison, measure the signal of telecommunication being attached to lip surrounding flexible skin sensor with Keithley 2400 table；
Fig. 6 implements in claim 5, by the artificial neural network recognizer software interface with MATLAB software identification letter " A "；
Fig. 7 implements in claim 6, by the artificial neural network recognizer software interface with MATLAB software identification letter " hello "；
Fig. 8 implements, in claim 7, namely to be identified the MATLAB interface of letter " Q " by the superposition of letter " K " and " U " by superposition identification；
Fig. 9 is the identification process flow diagram flow chart of artificial neural network recognizer；
Detailed description of the invention
The present invention is a kind of interactive intelligent translation system and method based on flexible electronic skin, including lip reading signal gathering unit, character recognition unit, translation unit, report unit, display unit, feedback reminding unit.Hardware specifically includes that the flexible skin sensor being attached to user's mouth surrounding, small display screen and bluetooth earphone and 3 the LED warning lights green, red, yellow for feedback user.Intelligent apparatus also includes blue tooth interface circuit, the wireless communication interface being connected with the exterior terminal such as mobile phone, computer, gives the lithium battery of every part power supply.
Described lip reading signal gathering unit, critical piece is flexible electronic skin, the manufacture method of flexible electronic skin is: Cu nano wire-graphene film cuts the strip of 2cm × 0.5cm, two ends elargol is stained with copper cash, embed in polydimethylsiloxane (PDMS) again, it is placed in 2h in 70 DEG C of air dry ovens, makes flexible electronic skin.Wherein, the preparation method of Cu nano wire-graphene film is as follows:
(1) in 20-25mL ethylene glycol solution, 20-42mgCu nano wire is added, 48-54mg ascorbic acid is added after being uniformly dispersed, 3-3.8mL graphene oxide is added after being uniformly dispersed, move in hydrothermal reaction kettle, it is placed in 120-160 DEG C of air dry oven and reacts 4-6h, it is cooled to room temperature, just obtains Cu nano wire-Graphene composite aquogel.
(2) Cu prepared nano wire-Graphene composite aquogel is placed in 0.5wt% hydrazine hydrate solution dialysis 16h, further take out to pour in 150mL deionized water and be uniformly dispersed, obtain suspension, then with core defecator sucking filtration, just obtain Cu nano wire-graphene film.
Flexible electronic skin has higher susceptiveness and stability, and its shape and size can be customized according to everyone nozzle type, it is ensured that flexible electronic skin can be close to the surrounding of lip, the vibration of lip when sensitive collection is spoken.
Described character recognition unit, the signal of telecommunication exported by flexible electronic skin, uses the recognizer of artificial neural network that the character of the signal of telecommunication collected Yu foundation being judged, data base contrasts, exports immediate result.
Data base sets up by training, and before user uses, first has to the nozzle type according to user and pronunciation custom is made and flexible skin sensor can be made completely to be close to the intelligent apparatus of lip surrounding.Then need to pay user to be trained, obtain meeting the lip reading vibration signal of telecommunication of 26 letters of user pronunciation custom and 48 International Phonetic Symbols, these signals are set up a data base.
In identification process, user brings the intelligent apparatus meeting oneself nozzle type that we design, when a user speaks, owing to the change of nozzle type makes lip four weekly assembly produce vibration, it is close to the flexible skin sensor of lip to follow the vibration of lip simultaneously and can produce the Light deformation on surface, thus causing the deformation of foil gauge in resistance strain gage sensor, foil gauge resistance value is changed, so that the magnitude of voltage of externally output also can change on foil gauge, this voltage signal is sent to high in the clouds by the blue tooth interface of device, recognizer by the artificial neural network of lip reading identification, signal in data base and these signals are carried out matching identification or superposition identification one by one, obtain the result of lip reading identification.Then these results are input in the electronic translation softwares such as CAJ and translate, and obtain all syntactic informations about this word.Then pass through the result that report when blue tooth interface passes to bluetooth earphone interior-excess is translated.And all syntactic informations about this phrase are passed to by blue tooth interface simultaneously and show in display screen.This process can be existing at mobile phone A PP interior-excess simultaneously.
Signal of telecommunication transmission between lip reading signal gathering unit and character recognition unit, for technological means commonly used in the art, it is possible to adopt copper cash as above directly to transmit, it is also possible to be transmitted by wireless network.The program of artificial neural network is write single-chip microcomputer (in character recognition unit), incoming for the signal of telecommunication collected single-chip microcomputer, in the process that single-chip microcomputer interior-excess now identifies, and exports result by serial communication interface.The incoming high in the clouds of data (character recognition unit) that will can also be collected very easily by the blue tooth interface of intelligent apparatus or home control network communication protocol, realize the identification of character beyond the clouds, terminal (character image) is delivered to by counter for recognition result, due to high in the clouds character vector storehouse more comprehensively and sufficient so that discrimination improves.
Described display unit, it is possible to be a small-sized display screen, in high in the clouds electronic translation software obtain the syntactic information such as the usage about this word and collocation can by the blue-tooth device display that to be sent on display screen real-time, for prior art well known in the art.Such as, being connected with the serial line interface of display driver circuit by the serial line interface of high in the clouds translation unit, the display driver circuit of display device drives the result of display screen display translation by data/address bus and address bus.The serial line interface of blue tooth interface circuit can also be connected with the serial line interface of single-chip microcomputer, be shown in the terminal such as mobile phone or computer by the bluetooth information such as grammer and collocation by translation result.
Described report unit, it is possible to be a little speaker.The result of translation is carried out real-time broadcasting by speaker, for prior art well known in the art.Such as, the EBI of single-chip microcomputer is connected with voice driven circuit, is then attached to the port of speaker, by the voice broadcast program of write in single-chip microcomputer by the result of identification by real-time the reading out of speaker.
Intelligent apparatus also includes feedback user and plays 3 LED of reminding effect.Translating after successfully when identifying, middle green LED lamp is bright, is used for reminding user, and this identifies that translation is over, it is possible to carry out translation next time；When can not identifying result, then the yellow LED lamp on right side is bright, is used for reminding user that this identification is broken down, and identifies and can not translate result, again lip reading pronunciation of typing；When, after three same typings, still can not identifying result, the red LED lamp being at this moment positioned at the lower left corner is bright, reminds the identification of this word of user or phrase to have no result, and user needs to be said differently or skip this word or expression.Above-mentioned functions can be realized by simple logic circuit, does not do detailed statement at this.
The recognizer of above-mentioned neutral net is: artificial neural network ANN, is a kind of engineering system simulating its structure and intelligent behavior on the understanding basis to human brain tissue structure and operating mechanism.Neural network filter process is divided into two steps, first it is learning process, by a large amount of learning samples, network is trained, constantly connection weights and threshold value are adjusted according to certain learning rules, finally making network have certain desired output, namely this output be correctly to be categorized into by training sample in its generic, now it is believed that network is study has arrived the inherent law between input sample.Followed by categorizing process, apply weights and threshold value that above learning process trains, the sample of arbitrary feeding network is classified.
Owing to, in English, it is aphonic for having some letter in a lot of word, this results in the phonetic symbol with pronunciation or when syllable identifies word, although the sound sent in speaker is accurately, but the spelling of the word of display is wrong, causes discrimination to reduce.Therefore, when writing identification software, finally the word of display once can be audited, can letter aphonic in English word, namely so-called mute adds according to the word-building of English pronunciation.On this basis, the later stage will change into, more comprehensively English pronunciation rule, the algorithm identified and screen and add.
The terminal of this intelligent translation system finally can being made the form of mobile phone A PP, as long as so opening this software just can see the result of translation very easily in real time, and reading aloud the result of translation, thus more convenient and in hgher efficiency.
Technical scheme is further illustrated below in conjunction with accompanying drawing and by detailed description of the invention.Should be appreciated that specific embodiment described herein is only in order to explain the present invention, is not intended to limit the present invention.
The present invention provides a kind of intelligent translation method.
Embodiment 1
A kind of interactive intelligent translation device based on flexible electronic skin, schematic appearance is as shown in Figure 1, attachment structure block diagram is as in figure 2 it is shown, this device includes: lip reading signal gathering unit, character recognition unit, translation unit, report unit, display unit, feedback reminding unit.It specifically includes that the flexible skin sensor being attached to user's mouth surrounding, small display screen and bluetooth earphone and 3 the LED warning lights green, red, yellow for feedback user.Intelligent apparatus also includes blue tooth interface circuit, the wireless communication interface being connected with the exterior terminal such as mobile phone, computer, gives the lithium battery of every part power supply.
As it is shown on figure 3, flexible electronic skin sensor is attached to the surrounding of mouth, the making material of flexible skin sensor is the extraordinary new material Graphene of electric conductivity so that its susceptiveness and stability are very good.The surrounding of user's lip it is close to, for gathering the signal of telecommunication of lip vibration when speaking during use.And the shape of flexible electronic skin needs the nozzle type according to user to make to measure, it is ensured that sensor can fully gather the characteristic quantity of lip vibration during user pronunciation.
As shown in Figure 4, when reading English alphabet " R " and " S ", the signal of telecommunication being attached to lip surrounding flexible skin sensor is measured with Keithley 2400 table, can be seen that from oscillogram the characteristic quantity of the oscillogram of each letter is different, there is obvious diversity, so that the identification realizing letter is possibly realized.
As it is shown in figure 5, during with different 3 times English phrase " nanomaterials " of tone liaison, measure the signal of telecommunication being attached to lip surrounding flexible skin sensor with Keithley 2400 table.From oscillogram it can be seen that repeatability is especially good, although the height of tone can affect the amplitude of oscillogram, but the characteristic quantity of waveform is constant, illustrate to realize identifying by the difference of each letter oscillogram characteristic quantity.
Embodiment 2
As shown in Figure 6,7, native system achieves man-to-man identification, also achieves the continuous identification together of several letter.In the identification of character, native system can pass through the algorithm of artificial neural network and realize simple English 26 alphabetical identifications, discrimination height very.And, native system is capable of once identifying 4,5 letters simultaneously, and ensures that the order of letter is constant, and in the recognition result namely exported, letter type and order are completely the same with what input.Such as, lip vibration signal when reading 5 letter " hello " is measured with flexible skin sensor, this signal waveform is inputted the identification system having built up in MATLAB, system can be passed through the recognizer of artificial neural network and each letter in this signal carries out with 26 the alphabetical data bases having built up contrast identification respectively, eventually exports recognition result " hello " on the interface of MATLAB.
Embodiment 3
As shown in Figure 8, native system achieves the superposition identification of letter.MATALB establishes the data base of the lip reading vibration signal of telecommunication of English alphabet " K " and " U ", then lip reading vibration signal of telecommunication when reading English alphabet " Q " is measured, it is inputted the identification system of MATLAB, the pronunciation [kju :] of letter " Q " can be identified the laminated structure of the phonetic symbol [ju :] being the phonetic symbol [kei] of letter " K " and alphabetical " U " by system respectively through the recognizer of neural network, then obtain, further according to the word-building of phonetic symbol, the lip reading vibration signal of telecommunication that this is letter " Q ", finally can export recognition result on the control panel of MATLAB is letter " Q ". the lip reading identification system that we set up can not only realize man-to-man identification, the power of superposition identification can also be realized.Namely due to the ultimate unit that syllable is pronunciation, the pronunciation of any word, it is all be decomposed into syllable one by one to read aloud.Lip vibration signal when someone reads 48 International Phonetic Symbols of English is measured respectively with flexible skin sensor, set up a MATLAB data base, then lip vibration signal when he reads any one English word is measured, input a signal in the MATLAB identification system having built up, system can by the recognizer of artificial neural network, by each section of waveform input signal respectively by comparing with the waveform of 48 International Phonetic Symbols data bases having built up successively smoothly, the corresponding International Phonetic Symbols of signal waveform each section or syllable is identified successively by recognizer, then the International Phonetic Symbols these identified in order form word according to the grammer of English pronunciation phonetic symbol, then the interface of MATLAB will reveal whether the letter that identifies.Thus, identifying letter with the superposition identification of phonetic symbol, the superposition identification followed by letter identifies phrase, finally identifies sentence with the superposition identification of phrase, and by that analogy, we finally can be achieved with the identification of complete people's works and expressions for everyday use.
Its use procedure is: first gather the lip reading vibration signal of telecommunication of 48 International Phonetic Symbols according to the nozzle type of user and pronunciation custom, these signals are set up a data base.When a user speaks, owing to the change of nozzle type makes lip four weekly assembly produce vibration, it is close to the flexible skin sensor of lip to follow the vibration of lip simultaneously and can produce the Light deformation on surface, thus causing resistance strain gage deformation, resistance value changes, on foil gauge, the magnitude of voltage of externally output also can change, this voltage signal is sent to high in the clouds by the blue tooth interface of device, recognizer by the artificial neural network of lip reading identification, signal in data base and these signals are carried out matching identification or superposition identification one by one, obtains the result of lip reading identification.Then these results are input in the electronic translation softwares such as CAJ and translate, and obtain all syntactic informations about this word.Then pass through the result that report when blue tooth interface passes to bluetooth earphone interior-excess is translated.And all syntactic informations about this phrase are passed to by blue tooth interface simultaneously and show in display screen.This process can be existing at mobile phone A PP interior-excess simultaneously.
The identification process flow diagram flow chart of artificial neural network recognizer as shown in Figure 9, the recognizer of above-mentioned neutral net is: artificial neural network ANN, is a kind of engineering system simulating its structure and intelligent behavior on the understanding basis to human brain tissue structure and operating mechanism.Neural network filter process is divided into two steps, first it is learning process, by a large amount of learning samples, network is trained, constantly connection weights and threshold value are adjusted according to certain learning rules, finally making network have certain desired output, namely this output be correctly to be categorized into by training sample in its generic, now it is believed that network is study has arrived the inherent law between input sample.Then it is exactly categorizing process, applies weights and threshold value that above learning process trains, the sample of arbitrary feeding network is classified.
Owing to, in English, it is aphonic for having some letter in a lot of word, this results in the phonetic symbol with pronunciation or when syllable identifies word, although the sound sent in speaker is accurately, but the spelling of the word of display is wrong, causes discrimination to reduce.Therefore, when writing identification software, the word of display finally can once be audited by native system, can letter aphonic in English word, and namely so-called mute adds according to the word-building of English pronunciation.On this basis, later stage will change into, more comprehensively English pronunciation rule, the algorithm identified and screen and add, such as " vowel rules of pronunciation in stressed syllable ", " rules of pronunciation of vowel combination " etc., improve the accuracy rate of identification with this.
The terminal of this identification system finally can being made the form of mobile phone A PP, as long as so opening this software just can see the result of translation very easily in real time, and reading aloud the result of translation, thus more convenient and in hgher efficiency.
Claims (8)
1. the interactive intelligent translation system that a flexible electronic skin combines with bluetooth earphone, it is characterised in that: include lip reading signal gathering unit, character recognition unit, translation unit, report unit；
Described lip reading signal gathering unit includes flexible electronic skin, the strain produced during for gathering lip motion, and exports as electronic signals；
Described character recognition unit is for comparing the signal of lip reading signal gathering unit collection with the data of storage in its data base, it is achieved character recognition；
Described translation unit is for translating the result of character recognition, for instance translator of Chinese is become English；
Described report unit is for reporting the translation result of translation unit.
2. system according to claim 1, it is characterised in that described system also includes display unit, for showing information such as the grammer of translation unit translation result and collocation in real time.
3. system according to claim 1, it is characterised in that described lip reading signal gathering unit, character recognition unit, translation unit, report unit pass sequentially through network and be connected.
4. system according to claim 1, it is characterized in that, described system also includes feedback reminding unit, and described feedback reminding unit is to realize the result according to this identification by 3 LED to remind operation next time, realizes by algorithm is write recognizer.
5. system according to claim 1, it is characterized in that, described flexible electronic skin prepares by the following method: Cu nano wire-graphene film cuts the strip of 2cm × 0.5cm, two ends elargol is stained with copper cash, embed in polydimethylsiloxane (PDMS) again, it is placed in 2h in 70 DEG C of air dry ovens, makes flexible electronic skin.The preparation method of described Cu nano wire-graphene film is as follows:
(1) in 20-25mL ethylene glycol solution, 20-42mgCu nano wire is added, 48-54mg ascorbic acid is added after being uniformly dispersed, 3-3.8mL graphene oxide is added after being uniformly dispersed, move in hydrothermal reaction kettle, it is placed in 120-160 DEG C of air dry oven and reacts 4-6h, it is cooled to room temperature, just obtains Cu nano wire-Graphene composite aquogel.
(2) Cu prepared nano wire-Graphene composite aquogel is placed in 0.5wt% hydrazine hydrate solution dialysis 16h, further take out to pour in 150mL deionized water and be uniformly dispersed, obtain suspension, then with core defecator sucking filtration, just obtain Cu nano wire-graphene film.
6. system according to claim 1, it is characterised in that described data base includes English alphabet data base and International Phonetic Symbols data base；Set up by the following method: flexible electronic skin is sticked in lip surrounding, gather and read aloud the strain that when 26 English alphabets or 48 International Phonetic Symbols, lip motion produces, and be stored in character recognition unit as electronic signals；Each alphabetical or each phonetic symbol has a characteristic of correspondence waveform；
7. the interpretation method of system described in a claim 1, it is characterised in that comprise the following steps:
(1) flexible electronic skin is sticked in lip surrounding, the strain produced when gathering lip motion, and it is sent to character recognition unit as electronic signals, each phonetic symbol or letter produce a signal waveform；
(2) character recognition unit utilizes data base, by artificial neural network recognizer, each waveform in the signal of telecommunication is identified；
(3) recognition result is translated by translation unit；
(4) translation result carries out voice broadcast by reporting unit.
8. method according to claim 7, it is characterised in that described recognition result is: after the letter identifying each wave band obtained is overlapped according to sequencing, the recognition result of composition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610128250.2A CN105807924A (en) | 2016-03-07 | 2016-03-07 | Flexible electronic skin based interactive intelligent translation system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610128250.2A CN105807924A (en) | 2016-03-07 | 2016-03-07 | Flexible electronic skin based interactive intelligent translation system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105807924A true CN105807924A (en) | 2016-07-27 |
Family
ID=56466913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610128250.2A Pending CN105807924A (en) | 2016-03-07 | 2016-03-07 | Flexible electronic skin based interactive intelligent translation system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105807924A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407289A (en) * | 2016-08-29 | 2017-02-15 | 乐视控股（北京）有限公司 | Method and device for processing foreign language audio information |
| CN108268452A (en) * | 2018-01-15 | 2018-07-10 | 东北大学 | A kind of professional domain machine synchronous translation device and method based on deep learning |
| CN108922537A (en) * | 2018-05-28 | 2018-11-30 | Oppo广东移动通信有限公司 | Audio identification methods, device, terminal, earphone and readable storage medium storing program for executing |
| CN113963528A (en) * | 2021-10-20 | 2022-01-21 | 浙江理工大学 | Man-machine interaction system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000338987A (en) * | 1999-05-28 | 2000-12-08 | Mitsubishi Electric Corp | Utterance start monitor, speaker identification device, voice input system, speaker identification system and communication system |
| CN202352332U (en) * | 2011-11-30 | 2012-07-25 | 李扬德 | Portable type lip language identifier |
| CN103020048A (en) * | 2013-01-08 | 2013-04-03 | 深圳大学 | Method and system for language translation |
| CN103294199A (en) * | 2013-06-09 | 2013-09-11 | 华东理工大学 | Silent information identifying system based on facial muscle sound signals |
| CN104575500A (en) * | 2013-10-24 | 2015-04-29 | 中国科学院苏州纳米技术与纳米仿生研究所 | Application of electronic skin in voice recognition, voice recognition system and voice recognition method |
| CN104801244A (en) * | 2015-04-09 | 2015-07-29 | 浙江理工大学 | Method for preparing three-dimensional graphene-copper nanowire composite aerogel |
-
2016
- 2016-03-07 CN CN201610128250.2A patent/CN105807924A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000338987A (en) * | 1999-05-28 | 2000-12-08 | Mitsubishi Electric Corp | Utterance start monitor, speaker identification device, voice input system, speaker identification system and communication system |
| CN202352332U (en) * | 2011-11-30 | 2012-07-25 | 李扬德 | Portable type lip language identifier |
| CN103020048A (en) * | 2013-01-08 | 2013-04-03 | 深圳大学 | Method and system for language translation |
| CN103294199A (en) * | 2013-06-09 | 2013-09-11 | 华东理工大学 | Silent information identifying system based on facial muscle sound signals |
| CN104575500A (en) * | 2013-10-24 | 2015-04-29 | 中国科学院苏州纳米技术与纳米仿生研究所 | Application of electronic skin in voice recognition, voice recognition system and voice recognition method |
| CN104801244A (en) * | 2015-04-09 | 2015-07-29 | 浙江理工大学 | Method for preparing three-dimensional graphene-copper nanowire composite aerogel |
Non-Patent Citations (1)
| Title |
|---|
| TAKEO YAMADA ET AL: "A Stretchable Carbon Nanotube Strain Sensor for Human-motion Detection", 《NATURE NANOTECHNOLOGY》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407289A (en) * | 2016-08-29 | 2017-02-15 | 乐视控股（北京）有限公司 | Method and device for processing foreign language audio information |
| CN108268452A (en) * | 2018-01-15 | 2018-07-10 | 东北大学 | A kind of professional domain machine synchronous translation device and method based on deep learning |
| CN108922537A (en) * | 2018-05-28 | 2018-11-30 | Oppo广东移动通信有限公司 | Audio identification methods, device, terminal, earphone and readable storage medium storing program for executing |
| CN108922537B (en) * | 2018-05-28 | 2021-05-18 | Oppo广东移动通信有限公司 | Audio recognition method, device, terminal, earphone and readable storage medium |
| CN113963528A (en) * | 2021-10-20 | 2022-01-21 | 浙江理工大学 | Man-machine interaction system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105807925A (en) | Flexible electronic skin based lip language identification system and method | |
| CN105551327A (en) | Interactive pronunciation correcting system and method based on soft electronic skin | |
| CN108108340B (en) | Dialogue interaction method and system for intelligent robot | |
| CN203861914U (en) | Pet robot | |
| CN105807924A (en) | Flexible electronic skin based interactive intelligent translation system and method | |
| CN205281861U (en) | Interactive intelligence learning machine | |
| CN105206123B (en) | A kind of deaf and dumb patient's ac equipment | |
| CN107274736A (en) | A kind of interactive Oral English Practice speech sound teaching apparatus in campus | |
| CN107067838A (en) | A kind of intelligent tutoring system | |
| CN107657906A (en) | A kind of blind person based on magnetic fluid reads smart electronicses book and its display methods | |
| CN101494816A (en) | Hearing-aid device and method suitable for anacusia patient | |
| CN104361787A (en) | System and method for converting signals | |
| CN108766416A (en) | Audio recognition method and Related product | |
| CN1331080C (en) | Virtual keyboard and robot control system by brain electric signal | |
| CN205177193U (en) | Deaf and dumb patient AC installation | |
| CN107862021A (en) | A kind of learning method and system based on intelligent microphone apparatus | |
| CN206907294U (en) | A kind of deaf-mute's Special alternating-current glasses | |
| CN203870835U (en) | Interactive teaching system based on electronic whiteboard | |
| CN115019820A (en) | Touch sensing and finger combined sounding deaf-mute communication method and system | |
| CN209625781U (en) | Bilingual switching device for child-parent education | |
| CN201025562Y (en) | A portable intelligent language learning machine | |
| CN202601021U (en) | Synchronous English translation exercise device | |
| CN207925131U (en) | Read aloud equipment | |
| CN111761588A (en) | Artificial intelligence traditional Chinese medicine meridian treatment service robot | |
| CN206700779U (en) | A kind of voice interaction toy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160727 |
|
| RJ01 | Rejection of invention patent application after publication |