CN109300478A - A kind of auxiliary Interface of person hard of hearing - Google Patents

A kind of auxiliary Interface of person hard of hearing Download PDF

Info

Publication number
CN109300478A
CN109300478A CN201811027365.8A CN201811027365A CN109300478A CN 109300478 A CN109300478 A CN 109300478A CN 201811027365 A CN201811027365 A CN 201811027365A CN 109300478 A CN109300478 A CN 109300478A
Authority
CN
China
Prior art keywords
hearing
interface
text
person hard
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811027365.8A
Other languages
Chinese (zh)
Inventor
申志远
熊宝霖
陈子龙
何殷勤
苟逸凡
吉翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN201811027365.8A priority Critical patent/CN109300478A/en
Publication of CN109300478A publication Critical patent/CN109300478A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present invention relates to a kind of auxiliary Interface of person hard of hearing, which includes: voice collecting unit: including microphone and filter, to receive interlocutor's voice of person hard of hearing, and being saved as audio file, and carries out background noise reduction pretreatment;Speech-to-text converting unit: being connect by unit interface with voice collecting unit, and the voice signal of audio file is converted to text results to read pretreated audio file, and by speech recognition;Interactive unit: it is connect by unit interface with speech-to-text converting unit, to show the text results by conversion to person hard of hearing.Compared with prior art, the present invention have many advantages, such as to manually control, display text, background noise reduction.

Description

A kind of auxiliary Interface of person hard of hearing
Technical field
The present invention relates to a kind of auxiliary Interfaces, more particularly, to a kind of auxiliary Interface of person hard of hearing.
Background technique
It is common one of the crippling sensory disturbance disease in China, person hard of hearing and normal person couple that dysaudia, which has become, There are larger obstacles when words exchange.The information interchange demand that dysaudia auxiliary passes through the various means satisfaction person that listens to barrier.It is auxiliary at present Mainly there are two directions for assistant's section: one is repaired to impaired hearing system, for example, impaired for sound conduction access Hearing aid, for the artificial cochlea etc. of sound-electric signal conversion missing;Another direction is converted into voice signal such as figure As or text information, realize the exchange needs for the person that listens to barrier.
Through the literature search of existing technologies, patent document CN201410153639.3 discloses a kind of with voice Identification and caption display function intelligent hearing aid, comprising: acquiring and identifying module, voice amplification module, message processing module and Projection module;Acquiring and identifying module is for acquiring voice messaging and the voice messaging after identification being simultaneously sent to voice amplification mould Block and message processing module;Voice amplification module is used for the amplification of received voice messaging and directional transmissions are gone out;Information processing Module is used to received voice messaging being converted to text information, and sends projection module for the text information after conversion;It throws Shadow module is used to project to the text information received the retina of user.The intelligent hearing aid has the following disadvantages it Place: one, acquiring and identifying module receives all voices near the person that listens to barrier without interruption, receives voice when not having dialogue demand, Need to listen to barrier person to pay attention to ambient conditions at the moment;Two, acquiring and identifying module does not carry out noise reduction process to voice messaging, but directly will Voice messaging after identification is sent to voice amplification module, is easy together to amplify ambient noise, and then influence voice after amplification Quality and voice-text conversion result accuracy;Three, it is transferred out after amplification module amplifies voice messaging, it is identical Voice messaging may be collected identification module and repeat to receive, and lead to endless loop, therefore have ignored the voice letter useful to the person that listens to barrier Breath.
Patent document CN201611178785.7 provide the auxiliary conversational system of deaf-mute and normal person a kind of, method and Smart phone, comprising: scene perception module, for perceiving and determining the session operational scenarios of deaf-mute and normal person;Data acquisition and Preprocessing module generates voice data, pre-processes to the voice data, generate voice number for acquiring normal person's speech According to;Speech recognition module identifies the voice data for receiving, and loads the speech recognition modeling of the corresponding session operational scenarios, root The voice data is recognized and converted into text information according to the speech recognition modeling;Voice synthetic module, for deaf-mute is defeated The content of text for entering dialogue is converted into voice messaging, and issues normal person.The system has the following disadvantages: one, only for The auxiliary of deaf-mute is talked with, and considers the demand of the person that listens to barrier of deaf type after language;Two, data acquisition and preprocessing module are according to right Words scene carries out starting point and end point detection to voice data, removal noise, although using automatic measurement technique, and it is manual Control starts/stops taped conversations and compares, and is more also easy to produce wrong voice data and uncertain delay time;Three, voice closes At module there are redundancy, text can be switched to rapidly after the content of text of deaf-mute's input dialogue, for normal person, vision ratio is listened Feel reaction faster.
Summary of the invention
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide a kind of person hard of hearing Assist Interface.
The purpose of the present invention can be achieved through the following technical solutions:
A kind of auxiliary Interface of person hard of hearing, the device include:
Voice collecting unit: including microphone and filter, to receive interlocutor's voice of person hard of hearing, and by its Audio file is saved as, and carries out background noise reduction pretreatment;
Speech-to-text converting unit: being connect by unit interface with voice collecting unit, pretreated to read Audio file, and the voice signal of audio file is converted to by text results by speech recognition;
Interactive unit: being connect by unit interface with speech-to-text converting unit, to show the text knot of conversion Fruit is to person hard of hearing.
Preferably, the speech-to-text converting unit comprising microprocessor and passes through communication interface and microprocessor The peripheral circuit of connection, the microprocessor are connect with microphone, and the peripheral circuit is connect with filter.
Preferably, the communication interface includes the external communication interface and voice-text of speech-to-text converting unit and cloud The internal communication interface of this converting unit.
Preferably, which further includes cloud server, and the cloud server and microprocessor pass through external communication The microprocessor of server beyond the clouds or local is arranged in interface communication, the speech recognition.
Preferably, the interaction display interface include the interaction display interface being connect with microprocessor and with periphery electricity The interlocutor of beginning/stopping voice collecting control button of road connection, person hard of hearing starts/stops according to dialogue state control Only voice collecting control button realization start/stop voice collecting.
Preferably, the unit interface is communication interface or electric interfaces.
Preferably, the filter passes through hardware or software realization.
Preferably, the display interface is display screen.
Preferably, the beginning/stopping voice collecting control button is physical entity button or virtual push button.
Preferably, when beginning/stopping voice collecting control button is virtual push button, interaction display interface, which is equipped with, to be made For the beginning/stopping voice collecting control button and text display box of virtual push button, when person hard of hearing is not opened with interlocutor Begin dialogue when, virtual push button is circle, when preparing to start dialogue, clicks after starting recording acquisition after virtual push button, virtually presses Button 32 becomes square, when preparing to terminate dialogue, stops recording acquisition after virtual push button with clicking, and show institute in text box Converting text result.
Preferably, the person hard of hearing is deaf type person hard of hearing after language.
Compared with prior art, the invention has the following advantages that
1, the present invention provides auxiliary dialogue function for type person hard of hearing deaf after language, and most of involved in the prior art And be deaf-mute, therefore, the present invention is without speech synthesis or text entry technique.
2, the present invention passes through and person hard of hearing/normal person interactive interface, can not only manually control beginning/stopping auxiliary Dialogue, moreover it is possible to show the text results of normal person's dialogic voice to interactive interface.Which overcome in the prior art using certainly The deficiency of dynamic detection beginning of conversation/stopping or whole recording, in addition, beginning/stopping auxiliary dialogue can be by person hard of hearing Interlocutor's control, talks with convenient for person hard of hearing and normal person.
3, the present invention is not limiting as the implementation of noise reduction process method, and which overcome do not use background to go in the prior art The deficiency for technology of making an uproar, in addition, also not limiting the model and deployment way of speech recognition technology, used model can be with speech recognition The development of technology and change, be easy to for existing optimal relevant art being integrated into device, which overcome in the prior art using solid Determine the deficiency of speech recognition technology model.
Detailed description of the invention
Fig. 1 is the auxiliary Interface structural schematic diagram of person hard of hearing provided by the invention.
Fig. 2 is the structural schematic diagram of one embodiment of the invention.
Fig. 3 is schematic diagram when not starting dialogue in one embodiment of the invention in interaction display interface.
Fig. 4 is the schematic diagram after starting dialogue in one embodiment of the invention in interaction display interface.
Fig. 5 is the schematic diagram after terminating dialogue in one embodiment of the invention in interaction display interface.
Description of symbols in figure:
1, voice collecting unit, 2, speech-to-text converting unit, 3, interactive unit, 4, unit interface, 11, microphone, 12, filter, 21, microprocessor, 22, peripheral circuit, 23, communication interface, 31, interaction display interface, 32, beginning/stopping language The control button of sound acquisition.
Specific embodiment
The present invention is described in detail combined with specific embodiments below.Following embodiment will be helpful to the technology of this field Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill of this field For personnel, without departing from the inventive concept of the premise, several changes and improvements can also be made.These belong to the present invention Protection scope.
Embodiment
As shown in Figure 1, the present invention provides a kind of auxiliary Interface of person hard of hearing, deaf type after especially a kind of language Person hard of hearing and normal person dialogue auxiliary device, the device include voice collecting unit 1, speech-to-text converting unit 2, Interactive unit 3 and each unit interface 4, voice collecting unit 1, speech-to-text converting unit 2, interactive unit 3 pass sequentially through Unit interface 4 is connected, specifically:
Voice collecting unit 1: interlocutor's voice of person hard of hearing is received, the sound of wav or other formats are saved as Frequency file, and the pretreatment of background noise reduction is carried out to saved audio file;
Speech-to-text converting unit 2: reading pretreated audio file, using speech recognition technology by voice signal Be converted to text results;
Interactive unit 3: institute's converting text is given to person hard of hearing as the result is shown, the interlocutor of person hard of hearing is according to dialogue State control start/stop voice collecting.
Voice collecting unit 1 includes microphone 11 and filter 12.Filter 12 can pass through hardware or software realization.Language Sound-text conversion units 2 include the peripheral circuit 22 and communication interface 23 of microprocessor 21, microprocessor.Speech-to-text conversion Local or cloud server can be deployed in using existing speech recognition technology in unit 2, communication interface 23 includes voice-text This converting unit 2 and the external communication interface in cloud and the internal communication interface of speech-to-text converting unit 2, interactive unit 3 are wrapped The display interface 31 and beginning/stopping voice collecting control button 32 containing interaction, interaction display interface be display screen or other Show medium, beginning/stopping voice collecting control button 32 is the virtual push button on physical entity button or display screen, single First interface 4 is communication interface or electric interfaces.
Preferred embodiment is as follows:
As shown in Fig. 2, USB of the voice collecting unit 1 of the auxiliary Interface of person hard of hearing using product happy (Bejoy) Insertion pore microphone 11 and filter 12 by software realization, wherein microphone 11 accesses Raspberry foundation The USB interface of Raspberry PI 3MODEL B+ is deployed in Raspberry by the program of the filter 12 of software realization In the Raspian operating system of PI 3MODEL B+, Wiener filtering is write using Python;The speech-to-text of the device turns Unit 2 is changed using the BCM2837B0 microprocessor 21 of Broadcom, the Raspberry PI 3MODEL of Raspberry foundation The peripheral circuit 22 and communication interface 23 of B+, wherein the speech recognition technology deployment of speech-to-text converting unit 2 beyond the clouds, is adopted With the online REST API (see http://ai.***.com/tech/speech/asr) of the speech recognition of Baidu, It is write in Raspbian operating system using Python and calls online REST API, communication interface 23 is to communicate with cloud The WiFi and universal input and output port (GPIO) of Raspberry PI 3MODEL B+;The interactive unit 3 of the device uses 3.5 Interaction display interface 31 and virtual push button 32 is presented in the LCD touch screen of inch Raspberry PI 3MODEL B+, wherein touches Screen is connected to the GPIO of Raspberry PI 3MODEL B+ by SPI, and virtual push button 32 is deployed in Raspberry In the Raspbian operating system of PI3MODEL B+, it is simultaneously real that graphical user's interactive interface (GUI) is write using Python and PyQT Existing virtual push button 32, as shown in figure 3, the virtual push button 32 of GUI is circle when person hard of hearing and normal person do not start dialogue, If starting recording acquisition after preparing the virtual push button 32 for starting to click GUI when dialogue with finger at this time, as shown in figure 4, virtually pressing Button 32 becomes square, if stopping recording acquisition after preparing the virtual push button 32 for terminating to click GUI when dialogue with finger at this time, with Institute's converting text result is shown in text box afterwards;The unit interface 4 of the device is using Raspberry PI 3MODEL B+'s Communication interface connects each unit with GPIO.
Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited to above-mentioned Particular implementation, those skilled in the art can make a variety of changes or modify within the scope of the claims, this not shadow Ring substantive content of the invention.In the absence of conflict, the feature in embodiments herein and embodiment can any phase Mutually combination.

Claims (10)

1. a kind of auxiliary Interface of person hard of hearing, which is characterized in that the device includes:
Voice collecting unit (1): including microphone (11) and filter (12), to receive interlocutor's language of person hard of hearing Sound, and it is saved as audio file, and carry out background noise reduction pretreatment;
Speech-to-text converting unit (2): it is connect by unit interface (4) with voice collecting unit (1), to read pre- place Audio file after reason, and the voice signal of audio file is converted to by text results by speech recognition;
Interactive unit (3): it is connect by unit interface (4) with speech-to-text converting unit (2), to show conversion Text results are to person hard of hearing.
2. a kind of auxiliary Interface of person hard of hearing according to claim 1, which is characterized in that the voice- Text conversion units (2) include microprocessor (21) and are connect by communication interface (23) with microprocessor (21) peripheral electric Road (22), the microprocessor (21) are connect with microphone (11), and the peripheral circuit (22) is connect with filter (12).
3. a kind of auxiliary Interface of person hard of hearing according to claim 2, which is characterized in that the communication interface (23) comprising speech-to-text converting unit (2) and the external communication interface in cloud and the inside of speech-to-text converting unit (2) Communication interface.
4. a kind of auxiliary Interface of person hard of hearing according to claim 3, which is characterized in that the device further includes Cloud server, the cloud server are communicated with microprocessor (21) by external communication interface, the speech recognition The microprocessor (21) of server beyond the clouds or local is set.
5. a kind of auxiliary Interface of person hard of hearing according to claim 2, which is characterized in that the interaction is aobvious Show that interface (31) includes the interaction display interface (31) connecting with microprocessor (21) and opens with what peripheral circuit (22) was connect Beginning/stopping voice collecting control button (32), the interlocutor of person hard of hearing starts according to dialogue state control/stop voice Acquisition control button (32) realization start/stop voice collecting.
6. a kind of auxiliary Interface of person hard of hearing according to claim 1, which is characterized in that between the unit Interface (4) is communication interface or electric interfaces.
7. a kind of auxiliary Interface of person hard of hearing according to claim 5, which is characterized in that the display interface It (31) is display screen.
8. a kind of auxiliary Interface of person hard of hearing according to claim 5, which is characterized in that described to start/stop Only the control button (32) of voice collecting is physical entity button or virtual push button.
9. a kind of auxiliary Interface of person hard of hearing according to claim 8, which is characterized in that when beginning/stopping The control button (32) of voice collecting be virtual push button when, interaction display interface (31) be equipped with as virtual push button beginning/ The control button (32) and text display box for stopping voice collecting, when person hard of hearing does not start dialogue with interlocutor, virtually Button is circle, when preparing to start dialogue, is clicked after starting recording acquisition after virtual push button, virtual push button 32 becomes square Shape stops recording acquisition after virtual push button with clicking, and show institute's converting text knot in text box when preparing to terminate dialogue Fruit.
10. a kind of auxiliary Interface of person hard of hearing according to claim 1, which is characterized in that the hearing Obstacle person is deaf type person hard of hearing after language.
CN201811027365.8A 2018-09-04 2018-09-04 A kind of auxiliary Interface of person hard of hearing Pending CN109300478A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811027365.8A CN109300478A (en) 2018-09-04 2018-09-04 A kind of auxiliary Interface of person hard of hearing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811027365.8A CN109300478A (en) 2018-09-04 2018-09-04 A kind of auxiliary Interface of person hard of hearing

Publications (1)

Publication Number Publication Date
CN109300478A true CN109300478A (en) 2019-02-01

Family

ID=65166298

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811027365.8A Pending CN109300478A (en) 2018-09-04 2018-09-04 A kind of auxiliary Interface of person hard of hearing

Country Status (1)

Country Link
CN (1) CN109300478A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111125055A (en) * 2019-11-22 2020-05-08 北京理工大学 Retrospective hearing-impaired person auxiliary dialogue system
CN111128180A (en) * 2019-11-22 2020-05-08 北京理工大学 Auxiliary dialogue system for hearing-impaired people
CN111127827A (en) * 2019-12-27 2020-05-08 钟楷文 Life assisting system for deaf and hearing-impaired patients

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070112911A (en) * 2006-05-23 2007-11-28 (주) 한 슬 Communication system for the deaf using speech recognition
CN201365285Y (en) * 2009-03-10 2009-12-16 胡礼斌 Hearing aid mobile phone applicable to the deaf
CN201860365U (en) * 2010-05-26 2011-06-08 康佳集团股份有限公司 Mobile phone device for deaf-mute
WO2015131028A1 (en) * 2014-02-28 2015-09-03 Ultratec,Inc. Semiautomated relay method and apparatus
CN106066633A (en) * 2015-04-24 2016-11-02 Jpw工业有限公司 The wearable display device being used together with lathe
CN107454947A (en) * 2016-09-26 2017-12-08 深圳市大疆创新科技有限公司 Unmanned aerial vehicle (UAV) control method, wear-type show glasses and system
CN107980110A (en) * 2016-12-08 2018-05-01 深圳市柔宇科技有限公司 Head-mounted display apparatus and its content input method
CN207612422U (en) * 2017-12-07 2018-07-13 杭州蓝斯特科技有限公司 A kind of visualization auditory prosthesis

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070112911A (en) * 2006-05-23 2007-11-28 (주) 한 슬 Communication system for the deaf using speech recognition
CN201365285Y (en) * 2009-03-10 2009-12-16 胡礼斌 Hearing aid mobile phone applicable to the deaf
CN201860365U (en) * 2010-05-26 2011-06-08 康佳集团股份有限公司 Mobile phone device for deaf-mute
WO2015131028A1 (en) * 2014-02-28 2015-09-03 Ultratec,Inc. Semiautomated relay method and apparatus
CN106066633A (en) * 2015-04-24 2016-11-02 Jpw工业有限公司 The wearable display device being used together with lathe
CN107454947A (en) * 2016-09-26 2017-12-08 深圳市大疆创新科技有限公司 Unmanned aerial vehicle (UAV) control method, wear-type show glasses and system
CN107980110A (en) * 2016-12-08 2018-05-01 深圳市柔宇科技有限公司 Head-mounted display apparatus and its content input method
CN207612422U (en) * 2017-12-07 2018-07-13 杭州蓝斯特科技有限公司 A kind of visualization auditory prosthesis

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111125055A (en) * 2019-11-22 2020-05-08 北京理工大学 Retrospective hearing-impaired person auxiliary dialogue system
CN111128180A (en) * 2019-11-22 2020-05-08 北京理工大学 Auxiliary dialogue system for hearing-impaired people
CN111127827A (en) * 2019-12-27 2020-05-08 钟楷文 Life assisting system for deaf and hearing-impaired patients

Similar Documents

Publication Publication Date Title
US8478578B2 (en) Mobile speech-to-speech interpretation system
CN103116576A (en) Voice and gesture interactive translation device and control method thereof
CN109300478A (en) A kind of auxiliary Interface of person hard of hearing
CN102298694A (en) Man-machine interaction identification system applied to remote information service
CN107644643A (en) A kind of voice interactive system and method
WO2016187910A1 (en) Voice-to-text conversion method and device, and storage medium
CN104811559A (en) Noise reduction method, communication method and mobile terminal
CN110148418B (en) Scene record analysis system, method and device
CN109346057A (en) A kind of speech processing system of intelligence toy for children
WO2014173325A1 (en) Gutturophony recognition method and device
CN111276150B (en) Intelligent voice-to-text and simultaneous interpretation system based on microphone array
CN111261139A (en) Character personification broadcasting method and system
CN105701686A (en) Voiceprint advertisement implementation method and device
CN105869636A (en) Speech recognition apparatus and method thereof, smart television set and control method thereof
CN113299309A (en) Voice translation method and device, computer readable medium and electronic equipment
CN109119077A (en) A kind of robot voice interactive system
KR20210124050A (en) Automatic interpretation server and method thereof
CN113763925A (en) Speech recognition method, speech recognition device, computer equipment and storage medium
CN112908336A (en) Role separation method for voice processing device and voice processing device thereof
CN111985252A (en) Dialogue translation method and device, storage medium and electronic equipment
JP2021076715A (en) Voice acquisition device, voice recognition system, information processing method, and information processing program
CN108735234A (en) A kind of device monitoring health status using voice messaging
JP2017191531A (en) Communication system, server, and communication method
CN110232919A (en) Real-time voice stream extracts and speech recognition system and method
CN109922397A (en) Audio intelligent processing method, storage medium, intelligent terminal and smart bluetooth earphone

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 200030 Dongchuan Road, Minhang District, Minhang District, Shanghai

Applicant after: SHANGHAI JIAO TONG University

Address before: 200030 Huashan Road, Shanghai, No. 1954, No.

Applicant before: SHANGHAI JIAO TONG University

CB02 Change of applicant information
RJ01 Rejection of invention patent application after publication

Application publication date: 20190201

RJ01 Rejection of invention patent application after publication