CN102521577A - Handwriting recognition, synthesis and tracking method of interactive multimedia device - Google Patents
Handwriting recognition, synthesis and tracking method of interactive multimedia device Download PDFInfo
- Publication number
- CN102521577A CN102521577A CN2011104269066A CN201110426906A CN102521577A CN 102521577 A CN102521577 A CN 102521577A CN 2011104269066 A CN2011104269066 A CN 2011104269066A CN 201110426906 A CN201110426906 A CN 201110426906A CN 102521577 A CN102521577 A CN 102521577A
- Authority
- CN
- China
- Prior art keywords
- handwriting
- user
- tracking
- standard letter
- person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Machine Translation (AREA)
Abstract
The invention relates to a handwriting recognition, synthesis and tracking method of an interactive multimedia device. The method comprises the following ordered steps that: after starting the interactive multimedia device, a user writes on the device, a handwriting recognition module converts the handwritten handwriting into standard fonts recognizable for a PC (Personal Computer); a voice synthesis module synthesizes the standard font into computer voice and outputs the voice; the user reads the standard fonts, and a reading tracking module processes the acquired user voice, and tracks the standard fonts corresponding to the reading in real time. The method is used in classroom teaching, and increases the interactivity and the enjoyment of the phonetic teaching; and the method also can be used for karaoke singing tracking in the entertainment field so as to improve user experience.
Description
?
Technical field
The present invention relates to the interactive multimedia apparatus field, especially a kind of person's handwriting identification of interactive multimedia equipment, synthetic and tracking.
Background technology
Traditional voice technology is used the computer standard font that only is confined to acquiescence and is synthesized; Along with popularizing of information tools such as smart mobile phone, palm PC, interactive electric whiteboard; And intelligent sound is synthetic, the development of speech recognition technology, and traditional voice technology can't satisfy user's request for utilization.At present, research and develop a kind of support, the phonetic synthesis of identifying the handwriting, the Intelligent Recognition of input voice and technology of instant tracking of identifying the handwriting and be necessary, can improve user's experience effect effectively based on handwriting recognition technology.
?
Summary of the invention
The object of the present invention is to provide a kind ofly can discern hand-written text handwriting, synthetic, and convert person's handwriting identification, the synthetic and tracking of the interactive multimedia equipment that massage voice reading comes out to.
For realizing above-mentioned purpose, the present invention has adopted following technical scheme: a kind of person's handwriting identification of interactive multimedia equipment, synthetic and tracking, and this method comprises the step of following order:
(1) start interactive multimedia equipment, the user writes on equipment, and the handwriting recognition module converts handwriting into PC discernible standard letter;
(2) the phonetic synthesis module synthesizes computerized speech and output with standard letter; The user reads aloud standard letter, read aloud tracking module the user speech of gathering handled, real-time follow-up with read aloud corresponding standard letter.
Can know that by technique scheme the present invention discerns handwriting through the handwriting recognition module, and convert the standard letter that computing machine can be discerned into; After conversion; The user can start the phonetic synthesis module, and the phonetic synthesis module synthesizes computerized speech to standard letter, also can start and read aloud tracking module; By reading aloud the voice that tracking module collection user reads aloud standard letter, and implement to follow the tracks of the standard letter that the user read aloud.The present invention is applied in the classroom instruction, has increased the interactive of phonetic teaching with interesting, and the Karaoke singing that also can be applied to entertainment field is followed, to improve user's experience effect.
Description of drawings
Fig. 1,2 is workflow diagram of the present invention.
Embodiment
A kind of person's handwriting identification of interactive multimedia equipment, synthetic and tracking; This method comprises the step of following order: (1) starts interactive multimedia equipment; The user writes on equipment, and the handwriting recognition module converts handwriting into PC discernible standard letter; (2) the phonetic synthesis module synthesizes computerized speech and output with standard letter; The user reads aloud standard letter, read aloud tracking module the user speech of gathering handled, real-time follow-up with read aloud corresponding standard letter, as shown in Figure 1.Described interactive multimedia equipment is made up of handwriting pad, PC, loudspeaker and microphone, and PC links to each other with handwriting pad, loudspeaker, microphone respectively through its USB interface.
As shown in Figure 2; After the handwriting recognition module converts user's handwriting into PC discernible standard letter; Whether judges starts the person's handwriting recognition function, if judged result is for being, then converts user's handwriting into PC discernible standard letter by the handwriting recognition module; Otherwise, return and continue to judge whether to start the person's handwriting recognition function.The user judges whether recognition result is consistent with handwriting, discerns if judged result, is then accomplished person's handwriting for being; Otherwise return and carry out person's handwriting identification again.
As shown in Figure 2; After the handwriting recognition module converts user's handwriting into PC discernible standard letter; Whether judges starts complex functionality, if judged result is for being, then by the phonetic synthesis module standard letter is synthesized computerized speech and exports through loudspeaker; Otherwise, return and continue to judge whether to start complex functionality.
As shown in Figure 2, after the handwriting recognition module converted user's handwriting into PC discernible standard letter, whether judges started following function; If judged result is for being; Then gather the voice of the standard letter that the user read aloud, and this voice messaging is sent to sound identification module, after sound identification module identification by microphone; Be sent to and read aloud tracking module, read aloud tracking module real-time follow-up and user and read aloud corresponding standard letter; Otherwise, return and continue to judge whether to start following function.Following function be meant through microphone with phonetic entry in PC; PC carries out serializing with it and handles through reading the standard letter that converts to, follows the tracks of and read aloud corresponding person's handwriting font in real time; Realize the high bright demonstration of person's handwriting text that massage voice reading arrives, the effect that the person's handwriting cursor is followed automatically.Following function realizes reads aloud that pronunciation is corresponding with written handwriting follows, and the individual difference of not imported source of sound influences, the tracking real-time response of voice of importing in the reciprocal process and person's handwriting, and error is less; The intelligent-tracking that the order that can also realize handwriting is read aloud, skipped, inverted order is read aloud.In addition, can also realize the intelligent-tracking that other languages and multilingual mixing are read aloud.
Be further described below in conjunction with Fig. 1,2 couples of the present invention.
Use special writing implement, like electronic pen, or finger is write on handwriting pad; At this moment, PC automatically identifies and produces corresponding trace information in order when writing, and this is mapped to the ISN of Chinese character; Handwriting is converted into grapholect, on handwriting pad, has write " intelligent sound technology " like the user, then PC can be caught the trace information that produces when writing automatically; Be mapped in the Hanzi internal code of storing in the PC, find out corresponding grapholect, and output in the recognition result.
The user starts the phonetic synthesis module, gets into the phonetic synthesis state, and the grapholect after the phonetic synthesis module will be discerned is converted into the voice that can understand, standard is smooth in real time and exports.
Tracking module is read aloud in user's startup; Through continuous speech being decomposed into units such as speech, phoneme, extract the correlated characteristic of voice, coupling acoustic model and pattern; Realization is to the identification and the understanding of natural-sounding; Use microphone input voice " intelligent sound technology " like the user, then PC can be the unit morpheme and extract correlated characteristic with speech, phoneme continuous voice decomposition, thereby realizes identification and the understanding of PC to natural-sounding.PC simultaneously, carries out serializing with the text of discerning and handles, and both results are mapped through identification and processing to language model, thereby realizes the instant effect of following the tracks of of massage voice reading.
Claims (6)
1. the person's handwriting of interactive multimedia equipment identification, synthetic and tracking, this method comprises the step of following order:
(1) start interactive multimedia equipment, the user writes on equipment, and the handwriting recognition module converts handwriting into PC discernible standard letter;
(2) the phonetic synthesis module synthesizes computerized speech and output with standard letter; The user reads aloud standard letter, read aloud tracking module the user speech of gathering handled, real-time follow-up with read aloud corresponding standard letter.
2. the person's handwriting identification of interactive multimedia equipment according to claim 1, synthetic and tracking; It is characterized in that: described interactive multimedia equipment is made up of handwriting pad, PC, loudspeaker and microphone, and PC links to each other with handwriting pad, loudspeaker, microphone respectively through its USB interface.
3. the person's handwriting identification of interactive multimedia equipment according to claim 2, synthetic and tracking; It is characterized in that: after the handwriting recognition module converts user's handwriting into PC discernible standard letter; Whether judges starts the person's handwriting recognition function; If judged result is for being, then convert user's handwriting into PC discernible standard letter by the handwriting recognition module; Otherwise, return and continue to judge whether to start the person's handwriting recognition function.
4. the person's handwriting identification of interactive multimedia equipment according to claim 2, synthetic and tracking; It is characterized in that: after the handwriting recognition module converts user's handwriting into PC discernible standard letter; Whether judges starts complex functionality; If judged result is for being, then standard letter is synthesized computerized speech and export through loudspeaker by the phonetic synthesis module; Otherwise, return and continue to judge whether to start complex functionality.
5. the person's handwriting identification of interactive multimedia equipment according to claim 2, synthetic and tracking; It is characterized in that: after the handwriting recognition module converts user's handwriting into PC discernible standard letter; Whether judges starts following function, if judged result is for being, then by the voice of the microphone collection standard letter that the user read aloud; And this voice messaging is sent to sound identification module; After sound identification module identification, be sent to and read aloud tracking module, read aloud tracking module real-time follow-up and user and read aloud corresponding standard letter; Otherwise, return and continue to judge whether to start following function.
6. the person's handwriting identification of interactive multimedia equipment according to claim 3, synthetic and tracking; It is characterized in that: after the handwriting recognition module converts user's handwriting into PC discernible standard letter; The user judges whether recognition result is consistent with handwriting; If judged result, is then accomplished person's handwriting for being and is discerned; Otherwise return and carry out person's handwriting identification again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104269066A CN102521577A (en) | 2011-12-20 | 2011-12-20 | Handwriting recognition, synthesis and tracking method of interactive multimedia device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104269066A CN102521577A (en) | 2011-12-20 | 2011-12-20 | Handwriting recognition, synthesis and tracking method of interactive multimedia device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102521577A true CN102521577A (en) | 2012-06-27 |
Family
ID=46292488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011104269066A Pending CN102521577A (en) | 2011-12-20 | 2011-12-20 | Handwriting recognition, synthesis and tracking method of interactive multimedia device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102521577A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105892815A (en) * | 2016-03-31 | 2016-08-24 | 北京小米移动软件有限公司 | Document marking method and device |
| CN110488997A (en) * | 2019-07-03 | 2019-11-22 | 深圳市九洲电器有限公司 | Voice-based clipboard implementation method and Related product |
| CN114398463A (en) * | 2021-12-30 | 2022-04-26 | 南京硅基智能科技有限公司 | Voice tracking method and device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060190256A1 (en) * | 1998-12-04 | 2006-08-24 | James Stephanick | Method and apparatus utilizing voice input to resolve ambiguous manually entered text input |
| CN101315666A (en) * | 2008-07-11 | 2008-12-03 | 中国科学院软件研究所 | Multi-channel hand-written Chinese error correction method based on voice |
| CN101377726A (en) * | 2007-08-31 | 2009-03-04 | 西门子（中国）有限公司 | Input method combining speech recognition with stroke recognition and terminal thereof |
| CN102156577A (en) * | 2011-03-28 | 2011-08-17 | 安徽科大讯飞信息科技股份有限公司 | Method and system for realizing continuous handwriting recognition input |
-
2011
- 2011-12-20 CN CN2011104269066A patent/CN102521577A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060190256A1 (en) * | 1998-12-04 | 2006-08-24 | James Stephanick | Method and apparatus utilizing voice input to resolve ambiguous manually entered text input |
| CN101377726A (en) * | 2007-08-31 | 2009-03-04 | 西门子（中国）有限公司 | Input method combining speech recognition with stroke recognition and terminal thereof |
| CN101315666A (en) * | 2008-07-11 | 2008-12-03 | 中国科学院软件研究所 | Multi-channel hand-written Chinese error correction method based on voice |
| CN102156577A (en) * | 2011-03-28 | 2011-08-17 | 安徽科大讯飞信息科技股份有限公司 | Method and system for realizing continuous handwriting recognition input |
Non-Patent Citations (1)
| Title |
|---|
| 王景中 等: "基于OCR技术的盲用阅读器设计", 《2009年研究生学术交流会通信与信息技术论文集》, 1 September 2009 (2009-09-01) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105892815A (en) * | 2016-03-31 | 2016-08-24 | 北京小米移动软件有限公司 | Document marking method and device |
| CN110488997A (en) * | 2019-07-03 | 2019-11-22 | 深圳市九洲电器有限公司 | Voice-based clipboard implementation method and Related product |
| CN114398463A (en) * | 2021-12-30 | 2022-04-26 | 南京硅基智能科技有限公司 | Voice tracking method and device, storage medium and electronic equipment |
| CN114398463B (en) * | 2021-12-30 | 2023-08-11 | 南京硅基智能科技有限公司 | Voice tracking method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110288077B (en) | Method and related device for synthesizing speaking expression based on artificial intelligence | |
| JP5616325B2 (en) | How to change the display based on user instructions | |
| CN110675854B (en) | Chinese and English mixed speech recognition method and device | |
| CN103714727A (en) | Man-machine interaction-based foreign language learning system and method thereof | |
| CN105426362A (en) | Speech Translation Apparatus And Method | |
| CN112765971B (en) | Text-to-speech conversion method and device, electronic equipment and storage medium | |
| CN106446406A (en) | Simulation system and simulation method for converting Chinese sentences into human mouth shapes | |
| CN103955454A (en) | Method and equipment for carrying out literary form conversion between vernacular Chinese and classical Chinese | |
| TW201314638A (en) | Learning machine with augmented reality mechanism | |
| CN105609098A (en) | Internet-based online learning system | |
| CN109300469A (en) | Simultaneous interpretation method and device based on machine learning | |
| Dai et al. | The sound of silence: end-to-end sign language recognition using smartwatch | |
| CN102521577A (en) | Handwriting recognition, synthesis and tracking method of interactive multimedia device | |
| CN205451551U (en) | Speech recognition driven augmented reality human -computer interaction video language learning system | |
| CN102063282A (en) | Chinese speech input system and method | |
| TWI574254B (en) | Speech synthesis method and apparatus for electronic system | |
| CN103455530A (en) | Portable-type device for creating textual word databases corresponding to personized voices | |
| CN202632566U (en) | English pronunciation teachinig device | |
| CN112201253A (en) | Character marking method and device, electronic equipment and computer readable storage medium | |
| CN111638783A (en) | Man-machine interaction method and electronic equipment | |
| CN201600791U (en) | Electronic device for learning Chinese characters | |
| CN206162525U (en) | Forestry english translation interactive installation | |
| CN104134081A (en) | Spelling method and device for hand input content | |
| CN203217570U (en) | Translation machine | |
| CN108717854A (en) | Method for distinguishing speek person based on optimization GFCC characteristic parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120627 |