CN109598003B - AR translation device and method - Google Patents
AR translation device and method Download PDFInfo
- Publication number
- CN109598003B CN109598003B CN201811542478.1A CN201811542478A CN109598003B CN 109598003 B CN109598003 B CN 109598003B CN 201811542478 A CN201811542478 A CN 201811542478A CN 109598003 B CN109598003 B CN 109598003B
- Authority
- CN
- China
- Prior art keywords
- translation
- information
- display screen
- camera
- earphone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 239000011521 glass Substances 0.000 claims abstract description 16
- 210000005069 ears Anatomy 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 9
- 210000001747 pupil Anatomy 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 208000032041 Hearing impaired Diseases 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses an AR translation device and method. The device comprises: the glasses comprise a glasses body, a camera, an AR display screen, an earphone, a microphone, a host and a handheld device; the AR display screen is positioned at the position where the spectacle lenses of the spectacle main body are installed; the camera is positioned in the middle of the two AR display screens; the microphone and the earphone are positioned at the position where the glasses body is contacted with the ears of the user; the host is respectively connected with the camera and the AR display screen, and is used for receiving the camera shooting information collected by the camera, storing, calculating and translating the camera shooting information to obtain translation information and sending the translation information to the AR display screen; the host is connected with the handheld device through a connecting wire and a hanging rope, the handheld device is provided with a loudspeaker, and the microphone is connected with the loudspeaker; the host is connected with the earphone and the loudspeaker respectively and used for sending the translation information to the user through the earphone and the loudspeaker in a voice mode. By adopting the invention, the seen region can be translated in real time.
Description
Technical Field
The invention relates to the field of voice translation, in particular to an AR translation device and method.
Background
With the integration of the world, trade among countries, enhancement of cultural communication, more and more communication among countries with different languages, and the requirement of translation is increased year by year. Speech translation devices are developing faster, but there are few or no devices for text translation, particularly wearable devices that facilitate translation at any time. The requirements of various forms of translation in international business activities are strong in international travel, and the current speech translation machines on the market cannot meet the requirements, such as the diversified requirements of translating characters into speech, translating speech into characters and translating characters into characters are not well met. The pushing-out of the AR translation function of the mobile phone solves a part of requirements, but the AR translation function needs to be carried out by holding the mobile phone by hand, is still troublesome, cannot liberate two hands and cannot be used for a long time.
Disclosure of Invention
The invention aims to provide an AR translation device and method, which can liberate both hands and realize real-time translation of a seen region.
In order to achieve the purpose, the invention provides the following scheme:
an AR translation device, comprising: the glasses comprise a glasses body, a camera, an AR display screen, an earphone, a microphone, a host and a handheld device; the AR display screen is positioned at the position where the spectacle lenses of the spectacle main body are installed; the camera is positioned in the middle of the two AR display screens; the microphone and the earphone are positioned at the position where the glasses body contacts with the ears of the user; the host is respectively connected with the camera and the AR display screen, and is used for receiving the camera shooting information collected by the camera, storing, calculating and translating the camera shooting information to obtain translation information, and sending the translation information to the AR display screen; the host is connected with the handheld device through a connecting wire and a hanging rope, the loudspeaker is arranged on the handheld device, and the microphone is connected with the loudspeaker; the host is respectively connected with the earphone and the loudspeaker, and the host is used for sending the translation information to a user in a voice form through the earphone and the loudspeaker.
Optionally, the AR translation apparatus further includes a battery, and the battery is located inside the handheld device.
Optionally, the outer surface of the AR display screen of the AR translation device is plated with a light shielding film.
Optionally, the glasses body is provided with a pupil distance adjusting plectrum, and the pupil distance adjusting plectrum is used for adjusting the pupil distance.
A method of AR translation, the method comprising:
acquiring character information;
determining the coordinate position of the text information on the AR display screen through the text information;
translating the text information to obtain text translation information;
playing the character translation information in a voice form;
and sending the character translation information to the coordinate position of the character information on the AR display screen.
Optionally, the obtaining of the text information specifically includes:
and acquiring text information through a camera.
Optionally, the determining, by the text information, a coordinate position of the text information on the AR display screen specifically includes:
recognizing characters through the character information, taking the length and width of the characters on an AR display screen as coordinate axes, taking the center of the characters as an original point, and recording a transverse coordinate value of the transverse width and a longitudinal coordinate value of the longitudinal length away from the original point;
and determining the coordinate position of the text information on the AR display screen according to the transverse coordinate value and the longitudinal coordinate value.
Optionally, the playing the text translation information in a form of voice specifically includes:
and playing the text translation information to a user through an earphone or a loudspeaker.
A method of AR translation, the method comprising:
acquiring pattern information;
translating the pattern information to obtain pattern translation information;
converting the pattern translation information into real name translation information;
playing the real object name translation information in a voice mode;
and sending the real object name translation information to an AR display screen for display.
A method of AR translation, the method comprising:
acquiring sound information;
translating the sound information to obtain sound translation information;
playing the sound translation information in a voice form;
and sending the sound translation information to an AR display screen for display.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the invention provides an AR translation device, which can liberate both hands and realize real-time translation of a seen region. The functions of translating patterns into characters and translating the characters into voice can be realized. Meanwhile, the function of translating the voice into the characters can be realized, wherein the function of translating the voice into the characters is also suitable for hearing-impaired people.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of an AR translation apparatus according to an embodiment of the present invention;
FIG. 2 is a flowchart of an AR translation method according to an embodiment of the present invention;
FIG. 3 is a flowchart of an AR translation method according to an embodiment of the present invention;
FIG. 4 is a flowchart of the AR translation method according to the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The invention aims to provide an AR translation device and method, which can liberate both hands and realize real-time translation of a seen region.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
FIG. 1 is a schematic diagram of an AR translation apparatus according to an embodiment of the present invention. As shown in fig. 1, an AR translation apparatus includes: the glasses comprise a glasses body 1, a camera 2, an AR display screen 3, an earphone 4, a microphone 5, a host 6 and a handheld device 7; the AR display screen 3 is positioned at the position where the spectacle lenses of the spectacle body 1 are installed; the camera 2 is positioned in the middle of the two AR display screens 3; the microphone 5 and the earphone 4 are positioned at the position where the glasses body 1 contacts with the ears of the user; the host 6 is respectively connected with the camera 2 and the AR display screen 3, and the host 6 is used for receiving the camera information collected by the camera 2, storing, calculating and translating the camera information to obtain translation information, and sending the translation information to the AR display screen 3; the host 6 is connected with the handheld device 7 through a connecting wire and a hanging rope 8, the handheld device 7 is provided with the loudspeaker 9 and a control key 10 of the loudspeaker, and the microphone 5 is connected with the loudspeaker 8; the host computer 6 is respectively connected with the earphone 4 and the loudspeaker 8, and the host computer 6 is used for sending the translation information to a user in a voice form through the earphone 4 and the loudspeaker 8.
The AR translation device further comprises a battery, and the battery is located inside the handheld device. The outer surface of the AR display screen of the AR translation device is plated with a light shielding film. The glasses body is provided with a pupil distance adjusting plectrum 11, and the pupil distance adjusting plectrum 11 is used for adjusting the pupil distance. The glasses body 1 of the AR translation device is connected with the handheld device 7 through the connecting line and the hanging rope 8, so that power supply and signal transmission can be realized, and meanwhile, when a user does not use the AR translation device, the AR translation device can be hung in the chest without being put into a pocket or a bag, so that the AR translation device is more convenient to use.
By adopting the invention, both hands can be liberated, and the seen region can be translated in real time. The functions of translating the patterns into characters and translating the characters into voice can be realized. Meanwhile, the function of translating voice into characters can be realized, wherein the function of translating voice into characters is also suitable for hearing-impaired people.
FIG. 2 is a flowchart of an AR translation method according to an embodiment of the present invention. As shown in fig. 2, a method of AR translation, the method comprising:
step 101: acquiring character information, and acquiring the character information through a camera;
step 102: determining the coordinate position of the text information on the AR display screen through the text information;
step 103: translating the text information to obtain text translation information;
step 104: and playing the text translation information in a voice form, and playing the text translation information to a user through an earphone or a loudspeaker.
Step 105: and sending the character translation information to the coordinate position of the character information on the AR display screen.
recognizing characters through the character information, taking the length and width of the characters on an AR display screen as coordinate axes, taking the center of the characters as an original point, and recording a transverse coordinate value of the transverse width and a longitudinal coordinate value of the longitudinal length away from the original point;
and determining the coordinate position of the text information on the AR display screen according to the transverse coordinate value and the longitudinal coordinate value.
The first embodiment is as follows:
firstly, the information shot in the camera shooting frame of the camera is consistent with the display information of the AR optical display lens. Specifically, the camera and the AR optical display lens are connected, and information shot by the camera is uploaded to the AR optical display lens to be displayed. When a user watches the external environment, the camera of the equipment is used for identifying the scenery in the front visual field of the user, and when characters are identified, the characters are marked with coordinates; specifically, when the characters are recognized, coordinates are printed on the characters, namely, the length and the width of the display screen are used as coordinate axes, the center of the characters is used as a point, the coordinate value of the transverse width of the point is recorded, the coordinate value of the longitudinal length of the point is recorded, and the coordinates of the characters are determined according to the two numerical values. Recording the coordinates of the characters in a camera viewfinder, and simultaneously translating the character information to a server through local translation processing or network uploading; and directly throwing the translated language and character information to a screen of an AR optical display lens display screen according to the coordinate record. Namely, the translated text is covered on the original text according to the coordinate record, and the real-time AR translation effect is realized.
The second embodiment is as follows:
firstly, the information shot in the camera shooting frame of the camera is consistent with the display information of the AR optical display lens. Specifically, the camera and the AR optical display lens are connected, and information shot by the camera is uploaded to the AR optical display lens for display. When a user watches the external environment, the camera of the equipment is used for carrying out offline or online identification on scenery in the visual field in front of the user, when characters are identified, more character contents often appear and are distributed in a plurality of areas, at this time, the displayed characters are smaller, the user can hardly see clearly through a screen, at this time, each section of characters can be marked as 1,2,3 and 4, the marks and the corresponding character contents are recorded, the coordinates of each mark are also recorded, and meanwhile, character information is translated through local translation processing or uploaded to a server through a network. And displaying the translated language and character information item by item below a display screen according to the mark record, and simultaneously placing the marked marks 1,2,3 and 4 on corresponding character areas according to coordinates to realize a real-time AR translation effect.
FIG. 3 is a flowchart of an AR translation method according to an embodiment of the present invention. As shown in fig. 3, a method of AR translation, the method comprising:
step 201: acquiring pattern information;
step 202: translating the pattern information to obtain pattern translation information;
step 203: converting the pattern translation information into real name translation information;
step 204: playing the real object name translation information in a voice mode, and playing the real object name translation information to a user through an earphone or a loudspeaker;
step 205: and sending the real object name translation information to an AR display screen for display.
The third concrete example:
firstly, the information shot in a camera shooting frame is consistent with the display information of the AR optical display lens; when a user watches the external environment, a camera of the equipment is used for scanning and identifying scenes in the front visual field of the user in an off-line or on-line manner, when a real object is identified, character information is identified and processed locally or uploaded to a server through a network, and the name of the real object is translated into language and character contents required by the user and displayed on an AR optical display lens display screen; then, the pronunciation of this real object is broadcast for the user to listen through the earphone, realizes the function of AR knowledge, and this real object contains static article such as a cup, still contains information that animal, flowers and plants, view, human organ can be discerned by image recognition technology.
FIG. 4 is a flowchart of an AR translation method according to a third embodiment of the present invention. As shown in fig. 4, a method of AR translation, the method comprising:
step 301: acquiring sound information;
step 302: translating the sound information to obtain sound translation information;
step 303: playing the sound translation information in a voice form;
step 304: and sending the sound translation information to an AR display screen for display.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the description of the method part.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (10)
1. An AR translation apparatus, comprising: the glasses comprise a glasses body, a camera, an AR display screen, an earphone, a microphone, a host and a handheld device; the AR display screen is positioned at the position where the spectacle lenses of the spectacle main body are installed; the camera is positioned in the middle of the two AR display screens; the microphone and the earphone are positioned at the position where the glasses body is contacted with the ears of the user; the host is respectively connected with the camera and the AR display screen, and is used for receiving camera shooting information collected by the camera, storing, calculating and translating the camera shooting information to obtain translation information, and sending the translation information to the AR display screen; the host is connected with the handheld device through a connecting wire and a hanging rope, the handheld device is provided with a loudspeaker, and the microphone is connected with the loudspeaker; the host is respectively connected with the earphone and the loudspeaker, and the host is used for sending the translation information to a user through the earphone and the loudspeaker in a voice form.
2. The apparatus for AR translation according to claim 1, wherein said AR translation apparatus further comprises a battery, said battery being located inside said handheld device.
3. The AR translation device according to claim 1, wherein an outer surface of an AR display screen of the AR translation device is coated with a light shielding film.
4. The AR translation device according to claim 1, wherein a interpupillary distance adjusting dial is provided on the glasses body, and the interpupillary distance adjusting dial is used for adjusting the interpupillary distance.
5. An AR translation method applied to the AR translation apparatus of any one of claims 1 to 4, the method comprising:
acquiring character information;
determining the coordinate position of the text information on the AR display screen through the text information;
translating the character information to obtain character translation information;
playing the text translation information in a voice mode;
and sending the character translation information to the coordinate position of the character information on the AR display screen.
6. The AR translation method according to claim 5, wherein said obtaining text information specifically comprises:
and acquiring character information through a camera.
7. The AR translation method according to claim 5, wherein the determining the coordinate position of the text information on the AR display screen through the text information specifically comprises:
recognizing characters through the character information, taking the length and width of the characters on an AR display screen as coordinate axes, taking the center of the characters as an original point, and recording a transverse coordinate value of the transverse width and a longitudinal coordinate value of the longitudinal length away from the original point;
and determining the coordinate position of the text information on the AR display screen according to the transverse coordinate value and the longitudinal coordinate value.
8. The AR translation method according to claim 5, wherein said playing said textual translation information in a form of speech specifically comprises:
and playing the text translation information to a user through an earphone or a loudspeaker.
9. An AR translation method applied to the AR translation apparatus of any one of claims 1 to 4, the method comprising:
acquiring pattern information;
translating the pattern information to obtain pattern translation information;
converting the pattern translation information into real name translation information;
playing the real object name translation information in a voice mode;
and sending the real name translation information to an AR display screen for displaying.
10. An AR translation method applied to the AR translation apparatus of any one of claims 1 to 4, the method comprising:
acquiring sound information;
translating the voice information to obtain voice translation information;
playing the sound translation information in a voice form;
and sending the sound translation information to an AR display screen for displaying.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811542478.1A CN109598003B (en) | 2018-12-17 | 2018-12-17 | AR translation device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811542478.1A CN109598003B (en) | 2018-12-17 | 2018-12-17 | AR translation device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109598003A CN109598003A (en) | 2019-04-09 |
| CN109598003B true CN109598003B (en) | 2022-12-16 |
Family
ID=65963645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811542478.1A Active CN109598003B (en) | 2018-12-17 | 2018-12-17 | AR translation device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109598003B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112002186B (en) * | 2020-09-04 | 2022-05-06 | 语惠科技(南京)有限公司 | Information barrier-free system and method based on augmented reality technology |
| CN113009699A (en) * | 2021-03-30 | 2021-06-22 | 朱亮 | Support intelligent glasses of multilingual characters conversion pronunciation |
| CN113093387A (en) * | 2021-04-12 | 2021-07-09 | 深圳市东鲤科技有限公司 | Translation AR glasses and text translation method thereof |
| CN115797815B (en) * | 2021-09-08 | 2023-12-15 | 荣耀终端有限公司 | AR translation processing method and electronic equipment |
| CN115617179A (en) * | 2022-12-01 | 2023-01-17 | 三亚航空旅游职业学院 | AR/VR glasses capable of translating and prompting in real time and working method thereof |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107561695A (en) * | 2016-06-30 | 2018-01-09 | 上海擎感智能科技有限公司 | A kind of intelligent glasses and its control method |
| US10866631B2 (en) * | 2016-11-09 | 2020-12-15 | Rockwell Automation Technologies, Inc. | Methods, systems, apparatuses, and techniques for employing augmented reality and virtual reality |
| CN107479691B (en) * | 2017-07-06 | 2021-01-08 | 捷开通讯(深圳)有限公司 | Interaction method, intelligent glasses and storage device thereof |
-
2018
- 2018-12-17 CN CN201811542478.1A patent/CN109598003B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109598003A (en) | 2019-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109598003B (en) | AR translation device and method | |
| CN110189340B (en) | Image segmentation method and device, electronic equipment and storage medium | |
| US11825012B2 (en) | Server, client terminal, control method, and storage medium | |
| CN104983511A (en) | Voice-helping intelligent glasses system aiming at totally-blind visual handicapped | |
| CN108700982A (en) | Information processing equipment, information processing method and program | |
| JP7143847B2 (en) | Information processing system, information processing method, and program | |
| CN110322760B (en) | Voice data generation method, device, terminal and storage medium | |
| CN105981375A (en) | Information processing apparatus, control method, program, and system | |
| US11546690B2 (en) | Processing audio and video | |
| CN206440890U (en) | Wearable Split intelligent glasses | |
| CN206039075U (en) | Intelligence translation glasses | |
| CN207408959U (en) | Mixed reality intelligent glasses with text and language process function | |
| CN108174236A (en) | A kind of media file processing method, server and mobile terminal | |
| CN109308178A (en) | A kind of voice drafting method and its terminal device | |
| CN111739517A (en) | Speech recognition method, speech recognition device, computer equipment and medium | |
| CN108462826A (en) | A kind of method and mobile terminal of auxiliary photo-taking | |
| CN104570354A (en) | Interactive glasses and visitor guide system | |
| CN109784128A (en) | Mixed reality intelligent glasses with text and language process function | |
| Salvi et al. | Smart glass using IoT and machine learning technologies to aid the blind, dumb and deaf | |
| JP2015169814A (en) | Method and system for supporting communication | |
| EP3113505A1 (en) | A head mounted audio acquisition module | |
| CN110837557B (en) | Abstract generation method, device, equipment and medium | |
| CN210166754U (en) | Virtual reality wears exchange device and virtual reality wears exchange system | |
| CN110133872A (en) | A kind of intelligent glasses can be realized multilingual intertranslation | |
| CN108830901A (en) | A kind of image processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240112 Address after: 519000, Shop B1-01, No. 237 Meijie Road, Xiangzhou District, Zhuhai City, Guangdong Province, Self made TB140 Shop Patentee after: Zhuhai Zhangyu Zhitang Traditional Chinese Medicine Hospital Co.,Ltd. Address before: 518000 706, Building 18, Wangtang Industrial Zone, Xili Street, Nanshan District, Shenzhen, Guangdong Patentee before: SHENZHEN CHUANZHI TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |