CN110695989A - Audio-visual interaction system for intelligent robot and interaction control method thereof - Google Patents
Audio-visual interaction system for intelligent robot and interaction control method thereof Download PDFInfo
- Publication number
- CN110695989A CN110695989A CN201910896540.5A CN201910896540A CN110695989A CN 110695989 A CN110695989 A CN 110695989A CN 201910896540 A CN201910896540 A CN 201910896540A CN 110695989 A CN110695989 A CN 110695989A
- Authority
- CN
- China
- Prior art keywords
- module
- instruction
- voice
- information
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J13/00—Controls for manipulators
- B25J13/003—Controls for manipulators by means of an audio-responsive input
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Multimedia (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention discloses an audio-visual interaction system for an intelligent robot and an interaction control method thereof, wherein the audio-visual interaction system comprises a robot body, wherein a visual interaction mechanism and an auditory interaction mechanism are connected to the robot body; the visual interaction mechanism comprises a motion capture module, one side of the motion capture module is connected with a mobile device module through a Bluetooth module, an identity information identification module is connected onto the motion capture module, the identity information identification module and the motion capture module are connected with a cloud storage module through an internet module, a motion instruction storage module and a motion identification comparison module are connected onto the identity information identification module, and the motion identification comparison module is connected with the motion instruction storage module. The invention has the beneficial effects that: can control the robot through pronunciation or action to, the recognition degree is high, and the error rate is little, and user experience degree is high, convenient popularization.
Description
Technical Field
The invention relates to the field of robot interaction, in particular to an audio-visual interaction system for an intelligent robot and an interaction control method thereof.
Background
Robots are the common name for automatic control machines (Robot) that include all machines that simulate human behavior or thought and other creatures (e.g., machine dogs, machine cats, etc.). There are many taxonomies and controversy to define robots in a narrow sense, and some computer programs are even referred to as robots. In the modern industry, robots refer to artificial machines that automatically perform tasks to replace or assist human work.
Most of the existing robots input instructions in advance, the robots repeat the instruction actions, if the instructions need to be modified, the instructions need to be modified through a computer, however, audio-visual interaction can be input through voice and actions, so that the robots can make corresponding instruction actions, the computer input further develops, but the mode is easy to make mistakes, and the recognition degree is low.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
The invention provides an audio-visual interactive system for an intelligent robot and an interactive control method thereof, aiming at the problems in the related art, so as to overcome the technical problems in the prior related art.
The technical scheme of the invention is realized as follows:
according to one aspect of the invention, an audio-visual interaction system for an intelligent robot is provided, which comprises a robot body, wherein a visual interaction mechanism and an auditory interaction mechanism are connected to the robot body;
the visual interaction mechanism comprises a motion capture module, one side of the motion capture module is connected with a mobile equipment module through a Bluetooth module, an identity information recognition module is connected onto the motion capture module, the identity information recognition module and the motion capture module are both connected with a cloud storage module through an internet module, a motion instruction storage module and a motion recognition comparison module are connected onto the identity information recognition module, the motion recognition comparison module is connected with the motion instruction storage module, a motion instruction input module and a motion instruction editing input module are connected onto the motion instruction storage module, the motion instruction input module is connected with the motion capture module, and a motion instruction sending module is connected onto the motion recognition comparison module;
the auditory interaction mechanism comprises a sound capture module, a sound instruction judgment module is connected to the sound capture module, a voice key information extraction module is connected to the sound instruction judgment module, a voice instruction checking and comparison module is connected to the voice key information extraction module, a voice instruction storage module is connected to the voice instruction checking and comparison module, a voice instruction recording module is connected to the voice instruction storage module, the voice instruction recording module is connected with the sound capture module, a voice information instruction sending module is connected to the voice instruction checking and comparison module, and the voice information instruction sending module and the action instruction sending module are respectively connected with the robot body.
Optionally, the motion capture module is connected with a capture information feedback module.
Optionally, the mobile device module includes one of a mobile phone, a tablet computer, a watch phone, and a smart wristband with a bluetooth connection.
Optionally, the identity information recognition module includes one of a password input recognition module, a face input recognition module, a special action input recognition module, a tone judgment recognition module, a posture characteristic input recognition module, and a fingerprint input recognition module.
Optionally, a voice prompt module i is connected to the action recognition and comparison module.
Optionally, a voice prompt module II is connected to the voice instruction checking and comparing module.
Optionally, the voice information instruction sending module and the action instruction sending module are respectively connected with the combined information comparison module, and the combined information comparison module is connected with the robot body and the combined information storage module.
Optionally, a depth sound recognition mechanism is arranged on the auditory interaction mechanism, and the depth sound recognition mechanism includes a tone information recording storage module, a tone judgment module, a first information on-off module and a second information on-off module, wherein the tone information recording storage module is connected with the sound capture module and the tone judgment module, the tone judgment module is connected with the voice key information extraction module through the second information on-off module, and the first information on-off module is located between the voice instruction judgment module and the voice key information extraction module.
Optionally, the robot body is provided with an instruction completion feedback module.
According to another aspect of the present invention, there is provided an interactive control method for an intelligent robot.
The method comprises the following steps:
the voice command and the action command are recorded into the voice command storage module and the action command storage module in advance;
voice instruction input and action instruction input are carried out by related personnel, recognition is carried out by the voice finger-fit checking and comparing module and the action recognition and comparing module, if the command is correct, the command is sent to the robot body, and the robot body carries out corresponding action;
if the instruction is incorrect, comparing the recognition voice prompt I and the recognition voice prompt II to perform voice prompt;
if the instruction action of related personnel is more, the combined information can be expanded through the combined information storage module, so that the instruction for operating the robot body is expanded, whether the instruction is a combined instruction is judged in the combined information judgment comparison module through the voice information instruction sending module and the action instruction sending module, if not, the instruction is sent to the robot body, and if so, the instruction is sent to the robot body through comparing the stored information in the combined information storage module;
after the robot finishes the relevant command action, the command finishing feedback module makes a corresponding action.
The invention has the beneficial effects that: can control the robot through pronunciation or action to, the recognition degree is high, and the error rate is little, and user experience degree is high, convenient popularization.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an audiovisual interaction system for an intelligent robot according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an identity information recognition module in an audiovisual interaction system for an intelligent robot according to an embodiment of the present invention;
fig. 3 is a flowchart of an interaction control method for an intelligent robot according to an embodiment of the present invention.
Reference numerals;
1. a robot body; 2. a visual interaction mechanism; 3. an auditory interaction mechanism; 4. a motion capture module; 5. a Bluetooth module; 6. a mobile device module; 7. an identity information recognition module; 8. an internet module; 9. a cloud storage module; 10. an instruction completion feedback module; 11. an action instruction storage module; 12. a motion recognition and comparison module; 13. an action instruction input module; 14. an action instruction editing input module; 15. an action instruction sending module; 16. a sound capture module; 17. a sound instruction judgment module; 18. a voice key information extraction module; 19. a voice instruction checking and comparing module; 20. a voice instruction storage module; 21. a voice instruction recording module; 22. a voice message instruction sending module; 23. a capture information feedback module; 24. a password input identification module; 25. a face input recognition module; 26. a special action input recognition module; 27. a tone judgment and identification module; 28. the posture characteristic input recognition module; 29. a fingerprint input identification module; 30. a first voice prompt module; 31. a second voice prompt module; 32. a combined information comparison module; 33. a combined information storage module; 34. the tone information recording and storing module; 35. a tone judging module; 36. a first information on-off module; 37. and a second information on-off module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
As shown in fig. 1-2, according to an embodiment of the present invention, there is provided an audiovisual interaction system for an intelligent robot, including a robot body 1, wherein a visual interaction mechanism 2 and an auditory interaction mechanism 3 are connected to the robot body 1;
wherein, the visual interaction mechanism 2 comprises a motion capture module 4, one side of the motion capture module 4 is connected with a mobile equipment module 6 through a Bluetooth module 5, the motion capture module 4 is connected with an identity information recognition module 7, the identity information recognition module 7 and the motion capture module 4 are both connected with a cloud storage module 9 through an internet module 8, the identity information identification module 7 is connected with an action instruction storage module 11 and an action identification comparison module 12, wherein, the action identification comparison module 12 is connected with the action instruction storage module 11, the action instruction storage module 11 is connected with an action instruction input module 13 and an action instruction editing input module 14, the action instruction recording module 13 is connected with the action capturing module 4, and the action identification comparison module 12 is connected with an action instruction sending module 15;
the auditory interaction mechanism 3 comprises a sound capture module 16, a sound instruction judgment module 17 is connected to the sound capture module 16, a voice key information extraction module 18 is connected to the sound instruction judgment module 17, a voice instruction verification and comparison module 19 is connected to the voice key information extraction module 18, a voice instruction storage module 20 is connected to the voice instruction verification and comparison module 19, a voice instruction recording module 21 is connected to the voice instruction storage module 20, the voice instruction recording module 21 is connected to the sound capture module 16, a voice information instruction sending module 22 is connected to the voice instruction verification and comparison module 19, the voice information instruction sending module 22 and the action instruction sending module 15 are connected to the robot body 1 respectively.
In addition, in one embodiment, the motion capture module 4 is connected with a captured information feedback module 23. The mobile equipment module comprises one of a mobile phone, a tablet personal computer, a watch phone and an intelligent wrist strap with Bluetooth connection. The identity information recognition module 7 comprises one of a password input recognition module 24, a human face input recognition module 25, a special action input recognition module 26, a tone judgment recognition module 27, a posture characteristic input recognition module 28 and a fingerprint input recognition module 29. The action recognition comparison module 12 is connected with a first voice prompt module 30. And a second voice prompt module 31 is connected to the voice instruction checking and comparing module 19. The voice information instruction sending module 22 and the action instruction sending module 15 are respectively connected with a combined information comparison module 32, and the combined information comparison module 32 is connected with the robot body 1 and a combined information storage module 33. The auditory interaction mechanism 3 is provided with a depth sound recognition mechanism, the depth sound recognition mechanism comprises a tone information recording storage module 34, a tone judgment module 35, a first information on-off module 36 and a second information on-off module 37, wherein the tone information recording storage module 34 is connected with the sound capture module 16 and the tone judgment module 35, the tone judgment module 35 is connected with the voice key information extraction module 18 through the second information on-off module 37, and the first information on-off module 36 is located between the voice instruction judgment module 17 and the voice key information extraction module 18. The robot body 1 is provided with an instruction completion feedback module 10.
As shown in fig. 3, according to an embodiment of the present invention, there is also provided an interactive control method for an intelligent robot.
The method comprises the following steps:
step S101, inputting a voice command and an action command into a voice command storage module and an action command storage module in advance;
step S103, voice instruction input and action instruction input are carried out through related personnel, recognition is carried out through a voice instruction checking and comparing module and an action recognition and comparing module, if the recognition is correct, the instruction is sent to the robot body, and the robot body carries out corresponding action;
step S105, if the instruction is incorrect, comparing the first voice prompt for recognition and the second voice prompt for recognition to perform voice prompt;
step S105, if the instruction action of the related personnel is more, expanding the combined information through the combined information storage module so as to further expand the instruction for operating the robot body, judging whether the combined instruction is the combined instruction or not in the combined information judgment and comparison module which is sent through the voice information instruction sending module and the action finger-joint sending module, if not, sending the combined instruction to the robot body, and if so, sending the combined instruction to the robot body by comparing the stored information in the combined information storage module;
and S107, after the robot finishes the related command action, the command finishing feedback module makes a corresponding action.
In conclusion, by means of the technical scheme, the robot can be controlled through voice or actions, and the robot recognition system is high in recognition degree, small in error rate, high in user experience degree and convenient to popularize.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. An audio-visual interaction system for an intelligent robot is characterized by comprising a robot body (1), wherein a visual interaction mechanism (2) and an auditory interaction mechanism (3) are connected to the robot body (1);
wherein, the visual interaction mechanism (2) comprises a motion capture module (4), one side of the motion capture module (4) is connected with a mobile equipment module (6) through a Bluetooth module (5), an identity information identification module (7) is connected on the motion capture module (4), the identity information identification module (7) and the motion capture module (4) are both connected with a cloud storage module (9) through an internet module (8), a motion instruction storage module (11) and a motion identification comparison module (12) are connected on the identity information identification module (7), wherein the motion identification comparison module (12) is connected with the motion instruction storage module (11), a motion instruction input module (13) and a motion instruction editing input module (14) are connected on the motion instruction storage module (11), the action instruction recording module (13) is connected with the action capturing module (4), and the action identification comparison module (12) is connected with an action instruction sending module (15);
the auditory interaction mechanism (3) comprises a sound capture module (16), the sound capture module (16) is connected with a sound instruction judgment module (17), the voice instruction judging module (17) is connected with a voice key information extracting module (18), the voice key information extraction module (18) is connected with a voice finger-joint check and comparison module (19), the voice instruction checking and comparing module (19) is connected with a voice finger-joint storage module (20), the voice instruction storage module (20) is connected with a voice instruction recording module (21), the voice order entry module (21) is connected with the sound capture module (16), the voice instruction checking and comparing module (19) is connected with a voice information instruction sending module (22), the voice information instruction sending module (22) and the action instruction sending module (15) are respectively connected with the robot body (1).
2. An audiovisual interaction system for an intelligent robot according to claim 1, characterized in that a captured information feedback module (23) is connected to the motion capture module (4).
3. An audiovisual interaction system for a smart robot according to claim 2, wherein said mobile device module comprises one of a cell phone, a tablet, a watch phone, a smart wristband with a bluetooth connection.
4. An audiovisual interaction system for an intelligent robot according to claim 3, characterized in that the identity information recognition module (7) comprises one of a password input recognition module (24), a face input recognition module (25), a special action input recognition module (26), a tone color judgment recognition module (27), a posture feature input recognition module (28) and a fingerprint input recognition module (29).
5. An audiovisual interaction system for an intelligent robot according to claim 4, characterized in that a first voice prompt module (30) is connected to the action recognition and comparison module (12).
6. An audiovisual interaction system for an intelligent robot according to claim 5, characterized in that a second voice prompt module (31) is connected to the voice instruction checking and comparing module (19).
7. The audio-visual interaction system for the intelligent robot is characterized in that the voice information instruction sending module (22) and the action instruction sending module (15) are respectively connected with a combined information comparison module (32), and the robot body (1) and a combined information storage module (33) are connected to the combined information comparison module (32).
8. The audio-visual interaction system for the intelligent robot is characterized in that a deep sound recognition mechanism is arranged on the audio interaction mechanism (3), and the deep sound recognition mechanism comprises a tone information recording storage module (34), a tone judgment module (35), a first information on-off module (36) and a second information on-off module (37), wherein the tone information recording storage module (34) is connected with the sound capture module (16) and the tone judgment module (35), the tone judgment module (35) is connected with the key voice information extraction module (18) through the second information on-off module (37), and the first information on-off module (36) is located between the sound instruction judgment module (17) and the key voice information extraction module (18).
9. An audiovisual interaction system for an intelligent robot according to claim 8, characterized in that the robot body (1) is provided with an instruction completion feedback module (10).
10. An interaction control method for an intelligent robot, for use of the audiovisual interaction system for an intelligent robot of claim 9, comprising the steps of:
the voice command and the action command are recorded into the voice command storage module and the action command storage module in advance;
voice instruction input and action instruction input are carried out by related personnel, recognition is carried out by the voice instruction checking and comparing module and the action recognizing and comparing module, if the command is correct, the command is sent to the robot body, and the robot body carries out corresponding action;
if the instruction is incorrect, comparing the recognition voice prompt I and the recognition voice prompt II to perform voice prompt;
if the instruction action of related personnel is more, the combined information can be expanded through the combined information storage module, so that the instruction for operating the robot body is expanded, whether the instruction is a combined instruction is judged in the combined information judgment comparison module through the voice information instruction sending module and the action instruction sending module, if not, the instruction is sent to the robot body, and if so, the instruction is sent to the robot body through comparing the stored information in the combined information storage module;
after the robot finishes the relevant command action, the command finishing feedback module makes a corresponding action.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910896540.5A CN110695989A (en) | 2019-09-20 | 2019-09-20 | Audio-visual interaction system for intelligent robot and interaction control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910896540.5A CN110695989A (en) | 2019-09-20 | 2019-09-20 | Audio-visual interaction system for intelligent robot and interaction control method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110695989A true CN110695989A (en) | 2020-01-17 |
Family
ID=69196300
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910896540.5A Pending CN110695989A (en) | 2019-09-20 | 2019-09-20 | Audio-visual interaction system for intelligent robot and interaction control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110695989A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001154693A (en) * | 1999-11-30 | 2001-06-08 | Sony Corp | Robot controller and robot control method and recording medium |
| CN1460050A (en) * | 2001-03-27 | 2003-12-03 | 索尼公司 | Action teaching apparatus and action teaching method for robot system, and storage medium |
| CN104985599A (en) * | 2015-07-20 | 2015-10-21 | 百度在线网络技术（北京）有限公司 | Intelligent robot control method and system based on artificial intelligence and intelligent robot |
| CN105234940A (en) * | 2015-10-23 | 2016-01-13 | 上海思依暄机器人科技有限公司 | Robot and control method thereof |
| US9302393B1 (en) * | 2014-04-15 | 2016-04-05 | Alan Rosen | Intelligent auditory humanoid robot and computerized verbalization system programmed to perform auditory and verbal artificial intelligence processes |
| CN106325142A (en) * | 2015-06-30 | 2017-01-11 | 芋头科技（杭州）有限公司 | Robot system and control method thereof |
| CN106873773A (en) * | 2017-01-09 | 2017-06-20 | 北京奇虎科技有限公司 | Robot interactive control method, server and robot |
| CN108748260A (en) * | 2018-04-20 | 2018-11-06 | 宋东峰 | A kind of audiovisual interactive intelligence robot |
-
2019
- 2019-09-20 CN CN201910896540.5A patent/CN110695989A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001154693A (en) * | 1999-11-30 | 2001-06-08 | Sony Corp | Robot controller and robot control method and recording medium |
| CN1460050A (en) * | 2001-03-27 | 2003-12-03 | 索尼公司 | Action teaching apparatus and action teaching method for robot system, and storage medium |
| US9302393B1 (en) * | 2014-04-15 | 2016-04-05 | Alan Rosen | Intelligent auditory humanoid robot and computerized verbalization system programmed to perform auditory and verbal artificial intelligence processes |
| CN106325142A (en) * | 2015-06-30 | 2017-01-11 | 芋头科技（杭州）有限公司 | Robot system and control method thereof |
| CN104985599A (en) * | 2015-07-20 | 2015-10-21 | 百度在线网络技术（北京）有限公司 | Intelligent robot control method and system based on artificial intelligence and intelligent robot |
| CN105234940A (en) * | 2015-10-23 | 2016-01-13 | 上海思依暄机器人科技有限公司 | Robot and control method thereof |
| CN106873773A (en) * | 2017-01-09 | 2017-06-20 | 北京奇虎科技有限公司 | Robot interactive control method, server and robot |
| CN108748260A (en) * | 2018-04-20 | 2018-11-06 | 宋东峰 | A kind of audiovisual interactive intelligence robot |
Non-Patent Citations (1)
| Title |
|---|
| 陈敏: "《认知计算导论》", 30 April 2017, 华中科技大学出版社 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106653021B (en) | Voice wake-up control method and device and terminal | |
| CN107773982B (en) | Game voice interaction method and device | |
| US8897500B2 (en) | System and method for dynamic facial features for speaker recognition | |
| CN109300347A (en) | Dictation auxiliary method based on image recognition and family education equipment | |
| CN111933108B (en) | Automatic testing method for intelligent voice interaction system of intelligent network terminal | |
| CN102842306A (en) | Voice control method and device as well as voice response method and device | |
| CN109634552A (en) | Report control method and terminal device applied to dictation | |
| CN112836691A (en) | Intelligent interviewing method and device | |
| US11527251B1 (en) | Voice message capturing system | |
| CN104538034A (en) | Voice recognition method and system | |
| CN110673499A (en) | Audio-visual interaction system for intelligent robot and interaction control method thereof | |
| CN107274731A (en) | Towards children's study method, facility for study and storage medium | |
| CN110473556A (en) | Audio recognition method, device and mobile terminal | |
| Chang et al. | A kinect-based gesture command control method for human action imitations of humanoid robots | |
| CN107808085A (en) | the fingerprint control method and system of intelligent terminal | |
| CN113299287A (en) | Multi-mode-based service robot interaction method, system and storage medium | |
| CN111370004A (en) | Man-machine interaction method, voice processing method and equipment | |
| CN107910006A (en) | Audio recognition method, device and multiple source speech differentiation identifying system | |
| CN110695989A (en) | Audio-visual interaction system for intelligent robot and interaction control method thereof | |
| CN110400560B (en) | Data processing method and device, storage medium and electronic device | |
| CN106251717A (en) | Intelligent robot speech follow read learning method and device | |
| EP4064031A1 (en) | Method and system for tracking in extended reality using voice commmand | |
| CN103594086A (en) | Voice processing system, device and method | |
| CN108831470A (en) | A kind of method and system by voice control BMS | |
| CN114548063A (en) | Electronic form filling method and device based on full-link voice recognition technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200117 |
|
| RJ01 | Rejection of invention patent application after publication |