CN105975930A - Camera angle calibration method during robot speech localization process - Google Patents
Camera angle calibration method during robot speech localization process Download PDFInfo
- Publication number
- CN105975930A CN105975930A CN201610287804.3A CN201610287804A CN105975930A CN 105975930 A CN105975930 A CN 105975930A CN 201610287804 A CN201610287804 A CN 201610287804A CN 105975930 A CN105975930 A CN 105975930A
- Authority
- CN
- China
- Prior art keywords
- photographic head
- face
- camera
- robot
- center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Manipulator (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a camera angle calibration method during a robot speech localization process. The method comprises a first step of speech coarse localization in which the camera is rotated to an approximate speech source direction according to the speech information, a second step of human face detection in which the camera is used for detecting whether a human face exists and a third step is carried out if the camera detects the human face, a third step of human face angle deviation calculation in which a rectangular coordinate axis is built with the camera center as the original point, the coordinates [X, Y] of the human face center are localized, and the coordinates [X, Y] are deviation between the human face center and the camera center, and a fourth step of angle adjustment in which the angle degreeX of the camera rotating along the X axis and the angle degreeY of the camera rotating along the Y axis are converted according to the deviation between the human face coordinates and the camera center and then, the camera is rotated to enable the human face center to be aligned with the camera center according to the degreeX and the degreeY. The camera alignement precision during the speech localization process is improved.
Description
Technical field
The present invention relates to voice field of locating technology, a kind of photographic head angle calibration system method of robot voice position fixing process.
Background technology
nullDevelopment along with electronic computer science，The research of intelligent robot is the most deep，Its function carried also gets more and more，There is vision、Audition、The height intelligent robot of the functions such as olfactory sensation has occurred，This robot is respectively provided with a set of identification system，By this identification system acquisition information，Through the acquisition of information recognition result that processor and the processing system carried thereof are gathered，Common such as face identification system，Utilize camera collection facial image，The identity of people is i.e. can recognize that after process，But it is inadequate for only realizing series of complex operation by a set of face identification system for robot intelligentized for height，Robot that such as can be mutual with people，It also carries the voice positioning function of 360 °，User calls robot，Robot judges general orientation residing for user according to voice messaging，The direction alignment face turning its photographic head again carries out image acquisition and identification，This technology is capable of 360 ° of indiscriminate locating and tracking，It may be said that the combination of more technology allows the intelligence degree of robot step up.
But, the voice location technology of robot there is also deviation at present, in above-mentioned voice position fixing process, voice messaging is only available to one orientation substantially of robot, rotate photographic head (i.e. rotary machine head part) and be difficult to be exactly found the center of face afterwards, photographic head may be caused to obtain image clear not, there is the possibility of identification error, operationally can only pass through the point that user oneself looks for being directed at photographic head, therefore the voice location technology of robot haves much room for improvement.
Summary of the invention
The present invention solves the problems referred to above, it is provided that a kind of photographic head angle calibration system method of robot voice position fixing process, which raises the photographic head alignment precision of voice position fixing process.
For achieving the above object, the technical solution used in the present invention is:
A kind of photographic head angle calibration system method of robot voice position fixing process, comprises the following steps:
The first step, voice coarse positioning: after robot collects sound wave, orient speech source position by voice locating module, according to this speech source positional information to the robot face with photographic head forwarded speech source orientation substantially；
Second step, Face datection: utilize the camera collection image of robot face, judging whether acquired image has face by face detection module, if there is face, carrying out the 3rd step；
3rd step, face angular deviation calculates: gather a facial image, detect face center, rectangular coordinates axle is set up for initial point with photographic head central point, centre coordinate [the X of face in image is oriented according to face center and rectangular coordinates axle, Y], this coordinate [X, Y] is also the deviation at face center and photographic head center；
4th step: angle adjustment: converse angle degreeX that photographic head rotates and angle degreeY rotated along Y-axis along X-axis according to face coordinate and photographic head central point deviation, then photographic head is finely tuned according to degreeX and degreeY value, make the central point of the central point alignment photographic head of face, it is achieved being accurately positioned of voice.
Further comprising the steps of: between described 3rd step and the 4th step and the deviation calculated compared with predetermined deviation standard value, if deviation is less than predetermined deviation standard value, perform the 4th step, this predetermined deviation standard value pre-sets according to photographic head parameter.
In rectangular coordinates in described second step, initial point is positioned at the geometric center of rectangle, and in described 3rd step, the computing formula of degreeX and degreeY is as follows:
degreeX = x / (width/2) * MAX_DEGREE_X / 2;
degreeY = y / (height/2) * MAX_DEGREE_Y / 2;
Wherein, width is rectangle length in X direction, and height is the rectangle length along Y direction, and MAX_DEGREE_X is photographic head maximum Radix Rumicis in X direction, and MAX_DEGREE_Y is the photographic head maximum Radix Rumicis along Y-direction.
Described MAX_DEGREE_X is 80 °, and MAX_DEGREE_Y is 40 °.
The invention has the beneficial effects as follows:
A kind of photographic head angle calibration system method of the robot voice position fixing process of the present invention, by after voice substantially positions, according to facial image in the image of camera collection and photographic head central point position in acquired image, calculate photographic head and need the rotation angle value of fine setting, facial direction according to this rotation angle value robot, make the photographic head alignment face of robot face, it is finely adjusted for the voice coarse positioning before robot, realize being accurately positioned of voice, not only increase the pickup effect of robot and voice interaction, improve the accuracy of speech recognition, and, robot can collect the direct picture of interactive object face, improve Consumer's Experience.
Accompanying drawing explanation
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the present invention, and the schematic description and description of the present invention is used for explaining the present invention, is not intended that inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the FB(flow block) of the present invention.
Detailed description of the invention
In order to make the technical problem to be solved, technical scheme and beneficial effect clearer, clear, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.
As it is shown in figure 1, a kind of photographic head angle calibration system method of robot voice position fixing process that the present invention discloses, it comprises the following steps:
The first step, voice coarse positioning: after robot collects sound wave, orient speech source position by voice locating module, according to this speech source positional information to the robot face with photographic head forwarded speech source orientation substantially；
Second step, Face datection: utilize the camera collection image of robot face, judging whether acquired image has face by face detection module, if there is face, carrying out the 3rd step；
3rd step, face angular deviation calculates: gather a facial image, detect face center, rectangular coordinates axle is set up for initial point with photographic head central point, centre coordinate [the X of face in image is oriented according to face center and rectangular coordinates axle, Y], this coordinate [X, Y] is also the deviation at face center and photographic head center；
4th step: angle adjustment: converse angle degreeX that photographic head rotates and angle degreeY rotated along Y-axis along X-axis according to face coordinate and photographic head central point deviation, then photographic head is finely tuned according to degreeX and degreeY value, make the central point of the central point alignment photographic head of face, it is achieved being accurately positioned of voice.
Further comprising the steps of: between described 3rd step and the 4th step and the deviation calculated compared with predetermined deviation standard value, if deviation is less than predetermined deviation standard value, perform the 4th step, this predetermined deviation standard value pre-sets according to photographic head parameter.
In rectangular coordinates in described second step, initial point is positioned at the geometric center of rectangle, and in described 3rd step, the computing formula of degreeX and degreeY is as follows:
degreeX = x / (width/2) * MAX_DEGREE_X / 2;
degreeY = y / (height/2) * MAX_DEGREE_Y / 2;
Wherein, width is rectangle length in X direction, and height is the rectangle length along Y direction, and MAX_DEGREE_X is photographic head maximum Radix Rumicis in X direction, and MAX_DEGREE_Y is the photographic head maximum Radix Rumicis along Y-direction.
Described MAX_DEGREE_X is 80 °, and MAX_DEGREE_Y is 40 °.
A kind of photographic head angle calibration system method of the robot voice position fixing process of the present invention, by after voice substantially positions, according to facial image in the image of camera collection and photographic head central point position in acquired image, calculate photographic head and need the rotation angle value of fine setting, facial direction according to this rotation angle value robot, make the photographic head alignment face of robot face, it is finely adjusted for the voice coarse positioning before robot, realize being accurately positioned of voice, not only increase the pickup effect of robot and voice interaction, improve the accuracy of speech recognition, and, robot can collect the direct picture of interactive object face, improve Consumer's Experience.
Described above illustrate and describes the preferred embodiments of the present invention, it is to be understood that the present invention is not limited to form disclosed herein, it is not to be taken as the eliminating to other embodiments, and can be used for other combinations various, amendment and environment, and can be modified by above-mentioned teaching or the technology of association area or knowledge in invention contemplated scope herein.And the change that those skilled in the art are carried out and change are without departing from the spirit and scope of the present invention, the most all should be in the protection domain of claims of the present invention.
Claims (4)
1. the photographic head angle calibration system method of a robot voice position fixing process, it is characterised in that comprise the following steps:
The first step, voice coarse positioning: after robot collects sound wave, orient speech source position by voice locating module, according to this speech source positional information to the robot face with photographic head forwarded speech source orientation substantially；
Second step, Face datection: utilize the camera collection image of robot face, judging whether acquired image has face by face detection module, if there is face, carrying out the 3rd step；
3rd step, face angular deviation calculates: gather a facial image, detect face center, rectangular coordinates axle is set up for initial point with photographic head central point, centre coordinate [the X of face in image is oriented according to face center and rectangular coordinates axle, Y], this coordinate [X, Y] is also the deviation at face center and photographic head center；
4th step: angle adjustment: converse angle degreeX that photographic head rotates and angle degreeY rotated along Y-axis along X-axis according to face coordinate and photographic head central point deviation, then photographic head is finely tuned according to degreeX and degreeY value, make the central point of the central point alignment photographic head of face, it is achieved being accurately positioned of voice.
The photographic head angle calibration system method of a kind of robot voice position fixing process the most as claimed in claim 1, it is characterized in that: further comprise the steps of: between described 3rd step and the 4th step and the deviation calculated is compared with predetermined deviation standard value, if deviation is less than predetermined deviation standard value, performing the 4th step, this predetermined deviation standard value pre-sets according to photographic head parameter.
The photographic head angle calibration system method of a kind of robot voice position fixing process the most as claimed in claim 1, it is characterized in that: in the rectangular coordinates in described second step, initial point is positioned at the geometric center of rectangle, and in described 3rd step, the computing formula of degreeX and degreeY is as follows:
degreeX = x / (width/2) * MAX_DEGREE_X / 2;
degreeY = y / (height/2) * MAX_DEGREE_Y / 2;
Wherein, width is rectangle length in X direction, and height is the rectangle length along Y direction, and MAX_DEGREE_X is photographic head maximum Radix Rumicis in X direction, and MAX_DEGREE_Y is the photographic head maximum Radix Rumicis along Y-direction.
The photographic head angle calibration system method of a kind of robot voice position fixing process the most as claimed in claim 3, it is characterised in that: described MAX_DEGREE_X is 80 °, and MAX_DEGREE_Y is 40 °.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610287804.3A CN105975930A (en) | 2016-05-04 | 2016-05-04 | Camera angle calibration method during robot speech localization process |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610287804.3A CN105975930A (en) | 2016-05-04 | 2016-05-04 | Camera angle calibration method during robot speech localization process |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105975930A true CN105975930A (en) | 2016-09-28 |
Family
ID=56994851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610287804.3A Pending CN105975930A (en) | 2016-05-04 | 2016-05-04 | Camera angle calibration method during robot speech localization process |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105975930A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106335071A (en) * | 2016-11-14 | 2017-01-18 | 上海木爷机器人技术有限公司 | The robot and robot control method |
| CN108446684A (en) * | 2018-05-22 | 2018-08-24 | 深圳腾视科技有限公司 | A kind of correcting mechanism and its calibration method of human face analysis system |
| CN108647633A (en) * | 2018-05-08 | 2018-10-12 | 腾讯科技（深圳）有限公司 | Recognition and tracking method, recognition and tracking device and robot |
| CN108664042A (en) * | 2017-03-29 | 2018-10-16 | 深圳大森智能科技有限公司 | The method for controlling rotation and system of robot eyes |
| CN108664889A (en) * | 2017-03-28 | 2018-10-16 | 卡西欧计算机株式会社 | Object detection device, object object detecting method and recording medium |
| CN108733215A (en) * | 2018-05-17 | 2018-11-02 | 华南农业大学 | One kind personalizes virtual assistant's direction of gaze control method |
| CN109000634A (en) * | 2018-06-04 | 2018-12-14 | 上海木木机器人技术有限公司 | A kind of based reminding method and system of navigation object travelling route |
| CN109361896A (en) * | 2018-10-17 | 2019-02-19 | 南京布拎信息科技有限公司 | A kind of recognition of face face positioning device and its application method |
| CN109459722A (en) * | 2018-10-23 | 2019-03-12 | 同济大学 | Voice interactive method based on face tracking device |
| CN109968351A (en) * | 2017-12-28 | 2019-07-05 | 深圳市优必选科技有限公司 | Robot and its control method, the device with store function |
| CN110139064A (en) * | 2018-09-29 | 2019-08-16 | 广东小天才科技有限公司 | Video call method of wearable device and wearable device |
| CN111128005A (en) * | 2019-11-22 | 2020-05-08 | 苏州马尔萨斯文化传媒有限公司 | Regional combined intelligent electronic billboard system based on multiple sensing |
| CN111147749A (en) * | 2019-12-31 | 2020-05-12 | 宇龙计算机通信科技(深圳)有限公司 | Photographing method, photographing device, terminal and storage medium |
| CN111708428A (en) * | 2019-03-18 | 2020-09-25 | 丰田自动车株式会社 | Communication system and method for controlling the same |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214291A (en) * | 2010-04-12 | 2011-10-12 | 云南清眸科技有限公司 | Method for quickly and accurately detecting and tracking human face based on video sequence |
| CN102567714A (en) * | 2011-12-14 | 2012-07-11 | 深圳市中控生物识别技术有限公司 | Method for correcting color image and black-and-white image based on double-camera face identification |
| CN103438906A (en) * | 2013-09-10 | 2013-12-11 | 上海海事大学 | Vision and satellite positioning sensor joint calibrating method suitable for robot navigation |
| CN103905733A (en) * | 2014-04-02 | 2014-07-02 | 哈尔滨工业大学深圳研究生院 | Method and system for conducting real-time tracking on faces by monocular camera |
| CN104284093A (en) * | 2014-10-22 | 2015-01-14 | 小米科技有限责任公司 | Panorama shooting method and device |
| CN105116994A (en) * | 2015-07-07 | 2015-12-02 | 百度在线网络技术（北京）有限公司 | Intelligent robot tracking method and tracking device based on artificial intelligence |
-
2016
- 2016-05-04 CN CN201610287804.3A patent/CN105975930A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102214291A (en) * | 2010-04-12 | 2011-10-12 | 云南清眸科技有限公司 | Method for quickly and accurately detecting and tracking human face based on video sequence |
| CN102567714A (en) * | 2011-12-14 | 2012-07-11 | 深圳市中控生物识别技术有限公司 | Method for correcting color image and black-and-white image based on double-camera face identification |
| CN103438906A (en) * | 2013-09-10 | 2013-12-11 | 上海海事大学 | Vision and satellite positioning sensor joint calibrating method suitable for robot navigation |
| CN103905733A (en) * | 2014-04-02 | 2014-07-02 | 哈尔滨工业大学深圳研究生院 | Method and system for conducting real-time tracking on faces by monocular camera |
| CN104284093A (en) * | 2014-10-22 | 2015-01-14 | 小米科技有限责任公司 | Panorama shooting method and device |
| CN105116994A (en) * | 2015-07-07 | 2015-12-02 | 百度在线网络技术（北京）有限公司 | Intelligent robot tracking method and tracking device based on artificial intelligence |
Non-Patent Citations (1)
| Title |
|---|
| WEIDONG SONG ETC: "Automatic Calibration Method Based on Improved Camera Calibration Template", 《 5TH IET INTERNATIONAL CONFERENCE ON WIRELESS, MOBILE AND MULTIMEDIA NETWORKS (ICWMMN 2013)》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106335071A (en) * | 2016-11-14 | 2017-01-18 | 上海木爷机器人技术有限公司 | The robot and robot control method |
| CN108664889A (en) * | 2017-03-28 | 2018-10-16 | 卡西欧计算机株式会社 | Object detection device, object object detecting method and recording medium |
| CN108664042A (en) * | 2017-03-29 | 2018-10-16 | 深圳大森智能科技有限公司 | The method for controlling rotation and system of robot eyes |
| CN109968351A (en) * | 2017-12-28 | 2019-07-05 | 深圳市优必选科技有限公司 | Robot and its control method, the device with store function |
| CN108647633A (en) * | 2018-05-08 | 2018-10-12 | 腾讯科技（深圳）有限公司 | Recognition and tracking method, recognition and tracking device and robot |
| CN108647633B (en) * | 2018-05-08 | 2023-12-22 | 腾讯科技（深圳）有限公司 | Identification tracking method, identification tracking device and robot |
| CN108733215A (en) * | 2018-05-17 | 2018-11-02 | 华南农业大学 | One kind personalizes virtual assistant's direction of gaze control method |
| CN108446684A (en) * | 2018-05-22 | 2018-08-24 | 深圳腾视科技有限公司 | A kind of correcting mechanism and its calibration method of human face analysis system |
| CN109000634A (en) * | 2018-06-04 | 2018-12-14 | 上海木木机器人技术有限公司 | A kind of based reminding method and system of navigation object travelling route |
| CN109000634B (en) * | 2018-06-04 | 2022-06-03 | 上海智蕙林医疗科技有限公司 | Navigation object traveling route reminding method and system |
| CN110139064A (en) * | 2018-09-29 | 2019-08-16 | 广东小天才科技有限公司 | Video call method of wearable device and wearable device |
| CN110139064B (en) * | 2018-09-29 | 2021-10-01 | 广东小天才科技有限公司 | Video call method of wearable device and wearable device |
| CN109361896A (en) * | 2018-10-17 | 2019-02-19 | 南京布拎信息科技有限公司 | A kind of recognition of face face positioning device and its application method |
| CN109361896B (en) * | 2018-10-17 | 2021-08-24 | 南京布拎信息科技有限公司 | Face positioning device for face recognition and use method thereof |
| CN109459722A (en) * | 2018-10-23 | 2019-03-12 | 同济大学 | Voice interactive method based on face tracking device |
| CN111708428A (en) * | 2019-03-18 | 2020-09-25 | 丰田自动车株式会社 | Communication system and method for controlling the same |
| CN111708428B (en) * | 2019-03-18 | 2023-08-15 | 丰田自动车株式会社 | Communication system and method for controlling the same |
| CN111128005A (en) * | 2019-11-22 | 2020-05-08 | 苏州马尔萨斯文化传媒有限公司 | Regional combined intelligent electronic billboard system based on multiple sensing |
| CN111147749A (en) * | 2019-12-31 | 2020-05-12 | 宇龙计算机通信科技(深圳)有限公司 | Photographing method, photographing device, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105975930A (en) | Camera angle calibration method during robot speech localization process | |
| Wang et al. | Study on eye gaze estimation | |
| Wang et al. | Estimating the eye gaze from one eye | |
| Venkateswarlu | Eye gaze estimation from a single image of one eye | |
| JP6815707B2 (en) | Face posture detection method, device and storage medium | |
| CN102830797B (en) | A kind of man-machine interaction method based on sight line judgement and system | |
| Licsár et al. | User-adaptive hand gesture recognition system with interactive training | |
| US20170371403A1 (en) | Gesture recognition using multi-sensory data | |
| CN108595008B (en) | Human-computer interaction method based on eye movement control | |
| WO2012101962A1 (en) | State-of-posture estimation device and state-of-posture estimation method | |
| US20150062010A1 (en) | Pointing-direction detecting device and its method, program and computer readable-medium | |
| CN111480164A (en) | Head pose and distraction estimation | |
| EP3531340B1 (en) | Human body tracing method, apparatus and device, and storage medium | |
| WO2020019504A1 (en) | Robot screen unlocking method, apparatus, smart device and storage medium | |
| US10991124B2 (en) | Determination apparatus and method for gaze angle | |
| KR20160129000A (en) | Real-time 3d gesture recognition and tracking system for mobile devices | |
| US9843789B2 (en) | Still-image extracting method and image processing device for implementing the same | |
| CN106371566A (en) | Correction module, method and computer readable recording medium for eye tracking | |
| US20060152478A1 (en) | Projection of synthetic information | |
| WO2018146922A1 (en) | Information processing device, information processing method, and program | |
| Neto et al. | Real-time head pose estimation for mobile devices | |
| Sun et al. | Visual hand tracking on depth image using 2-D matched filter | |
| CN116385538A (en) | Visual SLAM method, system and storage medium for dynamic scene | |
| WO2023077665A1 (en) | Palm position determination method and apparatus, and electronic device and storage medium | |
| Gao et al. | Palmprint recognition system with double-assistant-point on iOS mobile devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180904 Address after: 363000 a factory building of Jingcheng Malata Industrial Park, Nanjing County, Zhangzhou, Fujian. Applicant after: Zhangzhou Malata Technology Co., Ltd. Address before: 363000 Malata Nanjing Industrial Park, Zhangzhou, Fujian Applicant before: Nanjing Malata Technology Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160928 |
|
| WD01 | Invention patent application deemed withdrawn after publication |