CN106042005A - Bionic eye positioning tracking system and working method thereof - Google Patents

Abstract

The invention discloses a bionic eye positioning tracking system and a working method thereof. The system is mainly composed of six parts including a bionic eye pan-tilt module, a voice acquisition sensor array, a camera module, a pan-tilt movement control module, a controller module and an upper computer module. According to the bionic eye positioning tracking system and the working method of the bionic eye positioning tracking system, the three-dimensional space acoustic source localization technology and the visual processing algorithm are combined with the automatic control technology through the above system, the bionic eye positioning tracking system and the working method of the bionic eye positioning tracking system can be achieved, a bionic eye can face the center of the contour of the face of a speaker in real time, a bionic device can be more vivid, lively and friendly when communicating and interacting with people, and more interestingness is added.

Description

Bionic eye location tracking system and method for work thereof

Technical field

The present invention relates to various bionic eye to the location tracking system of target and method of work, particularly relate to bionic eye Binocular fixation is to the bionic eye location tracking system of face of speaking and method of work.

Background technology

At present, the various bio-robots (e.g., humanoid, the imitative animal shape etc.) sensory perceptual system to surrounding, mainly leads to Cross bionic eye and bionic ear to obtain external information.

Wherein, in the design of bionic eye, mainly serve as bionic eye by two photographic head of installation and obtain video information, To bionic eye overall structure founding mathematical models, some algorithms of application visual processes, it is achieved the three-dimensional information to target object Calculate and recognition of face etc..Technology related to this comparative maturity, and it is widely applied to industry, military affairs, medical treatment etc. Numerous areas.

But, the existing photographic head serving as bionic eye or be fixedly mounted on certain position, or can be by the behaviour of people Control travel direction adjusts, and typically cannot realize the automatic to regarding and tracking of bionic eye and current speaker, lack lively and friendly Communicative effect.

In the sound localization technology being similar to bionic ear, there is the sound signal collecting devices such as application mike and come real The application of existing sound localization, this technology is techniques known, e.g., following document: Zheng Zhenzhen, Feng Huajun, Shen Changyu etc. Three-dimensional acoustic localization algorithm [J] based on coordinate system transformation. journal of Zhejiang university, 2008,42 (2): 341-343 (under be called for short literary composition Offer 1), the most just refer to a kind of sound localization technology based on sodar time difference (TDOA), and give a kind of based on coordinate The Three-dimensional acoustic localization algorithm of system's conversion.

But, existing sound localization method is only to rely on the sound signal collecting devices such as mike to collect signal, Information source is single, stability is low, error is big, when sound-source signal is disturbed by extraneous factor, this sound localization method without Method realizes the precise positioning to speaker.

In exhibition or public occasion, under everybody surrounds and watches situation, how can be lively, accurately by the two of bionic eye photographic head Ground watches speaker attentively on the face, and can automatically follow the movement of speaker and rotate eyes (two photographic head), allows and talks with people Face can be in the picture centre that camera collection arrives in real time, there is no the research of this type of technology and method at present.

Summary of the invention

Cannot realize automatically watching attentively to solve existing bionic eye and follow the tracks of the technological deficiency of face of speaking, the present invention based on Bionic eye and the prior art of bionic ear, it is provided that a kind of bionic eye location tracking system, to realize two shootings of bionic eye Head is automatically watched attentively and traces into face of speaking.

Present invention simultaneously provides the method for work of this location tracking system.

For reaching above-mentioned purpose, the bionic eye location tracking system of the present invention is mainly made up of following six parts: bionic eye cloud Platform module, sound collection sensor array, camera module, The Cloud Terrace motion-control module, controller module, upper computer module； Wherein:

Described bionic eye The Cloud Terrace module includes: a The Cloud Terrace base, the sub-The Cloud Terrace of a carrying platform and two, left and right, its In, it is connected, by carrying platform on The Cloud Terrace base by a horizontal rotation joint between The Cloud Terrace base and carrying platform Horizontally rotate to imitate the neck rotation of bio-robot；Carrying platform is provided with two the sub-The Cloud Terraces in left and right, two, left and right Sub-The Cloud Terrace is respectively arranged with a horizontal rotating frame and a pitching swivel mount respectively, so, every sub-The Cloud Terrace all have horizontal rotation and Pitching rotates two cradle heads, and two photographic head are separately mounted on the pitching swivel mount in the middle of the sub-The Cloud Terrace in two, left and right, fill When the eyes of bionic eye, by the respective horizontal rotation of sub-The Cloud Terrace of two, left and right and the rotation of elevation rotary joint, can distinguish Adjust the photographic head being arranged on two the sub-The Cloud Terraces in left and right to point to, imitate the rotation of eyes, such bionic eye The Cloud Terrace module A total of five freedoms of motion, the center position on carrying platform, bind a neck that can horizontally rotate with carrying platform Portion's coordinate system.

Described sound collection sensor array serves as bionic ear in the present system, by mike identical for four characteristics, Centered by bionic eye The Cloud Terrace base fulcrum, it is arranged on around The Cloud Terrace base center by the space geometry relation of positive tetrahedron, and The Cloud Terrace base is propped up dot center's initial point as bionic eye The Cloud Terrace base coordinate system, so, by the acoustical signal collected and Sound localization algorithm (sound localization method that the document 1 etc. that can refer to mention in background technology is mentioned), obtains speaker's mouth D coordinates value relative to base coordinate system.

Described camera module is made up of two independent photographic head, is separately mounted in the middle of the sub-The Cloud Terrace in two, left and right On pitching swivel mount, serve as the eyes of bionic eye in the present system, be used for obtaining video information, and video data passed to upper Machine module.

Described The Cloud Terrace motion-control module includes: the horizontal rotation being separately mounted on two the sub-The Cloud Terraces in left and right drives electricity The horizontal rotation of machine and pitching rotary drive motor, a carrying platform drives motor, data transmission module.The Cloud Terrace motor control Module controls the driving motor of corresponding joint according to each driving electric machine rotation signal of the controller module output received Rotate.

Described controller module, on the one hand gathers the output signal of sound collection sensor array, runs sound localization Algorithm, it is thus achieved that speaker's mouth is relative to the D coordinates value of The Cloud Terrace base coordinate system, and by the three-dimensional seat of this speaker's mouth Mark passes to upper computer module；On the other hand receive control instruction and data message that upper computer module sends, control each and drive Dynamic electric machine rotation respective angles, the most also returns the kinestate of system to upper computer module.

Described upper computer module includes: the eye level corner α of bionic eye The Cloud Terrace₁And β₁Derivation program, to taking About carrying platform horizontal rotation the derivation program of rotational angle theta, bionic eye The Cloud Terrace module, two sub-The Cloud Terrace joint spaces solves Operation program, vision processing algorithm program and the signal procedure of controller.

The method of work of the present invention mainly comprises the steps of:

Step one, system initialization: system electrification, sound collection sensor array enters duty, upper computer module The video image that display camera module obtains, controller module controls all driving reset motors to initial angle.

Step 2, sound localization: controller module reads the voice data that sound collection sensor array gathers, and runs Sound localization algorithm obtains speaker's mouth D coordinates value relative to The Cloud Terrace base coordinate system, passes to upper by this coordinate figure Machine module.

Step 3, eye level corner solve: i.e. to two sub-The Cloud Terrace each level angles about bionic eye The Cloud Terrace module Solve.

Upper computer module relative to the D coordinates value of The Cloud Terrace base coordinate system, is converted to speaker according to speaker's mouth Mouth is relative to the D coordinates value of cervical region coordinate system, when, under the non-rotation status of forward neck portion, solving two eyes and turn respectively Level angle required when speaker's mouth；If wherein the level angle of any one exceedes ± and 30 degree (set point), this Time need cervical region is carried out to rotate and adjust, then turn to step 4；If the feathering angle of solve out two eyes does not all surpass Cross ± 30 degree (set point), at this moment need not carry out cervical region rotating adjust, then turn to step 5.

Step 4, neck joint is carried out rotate adjust: i.e. to carrying platform horizontal rotation corner solve.

Upper computer module first according to speaker's mouth relative to the D coordinates value of cervical region coordinate system, to meet carrying platform Turning to honest is condition towards speaker's mouth, and the anti-feathering angle solved needed for cervical region, upper computer module is by the water of cervical region The flat anglec of rotation issues controller module, controller module control the horizontal rotary drive motor of cervical region and rotate respective angles, it is achieved Carrying platform turns to honest in the face of speaker's mouth direction, and so, cervical region coordinate system also turns to honest in the face of speaker mouth side To, then turn to step 5, more relatively current cervical region coordinate system carries out the pass of two sub-The Cloud Terraces about bionic eye The Cloud Terrace module Joint space derivation, so that two eyes turn to speaker's mouth direction respectively.

About step 5, bionic eye The Cloud Terrace module, two sub-The Cloud Terrace joint spaces solve: upper computer module is according to speaker The relation of mouth relatively current cervical region coordinate system carries out two sub-The Cloud Terrace joint spaces about bionic eye The Cloud Terrace module and solves fortune Calculate, the most counter solve the pitching rotary drive motor of two the sub-The Cloud Terraces in left and right, horizontal rotation drives the angle that should rotate of motor Information, is then issued to controller module, by controller module control about two sub-The Cloud Terrace respective pitching rotary drive motors, Horizontal rotation drives electric machine rotation respective angles, to realize the eyes of bionic eye tentatively to the mouth of speaker.

So far, complete the process of two photographic head (eyes) tentatively alignment speaker's mouth.

Step 6, eyes adjustment in direction: i.e. to two each rotary joint corners of sub-The Cloud Terrace about bionic eye The Cloud Terrace module Correction.Upper computer module runs face recognition algorithms, obtains speaker face rectangular peripheral profile and asks for its centre bit Put the two-dimensional coordinate value in video image coordinate system, according to the side-play amount of this two-dimensional coordinate value Yu video image center, Obtain the pitching rotary drive motor of two the sub-The Cloud Terraces in left and right, the correction value of the horizontal rotation driving motor anglec of rotation respectively, so After above-mentioned anglec of rotation correction value is issued controller module again, by controller module control about two sub-The Cloud Terraces respective Pitching rotary drive motor, horizontal rotation drive motor fine setting respective angles, to realize to two eyes of bionic eye the most precisely Face mask center position towards speaker so that speaker face rectangular profile center occurs in the video of photographic head identification Picture centre.

If step 7 controller module detects when new speaker occurs, then circulation performs step 2, step 3, step Rapid four, step 5, step 6, carry out real-time tracking to emerging face of speaking；If not having new speaker to occur, then circulate Perform step 6 so that the corresponding degree of freedom of eyes adjusts according to the movement of face location of currently speaking.

Step 8, detection whole system the need of working on, the most then turn to step 7, it may be judged whether have new Speaker occurs, if it is not, shutdown of then deactivating.

Below in conjunction with operation principle advantages of the present invention is described:

Two photographic head serving as bionic eye are arranged on a The Cloud Terrace with five degree of freedom by the present invention；By three-dimensional Spatial sound source location technology, it is achieved the preliminary three-dimensional localization to speaker's mouth position；Solve according to The Cloud Terrace joint space Operation method, obtains and the corresponding corner in each joint of rotary platform, bionic eye is stared at the direction of Primary Location；In conjunction with face Recognition methods, finds out the accurate orientation at speaker's face mask center, and the corner of fine-adjustment tripod head associated joint, with by speaker Face mask move to serve as two camera collections of bionic eye picture centre till；When speaker walks about, bionic eye The change in location following this face of speaking is rotated adjustment automatically, to ensure that the face of speaker can be in photographic head in real time and adopt The picture centre of collection；During until another person lifts up one's voice again, bionic eye just can be noted again according to aforesaid work process again Depending on new face of speaking.

In a word: three dimensions sound localization technology, vision processing algorithm are combined by the present invention with automatic control technology, can To realize a kind of bionic eye location tracking system and method for work so that bionic eye can be in real time towards speaker's face mask Center, can allow bionic device when with people's exchange and interdynamic, it appears more lively, active and friendly, increase more interest.

Accompanying drawing explanation

Fig. 1 is bionic eye location tracking system overall relation block diagram；

Fig. 2 is bionic eye location tracking system population structure schematic diagram；

Fig. 3 is bionic eye The Cloud Terrace modular structure schematic diagram；

Fig. 4 is The Cloud Terrace motion-control module structural representation；

Spatial relationship schematic diagram between Fig. 5 speaker's face mask center and two camera image coordinate systems；

Each coordinate system relation schematic diagram when Fig. 6 is initial position；

Fig. 7 is bionic eye location tracking system workflow diagram.

Marginal data:

Fig. 1: 100-sound collection sensor array；200-camera module；300-bionic eye The Cloud Terrace module；400-The Cloud Terrace Motion-control module；500-controller module；600-upper computer module；

Fig. 2: 100-sound collection sensor array；101-1 mike；102-2 mike；103-3 mike； 104-4 mike；200-camera module；The left photographic head of 201-；The right photographic head of 202-；300-bionic eye The Cloud Terrace module；

Fig. 3: 301-The Cloud Terrace base；302-carrying platform；The left sub-The Cloud Terrace of 303-；The right sub-The Cloud Terrace of 304-；The left sub-The Cloud Terrace of 305- Horizontal rotating frame；The pitching swivel mount of the left sub-The Cloud Terrace of 306-；The horizontal rotating frame of the right sub-The Cloud Terrace of 307-；The right sub-The Cloud Terrace of 308- Pitching swivel mount；

The horizontal rotation of Fig. 4: 401-carrying platform drives motor；The horizontal rotation of the left sub-The Cloud Terrace of 402-drives motor； The pitching rotary drive motor of the left sub-The Cloud Terrace of 403-；The horizontal rotation of the right sub-The Cloud Terrace of 404-drives motor；The right sub-The Cloud Terrace of 405- Pitching rotary drive motor.

Detailed description of the invention

Below in conjunction with the accompanying drawings, bionic eye location tracking system and the method for work of the present invention are described.

The bionic eye location tracking system of the present invention is mainly made up of following six part: sound collection sensor array 100, camera module 200, bionic eye The Cloud Terrace module 300, The Cloud Terrace motion-control module 400, controller module 500, host computer Module 600；As it is shown in figure 1, wherein:

Described sound collection sensor array 100 serves as bionic ear in the present system, respectively by No. 1 mike 101,2 Number mike 103, No. 4 mikes 104 of 102, No. 3 mikes, as in figure 2 it is shown, in bionic eye The Cloud Terrace base 301 fulcrum being The heart, is arranged on around The Cloud Terrace base center according to the space geometry relation of positive tetrahedron, and 301 dot center of The Cloud Terrace base is made For bionic eye The Cloud Terrace base coordinate system O₀Initial point, as shown in Figure 6, so, by the acoustical signal collected and sound localization Algorithm, available speaker's mouth is relative to the D coordinates value of base coordinate system.

Described camera module 200 is made up of left photographic head 201 and right photographic head 202, as in figure 2 it is shown, by left shooting 201 are arranged on the pitching swivel mount 306 of left sub-The Cloud Terrace 303, and right photographic head 201 is arranged on the pitching of right sub-The Cloud Terrace 304 On swivel mount 308, serve as the eyes of bionic eye in the present system, be used for obtaining video information, and video data passed to upper Machine module 600.

Described bionic eye The Cloud Terrace module 300 includes: 301, carrying platform 302 of a The Cloud Terrace base, left sub-The Cloud Terrace 303, right sub-The Cloud Terrace 304, as it is shown on figure 3, wherein, is closed by a horizontal rotation between The Cloud Terrace base 301 and carrying platform 302 Joint is connected, and is imitated the neck rotation of bio-robot by carrying platform horizontally rotating on The Cloud Terrace base 301；Carrying Being provided with left sub-The Cloud Terrace 303 and right sub-The Cloud Terrace 304 on platform 302, left sub-The Cloud Terrace 303 has a horizontal rotating frame 305 and one Pitching swivel mount 306, right sub-The Cloud Terrace 304 has a horizontal rotating frame 307 and a pitching swivel mount 308, so, every sub-cloud Platform all has horizontal rotation and pitching rotates two cradle heads, and two photographic head are separately mounted in the middle of two the sub-The Cloud Terraces in left and right Pitching swivel mount on, serve as the eyes of bionic eye, rotated by the respective horizontal rotation of sub-The Cloud Terrace of two, left and right and pitching and close The rotation of joint, can adjust the photographic head being arranged on two the sub-The Cloud Terraces in left and right respectively and point to, imitate the rotation of eyes, so A total of five freedoms of motion of bionic eye The Cloud Terrace module 300, and at the center position of carrying platform 302, bind one Cervical region coordinate system O that can horizontally rotate with carrying platform 302 and rotate₁, as shown in Figure 6.

Described The Cloud Terrace motion-control module 400 includes: the horizontal rotation being arranged on left sub-The Cloud Terrace 303 drives motor 402 and pitching rotary drive motor 403, the horizontal rotation that is arranged on right sub-The Cloud Terrace 304 drive motor 404 and pitching to rotate to drive The horizontal rotation of 405, carrying platform 302 of galvanic electricity machine drives motor 401, data transmission module 406, as shown in Figure 4, The Cloud Terrace Motion-control module 400 controls according to each driving electric machine rotation signal of controller module 500 output received accordingly The driving motor in joint rotates.

Described controller module 500, on the one hand gathers the output signal of sound collection sensor array 100, operation sound Source location algorithm, it is thus achieved that speaker's mouth is relative to The Cloud Terrace base coordinate system O₀D coordinates value, and by the three of this speaker Dimension coordinate passes to upper computer module 600, as shown in Figure 1；On the other hand the control instruction sum that upper computer module 600 sends is received It is believed that breath, control each and drive electric machine rotation respective angles, also return the motion shape of system simultaneously to upper computer module 600 State.

Described upper computer module 600 includes: the eye level corner α of bionic eye The Cloud Terrace₁And β₁Derivation program, Carrying platform 302 is horizontally rotated two sub-The Cloud Terrace joint spaces about the derivation program of rotational angle theta, bionic eye The Cloud Terrace module Derivation program, vision processing algorithm program and the signal procedure of controller 500.

The present invention realizes bionic eye and mainly comprises the steps of the track and localization specific works method of face of speaking:

Step one, system initialization: system electrification, sound collection sensor array 100 enters duty, host computer mould Block 600 shows the video image that camera module 200 obtains, and controller module 500 controls to be arranged on the water on left sub-The Cloud Terrace 303 Flat rotary drive motor 402 and pitching rotary drive motor 403, the horizontal rotation being arranged on right sub-The Cloud Terrace 304 drive motor 404 and 405, carrying platform 302 of pitching rotary drive motor on horizontal rotation drive motor 401 reset to initial angle Degree.

Step 2, sound localization: controller module 500 reads the data that sound collection sensor array 100 gathers, and transports Row sound localization algorithm obtains speaker's mouth relative to The Cloud Terrace base coordinate system O₀D coordinates value, this coordinate figure is passed to Upper computer module 600.

Step 3, eye level corner solve: i.e. turn about 300 two sub-The Cloud Terrace each levels of bionic eye The Cloud Terrace module Angle α₁And β₁Solve.

Upper computer module 600 according to speaker's mouth relative to The Cloud Terrace base coordinate system O₀D coordinates value, be converted to Speaker's mouth is relative to cervical region coordinate system O₁D coordinates value, under the current non-rotation status of carrying platform 302, solve When left photographic head 201 and right photographic head 202 turn to speaker's mouth respectively, the horizontal rotation of left sub-The Cloud Terrace 303 drives the corner of motor Degree α₁With the gyration β that the horizontal rotation of right sub-The Cloud Terrace 304 drives motor₁If wherein the level angle of any one exceedes ± 30 ° (set point), at this moment needs to carry out carrying platform 302 cervical region rotating to adjust, then turns to step 4；If the left side solving out The horizontal rotation of sub-The Cloud Terrace 303 drives the gyration α of motor₁With the gyration β that the horizontal rotation of right sub-The Cloud Terrace 304 drives motor₁The most not Exceed ± 30 ° (set point), at this moment need not carry out carrying platform 302 rotating adjust, then turn to step 5.

Step 4, neck joint is carried out rotate adjust: i.e. carrying platform 302 is horizontally rotated rotational angle theta and solves.

Upper computer module 600 first according to speaker's mouth relative to cervical region coordinate system O₁D coordinates value, with meet take Carrying platform 302 turn to honest towards speaker's mouth be condition, the anti-feathering angle θ solved needed for carrying platform 302 cervical region. Feathering angle θ needed for carrying platform 302 cervical region is issued controller module 500, by controller module by upper computer module 600 500 horizontal rotations controlling carrying platform 302 drive motor 401 to rotate respective angles θ, it is achieved carrying platform 302 turns to honest In the face of speaker's mouth direction, so, cervical region coordinate system O₁X-axis also turn to honest in the face of speaker's mouth direction, then turn To step 5, more relatively current cervical region coordinate system O₁Carry out the joint space of two sub-The Cloud Terraces about bionic eye The Cloud Terrace module to ask Solve computing, so that left photographic head 201 and right photographic head 202 turn to speaker's mouth direction respectively

Step 5, about 300 two sub-The Cloud Terrace joint spaces of bionic eye The Cloud Terrace module solve: upper computer module 600 basis Speaker's mouth relatively current cervical region coordinate system O₁Relation carry out the joint of two sub-The Cloud Terraces about bionic eye The Cloud Terrace module Space derivation, the most anti-horizontal rotation solving left sub-The Cloud Terrace 303 drives motor 402 and pitching rotary drive motor 403 Anglec of rotation α₁、α₂, the horizontal rotation of right sub-The Cloud Terrace 304 drives motor 404 and the anglec of rotation of pitching rotary drive motor 405 Degree β₁、β₂, being then issued to controller module 500, controller module 500 horizontal rotation controlling left sub-The Cloud Terrace 303 drives motor 402 and the horizontal rotation of pitching rotary drive motor 403, right sub-The Cloud Terrace 304 drive motor 404 and pitching rotary drive motor 405 rotate respective angles, preliminary to the mouth of speaker with the left photographic head 201 and right photographic head 202 realizing bionic eye.

So far, complete the process of two photographic head (eyes) tentatively alignment speaker's mouth.

Step 6, eyes adjustment in direction: i.e. to two each rotary joint corners of sub-The Cloud Terrace about bionic eye The Cloud Terrace module Correction.

Upper computer module 600 runs face recognition algorithms, obtains speaker face rectangular peripheral profile and asks for wherein Heart position is in video image coordinate system O of left photographic head 201_l-X_lY_lVideo image coordinate system O with right photographic head 202_r-X_rY_r In two-dimensional coordinate value (X_l, Y_l) and (X_r, Y_r), as it is shown in figure 5, set picture traverse as W, height is H, according to this two-dimensional coordinate Value and the side-play amount of video image center (W/2, H/2), obtain the horizontal rotation driving motor of left sub-The Cloud Terrace 303 respectively 402 anglecs of rotation α₁Correction value Δ α₁With pitching rotary drive motor 403 anglec of rotation α₂Correction value Δ α₂, and right son The horizontal rotation of The Cloud Terrace 304 drives motor 404 anglec of rotation β₁Correction value Δ β₁With pitching rotary drive motor 405 anglec of rotation Degree β₂Correction value Δ β₂, then above-mentioned anglec of rotation correction value is issued controller module 500 again, by controller module 500 The horizontal rotation controlling left sub-The Cloud Terrace 303 drives motor 402 and pitching rotary drive motor 403, the level rotation of right sub-The Cloud Terrace 304 Turn driving motor 404 and pitching rotary drive motor 405 finely tunes respective angles, to realize left photographic head 201 and the right side of bionic eye Photographic head 201 is the most precisely towards speaker face rectangular profile center position so that in speaker face rectangular peripheral profile The heart occurs in the video image center of photographic head identification.

If step 7 controller module 500 detects when new speaker occurs, then circulation performs step 2, step Three, step 4, step 5, step 6, carries out real-time tracking to emerging face of speaking；If not having new speaker to occur, Then circulation performs step 6 so that the corresponding degree of freedom of eyes adjusts according to the movement of face location of currently speaking.

Step 8, detection whole system the need of working on, the most then turn to step 7, it may be judged whether have new Speaker occurs, if it is not, shutdown of then deactivating.

In method of work of the present invention: specific algorithm that the eye level corner mentioned in step 3 solves, step 4 carry To cervical region is carried out rotate the computational methods adjusted, two sub-The Cloud Terraces close about the bionic eye The Cloud Terrace module mentioned in step 5 Joint space method for solving is technology commonly used in the art.Understand for the ease of auditor, describe in detail separately below.

The present invention describes involved all angular units and is all taken as: degree, all length unit is all taken as: millimeter (mm).

The specific algorithm that the eye level corner mentioned in above-mentioned steps three solves comprises the steps of:

Step 3.1, as shown in Figure 6, sets up fixing The Cloud Terrace base coordinate system in bionic eye The Cloud Terrace module 300 respectively O₀-X₀Y₀Z₀, cervical region coordinate system O that horizontally rotates₁-X₁Y₁Z₁, left sub-The Cloud Terrace 303 base coordinate system O₂-X₂Y₂Z₂, left sub-The Cloud Terrace 303 horizontal rotation coordinate systems O₃-X₃Y₃Z₃, 303 left sub-The Cloud Terrace pitching rotating coordinate system O₄-X₄Y₄Z₄, right sub-The Cloud Terrace 304 basis is sat Mark system O₅-X₅Y₅Z₅Coordinate system, right sub-The Cloud Terrace 304 horizontally rotates coordinate system O₆-X₆Y₆Z₆, right sub-The Cloud Terrace 304 pitching rotating coordinate system O₇-X₇Y₇Z₇, it is briefly referred to as O₀、O₁、O₂、O₄、O₅、O₆、O₇Coordinate system, wherein:

O₀Coordinate system build 301 dot center of bionic eye The Cloud Terrace base in, obtains speaker's mouth through sound localization algorithm D coordinates value is the statement under this coordinate system, for the base coordinate system of whole bionic eye The Cloud Terrace；

O₁Coordinate system build carrying platform 302 center above in, bind with carrying platform 302 together with, be equivalent to cervical region Coordinate system, O₁Coordinate system follows horizontally rotating and rotating of carrying platform 302, and its corner θ represents；

O₂In the bottom center plane of the horizontal rotation joints axes that coordinate system build left sub-The Cloud Terrace 303 in, with carrying platform 302 bindings together, for left sub-The Cloud Terrace 303 base coordinate system；

O₃Its initial point of coordinate system just build the surface of the horizontal rotation joints axes of left sub-The Cloud Terrace 303, and and pitching in Joints axes is contour, O₃Coordinate system is bundled on left sub-The Cloud Terrace 303 horizontal rotating frame 305 rightly, along with left sub-The Cloud Terrace 303 The horizontally rotating and rotate of horizontal rotating frame 305, its corner α₁Represent；

O₄Coordinate origin and O₃Coordinate origin overlaps, O₄Coordinate system is bundled in left sub-The Cloud Terrace 303 pitching rightly and rotates On frame 306, rotate along with the pitch rotation of left sub-The Cloud Terrace 303 pitching swivel mount 306, because O₄The X of coordinate system₄Axle positive direction Just with the optical axis coincidence of the left photographic head 201 being fixed on pitching swivel mount 306, as it is shown in figure 5, therefore, O₄Coordinate system is also referred to as For left photographic head 201 coordinate system, left photographic head 201 will be with left sub-The Cloud Terrace 303 horizontal rotating frame 305 and pitching swivel mount The rotation of 306 and rotate, O₄Coordinate system is with the corner α of left sub-The Cloud Terrace 303 pitching swivel mount 306 pitching₂Represent；

O₅In the bottom center plane of the horizontal rotation joints axes that coordinate system build right sub-The Cloud Terrace 304 in, with carrying platform 302 bindings together, for right sub-The Cloud Terrace 304 base coordinate system；

O₆Its initial point of coordinate system just build the surface of the horizontal rotation joints axes of right sub-The Cloud Terrace 304, and and pitching in Joints axes is contour, O₆Coordinate system is bundled on right sub-The Cloud Terrace horizontal rotating frame 307 rightly, along with right sub-The Cloud Terrace 304 level The horizontally rotating and rotate of swivel mount 307, its corner β₁Represent；

O₇Coordinate origin and O₆Coordinate origin overlaps, O₇Coordinate system is bundled in right sub-The Cloud Terrace 304 pitching rightly and rotates On frame 308, rotate along with the pitch rotation of right sub-The Cloud Terrace 304 pitching swivel mount 308, because O₇The X of coordinate system₇Axle positive direction Just with the optical axis coincidence of the right photographic head 201 being fixed on pitching swivel mount 308, therefore, O₇Coordinate system is also referred to as right photographic head Coordinate system, right shooting 201 will be with right sub-The Cloud Terrace 304 horizontal rotating frame 307 and the rotation of pitching swivel mount 308 and turns Dynamic, O₇Coordinate system is with the corner β of right sub-The Cloud Terrace 304 pitching swivel mount 308 pitching₂Represent；

Step 3.2, spatial correlation describe:

As shown in Figure 6, by cervical region coordinate system O₁Relative to The Cloud Terrace base coordinate system O₀Translation and horizontal rotation relation square Battle array A₀Describe, by O₂Coordinate system is relative to O₁The translation relation matrix A of coordinate system₁Describe, by O₃Coordinate system is relative to O₂Coordinate The translation of system and horizontal rotation relation matrix A₂Describe, by O₄Coordinate system is relative to O₃The pitching rotation relationship square of coordinate system Battle array A₃Describe, by O₅Coordinate system is relative to O₁The translation relation matrix A of coordinate system₄Describe, by O₆Coordinate system is relative to O₅Coordinate The translation of system and horizontal rotation relation matrix A₅Describe, by O₇Coordinate system is relative to O₆The pitching rotation relationship square of coordinate system Battle array A₆Describe, by speaker's mouth relative to The Cloud Terrace base coordinate system O₀Vector correlation with matrix P describe, by speaker's mouth Relative to left photographic head coordinate system O₄Vector correlation matrix P_LDescribe, by the mouth of speaker relative to right photographic head coordinate It is O₇Vector correlation matrix P_RDescribe；

Spatial relation description according to above-mentioned coordinate system can be expressed as following series matrix:

$\begin{array}{cc}{A}_0=\left[\begin{array}{cccc}cos\mathrm{\θ}& -sin\mathrm{\θ}& 0& 0\\ sin\mathrm{\θ}& \cos \mathrm{\θ}& 0& 0\\ 0& 0& 1& {\mathrm{Pz}}_1\\ 0& 0& 0& 1\end{array}\right]---\left(1\right)& A_1=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& P_y\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]---\left(2\right)\end{array}$

$\begin{array}{cc}{A}_2=\left[\begin{array}{cccc}{\mathrm{cos\α}}_1& -{\mathrm{sin\α}}_1& 0& 0\\ {\mathrm{sin\α}}_1& {\mathrm{cos\α}}_1& 0& 0\\ 0& 0& 1& {\mathrm{Pz}}_2\\ 0& 0& 0& 1\end{array}\right]---\left(3\right)& A_3=\left[\begin{array}{cccc}{\mathrm{cos\α}}_2& 0& {\mathrm{sin\α}}_2& 0\\ 0& 1& 0& 0\\ -{\mathrm{sin\α}}_2& 0& {\mathrm{cos\α}}_2& 0\\ 0& 0& 0& 1\end{array}\right]---\left(4\right)\end{array}$

$\begin{array}{cc}{A}_4=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& -P_y\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]---\left(5\right)& A_5=\left[\begin{array}{cccc}{\mathrm{cos\β}}_1& -{\mathrm{sin\β}}_1& 0& 0\\ {\mathrm{sin\β}}_1& {\mathrm{cos\β}}_1& 0& 0\\ 0& 0& 1& {\mathrm{Pz}}_2\\ 0& 0& 0& 1\end{array}\right]---\left(6\right)\end{array}$

$\begin{array}{cc}{A}_6=\left[\begin{array}{cccc}{\mathrm{cos\β}}_2& 0& {\mathrm{sin\β}}_2& 0\\ 0& 1& 0& 0\\ -{\mathrm{sin\β}}_2& 0& {\mathrm{cos\β}}_2& 0\\ 0& 0& 0& 1\end{array}\right]---\left(7\right)& P={\left[\begin{array}{cccc}X& Y& Z& 1\end{array}\right]}^T---\left(8\right)\end{array}$

P_L=[X_L Y_L Z_L 1]^T (9)P_R=[X_R Y_R Z_R 1]^T (10)

In matrix: Pz₁For O₁Coordinate origin is relative to O₀The height of coordinate origin, for known；P_yFor cervical region coordinate system O₁Distance between initial point and two, left and right sub-The Cloud Terrace base coordinate system initial point, for known；Pz₂For O₃Coordinate origin relative to Coordinate origin and O₆Coordinate origin is relative to O₅The height of coordinate origin, for known；

θ is O₁Coordinate system is relative to O₀The horizontal rotation angle of coordinate system, is also the horizontal rotary drive motor of cervical region 401 Gyration, seen from above, is just counterclockwise, is negative clockwise, can read at any time, for known quantity；

α₁For O₃Coordinate system is relative to O₂The horizontal rotation angle of coordinate system, is also that the horizontal rotation of left sub-The Cloud Terrace drives motor The gyration of 402, this corner serves as horizontally rotating of left eye, seen from above, is just counterclockwise, is negative clockwise, is demand solution Amount；

α₂For O₄Coordinate system is relative to O₃The pitching anglec of rotation of coordinate system, is also left sub-The Cloud Terrace pitching rotary drive motor The gyration of 403, this corner serves as the pitch rotation of left eye, and from the side, nutation is just, faces upward as negative, is demand solution amount；

β₁For O₆Coordinate system is relative to O₅The horizontal rotation angle of coordinate system, is also that the horizontal rotation of right sub-The Cloud Terrace drives motor The gyration of 404, this corner serves as horizontally rotating of right eye, seen from above, is just counterclockwise, is negative clockwise, is demand solution Amount；

β₂For O₇Coordinate system is relative to O₆The pitching anglec of rotation of coordinate system, is also right sub-The Cloud Terrace pitching rotary drive motor The gyration of 405, this corner serves as the pitch rotation of right eye, and from the side, nutation is just, faces upward as negative, is demand solution amount；

X, Y, Z are the D coordinates value of speaker's mouth, and via controller module 500 is run sound localization algorithm and obtained in real time , as known quantity when calculating herein；

When the sub-The Cloud Terrace in two, left and right is towards dead ahead, the feathering angle of two the sub-The Cloud Terraces in left and right is respectively as follows: α₁=0 °, β₁=0 °；

When left photographic head 201 is just to acoustic target, must there is Y_L=0, Z_L=0, X_LIt it is demand solution amount；

When right photographic head 202 is just to acoustic target, must there is Y_R=0, Z_R=0, X_RIt it is demand solution amount；

Step 3.3, in the case of cervical region does not rotates, carry out bionic eye The Cloud Terrace left sub-The Cloud Terrace 303 horizontal rotation drive electricity The gyration α of machine₁With the gyration β that the horizontal rotation of right sub-The Cloud Terrace 304 drives motor₁Derivation:

According to the spatial relationship between each coordinate system above-mentioned, can be by speaker's mouth relative to The Cloud Terrace base coordinate system The vector of D coordinates value describe P and left photographic head 201, right photographic head 202 coordinate system X-axis honest towards speaker's mouth time Vector P is described_L、P_RSet up following relational expression:

$\left\{\begin{array}{c}P=A_0\·A_1\·A_2\·A_3\·P_L\\ P=A_0\·A_4\·A_5\·A_6\·P_R\end{array}\right.---\left(11\right)$

Now, the horizontal rotation angle α needed for two the sub-The Cloud Terraces in left and right need to only be calculated₁With β₁, by aforementioned each formula and Know that condition can be derived acquisition:

$\left\{\begin{array}{c}{\mathrm{\α}}_1=\arctan \frac{Y-Py}{X}\\ {\mathrm{\β}}_1=\arctan \frac{Y+Py}{X}\end{array}\right.---\left(12\right)$

So, in the case of neck joint does not rotates (θ=0 °), solve and obtained eyes and see to speaker's mouth position Time, the horizontal rotation angle α needed for the sub-The Cloud Terrace in two, left and right₁With β₁, i.e. obtained the horizontal rotation of left sub-The Cloud Terrace 303 and driven motor Gyration α₁With the gyration β that the horizontal rotation of right sub-The Cloud Terrace 304 drives motor₁。

The computational methods that cervical region carries out rotating adjustment mentioned in above-mentioned steps four are:

If neck joint being rotated after adjusting, cervical region coordinate system O₁Should be honest towards speaker's mouth, if after Tiao Zhenging The mouth of speaker is relative to cervical region coordinate system O₁Vector correlation matrix P₁Describe, if O after Tiao Zhenging₁Cervical region coordinate system relative to The Cloud Terrace base coordinate system O₀Translation and horizontal rotation relation B describe, then they can be write as lower column matrix:

P₁=[X₁ Y₁ Z₁ 1]^T (13)In matrix: (X₁, Y₁, Z₁) it is that speaker's mouth is in cervical region coordinate system O₁In coordinate statement, when cervical region coordinate system O₁X₁Axle is honest towards saying Y during words people's mouth₁=0, it is definite value, X₁、Z₁It it is demand solution amount；θ is the anglec of rotation of the horizontal rotary drive motor of cervical region 401, from Top is seen, is just counterclockwise, is negative clockwise, is demand solution amount；Pz₁For coordinate system relative to O₀The height of coordinate system, can measure Arrive, for known quantity；

According to the spatial relationship between each coordinate system above-mentioned, can be by speaker's mouth relative to The Cloud Terrace base coordinate system O₀ The vector of D coordinates value describe P with relative to cervical region coordinate system O after adjusting₁Vector P is described₁Set up following relational expression:

P=B P₁ (15)

(15) formula that (8) formula and known conditions substituted into can be derived acquisition:

$\left\{\begin{array}{c}\mathrm{\θ}=\arctan \frac{Y}{X}\\ X_1=\sqrt{X^2+Y^2}\\ Z_1=Z-{\mathrm{Pz}}_1\end{array}\right.---\left(16\right)$

Above-mentioned required bionic eye neck joint horizontal rotation angle θ, is, if making carrying platform 302 just to speaking The mouth direction of people, carrying platform 302 is required obtains the anglec of rotation, is also the rotation needed for the horizontal rotary drive motor of cervical region 401 Angle, θ.

Mentioning in above-mentioned steps five, about bionic eye The Cloud Terrace module, two sub-The Cloud Terrace joint space method for solving are:

Now, neck joint adjusts, according to the relationship description formula between each coordinate system of aforementioned foundation without making to rotate (1), when two, left and right photographic head is all directed at the mouth position of same speaker, should there be formula (11)-(10) equally.

Can be derived acquisition by aforementioned each formula and known conditions:

$\left\{\begin{array}{c}{\mathrm{\α}}_1=\arctan \frac{Y-Py}{X}\\ {\mathrm{\α}}_2=\arctan \frac{{\mathrm{Pz}}_1+{\mathrm{Pz}}_2-Z}{\sqrt{{(Y-Py)}^2+X^2}}\\ X_L=\sqrt{{(Y-Py)}^2+X^2+{({\mathrm{Pz}}_1+{\mathrm{Pz}}_2-Z)}^2}\\ {\mathrm{\β}}_1=\arctan \frac{Y+Py}{X}\\ {\mathrm{\β}}_2=\arctan \frac{{\mathrm{Pz}}_1+{\mathrm{Pz}}_2-Z}{\sqrt{{(Y+Py)}^2+X^2}}\\ X_R=\sqrt{{(Y+Py)}^2+X^2+{({\mathrm{Pz}}_1+{\mathrm{Pz}}_2-Z)}^2}\end{array}\right.---\left(17\right)$

Above-mentioned anglec of rotation α solving the left and right corresponding degree of freedom of sub-The Cloud Terrace₁、α₂、β₁、β₂Process, be and anti-solve left sub-cloud The horizontal rotation of platform 303 drives motor 402 anglec of rotation α₁With pitching rotary drive motor 403 anglec of rotation α₂, right sub-The Cloud Terrace The horizontal rotation of 304 drives anglec of rotation β of motor 404₁Anglec of rotation β with pitching rotary drive motor 405₂Calculating Journey, wherein, X_LIt is that the photocentre of left photographic head 201 is to the distance between speaker's mouth, X_RIt is that right photographic head photocentre is to mouth of speaking Distance between portion.

The computational methods step of the eyes adjustment in direction value that step 6 of the present invention is carried is as follows: following all angular units are equal It is taken as: degree (°), all Parameter units relevant to video image are all taken as: pixel；

Step 6.1, upper computer module 600 read and show what the left photographic head 201 of bionic eye and right photographic head 202 obtained Video image, if picture traverse is W, height is H；

Step 6.2, upper computer module 600 run face recognition algorithms, know at left photographic head 201, right photographic head 202 respectively In other video image, calculate the face rectangular peripheral profile center video image coordinate at left photographic head 201 of speaking It is O_l-X_lY_lVideo image coordinate system O with right photographic head 201_r-X_rY_rIn two-dimensional coordinate value: (X_l, Y_l)、(X_r, Y_r), such as figure Shown in 5.

Step 6.3, take video image coordinate system O of left photographic head 201_l-X_lY_lSit with the video image of right photographic head 201 Mark system O_r-X_rY_rCenter position coordinates value be: (W/2, H/2), then left photographic head 201 identify video image in, speak Two-dimensional coordinate value (the X of face center_l, Y_l) and two-dimensional coordinate value (W/2, the H/2) side-play amount of video image center For:

$\left\{\begin{array}{c}{\mathrm{\ΔX}}_l=X_l-W/2\\ {\mathrm{\ΔY}}_l=Y_l-H/2\end{array}\right.---\left(18\right)$

In the video image that right photographic head 201 identifies, the two-dimensional coordinate value (X of speaker face center_r, Y_r) with Two-dimensional coordinate value (W/2, the H/2) side-play amount of video image center is:

$\left\{\begin{array}{c}{\mathrm{\ΔX}}_r=X_r-W/2\\ {\mathrm{\ΔY}}_r=Y_r-H/2\end{array}\right.---\left(19\right)$

Step 6.4, ask for eyes corresponding freedom degree rotating angle correction according to side-play amount:

Left eye horizontal rotation freedom degree rotating angle correction value: Δ α₁=arctan (Δ X_l/length)； (20)

Left eye pitching rotary freedom anglec of rotation correction value: Δ α₂=arctan (Δ Y_l/length)； (21)

Right eye horizontal rotation freedom degree rotating angle correction value: Δ β₁=arctan (Δ X_r/length)； (22)

Right eye pitching rotary freedom anglec of rotation correction value: Δ β₂=arctan (Δ Y_r/length)； (23)

Wherein, length is the focal length parameter of photographic head, sets according to the actual parameter of photographic head；Anglec of rotation correction value Δα₁、Δα₂、Δβ₁、Δβ₂In all comprise positive and negative number two kinds of situations.

After respective drive motor rotates above-mentioned correction angle, speaker face rectangular peripheral profile can be adjusted in image Heart position.

Two embodiments are set forth below and further illustrate present invention application in practice

Embodiment:

Embodiment 1: bionic eye cradle head structure design parameter is: P_z1=150mm, P_z2=120mm, P_y1=120mm, it is assumed that Run sound localization algorithm by controller module 500 and obtain speaker's mouth relative to The Cloud Terrace base coordinate system O₀Three-dimensional coordinate Value P=[6,000 2,000 3,000 1]^T；First, obtain according to formula (12) formula and known conditions in step 3 of the present invention:

α₁=17.398 °, β₁=19.461 °

Two angle values, all within ± 30 °, at this moment need not carry out carrying platform 302 cervical region rotating adjust；

Then, according to step 5 of the present invention, known conditions is substituted into formula (17) and obtains:

α₁=17.398 °, α₂=-23.472 °, X_L=6854.7mm, β₁=19.461 °, β₂=-23.2214 °, X_R= 6924.4mm；

If above-mentioned result of calculation and known conditions being taken back formula (11) verify, it can be deduced that speaker's mouth phase For The Cloud Terrace base coordinate system O₀D coordinates value:

' P=[6000.1 1999.7 2999.8 1]^T

Understand ' P ≈ P, and error is in allowed band, demonstrates the correctness of angle method for solving of the present invention.

Finally, according to step 6 of the present invention, face rectangular peripheral profile of speaking center is at the video of left photographic head 201 Image coordinate system O_l-X_lY_lVideo image coordinate system O with right photographic head 201_r-X_rY_rIn two-dimensional coordinate value be respectively as follows: (520, 100), (130,110), be respectively as follows: with two-dimensional coordinate value (320, the 240) side-play amount of video image center

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{\ΔX}}_l=520-320\\ {\mathrm{\ΔY}}_l=100-240\end{array}\right.& \left\{\begin{array}{c}{\mathrm{\ΔX}}_r=130-320\\ {\mathrm{\ΔY}}_r=110-240\end{array}\right.\end{array}$

Photographic head focal length parameter is taken as: length=3000, above-mentioned condition is brought into formula (20)-(23) and show that each revolves Corner correction value:

Δα₁=3.81 °, Δ α₂=-2.61 °, Δ β₁=-3.62 °, Δ β₂=-2.48 °

After respective drive motor rotates above-mentioned correction angle, speaker face rectangular peripheral profile can be adjusted in image Heart position.

Embodiment 2: bionic eye cradle head structure design parameter is: P_z1=150mm, P_z2=120mm, P_y1=120mm, it is assumed that Run sound localization algorithm by controller module 500 and obtain speaker's mouth relative to The Cloud Terrace base coordinate system O₀Three-dimensional coordinate Value P=[3,000 3,000 3,000 1]^T；Obtain according to (12) formula and known conditions in step 3 of the present invention:

α₁=43.834 °, β₁=46.1267 °,

Visible, they are all beyond ± 30 ° of scopes, according to step 4 of the present invention, first have to enter carrying platform 302 cervical region Row rotates and adjusts, and known conditions substitution (16) formula can be tried to achieve: θ=45.0033 °, and control carrying platform 302 anglec of rotation θ= After 45.0033 °, make carrying platform 302 just to the mouth direction of speaker；Then, relatively current cervical region coordinate system O₁Enter Row step 5 computing, by P=[3,000 3,000 3,000 1]^T, θ=45.0033 ° and other known conditions substitution formula (17), : α₁=-1.6204 °, α₂=-32.7519 °, X_L=5046.5mm, β₁=1.6204 °, β₂=-32.7519 °, X_R= 5046.5mm, now, α₁=-1.6204 °, β₁=1.6204 °, all within ± 30 °；If by above-mentioned result of calculation and known Condition is taken back formula (11) and is verified, it can be deduced that speaker's mouth is relative to The Cloud Terrace base coordinate system O₀D coordinates value For:

' P=[3000.1 2999.8 2999.9 1]^T

Understand ' P ≈ P, and error is in allowed band, demonstrates the correctness of angle method for solving of the present invention.

Finally, according to step 6 of the present invention, face rectangular peripheral profile of speaking center is at the video of left photographic head 201 Image coordinate system O_l-X_lY_lVideo image coordinate system O with right photographic head 201_r-X_rY_rIn two-dimensional coordinate value be respectively as follows:

(400,340), (240,350), with two-dimensional coordinate value (320,240) the side-play amount difference of video image center For:

$\begin{array}{cc}\left\{\begin{array}{c}{\mathrm{\ΔX}}_l=400-320\\ {\mathrm{\ΔY}}_l=340-240\end{array}\right.& \left\{\begin{array}{c}{\mathrm{\ΔX}}_r=240-320\\ {\mathrm{\ΔY}}_r=350-240\end{array}\right.\end{array}$

Photographic head focal length parameter is taken as: length=3000, above-mentioned condition is brought into formula (20)-(23) and show that each revolves Corner correction value:

Δα₁=1.53 °, Δ α₂=1.90 °, Δ β₁=-1.53 °, Δ β₂=2.09 °

After respective drive motor rotates above-mentioned correction angle, speaker face rectangular peripheral profile can be adjusted in image Heart position.

The technology such as sound localization, recognition of face and visual processes and algorithm are known technology, are not described in detail at this, this Invention combines above-mentioned technology and the design of bionic eye cradle head structure and joint space derivation algorithm, it is achieved that bionic eye is to face of speaking Being accurately positioned and following the trail of of portion's rectangular peripheral profile.

Claims (3)

1. a bionic eye location tracking system, it is characterised in that it is mainly made up of following six parts:

Bionic eye The Cloud Terrace module, sound collection sensor array, camera module, The Cloud Terrace motion-control module, controller module And upper computer module；Wherein:

Described bionic eye The Cloud Terrace module includes: a The Cloud Terrace base, the sub-The Cloud Terrace of a carrying platform and two, left and right, wherein, It is connected, by carrying platform level on The Cloud Terrace base by a horizontal rotation joint between The Cloud Terrace base and carrying platform Rotate the neck rotation imitating bio-robot；Carrying platform is provided with two the sub-The Cloud Terraces in left and right, the sub-cloud in two, left and right Platform is respectively arranged with a horizontal rotating frame and a pitching swivel mount respectively, and so, every sub-The Cloud Terrace all has horizontal rotation and pitching Rotating two cradle heads, two photographic head are separately mounted on the pitching swivel mount in the middle of the sub-The Cloud Terrace in two, left and right, serve as imitative The eyes of raw eye, by the respective horizontal rotation of sub-The Cloud Terrace of two, left and right and the rotation of elevation rotary joint, can adjust respectively The photographic head being arranged on two the sub-The Cloud Terraces in left and right points to, and imitates the rotation of eyes；

Described sound collection sensor array serves as bionic ear in the present system, by mike identical for four characteristics, with imitative Centered by raw eye The Cloud Terrace base fulcrum, it is arranged on around The Cloud Terrace base center by the space geometry relation of positive tetrahedron, and by cloud Platform base props up dot center's initial point as bionic eye The Cloud Terrace base coordinate system, so, by the acoustical signal collected and sound source Location algorithm, obtains speaker's mouth D coordinates value relative to base coordinate system；

Described camera module is made up of two independent photographic head, is separately mounted to the pitching in the middle of the sub-The Cloud Terrace in two, left and right On swivel mount, serve as the eyes of bionic eye in the present system, be used for obtaining video information, and video data is passed to host computer mould Block；

Described The Cloud Terrace motion-control module includes: be separately mounted to left and right two sub-The Cloud Terraces on horizontal rotation drive motor and Pitching rotary drive motor, the horizontal rotation of a carrying platform drive motor and data transmission module；The Cloud Terrace motor control mould Tuber enters to the driving motor controlling corresponding joint according to each driving electric machine rotation signal of the controller module output received Row rotates；

Described controller module, on the one hand gathers the output signal of sound collection sensor array, runs sound localization algorithm, Obtain speaker's mouth D coordinates value relative to The Cloud Terrace base coordinate system, and the three-dimensional coordinate of this speaker's mouth is passed to Upper computer module；On the other hand receive control instruction and data message that upper computer module sends, control each and drive motor Rotate respective angles, the most also return the kinestate of system to upper computer module；

Described upper computer module includes: the eye level corner α of bionic eye The Cloud Terrace₁And β₁Derivation program, flat to carrying The derivation of two sub-The Cloud Terrace joint spaces about platform horizontal rotation the derivation program of rotational angle theta, bionic eye The Cloud Terrace module Program, vision processing algorithm program and the signal procedure with controller.

2. the method for work of a bionic eye location tracking system as claimed in claim 1, it is characterised in that mainly comprise as Lower step:

Step one, system initialization: system electrification, sound collection sensor array enters duty, and upper computer module shows The video image that camera module obtains, controller module controls all driving reset motors to initial angle；

Step 2, sound localization: controller module reads the voice data that sound collection sensor array gathers, and runs sound source Location algorithm obtains speaker's mouth D coordinates value relative to The Cloud Terrace base coordinate system, and this coordinate figure is passed to host computer mould Block；

Step 3, eye level corner solve: i.e. seek two sub-The Cloud Terrace each level angles about bionic eye The Cloud Terrace module Solve；

Upper computer module relative to the D coordinates value of The Cloud Terrace base coordinate system, is converted to speaker's mouth according to speaker's mouth Relative to the D coordinates value of cervical region coordinate system, when, under the non-rotation status of forward neck portion, solving two eyes and turn to respectively Level angle required during words people's mouth；If wherein the level angle of any one exceedes ± and 30 degree, at this moment need cervical region is entered Row rotates and adjusts, then turn to step 4；If the feathering angle of solve out two eyes is all not less than ± 30 degree, the most not Need to carry out cervical region rotating to adjust, then turn to step 5；

Step 4, neck joint is carried out rotate adjust: i.e. to carrying platform horizontal rotation corner solve；

Upper computer module first according to speaker's mouth relative to the D coordinates value of cervical region coordinate system, turn to meeting carrying platform Honest is condition towards speaker's mouth, and the anti-feathering angle solved needed for cervical region, the level of cervical region is revolved by upper computer module Corner issues controller module, controller module control the horizontal rotary drive motor of cervical region and rotate respective angles, it is achieved carry Platform turns to honest in the face of speaker's mouth direction, and so, cervical region coordinate system also turns to honest in the face of speaker's mouth direction, so Rear steering step 5, more relatively current cervical region coordinate system carries out the joint space of two sub-The Cloud Terraces about bionic eye The Cloud Terrace module Derivation, so that two eyes turn to speaker's mouth direction respectively；

About step 5, bionic eye The Cloud Terrace module, two sub-The Cloud Terrace joint spaces solve:

Upper computer module is carried out about bionic eye The Cloud Terrace module according to the relation of speaker's mouth relatively current cervical region coordinate system Two sub-The Cloud Terrace joint space derivations, the most anti-pitching rotary drive motor solving two the sub-The Cloud Terraces in left and right and level rotation Turn the angle information that driving motor should rotate, be then issued to controller module, by controller module control about two sub-clouds Platform respective pitching rotary drive motor and horizontal rotation drive electric machine rotation respective angles, preliminary to realize the eyes of bionic eye To the mouth of speaker；

So far, complete two photographic head are tentatively directed at the process of speaker's mouth；

Step 6, eyes adjustment in direction: i.e. two each rotary joint corners of sub-The Cloud Terrace about bionic eye The Cloud Terrace module are repaiied Just；

Upper computer module runs face recognition algorithms, obtains speaker face rectangular peripheral profile and asks for its center and exist Two-dimensional coordinate value in video image coordinate system, according to the side-play amount of this two-dimensional coordinate value Yu video image center, respectively Obtain pitching rotary drive motor and the correction value of the horizontal rotation driving motor anglec of rotation of two the sub-The Cloud Terraces in left and right, then will Above-mentioned anglec of rotation correction value issues controller module again, by controller module control about two respective pitching of sub-The Cloud Terrace Rotary drive motor and horizontal rotation drive motor fine setting respective angles, with realize to two eyes of bionic eye the most precisely towards The face mask center position of speaker so that speaker face rectangular profile center occurs in the video image of photographic head identification Center；

If step 7 controller module detects when new speaker occurs, then circulation perform step 2, step 3, step 4, Step 5 and step 6, carry out real-time tracking to emerging face of speaking；If not having new speaker to occur, then circulate execution Step 6 so that the corresponding degree of freedom of eyes adjusts according to the movement of face location of currently speaking；

Step 8, detection whole system the need of working on, the most then turn to step 7, it may be judged whether have new speaking People occurs, if it is not, shutdown of then deactivating.

3. method of work as claimed in claim 2, it is characterised in that the calculating side of the eyes adjustment in direction value that step 6 is carried Method step is as follows:

The most all angular units are all taken as: degree (°), all Parameter units relevant to video image are all taken as: pixel；

Step 6.1, upper computer module (600) read and show that the left photographic head (201) of bionic eye and right photographic head (202) obtain Video image, if picture traverse is W, height be H；

Step 6.2, upper computer module (600) run face recognition algorithms, respectively at left photographic head (201), right photographic head (202) In the video image identified, calculate the face rectangular peripheral profile center video image in left photographic head (201) of speaking Coordinate system O_l-X_lY_lVideo image coordinate system O with right photographic head (201)_r-X_rY_rIn two-dimensional coordinate value: (X_l, Y_l)、(X_r, Y_r)；

Step 6.3, take video image coordinate system O of left photographic head (201)_l-X_lY_lSit with the video image of right photographic head (201) Mark system O_r-X_rY_rCenter position coordinates value be: (W/2, H/2), then in the video image that left photographic head (201) identifies, say Two-dimensional coordinate value (the X of words face center_l, Y_l) and two-dimensional coordinate value (W/2, the H/2) skew of video image center Amount is:

$\left\{\begin{array}{c}{\mathrm{\ΔX}}_l=X_l-W/2\\ {\mathrm{\ΔY}}_l=Y_l-H/2\end{array}\right.$

In the video image that right photographic head (201) identifies, the two-dimensional coordinate value (X of speaker face center_r, Y_r) with regard Frequently two-dimensional coordinate value (W/2, the H/2) side-play amount of image center location is:

$\left\{\begin{array}{c}{\mathrm{\ΔX}}_r=X_r-W/2\\ {\mathrm{\ΔY}}_r=Y_r-H/2\end{array}\right.$

Step 6.4, ask for eyes corresponding freedom degree rotating angle correction according to side-play amount:

Left eye horizontal rotation freedom degree rotating angle correction value: Δ α₁=arctan (Δ X_l/length)；

Left eye pitching rotary freedom anglec of rotation correction value: Δ α₂=arctan (Δ Y_l/length)；

Right eye horizontal rotation freedom degree rotating angle correction value: Δ β₁=arctan (Δ X_r/length)；

Right eye pitching rotary freedom anglec of rotation correction value: Δ β₂=arctan (Δ Y_r/length)；

Wherein, length is the focal length parameter of photographic head, sets according to the actual parameter of photographic head；Anglec of rotation correction value Δ α₁、 Δα₂、Δβ₁、Δβ₂In all comprise positive and negative number two kinds of situations；

After respective drive motor rotates above-mentioned correction angle, thus speaker face rectangular peripheral profile is adjusted to the center of image Position.