CN109376621A

CN109376621A - A kind of sample data generation method, device and robot

Info

Publication number: CN109376621A
Application number: CN201811162694.3A
Authority: CN
Inventors: 孔祥晖; 秦林婵; 黄通兵
Original assignee: Beijing Qixin Yiwei Information Technology Co Ltd
Current assignee: Beijing 7Invensun Technology Co Ltd; Beijing Qixin Yiwei Information Technology Co Ltd
Priority date: 2018-09-30
Filing date: 2018-09-30
Publication date: 2019-02-22

Abstract

This application discloses a kind of sample data generation methods, image including acquiring corresponding user, the eyeball characteristic information of the user is extracted from the image of the corresponding user, the mood label of the user is determined by mood decision condition and the eyeball characteristic information, sample data is generated according to described image and the mood label, the sample data is for training human perceptual model.Since eyeball feature can objectively reflect the mood of user, the mood label of user can be thus determined based on the eyeball feature, to realize the automatic identification of user emotion, and the accuracy rate with higher when being identified to user emotion, therefore, the sample data of high quality can be generated according to the image of user and the higher mood label of accuracy rate.Disclosed herein as well is a kind of robots for the human perceptual model that sample data generating means and the built-in sample data training generated using this method are obtained.

Description

A kind of sample data generation method, device and robot

Technical field

This application involves but be not limited to technical field of data processing more particularly to a kind of sample data generation method, device And robot.

Background technique

With the development of artificial intelligence (Artificial Intelligence, AI), the robot based on AI meet the tendency of and It is raw.Current robot is mainly based upon voice realization and interacts with user, is based on this, and robot has the energy of " listening " and " saying " Power can listen attentively to the demand of user and make a response.In order to promote interactive experience, industry, which is proposed, is superimposed vision in fact with the sense of hearing The technical solution now interacted with user.By increasing visual capacity on the basis of the sense of hearing, robot identification user can be made to exist Mood when interaction, and then humanized service can be provided according to the mood of user.

Robot identifies what mood of the user in interaction was realized generally by human perceptual model.Human perceptual model It can be obtained by a large amount of sample data and machine learning algorithm training.Wherein, the quality of sample data directly affects The accuracy rate of human perceptual model.

Industry from internet mainly using big data is obtained at present, and then manually the mode of mark big data obtains sample Data.The experience of mark personnel is depended primarily on due to manually marking, and marks personnel to the visual perceptions information such as mood Judgment criteria disunity, so cause mark accuracy rate it is not high, greatly affected the quality of sample data.Therefore, it needs A kind of sample data generation method is provided to obtain the sample data of high quality.

Summary of the invention

In view of this, this application provides a kind of sample data generation method, this method can be mentioned according in user images The mood of the eyeball Automatic feature recognition user taken, and then generate the sample data of high quality.Accordingly, present invention also provides A kind of sample data generating means and robot.

The application first aspect provides a kind of sample data generation method, which comprises

Acquire the image of corresponding user；

The eyeball characteristic information of the corresponding user is extracted from the image of the corresponding user；

The mood label of the corresponding user is determined by mood decision condition and the eyeball characteristic information；

Sample data is generated according to the image of the corresponding user and the mood label, the sample data is for training Human perceptual model.

Optionally, the eyeball characteristic information includes pupil information；

Then the mood label that the user is determined by mood decision condition and the eyeball characteristic information includes:

Feelings by the determination of mood decision condition and the matched mood label of the pupil information, as the corresponding user Thread label.

Optionally, the method also includes:

The vision area-of-interest of the corresponding user is determined according to the image of the corresponding user and eyeball tracking algorithm；

It is then described to include: according to the image of the corresponding user and mood label generation sample data

Sample number is generated according to the image of the corresponding user, the mood label and the vision area-of-interest According to.

Optionally, the method also includes:

The corresponding eyeball optics characteristic information of image or the eyeball electricity that the corresponding user is obtained from eye movements system are special Reference breath, and determine that the visual impression of the corresponding user is emerging according to the eyeball optics characteristic information or eyeball electrical characteristic information Interesting region；

Optionally, the image for acquiring corresponding user includes:

Multiple images of user are corresponded in the acquisition time period；

Then the method also includes:

Determine the corresponding user in the eyeball of the time cycle according to multiple described images and eyeball tracking algorithm Rotation direction；

The psychological activity of the corresponding user is determined by the first psychological activity decision condition and the Rotation of eyeball direction Class label；

For every image in multiple described images, respectively according to every image, the corresponding mood label of every image And the psychological activity class label of the corresponding user generates sample data.

Optionally, the image for acquiring corresponding user includes:

Multiple images of user are corresponded in the acquisition time period；

Then the method also includes:

For every image in multiple described images, the corresponding eyelid of image of every corresponding user is extracted respectively Location information, the eyelid location information that the image of user is corresponded to according to every determine the corresponding user in week time The blink characteristic information of phase；

By the second psychological activity decision condition and the blink characteristic information, the psychological activity of the corresponding user is determined Class label；

Optionally, the image for acquiring corresponding user includes:

Multiple images of user are corresponded in the acquisition time period；

Then the method also includes:

For every image in multiple described images, the corresponding limbs of image of every corresponding user are extracted respectively Characteristic information；

Determine the corresponding limbs characteristic information of multiple images and the corresponding mood mark of multiple described images The incidence relation of label；

Determine that the habit label of the user, the habit label exist for characterizing the user according to the incidence relation Operate corresponding emotional state when certain limb action；

Then generating sample data according to the image of the corresponding user and the mood label includes:

For every image in multiple images of the corresponding user, respectively according to every image and every image Corresponding mood label and the habit label of the corresponding user generate sample data.

Optionally, the method also includes:

The face feature information and/or limbs feature letter of the corresponding user are extracted from the image of the corresponding user Breath；

According to the determination of the first mood decision condition and the matched first mood label of the eyeball characteristic information；

According to the determination of the second mood decision condition and the face feature information and/or limbs characteristic information matched second Mood label；

The mood label of the user is determined according to the first mood label and the second mood label.

Optionally, the image for acquiring corresponding user includes:

Image of the corresponding user under true interaction scenarios is acquired by camera.

The application second aspect provides a kind of sample data generating means, and described device includes:

Acquisition unit, for acquiring the image of corresponding user；

Extraction unit, for extracting the eyeball characteristic information of the corresponding user from the image of the corresponding user；

Determination unit, for determining the mood of the corresponding user with the eyeball characteristic information by mood decision condition Label；

Generation unit, for generating sample data, the sample according to the image of the corresponding user and the mood label Notebook data is for training human perceptual model.

The determination unit is specifically used for:

Optionally, the determination unit is also used to:

Then the generation unit is specifically used for:

Optionally, the determination unit is also used to:

Then the generation unit is specifically used for:

Optionally, the acquisition unit is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

The determination unit is also used to:

Then the generation unit is specifically used for:

Optionally, the acquisition unit is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

The extraction unit is also used to:

For every image in multiple described images, the corresponding eyelid of image of every corresponding user is extracted respectively Location information；

The determination unit is also used to:

Determine the user in the time cycle according to the eyelid location information of the image of corresponding user described in every Blink characteristic information；

Then the generation unit is specifically used for:

Optionally, the acquisition unit is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

Then the extraction unit is also used to:

The determination unit is also used to: determining the corresponding limbs characteristic information of multiple images and multiple described figures As the incidence relation of corresponding mood label；

Determine that the habit label of the corresponding user, the habit label are described right for characterizing according to the incidence relation Using family when operating certain limb action corresponding emotional state；

Then the generation unit is specifically used for:

For every image in multiple images of the corresponding user, respectively according to every image and every image Corresponding mood label and the habit label of the user generate sample data.

Optionally, the extraction unit is also used to:

The face feature information and/or limbs characteristic information of the user are extracted from described image；

Then the determination unit is specifically used for:

Optionally, the acquisition unit is specifically used for:

The application third aspect provides a kind of robot, and the robot includes processor and memory:

The memory is used to store the program code of human perceptual model, and said program code is transferred to the place Manage device；The human perceptual model is the sample data generated using sample data generation method described in the application first aspect The model that training obtains, identifies the mood of the user when for interacting with user；

The processor is used to run the human perceptual model according to said program code, to realize with the user's Interaction.

As can be seen from the above technical solutions, the embodiment of the present application has the advantage that

The embodiment of the present application provides a kind of sample data generation method, and this method is realized based on image processing techniques , the image of corresponding user is acquired first, and corresponding user is then extracted from the image of corresponding user by image processing techniques Eyeball feature, since eyeball feature can objectively reflect the mood of user, it is thus possible to based on the eyeball feature determine pair Using the mood label at family, to realize the automatic identification of user emotion, and have when being identified to user emotion compared with Therefore high accuracy rate the sample of high quality can be generated according to the image of corresponding user and the higher mood label of accuracy rate Notebook data.

Detailed description of the invention

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of application without any creative labor, may be used also for those of ordinary skill in the art To obtain other drawings based on these drawings.

Fig. 1 is a kind of schematic diagram of a scenario of sample data generation method in the embodiment of the present application；

Fig. 2 is a kind of flow chart of sample data generation method in the embodiment of the present application；

Fig. 3 is a kind of flow chart of sample data generation method in the embodiment of the present application；

Fig. 4 is a kind of flow chart of sample data generation method in the embodiment of the present application；

Fig. 5 is a kind of flow chart of sample data generation method in the embodiment of the present application；

Fig. 6 is a kind of flow chart of sample data generation method in the embodiment of the present application；

Fig. 7 is a kind of structural schematic diagram of sample data generating means in the embodiment of the present application.

Specific embodiment

In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application Attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only this Apply for a part of the embodiment, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art exist Every other embodiment obtained under the premise of creative work is not made, shall fall in the protection scope of this application.

The description and claims of this application and term " first ", " second ", " third ", " in above-mentioned attached drawing The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage The data that solution uses in this way are interchangeable under appropriate circumstances, so that embodiments herein described herein for example can be to remove Sequence other than those of illustrating or describe herein is implemented.In addition, term " includes " and " having " and theirs is any Deformation, it is intended that cover it is non-exclusive include, for example, containing the process, method of a series of steps or units, system, production Product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include be not clearly listed or for this A little process, methods, the other step or units of product or equipment inherently.

This technical problem is trained for the second-rate influence human perceptual model of sample data obtained in the prior art, The application is based on image procossing principle and proposes a kind of sample data generation method, acquires the image of corresponding user first, then The eyeball feature for extracting corresponding user from the image of corresponding user by image processing techniques, since the eyeball feature being capable of visitor See the mood of ground reflection user, it is thus possible to the mood label that corresponding user is determined based on the eyeball feature, to realize user The automatic identification of mood, and the accuracy rate with higher when being identified to user emotion, therefore, according to corresponding user's The sample data of high quality can be generated in image and the higher mood label of accuracy rate.

It is appreciated that may be implemented to automatically generate the sample data of high quality by this method, sample data instruction is utilized When practicing human perceptual model, enable to human perceptual model accuracy rate with higher, also, due to sample data quality compared with Height, interference content is less, reduces trained difficulty, improves training effectiveness, shortens cycle of training.

Sample data generation method provided by the present application can be applied to have in the processing equipment of graphics capability.Place Reason equipment can be terminal device or server with graphics processor (Graphics Processing Unit, GPU).Its In, terminal device can be personal computer (personal computer, PC), work station or robot, mobile phone etc..For It is easy to understand, is illustrated hereinafter with robot, do not constitute the restriction to technical scheme.

Sample data generation method provided by the present application can be stored in processing equipment in the form of application program, had When body is realized, processing equipment realizes sample data generation method by executing application.It should be noted that application program It can be independent application program, be also possible to be integrated in functional module or plug-in unit in other applications, the application is to this It is not construed as limiting.

To keep the technical solution of the application clearer, the embodiment of the present application is mentioned below with reference to the application scenarios of robot The sample data generation method of confession is introduced.

The scene framework figure of sample data generation method shown in Figure 1 includes robot 10 in the application scenarios With user 20, robot 10 is shot user 20 by the camera of itself, when interacting with user 20 to acquire user Then 20 image extracts the eyeball characteristic information of user 20 from the image of user 20, passes through mood decision condition and eyeball Characteristic information can determine the mood label of user 20, generate sample number according to the image of user 20 and corresponding mood label According to the sample data can be used for training human perceptual model.

Wherein, the eyeball feature based on user is to the Emotion identification of user accuracy rate with higher, thus sample data In mood label can accurately identify the mood of the corresponding user of image, quality with higher, by the sample of high quality Notebook data can be improved the accuracy rate of human perceptual model, and reduce trained difficulty, improve for training human perceptual model Training effectiveness shortens cycle of training.

Next, describing in detail in conjunction with attached drawing to sample data generation method provided by the embodiments of the present application.Referring to The flow chart of sample data generation method shown in Fig. 2, this method comprises:

S201: the image of corresponding user is acquired.

For robot, visual ability can be made it have by training human perceptual model, be based on the view Feel that sensing capability may be implemented to identify the mood etc. of user.For this purpose, robot needs to obtain sample data, for training Human perceptual model.In specific implementation, robot acquires the image of corresponding user first, so as to according to the image of corresponding user Generate sample data.Wherein, corresponding user refers to the user for specifying and interacting in interaction scenarios.It is more when existing in interaction scenarios When a user, robot can be interacted by human perceptual model with designated user namely corresponding user.For this purpose, robot Need to acquire the image of corresponding user in advance, to train human perceptual model.

In some possible implementations, robot can acquire the corresponding user by camera and really interact Image under scene.With the artificial example of family's house keeper's machine, house keeper robot of family can when being interacted with user, by itself Camera acquires image of the user under the true interaction scenarios.

S202: the eyeball characteristic information of the corresponding user is extracted from the image of the corresponding user.

Eyes are an important carriers for reflecting mood.When user is in different emotional states, the state of eyes is often Different, especially there are biggish differences for the state of eyeball.In order to identify the mood of user, figure of the robot from corresponding user The eyeball characteristic information of corresponding user is extracted as in.

Wherein, eyeball can be divided into wall of eyeball and content two parts, and wall of eyeball includes outer layer fiber film, middle layer tunica vasculose And inner retina, include cornea in tunica fibrosa, includes iris in tunica vasculose, the aperture among iris forms pupil.Eyeball Characteristic information may include pupil information, and as the specific example of the application, pupil information may include pupil size.? When specific implementation, robot can extract the eyeball characteristic information of user by image processing techniques from image.It is some can In the implementation of energy, the image of user can be converted to gray level image by robot, and it is thick to be then based on small gray value clustering procedure Iris center is positioned, then determines pupil profile based on edge extraction techniques, pupil size, machine can be determined according to pupil profile People can be using pupil size as eyeball characteristic information.In another possible implementation, robot can also to image into Row binaryzation realizes the preliminary extraction to pupil, then transports substantially with burn into expansion, the opening and closing operation etc. in mathematical morphology Factor removal will be blocked in pupil gray level image by calculating combining form reconstructing method, then remove noise spot pair by smothing filtering The interference of pupil can determine that pupil size, robot can be by Ruler for measuring pupil after extracting pupil region according to pupil region It is very little to be used as eyeball characteristic information.

S203: the mood label of the user is determined by mood decision condition and the eyeball characteristic information.

Mood decision condition characterizes the corresponding relationship between eyeball characteristic information and mood.Mood decision condition specifically may be used To show as the ternary mapping relations of " eyeball characteristic parameter ", " parameter value range ", " mood label ", for example, if " eyeball is special Sign parameter " falls into one " parameter value range ", then can determine that one " mood label ", the mood label are and the eyeball feature The mood label of information matches.

Based on this, robot can and the eyeball characteristic information matched mood mark determining according to mood decision condition Label, then determine the mood label of the corresponding user according to the mood label.It, can be directly by the mood when specific implementation Mood label of the label as the user, can also be by the mood label in conjunction with the mood label obtained by other means With the mood label of the determination user.

As the specific example of the application, eyeball characteristic information can be pupil information.If eyeball characteristic information is Pupil information, then robot can be by the determination of mood decision condition and the matched mood label of the pupil information, as correspondence The mood label of user.In this example, eyeball characteristic information is pupil information, and eyeball characteristic parameter can be pupil size, First mood decision condition is represented by " pupil size ", " pupil size value range ", " mood label " ternary mapping pass System, if the first mood decision condition is, pupil size is greater than first threshold, then corresponding mood label is happiness, and pupil size is small In second threshold, then corresponding mood label is sorrow, then robot is after the pupil size for extracting user in image, Ke Yigen According to the corresponding relationship of pupil size and mood, the determining and matched mood label of current pupil size is then based on and current pupil The matched mood label of pore size determines the mood label of user.

Wherein, first threshold and second threshold can be arranged based on experience value, and in specific implementation, robot can collect Pupil size of the user in a period of time generally can be than being in normal emotional state when user's excitation time pupil obviously amplifies When quadruplication, and when user's anger, pupil is obviously reduced, be based on this, can will in the pupil size of collection obviously amplification or The data removal being obviously reduced, then the average value of pupil size is calculated, using the average value as a reference value of pupil size, then Based on a reference value, first threshold and second threshold are set.For example, first threshold can be used as by four times of a reference value, by benchmark The half of value is as second threshold.It is to be appreciated that different user or user can be in a reference value in different time periods Different, therefore, first threshold and second threshold can adaptively be adjusted according to the pupil size data of acquisition.

S204: sample data is generated according to the image of the corresponding user and the mood label, the sample data is used In training human perceptual model.

In specific implementation, robot is raw according to the image of the correspondence user of acquisition and mood label corresponding with image At sample data, which is used to train human perceptual model, then can use machine learning algorithm and sample number According to mood label the parameter of human perceptual model is optimized so that human perceptual model prediction mood label and sample The mood label of data is consistent.

In the present embodiment, mood label can be indicated by way of vector, then generated according to image and mood label Sample data can be expressed as the form of " picture " and " one-dimensional label vector ", wherein the one-dimensional label vector characterizes user Mood.

From the foregoing, it will be observed that the embodiment of the present application provides a kind of sample data generation method, this method is based on image procossing What technology was realized, the image of corresponding user is acquired first, is then extracted from the image of corresponding user by image processing techniques The eyeball feature of corresponding user, since eyeball feature can objectively reflect the mood of user, it is thus possible to special based on the eyeball Sign determines the mood label of corresponding user, to realize the automatic identification of user emotion, and identifies to user emotion When accuracy rate with higher height therefore can be generated according to the image of corresponding user and the higher mood label of accuracy rate The sample data of quality.

Embodiment illustrated in fig. 2 is mainly based upon the mood of eyeball characteristic information identification user, to generate the sample of high quality Notebook data is also based on the carrier such as face or limbs letter of other expression moods to improve the accuracy of Emotion identification Cease the mood for determining user.

A kind of implementation of sample data generation method provided by the embodiments of the present application is described in detail below, is joined See Fig. 3, this method comprises:

S301: the image of corresponding user is acquired.

S302: the eyeball characteristic information of the corresponding user is extracted from the image of the corresponding user.

S303: according to the determination of the first mood decision condition and the matched first mood label of the eyeball characteristic information.

First mood decision condition characterizes the corresponding relationship between eyeball characteristic information and mood.First mood determines item Part can specifically show as the ternary mapping relations of " eyeball characteristic parameter ", " parameter value range ", " mood label ", for example, If " eyeball characteristic parameter " falls into one " parameter value range ", can determine one " mood label ", the mood label be with The matched mood label of the eyeball characteristic information.Based on this, robot can according to the first mood decision condition it is determining with it is described The matched first mood label of eyeball characteristic information.S304: extract the corresponding user's from the image of the corresponding user Face feature information and/or limbs characteristic information.

For user under different emotional states, facial state is typically also different.For example, user is in positive, glad Emotional state when, the corners of the mouth can raise up, and when in passive, sad emotional state, eyelid is micro- to be closed, mouth sagging, in this way, also The mood of user can be determined based on facial state.Similar, user can unconsciously implement corresponding under different emotional states Limb action.For example, when user is in the emotional state of indignation movement of clenching fist can be implemented, in this way, being also based on limbs shape State determines the mood of user.

In specific implementation, robot extracted from the image of corresponding user corresponding user face feature information and/or Limbs characteristic information corresponds to the mood of user for identification.Wherein, robot extract corresponding user face feature information and/ Or limbs characteristic information can be by the way of similar with eyeball characteristic information is extracted, for example, converting the image into binary picture Picture or gray level image, then obtain face feature information and/or limbs characteristic information by way of edge extracting.

S305: it is determined according to the second mood decision condition and is matched with the face feature information and/or limbs characteristic information The second mood label.

Second mood decision condition characterizes at least one of face feature information and limbs characteristic information between mood Corresponding relationship.In specific implementation, if robot extracts face feature information from image, facial characteristics can be believed Breath matched with the second mood decision condition, obtain with the matched second mood label of face feature information, if robot from Limbs characteristic information is extracted in image, then limbs characteristic information can be matched with the second mood decision condition, be obtained With the matched second mood label of limbs characteristic information, if robot extracts face feature information and limbs feature from image Information, then it is available with the matched second mood label of face feature information and with matched second feelings of limbs characteristic information The quantity of thread label namely the second mood label can be two.

It should be noted that the execution sequence of S302 and S304 can be arbitrary, such as it can be and be performed simultaneously, It can be and successively executed according to the sequence of setting, and S303 and S305 are executed after S302 and S304 respectively.

S306: the mood mark of the corresponding user is determined according to the first mood label and the second mood label Label.

First mood label is the mood label determined based on eyeball characteristic information, and the second mood label is based on facial special The mood label that reference breath and/or limbs characteristic information determine, robot can be according to the first mood label and the second mood mark Label determine the mood label of corresponding user.

Robot determines the mood label of corresponding user according to the first mood label and the second mood label, specifically can be The mood label of corresponding user is determined according to the first mood label, the second mood label and decision plan.Wherein, decision plan It can be arranged according to actual needs, a specific example as the application can be only in order to improve Emotion identification accuracy rate When the first mood label is consistent with the second mood label, using the first mood label as the mood label of corresponding user；As Another specific example of the application can will use voting mechanism by the first feelings when the quantity of the second mood label is two Mood label of the higher mood label of poll as corresponding user in thread label and the second mood label, for example, the first feelings Thread label is " happiness ", and two the second mood labels respectively " are liked " and " sorrow ", then is based on above-mentioned strategy, can be by " happiness " conduct pair Using the mood label at family.

S307: sample data is generated according to the image of the corresponding user and the mood label, the sample data is used In training human perceptual model.

The specific implementation of this step may refer to the description of embodiment illustrated in fig. 2 related content, and details are not described herein.

From the foregoing, it will be observed that the embodiment of the present application provides a kind of sample data generation method, corresponding user is acquired in this method Image, the eyeball characteristic information of corresponding user is not only extracted from image, also extract corresponding user face feature information and/ Or limbs characteristic information, in this way, can be special based at least one of face feature information and limbs characteristic information and eyeball Reference ceases the mood label for determining user, which is capable of the mood of relatively accurately identity user, therefore, is based on user Image and corresponding mood label generate sample data quality with higher.

When interacting with user in addition to the mood of identification user, the vision area-of-interest of user can also be determined, in this way, Be conducive to robot to be the corresponding content of user's recommendation according to the vision area-of-interest of user or give other responses, mention High user experience.For this purpose, the label of vision area-of-interest can be increased, in sample data to train human perceptual model to know The ability of the vision area-of-interest of other user.

Wherein, vision area-of-interest is to refer to generate powerful stimulation to human visual system, and information content is great Region.Vision area-of-interest can be determined by a variety of implementations, can substantially be classified as contact and contactless two Class will be described in detail respectively below.

Contact mode determines that vision area-of-interest includes electroculogram, iris-corneoscleral limbus, double purkinje images and connects Touch the several ways such as mirror, wherein electroculogram is to record by placing a pair of electrodes on skin in inside and outside give a tongue-lashing in Rotation of eyeball Its potential value, so that it is determined that vision area-of-interest.Iris-corneoscleral limbus method is also referred to as infrared electro bounce technique, is by eye Two infrared ray photosensitive tubes are nearby installed in portion, with sightless Infrared irradiation eye, two at iris and corneoscleral limbus or so The infrared light for dividing reflection, is received by two infrared photodiodes respectively.When eyeball moves to the left or to the right, two photosensitive tubes are connect The infrared ray of receipts can change, and can measure eye movement using the differential signal, to obtain the vision area-of-interest of user. Infrared electro bounce technique can be realized by infrared camera.Purkinje image is formed by several optical interfaces reflection of eyes Image, direction of gaze and blinkpunkt can be determined by the measurement to two purkinje images, and then determine the view of user Feel area-of-interest.Contact lense rule is that one piece of reflecting mirror is fixed on cornea or sclera, by fixed beam when eye movement It is reflected into different directions, to obtain eye movement signal, the vision area-of-interest of user can be determined according to the eye movement signal.

Contactless mode determines that vision area-of-interest includes several sides such as pupil-corneal reflection vector, corneal reflection Formula.Specifically, cornea can reflect the light fallen on, and when eye movement, light is mapped to cornea with the angle changed, obtain difference Direction it is reflective, using anterior corneal surface formed dotted line moved, the position of the virtual image in real-time detection image with the rotation of eyeball, Eye movement signal can be obtained after signal processing, so realize and vision area-of-interest is determined by corneal reflection mode.Pupil- Corneal reflection vector is then to obtain eyeball image by fixed eye camera to extract eye using bright pupil hole and dark pupil principle Pupil in ball image makees corneal reflection point data using the relative position of corneal reflection method correction eye camera and eyeball For the basic point of the relative position of eye camera and eyeball, pupil center location coordinate is the blinkpunkt for indicating user.It needs to illustrate , eye camera can be controlled by microelectromechanical-systems (Micro-Electro-Mechanical System, MEMS), with Realize visual pursuit.

Based on the specific implementation of above-mentioned determining vision area-of-interest, present invention also provides the generations of sample data A kind of specific implementation of method participates in Fig. 4, this method comprises:

S401: the image of corresponding user is acquired.

S402: the eyeball characteristic information of the corresponding user is extracted from the image of the corresponding user.

S403: the mood label of the user is determined by mood decision condition and the eyeball characteristic information.

The specific implementation of S401 to S403 may refer to the description of embodiment illustrated in fig. 2 related content, and details are not described herein.

S404: determine that the vision of the corresponding user is interested according to the image of the corresponding user and eyeball tracking algorithm Region.

Robot extracts the eyeballs characteristic informations such as pupil information, cornea information from the image of the correspondence user of acquisition, Then the pupil information of extraction and cornea information etc. are inputted into eyeball tracking algorithm to determine the vision area-of-interest of user.Its In, eyeball tracking algorithm can be pupil-corneal reflection method, and pupil information may include the information such as pupil size, pupil position At least one of, cornea information may include hot spot reflective information, in order to determine vision area-of-interest, in some situations Under, iris information can also be obtained, then chases after at least one of pupil information, iris information and cornea information input eyeball Track algorithm, to determine the vision area-of-interest of user.

In order to make it easy to understand, being illustrated in conjunction with specific example.Firstly, robot control camera shoots user's face figure Then it is special to extract the eyeballs such as pupil information, cornea information using image processing algorithm by picture, image resolution ratio 1920*1080p Reference breath, pupil information, cornea information are embodied in the coordinate data (x, y) in image, and robot recycles eyeball tracking Algorithm calculates blinkpunkt information, and the point data of watching attentively of output is the coordinate data (x1, y1) on display screen screen, or is used Coordinate data (x2, y2, z2) in the visual field of family, the vision area-of-interest of user can be determined based on above-mentioned data.

Wherein, a bit in (x1, y1) characterization display this plane of screen, which can be the display of robot Screen, robot can determine the corresponding uniform resource locator of display content (the Uniform Resource on display screen Locator, URL), be based on the URL, can determine the keyword of the current attentinal contents of user, in this way, make robot more in order to Solve user；(x2, y2, z2) then characterizes a bit in space, based on the spatial model constructed in advance, can determine that the point is corresponding Object, in this way, robot can determine user concern spatial object.

The execution sequence of S402 and S404 is not construed as limiting, and may be performed simultaneously, can also be according to actual needs according to setting Sequence execute.

S405: sample is generated according to the image of the corresponding user, the mood label and the vision area-of-interest Notebook data.

In specific implementation, robot according to the image of the correspondence user of acquisition, mood label corresponding with image with And vision area-of-interest generates sample data, which is used to train human perceptual model, then can use machine Learning algorithm and the mood label of sample data, vision area-of-interest optimize the parameter of human perceptual model, make Mood label, the vision area-of-interest for obtaining human perceptual model prediction are emerging with mood label, the visual impression of sample data respectively Interesting region is consistent.In this way, when interacting with user, the mood of human perceptual model identification user and interested can use Region.

In the present embodiment, vision area-of-interest can use similar with mood label, psychological activity class label Mode indicates, i.e., is indicated by way of vector, in this way, raw according to image, mood label and vision area-of-interest At sample data can be expressed as the form of " picture " and " two-dimensional tag vector ", wherein two-dimensional tag vector table requisition The mood and vision area-of-interest at family.

From the foregoing, it will be observed that the embodiment of the present application provides a kind of sample data generation method, this method acquires corresponding user's Then image extracts the eyeball feature of corresponding user by image processing techniques from the image of corresponding user, due to eyeball spy Sign can objectively reflect the mood of user, it is thus possible to the mood label of corresponding user is determined based on the eyeball feature；It is based on The image of corresponding user can also determine the vision area-of-interest of corresponding user, according to the image of corresponding user, corresponding feelings The sample data of high quality can be generated in thread label and vision area-of-interest, and the sample data is from multiple dimensions to application Family is labeled, and is trained obtained human perceptual model that can not only perceive the mood of corresponding user based on the sample data, is gone back Can perceive the psychological activity type of corresponding user, thus can mood based on user and psychological activity type make preferably Response.

Embodiment illustrated in fig. 4 is the vision region of interest that corresponding user is determined using the image of the correspondence user acquired It domain can also be according to the eye movements system for being exclusively used in eye movement tracking in the embodiment of the present application in other possible implementations Determine the vision area-of-interest of corresponding user.Specifically it may refer to Fig. 5, this method comprises:

S501: the image of corresponding user is acquired.

S502: the eyeball characteristic information of the corresponding user is extracted from the image of the corresponding user.

S503: the mood label of the user is determined by mood decision condition and the eyeball characteristic information.

The specific implementation of S501 to S503 may refer to the description of embodiment illustrated in fig. 2 related content, and details are not described herein.

S504: the corresponding eyeball optics characteristic information of image or eyeball electricity of the corresponding user are obtained from eye movements system Characteristic information is learned, and determines the vision of the corresponding user according to the eyeball optics characteristic information or eyeball electrical characteristic information Area-of-interest.

Wherein, the eye movements system includes microelectromechanical-systems, the eye movement tracing system based on infrared camera, based on connecing Touch the eye movement tracing system of mirror or the eye movement tracing system based on detection electrode.Different eye movements systems realize eyeball fortune The principle of dynamic tracking is different.For example, the eye movement tracing system based on infrared camera is to realize eyeball by infrared ray Motion tracking, it is infrared that robot can obtain the reflections such as the cornea of eyeball from the eye movement tracing system based on infrared camera The eyeball optics characteristic information such as direction of line determines the vision area-of-interest of user.In another example the eyeball based on detection electrode Motion tracking system is to realize that eye movement is tracked by being embedded in the current potential that the detection electrode of eyeball detects, therefore, machine The eyes such as electroculogram of current potential when device people can obtain record Rotation of eyeball from the eye movement tracing system based on detection electrode Then ball electrical characteristic information determines the vision area-of-interest of user according to the eyeball electrical characteristic information.

It should be noted that eyeball optics characteristic information is not limited to the direction of infrared ray, it can also be including corneal reflection etc. Information, eyeball electrical characteristic information can also be the information such as electric current or capacitor, and the present embodiment is not construed as limiting this.

The execution sequence of S502 and S504 is not construed as limiting, and may be performed simultaneously, can also be according to actual needs according to setting Sequence execute.

S505: sample is generated according to the image of the corresponding user, the mood label and the vision area-of-interest Notebook data.

The specific implementation of this step can participate in the description of S405 related content, and details are not described herein.

It is appreciated that robot when interacting with user, can not only identify the mood of user by human perceptual model, It can also identify the current psychological activity type of user, determine that the current psychological activity type of user is that vision is recalled or vision is thought As in this way, robot can provide humanized service according to the mood and psychological activity type of user.

With reference to the accompanying drawing, it is situated between to a kind of implementation of sample data generation method provided by the embodiments of the present application It continues, referring to Fig. 6, this method comprises:

S601: multiple images of user are corresponded in the acquisition time period.

In specific implementation, robot can be to correspond to multiple images of user in continuous acquisition a period of time.Wherein, when Between the period can be arranged according to actual needs, for example, it is desired to when being identified to the current psychological activity type of user, can with point Clock acquires multiple images of user in the time cycle as the time cycle.

S602: for every image in multiple described images, the eyeball characteristic information of the user is extracted respectively.

Robot can use mode identical with embodiment illustrated in fig. 2, extract the eyeball of user respectively to every image Characteristic information, in this way, robot can obtain the corresponding eyeball characteristic information of multiple images.

S603: the corresponding mood mark of every image is determined according to the first mood decision condition and the eyeball characteristic information Label.

In specific implementation, robot can use identical processing mode to every image, with every image pair of determination The mood label answered.By taking an image as an example, robot can be carried out the eyeball characteristic information and the first mood decision condition Matching, obtain with the matched first mood label of the eyeball characteristic information, as the corresponding mood label of the image.By multiple Each Zhang Jun of image is handled in a manner described, available mood label corresponding with each image.

S604: determine the corresponding user in the time cycle according to multiple described images and eyeball tracking algorithm Rotation of eyeball direction.

Multiple images are image of the user within the time cycle, by using eyeball tracking algorithm to the eyeball of user Movement is tracked, and user can be determined in the Rotation of eyeball direction of the time cycle.In specific implementation, robot can be with Whether rotated according to the eyeball of a variety of image recognition users, then determines user in the Rotation of eyeball of the time cycle again Direction.

In some possible implementations, for every image in multiple described images, robot can be according to eye Ball tracing algorithm determines the corresponding user's direction of gaze of every image respectively, wherein determining that the specific implementation of user's direction of gaze can With embodiment related content shown in Figure 4 description.Then, robot can be according to the corresponding user side of watching attentively of every image To judging whether the user within the time cycle occurs Rotation of eyeball.Specifically, user's direction of gaze can pass through angle Degree indicates that robot can judge user whether at this according to the angle change of the corresponding user's direction of gaze of multiple images Between Rotation of eyeball occurs in the period.For example, in multiple images the corresponding user's direction of gaze of any two images angle change Greater than angle change threshold value, then judge that within the time cycle Rotation of eyeball occurs for user really.

When robot judges that within the time cycle Rotation of eyeball occurs for user, then can respectively be corresponded to based on multiple images Pupil position determine Rotation of eyeball direction.Specifically, for every image in multiple described images, every figure is determined respectively As corresponding pupil position；Then according to the corresponding pupil position of every image, the Rotation of eyeball direction of the user is determined.Example Such as, upper right side of the forward corresponding pupil position of image of a certain timing for eyeball, the figure of a certain timing rearward in multiple images As the upper left side that corresponding pupil position is eyeball, then show that Rotation of eyeball direction of the user within the time cycle is upper left Side.

S605: the psychology of the corresponding user is determined by the first psychological activity decision condition and the Rotation of eyeball direction Class of activity label.

Psychological activity refers to the thinking that people is carried out before carrying out the activity such as language, behavior, expression.In general, psychological Activity can be divided into following several classifications, i.e., in the vision imagination, vision are recalled, touched and be in a bad mood by emotion or body Heart dialogue etc., is based on this, psychological activity class label include the vision imagination, vision recall, touched by emotion or body and Several class labels such as the heart dialogue being in a bad mood.

First psychological activity decision condition characterizes the corresponding relationship between Rotation of eyeball direction and psychological activity classification.The One psychological activity decision condition can specifically show as this characteristic parameter of Rotation of eyeball direction, characteristic ginseng value, psychological activity The ternary mapping relations of class label.In specific implementation, the first psychology can be arranged by conditional statement as described below to live Dynamic decision condition can determine " a psychological activity if it is a certain " direction " that the conditional statement, which can be " Rotation of eyeball direction ", Class label ".

In order to make it easy to understand, illustrating.For example, the first psychological activity class label can be, if Rotation of eyeball direction Turn for upper right, it is determined that psychological activity class label is the vision imagination；If Rotation of eyeball direction turns for bottom right, it is determined that psychology is living Dynamic class label is to be touched by emotion touch or body；If Rotation of eyeball direction is supralevoversion, it is determined that psychological activity classification Label is recalled for vision；If Rotation of eyeball direction turns for lower-left, it is determined that psychological activity class label is the heart pair being in a bad mood Words.

In specific implementation, the corresponding relationship that robot is characterized according to the psychological activity decision condition, determining and current eye The matched psychological activity class label of ball rotation direction, the psychological activity class label are the psychological activity classification mark of user Label.Further, robot can be with Rotation of eyeball frequency of the counting user within the time cycle, if user is in a direction Rotation of eyeball frequency reach the first predeterminated frequency, then can determine that user is in nervous, the uneasy or state of alert.

As the extension of the present embodiment, robot is in addition to can be true based on Rotation of eyeball direction of the user within the time cycle The mood label for determining user is also based on the psychological activity class that blink characteristic information of the user within the time cycle determines user Distinguishing label.Wherein, blink characteristic information refers to the information of characterization user's blink state, is specifically as follows frequency of wink and/or blinks It at the moment grows, frequency of wink refers to that number of winks of the user within the unit time, blink duration can be understood as after primary open eyes It quickly closes one's eyes and arrives the time opened eyes next time again.Next to the psychological activity classification mark for determining user based on blink characteristic information Label are described in detail.

In specific implementation, within the acquisition time period after multiple images of user, for every figure in multiple images Picture, robot can extract the corresponding eyelid location information of image respectively, wherein eyelid location information can pass through coordinates table Show, then according to the eyelid location information of every image determine the user in the blink characteristic information of the time cycle, such as Frequency of wink and blink duration, robot can determine user according to the second psychological activity decision condition and blink characteristic information Psychological activity class label.

Wherein, the second psychological activity decision condition characterizes the corresponding pass between blink characteristic information and psychological activity classification System.First psychological activity decision condition can specifically show as blink characteristic parameter, characteristic parameter value range, psychological activity class The ternary mapping relations of distinguishing label.In specific implementation, the second psychological activity can be arranged by conditional statement as described below Decision condition, if the value that the conditional statement can be a certain " blink characteristic parameter " falls into a certain " characteristic parameter value model Enclose ", it is determined that a certain " psychological activity class label ".Wherein, psychological activity class label in addition to vision described above recalls, The labels such as the vision imagination, can also include anxiety, be intended to prevent at the moment things enter the psychological activity class label in the visual field.

As the specific example of the application, the second psychological activity decision condition can be, if user is in the time The frequency of wink in period is greater than the second predeterminated frequency, it is determined that psychological activity class label is anxiety, if user is in the time The blink duration in period is greater than preset duration, it is determined that psychological activity class label is that things enters the visual field at the moment for intention prevention. Wherein, the first predeterminated frequency, the second predeterminated frequency and preset duration can be arranged based on experience value.

According to the picture of the user of robot acquisition whithin a period of time, the mobile speed of the eyes of user can also be determined Degree, is denoted as saccadic speed, that reflects the transfers of blinkpunkt.Saccadic speed based on user can determine the current feelings of user Thread and/or psychological activity type.

Wherein, the execution sequence of S602 and S604 can be arranged according to actual needs, for example, may be performed simultaneously, It can be executed according to preset sequencing, S603 and S605 is executed after S602 and S604 respectively, then executes S606.

S606: for every image in multiple described images, respectively according to every image, the corresponding mood of every image Label and the psychological activity class label of the corresponding user generate sample data.

In specific implementation, for every image in multiple images, robot is corresponding according to every image, every image Mood label and the psychological activity class label of corresponding user generate sample data, in this way, available multiple sample numbers According to.The sample data is used to train human perceptual model, then it can be using machine learning algorithm and the mood of sample data Label, psychological activity class label optimize the parameter of human perceptual model, so that the mood of human perceptual model prediction Label, psychological activity class label are consistent with the mood label of sample data, psychological activity class label respectively.

In the present embodiment, psychological activity class label can be indicated using mode similar with mood label, that is, be passed through The form of vector is indicated, in this way, can according to the sample data that image, mood label and psychological activity class label generate To be expressed as the form of " picture " and " two-dimensional tag vector ", wherein the mood and the heart at two-dimensional tag vector table requisition family Manage Activity Type.

From the foregoing, it will be observed that the embodiment of the present application provides a kind of sample data generation method, this method acquires corresponding user's Then image extracts the eyeball feature of corresponding user by image processing techniques from the image of corresponding user, due to eyeball spy Sign can objectively reflect the mood of user, it is thus possible to the mood label of corresponding user is determined based on the eyeball feature；It is based on The image of corresponding user can also determine that the rotation direction of eyeball, the rotation direction of eyeball characterize the psychological activity class of user Type, and hence it is also possible to the psychological activity class label of corresponding user be determined, according to image, the corresponding mood mark of corresponding user Label and psychological activity class label psychological activity class label can be generated the sample data of high quality, and the sample data is from more A dimension is labeled corresponding user, and the human perceptual model obtained based on sample data training can not only perceive correspondence The mood of user can also perceive the psychological activity type of corresponding user, thus can mood and psychological activity based on user Type makes better response.

In the embodiment of the present application, robot can not only identify that the mood, psychological activity type, vision of user are interested Region can also determine the habit of user by the data within a time cycle, after the habit for obtaining user, Ke Yigen According to user habit carry out more responsive to.

In some possible implementations, robot can to correspond to multiple images of user in the acquisition time period, In, which can be arranged based on experience value, and as the specific example of the application, which can be one Month, a season or 1 year.Then, robot extracts every institute for every image in multiple images of acquisition respectively The corresponding limbs characteristic information of image of corresponding user is stated, which can be the information such as the limb action of user, It is then determined the association of the corresponding limbs characteristic information of multiple images and the corresponding mood label of multiple described images Relationship determines the habit label of the user according to the incidence relation.Wherein, habit label is being grasped for characterizing the user Make corresponding emotional state when certain limb action.In this way, robot can divide according to every image in multiple described images Sample data is not generated according to every image and the corresponding mood label of every image and the habit label of the user.

In specific implementation, robot can be directed to every image, establish limbs characteristic information and mood label respectively Numerical value pair, then according to the numerical value of limbs characteristic information and mood label to counting, when the corresponding limb of certain image The numerical value pair of body characteristics information and mood label and existing numerical value to it is identical when, then the limbs characteristic information and mood mark The numerical value of label adds one to the number of appearance, after all images of acquisition are counted, according to limbs characteristic information and feelings The numerical value of thread label sorts to the number of appearance, and number is more, and relevance is stronger, and such robot, which realizes, counts multiple images The incidence relation of corresponding limbs characteristic information and the corresponding mood label of multiple described images.Robot can benefit The habit label of corresponding user is determined with the incidence relation, such as can be by limbs characteristic information and the numerical value pair of mood label The number of appearance is greater than the limbs characteristic information of preset times and the numerical value of mood label to the habit mark as corresponding user Label characterize corresponding user mood corresponding to the mood label when implementing the corresponding limb action of the limbs characteristic information State.

In the present embodiment, the habit label of user can use similar with mood label, psychological activity class label Mode indicates, i.e., is indicated by way of vector, in this way, the sample generated according to image, mood label and habit label Notebook data can be expressed as the form of " picture " and " two-dimensional tag vector ", wherein the feelings at two-dimensional tag vector table requisition family Thread and habit.

It can be seen that this method is by acquiring corresponding user's present invention also provides a kind of sample data generation method Then image determines the mood label of corresponding user according to the eyeball characteristic information extracted from the image of corresponding user, also, According to multiple images of corresponding user in a period of time, the corresponding mood label of every image of identification and limbs feature Information, and according to the habit label of the determining corresponding user of incidence relation between limbs characteristic information and mood label, Jin Ergen Sample data is generated according to image, mood label and the habit label of corresponding user, which can not only accurate identification The mood of user, also from habit, this dimension is identified user, thus quality with higher.

It should be noted that the improvement that Fig. 3 is carried out on the basis of being embodiment shown in Fig. 2 to embodiment illustrated in fig. 6, In specific implementation, it can according to actual needs combine above-described embodiment, to obtain the sample data of different dimensions.

For example, Fig. 3, Fig. 4, embodiment illustrated in fig. 6 can be combined, according to image, mood label, vision area-of-interest Sample data is generated with psychological activity class label, which can specifically be expressed as " picture " and " three-dimensional label vector " Form, the three-dimensional vector characterize user mood, psychological activity type and vision area-of-interest.In another example robot Sample number can also be generated according to image, mood label, psychological activity class label, vision area-of-interest and habit label According to the sample data can specifically be expressed as the form of " picture " and " four-dimensional label vector ", and four-dimension label vector characterization is used Mood, psychological activity type, vision area-of-interest and the habit at family.That is, on the basis based on picture and mood label On, robot can also according to psychological activity class label, vision area-of-interest and habit label any one or it is more Kind generates sample data.

Implement the specific implementation for the sample data generation method that power provides based on the application, the embodiment of the present application also mentions Sample data generating means have been supplied, sample data generating means will be introduced from the angle of function modoularization below, referring to Fig. 7, the device include:

Acquisition unit 710, for acquiring the image of corresponding user；

Extraction unit 720, for extracting the eyeball characteristic information of the corresponding user from the image of the corresponding user；

Determination unit 730, for determining the corresponding user's with the eyeball characteristic information by mood decision condition Mood label；

Generation unit 740, it is described for generating sample data according to the image of the corresponding user and the mood label Sample data is for training human perceptual model.

Optionally, optionally, the eyeball characteristic information includes pupil information；

The determination unit 730 is specifically used for:

Mood mark by the determination of mood decision condition and the matched mood label of the pupil information, as the user Label.

Optionally, the determination unit 730 is also used to:

Then the generation unit 740 is specifically used for:

Optionally, the determination unit 730 is also used to:

Then the generation unit 740 is specifically used for:

Optionally, the acquisition unit 710 is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

The determination unit 730 is also used to:

Then the generation unit 740 is specifically used for:

Optionally, the acquisition unit 710 is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

The extraction unit 720 is also used to:

The determination unit 730 is also used to:

Determine the corresponding user in week time according to the eyelid location information of the image of corresponding user described in every The blink characteristic information of phase；

Then the generation unit 740 is specifically used for:

Optionally, the acquisition unit 710 is specifically used for:

Multiple images of user are corresponded in the acquisition time period；

Then the extraction unit 720 is specifically used for:

The determination unit 730 is also used to: determining the corresponding limbs characteristic information of multiple images and described more Open the incidence relation of the corresponding mood label of image；

The habit label of the user is determined according to the incidence relation, the habit label is for characterizing described pair of application Family corresponding emotional state when operating certain limb action；

Then the generation unit 740 is specifically used for:

Optionally, the extraction unit 720 is also used to:

Then the determination unit 730 is specifically used for:

Optionally, the acquisition unit 710 is specifically used for:

From the foregoing, it will be observed that the application, which implements power, provides a kind of sample data generating means, which is based on image procossing Technology realizes what sample data generated, acquires the image of corresponding user first, then passes through image processing techniques from corresponding user Image in extract the eyeball feature of corresponding user, since eyeball feature can objectively reflect the mood of user, it is thus possible to The mood label that corresponding user is determined based on the eyeball feature, to realize the automatic identification of user emotion, and to user Accuracy rate with higher when mood is identified, therefore, according to the image and the higher mood mark of accuracy rate of corresponding user Label can automatically generate the sample data of high quality.

Based on sample data generation method provided by the embodiments of the present application, the embodiment of the present application also improves a kind of machine People, the robot includes processor and memory:

The memory is used to store the program code of human perceptual model, and said program code is transferred to the place Manage device；The human perceptual model is to be instructed using the sample data that sample data generation method provided by the embodiments of the present application generates The model got identifies the mood of the user when for interacting with user；

It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.

In several embodiments provided herein, it should be understood that disclosed device and method can pass through it Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or unit Letter connection can be electrical property, mechanical or other forms.

The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.

It should be appreciated that in this application, " at least one (item) " refers to one or more, and " multiple " refer to two or two More than a."and/or" indicates may exist three kinds of relationships, for example, " A and/or B " for describing the incidence relation of affiliated partner It can indicate: only exist A, only exist B and exist simultaneously tri- kinds of situations of A and B, wherein A, B can be odd number or plural number.Word Symbol "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or"." at least one of following (a) " or its similar expression, refers to Any combination in these, any combination including individual event (a) or complex item (a).At least one of for example, in a, b or c (a) can indicate: a, b, c, " a and b ", " a and c ", " b and c ", or " a and b and c ", and wherein a, b, c can be individually, can also To be multiple.

The above, above embodiments are only to illustrate the technical solution of the application, rather than its limitations；Although referring to before Embodiment is stated the application is described in detail, those skilled in the art should understand that: it still can be to preceding Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features；And these It modifies or replaces, the spirit and scope of each embodiment technical solution of the application that it does not separate the essence of the corresponding technical solution.

Claims

1. a kind of sample data generation method, which is characterized in that the described method includes:

Acquire the image of corresponding user；

The mood label of the corresponding user is determined with the eyeball characteristic information by mood decision condition；

Sample data is generated according to the image of the corresponding user and the mood label, the sample data is for training vision Sensor model.

2. the method according to claim 1, wherein the eyeball characteristic information includes pupil information；

Then the mood label that the corresponding user is determined by mood decision condition and the eyeball characteristic information includes:

Mood mark by the determination of mood decision condition and the matched mood label of the pupil information, as the corresponding user Label.

3. the method according to claim 1, wherein the method also includes:

Sample data is generated according to the image of the corresponding user, the mood label and the vision area-of-interest.

4. the method according to claim 1, wherein the method also includes:

The corresponding eyeball optics characteristic information of image or eyeball electrical characteristic letter of the corresponding user are obtained from eye movements system It ceases, and determines the vision region of interest of the corresponding user according to the eyeball optics characteristic information or eyeball electrical characteristic information Domain；

5. the method according to claim 1, wherein the image for acquiring corresponding user includes:

Multiple images of user are corresponded in the acquisition time period；

Then the method also includes:

Determine the corresponding user in the Rotation of eyeball of the time cycle according to multiple described images and eyeball tracking algorithm Direction；

The psychological activity classification of the corresponding user is determined by the first psychological activity decision condition and the Rotation of eyeball direction Label；

6. the method according to claim 1, wherein the image for acquiring corresponding user includes:

Multiple images of the corresponding user in the acquisition time period；

Then the method also includes:

For every image in multiple described images, the corresponding eyelid position of image of every corresponding user is extracted respectively Information determines the corresponding user in the time cycle according to the eyelid location information of the image of corresponding user described in every Blink characteristic information；

By the second psychological activity decision condition and the blink characteristic information, the psychological activity classification of the corresponding user is determined Label；

7. the method according to claim 1, wherein the image for acquiring corresponding user includes:

Multiple images of the corresponding user in the acquisition time period；

Then the method also includes:

For every image in multiple described images, the corresponding limbs feature of image of every corresponding user is extracted respectively Information；

Determine the corresponding limbs characteristic information of multiple images and the corresponding mood label of multiple described images Incidence relation；

The habit label of the corresponding user is determined according to the incidence relation, the habit label is for characterizing described pair of application Family corresponding emotional state when operating certain limb action；

It is corresponding according to every image and every image respectively for every image in multiple images of the corresponding user Mood label and the corresponding user habit label generate sample data.

8. according to claim 1 to method described in 7 any one, which is characterized in that the method also includes:

The face feature information and/or limbs characteristic information of the corresponding user are extracted from the image of the corresponding user；

According to the determination of the second mood decision condition and the face feature information and/or matched second mood of limbs characteristic information Label；

The mood label of the corresponding user is determined according to the first mood label and the second mood label.

9. according to claim 1 to method described in 7 any one, which is characterized in that the image packet for acquiring corresponding user It includes:

10. a kind of sample data generating means, which is characterized in that described device includes:

Acquisition unit, for acquiring the image of corresponding user；

Determination unit, for determining the mood mark of the corresponding user with the eyeball characteristic information by mood decision condition Label；

Generation unit, for generating sample data, the sample number according to the image of the corresponding user and the mood label According to for training human perceptual model.

11. a kind of robot, which is characterized in that the robot includes processor and memory:

The memory is used to store the program code of human perceptual model, and said program code is transferred to the processing Device；The human perceptual model is the sample generated using sample data generation method described in claim 1 to 9 any one The model that data training obtains, for identifying the mood of the corresponding user when interactive with user；

The processor is used to run the human perceptual model according to said program code, to realize with the corresponding user's Interaction.