CN108334806A

CN108334806A - Image processing method, device and electronic equipment

Info

Publication number: CN108334806A
Application number: CN201710282661.1A
Authority: CN
Inventors: 吴昊; 张振伟; 欧义挺; 董晓龙; 戚广全; 谢俊驰; 谢斯豪; 梁雪; 段韧; 张新磊
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2017-04-26
Filing date: 2017-04-26
Publication date: 2018-07-27
Anticipated expiration: 2037-04-26
Also published as: CN108334806B

Abstract

The present invention relates to a kind of image processing method, device and electronic equipment, the method includes：Obtain the picture frame acquired from reality scene；The described image frame of acquisition is played frame by frame according to the sequential of acquisition；Obtain the face affective characteristics recognition result that the facial image that identification described image frame includes obtains；According to the face affective characteristics recognition result, corresponding affective characteristics image is searched；Obtain display location of the affective characteristics image in currently playing picture frame；According to the display location, the affective characteristics image is rendered in currently playing picture frame.Scheme provided by the present application improves image processing efficiency.

Description

Image processing method, device and electronic equipment

Technical field

The present invention relates to field of computer technology, more particularly to a kind of image processing method, device and electronic equipment.

Background technology

With the development of computer technology, image processing techniques is also constantly progressive.User can by profession image at Reason software handles image so that treated image appearance is more preferable.User can also by image processing software, The material provided by image processing software is provided in image, allows treated image that can transmit more information.

However, current image procossing mode, needs the material database of user's unfolded image processing software, material database is browsed, Suitable material, the position of adjustment material in the picture is selected to complete image procossing to confirm modification from material database.In It is that current image procossing mode needs a large amount of manual operation, time-consuming, causes image processing process efficiency low.

Invention content

Based on this, it is necessary to be directed to traditional low problem of image processing process efficiency, provide a kind of image processing method, dress It sets and electronic equipment.

A kind of image processing method, the method includes：

Obtain the picture frame acquired from reality scene；

The described image frame of acquisition is played frame by frame according to the sequential of acquisition；

Obtain the face affective characteristics recognition result that the facial image that identification described image frame includes obtains；

According to the face affective characteristics recognition result, corresponding affective characteristics image is searched；

Obtain display location of the affective characteristics image in currently playing picture frame；

According to the display location, the affective characteristics image is rendered in currently playing picture frame.

A kind of image processing apparatus, described device include：

Acquisition module, for obtaining the picture frame acquired from reality scene；

Playing module, for playing the described image frame of acquisition frame by frame according to the sequential of acquisition；

Recognition result acquisition module, the face emotion obtained for obtaining the facial image that identification described image frame includes Feature recognition result；

Searching module, for according to the face affective characteristics recognition result, searching corresponding affective characteristics image；

Display location acquisition module, for obtaining displaying position of the affective characteristics image in currently playing picture frame It sets；

Rendering module, for according to the display location, the affective characteristics figure to be rendered in currently playing picture frame Picture.

A kind of electronic equipment, including memory and processor store computer-readable instruction in the memory, described When computer-readable instruction is executed by the processor so that the processor executes following steps：

Obtain the picture frame acquired from reality scene；

Above-mentioned image processing method, device and electronic equipment play the picture frame for the scene that reflects reality so that broadcasting Picture frame can reflect reality scene.Obtain the face affective characteristics identification knot that the facial image that identification picture frame includes obtains Fruit, so that it may to automatically determine the heartbeat conditions of the personage in reality scene.Affective characteristics image is got currently playing After display location in picture frame, according to the display location, affective characteristics image is rendered in currently playing picture frame, so that it may Automatically virtual affective characteristics image to be combined with figure picture in reality scene, the emotion shape of personage in the scene that reflects reality Condition.Because avoiding manually-operated tedious steps, image processing efficiency is greatly improved.

Description of the drawings

Fig. 1 is the applied environment figure of image processing method in one embodiment；

Fig. 2 is the internal structure schematic diagram of the electronic equipment for realizing image processing method in one embodiment；

Fig. 3 is the flow diagram of image processing method in one embodiment；

Fig. 4 is the flow diagram of image processing method in another embodiment；

Fig. 5 is the contrast schematic diagram at interface before and after drawing affective characteristics image in one embodiment；

Fig. 6 is the contrast schematic diagram at interface before and after showing the text identified according to voice data in one embodiment；

Fig. 7 is the structure diagram of image processing apparatus in one embodiment；

Fig. 8 is the structure diagram of image processing apparatus in another embodiment.

Specific implementation mode

In order to make the purpose , technical scheme and advantage of the present invention be clearer, with reference to the accompanying drawings and embodiments, right The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.

Fig. 1 is the applied environment figure of image processing method in one embodiment.Referring to Fig.1, the image processing method application In image processing system.Image processing system includes terminal 110 and server 120, and terminal 110 passes through network and server 120 Connection.Terminal 110 can be specifically terminal console or mobile terminal, and mobile terminal specifically can be with mobile phone, tablet computer or notes At least one of this computer etc..Server 120 can be specifically independent physical server, can also be physical server collection Group.Terminal 110 can be used for obtaining the picture frame acquired from reality scene, by the picture frame of acquisition according to acquisition sequential frame by frame It plays.Terminal 110 can be obtained when playing picture frame identifies that the face emotion that the facial image that the picture frame includes obtains is special Sign recognition result searches corresponding affective characteristics image, then obtain the affective characteristics according to the face affective characteristics recognition result Display location of the image in currently playing picture frame, according to the display location, being rendered in currently playing picture frame should Affective characteristics image.Wherein, including the above process that the facial image that picture frame includes is identified can be in terminal 110 Upper progress can also carry out on server 120.

Fig. 2 is the internal structure schematic diagram of electronic equipment in one embodiment.The electronic equipment can be the end in above-mentioned Fig. 1 End 110.As shown in Fig. 2, the electronic equipment includes processor, non-volatile memory medium, the memory connected by system bus Reservoir and network interface, voice collection device, loud speaker, display screen and input unit.Wherein, the non-volatile of electronic equipment is deposited Storage media is stored with operating system, further includes a kind of image processing apparatus, the image processing apparatus for realizing a kind of image at Reason method.The processor supports the operation of entire terminal for providing calculating and control ability.Built-in storage in electronic equipment Operation for the image processing apparatus in non-volatile memory medium provides environment, and can store computer in the built-in storage can Reading instruction when the computer-readable instruction is executed by the processor, may make the processor to execute a kind of image processing method Method.Network interface is used to carry out network communication with server 120, such as sends the picture frame of acquisition to server 120, receives service The face affective characteristics recognition result etc. that device 120 returns.The display screen of electronic equipment can be liquid crystal display or electronic ink Water display screen etc., input unit can be the touch layers covered on display screen, can also be the button being arranged in terminal enclosure, rail Mark ball or Trackpad can also be external keyboard, Trackpad or mouse etc..The electronic equipment can be terminal console, movement Either intelligent wearable device mobile terminal specifically can be at least one in mobile phone, tablet computer or laptop etc. for terminal Kind.It will be understood by those skilled in the art that structure shown in Figure 2, only with the relevant part-structure of application scheme Block diagram, does not constitute the restriction for the electronic equipment being applied thereon to application scheme, and specific electronic equipment may include Than more or fewer components as shown in the figure, either combines certain components or arranged with different components.

As shown in figure 3, in one embodiment, providing a kind of image processing method.The present embodiment is mainly in this way It is illustrated applied to the terminal 110 in above-mentioned Fig. 1.With reference to Fig. 3, which specifically comprises the following steps：

S302 obtains the picture frame acquired from reality scene.

Wherein, reality scene is scene present in natural world.Picture frame is the picture frame that can form dynamic menu Unit in sequence, for recording the picture in certain moment reality scene.

In one embodiment, terminal specifically can be according to fixed or dynamic frame per second, the acquired image frames from reality scene, Obtain the picture frame collected.Wherein, fixed or dynamic frame per second can be such that picture frame is fixed according to this or dynamic frame per second Continuous dynamic menu is formed when broadcasting.

In one embodiment, terminal can acquire the figure of reality scene by camera under the current visual field of camera As frame, the picture frame collected is obtained.Wherein, the visual field of camera can change because of the variation of the posture of terminal and position.

In one embodiment, terminal can provide AR (Augmented Reality, augmented reality) by social networking application Screening-mode, and after selecting the AR screening-modes, the acquired image frames from reality scene obtain the picture frame collected. Wherein, social networking application is that the application of network social intercourse interaction can be carried out based on social networks.Social networking application includes that instant messaging is answered With, SNS (Social Network Service, social network sites) application, live streaming apply or take pictures using etc..

In one embodiment, terminal can receive the picture frame acquired in the slave reality scene that another terminal is sent, obtain The picture frame of reception.For example, when terminal establishes video session by the social networking application operated in terminal, other session sides are received The picture frame acquired in the slave reality scene that corresponding terminal is sent.

In one embodiment, the screening-mode that terminal can be provided by the way that application is broadcast live, acquires image from reality scene Frame, using the picture frame of acquisition as live data, to be broadcast live by the way that application is broadcast live.Terminal also can receive another terminal and send , by be broadcast live application provide screening-mode from reality scene acquired image frames, using the picture frame received as be broadcast live Data, with the live streaming initiated by the way that application plays other users are broadcast live by the way that application is broadcast live.

S304 plays the picture frame of acquisition according to the sequential of acquisition frame by frame.

Wherein, the time sequencing when sequential of acquisition refers to acquired image frames can be recorded by picture frame in acquisition The magnitude relationship of timestamp indicates.Play frame by frame refers to being played by picture frame.

Specifically, terminal, according to timestamp ascending order, can play the image of acquisition one by one according to the frame per second of acquired image frames Frame.Terminal can directly play the picture frame of acquisition, can also the picture frame of acquisition be stored in caching according to the sequential of acquisition Area, and take out picture frame from buffer area by the sequential of acquisition and play.

In one embodiment, the image that terminal can will acquire in the slave reality scene of another terminal received transmission Frame, according to timestamp ascending order, plays the picture frame received one by one according to the frame per second of another terminal acquired image frames.Terminal can Directly to play the picture frame received, the picture frame received can also be stored in buffer area according to the sequential of acquisition, and Picture frame is taken out by the sequential of acquisition from buffer area to play.

S306 obtains the face affective characteristics recognition result that the facial image that identification picture frame includes obtains.

Wherein, affective characteristics are the features for reacting human or animal's emotion.Affective characteristics are that computer can recognize that and handle Feature.Such as happy, melancholy or indignation of affective characteristics etc..Face affective characteristics refer to the emotion spy reflected by human face expression Sign.

In one embodiment, terminal can be detected in the acquired image frames from reality scene in the image collected frame Whether include facial image.If terminal includes to the picture frame when the picture frame of judgement acquisition includes facial image Facial image carry out Expression Recognition, obtain the obtained face affective characteristics recognition result of identification.

In one embodiment, terminal can acquire reality scene by camera under the current visual field of camera After picture frame, the image data that the picture frame of acquisition includes is extracted, and detects whether the image data includes face characteristic number According to.If it includes face characteristic data that terminal, which detects in the image data, judge that the picture frame includes facial image.Terminal It can be further from face characteristic extracting data expressive features data, according to the expressive features data of extraction, in local to acquisition Picture frame include facial image carry out Expression Recognition, obtain face affective characteristics recognition result.Wherein, expressive features number According to can be for reflecting the distance between the profile of face, glasses, nose, mouth and each face's organ etc. therein one Kind or various features information.

For example, people, when feeling happy, the corners of the mouth can all raise up, if terminal includes face in picture frame The expressive features data that characteristic extracts raise up for the corners of the mouth, then can indicate that the affective characteristics that face reflects in the picture frame are Happily.For people when feeling surprised, the amplitude that mouth opens is larger, if terminal includes face characteristic data in picture frame The expressive features data extracted are that mouth opening amplitude is larger, then can indicate that the affective characteristics that face reflects in the picture frame are frightened It is surprised.

In one embodiment, the picture frame including facial image that detection obtains can be also sent to server by terminal, Server carries out Expression Recognition after the picture frame for receiving terminal transmission, to the facial image that the picture frame includes and obtains people Face affective characteristics recognition result, then the face affective characteristics recognition result that identification obtains is fed back into terminal, terminal obtains service The face affective characteristics recognition result that device returns.

In one embodiment, the picture frame that terminal can also acquire in receiving the slave reality scene that another terminal is sent Afterwards, whether detect in the picture frame received includes facial image.If the picture frame that terminal is received in judgement includes face When image, Expression Recognition can be carried out to the facial image that the picture frame includes in local, obtain corresponding face affective characteristics Recognition result；Also the picture frame can be sent to server so that server the facial image for including to the picture frame into After row identification, face affective characteristics recognition result is returned.

S308 searches corresponding affective characteristics image according to face affective characteristics recognition result.

Wherein, affective characteristics image refers to the image that can reflect affective characteristics.Reflect sad affective characteristics image such as Image including tears includes the image etc. of scene of raining.The affective characteristics image of reflection indignation such as includes the figure of flame As etc..Affective characteristics image can be the image that terminal is crawled from internet, can also be terminal according to by the terminal packet The image of the picture pick-up device shooting included.Affective characteristics image can be dynamic picture, can also be static images.

In one embodiment, terminal can select the affective characteristics that can carry out image procossing in advance, and corresponding to selected Affective characteristics configure corresponding affective characteristics image.Terminal obtains face feelings after getting face affective characteristics recognition result Feel the corresponding affective characteristics image of affective characteristics of feature recognition result characterization.

In one embodiment, terminal can establish affective characteristics image library in advance, and will reflect in affective characteristics image library The affective characteristics image of identical affective characteristics maps to identical affective characteristics.Terminal is getting face affective characteristics identification knot After fruit, affective characteristics and the matched emotion of face affective characteristics recognition result that reflection can be searched in affective characteristics image library are special Levy image.

In one embodiment, the affective characteristics image library that terminal is established in advance may include multiple affective characteristics image sets It closes, each affective characteristics image collection reflects a kind of affective characteristics.Terminal is looked into after getting face affective characteristics recognition result The affective characteristics image collection for looking for the affective characteristics reflected in affective characteristics image library consistent with face affective characteristics recognition result, Affective characteristics image is chosen from the affective characteristics image collection found.

S310 obtains display location of the affective characteristics image in currently playing picture frame.

Wherein, display location of the affective characteristics image in currently playing picture frame, the display location indicate that emotion is special Levy image region shared in currently playing picture frame.Display location can be by affective characteristics image in currently playing figure As the coordinate representation in coordinate system of the region shared in frame in currently playing picture frame.

In one embodiment, terminal can obtain the exhibition of the affective characteristics image together when searching affective characteristics image Show position.Terminal specifically can be from the corresponding drafting mode of affective characteristics image found be locally obtained, according to the drafting of acquisition Mode determines the display location of affective characteristics image.

Further, the drafting mode of affective characteristics image can be dynamic following object of reference.Specifically, terminal can determine The affective characteristics image found needs display location of the object of reference of dynamic following in currently playing picture frame, further according to ginseng Display location of the affective characteristics image in currently playing picture frame is determined according to the display location of object.

The drafting mode of affective characteristics image can also be static display.Specifically, for the affective characteristics of static display The display area of the affective characteristics image in currently playing picture frame can be directly arranged in image, terminal in advance, and terminal is needing Drawing the affective characteristics image can directly acquire.

S312 renders affective characteristics image according to display location in currently playing picture frame.

Specifically, terminal can render affective characteristics image in currently playing picture frame at the display location of acquisition. Terminal can obtain the corresponding pattern data of affective characteristics image, to according to the display location of the pattern data and acquisition, broadcast Affective characteristics image is rendered in the picture frame put.In one embodiment, affective characteristics image be include one group of image frame sequence Dynamic image.Terminal can render the image that dynamic image includes one by one according to the corresponding frame per second of dynamic image and display location Frame.

In one embodiment, display location can be affective characteristics image relative to a certain in currently playing picture frame The position of specific region；Terminal can track the specific region in the picture frame of broadcasting, to according to the display location and tracking The specific region arrived determines position of the affective characteristics image relative to the specific region tracked in currently playing picture frame, To render affective characteristics image according to determining position.Wherein, specific region is can to indicate specific in reality scene in image Region, which can be human face region etc..

Above-mentioned image processing method plays the picture frame for the scene that reflects reality so that the picture frame of broadcasting can reflect Reality scene.Obtain the face affective characteristics recognition result that the facial image that identification picture frame includes obtains, so that it may with automatic Ground determines the heartbeat conditions of the personage in reality scene.Get displaying of the affective characteristics image in currently playing picture frame Behind position, according to the display location, affective characteristics image is rendered in currently playing picture frame, so that it may with automatically will be virtual Affective characteristics image combined with figure picture in reality scene, the heartbeat conditions of personage in the scene that reflects reality.Because avoiding people The tedious steps of work operation, greatly improve image processing efficiency.

In one embodiment, step S306 is specifically included：The size of picture frame is adjusted to pre-set dimension；After adjustment The direction of picture frame is rotated to the direction for meeting affective characteristics identification condition；Postrotational picture frame is sent to server；It receives The face affective characteristics recognition result for the picture frame sent that server returns.

Wherein, pre-set dimension refers to the size of pre-set picture frame.Meeting the direction of affective characteristics identification condition is The direction for referring to picture frame when can carry out affective characteristics identification.

In one embodiment, terminal can be from the image spy of the preset picture frame including facial image of server pull Sign, which refers to the feature that can be carried out the picture frame of Expression Recognition and should have.For example, the size or figure of picture frame As the direction etc. of frame.

Specifically, the picture frame that terminal is acquired in acquisition from reality scene, and pick out the image including facial image After frame, whether the size of the detectable picture frame including facial image screened meets pre-set dimension.If detection filters out The size of the picture frame including facial image come does not meet pre-set dimension, then carries out size adjusting to the picture frame.

The size for the picture frame including facial image that terminal can be screened in detection meets pre-set dimension or right After incongruent image framing control size, the current direction of detection image frame.If the current direction of picture frame does not meet emotion spy Identification condition is levied, then rotates direction to the direction for meeting affective characteristics identification condition of picture frame.

Terminal can meet affective characteristics identification condition in the current direction of picture frame or be revolved to incongruent picture frame After turning direction, picture frame is sent to server.Server is extracted and is wrapped in picture frame after the picture frame for receiving terminal transmission The expressive features data included carry out the facial image that the picture frame received includes according to the expressive features data of extraction Expression Recognition obtains face affective characteristics recognition result, then the face affective characteristics recognition result that identification obtains is fed back to end End.

In one embodiment, the picture frame that terminal is acquired in acquisition from reality scene, and pick out including face figure After the picture frame of picture, picture frame can carry out to minification processing, and will reduce that treated picture frame saves as JPEG (Joint Photographic Experts Group Joint Photographic Experts Groups) format.Terminal can wrap in detection image frame again The direction of the facial image included, and the direction for the facial image for including in picture frame does not meet the side that affective characteristics identify condition Xiang Shi rotates the direction of picture frame.

Wherein, jpeg format refers to the picture format compressed according to International Standard of image compression.Meet affective characteristics When the direction of identification condition can be specifically that the angle of the central axes of facial image and vertical direction is not more than 45 degree in picture frame Direction.

In above-described embodiment, before facial image carries out Expression Recognition in by server to picture frame, picture frame is adjusted Size and direction so that picture frame meet carry out Expression Recognition condition, Expression Recognition speed and accuracy can be improved, may be used also Reduce hardware resource consumption.

In one embodiment, after step S306, which further includes：It is recorded when extracting acquired image frames Voice data；Obtain the speech emotional feature recognition result that identification voice data obtains.Step S308 is specifically included：According to people Face affective characteristics recognition result and speech emotional feature recognition are as a result, search corresponding affective characteristics image.

Specifically, terminal can record the voice data in reality scene simultaneously in the acquired image frames from reality scene, And when playing the picture frame of acquisition, the voice data of recording is played simultaneously.Terminal can specifically call voice collection device to acquire Voice data is corresponded to acquisition time and is stored in buffer area by the voice data that ambient sound is formed.

Terminal can extract current progress expression when the facial image that the picture frame to acquisition includes carries out Expression Recognition Acquisition time corresponding to the picture frame of identification intercepts the voice data piece of predetermined time period from the voice data of buffer area Section, and the acquisition time that the corresponding acquisition time section covering of voice data segment extracted obtains.The voice data piece of extraction Section is to acquire the voice data recorded when the picture frame.Wherein, predetermined time period is pre-set interception voice data The time span of segment, predetermined time period can be specifically 5 seconds or 10 seconds etc..

In one embodiment, terminal can be intercepted from the voice data of buffer area using the acquisition time of acquisition as midpoint The voice data segment of predetermined time period.For example, currently the acquisition time corresponding to the picture frame of progress Expression Recognition is 30 divide 15 seconds when 1 day 18 October in 2016, and predetermined time period is 5 seconds, then can divide 15 seconds with when 1 day 18 October in 2016 30 For midpoint, 30 30 divide 17 seconds when dividing 13 seconds to 2016 on October 1,18 when interception acquisition time section is 1 day 18 October in 2016 Voice data segment.

In one embodiment, when the picture frame that terminal acquires in the slave reality scene for receiving another terminal transmission, It can receive the voice data recorded in acquired image frames that another terminal is sent.The voice data of reception can be stored in slow by terminal Area is deposited, when playing picture frame by the sequential of acquisition, voice data taking-up is played simultaneously.

Terminal can extract current progress table when the facial image for including to the picture frame received carries out Expression Recognition Acquisition time corresponding to the picture frame of feelings identification, intercepts the voice data of predetermined time period from the voice data of buffer area Segment, and the acquisition time that the corresponding acquisition time section covering of voice data segment extracted obtains.The voice data of extraction Segment is to acquire the voice data recorded when the picture frame

Terminal is after obtaining the voice data that the when of acquiring the current picture frame for carrying out Expression Recognition records, to the language of acquisition Sound data are identified, and obtain speech emotional feature recognition result.

In one embodiment, the speech emotional feature recognition knot that identification voice data obtains is obtained in image processing method The step of fruit, specifically includes：The voice data of extraction is identified as text；Search the affective characteristics keyword included by text；Root It is investigated that the affective characteristics keyword found, obtains speech emotional feature recognition result corresponding with voice data.

Specifically, terminal can carry out feature extraction to voice data, obtain voice feature data to be identified, be then based on Acoustic model carries out voice sub-frame processing to voice feature data to be identified and obtains multiple phonemes, according to candidate in candidate character library The correspondence of word and phoneme converts multiple phonemes that processing obtains to character string, recycles language model adjustment conversion Obtained character string, to obtain meeting the text of natural language pattern.

Wherein, text is the character representation form of voice data.Acoustic model such as GMM (Gaussian Mixture Model gauss hybrid models) or DNN (Deep Neural Network deep neural networks) etc..Candidate character library includes candidate word Phoneme corresponding with candidate word.The character sequence that language model is used to be identified according to natural language mode adjustment acoustic model Row, such as N-Gram models (CLM, Chinese Language Model Chinese language models) etc..

Affective characteristics key word library can be arranged in terminal in advance, and affective characteristics key word library includes that several affective characteristics are crucial Word, and it is special to reflect that the affective characteristics keyword of identical affective characteristics maps to identical emotion in affective characteristics key word library Sign.Affective characteristics key word library is storable in during file, database either cache when needed from file, database or slow Deposit middle acquisition.Terminal by the voice data of extraction after being identified as text, the character and feelings that include by the text that identification obtains Feel each affective characteristics Key comparison in feature critical character library.When there are characters and emotion in affective characteristics key word library in text When feature critical word matches, matched affective characteristics keyword is obtained, the corresponding affective characteristics of affective characteristics keyword are obtained For speech emotional feature recognition result.

It illustrates, it is assumed that the text that terminal recognition voice data obtains is " I am very happy today ", including emotion Feature critical word " happy ", the affective characteristics that " happy " maps to are " happy ", then speech emotional feature recognition result " is opened The heart ".Assuming that the text that terminal recognition voice data obtains is " I am very delight ", it is " high including affective characteristics keyword It is emerging ", the affective characteristics that " happiness " maps to are " happy ", then speech emotional feature recognition result is also " happy ".

In above-described embodiment, text identification, the expression feelings for including according to text are carried out by the voice data to recording The character of feature is felt to obtain speech emotional feature recognition as a result, improving the accuracy of speech emotional feature recognition result.

In one embodiment, the speech emotional feature that terminal can also be obtained according to the corresponding acoustic feature of voice data Recognition result.Terminal specifically can carry out acoustic feature extraction to voice data, special with emotion according to the acoustic feature established in advance The correspondence of sign obtains corresponding affective characteristics, obtains speech emotional feature recognition result.

In one embodiment, acoustic feature includes tone color and prosodic features.Tone color refers to the spy that sounding body makes a sound Color, for different sounding bodies since material, structure are different, the tone color made a sound is also just different.Physically joined by frequency spectrum It counts to characterize tone color.Prosodic features refers to the basic tone and rhythm that sounding body makes a sound, and is physically being joined by fundamental frequency Number, duration distribution and signal strength characterize prosodic features.

For example, people, when feeling happy, the rhythm can be shown as cheerful and light-hearted when speaking, if terminal is in voice number It is higher according to basic tone in the prosodic features extracted and when rhythm is very fast, it can indicate that the affective characteristics of voice data reflection are Happily.

In the present embodiment, acoustic feature extraction is carried out by the voice data to recording, is indicated according in acoustic feature The parameter of affective characteristics obtains speech emotional feature recognition as a result, improving the accuracy of speech emotional feature recognition result.

In one embodiment, known according to face affective characteristics recognition result and speech emotional feature in image processing method Not as a result, the step of searching corresponding affective characteristics image may particularly include：When face affective characteristics recognition result and voice feelings When feeling the matching of feature recognition result, corresponding affective characteristics image is searched according to face affective characteristics recognition result.

Specifically, the face emotion that terminal is obtained in the Expression Recognition for obtaining the facial image for including according to picture frame is special Levy recognition result, and the speech emotional feature recognition result that the voice data recorded of according to acquired image frames when identifies Afterwards, face affective characteristics recognition result and speech emotional feature recognition result are compared, is tied when face affective characteristics identify When fruit matches with speech emotional feature recognition result, corresponding affective characteristics figure is searched according to face affective characteristics recognition result Picture.

In one embodiment, in image processing method corresponding emotion spy is searched according to face affective characteristics recognition result Image is levied, including：The affective characteristics type and recognition result confidence level that extraction face affective characteristics recognition result includes；Search with The corresponding affective characteristics image collection of affective characteristics type；From affective characteristics image collection, pick out and recognition result confidence Spend corresponding affective characteristics image.

Wherein, affective characteristics type refers to the type of the reflected affective characteristics of face.Such as " happy ", " sad " or " indignation " etc..Recognition result confidence level indicates that face affective characteristics recognition result is the credible journey of the true affective characteristics of face Degree, recognition result confidence level is higher, indicates that face affective characteristics recognition result is that the possibility of the true affective characteristics of face is got over It is high.

Specifically, the affective characteristics image library that terminal is established in advance may include multiple affective characteristics image collections, each Affective characteristics image collection reflects a kind of affective characteristics type.Terminal can correspond to face affective characteristics recognition result confidence level one One one affective characteristics image of mapping.Terminal searches affective characteristics image library after getting face affective characteristics recognition result The affective characteristics of the middle reflection affective characteristics image set consistent with the affective characteristics type that face affective characteristics recognition result includes It closes, the recognition result confidence level for including with face affective characteristics recognition result is chosen from the affective characteristics image collection found Corresponding affective characteristics image.

In above-described embodiment, the recognition result confidence level for including to different face affective characteristics recognition results is respectively set Corresponding affective characteristics image visualizes the credible of reflection face affective characteristics recognition result by affective characteristics image Degree so that processing result image is more acurrate.

In one embodiment, when face affective characteristics recognition result is matched with speech emotional feature recognition result, eventually The feelings for holding the affective characteristics that can also reflect in the affective characteristics image library found to include with face affective characteristics recognition result Feel in the consistent affective characteristics image collection of characteristic type, randomly selects an affective characteristics image.

In the present embodiment, when face affective characteristics recognition result is matched with speech emotional feature recognition result, according to Face affective characteristics recognition result searches corresponding affective characteristics image, the in this way guarantee in speech emotional feature recognition result Under, carry out image procossing according to face affective characteristics recognition result so that processing result image is more acurrate.

In one embodiment, known according to face affective characteristics recognition result and speech emotional feature in image processing method Not as a result, the step of searching corresponding affective characteristics image may particularly include：When face affective characteristics recognition result and voice feelings When feeling feature recognition result mismatch, corresponding affective characteristics image is searched according to speech emotional feature recognition result.

Specifically, the face emotion that terminal is obtained in the Expression Recognition for obtaining the facial image for including according to picture frame is special Levy recognition result, and the speech emotional feature recognition result that the voice data recorded of according to acquired image frames when identifies Afterwards, face affective characteristics recognition result and speech emotional feature recognition result are compared, is tied when face affective characteristics identify When fruit mismatches with speech emotional feature recognition result, corresponding affective characteristics figure is searched according to speech emotional feature recognition result Picture.

In one embodiment, terminal can also obtain the degree adverb that the text that voice data identifies includes. Degree adverb is used to indicate the intensity of emotion, such as：" very ", " very " or " and its " etc..Terminal-pair voice data is known The speech emotional feature recognition result not obtained specifically may include affective characteristics type and emotion intensity.

Specifically, the affective characteristics image library that terminal is established in advance may include multiple affective characteristics image collections, each Affective characteristics image collection reflects a kind of affective characteristics type.Terminal can correspond to emotion intensity and map an emotion one by one Characteristic image.It is special to search the emotion reflected in affective characteristics image library after getting speech emotional feature recognition result for terminal The sign affective characteristics image collection consistent with the affective characteristics type that speech emotional feature recognition result includes, from the feelings found Feel and chooses affective characteristics corresponding with the emotion intensity that speech emotional feature recognition result includes in characteristic image set Image.

In the present embodiment, it when face affective characteristics recognition result and speech emotional feature recognition result mismatch, presses Corresponding affective characteristics image is searched according to speech emotional feature recognition result, this emotion with the expression of true voice data is special Recognition result is levied to carry out image procossing so that processing result image is more acurrate.

In above-described embodiment, face affective characteristics recognition result is considered with speech emotional feature recognition as a result, searching Reflect in picture frame so the affective characteristics image of the affective characteristics of performance so that processing result image is more acurrate.

In one embodiment, step S310 specifically includes display of the determining facial image in currently playing picture frame Position；Inquire the relative position of affective characteristics image and facial image；According to display location and relative position, affective characteristics are determined Display location of the image in currently playing picture frame.

In the present embodiment, display location of the affective characteristics image in currently playing picture frame refers to affective characteristics figure As the physical location being shown in currently playing picture frame.Terminal can obtain and search when searching affective characteristics image Object of reference when the affective characteristics Image Rendering arrived.Object of reference can be specifically the facial image that picture frame includes.

Specifically, terminal can obtain display location of the object of reference in currently playing picture frame and affective characteristics image with The relative position of object of reference, terminal is further according to display location of the object of reference in currently playing picture frame and affective characteristics image Display location of the affective characteristics image in currently playing picture frame is determined with the relative position of object of reference.Affective characteristics image Display location in currently playing picture frame can be specifically the seat of pixel coordinate section or other default positioning methods Mark section.Pixel refers to the least unit that can be shown on computer screen.In the present embodiment, pixel can be logical pixel Or physical picture element.

In above-described embodiment, by the way that the relative position of affective characteristics image and facial image is arranged, make affective characteristics image Position relative to facial image is shown, so that the display location of affective characteristics image is more reasonable.

In one embodiment, after step S312, which further includes the tracking in the picture frame of broadcasting The movement locus of facial image；According to the movement locus of tracking, affective characteristics image is followed to the face in the picture frame of broadcasting Image moves.

Wherein, the movement locus of facial image refers to the rail for the facial image formation that the picture frame continuously played includes Mark.Specifically, the display location of affective characteristics image can be affective characteristics image relative to people in currently playing picture frame The position of face image；Terminal can track the facial image in currently playing picture frame in the picture frame of broadcasting, to press According to the display location and the facial image that tracks, determine in currently playing picture frame affective characteristics image relative to tracking Facial image position, to render affective characteristics image according to determining position.

In above-described embodiment, affective characteristics image follows facial image to be shown, to intelligently by affective characteristics figure As being associated with the face in reality scene, new interaction mode is provided.

As shown in figure 4, in a specific embodiment, image processing method includes：

S402 obtains the picture frame acquired from reality scene.

S404 plays the picture frame of acquisition according to the sequential of acquisition frame by frame.

S406 adjusts the size of picture frame to pre-set dimension；The direction of picture frame after adjustment is rotated to meeting emotion The direction of feature recognition condition；Postrotational picture frame is sent to server；The face affective characteristics that server returns are received to know Other result.

S408 extracts the voice data recorded when acquired image frames；It is special to obtain the speech emotional that identification voice data obtains Levy recognition result.

S410, judges whether face affective characteristics recognition result matches with speech emotional feature recognition result；If so, jumping Go to step S412；If it is not, then jumping to step S414.

S412, the affective characteristics type and recognition result confidence level that extraction face affective characteristics recognition result includes；It searches Affective characteristics image collection corresponding with affective characteristics type；From affective characteristics image collection, picks out and set with recognition result The corresponding affective characteristics image of reliability.

S414 searches corresponding affective characteristics image according to speech emotional feature recognition result.

S416 determines display location of the facial image in currently playing picture frame；Inquire affective characteristics image and people The relative position of face image；According to display location and relative position, determine affective characteristics image in currently playing picture frame Display location.

S418 renders affective characteristics image according to display location in currently playing picture frame.

S420 tracks the movement locus of facial image in the picture frame of broadcasting；According to the movement locus of tracking, by feelings Sense characteristic image follows the facial image movement in the picture frame of broadcasting.

In the present embodiment, from reality scene acquired image frames and according to the sequential of acquisition play, by acquisition The face affective characteristics recognition result for the facial image that picture frame includes, it will be able to determine personage's emotion in reflection facial image The affective characteristics image of feature, and be shown.Emotion is directly carried out according to the picture frame acquired in reality scene immediately in this way The displaying of characteristic image can avoid manually choosing affective characteristics image and manually adjusting affective characteristics image being shown And the workload introduced, image processing efficiency is improved, and image procossing is real-time.

In one embodiment, terminal, can also be in currently playing picture frame after identifying to obtain text to voice data The text that middle display identification obtains.Terminal can specifically be drawn in currently playing picture frame for carrying out display text content Component shows that identification obtains text in assembly.In the present embodiment, it is identified by being shown in currently playing picture frame To text can overcome the obstacle of deaf and dumb human world interaction, improve the practicability of image procossing.

Fig. 5 shows the contrast schematic diagram at interface before and after drafting affective characteristics image in one embodiment.It is with reference to 5 left side of figure The interface schematic diagram before affective characteristics image is drawn, which includes facial image 510, with reference to 5 right drafting emotion of figure Interface schematic diagram after characteristic image, the interface schematic diagram include facial image 510 and affective characteristics image 520, affective characteristics Image 520 includes indicating that affective characteristics are happy affective characteristics image 521 and indicate the affective characteristics that affective characteristics are sad Image 522.

Terminal is carrying out what Expression Recognition obtained according to the facial image 510 in the interface before drafting affective characteristics image Face affective characteristics recognition result, and the speech emotional feature recognition result that the voice data of recording identifies is found After corresponding affective characteristics image.If the affective characteristics that the left sides Fig. 5 that terminal determines, which include facial image 510, to be reflected are happy, The facial image 510 is tracked in currently playing picture frame, and is drawn in corresponding position and indicated that affective characteristics are happy Affective characteristics image 521.If the affective characteristics that the left sides Fig. 5 that terminal determines, which include facial image 510, to be reflected are sad, current The facial image 510 is tracked in the picture frame of broadcasting, and is drawn in corresponding position and indicated that affective characteristics are that sad emotion is special Levy image 522.

Fig. 6 shows the comparison signal at interface before and after the text for showing in one embodiment and being identified according to voice data Figure.With reference to the left interface schematic diagram to show before the text identified according to voice data of figure 6, which includes people Face image 610, with reference to the right interface schematic diagram to show after the text identified according to voice data of figure 6, interface signal Figure includes facial image 610, affective characteristics image 620 and text 630.Wherein, text 630 by terminal according to acquire the picture frame When the voice data recorded identify to obtain, can be specifically " I am good sad today ", the affective characteristics of reflection be it is sad, then can be The facial image 610, and the text 630 obtained in the display identification of corresponding position are tracked in currently playing picture frame, may be used also It is drawn in corresponding position and indicates that affective characteristics are sad affective characteristics image 620.

Fig. 7 is the structure diagram of image processing apparatus 700 in one embodiment.With reference to Fig. 7, the image processing apparatus 700 Including：Picture frame acquisition module 701, playing module 702, recognition result acquisition module 703, searching module 704, display location obtain Modulus block 705 and rendering module 706.

Picture frame acquisition module 701, for obtaining the picture frame acquired from reality scene.

Playing module 702, for playing the picture frame of acquisition frame by frame according to the sequential of acquisition.

Recognition result acquisition module 703, the face emotion obtained for obtaining the facial image that identification picture frame includes Feature recognition result.

Searching module 704, for according to face affective characteristics recognition result, searching corresponding affective characteristics image.

Display location acquisition module 705, for obtaining displaying position of the affective characteristics image in currently playing picture frame It sets.

Rendering module 706, for according to display location, affective characteristics image to be rendered in currently playing picture frame.

Above-mentioned image processing apparatus 700 plays the picture frame for the scene that reflects reality so that the picture frame of broadcasting can be anti- Reflect reality scene.Obtain the face affective characteristics recognition result that the facial image that identification picture frame includes obtains, so that it may with certainly The heartbeat conditions of the personage in reality scene are determined dynamicly.Get exhibition of the affective characteristics image in currently playing picture frame After showing position, according to the display location, affective characteristics image is rendered in currently playing picture frame, so that it may with automatically will be empty Quasi- affective characteristics image is combined with figure picture in reality scene, the heartbeat conditions of personage in the scene that reflects reality.Because avoiding Manually-operated tedious steps, greatly improve image processing efficiency.

In one embodiment, recognition result acquisition module 703 is additionally operable to the size of adjustment picture frame to pre-set dimension；It will The direction of picture frame after adjustment is rotated to the direction for meeting affective characteristics identification condition；Postrotational picture frame is sent to service Device；Receive the face affective characteristics recognition result for the picture frame sent that server returns.

In the present embodiment, before facial image carries out Expression Recognition in by server to picture frame, picture frame is adjusted Size and direction so that picture frame meet carry out Expression Recognition condition, Expression Recognition speed and accuracy can be improved, may be used also Reduce hardware resource consumption.

In one embodiment, recognition result acquisition module 703 is additionally operable to the voice number recorded when extraction acquired image frames According to；Obtain the speech emotional feature recognition result that identification voice data obtains.Searching module 704 is additionally operable to special according to face emotion Recognition result and speech emotional feature recognition are levied as a result, searching corresponding affective characteristics image.

In the present embodiment, face affective characteristics recognition result is considered with speech emotional feature recognition as a result, searching Reflect in picture frame so the affective characteristics image of the affective characteristics of performance so that processing result image is more acurrate.

In one embodiment, recognition result acquisition module 703 is additionally operable to the voice data of extraction being identified as text；It looks into Look for the affective characteristics keyword included by text；According to the affective characteristics keyword found, obtain corresponding with voice data Speech emotional feature recognition result.

In the present embodiment, text identification, the expression feelings for including according to text are carried out by the voice data to recording The character of feature is felt to obtain speech emotional feature recognition as a result, improving the accuracy of speech emotional feature recognition result.

In one embodiment, searching module 704 is additionally operable to when face affective characteristics recognition result and speech emotional feature When recognition result matches, corresponding affective characteristics image is searched according to face affective characteristics recognition result.

In the present embodiment, the recognition result confidence level for including to different face affective characteristics recognition results is respectively set Corresponding affective characteristics image visualizes the credible of reflection face affective characteristics recognition result by affective characteristics image Degree so that processing result image is more acurrate.

In one embodiment, it is special to be additionally operable to the emotion that extraction face affective characteristics recognition result includes for searching module 704 Levy type and recognition result confidence level；Search affective characteristics image collection corresponding with affective characteristics type；From affective characteristics figure During image set closes, affective characteristics image corresponding with recognition result confidence level is picked out.

In one embodiment, searching module 704 is additionally operable to when face affective characteristics recognition result and speech emotional feature When recognition result mismatches, corresponding affective characteristics image is searched according to speech emotional feature recognition result.

In one embodiment, display location acquisition module 705 is additionally operable to determine facial image in currently playing image Display location in frame；Inquire the relative position of affective characteristics image and facial image；According to display location and relative position, really Pledge love to feel display location of the characteristic image in currently playing picture frame.

In the present embodiment, by the way that the relative position of affective characteristics image and facial image is arranged, make affective characteristics image Position relative to facial image is shown, so that the display location of affective characteristics image is more reasonable.

As shown in figure 8, in one embodiment, image processing apparatus 700 further includes rendering to follow module 707.

Rendering follows module 707, in the picture frame of broadcasting, tracking the movement locus of facial image；According to tracking Movement locus, affective characteristics image is followed the facial image in the picture frame of broadcasting move.

In the present embodiment, affective characteristics image follows facial image to be shown, to intelligently by affective characteristics figure As being associated with the face in reality scene, new interaction mode is provided.

In one embodiment, a kind of computer readable storage medium, is stored thereon with computer-readable instruction, the calculating Machine readable instruction realizes following steps when being executed by processor：

Obtain the picture frame acquired from reality scene；

The picture frame of acquisition is played frame by frame according to the sequential of acquisition；

Obtain the face affective characteristics recognition result that the facial image that identification picture frame includes obtains；

According to face affective characteristics recognition result, corresponding affective characteristics image is searched；

According to display location, affective characteristics image is rendered in currently playing picture frame.

The computer-readable instruction stored on above computer readable storage medium storing program for executing when executed, by the scene that reflects reality Picture frame play so that the picture frame of broadcasting can reflect reality scene.Obtain the facial image that identification picture frame includes Obtained face affective characteristics recognition result, so that it may to automatically determine the heartbeat conditions of the personage in reality scene.It gets After display location of the affective characteristics image in currently playing picture frame, according to the display location, in currently playing image Affective characteristics image is rendered in frame, so that it may automatically virtual affective characteristics image to be combined with figure picture in reality scene, The heartbeat conditions of personage in the scene that reflects reality.Because avoiding manually-operated tedious steps, image procossing is greatly improved Efficiency.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held Capable acquisition identifies that the step of face affective characteristics recognition result that the facial image that picture frame includes obtains includes：Adjustment figure As the size of frame is to pre-set dimension；The direction of picture frame after adjustment is rotated to the direction for meeting affective characteristics identification condition； Postrotational picture frame is sent to server；Receive the face affective characteristics identification for the picture frame sent that server returns As a result.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held After row obtains the step of face affective characteristics recognition result that the facial image that identification picture frame includes obtains, it can also carry out Following steps：Extract the voice data recorded when acquired image frames；The speech emotional feature that identification voice data obtains is obtained to know Other result.Execute according to face affective characteristics recognition result, the step of searching corresponding affective characteristics image includes：According to people Face affective characteristics recognition result and speech emotional feature recognition are as a result, search corresponding affective characteristics image.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held Capable acquisition identifies that the step of speech emotional feature recognition result that voice data obtains includes：The voice data of extraction is identified For text；Search the affective characteristics keyword included by text；According to the affective characteristics keyword found, obtain and voice number According to corresponding speech emotional feature recognition result.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held It is capable according to face affective characteristics recognition result and speech emotional feature recognition as a result, searching the step of corresponding affective characteristics image Suddenly include：When face affective characteristics recognition result is matched with speech emotional feature recognition result, know according to face affective characteristics Other result searches corresponding affective characteristics image.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held Capable the step of searching corresponding affective characteristics image according to face affective characteristics recognition result includes：Extract face affective characteristics The affective characteristics type and recognition result confidence level that recognition result includes；Search affective characteristics figure corresponding with affective characteristics type Image set closes；From affective characteristics image collection, affective characteristics image corresponding with recognition result confidence level is picked out.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held It is capable according to face affective characteristics recognition result and speech emotional feature recognition as a result, searching the step of corresponding affective characteristics image Suddenly include：When face affective characteristics recognition result and speech emotional feature recognition result mismatch, according to speech emotional feature Recognition result searches corresponding affective characteristics image.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held The step of display location of the capable acquisition affective characteristics image in currently playing picture frame includes：Determine that facial image is being worked as Display location in the picture frame of preceding broadcasting；Inquire the relative position of affective characteristics image and facial image；According to display location And relative position, determine display location of the affective characteristics image in currently playing picture frame.

In one embodiment, the computer-readable instruction stored on computer readable storage medium when executed, is held After the step of going according to display location, affective characteristics image is rendered in currently playing picture frame, following step can also carry out Suddenly：In the picture frame of broadcasting, the movement locus of facial image is tracked；According to the movement locus of tracking, by affective characteristics image The facial image in the picture frame of broadcasting is followed to move.

In one embodiment, a kind of electronic equipment, including memory and processor, storing computer in memory can Reading instruction, when computer-readable instruction is executed by processor so that processor executes following steps：

Obtain the picture frame acquired from reality scene；

The picture frame for the scene that reflects reality is played, is made when executing above-mentioned steps by processor by above-mentioned electronic equipment The picture frame that must be played can reflect reality scene.It is special to obtain the face emotion that the facial image that identification picture frame includes obtains Levy recognition result, so that it may to automatically determine the heartbeat conditions of the personage in reality scene.Affective characteristics image is got to work as After display location in the picture frame of preceding broadcasting, according to the display location, affective characteristics are rendered in currently playing picture frame Image, so that it may automatically virtual affective characteristics image to be combined with figure picture in reality scene, people in the scene that reflects reality The heartbeat conditions of object.Because avoiding manually-operated tedious steps, image processing efficiency is greatly improved.

In one embodiment, electronic equipment when executing computer-readable instruction by processor, know by the acquisition of execution The step of face affective characteristics recognition result that the facial image that other picture frame includes obtains includes：Adjust the size of picture frame To pre-set dimension；The direction of picture frame after adjustment is rotated to the direction for meeting affective characteristics identification condition；After sending rotation Picture frame to server；Receive the face affective characteristics recognition result for the picture frame sent that server returns.

In one embodiment, electronic equipment executes when executing computer-readable instruction by processor and obtains identification After the step of face affective characteristics recognition result that the facial image that picture frame includes obtains, following steps are can also carry out： Extract the voice data recorded when acquired image frames；Obtain the speech emotional feature recognition result that identification voice data obtains.Place Manage that device executes according to face affective characteristics recognition result, the step of searching corresponding affective characteristics image includes：According to face Affective characteristics recognition result and speech emotional feature recognition are as a result, search corresponding affective characteristics image.

In one embodiment, electronic equipment when executing computer-readable instruction by processor, know by the acquisition of execution The step of speech emotional feature recognition result that other voice data obtains includes：The voice data of extraction is identified as text；It looks into Look for the affective characteristics keyword included by text；According to the affective characteristics keyword found, obtain corresponding with voice data Speech emotional feature recognition result.

In one embodiment, electronic equipment by processor execute computer-readable instruction when, execution according to people Face affective characteristics recognition result and speech emotional feature recognition are as a result, the step of searching corresponding affective characteristics image includes：When When face affective characteristics recognition result is matched with speech emotional feature recognition result, searched according to face affective characteristics recognition result Corresponding affective characteristics image.

In one embodiment, electronic equipment by processor execute computer-readable instruction when, execution according to people Face affective characteristics recognition result search corresponding affective characteristics image the step of include：Extract face affective characteristics recognition result packet The affective characteristics type and recognition result confidence level included；Search affective characteristics image collection corresponding with affective characteristics type；From In affective characteristics image collection, affective characteristics image corresponding with recognition result confidence level is picked out.

In one embodiment, electronic equipment by processor execute computer-readable instruction when, execution according to people Face affective characteristics recognition result and speech emotional feature recognition are as a result, the step of searching corresponding affective characteristics image includes：When When face affective characteristics recognition result is mismatched with speech emotional feature recognition result, looked into according to speech emotional feature recognition result Look for corresponding affective characteristics image.

In one embodiment, electronic equipment by processor when executing computer-readable instruction, the acquisition feelings of execution The step of feeling display location of the characteristic image in currently playing picture frame include：Determine facial image in currently playing figure As the display location in frame；Inquire the relative position of affective characteristics image and facial image；According to display location and relative position, Determine display location of the affective characteristics image in currently playing picture frame.

In one embodiment, electronic equipment is executed when executing computer-readable instruction by processor according to displaying Position, in currently playing picture frame render affective characteristics image the step of after, can also carry out following steps：In broadcasting In picture frame, the movement locus of facial image is tracked；According to the movement locus of tracking, affective characteristics image is followed to the figure of broadcasting As the facial image movement in frame.

One of ordinary skill in the art will appreciate that realizing all or part of flow in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the program can be stored in a non-volatile computer and can be read In storage medium, the program is when being executed, it may include such as the flow of the embodiment of above-mentioned each method.Wherein, the storage is situated between Matter can be magnetic disc, CD, read-only memory (Read-Only Memory, ROM) etc..

Each technical characteristic of above example can be combined arbitrarily, to keep description succinct, not to above-described embodiment In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance Shield is all considered to be the range of this specification record.

Several embodiments of the invention above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously Cannot the limitation to the scope of the claims of the present invention therefore be interpreted as.It should be pointed out that for those of ordinary skill in the art For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention Protect range.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.

Claims

1. a kind of image processing method, the method includes：

Obtain the picture frame acquired from reality scene；

2. according to the method described in claim 1, it is characterized in that, the face figure for obtaining identification described image frame and including As obtained face affective characteristics recognition result, including：

The size of described image frame is adjusted to pre-set dimension；

The direction of described image frame after adjustment is rotated to the direction for meeting affective characteristics identification condition；

Postrotational described image frame is sent to server；

Receive the face affective characteristics recognition result for the described image frame sent that the server returns.

3. according to the method described in claim 1, it is characterized in that, the face figure for obtaining identification described image frame and including After the face affective characteristics recognition result that picture obtains, the method further includes：

The voice data recorded when extraction acquisition described image frame；

Obtain the speech emotional feature recognition result for identifying that the voice data obtains；

It is described that corresponding affective characteristics image is searched according to the face affective characteristics recognition result, including：

According to the face affective characteristics recognition result and the speech emotional feature recognition as a result, searching corresponding affective characteristics Image.

4. according to the method described in claim 3, it is characterized in that, described obtain the voice feelings for identifying that the voice data obtains Feature recognition is felt as a result, including：

The voice data of extraction is identified as text；

Search the affective characteristics keyword included by the text；

According to the affective characteristics keyword found, speech emotional feature recognition corresponding with the voice data is obtained As a result.

5. according to the method described in claim 3, it is characterized in that, described according to the face affective characteristics recognition result and institute Predicate sound affective characteristics recognition result searches corresponding affective characteristics image, including：

When the face affective characteristics recognition result is matched with the speech emotional feature recognition result, according to the face feelings Feel feature recognition result and searches corresponding affective characteristics image.

6. according to the method described in claim 5, it is characterized in that, described search according to the face affective characteristics recognition result Corresponding affective characteristics image, including：

Extract the affective characteristics type and recognition result confidence level that the face affective characteristics recognition result includes；

Search affective characteristics image collection corresponding with the affective characteristics type；

From the affective characteristics image collection, affective characteristics image corresponding with the recognition result confidence level is picked out.

7. according to the method described in claim 3, it is characterized in that, described according to the face affective characteristics recognition result and institute Predicate sound affective characteristics recognition result searches corresponding affective characteristics image, including：

When the face affective characteristics recognition result and the speech emotional feature recognition result mismatch, according to the voice Affective characteristics recognition result searches corresponding affective characteristics image.

8. method according to any one of claim 1 to 7, which is characterized in that described to obtain the affective characteristics image Display location in currently playing picture frame, including：

Determine display location of the facial image in currently playing picture frame；

Inquire the relative position of the affective characteristics image and the facial image；

According to the display location and the relative position, determine the affective characteristics image in currently playing picture frame Display location.

9. according to the method described in claim 8, it is characterized in that, described according to the display location, in currently playing figure After rendering the affective characteristics image in frame, the method further includes：

In the picture frame of broadcasting, the movement locus of facial image is tracked；

According to the movement locus of tracking, the affective characteristics image is followed the facial image in the picture frame of broadcasting move.

10. a kind of image processing apparatus, which is characterized in that described device includes：

Picture frame acquisition module, for obtaining the picture frame acquired from reality scene；

Recognition result acquisition module, the face affective characteristics obtained for obtaining the facial image that identification described image frame includes Recognition result；

Display location acquisition module, for obtaining display location of the affective characteristics image in currently playing picture frame；

Rendering module, for according to the display location, the affective characteristics image to be rendered in currently playing picture frame.

11. device according to claim 10, which is characterized in that the recognition result acquisition module is additionally operable to described in adjustment The size of picture frame is to pre-set dimension；The direction of described image frame after adjustment is rotated to meeting affective characteristics and identifies condition Direction；Postrotational described image frame is sent to server；Receive the described image frame for transmission that the server returns Face affective characteristics recognition result.

12. device according to claim 10, which is characterized in that the recognition result acquisition module is additionally operable to extraction acquisition The voice data recorded when described image frame；Obtain the speech emotional feature recognition result for identifying that the voice data obtains；

The searching module be additionally operable to according to the face affective characteristics recognition result and the speech emotional feature recognition as a result, Search corresponding affective characteristics image.

13. device according to claim 12, which is characterized in that the searching module is additionally operable to when the face emotion is special When sign recognition result is matched with the speech emotional feature recognition result, phase is searched according to the face affective characteristics recognition result The affective characteristics image answered.

14. the device according to any one of claim 10 to 13, which is characterized in that the display location acquisition module is also For determining display location of the facial image in currently playing picture frame；Inquire the affective characteristics image with it is described The relative position of facial image；According to the display location and the relative position, determine the affective characteristics image current Display location in the picture frame of broadcasting.

15. a kind of electronic equipment, which is characterized in that including memory and processor, storing computer in the memory can Reading instruction, when the computer-readable instruction is executed by the processor so that the processor executes following steps：

Obtain the picture frame acquired from reality scene；