CN110378406A - Image emotional semantic analysis method, device and electronic equipment - Google Patents
Image emotional semantic analysis method, device and electronic equipment Download PDFInfo
- Publication number
- CN110378406A CN110378406A CN201910627791.3A CN201910627791A CN110378406A CN 110378406 A CN110378406 A CN 110378406A CN 201910627791 A CN201910627791 A CN 201910627791A CN 110378406 A CN110378406 A CN 110378406A
- Authority
- CN
- China
- Prior art keywords
- image
- theme
- network
- input
- analysis method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Transfer Between Computers (AREA)
Abstract
A kind of Image emotional semantic analysis method, device and electronic equipment are provided in the embodiment of the present disclosure, belong to technical field of image processing, this method comprises: inputting an image into first network to determine the theme of described image；Determine the image subject emotion attribute of described image theme；The image that image subject emotion attribute has been determined is input to the second network to quantify described image theme emotion attribute, wherein second network includes sub-network corresponding with each theme, and the image of corresponding theme is input into the sub-network of corresponding theme.By the processing scheme of the disclosure, the emotion of image subject can be not only obtained, but also can quantify to the emotion.
Description
Technical field
This disclosure relates to which technical field of image processing more particularly to a kind of Image emotional semantic analysis method, device and electronics are set
It is standby.
Background technique
In recent years, with the rapid development of multimedia technology, the media such as image, video have become in network social intercourse culture
A kind of mainstream, people more express the emotion or idea of oneself using multimedia content.
Since the value volume and range of product of image, video increases considerably, user directly searches interested content and needs to expend
A large amount of time and efforts, it is therefore desirable to a large amount of image, video content are screened, find out user it is interested in
Hold, efficiently and accurately to be pushed.But selected if the content pushed is not added, user may be made to touch all
Such as the content of the Negative sentiments of violence or large scale camera lens etc, lead to bad user experience.
For this reason, it may be necessary to which the emotion to image is analyzed, to pick out the image of negative emotion, and in turn to the figure of classification
The negative degree of picture carries out quantitative analysis.
Summary of the invention
In view of this, the embodiment of the present disclosure provides a kind of image quality measure method, apparatus and electronic equipment, at least partly
Solve problems of the prior art.
In a first aspect, the embodiment of the present disclosure provides a kind of Image emotional semantic analysis method, comprising:
First network is input an image into determine the theme of described image；
Determine the image subject emotion attribute of described image theme；
The image that image subject emotion attribute has been determined is input to the second network to quantify described image theme emotion category
Property,
Wherein, second network includes sub-network corresponding with each theme, and the image of corresponding theme is defeated
Enter the sub-network to corresponding theme.
According to a kind of specific implementation of the embodiment of the present disclosure, the first network is MobileNets network model,
The sub-network is multilayer perceptron, and the full articulamentum of the first network is connect with the input of second network.
According to a kind of specific implementation of the embodiment of the present disclosure, the first network that inputs an image into is described in determination
The theme of image, comprising:
Obtain the training data of each theme；
The training data of each theme is pre-processed；
Pretreated training data is input to the first network and carries out classification based training until network convergence；And
Described image is input to the first network to determine the theme of described image.
It is described that the image of image subject emotion attribute will be determined according to a kind of specific implementation of the embodiment of the present disclosure
The second network is input to quantify described image theme emotion attribute, comprising:
Obtain the training data of each theme；
The training data of each theme is pre-processed；
The pretreated training data of each theme is input to sub-network corresponding with the theme to be trained
Until network convergence；And
The image of corresponding theme is input to the sub-network of corresponding theme to quantify the theme emotion attribute of described image.
According to a kind of specific implementation of the embodiment of the present disclosure, the training data to each theme is located in advance
Reason, comprising:
Original image is zoomed in and out in such a way that Aspect Ratio is fixed, most will be scaled 256 by short side；
The image of 224 × 224 sizes is cut out from the image after scaling；
Mirror image switch is carried out to image；And
Every figure is subtracted each other with mean value figure to obtain the training data.
According to a kind of specific implementation of the embodiment of the present disclosure, the image subject emotion of the determining described image theme
Attribute, comprising:
Specific image theme is associated with specific image theme emotion attribute；Or
The image subject emotion attribute of described image theme is determined by searching for table.
It is described that the image of image subject emotion attribute will be determined according to a kind of specific implementation of the embodiment of the present disclosure
It is input to the second network, comprising:
According to the output of the first network, the top-N theme of described image is obtained, wherein described in the instruction of top-N theme
The top n theme of maximum probability, N are positive integer in all themes of image；And
Described image is separately input to sub-network corresponding with the top-N theme.
It is described that the image of image subject emotion attribute will be determined according to a kind of specific implementation of the embodiment of the present disclosure
It is input to the second network, further includes:
The probability of the top-N theme of described image is obtained according to the first network；
Obtain the theme that probability in the top-N theme is greater than predetermined threshold；And
Described image is separately input to son corresponding greater than the theme of predetermined threshold with probability in the top-N theme
Network.
According to a kind of specific implementation of the embodiment of the present disclosure, the training data for obtaining each theme, comprising:
The positive sample and negative sample of the training data of each theme are obtained, the positive sample includes the image of the theme,
The negative sample includes the image that normal picture and the first network are directed to the subject classification mistake.
According to a kind of specific implementation of the embodiment of the present disclosure, the training data for obtaining each theme, comprising:
The positive sample and negative sample of the training data of each theme are obtained, the positive sample includes that the first network is directed to
The correct image of subject classification, the negative sample include normal picture and the first network for the subject classification
The image of mistake.
Second aspect, the embodiment of the present disclosure additionally provide a kind of Image emotional semantic analytical equipment, which includes:
Theme determining module, image are input into the first network of the theme determining module to determine the master of described image
Topic；
Image subject emotion attribute determining module determines the image subject emotion attribute of described image theme；
Quantization modules, it is determined that the image of image subject emotion attribute be input into the second network of the quantization modules with
Quantify described image theme emotion attribute,
Wherein, second network includes sub-network corresponding with each theme, and the image of corresponding theme is defeated
Enter the sub-network to corresponding theme.
The third aspect, the embodiment of the present disclosure additionally provide a kind of electronic equipment, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out the image in aforementioned first aspect or any implementation of first aspect
Sentiment analysis method.
Fourth aspect, the embodiment of the present disclosure additionally provide a kind of non-transient computer readable storage medium, the non-transient meter
Calculation machine readable storage medium storing program for executing stores computer instruction, and the computer instruction is for making the computer execute aforementioned first aspect or the
Image emotional semantic analysis method in any implementation of one side.
5th aspect, the embodiment of the present disclosure additionally provide a kind of computer program product, which includes
The calculation procedure being stored in non-transient computer readable storage medium, the computer program include program instruction, when the program
When instruction is computer-executed, the computer is made to execute the image in aforementioned first aspect or any implementation of first aspect
Sentiment analysis method.
Image emotional semantic analytical plan in the embodiment of the present disclosure, including first network is input an image into the determination figure
The theme of picture；Determine the image subject emotion attribute of described image theme；The image that image subject emotion attribute has been determined is defeated
Enter to the second network to quantify described image theme emotion attribute, wherein second network includes corresponding with each theme
Sub-network, and the image of corresponding theme is input into the sub-network of corresponding theme.It, not only can be with by the scheme of the disclosure
The emotion of image subject is obtained, and the emotion can be quantified.
Detailed description of the invention
It, below will be to needed in the embodiment attached in order to illustrate more clearly of the technical solution of the embodiment of the present disclosure
Figure is briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present disclosure, for this field
For those of ordinary skill, without creative efforts, it can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is a kind of flow diagram for Image emotional semantic analysis method that the embodiment of the present disclosure provides；
Fig. 2 is the structural scheme of mechanism of the sub-network in the second network that the embodiment of the present disclosure provides；
Fig. 3 is the structural schematic diagram that the Image emotional semantic that the embodiment of the present disclosure provides analyzes network；
Fig. 4 is the pretreated flow diagram of training data that the embodiment of the present disclosure provides；
Fig. 5 is a kind of structural schematic diagram for Image emotional semantic analytical equipment that the embodiment of the present disclosure provides；
Fig. 6 is the electronic equipment schematic diagram that the embodiment of the present disclosure provides.
Specific embodiment
The embodiment of the present disclosure is described in detail with reference to the accompanying drawing.
Illustrate embodiment of the present disclosure below by way of specific specific example, those skilled in the art can be by this specification
Disclosed content understands other advantages and effect of the disclosure easily.Obviously, described embodiment is only the disclosure
A part of the embodiment, instead of all the embodiments.The disclosure can also be subject to reality by way of a different and different embodiment
It applies or applies, the various details in this specification can also be based on different viewpoints and application, in the spirit without departing from the disclosure
Lower carry out various modifications or alterations.It should be noted that in the absence of conflict, the feature in following embodiment and embodiment can
To be combined with each other.Based on the embodiment in the disclosure, those of ordinary skill in the art are without creative efforts
Every other embodiment obtained belongs to the range of disclosure protection.
It should be noted that the various aspects of embodiment within the scope of the appended claims are described below.Ying Xian
And be clear to, aspect described herein can be embodied in extensive diversified forms, and any specific structure described herein
And/or function is only illustrative.Based on the disclosure, it will be understood by one of ordinary skill in the art that one described herein
Aspect can be independently implemented with any other aspect, and can combine the two or both in these aspects or more in various ways.
For example, carry out facilities and equipments in terms of any number set forth herein can be used and/or practice method.In addition, can make
With other than one or more of aspect set forth herein other structures and/or it is functional implement this equipment and/or
Practice the method.
It should also be noted that, diagram provided in following embodiment only illustrates the basic structure of the disclosure in a schematic way
Think, component count, shape and the size when only display is with component related in the disclosure rather than according to actual implementation in schema are drawn
System, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel can also
It can be increasingly complex.
In addition, in the following description, specific details are provided for a thorough understanding of the examples.However, fields
The skilled person will understand that the aspect can be practiced without these specific details.
The embodiment of the present disclosure provides a kind of Image emotional semantic analysis method.Image emotional semantic analysis method provided in this embodiment can
To be executed by a computing device, which can be implemented as software, or be embodied as the combination of software and hardware, the meter
It calculates device and can integrate and be arranged in server, terminal device etc..
Referring to Fig. 1, a kind of Image emotional semantic analysis method of embodiment of the present disclosure offer, comprising:
S100: first network is input an image into determine the theme of described image.
Generally, for an image, it includes specific themes, for example, trees theme, sky theme, traffic accident master
It inscribes, theme of fighting, human skeleton theme etc..
In the embodiments of the present disclosure, the theme of image is determined first with first network.First network for example can be
MobileNets network model.
MobileNets network model is based on a kind of streamlined structure and separates convolution using depth to construct light-duty weight
Deep neural network, compared with other classificatory popular network models of ImageNet, MobileNets shows very strong
Performance.
MobileNets network model separates convolution (depthwise sparable convolutions) with depth level
Traditional Standard convolution (standard convolutions) is substituted to solve the computational efficiency of convolutional network and asking for parameter amount
Topic.MobileNets network model is based on depth level and separates convolution (depthwise sparable convolutions),
Standard convolution can be resolved into one convolution of a depth convolution sum (1 × 1 convolution kernel).Depth convolution is by each convolution kernel
It is applied to each channel, and 1 × 1 convolution is used to combine the output of channel convolution.It is deep using 3 × 3 convolution kernel
Degree grade, which separates convolution, which compares Standard convolution, can reduce about 9 times of calculation amount.
Therefore, in the present invention, image is input into the theme that image is determined in MobileNets network model.It should
Although also can be used such as note that in the present invention, the theme of image is determined using MobileNets network model
Other sorter network models of GoogleNet and VGG16 etc determine the theme of image.
S200: the image subject emotion attribute of described image theme is determined.
After the theme that inputted image has been determined in the network model by MobileNets etc, the image is determined
Image subject emotion attribute.
Such as image of trees theme, sky theme etc can bring positive, positive emotion to people, and such as traffic accident
The image of theme etc can bring negative, passive emotion to people.
In the present example embodiment, main in the theme emotion attribute of the theme for determining image and then the determining image
Emotion attribute is inscribed for example including passive, positive and common.Although should be appreciated that in the present example embodiment, theme emotion category
Property include it is passive, positive and common, but the invention is not restricted to this, and the theme emotion attribute of the embodiment of the present disclosure for example can be with
Theme emotion attribute including other very passive, very positive etc. classifications.
Determine that image subject emotion attribute for example can be by by specific image theme and specific image theme emotion attribute
It is associated.It specifically, can be directly by specific master such as after the theme for obtaining image by MobileNets network model
Topic is classified as one of passive, positive and common.For example, if the image subject obtained by MobileNets network model
For ocean, then what can directly be sorted out is positive.That is institute can directly pass through step S100 in the embodiments of the present disclosure
In MobileNets network model obtain image image subject emotion attribute.
Alternatively, the image subject emotion attribute of image can be determined by individual module.For example, can pass through
Image subject emotion attribute determining module determines the emotion attribute of the image for obtaining theme.It, can be with according to one embodiment
The look-up table that image subject and emotion attribute are stored in image subject emotion attribute determining module, in this way, according to step
After S100 obtains the theme of image, its emotion attribute can be obtained according to the look-up table.It, can be with people according to another embodiment
Building site determines the emotion attribute of the image of the theme.According to another embodiment, can by step S100 such as
The network model of MobileNets network model etc judges automatically image subject emotion attribute, in the embodiments of the present disclosure,
The image subject emotion attribute of inputted image is judged by housebroken network.
S300: the image that image subject emotion attribute has been determined is input to the second network to quantify described image theme feelings
Feel attribute.Second network includes sub-network corresponding with each theme, and the image of corresponding theme is input into phase
Answer the sub-network of theme.
The theme of image is being determined by first network, thus after solving the diverse problems of image, it is also necessary to solve
Certainly subjectivity problem.That is, for same subject, for example, traffic accident, also can to the degree of user's bring Negative Affect
It is had differences because of the miserable degree of traffic accident.Therefore, it is also desirable to quantify to picture material.
In the embodiments of the present disclosure, individual network is set for each theme, with the image by the network to the theme
Content is quantified.Specifically, for example, individually having the second network of traffic accident label for the image setting of traffic accident theme
(sub-network), there is the second network of traffic accident label to quantify to the content of the image of traffic accident theme for this.In second network
Sub-network for example can be multilayer perceptron, be a kind of feed forward Artificial Network model, by multiple data sets of input
It is mapped on the data set of single output.
Referring to fig. 2, it illustrates the structural schematic diagrams according to the multilayer perceptron of the embodiment of the present disclosure.It includes input
Layer, hidden layer and output layer, other than input layer, every layer of activation primitive can for example use sigmod.In addition, in the disclosure
In embodiment, using two layers of hidden layer (full connection), the number of plies is respectively 2048 and 1024.
The detail of multilayer perceptron for example may refer tohttps://www.cnblogs.com/ooon/p/ 5577241.html, entire contents are incorporated herein by reference.
In this way, being determined that the image of theme (for example, fire) passes through sub-network corresponding with the theme by first network
(multilayer perceptron) is further quantified.
It in the embodiments of the present disclosure, for example, can be using the probability value of the softmax layer of sub-network as to image subject feelings
The quantized result of sense.
In this way, can not only obtain the feelings of image subject by the Image emotional semantic analysis method according to the embodiment of the present disclosure
Sense, and the emotion can be quantified.
According to the embodiment of the present disclosure specific embodiment, the first network is MobileNets network model,
The sub-network is multilayer perceptron, and the full articulamentum of the first network is connect with the input of the sub-network.
Referring to Fig. 3, it illustrates the structural schematic diagrams that network is analyzed according to the Image emotional semantic of the embodiment of the present disclosure.
In the embodiments of the present disclosure, the image of input is divided into 27 negative themes by the MobileNets network model
And other normal themes amount to 28 themes.
Although in the present example embodiment, the image of input is divided into 27 negative themes and other normal themes,
Input picture can be divided into any number of negative theme by however, the present invention is not limited thereto, the MobileNets network model
With any number of front theme.It so, it is possible preliminarily to solve the problems, such as that image is multifarious.
Specifically, table 1 shows 27 negative themes according to the embodiment of the present disclosure.
Table 1: the negative theme of the embodiment of the present disclosure and division
In addition, in the embodiments of the present disclosure, in order to be determined that the image of theme carries out further quantitative evaluation, by institute
The full articulamentum for stating first network is connect with the input of the sub-network, is commented with carrying out quantization to image using the sub-network
Estimate.
According to the embodiment of the present disclosure specific embodiment, the first network that inputs an image into is described in determination
The theme of image, comprising:
Obtain the training data of each theme.
The training data of each theme is pre-processed；
Pretreated training data is input to the first network and carries out classification based training until network convergence；And
Described image is input to the first network to determine the theme of described image.
In order to classify to input picture to obtain its theme, 40000 are being chosen in the embodiments of the present disclosure commonly just
The normal training data of sample and the negative theme sample of 27 classes as first network.Theme sample negative for 27 classes, every class can
To select 700~7000 samples, such as theme negative for fire, 5000 fire theme samples can be chosen.
Although should be appreciated that in the embodiments of the present disclosure, with certain amount of common sample and negative theme pattern representation
The present invention, but the invention is not restricted to this, common sample and negative theme sample can choose as other suitable data.
For selected sample, for example, can method according to Fig.4, pre-processed.
S401: original input picture is zoomed in and out in such a way that Aspect Ratio is fixed, will most short side be scaled
256。
In the embodiments of the present disclosure, the scaling of image, which refers to the process of, is adjusted the size of digital picture.Image is put
Interpolated value method can be used greatly, i.e., inserted between pixel using suitable interpolation algorithm on the basis of original image pixel
Enter new element, interpolation algorithm for example including the image interpolation algorithm based on edge, the image interpolation algorithm based on region and
The algorithm of other known or following exploitations.In addition, downscaled images for example can be using CV_INETR_AREA come interpolation.
In addition, although original input picture is zoomed to most short side in such a way that Aspect Ratio is fixed in the above description
It is 256, but the embodiment of the present disclosure is without being limited thereto, but most short side can be zoomed into other suitable numerical value, and scale
Mode can be the other modes except Aspect Ratio fixed form.But for convenience of description, below with will be original defeated
Enter image and zoom in such a way that Aspect Ratio is fixed for most short side is 256 to be illustrated.
S402: after original input picture to be zoomed to most short side and is 256, from the upper left of image, upper right, lower-left, the right side
Lower and intermediate shear goes out the figure of 224 × 224 sizes.
Since the input of quality evaluation network model is 224 × 224, therefore, it is necessary to 224 × 224 are cut out from image
The figure of size.It is understood that the position for the image sheared be not limited to the upper left of image, upper right, lower-left, bottom right and in
Between, but can be other suitable positions.
S403: image is subjected to mirror image switch.
In the embodiments of the present disclosure, the problem of due to the image from smart phone in the prevalence of rotation or mirror image,
Therefore it needs to be operated such that image is in normal condition by carrying out mirror image switch.
S404: every figure is subtracted each other with mean value figure.
Due in most cases, and it is not concerned with the whole light levels of input picture.More precisely, image block
Average brightness value information is not key message, it is possible to subtract this value to carry out mean value reduction.Specifically, in disclosure reality
It applies in example, every figure is subtracted each other with mean value figure, to be normalized.
After obtaining training data by method as described above, it is straight that classification based training is carried out to MobileNets network model
To network convergence.In addition, in the training process, being trained using batch gradient descent method.Criticize the example of gradient descent method for example
Including batch gradient descent method BGD, small lot gradient descent method MBGD and stochastic gradient descent method SGD etc..
After obtaining housebroken MobileNets network model, trained first net can be input to image
Network is to determine the theme of described image.
It is described that the image of image subject emotion attribute will be determined according to the embodiment of the present disclosure specific embodiment
The second network is input to quantify described image theme emotion attribute, comprising:
Obtain the training data of each theme；
The training data of each theme is pre-processed；
The pretreated training data of each theme is input to sub-network corresponding with the theme to be trained
Until network convergence；And
The image of corresponding theme is input to the sub-network of corresponding theme to quantify the theme emotion attribute of described image.
In the embodiments of the present disclosure, individual network model, and the acquisition of training data, instruction are trained for each theme
The training of the pretreatment and network model of practicing data can be similar to first network model, and details are not described herein.
It in the embodiments of the present disclosure, such as can be using the probability of the softmax of sub-network layer as the theme feelings of the image
Feel the quantized value of attribute.
It is described that the image of image subject emotion attribute will be determined according to the embodiment of the present disclosure specific embodiment
It is input to the second network, comprising:
According to the output of the first network, the top-N theme of described image is obtained, wherein described in the instruction of top-N theme
The top n theme of maximum probability, N are positive integer in all themes of image；And
Described image is separately input to sub-network corresponding with the top-N theme.
When the input picture is divided into multiple images theme by the first network, according to the first network
Softmax layers of probability selects theme label of the theme (top-N theme) of predetermined quantity as described image.In addition, this public affairs
Embodiment is opened also according to the threshold value of setting to determine whether the image to be input in corresponding second network.
In general, an image may be comprising multiple themes (also including sky for example, both including trees), this
In the case of, which may be divided into multiple themes by the first network.In this case, it is called together to be promoted
It returns, the first network is only using themes several before maximum probability as the theme label of the image.
Specifically, in the embodiments of the present disclosure, using two theme labels of softmax layers of maximum probability as the input
The theme label of image.That is, the label of the first network output top-2.
It is not that can all enter corresponding second network, but go back after obtaining the top-2 label of input picture
It must satisfy specific condition.Specifically, in the embodiments of the present disclosure, it is desirable that softmax layers corresponding with the top-2 label
Probability is greater than predetermined threshold.In other words, softmax layers will be met in two theme labels of softmax layers of maximum probability
Probability be greater than predetermined threshold image be input in corresponding second network.It so, it is possible to further increase and recall.
According to the embodiment of the present disclosure specific embodiment, the training data for obtaining each theme, including obtain
The positive sample and negative sample of the training data of each theme are taken, the positive sample includes the image of the theme, the negative sample
The image of the subject classification mistake is directed to including normal picture and the first network.Alternatively, the positive sample packet
The first network is included for the correct image of the subject classification, the negative sample includes normal picture and first net
Network is directed to the image of the subject classification mistake.
As described above, in the embodiments of the present disclosure, for each theme one network (the second network, refined net) of training.
During training each second network, training data may include under image and the theme label under the theme label
Normal picture.Image and normal picture under the theme label can be obtained by manual type.Alternatively, the theme label
Under image and normal picture can also be obtained by the first network.That is, for example can be defeated by first network
Positive sample of the image of fire theme out as the second network training data, and using other images as the second network training
The negative sample of data.
However, since first network possibly can not completely correctly classify to the theme of image, in disclosure reality
It applies in example, also using the image of the classification error of first network as the negative sample of the training data of the second network.
800 fire figures are obtained by first network by taking fire theme as an example, such as 1000 input pictures, but
It is that only 600 fire images are true fire images, remaining 200 are non-fire images.Then this 200 non-fire figures
As the image of the as classification error of first network, also referred to as badcase.Then in the embodiments of the present disclosure, by this 200 Zhang Feihuo
Negative sample of the calamity image as the second network, and can be using 600 true fire images as the positive sample of the second network.
It so, it is possible the precision for further promoting network.
Fig. 5 shown device 500 can it is corresponding execute above method embodiment in content comprising:
Theme determining module 501, image are input into the first network of the theme determining module to determine described image
Theme；
Image subject emotion attribute determining module 502 determines the image subject emotion attribute of described image theme；
Quantization modules 503 are determined that the image of image subject emotion attribute is input into the second net of the quantization modules
Network to quantify described image theme emotion attribute,
Wherein, second network includes sub-network corresponding with each theme, and the image of corresponding theme is defeated
Enter the sub-network to corresponding theme.
The part that the present embodiment is not described in detail, referring to the content recorded in above method embodiment, details are not described herein.
Referring to Fig. 6, the embodiment of the present disclosure additionally provides a kind of electronic equipment 60, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out Image emotional semantic analysis method in preceding method embodiment.
The embodiment of the present disclosure additionally provides a kind of non-transient computer readable storage medium, and the non-transient computer is readable to deposit
Storage media stores computer instruction, and the computer instruction is for making the computer execute Image emotional semantic point in preceding method embodiment
Analysis method.
The embodiment of the present disclosure additionally provides a kind of computer program product, and the computer program product is non-temporary including being stored in
Calculation procedure on state computer readable storage medium, the computer program include program instruction, when the program instruction is calculated
When machine executes, the computer is made to execute Image emotional semantic analysis method in preceding method embodiment.
Below with reference to Fig. 6, it illustrates the structural schematic diagrams for the electronic equipment 60 for being suitable for being used to realize the embodiment of the present disclosure.
Electronic equipment in the embodiment of the present disclosure can include but is not limited to such as mobile phone, laptop, Digital Broadcasting Receiver
Device, PDA (personal digital assistant), PAD (tablet computer), PMP (portable media player), car-mounted terminal are (such as vehicle-mounted
Navigation terminal) etc. mobile terminal and such as number TV, desktop computer etc. fixed terminal.Electronics shown in Fig. 6
Equipment is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in fig. 6, electronic equipment 60 may include processing unit (such as central processing unit, graphics processor etc.) 601,
It can be loaded into random access storage according to the program being stored in read-only memory (ROM) 602 or from storage device 608
Program in device (RAM) 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with the behaviour of electronic equipment 60
Various programs and data needed for making.Processing unit 601, ROM 602 and RAM 603 are connected with each other by bus 604.It is defeated
Enter/export (I/O) interface 605 and is also connected to bus 604.
In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, figure
As the input unit 606 of sensor, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaking
The output device 607 of device, vibrator etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.It is logical
T unit 609 can permit electronic equipment 60 and wirelessly or non-wirelessly be communicated with other equipment to exchange data.Although showing in figure
The electronic equipment 60 with various devices is gone out, it should be understood that being not required for implementing or having all devices shown.
It can alternatively implement or have more or fewer devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608
It is mounted, or is mounted from ROM 602.When the computer program is executed by processing unit 601, the embodiment of the present disclosure is executed
Method in the above-mentioned function that limits.
It should be noted that the above-mentioned computer-readable medium of the disclosure can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In open, computer-readable signal media may include in a base band or as the data-signal that carrier wave a part is propagated,
In carry computer-readable program code.The data-signal of this propagation can take various forms, including but not limited to
Electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable and deposit
Any computer-readable medium other than storage media, the computer-readable signal media can send, propagate or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF (radio frequency) etc. are above-mentioned
Any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.
Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are by the electricity
When sub- equipment executes, so that the electronic equipment: obtaining at least two internet protocol addresses；Send to Node evaluation equipment includes institute
State the Node evaluation request of at least two internet protocol addresses, wherein the Node evaluation equipment is internet from described at least two
In protocol address, chooses internet protocol address and return；Receive the internet protocol address that the Node evaluation equipment returns；Its
In, the fringe node in acquired internet protocol address instruction content distributing network.
Alternatively, above-mentioned computer-readable medium carries one or more program, when said one or multiple programs
When being executed by the electronic equipment, so that the electronic equipment: receiving the Node evaluation including at least two internet protocol addresses and request；
From at least two internet protocol address, internet protocol address is chosen；Return to the internet protocol address selected；Wherein,
The fringe node in internet protocol address instruction content distributing network received.
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof
Machine program code, above procedure design language include object oriented program language-such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present disclosure can be realized by way of software, can also be by hard
The mode of part is realized.Wherein, the title of unit does not constitute the restriction to the unit itself under certain conditions, for example, the
One acquiring unit is also described as " obtaining the unit of at least two internet protocol addresses ".
It should be appreciated that each section of the disclosure can be realized with hardware, software, firmware or their combination.
The above, the only specific embodiment of the disclosure, but the protection scope of the disclosure is not limited thereto, it is any
Those familiar with the art is in the technical scope that the disclosure discloses, and any changes or substitutions that can be easily thought of, all answers
Cover within the protection scope of the disclosure.Therefore, the protection scope of the disclosure should be subject to the protection scope in claims.
Claims (13)
1. a kind of Image emotional semantic analysis method characterized by comprising
First network is input an image into determine the theme of described image；
Determine the image subject emotion attribute of described image theme；
The image for having determined image subject emotion attribute is input to the second network to quantify described image theme emotion attribute,
Wherein, second network includes sub-network corresponding with each theme, and the image of corresponding theme is input into
The sub-network of corresponding theme.
2. Image emotional semantic analysis method according to claim 1, which is characterized in that the first network is MobileNets
Network model, the sub-network are multilayer perceptron, and the input of the full articulamentum of the first network and the sub-network
Connection.
3. Image emotional semantic analysis method according to claim 1, which is characterized in that described to input an image into first network
To determine the theme of described image, comprising:
Obtain the training data of each theme；
The training data of each theme is pre-processed；
Pretreated training data is input to the first network and carries out classification based training until network convergence；And
Described image is input to the first network to determine the theme of described image.
4. Image emotional semantic analysis method according to claim 1, which is characterized in that described that image subject emotion will be determined
The image of attribute is input to the second network to quantify described image theme emotion attribute, comprising:
Obtain the training data of each theme；
The training data of each theme is pre-processed；
By the pretreated training data of each theme be input to sub-network corresponding with the theme be trained until
Network convergence；And
The image of corresponding theme is input to the sub-network of corresponding theme to quantify the theme emotion attribute of described image.
5. Image emotional semantic analysis method according to claim 3 or 4, which is characterized in that the training to each theme
Data are pre-processed, comprising:
Original image is zoomed in and out in such a way that Aspect Ratio is fixed, most will be scaled 256 by short side；
The image of 224 × 224 sizes is cut out from the image after scaling；
Mirror image switch is carried out to image；And
Every figure is subtracted each other with mean value figure to obtain the training data.
6. Image emotional semantic analysis method according to claim 1, which is characterized in that the figure of the determining described image theme
As theme emotion attribute, comprising:
Specific image theme is associated with specific image theme emotion attribute；Or
The image subject emotion attribute of described image theme is determined by the first network.
7. Image emotional semantic analysis method according to claim 1, which is characterized in that described that image subject emotion will be determined
The image of attribute is input to the second network, comprising:
According to the output of the first network, the top-N theme of described image is obtained, wherein top-N theme indicates described image
All themes in maximum probability top n theme, N is positive integer；And
Described image is separately input to sub-network corresponding with the top-N theme.
8. Image emotional semantic analysis method according to claim 7, which is characterized in that described that image subject emotion will be determined
The image of attribute is input to the second network, further includes:
The probability of the top-N theme of described image is obtained according to the first network；
Obtain the theme that probability in the top-N theme is greater than predetermined threshold；And
Described image is separately input to subnet corresponding greater than the theme of predetermined threshold with probability in the top-N theme
Network.
9. Image emotional semantic analysis method according to claim 4, which is characterized in that the training number for obtaining each theme
According to, comprising:
The positive sample and negative sample of the training data of each theme are obtained, the positive sample includes the image of the theme, described
Negative sample includes the image that normal picture and the first network are directed to the subject classification mistake.
10. Image emotional semantic analysis method according to claim 4, which is characterized in that the training for obtaining each theme
Data, comprising:
The positive sample and negative sample of the training data of each theme are obtained, the positive sample includes the first network for described
The correct image of subject classification, the negative sample include normal picture and the first network for the subject classification mistake
Image.
11. a kind of Image emotional semantic analytical equipment characterized by comprising
Theme determining module, image are input into the first network of the theme determining module to determine the theme of described image；
Image subject emotion attribute determining module determines the image subject emotion attribute of described image theme；
Quantization modules are determined that the image of image subject emotion attribute is input into the second network of the quantization modules to quantify
Described image theme emotion attribute,
Wherein, second network includes sub-network corresponding with each theme, and the image of corresponding theme is input into
The sub-network of corresponding theme.
12. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
At least one processor；And
The memory being connect at least one described processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one described processor, and described instruction is by described at least one
It manages device to execute, so that at least one described processor is able to carry out the analysis of Image emotional semantic described in aforementioned any claim 1-10
Method.
13. a kind of non-transient computer readable storage medium, which stores computer instruction,
The computer instruction is for making the computer execute Image emotional semantic analysis method described in aforementioned any claim 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627791.3A CN110378406A (en) | 2019-07-12 | 2019-07-12 | Image emotional semantic analysis method, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627791.3A CN110378406A (en) | 2019-07-12 | 2019-07-12 | Image emotional semantic analysis method, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110378406A true CN110378406A (en) | 2019-10-25 |
Family
ID=68252847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910627791.3A Pending CN110378406A (en) | 2019-07-12 | 2019-07-12 | Image emotional semantic analysis method, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378406A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114693809A (en) * | 2022-02-25 | 2022-07-01 | 智己汽车科技有限公司 | Method and equipment for switching automotive electronic interior style themes |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295568A (en) * | 2016-08-11 | 2017-01-04 | 上海电力学院 | The mankind's naturalness emotion identification method combined based on expression and behavior bimodal |
| CN107341506A (en) * | 2017-06-12 | 2017-11-10 | 华南理工大学 | A kind of Image emotional semantic classification method based on the expression of many-sided deep learning |
| CN108427920A (en) * | 2018-02-26 | 2018-08-21 | 杭州电子科技大学 | A kind of land and sea border defense object detection method based on deep learning |
| CN109685110A (en) * | 2018-11-28 | 2019-04-26 | 北京陌上花科技有限公司 | Training method, image classification method and device, the server of image classification network |
-
2019
- 2019-07-12 CN CN201910627791.3A patent/CN110378406A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295568A (en) * | 2016-08-11 | 2017-01-04 | 上海电力学院 | The mankind's naturalness emotion identification method combined based on expression and behavior bimodal |
| CN107341506A (en) * | 2017-06-12 | 2017-11-10 | 华南理工大学 | A kind of Image emotional semantic classification method based on the expression of many-sided deep learning |
| CN108427920A (en) * | 2018-02-26 | 2018-08-21 | 杭州电子科技大学 | A kind of land and sea border defense object detection method based on deep learning |
| CN109685110A (en) * | 2018-11-28 | 2019-04-26 | 北京陌上花科技有限公司 | Training method, image classification method and device, the server of image classification network |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114693809A (en) * | 2022-02-25 | 2022-07-01 | 智己汽车科技有限公司 | Method and equipment for switching automotive electronic interior style themes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110381368A (en) | Video cover generation method, device and electronic equipment | |
| US11182447B2 (en) | Customized display of emotionally filtered social media content | |
| CN110401844A (en) | Generation method, device, equipment and the readable medium of net cast strategy | |
| CN110267097A (en) | Video pushing method, device and electronic equipment based on characteristic of division | |
| CN110378474A (en) | Fight sample generating method, device, electronic equipment and computer-readable medium | |
| CN110363753A (en) | Image quality measure method, apparatus and electronic equipment | |
| CN109598304B (en) | Disaggregated model calibration method, device, equipment and readable medium | |
| CN110378410A (en) | Multi-tag scene classification method, device and electronic equipment | |
| CN110222726A (en) | Image processing method, device and electronic equipment | |
| CN110278447A (en) | Video pushing method, device and electronic equipment based on continuous feature | |
| CN110573837A (en) | Navigation method, navigation device, storage medium and server | |
| CN109828906A (en) | UI automated testing method, device, electronic equipment and storage medium | |
| CN109993638A (en) | Method, apparatus, medium and the electronic equipment of Products Show | |
| CN110399847A (en) | Extraction method of key frame, device and electronic equipment | |
| CN114417174B (en) | Content recommendation method, device, equipment and computer storage medium | |
| CN111984803A (en) | Multimedia resource processing method and device, computer equipment and storage medium | |
| CN110069997B (en) | Scene classification method and device and electronic equipment | |
| CN110362698A (en) | A kind of pictorial information generation method, device, mobile terminal and storage medium | |
| CN110442803A (en) | Data processing method, device, medium and the calculating equipment executed by calculating equipment | |
| CN110059748A (en) | Method and apparatus for output information | |
| CN110378406A (en) | Image emotional semantic analysis method, device and electronic equipment | |
| CN116775915A (en) | Resource recommendation method, recommendation prediction model training method, device and equipment | |
| CN109977925A (en) | Expression determines method, apparatus and electronic equipment | |
| CN110378936A (en) | Optical flow computation method, apparatus and electronic equipment | |
| CN112036519B (en) | Multi-bit sigmoid-based classification processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191025 |