CN109858482A - A kind of image key area detection method and its system, terminal device - Google Patents
A kind of image key area detection method and its system, terminal device Download PDFInfo
- Publication number
- CN109858482A CN109858482A CN201910042460.3A CN201910042460A CN109858482A CN 109858482 A CN109858482 A CN 109858482A CN 201910042460 A CN201910042460 A CN 201910042460A CN 109858482 A CN109858482 A CN 109858482A
- Authority
- CN
- China
- Prior art keywords
- image
- key area
- gradient
- activation
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of image key area detection method and its systems, terminal device, it trains a sorter network to commodity to be sorted using deep learning method, forward inference is carried out to commodity to be sorted with the neural network, obtain the classification and activation figure of commodity, it is based further on the gradient with the coding of the categorical match of commodity as neural network, carry out backpropagation, obtain the gradient activation figure that can reflect the key area of image to be processed, so as to be rejected with the little insignificant region of the required classification information degree of correlation, to reduce the background interference of original image to be processed, and the key area needed for obtaining.It is different from the method that existing needs are manually marked, technical solution provided by the present invention, it can be obtained by the self training of neural network, the key area that detection method and its system in this way obtains can have more preferably robustness compared to the key area by artificially marking by hand.
Description
[technical field]
The present invention relates to artificial intelligence field, in particular to a kind of image key area detection method and its system, end
End equipment.
[background technique]
With the sharp increase of image procossing amount, how image is efficiently marked, classification processing, increasingly obtain people
Concern.In the prior art, it is mainly based upon the priori knowledge of people, the key area in image, key point are carried out manual
Mark, mark key content neural network can be allowed to be learnt.But the division of such key area needs to expend
A large amount of human and material resources and time cost, and due to the difference of personal experience, there is also the problems of mark inaccuracy, especially
It is for package goods as bottle, chest, sack or the box with similar packaging label etc. with similar features
When, since feature difference is smaller, it is difficult to define suitable key point or key area according to personal experience, such as
Some marks classification during, often will appear by with similar packaging label chest or box select it is identical
Key area.
[summary of the invention]
It is difficult to the technical issues of defining suitable key area to solve the prior art, the present invention provides a kind of image pass
Keypad area detecting method and its system, terminal device.
The present invention is in order to solve the above technical problems, offer the following technical solution: a kind of image key area detection method,
It comprises the following steps that step S1, and training obtains a neural network；Step S2 inputs an image to be processed, is based on the nerve
Network handles handle image and carry out forward inference, to obtain the classification results of commodity and required activation figure in image to be processed
(activation map)；Classification results are converted to coding result by step S3, and using coding result as the neural network
In gradient carry out backpropagation, with obtain needed for gradient map (gradient map)；And step S4, by activation figure and gradient map
Synthesis show that gradient activation figure (gradient activation map), the gradient activation figure can represent image to be processed
Key area.
Preferably, after above-mentioned steps S4, include the following steps: step S5, gradient activation figure is converted into thermodynamic chart；
And step S6, thermodynamic chart and image to be processed are overlapped, key area is obtained, optimizes new classification based on the key area
Neural network.
Preferably, in above-mentioned steps S6, optimized based on the key area new Classification Neural specifically include it is following
Step: key area being cut out to obtain new cutting image, optimizes new Classification Neural to continue training；Or it will be wait locate
It manages and increases a channel in image to store the thermodynamic chart, and continue training and optimize new Classification Neural.
Preferably, after above-mentioned steps S1, before carrying out step S2, it is also necessary to which the neural network in judgment step S1 is
No training convergence, if so, S2 is entered step, if it is not, then return step S1 continues to train；Activation obtained in step s 2
Figure is specially to carry out in forward inference in neural network, the activation figure on the last one convolutional layer；
Preferably, in above-mentioned steps S3, coding result is converted to classification results, is specifically included: to classification results into
Row one hot encodes (efficient coding), the corresponding one hot coding of each classification.
Preferably, activation figure and gradient map synthesis are obtained into gradient activation figure in above-mentioned steps S4, specifically included as follows
Step: the average value of the corresponding gradient map of the last one convolutional layer is calculated；By the logical of the average value of acquisition and corresponding convolutional layer
Road number, which is multiplied, obtains the product value of corresponding gradient map；Product value is weighted and averaged operation with the activation figure respectively, to obtain
It obtains required gradient and activates figure.
The present invention in order to solve the above technical problems, provide a kind of another technical solution: image key area detection system,
Comprising: which training module, is configurable for training and obtains a neural network；Activation module is obtained, is configured as using
In inputting an image to be processed, forward inference is carried out to image to be processed based on the neural network, to obtain in image to be processed
The classification results of commodity and required activation figure；Gradient module is obtained, is configurable for being converted to classification results into coding
As a result, and using coding result as in the neural network gradient carry out backpropagation, to obtain required gradient map；And it obtains
Gradient activates module, is configurable for obtaining activation figure and gradient map synthesis into gradient activation figure, the gradient activation
Figure can represent the key area of image to be processed.
Preferably, described image key area detection system further include: judgment module, for judging in the training pattern
Whether the neural network that training obtains trains convergence；Image conversion module, for gradient activation figure to be converted to thermodynamic chart；And it obtains
Key area module is taken, obtains key area for thermodynamic chart and image to be processed to be overlapped, and be based on the key area
To optimize new Classification Neural.
Preferably, described image conversion module further include: average calculation unit, for calculating the corresponding ladder of different convolutional layers
Spend the average value of figure；Product computing unit, for by the average value of the corresponding gradient map of different convolutional layers and different convolutional layers
Port number be multiplied and obtain the product value of corresponding gradient map；And weighted average calculation unit, for by product value respectively with activation
Figure is weighted and averaged operation, activates figure with the gradient needed for obtaining.
The present invention is in order to solve the above technical problems, provide a kind of another technical solution: terminal device, it is characterised in that:
The terminal device includes storage unit and processing unit, and for the storage unit for storing computer program, the processing is single
Member is for executing the step in above-mentioned image key area detection method by the computer program that the storage unit stores.
Compared with prior art, described image key area detection method and its system, terminal provided by the present invention are set
It is standby have it is following the utility model has the advantages that
Image key area detection method and its system provided by the present invention, using deep learning method to be sorted
Commodity train a sorter network, carry out forward inference to commodity to be sorted with the neural network, obtain the classification of commodity
Scheme with activation, is based further on the gradient with the coding of the categorical match of commodity as neural network, carries out backpropagation, obtain
Can reflect the gradient activation figure of the key area of image to be processed, so as to by with little non-of the required classification information degree of correlation
Important area is rejected, so as to effectively reduce the background interference of original image to be processed, so as to further concentrate on discrimination
Higher key area.
It is different from the method that existing needs are manually marked, image key area detection method provided by the present invention
And its system, it can be obtained by the self training of neural network, the key that detection method and its system in this way obtains
Region can have more preferably robustness (Robust) compared to the key area by artificially marking by hand.
Further, in the present invention, the gradient activation figure of acquisition is converted into thermodynamic chart, to realize neural network
Visualization, obtain key area of the neural network for such commodity area of interest, as such commodity.It in this way can be with
It is automatically performed the detection to commodity key area, and can use these key areas, further improves sorter network
Performance.
And in the present invention, the key area based on acquisition is placed into new Classification Neural and is trained, can be into
The new Classification Neural of one-step optimization, to facilitate the performance of fast lifting neural network, and such automatic detection
Process can save a large amount of time and human cost.
In order to further increase the robustness of trained neural network, summarize in described image key area detection method, also
Including judging whether neural network trains convergence, if not converged, continue with data set and neural network is trained, thus
It can guarantee accuracy of the neural network of training acquisition to commodity classification.
In the present invention, it is encoded using classification results of the one hot coding mode to commodity, it can be directly by commodity
Classification results are converted into corresponding multi-C vector, and one hot is encoded the gradient as neural network, to carry out backpropagation,
To obtain its corresponding gradient map, the selection based on one hot coding obtains required key area, key area and one
Corresponding classification is related in hot coding, and the selection based on one hot coding can further improve to be examined from the image to be processed
Survey the flexibility ratio and accuracy of key area.
In the present invention, the gradient map for obtaining activation figure and its backpropagation acquisition using neural network forward inference is comprehensive
To obtain gradient activation figure, can reflect in image to be processed select merchandise classification important area, without artificially into
Rower note can greatly reduce human and material resources and time cost, also can avoid due to manually marking image key area inaccuracy
Problem.
In the present invention, key area is cut out to obtain new cutting image or will increase by one in image to be processed
Channel is used equally for advanced optimizing new classification mind to store the processing mode of key area of two kinds of the thermodynamic chart to acquisition
Through network, so as to improve the performance of commodity classification neural network.
The present invention also provides a terminal devices comprising storage unit and processing unit, the storage unit is for storing
Computer program, the computer program that the processing unit is used to store by the storage unit execute described image key area
Step in area detecting method.Therefore, the terminal device also has and identical with above-mentioned image key area detection method has
Beneficial effect, details are not described herein.
[Detailed description of the invention]
Figure 1A is the step flow diagram of image key area detection method provided by first embodiment of the invention.
Figure 1B is the step flow diagram of another embodiment of the detection method of image key area shown in Figure 1A.
Fig. 2 be after step S1 shown in Figure 1A and before executing step S2 progress neural network whether train it is convergent
The flow diagram of judgment step.
Fig. 3 is the detailed process step schematic diagram of step S4 shown in Fig. 1.
Fig. 4 A is the detailed process step schematic diagram of step S6 shown in Fig. 2.
Fig. 4 B is the detailed process step schematic diagram of another embodiment of step S6 in Fig. 4 A.
Fig. 5 is the module diagram of image key area detection system provided by second embodiment of the invention.
Fig. 6 is the functional block diagram of another embodiment of the detection system of image key area shown in Fig. 5.
Fig. 7 is the specific module diagram of image conversion module shown in Fig. 6.
Fig. 8 is the module diagram of terminal device provided by third embodiment of the invention.
Attached drawing mark explanation:
20, image key area detection system；21, training module；22, obtain activation module；23, obtain gradient artwork
Block；231, coding module；24, it obtains gradient and activates module；25, judgment module；26, image conversion module；261, mean value meter
Calculate unit；262, product computing unit；263, weighted average calculation unit；27, obtain key area module；
30, terminal device；31, storage unit；32, processing unit；33, importation；34, output par, c；35, communication unit
Point.
[specific embodiment]
In order to make the purpose of the present invention, technical solution and advantage are more clearly understood, below in conjunction with attached drawing and embodiment,
The present invention will be described in further detail.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention,
It is not intended to limit the present invention.
Figure 1A is please referred to, the first embodiment of the present invention provides a kind of image key area detection method comprising as follows
The step of:
Step S1, training obtain a neural network；
Step S2 inputs an image to be processed, forward inference is carried out to the neural network based on image to be processed, to obtain
The classification results of commodity and required activation figure (activation map) in image to be processed；
Classification results are converted to coding result by step S3, and using coding result as the gradient in the neural network
Backpropagation is carried out, to obtain required gradient map (gradient map)；And
Activation figure and gradient map synthesis are obtained gradient activation figure (gradient activation map), institute by step S4
The key area of image to be processed can be represented by stating gradient activation figure.
Optionally, in order to further obtain more accurate key area, and the key area of acquisition is further utilized,
After above-mentioned steps S4, as shown in fig. 1b, described image key area detection method may also include the steps of:
Gradient activation figure is converted to thermodynamic chart by step S5；And
Thermodynamic chart and image to be processed are overlapped, obtain key area by step S6, based on the key area to optimize
New Classification Neural.
In above-mentioned step S1, the neural network includes the nerve net to can be used for classifying to object to be sorted
Network, specifically, used sorter network can be arbitrary convolutional neural networks (Convolutional Neural here
Networks, CNN), it specifically may include but be not only restricted to: alexnet neural network, vgg neural network or resnet nerve
Any one of network.
In the present embodiment, by taking commodity classification as an example, in the image to be processed that processing includes commodity to be sorted, often
It needs for the object as bottle, chest, belt or box, since the feature of different types of commodity is more and different
The feature difference of commodity is larger, so that the difficulty for making it classify increases.
In the present embodiment, the suitable neural network to commodity classification is selected, and is trained, image key can be improved
The accuracy of region detection.
Specifically, in order to improve it is described training obtain neural network stability, as shown in Figure 2, in above-mentioned steps
After S1, before carrying out step S2, it is also necessary to include the following steps:
Whether the neural network in step S1-2, judgment step S1 trains convergence, if so, S2 is entered step, if it is not, then
Return step S1 continues to train.
Specifically, in some embodiments of the invention, judging whether neural network trains convergence can be based on instruction
Practice the loss function (loss function) of neural network to determine, wherein loss function can directly react the pre- of neural network
The accuracy of measured value.When the variation of loss function is little, then can indicate neural metwork training to optimal state, i.e., it is believed that
The accuracy of training neural network is more excellent.In the present invention, the loss function may include but be not only restricted to quadratic loss function,
Logarithm loss function, cross entropy loss function etc..
In above-mentioned steps S2, image to be processed is inputted into the neural network of training acquisition, carries out forward inference, is chosen
Suitable convolution kernel (filter) obtains corresponding characteristic pattern (feature map)；During specific forward inference, often
One convolutional layer may include a convolution kernel or multiple convolution kernels, and each convolution kernel of convolutional layer has the image of a concern special
Sign, for example it can be vertical edge, horizontal edge, color or the texture etc. of image to be processed.It can be corresponded in forward inference
Multilayer convolutional layer is generated, in the present invention, convolution nuclear volume corresponding to the convolutional layer further away from input layer is more, and its energy
The characteristic information of embodiment is also more careful, so that the feature of detectable identification is also more.After completing forward inference, it can get wait locate
Manage the classification results of commodity in image.
It is corresponding on the last one convolutional layer during neural network carries out forward inference in above-mentioned steps S2
Characteristic pattern is to activate figure, which can embody High-level Image Semantic Information (High level feature), wherein image
High-layer semantic information can directly reflect the classification information of image commodity to be processed.
For example, in some specific embodiments, need to bottle, chest, sack or the box in image to be processed this
The object of sample is classified, and the feature for being used to classify can be label character information, the shape of bottle, chest, sack or box
Or color.Corresponding activation figure on the last one convolutional layer reflects the corresponding classification knot of bottle, chest, sack or box
The category feature of fruit.
In above-mentioned steps S3, classification results are converted into coding result, concretely: based on point obtained in step S2
Class result carries out one hot coding (efficient coding), specifically, one hot coding be classified variable as two into
The expression of vector processed, this requires classification value being mapped to integer value first.Then, each integer value be expressed as again binary system to
Amount, other than the index of integer, other are all zeros, are noted as 1.That is, in the present invention, each classification corresponds to
One one hot coding.
Specifically, it is assumed that the classification results encoded have 6 classifications, carry out one to first classification therein
Hot is encoded to (1,0,0,0,0,0), and if encoded to the 4th classification therein, one hot coding then corresponds to
For (0,0,0,1,0,0).
Further one hot coding to the gradient of the commodity of the category and is carried out anti-as neural network in step s3
To propagation, to obtain the gradient map on the last one convolutional layer.In the neural network, each convolutional layer can all have one
Gradient map.The one hot that can reflect commodity classification coding is equal to the gradient of neural network, corresponding gradient map can be based on
Directly reflection is by after this convolutional layer, neural network is to improve or reduce to the accuracy probability of the commodity classification.
For example, select what is measured in above-mentioned steps S2 to be classified as first category, then by its corresponding one hot coding (1,
0,0,0,0,0) to the gradient of the first category commodity and backpropagation is carried out as the neural network, to can get multiple
Convolutional layer corresponds to the gradient map of the first category commodity, takes the gradient map on the last one convolutional layer.It is obtained last
Gradient map on a convolutional layer can reflect the key area of first category commodity.Such as need to obtain the key of other classification commodity
Region can then replace one hot coding, the coding of classification commodity needed for keeping one hot coding corresponding.
For the quantity and mode classified above only as an example, in some specific embodiments, one hot coding can also be right
It should be more than ten of classification, tens classifications, several hundred a classifications or thousands of a classifications, in this as limitation.
In above-mentioned steps S4, as shown in Figure 3, activation figure and gradient map synthesis are shown that gradient activation figure specifically includes
Following step:
Step S41 calculates the average gradient in the corresponding gradient map of the last one convolutional layer per one-dimensional channel respective pixel
Value；Specifically, in the present invention, the output of a convolutional layer shares P and ties up channel, can correspond respectively to P key point position, roll up
The output of product feature is a W × H × P dimension tensor, wherein P indicates port number, the corresponding gradient map for being expressed as output of W
Width, the height of the corresponding gradient map for being expressed as output of H；Convolutional layer can be expressed as the matrix of W × H dimension per one-dimensional channel.
Average gradient value that step S41 is obtained and corresponding port number are multiplied the multiplying of the corresponding gradient map of acquisition by step S42
Product value.
Product value is weighted and averaged operation with the step S2 activation figure obtained respectively, needed for acquisition by step S43
Gradient activation figure；Specifically, activation figure obtained corresponds to floating number in step s 2, after being weighted and averaged operation,
The weighted average acquired is assigned to corresponding pixel points, so that the gradient needed for obtaining activates figure.In this step, activation figure
It is selected in the characteristic pattern of the last one convolutional layer during forward inference, it includes high-layer semantic informations, the phase with classification task
Guan Du is larger, therefore can be further improved the accuracy of classification.
Based on above-mentioned step it is found that the input in each channel may be converted into swashing with an equal amount of gradient of original image
Figure living.The stronger region of response ratio, can represent a regional area in original image in gradient activation figure.To any commodity
Generating a general part mapping in the last one convolutional layer has critically important effect to highlight those for prediction data
Region, it is believed that respond most strong position in gradient activation figure and be used as in original image and correspond to key area.
In above-mentioned steps S5, gradient activation figure is further converted into thermodynamic chart and is specifically comprised the following steps:
The average value in the channel one by one of the gradient activation figure acquired, average value are to be directed to a selected class categories
Thermodynamic chart.Wherein, gradient activation figure being converted into thermodynamic chart can be such that the key area of image to be processed visualizes, so as to more
Intuitively to obtain range and the position of key area, so that the accuracy of classification can be improved.
In above-mentioned steps S6, Fig. 4 A is please referred to, thermodynamic chart and image to be processed are overlapped, key area is obtained,
Based on the key area to optimize new Classification Neural, following step is specifically included:
Thermodynamic chart and image to be processed are overlapped by step S61, and the overlapping region of thermodynamic chart and image to be processed is made
For key area；Wherein, as an example of the invention, herein using Open CV function realize thermodynamic chart with it is to be processed
The superposition of image.
The key area is optimized new Classification Neural to continue training by step S62.
Specifically, further as shown in Figure 4 B, the step S62 can specifically be subdivided into following steps:
Step S621 cuts out key area to obtain new cutting image, to continue the new classification nerve of training optimization
Network；Or
Step S622 will increase a channel to store thermodynamic chart in image to be processed, and continue the new classification of training optimization
Neural network；Wherein, the channel be similar to RGB triple channel or gray level image channel, the channel to store thermodynamic chart,
And can be overlapped with image to be processed, corresponding key area is obtained to extract.
In above-mentioned steps S621 and step S622, the step corresponded to can be further by the key area of acquisition to instruct
Practice and optimize new Classification Neural, to facilitate the performance of promotion neural network, image provided in the present embodiment is crucial
Method for detecting area can realize the testing process of automation, without being manually labeled, can save a large amount of time and manpower,
To realize the fast lifting of Classification Neural performance.
Please continue to refer to Fig. 5, the second embodiment of the present invention provides a kind of image key area detection system 20, packet
It includes:
Training module 21 is configurable for training and obtains a neural network；
Activation module 22 is obtained, one image to be processed of input is configurable for, place is treated based on the neural network
It manages image and carries out forward inference, to obtain the classification results of commodity and required activation figure in image to be processed；
Gradient module 23 is obtained, is configurable for being converted to classification results into coding result, and by coding result
Backpropagation is carried out as the gradient in the neural network, to obtain required gradient map；And
It obtains gradient and activates module 24, be configurable for activation figure and gradient map synthesis showing that gradient activates
Figure.
Further comprise in above-mentioned acquisition gradient module 23 please continue to refer to Fig. 5:
Coding module 231, for carrying out one hot coding to classification results, a corresponding one so that each is classified
Hot coding.Wherein, one hot coding can correspond to putting in order for classification results, wherein assuming that classification results have 6 classifications,
It is encoded to (1,0,0,0,0,0) in the then corresponding one hot of first classification, and the corresponding one hot coding of the 4th classification
Then correspond to (0,0,0,1,0,0).
In the present embodiment, the one hot that can reflect commodity classification coding is equal to the gradient of neural network.Based on it
Corresponding gradient map can directly judge that neural network is passed through after this convolutional layer, for the accuracy probability of the commodity classification
It is to improve or reduce.
In order to further judge that above-mentioned training module trains the stability of the neural network obtained, referring to Fig. 6, the figure
As key area detection system further include:
Whether judgment module 25, the neural network for judging that training obtains in the training pattern train convergence；
The step of specific judgement of the judgment module 25 includes:
If judging neural network trained convergence, the acquisition activation module 22 is used for into the neural network
An image to be processed is inputted, forward inference is carried out to image to be processed based on the neural network, to obtain quotient in image to be processed
The classification results of product and required activation figure；
If judging, also training convergence, training module 21 do not continue to be trained the neural network neural network.
Specifically judge whether neural network trains convergent definitions relevant identical as content described in first embodiment,
This is repeated no more.
Optionally, in order to find out more suitable key area, and by the key area of acquisition to optimize new classification nerve
Network, please continue to refer to Fig. 6, described image key area detection system 20 can also further comprise:
Image conversion module 26, for gradient activation figure to be converted to thermodynamic chart；And
Key area module 27 is obtained, for being overlapped thermodynamic chart and image to be processed to obtain key area, and base
In the key area to optimize new Classification Neural.
Specifically, please continue to refer to Fig. 7, described image conversion module 26 can also further comprise:
Average calculation unit 261, for calculating the average value of the corresponding gradient map of the last one convolutional layer；It will specially obtain
After the average gradient value of each pixel of the gradient map on correspondence convolutional layer obtained, then obtain being averaged for the gradient map
Value.
Product computing unit 262, for by the logical of the average value of the corresponding gradient map of the convolutional layer and corresponding convolutional layer
Road number, which is multiplied, obtains the product value of corresponding gradient map；And
Weighted average calculation unit 263, for product value to be weighted and averaged operation with activation figure respectively, to obtain
The gradient needed activates figure.Specifically, after being weighted and averaged operation, the weighted average acquired is assigned to corresponding pixel points,
To which the gradient needed for obtaining activates figure.
Referring to Fig. 8, the third embodiment of the present invention provides one for implementing above-mentioned image key area detection method
Terminal device 30, the terminal device 30 include storage unit 31 and processing unit 32, and the storage unit 31 is based on storing
Calculation machine program, it is crucial that the computer program that the processing unit 32 is used to store by the storage unit 31 executes described image
Step in method for detecting area.
In some specific embodiments of the present invention, the terminal device 20 can be hardware, be also possible to software.Work as end
When end equipment is hardware, the various electronic equipments of video playing, including but not limited to intelligence are can be with display screen and supported
It can mobile phone, tablet computer, E-book reader, MP3 player (Moving Picture Experts Group Audio
Layer III, dynamic image expert's compression standard audio level 3), MP4 (Moving Picture Experts Group
Audio Layer IV, dynamic image expert's compression standard audio level 4) player, pocket computer on knee and desk-top meter
Calculation machine etc..When terminal device is software, may be mounted in above-mentioned cited electronic equipment.It may be implemented into multiple
Software or software module (such as providing multiple softwares of Distributed Services or software module) also may be implemented into single soft
Part or software module.It is not specifically limited herein.
The storage unit 31 includes the storage unit of read-only memory (ROM), random access storage device (RAM) and hard disk etc.
Point etc., the processing unit 32 according to the program being stored in the read-only memory (ROM) or can be loaded into random visit
It asks the program in memory (RAM) and executes various movements appropriate and processing.In random access storage device (RAM), also deposit
It contains the terminal device 30 and operates required various programs and data.
As shown in Figure 8, the terminal device 30 may also include the importation 33 of keyboard, mouse etc.；The terminal is set
Standby 30 can also further comprise the output par, c of cathode-ray tube (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.
34；And the terminal device 30 can further comprise the communications portion of the network interface card of LAN card, modem etc.
35.The communications portion 35 executes communication process via the network of such as internet.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, disclosed embodiment of this invention may include a kind of computer program product comprising be carried on meter
Computer program on calculation machine readable medium, the computer program include the program generation for method shown in execution flow chart
Code.In such embodiments, which can be downloaded and installed from network by communications portion 35.
When the computer program is executed by the processing unit 32, the described image key area detection of the application is executed
The above-mentioned function of being limited in method.It should be noted that computer-readable medium described herein can be it is computer-readable
Signal media or computer readable storage medium either the two any combination.Computer readable storage medium is for example
It may be-but not limited to-system, device or the device of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or any
Above combination.The more specific example of computer readable storage medium can include but is not limited to: lead with one or more
The electrical connection of line, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type can
Program read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device,
Magnetic memory device or above-mentioned any appropriate combination.
In this application, computer readable storage medium can also be any tangible medium for including or store program, should
Program can be commanded execution system, device or device use or in connection.And in this application, computer can
The signal media of reading may include in a base band or as the data-signal that carrier wave a part is propagated, wherein carrying computer
Readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal, optical signal
Or above-mentioned any appropriate combination.Computer-readable signal media can also be appointing other than computer readable storage medium
What computer-readable medium, the computer-readable medium can be sent, propagated or transmitted for by instruction execution system, device
Either device use or program in connection.The program code for including on computer-readable medium can be fitted with any
When medium transmission, including but not limited to: wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
One or more programming languages or combinations thereof can be used to write the calculating for executing operation of the invention
Machine program code, described program design language include object oriented program language -- such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing of the invention illustrate the system according to the various embodiments of the application, method
With the architecture, function and operation in the cards of computer program product.In this regard, each of flowchart or block diagram
Box can represent a part of a module, program segment or code, and a part of the module, program segment or code includes one
A or multiple executable instructions for implementing the specified logical function.It should also be noted that in some implementations as replacements,
Function marked in the box can also occur in a different order than that indicated in the drawings.For example, two succeedingly indicate
Box can actually be basically executed in parallel, they can also be executed in the opposite order sometimes, herein based on the function being related to
Can and determine.It is significant to note that in each box and block diagram and or flow chart in block diagram and or flow chart
The combination of box can be realized with the dedicated hardware based system for executing defined functions or operations, or can be used
The combination of specialized hardware and computer instruction is realized.
Involved unit can be realized by way of software in an embodiment of the present invention, can also pass through hardware
Mode realize.Described unit also can be set in the processor, for example, can be described as: a kind of image key area
Domain detection system includes training module, obtains activation module, obtains gradient module and obtain gradient activation module.Its
In, the title of these modules does not constitute the restriction to the module itself under certain conditions.
As on the other hand, the fourth embodiment of the present invention additionally provides a kind of computer-readable medium, which can
Reading medium can be included in device described in above-described embodiment；It is also possible to individualism, and without the supplying dress
In setting.Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are by the device
When execution, so that the device: training obtains a neural network；An image to be processed is inputted, based on the neural network to be processed
Image carries out forward inference, to obtain the classification results of commodity and required activation figure in image to be processed；Classification results are converted
For coding result, and backpropagation is carried out using coding result as the gradient in the neural network, to obtain required gradient map；
And activation figure and gradient map synthesis are shown that gradient activation figure, the gradient activation figure can represent the key area of image to be processed
Domain.
Compared with prior art, described image key area detection method and its system, terminal provided by the present invention are set
It is standby have it is following the utility model has the advantages that
Image key area detection method and its system provided by the present invention, using deep learning method to be sorted
Commodity train a sorter network, carry out forward inference to commodity to be sorted with the neural network, obtain the classification of commodity
Scheme with activation, is based further on the gradient with the coding of the categorical match of commodity as neural network, carries out backpropagation, obtain
Can reflect the gradient activation figure of the key area of image to be processed, so as to by with little non-of the required classification information degree of correlation
Important area is rejected, so as to effectively reduce the background interference of original image to be processed, so as to further concentrate on discrimination
Higher key area.
It is different from the method that existing needs are manually marked, image key area detection method provided by the present invention
And its system, it can be obtained by the self training of neural network, the key that detection method and its system in this way obtains
Region can have more preferably robustness compared to the key area by artificially marking by hand.
Further, in the present invention, the gradient activation figure of acquisition is converted into thermodynamic chart, to realize neural network
Visualization, obtain key area of the neural network for such commodity area of interest, as such commodity.It in this way can be with
It is automatically performed the detection to commodity key area, and can use these key areas, further improves sorter network
Performance.
And in the present invention, the key area based on acquisition is placed into new Classification Neural and is trained, can be into
The new Classification Neural of one-step optimization, to facilitate the performance of fast lifting neural network, and such automatic detection
Process can save a large amount of time and human cost.
In order to further increase the robustness of trained neural network, summarize in described image key area detection method, also
Including judging whether neural network trains convergence, if not converged, continue with data set and neural network is trained, thus
It can guarantee accuracy of the neural network of training acquisition to commodity classification.
In the present invention, it is encoded using classification results of the one hot coding mode to commodity, it can be directly by commodity
Classification results are converted into corresponding multi-C vector, and one hot is encoded the gradient as neural network, to carry out backpropagation,
To obtain its corresponding gradient map, the selection based on one hot coding obtains required key area, key area and one
Corresponding classification is related in hot coding, and the selection based on one hot coding can further improve to be examined from the image to be processed
Survey the flexibility ratio and accuracy of key area.
In the present invention, the gradient map for obtaining activation figure and its backpropagation acquisition using neural network forward inference is comprehensive
To obtain gradient activation figure, can reflect in image to be processed select merchandise classification important area, without artificially into
Rower note can greatly reduce human and material resources and time cost, also can avoid due to manually marking image key area inaccuracy
Problem.
In the present invention, key area is cut out to obtain new cutting image or will increase by one in image to be processed
Channel is used equally for advanced optimizing new classification mind to store the processing mode of key area of two kinds of the thermodynamic chart to acquisition
Through network, so as to improve the performance of commodity classification neural network.
The present invention also provides a terminal devices comprising storage unit and processing unit, the storage unit is for storing
Computer program, the computer program that the processing unit is used to store by the storage unit execute described image key area
Step in area detecting method.Therefore, the terminal device also has and identical with above-mentioned image key area detection method has
Beneficial effect, details are not described herein.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in original of the invention
Made any modification within then, equivalent replacement and improvement etc. should all be comprising within protection scope of the present invention.
Claims (10)
1. a kind of image key area detection method, it is characterised in that: it is comprised the following steps that
Step S1, training obtain a neural network；
Step S2 inputs an image to be processed, forward inference is carried out to image to be processed based on the neural network, to obtain wait locate
Manage the classification results of commodity and required activation figure in image；
Classification results are converted to coding result, and carried out coding result as the gradient in the neural network by step S3
Backpropagation, to obtain required gradient map；And
Activation figure and gradient map synthesis are shown that gradient activation figure, the gradient activation figure can represent image to be processed by step S4
Key area.
2. image key area detection method as described in claim 1, it is characterised in that: after above-mentioned steps S4, including such as
Lower step:
Gradient activation figure is converted to thermodynamic chart by step S5；And
Thermodynamic chart and image to be processed are overlapped, obtain key area by step S6, optimize new point based on the key area
Neural network.
3. image key area detection method as claimed in claim 2, it is characterised in that: in above-mentioned steps S6, be based on the pass
Key range is to optimize new Classification Neural specifically includes the following steps: key area is cut out to obtain new cutting figure
Picture optimizes new Classification Neural to continue training；Or will in image to be processed increase a channel to store the thermodynamic chart,
And continues training and optimize new Classification Neural.
4. image key area detection method as described in claim 1, it is characterised in that: after above-mentioned steps S1, walked
Before rapid S2, it is also necessary to which whether the neural network in judgment step S1 trains convergence, if so, S2 is entered step, if it is not, then returning
Step S1 is returned to continue to train；Activation figure obtained in step s 2, is specially to carry out in forward inference in neural network, most
Activation figure on the latter convolutional layer.
5. image key area detection method as described in claim 1, it is characterised in that: in above-mentioned steps S3, tied to classification
Fruit is converted to coding result, specifically includes: carrying out one hot coding to classification results, each corresponding one hot that classifies
Coding.
6. image key area detection method as described in claim 1, it is characterised in that: in above-mentioned steps S4 will activation figure with
Gradient map synthesis obtains gradient activation figure, specifically comprises the following steps: to calculate the flat of the corresponding gradient map of the last one convolutional layer
Mean value；By the product value of the average value of acquisition gradient map corresponding with the multiplication acquisition of the port number of corresponding convolutional layer；By product value
It is weighted and averaged operation with the activation figure respectively, figure is activated with the gradient needed for obtaining.
7. a kind of image key area detection system, it is characterised in that: comprising:
Training module is configurable for training and obtains a neural network；
Activation module is obtained, one image to be processed of input is configurable for, based on the neural network to image to be processed
Forward inference is carried out, to obtain the classification results of commodity and required activation figure in image to be processed；
Gradient module is obtained, is configurable for being converted to classification results into coding result, and using coding result as institute
The gradient stated in neural network carries out backpropagation, to obtain required gradient map；And
It obtains gradient and activates module, be configurable for activation figure and gradient map synthesis obtaining gradient activation figure, it is described
Gradient activation figure can represent the key area of image to be processed.
8. image key area detection system as claimed in claim 7, it is characterised in that: described image key area detection system
Further include:
Whether judgment module, the neural network for judging that training obtains in the training pattern train convergence；
Image conversion module, for gradient activation figure to be converted to thermodynamic chart；And
Key area module is obtained, obtains key area for thermodynamic chart and image to be processed to be overlapped, and be based on the pass
Key range is to optimize new Classification Neural.
9. image key area detection system as claimed in claim 7, it is characterised in that: described image conversion module further include:
Average calculation unit, for calculating the average value of the corresponding gradient map of different convolutional layers；
Product computing unit, for by the port number phase of the average value of the corresponding gradient map of different convolutional layers and different convolutional layers
Multiply the product value for obtaining corresponding gradient map；And
Weighted average calculation unit, for product value to be weighted and averaged operation with activation figure respectively, to obtain required ladder
Degree activation figure.
10. a kind of terminal device, it is characterised in that: the terminal device includes storage unit and processing unit, and the storage is single
Member is for storing computer program, described in the computer program execution that the processing unit is used to store by the storage unit
Step in any one of claim 1-6 described image key area detection method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910042460.3A CN109858482B (en) | 2019-01-16 | 2019-01-16 | Image key area detection method and system and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910042460.3A CN109858482B (en) | 2019-01-16 | 2019-01-16 | Image key area detection method and system and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109858482A true CN109858482A (en) | 2019-06-07 |
| CN109858482B CN109858482B (en) | 2020-04-14 |
Family
ID=66894938
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910042460.3A Active CN109858482B (en) | 2019-01-16 | 2019-01-16 | Image key area detection method and system and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109858482B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110504029A (en) * | 2019-08-29 | 2019-11-26 | 腾讯医疗健康(深圳)有限公司 | A kind of medical image processing method, medical image recognition method and device |
| CN110517771A (en) * | 2019-08-29 | 2019-11-29 | 腾讯医疗健康(深圳)有限公司 | A kind of medical image processing method, medical image recognition method and device |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111723695A (en) * | 2020-06-05 | 2020-09-29 | 广东海洋大学 | Improved Yolov 3-based driver key sub-area identification and positioning method |
| CN111883177A (en) * | 2020-07-15 | 2020-11-03 | 厦门熙重电子科技有限公司 | Voice key information separation method based on deep learning |
| CN113743543A (en) * | 2021-11-05 | 2021-12-03 | 武汉大学 | Image classification training method and device, server and storage medium |
| US12033374B2 (en) | 2020-02-13 | 2024-07-09 | Tencent Technology (Shenzhen) Company Limited | Image processing method, apparatus, and device, and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1443446A1 (en) * | 2001-10-05 | 2004-08-04 | Riken | Method of presuming domain linker region of protein |
| CN105718878A (en) * | 2016-01-19 | 2016-06-29 | 华南理工大学 | Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network |
| CN105718945A (en) * | 2016-01-20 | 2016-06-29 | 江苏大学 | Apple picking robot night image identification method based on watershed and nerve network |
| WO2016145379A1 (en) * | 2015-03-12 | 2016-09-15 | William Marsh Rice University | Automated Compilation of Probabilistic Task Description into Executable Neural Network Specification |
| CN106778590A (en) * | 2016-12-09 | 2017-05-31 | 厦门大学 | It is a kind of that video detecting method is feared based on convolutional neural networks model cruelly |
| US20180005113A1 (en) * | 2016-06-29 | 2018-01-04 | Fujitsu Limited | Information processing apparatus, non-transitory computer-readable storage medium, and learning-network learning value computing method |
| CN108564109A (en) * | 2018-03-21 | 2018-09-21 | 天津大学 | A kind of Remote Sensing Target detection method based on deep learning |
-
2019
- 2019-01-16 CN CN201910042460.3A patent/CN109858482B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1443446A1 (en) * | 2001-10-05 | 2004-08-04 | Riken | Method of presuming domain linker region of protein |
| WO2016145379A1 (en) * | 2015-03-12 | 2016-09-15 | William Marsh Rice University | Automated Compilation of Probabilistic Task Description into Executable Neural Network Specification |
| CN105718878A (en) * | 2016-01-19 | 2016-06-29 | 华南理工大学 | Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network |
| CN105718945A (en) * | 2016-01-20 | 2016-06-29 | 江苏大学 | Apple picking robot night image identification method based on watershed and nerve network |
| US20180005113A1 (en) * | 2016-06-29 | 2018-01-04 | Fujitsu Limited | Information processing apparatus, non-transitory computer-readable storage medium, and learning-network learning value computing method |
| CN106778590A (en) * | 2016-12-09 | 2017-05-31 | 厦门大学 | It is a kind of that video detecting method is feared based on convolutional neural networks model cruelly |
| CN108564109A (en) * | 2018-03-21 | 2018-09-21 | 天津大学 | A kind of Remote Sensing Target detection method based on deep learning |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110504029A (en) * | 2019-08-29 | 2019-11-26 | 腾讯医疗健康(深圳)有限公司 | A kind of medical image processing method, medical image recognition method and device |
| CN110517771A (en) * | 2019-08-29 | 2019-11-29 | 腾讯医疗健康(深圳)有限公司 | A kind of medical image processing method, medical image recognition method and device |
| WO2021036616A1 (en) * | 2019-08-29 | 2021-03-04 | 腾讯科技（深圳）有限公司 | Medical image processing method, medical image recognition method and device |
| CN110504029B (en) * | 2019-08-29 | 2022-08-19 | 腾讯医疗健康(深圳)有限公司 | Medical image processing method, medical image identification method and medical image identification device |
| CN111046939A (en) * | 2019-12-06 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | CNN (CNN) class activation graph generation method based on attention |
| CN111046939B (en) * | 2019-12-06 | 2023-08-04 | 中国人民解放军战略支援部队信息工程大学 | Attention-based CNN class activation graph generation method |
| US12033374B2 (en) | 2020-02-13 | 2024-07-09 | Tencent Technology (Shenzhen) Company Limited | Image processing method, apparatus, and device, and storage medium |
| CN111723695A (en) * | 2020-06-05 | 2020-09-29 | 广东海洋大学 | Improved Yolov 3-based driver key sub-area identification and positioning method |
| CN111883177A (en) * | 2020-07-15 | 2020-11-03 | 厦门熙重电子科技有限公司 | Voice key information separation method based on deep learning |
| CN111883177B (en) * | 2020-07-15 | 2023-08-04 | 厦门熙重电子科技有限公司 | Voice key information separation method based on deep learning |
| CN113743543A (en) * | 2021-11-05 | 2021-12-03 | 武汉大学 | Image classification training method and device, server and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109858482B (en) | 2020-04-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858482A (en) | A kind of image key area detection method and its system, terminal device | |
| Guo et al. | Supplier selection based on hierarchical potential support vector machine | |
| CN110929774B (en) | Classification method, model training method and device for target objects in image | |
| CN108229341A (en) | Sorting technique and device, electronic equipment, computer storage media, program | |
| WO2019157946A1 (en) | Anti-money laundering method, apparatus, and device | |
| CN109508681A (en) | The method and apparatus for generating human body critical point detection model | |
| CN108509457A (en) | A kind of recommendation method and apparatus of video data | |
| CN109165645A (en) | A kind of image processing method, device and relevant device | |
| CN109345302A (en) | Machine learning model training method, device, storage medium and computer equipment | |
| CN109460514A (en) | Method and apparatus for pushed information | |
| CN107220217A (en) | Characteristic coefficient training method and device that logic-based is returned | |
| CN108960514A (en) | It shows the method, apparatus of prediction model and adjusts the method, apparatus of prediction model | |
| CN107077625A (en) | The deep convolutional neural networks of layering | |
| CN106484766B (en) | Searching method and device based on artificial intelligence | |
| CN108171191B (en) | Method and apparatus for detecting face | |
| CN104573706A (en) | Object identification method and system thereof | |
| CN106600052A (en) | User attribute and social network detection system based on space-time locus | |
| CN108280451A (en) | Semantic segmentation and network training method and device, equipment, medium, program | |
| CN109345553A (en) | A kind of palm and its critical point detection method, apparatus and terminal device | |
| CN109255564A (en) | Pick-up point address recommendation method and device | |
| CN110263161A (en) | A kind of processing method of information, device and equipment | |
| CN110019790A (en) | Text identification, text monitoring, data object identification, data processing method | |
| CN109871809A (en) | A kind of machine learning process intelligence assemble method based on semantic net | |
| CN110310114A (en) | Object classification method, device, server and storage medium | |
| CN107203558A (en) | Object recommendation method and apparatus, recommendation information treating method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |