CN108921825A - The method and device of the facial skin points shape defect of detection based on deep learning - Google Patents
The method and device of the facial skin points shape defect of detection based on deep learning Download PDFInfo
- Publication number
- CN108921825A CN108921825A CN201810600716.3A CN201810600716A CN108921825A CN 108921825 A CN108921825 A CN 108921825A CN 201810600716 A CN201810600716 A CN 201810600716A CN 108921825 A CN108921825 A CN 108921825A
- Authority
- CN
- China
- Prior art keywords
- convolutional neural
- neural networks
- skin
- image block
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30088—Skin; Dermal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of method and device of facial skin points shape defect of the detection based on deep learning, and wherein method includes：Obtain training dataset；Convolutional neural networks are constructed, convolutional neural networks are trained to obtain detection model using training dataset, wherein convolutional neural networks have following feature：The feature extraction of multilayer convolution is carried out to obtain the characteristic pattern of different levels to sample, then it is sampled using more sizes and the sample boxes of various shapes to obtain the corresponding sampled result of different levels, then classify using softmax loss function for all sampled results and utilize L1 loss function predicted position, by the front end of the penalty values being calculated passback convolutional neural networks, convolutional neural networks parameter is adjusted using gradient descent method；Testing image is cut into test image block, all test image blocks are then inputted into detection model to obtain corresponding image block testing result with traversing, all image block test results are then spliced into complete image testing result.
Description
Technical field
The present invention relates to a kind of computer and software technology fields, and in particular to a kind of detection face skin in deep learning
The method and device of skin spot defect.
Background technique
With the improvement of living standards, the public beauty health for starting to pursue skin of face.Common skin of face defect
According to shape, area and distribution, it can substantially be divided into two class of skin of face spot defect and skin of face spot defect.Wherein
Skin of face spot defect includes mole, acne, acne print etc., they have area smaller, and shape approximation is dotted, usually independent
The characteristics of distribution.
The batch processing or quantitative statistics of skin defect are generally used for for the automatic detection of such dotted skin defect
Analysis.Existing dotted skin defect automatic detection algorithm is mostly based on traditional computer vision technique, i.e., according to face figure
The color or luminance information of picture in original image rank or mark off the thumbnail rank come, use threshold method and connected domain analysis
Method determines the position of dotted skin defect.Part detection algorithm uses the support vector machines based on machine learning, by collecting one
The image for having mark of fixed number amount, a small number of characteristics of image that analysis manually determines, such as contrast, diversity, homogenieity, energy
Deng dotted skin defect is positioned and is distinguished.When facial image meets such as fine definition, high-resolution, maintaining uniform illumination, skin
When the stringent conditions such as colour cast is white, skin defect is less, the prior art can basically reach testing goal, but in most cases
The facial image of acquisition can not meet the ideal conditions, and the prior art can not accurately detect dotted skin defect.
Summary of the invention
In view of this, the present invention provides the method and dress of a kind of facial skin points shape defect of the detection based on deep learning
It sets, is able to solve the technical issues of scientific and precise reference can not be provided for the assessment and treatment of skin defect of the prior art.
To achieve the above object, according to an aspect of the invention, there is provided a kind of detection face based on deep learning
The method of skin spot defect, including：Training dataset is obtained, the sample that the training data is concentrated is having a size of m pixel × n
Pixel, include the image block of skin of face spot defect markup information, wherein m and n be positive integer；Construct convolutional Neural net
Network is trained to obtain detection model the convolutional neural networks using the training dataset, wherein the convolution mind
There is following feature through network：The feature extraction of multilayer convolution is carried out to obtain the characteristic pattern of different levels to sample, is then directed to
The characteristic pattern of the different levels is using more sizes and the sampling of the sample boxes of various shapes is to obtain the corresponding sampling of different levels
As a result, then classify using softmax loss function for all corresponding sampled results of different levels and utilizing L1
Loss function predicted position adjusts the front end of the penalty values being calculated passback convolutional neural networks using gradient descent method
Convolutional neural networks parameter；Testing image is cut into multiple having a size of m pixel × n-pixel test image block, then traversed
The multiple test image block is inputted the detection model to obtain corresponding image block testing result, then by all institutes by ground
It states image block test result and is spliced into complete image testing result.
Optionally, the length-width ratio a of the sample boxes meets condition：1:3≤a≤3:1.
Optionally, the skin of face spot defect includes one or more combination below：Mole, acne, acne
Print.
Optionally, further include：The training dataset and testing image are pre-processed, the pretreatment includes：Into
Row invalid data filters out, and carries out brightness homogenization.
Optionally, 100≤m≤600, and 100≤n≤600.
To achieve the above object, according to another aspect of the present invention, it is also proposed that a kind of detection faces based on deep learning
The device of portion's skin spot defect, including：Module is obtained, for obtaining training dataset, the sample that the training data is concentrated
For having a size of m pixel × n-pixel, include the image block of skin of face spot defect markup information, wherein m and n is positive whole
Number；Modeling module is trained the convolutional neural networks using the training dataset for constructing convolutional neural networks
To obtain detection model, wherein the convolutional neural networks have following feature：To sample carry out the feature extraction of multilayer convolution with
The characteristic pattern of different levels is obtained, then utilizes more sizes and the sample boxes of various shapes for the characteristic pattern of the different levels
Then sampling is used with obtaining the corresponding sampled result of different levels for all corresponding sampled results of different levels
Softmax loss function carries out L1 loss function predicted position of classifying and utilize, by the penalty values being calculated passback convolution mind
Front end through network adjusts convolutional neural networks parameter using gradient descent method；Detection module, for testing image to be cut into
It is multiple having a size of m pixel × n-pixel test image block, then the multiple test image block is inputted the detection by traversal ground
Then all described image block test results are spliced into complete image detection to obtain corresponding image block testing result by model
As a result.
Optionally, the length-width ratio a of the sample boxes meets condition：1:3≤a≤3:1.
Optionally, the skin of face spot defect includes one or more combination below：Mole, acne, acne
Print.
Optionally, further include：Preprocessing module, for being located in advance to the training dataset and the testing image
Reason, the pretreatment include：It carries out invalid data to filter out, carries out brightness homogenization.
Optionally, 100≤m≤600, and 100≤n≤600.
According to the technique and scheme of the present invention, it can then be examined based on neural network and depth learning technology training detection model
Skin of face spot defect is surveyed, key technology point therein is：The characteristic pattern of different levels is sampled during training pattern
Then all sampled results are trained, it means that comprehensive study of the model towards progress multi-layer when the study stage,
Various sizes of target is detected from the characteristic pattern of different levels.Specifically, it is detected on low-level, large-sized characteristic pattern
The skin of face spot defect of smaller size, high-level, small size characteristic pattern on detect the skin of face of larger size
Spot defect can comprehensively detect various sizes of skin of face spot defect in this way with less computing overhead.The present invention
Technical solution at least have the advantages that：(1) can identify defect whether there is and existing position, and can be into
One step confirms defect kind, and accuracy is high, and wrong report is failed to report few；(2) intelligence degree is high, has self-learning characteristics, is not necessarily to people
Work determines characteristic factor；(3) algorithm simplicity is apparent, low to hardware requirement in the test application stage after the completion of model training；(4) may be used
Have a wide range of application, influenced by factors such as pore hair, illumination condition and shooting angle small, shows very strong robustness.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein：
Fig. 1 is the process of the method for the facial skin points shape defect of the detection based on deep learning according to an embodiment of the present invention
Schematic diagram；
Fig. 2 is the sample instantiation figure that the training data of the embodiment of the present invention is concentrated；
Fig. 3 is the mind in the method for the facial skin points shape defect of the detection based on deep learning according to an embodiment of the present invention
Through network algorithm schematic diagram；
Fig. 4 is the structure of the device of the facial skin points shape defect of the detection based on deep learning according to an embodiment of the present invention
Schematic diagram.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
To more fully understand those skilled in the art, lower different type skin of face spot defect is described in detail below
Relevant knowledge.
(1) mole
Mole also abbreviation mole, macle or black mole, are most commonly in by what the mole cell normally containing pigment was constituted
Benign tumour of skin, occasionally in mucomembranous surface.There are many types for clinical manifestation.Color is in deep brown or ink black more, is not had also
Coloured no mole.Canceration, significant can occur under certain condition for some types.Mole mostly occurs in face, neck, back
Grade portions, it is seen that in any normal human.Can be i.e. existing at birth, or gradually show in one's early years after life.Majority increasess slowly,
Or lasting for years and unchanged, but spontaneous regression seldom occurs.
(2) acne.Acne is commonly called as " whelk " or " small pox ".Acne is a kind of chronic inflammation of pilosebaceous unit
Dermatoses are mainly apt to occur in teenager, very big to teen-age psychology and social influence, but tend to nature after puberty and subtract
Light or recovery from illness.With the characteristics of the pleomorphism skin lesions such as acne, papule, warts, tubercle of the clinical manifestation to be apt to occur in face.Acne it is non-
Inflammatory skin lesion shows as open and closed comedones.The typical skin lesion of closed comedones (also known as hoary hair) is about 1 millimeter big
Small colour of skin papule, without obvious hair follicle opening.Open comedones (also known as blackhead) show as what dome-shaped papule companion significantly expanded
Hair follicle opening.Acne, which further develop, can evolve into various inflammatory skin lesions, show as inflamed papules, warts, tubercle and tumour.
Inflamed papules take on a red color, and 1 to 5 millimeter of diameter is differed；Warts is in the same size, has been filled with white fester；Tubercle diameter is greater than 5
Millimeter, touching have scleroma and feeling of pain；The position of tumour is deeper, is filled with the mixture of fester and blood.These skin lesions may be used also
Fusion forms big inflammatory patch and sinus etc..Inflammatory skin lesion usually leaves pigmentation, erythema perstans, recess after subsiding
Property or hypertrophic scar.Acne is clinically divided into 3 degree, 4 grades according to acne lesions property and severity：1 grade (slight)：Only
There are acne；2 grades (moderate)：In addition to acne, there are also some inflamed papules；3 grades (moderate)：In addition to acne, there are also more inflammatories
Papule or warts；4 grades (severe)：In addition to having acne, inflamed papules and warts, nodosity, tumour or scar are gone back.
(3) acne print.
Most of whelks can all leave acne print (pockmark) acne print and focus on prevention, and whelk wants treated as soon as possible to prevent leaving forever
Long property spot, more early treatment, are more not easy to leave acne print.As the metabolic acne print of skin also can slowly desalinate, shoal.It is fresh
Acne print be all it is red, it is outmoded after color can slowly deepen in dark brown, will slowly shoal later, this is the nature of acne print
Evolution process.
From the foregoing, it will be observed that the appearance of above-mentioned three types skin of face spot defect is different.Actually their origin cause of formation
Also different, corresponding treatment method and therapeutic agent are also different.If detecting classification error, treatment failure will lead to.Cause
This needs proposes a kind of method and apparatus for accurately and reliably detecting facial skin points shape defect.
Fig. 1 is the process of the method for the facial skin points shape defect of the detection based on deep learning according to an embodiment of the present invention
Schematic diagram.As shown in Figure 1, this method may include following step S1 to step S3.
Step S1：Training dataset is obtained, the sample that training data is concentrated is having a size of m pixel × n-pixel, includes
The image block of skin of face spot defect markup information, wherein m and n is positive integer.Wherein：Skin of face spot defect can wrap
Include one or more combination below：Mole, acne, acne print.
Such as：A large amount of (it is recommended that being greater than 5000) the original face image datas collected early period can manually be marked
Note, the form of mark draw at dotted skin defect in original facial image and are just included dotted skin defect
Rectangle frame, and tag along sort has been marked, illustrate that skin defect is specially one of mole, acne and acne print or a variety of
Combination.Then interception size is m pixel × n-pixel image block as training from the original facial image after these marks
Sample in data set.The numerical value of m and n should not be too large or too small, it is proposed that reasonable value makes typical dotted skin defect exist
Area in image block is not less than the 1% of the image block gross area.Optionally, 100≤m≤600, and 100≤n≤600.Fig. 2
It is the sample instantiation figure that the training data of the embodiment of the present invention is concentrated.It should be noted that being lacked in interception comprising dotted skin
When sunken image block, the interception position of adjustable image block is made so that dotted skin defect is located at the different location of image block
For a kind of means of positive sample for expanding training dataset.In addition, usual original facial image midpoint shape skin defect institute occupied area
Domain is smaller, negative sample quantity often far super positive sample quantity, can only selected part negative sample so that training data is concentrated just
Negative sample quantity balance.
Step S2：Convolutional neural networks are constructed, convolutional neural networks are trained to be examined using training dataset
Survey model, wherein convolutional neural networks have following feature：The feature extraction of multilayer convolution is carried out to obtain different levels to sample
Characteristic pattern, then for the characteristic pattern of different levels using more sizes and the sample boxes of various shapes are sampled to obtain different layers
The corresponding sampled result of grade, wherein characteristic pattern level is higher, and sample boxes size is bigger, then corresponding for all different levels
Sampled result using softmax loss function carry out classify and utilize L1 loss function predicted position, the damage that will be calculated
Mistake value returns the front end of convolutional neural networks, adjusts convolutional neural networks parameter using gradient descent method.
Specifically, dual-stage detection method (can be different from, dual-stage detection is needed using single phase in step s 2
First carry out a target area preextraction, then to the region extracted carry out classification and boundary determine, single phase then this two
Step content disposably combines completion, and comparatively efficiency increases) algorithm of target detection, with the segment sample in the training set
This is input, constructs convolutional neural networks, carries out the feature extraction of multilayer convolution to segment to be detected, obtains the feature of different levels
Figure carries out uniform intensive sampling (sample boxes use different scale and length-width ratio) for several selected characteristic patterns, uses
Softmax loss function predicts dotted skin defect tag along sort, predicts dotted skin defect detection block using L1 loss function
Position, the penalty values being calculated are passed back into convolutional neural networks, using gradient descent method adjustment convolutional neural networks ginseng
Number, new penalty values are calculated with parameter adjusted.Iterative cycles according to this, until model is restrained, penalty values drop to acceptable model
It encloses.
To more fully understand those skilled in the art, Fig. 3 is the detection faces based on deep learning of the embodiment of the present invention
Neural network algorithm schematic diagram in the method for portion's skin spot defect.
As shown in figure 3, using VGG16 as the basic model of convolutional neural networks, by the full connection in VGG16 network structure
Layer removes, and 6 layers of convolutional layer is increased newly, then to the last one convolutional layer (i.e. conv4_3) of VGG16 and all newly-increased convolutional layers
Each characteristic pattern of (conv6, conv7, conv8_2, conv9_2, conv10_2 and conv11_2) output carries out detection operation.
To the characteristic pattern of different levels carry out detection operation the advantages of be, various sizes of dotted skin defect can all be reached compared with
Good detection effect.The information that the dotted skin defect of small size is remained in low-level, large-sized characteristic pattern, with characteristic pattern
The raising of level, level of abstraction are deepened, and the information of the dotted skin defect of small size is gradually lost, the large scale more complex to structure
The sensitivity of dotted skin defect is promoted.Therefore the characteristic pattern for including different levels is conducive to cover various sizes of dotted skin
Skin defect.It is to the detection operation that selected characteristic pattern carries out：The sampled point of intensive sampling, sampled point are used for characteristic pattern setting
It is uniformly distributed on characteristic pattern.The different sample boxes of several sizes, length-width ratio are configured for each sampled point.It is each
The sample boxes quantity of a sampled point configuration can be 3 to 7.The size of sample boxes is related to characteristic pattern level size, feature figure layer
Grade is higher, and characteristic pattern size is smaller, and sample boxes size is bigger.For example, can be sampled to each on a certain hierarchy characteristic figure
Point five sample boxes of configuration, this five sample boxes area equations but length-width ratio is respectively 3:1,2:1,1:1,1:2,1:3.It will sampling
Frame is matched with dotted skin defect callout box, otherwise it is negative sample that the sample boxes that overlapping area is higher than given threshold, which are positive sample,.
Softmax function and L1 function penalty values are calculated for selected positive negative sample.
It should be noted that the length-width ratio a of sample boxes meets condition：1:3≤a≤3:1.This is because dotted skin defect
It may not be standard circular, it may be possible to ellipse.If square sample frame is only used only to be possible to be inaccurate.Using a variety of
The sample boxes of shape can make neural network that can learn to more information, and the model that training obtains has stronger robustness.
But also it is not recommended that deviateing very much 1 using length-width ratio：1 sample boxes do not have practical significance, can also increase calculating consumption.
Step S3：Testing image is cut into multiple having a size of m pixel × n-pixel test image block, then traversal ground
By multiple test image blocks input detection model to obtain corresponding image block testing result, then all image blocks are tested and are tied
Fruit is spliced into complete image testing result.
Specifically, testing image will be first cut into it is multiple having a size of m pixel × n-pixel test image block, then will be each
A test image block traversal is input in the detection model that step S2 is obtained.Model will export detection in test image block and obtain
Several dotted skin defect detection blocks, non-maxima suppression then is carried out to these dotted skin defect detection blocks, is removed
Possible duplicate detection block, obtains the corresponding testing result of test image block.Then it will test frame coordinate from test image block
Rank is converted to original test image rank, is equivalent to and executes splicing movement, obtains the testing result of complete image level.
From the foregoing, it will be observed that the method for the facial skin points shape defect of the detection according to an embodiment of the present invention based on deep learning,
Facial skin points shape defect, key therein can be then detected based on neural network and depth learning technology training detection model
Technical point is：During training pattern then the characteristic pattern sampling of different levels is trained all sampled results, this
Mean that model detects different rulers from the characteristic pattern of different levels towards the comprehensive study for carrying out multi-layer when the study stage
Very little target.Specifically, the skin of face spot defect that smaller size is detected on low-level, large-sized characteristic pattern,
High-level, small size characteristic pattern on detect the skin of face spot defect of larger size, in this way can be with less computing overhead
Comprehensively to detect various sizes of skin of face spot defect.Technical solution of the present invention at least has following beneficial to effect
Fruit：(1) can identify defect whether there is and existing position, and can further confirm that defect kind, and accuracy is high, accidentally
Report is failed to report few；(2) intelligence degree is high, has self-learning characteristics, determines characteristic factor without artificial；(3) algorithm simplicity is bright
It is clear, it is low to hardware requirement in the test application stage after the completion of model training；(4) can have a wide range of application, by pore hair, illumination item
The influence of the factors such as part and shooting angle is small, shows very strong robustness.
Optionally, in order to improve the accuracy rate of detection, training dataset and testing image can also be pre-processed, in advance
Processing includes：It carries out invalid data to filter out, carries out brightness homogenization.Detail is as follows：(1) invalid data is carried out to filter out.This
One processing is particularly important for testing image.Specifically, it can first determine face location, remove the background area other than face,
Invalid data can be filtered out in this way, reduced and calculated cost, improve efficiency.(2) brightness homogenization is carried out.Since face is not to put down
Smooth, the positions such as forehead, nose may have bloom, cheek, and the wing of nose may have shade, so needing to carry out brightness homogenization behaviour
Make.Specifically, then first pre-set image Block Brightness target mean carries out median filtering Fuzzy Processing to original picture block and obtains mould
Image block is pasted, original picture block and blurred picture block are then made into color space conversion respectively, converts to Lab color space, connects
The difference of the channel calculating blurred picture block L (luminance channel) mean value and luma target mean value.The difference that previous step calculates is added
In the channel original picture block L, finally original picture block is converted to rgb color space, the image block after obtaining brightness homogenization.
Optionally, the method for the facial skin points shape defect of detection of the invention can also be counted during model training
According to sampling operation and data enhancement operations, so that sample data is more abundant and optimizes.(1) data sampling：In training dataset
The distribution of data of all categories is possible and uneven.When deep learning model is trained each time, random take out need to be concentrated from training data
Access is according to formation training data batch.When forming training data batch, the selected probability of data of all categories with it is all kinds of
Other data bulk is inversely proportional, i.e. the probability that is selected of the smaller classification of data volume is higher, and the bigger classification of data volume is selected
Probability is lower.After this otherness sampling operation, the data category distribution into model is answered generally uniform, makes a small number of classifications can also
Sufficiently to be learnt by model.(2) data enhance：To reduce over-fitting risk, increase data volume, in each training data batch shape
Every image is overturn that is, according to certain probability at rear progress data enhancing processing, rotates and is enlarged, retaining
On the basis of key feature, artificial expanding data amount.
Fig. 4 is the structure of the device of the facial skin points shape defect of the detection based on deep learning according to an embodiment of the present invention
Schematic diagram.As shown in figure 4, the apparatus may include obtain module 100, modeling module 200, detection module 300.
Module 100 is obtained for obtaining training dataset, the sample that training data is concentrated is having a size of m pixel × n-pixel
, image block that include skin of face spot defect markup information, wherein m and n is positive integer.Wherein：Skin of face is dotted
Defect may include one or more combination below：Mole, acne, acne print.
Such as：A large amount of (it is recommended that being greater than 5000) the original face image datas collected early period can manually be marked
Note, the form of mark draw at dotted skin defect in original facial image and are just included dotted skin defect
Rectangle frame, and tag along sort has been marked, illustrate that skin defect is specially one of mole, acne and acne print or a variety of
Combination.Then interception size is m pixel × n-pixel image block as training from the original facial image after these marks
Sample in data set.The numerical value of m and n should not be too large or too small, it is proposed that reasonable value makes typical dotted skin defect exist
Area in image block is not less than the 1% of the image block gross area.Optionally, 100≤m≤600, and 100≤n≤600.Fig. 2
It is the sample instantiation figure that the training data of the embodiment of the present invention is concentrated.It should be noted that being lacked in interception comprising dotted skin
When sunken image block, the interception position of adjustable image block is made so that dotted skin defect is located at the different location of image block
For a kind of means of positive sample for expanding training dataset.In addition, usual original facial image midpoint shape skin defect institute occupied area
Domain is smaller, negative sample quantity often far super positive sample quantity, can only selected part negative sample so that training data is concentrated just
Negative sample quantity balance.
Modeling module 200 is trained convolutional neural networks using training dataset for constructing convolutional neural networks
To obtain detection model, wherein convolutional neural networks have following feature：The feature extraction of multilayer convolution is carried out to obtain to sample
Then the characteristic pattern of different levels is sampled for the characteristic pattern of different levels using more sizes and the sample boxes of various shapes to obtain
To the corresponding sampled result of different levels, wherein characteristic pattern level is higher, and sample boxes size is bigger, then for all differences
The corresponding sampled result of level carries out L1 loss function predicted position of classifying and utilize using softmax loss function, will calculate
The front end of obtained penalty values passback convolutional neural networks, adjusts convolutional neural networks parameter using gradient descent method.
Specifically, modeling module 200 is used to (be different from dual-stage detection method, dual-stage detection needs using single phase
A target area preextraction is first carried out, then classification and boundary determination are carried out to the region extracted, single phase is then this
Two step contents disposably combine completion, and comparatively efficiency increases) algorithm of target detection, with the segment in the training set
Sample is input, constructs convolutional neural networks, carries out the feature extraction of multilayer convolution to segment to be detected, obtains the spy of different levels
Sign figure carries out uniform intensive sampling (sample boxes use different scale and length-width ratio) for several selected characteristic patterns, uses
Softmax loss function predicts dotted skin defect tag along sort, predicts dotted skin defect detection block using L1 loss function
Position, the penalty values being calculated are passed back into convolutional neural networks, using gradient descent method adjustment convolutional neural networks ginseng
Number, new penalty values are calculated with parameter adjusted.Iterative cycles according to this, until model is restrained, penalty values drop to acceptable model
It encloses.
It should be noted that the length-width ratio a of sample boxes meets condition：1:3≤a≤3:1.This is because dotted skin defect
It may not be standard circular, it may be possible to ellipse.If square sample frame is only used only to be possible to be inaccurate.Using a variety of
The sample boxes of shape can make neural network that can learn to more information, and the model that training obtains has stronger robustness.
But also it is not recommended that deviateing very much 1 using length-width ratio：1 sample boxes do not have practical significance, can also increase calculating consumption.
Detection module 300 is multiple having a size of m pixel × n-pixel test image block for testing image to be cut into, so
Multiple test image blocks are inputted detection model to obtain corresponding image block testing result, then by all images by traversal ground afterwards
Block test result is spliced into complete image testing result.
Specifically, detection module 300 is multiple having a size of the survey of m pixel × n-pixel for will first be cut into testing image
Image block is tried, then each test image block traversal is input in the detection model that step S2 is obtained.Model will export test
Then several dotted skin defect detection blocks that detection obtains in image block carry out these dotted skin defect detection blocks non-
Maximum inhibits, and removes possible duplicate detection block, obtains the corresponding testing result of test image block.Then it will test frame seat
Mark is converted to original test image rank from test image block rank, is equivalent to and executes splicing movement, obtains complete image level
Other testing result.
From the foregoing, it will be observed that the device of the facial skin points shape defect of the detection according to an embodiment of the present invention based on deep learning,
Facial skin points shape defect, key therein can be then detected based on neural network and depth learning technology training detection model
Technical point is：During training pattern then the characteristic pattern sampling of different levels is trained all sampled results, this
Mean that model detects different rulers from the characteristic pattern of different levels towards the comprehensive study for carrying out multi-layer when the study stage
Very little target.Specifically, the skin of face spot defect that smaller size is detected on low-level, large-sized characteristic pattern,
High-level, small size characteristic pattern on detect the skin of face spot defect of larger size, in this way can be with less computing overhead
Comprehensively to detect various sizes of skin of face spot defect.Technical solution of the present invention at least has following beneficial to effect
Fruit：(1) can identify defect whether there is and existing position, and can further confirm that defect kind, and accuracy is high, accidentally
Report is failed to report few；(2) intelligence degree is high, has self-learning characteristics, determines characteristic factor without artificial；(3) algorithm simplicity is bright
It is clear, it is low to hardware requirement in the test application stage after the completion of model training；(4) can have a wide range of application, by pore hair, illumination item
The influence of the factors such as part and shooting angle is small, shows very strong robustness.
Optionally, in order to improve the accuracy rate of detection, the detection skin of face based on deep learning of the embodiment of the present invention
The device of spot defect can also include preprocessing module.The preprocessing module is used to carry out training dataset and testing image
Pretreatment, pretreatment include：It carries out invalid data to filter out, carries out brightness homogenization.Detail is as follows：(1) invalid number is carried out
According to filtering out.This processing is particularly important for testing image.Specifically, it can first determine face location, remove other than face
Background area can filter out invalid data in this way, reduce and calculate cost, improve efficiency.(2) brightness homogenization is carried out.Due to face
Portion is not flat, and the positions such as forehead, nose may have bloom, cheek, and the wing of nose may have shade, so needing to carry out brightness
Homogenization operation.Specifically, then first pre-set image Block Brightness target mean carries out the fuzzy place of median filtering to original picture block
Reason obtains blurred picture block, and original picture block and blurred picture block are then made color space conversion, conversion to Lab color respectively
Space then calculates the difference of the channel blurred picture block L (luminance channel) mean value and luma target mean value.Previous step is calculated
Difference is added in the channel original picture block L, finally converts original picture block to rgb color space, after obtaining brightness homogenization
Image block.
Optionally, the modeling module in the device of the facial skin points shape defect of detection of the invention is also used in model training
Data sampling operation and data enhancement operations are carried out in the process, so that sample data is more abundant and optimizes.(1) data sampling：
In training dataset the distribution of data of all categories may and it is uneven.It, need to be from training number when deep learning model is trained each time
Data, which are randomly selected, according to concentration forms training data batch.When forming training data batch, data of all categories are selected
Probability be inversely proportional with data bulk of all categories, i.e. the probability that is selected of the smaller classification of data volume is higher, and data volume is bigger
The probability that classification is selected is lower.After this otherness sampling operation, the data category distribution into model is answered generally uniform, is made
A small number of classifications can also sufficiently be learnt by model.(2) data enhance：To reduce over-fitting risk, increase data volume, in each instruction
Practice progress data enhancing processing after data batch is formed to overturn every image that is, according to certain probability, rotates and amplify
Operation, on the basis of retaining key feature, artificial expanding data amount.
Describe basic principle of the invention in conjunction with specific embodiments above, in the apparatus and method of the present invention, it is clear that
Each component or each step can be decomposed and/or be reconfigured.These decompose and/or reconfigure should be regarded as it is of the invention etc.
Efficacious prescriptions case.Also, the step of executing above-mentioned series of processes can execute according to the sequence of explanation in chronological order naturally, still
It does not need centainly to execute sequentially in time.Certain steps can execute parallel or independently of one another.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (10)
1. a kind of method of the facial skin points shape defect of detection based on deep learning, which is characterized in that including：
Training dataset is obtained, the sample that the training data is concentrated is having a size of m pixel × n-pixel, includes facial skin
The image block of skin spot defect markup information, wherein m and n is positive integer；
Convolutional neural networks are constructed, the convolutional neural networks are trained using the training dataset to obtain detection mould
Type, wherein the convolutional neural networks have following feature：The feature extraction of multilayer convolution is carried out to obtain different levels to sample
Characteristic pattern, then for the characteristic pattern of the different levels using more sizes and the sample boxes of various shapes are sampled to obtain not
Then the corresponding sampled result with level uses softmax loss function for all corresponding sampled results of different levels
L1 loss function predicted position of classifying and utilize is carried out, the front end of the penalty values being calculated passback convolutional neural networks is adopted
Convolutional neural networks parameter is adjusted with gradient descent method；
Testing image is cut into multiple having a size of m pixel × n-pixel test image block, then traversal ground is by the multiple survey
It tries image block and inputs the detection model to obtain corresponding image block testing result, then all described image blocks are tested and are tied
Fruit is spliced into complete image testing result.
2. the method according to claim 1, wherein the length-width ratio a of the sample boxes meets condition：1:3≤a≤
3:1。
3. the method according to claim 1, wherein the skin of face spot defect include it is below a kind of or
A variety of combinations：Mole, acne, acne print.
4. the method according to claim 1, wherein further including：
The training dataset and testing image are pre-processed, the pretreatment includes：It carries out invalid data to filter out, carry out
Brightness homogenization.
5. the method according to claim 1, wherein 100≤m≤600, and 100≤n≤600.
6. a kind of device of the facial skin points shape defect of detection based on deep learning, which is characterized in that including：
Obtain module, for obtaining training dataset, the sample that the training data is concentrated be having a size of m pixel × n-pixel,
It include the image block of skin of face spot defect markup information, wherein m and n is positive integer；
Modeling module instructs the convolutional neural networks using the training dataset for constructing convolutional neural networks
Practice to obtain detection model, wherein the convolutional neural networks have following feature：The feature extraction of multilayer convolution is carried out to sample
To obtain the characteristic pattern of different levels, the sampling of more sizes and various shapes then is utilized for the characteristic pattern of the different levels
Frame is sampled to obtain the corresponding sampled result of different levels, is then used for all corresponding sampled results of different levels
Softmax loss function carries out L1 loss function predicted position of classifying and utilize, by the penalty values being calculated passback convolution mind
Front end through network adjusts convolutional neural networks parameter using gradient descent method；
Detection module, it is multiple having a size of m pixel × n-pixel test image block for testing image to be cut into, then traverse
The multiple test image block is inputted the detection model to obtain corresponding image block testing result, then by all institutes by ground
It states image block test result and is spliced into complete image testing result.
7. device according to claim 6, which is characterized in that the length-width ratio a of the sample boxes meets condition：1:3≤a≤
3:1。
8. device according to claim 6, which is characterized in that the skin of face spot defect include it is below a kind of or
A variety of combinations：Mole, acne, acne print.
9. device according to claim 6, which is characterized in that further include：
Preprocessing module, for pre-processing to the training dataset and the testing image, the pretreatment includes：Into
Row invalid data filters out, and carries out brightness homogenization.
10. device according to claim 6, which is characterized in that 100≤m≤600, and 100≤n≤600.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810600716.3A CN108921825A (en) | 2018-06-12 | 2018-06-12 | The method and device of the facial skin points shape defect of detection based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810600716.3A CN108921825A (en) | 2018-06-12 | 2018-06-12 | The method and device of the facial skin points shape defect of detection based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108921825A true CN108921825A (en) | 2018-11-30 |
Family
ID=64419171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810600716.3A Pending CN108921825A (en) | 2018-06-12 | 2018-06-12 | The method and device of the facial skin points shape defect of detection based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108921825A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109858521A (en) * | 2018-12-29 | 2019-06-07 | 国际竹藤中心 | A kind of bamboo category identification method based on artificial intelligence deep learning |
| CN109934814A (en) * | 2019-03-15 | 2019-06-25 | 英业达科技有限公司 | Surface defect detecting system and its method |
| CN110008887A (en) * | 2019-03-29 | 2019-07-12 | 南京所由所以信息科技有限公司 | A kind of acne automatic grading method based on deep learning |
| CN110033443A (en) * | 2019-04-04 | 2019-07-19 | 武汉精立电子技术有限公司 | A kind of feature extraction network and its defects of display panel detection method |
| CN110458770A (en) * | 2019-07-26 | 2019-11-15 | 复旦大学附属中山医院 | A kind of medical image brightness homogenization bearing calibration |
| CN110796115A (en) * | 2019-11-08 | 2020-02-14 | 厦门美图之家科技有限公司 | Image detection method and device, electronic equipment and readable storage medium |
| CN111179253A (en) * | 2019-12-30 | 2020-05-19 | 歌尔股份有限公司 | Product defect detection method, device and system |
| CN111310775A (en) * | 2018-12-11 | 2020-06-19 | Tcl集团股份有限公司 | Data training method and device, terminal equipment and computer readable storage medium |
| CN111754497A (en) * | 2020-06-28 | 2020-10-09 | 深圳大学 | Industrial defect detection method and system based on geometric algebra |
| CN112712503A (en) * | 2020-12-30 | 2021-04-27 | 厦门福信光电集成有限公司 | Display panel appearance detection method based on deep learning |
| TWI728369B (en) * | 2019-05-24 | 2021-05-21 | 臺北醫學大學 | Method and system for analyzing skin texture and skin lesion using artificial intelligence cloud based platform |
| CN113838028A (en) * | 2021-09-24 | 2021-12-24 | 无锡祥生医疗科技股份有限公司 | Carotid artery ultrasonic automatic Doppler method, ultrasonic equipment and storage medium |
| CN116596867A (en) * | 2023-05-05 | 2023-08-15 | 理工特智科技(重庆)有限公司 | Target detection method for welding image |
| US11756332B2 (en) | 2020-06-30 | 2023-09-12 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Image recognition method, apparatus, device, and computer storage medium |
| CN117152099A (en) * | 2023-09-05 | 2023-12-01 | 深圳伯德睿捷健康科技有限公司 | Skin pore or blackhead detection method, system and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106469302A (en) * | 2016-09-07 | 2017-03-01 | 成都知识视觉科技有限公司 | A kind of face skin quality detection method based on artificial neural network |
| CN106548169A (en) * | 2016-11-02 | 2017-03-29 | 重庆中科云丛科技有限公司 | Fuzzy literal Enhancement Method and device based on deep neural network |
| CN107909566A (en) * | 2017-10-28 | 2018-04-13 | 杭州电子科技大学 | A kind of image-recognizing method of the cutaneum carcinoma melanoma based on deep learning |
| CN107945173A (en) * | 2017-12-11 | 2018-04-20 | 深圳市宜远智能科技有限公司 | A kind of skin disease detection method and system based on deep learning |
-
2018
- 2018-06-12 CN CN201810600716.3A patent/CN108921825A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106469302A (en) * | 2016-09-07 | 2017-03-01 | 成都知识视觉科技有限公司 | A kind of face skin quality detection method based on artificial neural network |
| CN106548169A (en) * | 2016-11-02 | 2017-03-29 | 重庆中科云丛科技有限公司 | Fuzzy literal Enhancement Method and device based on deep neural network |
| CN107909566A (en) * | 2017-10-28 | 2018-04-13 | 杭州电子科技大学 | A kind of image-recognizing method of the cutaneum carcinoma melanoma based on deep learning |
| CN107945173A (en) * | 2017-12-11 | 2018-04-20 | 深圳市宜远智能科技有限公司 | A kind of skin disease detection method and system based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| WEI LIU ET AL: "SSD: Single Shot MultiBox Detector", 《ECCV 2016》 * |
| 张毅: "基于亮度归一化的人脸识别的研究及应用", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310775B (en) * | 2018-12-11 | 2023-08-25 | Tcl科技集团股份有限公司 | Data training method, device, terminal equipment and computer readable storage medium |
| CN111310775A (en) * | 2018-12-11 | 2020-06-19 | Tcl集团股份有限公司 | Data training method and device, terminal equipment and computer readable storage medium |
| CN109858521B (en) * | 2018-12-29 | 2021-01-01 | 国际竹藤中心 | Bamboo species identification method based on artificial intelligence deep learning |
| CN109858521A (en) * | 2018-12-29 | 2019-06-07 | 国际竹藤中心 | A kind of bamboo category identification method based on artificial intelligence deep learning |
| CN109934814A (en) * | 2019-03-15 | 2019-06-25 | 英业达科技有限公司 | Surface defect detecting system and its method |
| CN109934814B (en) * | 2019-03-15 | 2022-02-01 | 英业达科技有限公司 | Surface defect detection system and method thereof |
| CN110008887A (en) * | 2019-03-29 | 2019-07-12 | 南京所由所以信息科技有限公司 | A kind of acne automatic grading method based on deep learning |
| CN110033443A (en) * | 2019-04-04 | 2019-07-19 | 武汉精立电子技术有限公司 | A kind of feature extraction network and its defects of display panel detection method |
| TWI728369B (en) * | 2019-05-24 | 2021-05-21 | 臺北醫學大學 | Method and system for analyzing skin texture and skin lesion using artificial intelligence cloud based platform |
| CN110458770A (en) * | 2019-07-26 | 2019-11-15 | 复旦大学附属中山医院 | A kind of medical image brightness homogenization bearing calibration |
| CN110796115A (en) * | 2019-11-08 | 2020-02-14 | 厦门美图之家科技有限公司 | Image detection method and device, electronic equipment and readable storage medium |
| CN110796115B (en) * | 2019-11-08 | 2022-12-23 | 厦门美图宜肤科技有限公司 | Image detection method and device, electronic equipment and readable storage medium |
| CN111179253B (en) * | 2019-12-30 | 2023-11-24 | 歌尔股份有限公司 | Product defect detection method, device and system |
| CN111179253A (en) * | 2019-12-30 | 2020-05-19 | 歌尔股份有限公司 | Product defect detection method, device and system |
| CN111754497A (en) * | 2020-06-28 | 2020-10-09 | 深圳大学 | Industrial defect detection method and system based on geometric algebra |
| CN111754497B (en) * | 2020-06-28 | 2023-09-01 | 深圳大学 | Industrial defect detection method and system based on geometric algebra |
| US11756332B2 (en) | 2020-06-30 | 2023-09-12 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Image recognition method, apparatus, device, and computer storage medium |
| CN112712503A (en) * | 2020-12-30 | 2021-04-27 | 厦门福信光电集成有限公司 | Display panel appearance detection method based on deep learning |
| CN112712503B (en) * | 2020-12-30 | 2023-05-02 | 厦门福信光电集成有限公司 | Display panel appearance detection method based on deep learning |
| CN113838028A (en) * | 2021-09-24 | 2021-12-24 | 无锡祥生医疗科技股份有限公司 | Carotid artery ultrasonic automatic Doppler method, ultrasonic equipment and storage medium |
| CN116596867A (en) * | 2023-05-05 | 2023-08-15 | 理工特智科技(重庆)有限公司 | Target detection method for welding image |
| CN116596867B (en) * | 2023-05-05 | 2024-02-13 | 理工特智科技(重庆)有限公司 | Target detection method for welding image |
| CN117152099A (en) * | 2023-09-05 | 2023-12-01 | 深圳伯德睿捷健康科技有限公司 | Skin pore or blackhead detection method, system and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921825A (en) | The method and device of the facial skin points shape defect of detection based on deep learning | |
| CN107240091B (en) | Capsule endoscope image preprocessing system and method | |
| Lau et al. | Automatically early detection of skin cancer: Study based on nueral netwok classification | |
| CN109118485A (en) | Digestive endoscope image classification based on multitask neural network cancer detection system early | |
| CN107977671A (en) | A kind of tongue picture sorting technique based on multitask convolutional neural networks | |
| CN109670510A (en) | A kind of gastroscopic biopsy pathological data screening system and method based on deep learning | |
| CN107274386A (en) | A kind of cervical cell liquid-based smear artificial intelligence aids in diagosis system | |
| CN107330892A (en) | A kind of sunflower disease recognition method based on random forest method | |
| CN108334860A (en) | The treating method and apparatus of cell image | |
| CN108846311A (en) | The method and device of the facial pieces of skin shape defect of detection based on deep learning | |
| CN105657580B (en) | A kind of capsule endoscope video abstraction generating method | |
| CN104299242B (en) | Fluoroscopic visualization eye fundus image extracting method based on NGC ACM | |
| Renugambal et al. | Application of image processing techniques in plant disease recognition | |
| Irshad et al. | Multi-channels statistical and morphological features based mitosis detection in breast cancer histopathology | |
| Ji et al. | In-field automatic detection of maize tassels using computer vision | |
| Wan et al. | Retinal image enhancement using cycle-constraint adversarial network | |
| Kanth et al. | Identification of different stages of Diabetic Retinopathy using artificial neural network | |
| CN105975952A (en) | Beard detection method and system in video image | |
| CN110288574A (en) | A kind of adjuvant Ultrasonographic Diagnosis hepatoncus system and method | |
| Santhakumar et al. | Machine learning algorithm for retinal image analysis | |
| Yadav et al. | Automatic Cataract Severity Detection and Grading Using Deep Learning | |
| Khoshdeli et al. | Deep learning models delineates multiple nuclear phenotypes in h&e stained histology sections | |
| Zhao et al. | Convolutional neural networks for branch retinal vein occlusion recognition? | |
| Iqbal et al. | Automatic diagnosis of diabetic retinopathy using fundus images | |
| CN105874508A (en) | Method and apparatus for cervical image analysis with improved reliability |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181130 |