CN111881789B - Skin color identification method, device, computing equipment and computer storage medium - Google Patents
Skin color identification method, device, computing equipment and computer storage medium Download PDFInfo
- Publication number
- CN111881789B CN111881789B CN202010676613.2A CN202010676613A CN111881789B CN 111881789 B CN111881789 B CN 111881789B CN 202010676613 A CN202010676613 A CN 202010676613A CN 111881789 B CN111881789 B CN 111881789B
- Authority
- CN
- China
- Prior art keywords
- skin color
- skin
- target
- color
- category
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000001514 detection method Methods 0.000 claims abstract description 55
- 239000013598 vector Substances 0.000 claims description 97
- 238000004891 communication Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 11
- 239000003086 colorant Substances 0.000 abstract description 7
- 238000012549 training Methods 0.000 description 9
- 238000011176 pooling Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 241001085205 Prenanthella exigua Species 0.000 description 1
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention relates to the technical field of face recognition and discloses a skin color recognition method, a device, a computing device and a computer storage medium, wherein the method comprises the following steps: acquiring a face image to be identified; inputting the face image to be identified into a skin color type detection model to obtain a target skin color type and a target skin color candidate area, wherein the target skin color type is the skin color type of the skin corresponding to the face image identified and determined by the skin color type detection model; according to the target skin color candidate region, determining a first skin color template with highest similarity with the target skin color candidate region in a plurality of preset skin color templates, wherein one skin color template corresponds to one skin color in one skin color category, and the one skin color category comprises a plurality of skin colors; and if the skin color category corresponding to the first skin color template is the target skin color category, determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image. Through the mode, the embodiment of the invention realizes skin color identification.
Description
Technical Field
The embodiment of the invention relates to the technical field of face recognition, in particular to a skin color recognition method, a skin color recognition device, a computing device and a computer storage medium.
Background
Face recognition is a biological recognition technology for carrying out identity recognition based on facial feature information of people. The face image contains face feature information, and therefore, the face image is the basis of face recognition.
With the improvement of living matter level of people, demands of people on personal image design are rapidly increasing. The user is required to select proper foundation color number, make-up, accessory and the like according to the skin color of the user, so that the quick and accurate acquisition of the skin color of the user is the basis for personal image design of the user.
At present, the skin color recognition of the human face is mainly carried out by dividing an effective area of the human face and setting a threshold value for the effective area of the human face. When the difference between specific skin colors is subtle, the skin colors cannot be accurately identified in this way.
Disclosure of Invention
In view of the above problems, embodiments of the present invention provide a skin color recognition method, apparatus, computing device, and computer storage medium, which are used to solve the problem of inaccurate skin color recognition in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a skin color recognition method, the method including:
Acquiring a face image to be identified;
Inputting the face image to be identified into a skin color type detection model to obtain a target skin color type and a target skin color candidate region, wherein the target skin color type is the skin color type which the skin corresponding to the face image identified and determined by the skin color type detection model belongs to, the target skin color candidate region is the skin color candidate region which is obtained by processing the face image to be identified by the skin color type detection model, and the target skin color candidate region is used for indicating that the skin color type which the skin corresponding to the face image belongs to is the target skin color type;
According to the target skin color candidate region, determining a first skin color template with highest similarity with the target skin color candidate region in a plurality of preset skin color templates, wherein one skin color template corresponds to one skin color in one skin color category, and one skin color category comprises a plurality of skin colors;
And if the skin color category corresponding to the first skin color template is the target skin color category, determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image.
In an alternative, the method further comprises:
If the skin tone category corresponding to the target skin tone is not the target skin tone category, determining a second skin tone template with highest similarity with the target skin tone candidate area in a plurality of skin tone templates corresponding to the target skin tone category, and determining the skin tone corresponding to the second skin tone template as the skin tone of the skin corresponding to the face image.
In an optional manner, the inputting the face image to be identified into a skin color category detection model to obtain a target skin color category and a target skin color candidate region includes:
Inputting the face image to be recognized into a skin color type detection model to obtain a plurality of skin color candidate areas obtained by processing the face image to be recognized by the skin color type detection model and score vectors corresponding to each skin color candidate area, wherein one score vector comprises a plurality of scores which are respectively used for indicating the probability that the skin color candidate area belongs to each skin color type;
According to the score vector corresponding to each skin color candidate region, determining a skin color category score corresponding to each skin color candidate region, wherein the skin color category score is the maximum score in the score vector corresponding to the skin color candidate region;
Determining a skin color category corresponding to a skin color candidate region corresponding to the maximum skin color category score as a target skin color category;
And determining the skin color candidate region corresponding to the target skin color category as a target skin color candidate region.
In an optional manner, the determining the skin tone candidate region corresponding to the target skin tone category as the target skin tone candidate region includes:
and in the skin color candidate areas corresponding to the target skin color category, the first n skin color candidate areas with the highest skin color category score are determined as target skin color candidate areas, and n is a positive integer.
In an optional manner, the determining, according to the target skin color candidate region, a first skin color template with highest similarity to the target skin color candidate region from a plurality of preset skin color templates includes:
Determining a target color vector corresponding to the target candidate region, wherein the target color vector is used for representing color characteristics of the target candidate region on different color channels;
calculating the similarity distance between the target color vector and the color vectors corresponding to the skin color templates respectively, wherein one color vector is used for indicating the color characteristics of one skin color template on different color channels;
And determining a skin color template corresponding to the minimum similarity distance as the first skin color template.
In an optional manner, the determining the target color vector corresponding to the target candidate region includes:
Restoring the target candidate region into a target image with the same size as the face image to be identified;
And calculating the characteristic value of the target image on each color channel to obtain a target color vector corresponding to the target candidate region.
In an optional manner, before calculating the similarity distance between the target color vector and the color vectors corresponding to the skin tone templates, the method further includes:
acquiring a plurality of skin color images corresponding to each skin color template;
calculating the characteristic value of each skin color image in the plurality of skin color images on each color channel to obtain the color vector of each skin color image;
and calculating the average value of the color vector values of the multiple skin color images corresponding to the same skin color template to obtain the color vector value corresponding to each skin color template.
According to another aspect of the embodiment of the present invention, there is provided a skin color recognition apparatus including:
The acquisition module is used for acquiring the face image to be identified;
the input module is used for inputting the face image to be identified into a skin color type detection model to obtain a target skin color type and a target skin color candidate area, wherein the target skin color type is the skin color type of the skin corresponding to the face image identified and determined by the skin color type detection model, the target skin color candidate area is the skin color candidate area obtained by processing the face image to be identified by the skin color type detection model, and the target skin color candidate area is used for indicating that the skin color type of the skin corresponding to the face image is the target skin color type;
The first determining module is used for determining a first skin color template with highest similarity with the target skin color candidate area in a plurality of preset skin color templates according to the target skin color candidate area, wherein one skin color template corresponds to one skin color in one skin color category, and the one skin color category comprises a plurality of skin colors;
and the second determining module is used for determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image when the skin color category corresponding to the first skin color template is the target skin color category.
According to another aspect of an embodiment of the present invention, there is provided a computing device including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
The memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the skin color identification method.
According to another aspect of the embodiments of the present invention, there is provided a computer readable storage medium, where at least one executable instruction is stored, where the executable instruction when executed on a computing device/apparatus causes the computing device/apparatus to perform operations corresponding to the skin tone identification method described above.
According to the embodiment of the invention, the target skin tone candidate region is determined through the skin tone type detection model, the target skin tone template is determined according to the similarity between the target skin tone candidate region and each skin tone template, and when the type corresponding to the target skin tone template is consistent with the target skin tone type, the skin tone corresponding to the face image to be identified is the skin tone corresponding to the target skin tone template. By the method, the skin color category detection model can obtain the target skin color category corresponding to the target skin color candidate region, and the skin color corresponding to the face image to be identified is determined through the similarity between the target skin color candidate region and each skin color template on the basis of the target skin color category. Because the distances between different skin tone templates and the target skin tone candidate region can reflect the similarity between the target skin tone candidate region and each skin tone template, even if the skin tone templates with nuances exist, the skin tone template which is most similar to the target skin tone candidate region can be distinguished by calculating the distance between the skin tone template and the target skin tone candidate region. Therefore, the skin color corresponding to the face image to be recognized can be accurately recognized through the mode.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present invention can be more clearly understood, and the following specific embodiments of the present invention are given for clarity and understanding.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 shows a flow chart of a skin color recognition method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a skin color category detection model in a skin color identification method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a skin color recognition method according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of a skin color recognition device according to an embodiment of the present invention;
FIG. 5 illustrates a schematic diagram of a computing device provided by an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein.
The application scene of the embodiment of the invention is face skin color recognition. The embodiment of the invention can be applied to any scene needing face skin color recognition, for example, under the scene needing to recognize the user race, the user race is determined through face skin color recognition. In the fields of cosmetology and personal image design, the embodiment of the invention can be applied to a cosmetology application program or equipment, and the application program or equipment can determine the skin color of the human face by collecting the human face image, so that proper cosmetology, accessories, skin care products and the like are recommended for a user according to the skin color of the human face. The following describes the embodiments of the present invention.
Fig. 1 shows a flowchart of a skin tone recognition method according to an embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
Step 110: and acquiring a face image to be identified.
The face image to be identified is a face image which needs to be subjected to skin color identification. The face image to be identified can be manually input or can be acquired on site through the image acquisition equipment. For example, the face image to be identified is acquired by a camera. In order to obtain face images with proper sizes conveniently, a face outline can be arranged in a shooting area of the camera, and a user is prompted to adjust shooting angles and distances.
Step 120: and inputting the face image to be identified into a skin color category detection model to obtain a target skin color category and a target skin color candidate region.
In this step, the skin color class detection model is obtained by training the target detection model with a plurality of sets of training data. Each set of training data comprises a face image, coordinates of skin color recognition areas in the face image, and skin color categories corresponding to the face image. The coordinates of the skin color recognition area in the face image are marked manually. When labeling, a region with a cleaner face in the face image is selected, so that hair shielding, obvious skin color brightness change and abnormal face regions are avoided. The skin color category corresponding to the face image can be identified by the preset label corresponding to each skin color category, and the embodiment of the invention is not limited to the specific form of the preset label corresponding to the skin color category. For example, in one embodiment, the skin tone categories include six of bright white, red, natural, wheat, dark and dark, represented using the Arabic numerals 1, 2, 3, 4, 5, 6, respectively.
The object detection model in the embodiment of the invention can be any object detection model in the prior art, for example, a convolutional neural network model, a single-lens multi-box detector (single shot multibox detector, SSD) and the like. Training any target detection model through multiple sets of training data to obtain a skin color class detection model. The skin color category detection model comprises a convolution layer, a pooling layer and a full connection layer. The convolution layer receives the face image and performs feature extraction on the face image. The feature extraction results of the face images of different convolution cores in the convolution layer are that a plurality of skin color candidate areas are obtained. And the pooling layer reduces the dimension of the features extracted by the convolution layer. Embodiments of the present invention are not limited to a particular pooling scheme of the pooling layer, e.g., in a particular implementation, the pooling layer selects maximum pooling. The full connection layer processes the feature of the pooling layer after dimension reduction to obtain coordinates of a plurality of skin color candidate areas corresponding to the input face image and score vectors corresponding to the coordinates of each skin color candidate area. The specific training process is a conventional training process of each target detection model, and will not be described herein.
In some embodiments, the number of convolution kernels of the feature extraction module in the target detection model is randomly reduced, and because the detection targets in the embodiment of the invention are skin color candidate areas in the face image, namely, areas capable of representing the skin colors of the face in the face image, the detection targets are fewer, so that the training speed of the skin color type detection model can be increased while the training of the skin color type detection model is ensured by randomly reducing the number of convolution kernels.
After the face image to be recognized is input into a skin color category detection model, coordinates of a plurality of skin color candidate areas corresponding to the face image to be recognized and a score vector corresponding to each skin color candidate area are obtained, wherein the score vector comprises a plurality of scores, the number of scores contained in the score vector is the same as the number of skin color categories, and each score represents the probability that the skin color candidate area belongs to each skin color category.
The target skin tone candidate region is a candidate region for determining a skin tone class in the face image to be identified. And determining the skin color category and the skin color category score corresponding to each skin color candidate region according to the maximum score in the score vector corresponding to each skin color candidate region in the obtained multiple skin color candidate regions. Wherein the skin color category score is the maximum value in the score vector corresponding to the candidate region. The skin color category corresponding to the skin color candidate region with the highest skin color category score is the target skin color category, and the skin color candidate region corresponding to the target skin color category is the target skin color candidate region.
Step 130: and determining a first skin color template with highest similarity with the target skin color candidate region from a plurality of preset skin color templates according to the target skin color candidate region.
In this step, each skin tone template corresponds to a skin tone, and the same skin tone category corresponds to at least one skin tone template. The skin color class is obtained by identification according to a skin color identification model, the skin color class is coarser in skin color division, and the skin color corresponding to the skin color template is finer. For example, the skin color class is fair, and the six skin color templates corresponding to the skin color class respectively represent different fair degrees.
In some embodiments, the similarity between the target skin tone candidate region and each skin tone template is determined based on a similarity distance between a target color vector corresponding to the target skin tone candidate region and a color vector corresponding to each skin tone template. The color vector of the target skin color candidate region is used for representing the color characteristics of the target candidate region on different color channels. The color vectors of the skin tone templates are used to characterize the color characteristics of the skin tone templates over different color channels.
The color vector in the embodiment of the invention can be a color vector formed in any color space, and the target color vector corresponding to the target skin color candidate area and the color vector corresponding to each skin color template are color vectors in the same color space. Preferably, the color vector is a color vector in YCrCb color space, and the color vector in YCrCb color space can separate the brightness features in the skin color candidate region, so that the obtained color vector is more accurate.
In some embodiments, the color vector p= [ C Y,CCr,CCb,SY,SCr,SCb ] corresponding to the target skin tone candidate region, where C Y,CCr,CCb represents the average value of all pixels of the target skin tone candidate region on the Y channel, the Cr channel, and the Cb channel, and S Y,SCr,SCb represents the variance of all pixels of the target skin tone candidate region on the Y channel, the Cr channel, and the Cb channel, respectively. Features of the target skin color candidate region can be reflected through the mean and the variance. Wherein,
Where m×n represents the size of the target skin tone candidate region, C Yi represents the i-th pixel of the target skin tone candidate region on the Y channel, C Cri represents the i-th pixel of the target skin tone candidate region on the Cr channel, and C Cbi represents the i-th pixel of the target skin tone candidate region on the Cb channel. The size of the target skin color candidate region is consistent with the size of the skin color template, and if the size of the target skin color candidate region is inconsistent with the size of the skin color template, the size of the target skin color candidate region is converted into the size consistent with the size of the skin color template through the size operation, so that the calculation accuracy is ensured.
Assuming that the color vector of one of the skin tone templates is t= [ C Y',CCr',CCb',SY',SCr',SCb' ], the similarity distance between the target skin tone candidate region and the skin tone template is:
Step 140: and if the skin color category corresponding to the first skin color template is the target skin color category, determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image.
In the step, if the skin color category corresponding to the target skin color template is the target skin color category, the skin color of the target skin color template obtained through similarity is consistent with the skin color category obtained through the identification of the skin color category detection model, and the skin color corresponding to the target skin color template is finer than the target skin color category. In this case, the skin color corresponding to the target skin color template is used as the skin color corresponding to the face image to be identified.
According to the embodiment of the invention, the target skin tone candidate region is determined through the skin tone type detection model, the target skin tone template is determined according to the similarity between the target skin tone candidate region and each skin tone template, and when the type corresponding to the target skin tone template is consistent with the target skin tone type, the skin tone corresponding to the face image to be identified is the skin tone corresponding to the target skin tone template. By the method, the skin color category detection model can obtain the target skin color category corresponding to the target skin color candidate region, and the skin color corresponding to the face image to be identified is determined through the similarity between the target skin color candidate region and each skin color template on the basis of the target skin color category. Because the distances between different skin tone templates and the target skin tone candidate region can reflect the similarity between the target skin tone candidate region and each skin tone template, even if the skin tone templates with nuances exist, the skin tone template which is most similar to the target skin tone candidate region can be distinguished by calculating the distance between the skin tone template and the target skin tone candidate region. Therefore, the skin color corresponding to the face image to be recognized can be accurately recognized through the mode.
In some embodiments, if the skin tone category corresponding to the target skin tone is not the target skin tone category, determining a second skin tone template with the highest similarity to the target skin tone candidate region from a plurality of skin tone templates corresponding to the target skin tone category, and determining the skin tone corresponding to the second skin tone template as the skin tone of the skin corresponding to the face image. The implementation manner of determining the second skin tone template with the highest similarity to the target candidate region in the multiple skin tone templates corresponding to the target skin tone category may refer to the implementation manner of determining the first skin tone template with the highest similarity to the target skin tone candidate region in the preset multiple skin tone templates, which is not described herein. If the skin color category corresponding to the target skin color template is not the target skin color category, the skin color of the target skin color template obtained through similarity calculation is inconsistent with the skin color category obtained through the skin color category detection model. Because the accuracy of the identification result of the skin color type detection model is high, a standard skin color template corresponding to the minimum distance value is determined in the skin color templates corresponding to the target skin color type, so that finer skin colors under the target skin color type can be obtained.
Fig. 3 shows a flowchart of a skin tone recognition method according to another embodiment of the present invention. In the embodiment of the invention, a plurality of target skin color candidate areas are provided. As shown in fig. 3, an embodiment of the present invention includes the steps of:
Step 201: and acquiring a face image to be identified.
Step 202: inputting the face image to be identified into a skin color type detection model to obtain a plurality of skin color candidate areas obtained by processing the face image to be identified by the skin color type detection model and score vectors corresponding to each skin color candidate area.
Step 203: and determining the skin color category score corresponding to each skin color candidate region according to the score vector corresponding to each skin color candidate region.
In this step, each skin color candidate region corresponds to a score vector, and the elements in the score vectors corresponding to the skin color candidate regions are the same, which respectively represent the probability that each skin color candidate region belongs to each skin color category. The maximum value in the score vector is the skin color category score, and the skin color category corresponding to the skin color category score is the skin color category corresponding to the skin color candidate region.
Step 204: and determining the skin color category corresponding to the skin color candidate region corresponding to the maximum skin color category score as the target skin color category.
In the step, after obtaining the skin color category scores corresponding to the skin color candidate areas, determining the skin color category corresponding to the skin color candidate area with the highest skin color category score as the target skin color category. For example, the number of skin color candidate regions is ten, the skin color category scores are a1 to a10 respectively, and the skin color categories corresponding to the ten skin color candidate regions may be the same or different. If the maximum value of a1 to a10 is a5 and the skin color category corresponding to a5 is fair, the fair is the target skin color category.
Step 205: and determining the first n skin color candidate areas with the highest skin color class scores as target skin color candidate areas in the skin color candidate areas corresponding to the target skin color class.
In this step, the skin tone candidate region corresponding to the target skin tone category is determined according to the skin tone candidate region having the highest skin tone category score in all the skin tone candidate regions. Among all the skin tone candidate areas, at least one skin tone candidate area with the skin tone category being the target skin tone category is provided, and the skin tone category scores corresponding to the skin tone candidate areas with the at least one skin tone category being the target skin tone category may be the same or different. And selecting the first n Zhang Fuse candidate areas with the highest skin color class scores from all the skin color candidate areas corresponding to the target skin color classes to obtain n target skin color candidate areas. n is a natural number greater than 0, and a specific value of n can be set by those skilled in the art according to a specific application scenario, which is not limited in the embodiment of the present invention. For example, in one embodiment, n=2, the target skin color category is fair. The total of five skin color candidate areas corresponding to white skin is that the skin color category scores corresponding to the five skin color candidate areas are a1, a3, a5, a7 and a8 respectively. And the first 2 skin color category scores with the largest skin color category score are a5 and a1, and the skin color candidate region corresponding to the a5 and the a1 is the target skin color candidate region.
Step 206: and determining a first skin color template with highest similarity with the target skin color candidate region from a plurality of preset skin color templates according to the target skin color candidate region.
In this step, the calculation method of the color vector value of each target skin color candidate region is the same as the calculation method of the target skin color candidate region in step 130 in fig. 1, please refer to the specific description of step 130 in fig. 1, and details are not repeated here. Taking the average value of the color vector values of the n target skin color candidate areas as a target color vector value. Assuming that the color vector values of the n target skin tone candidate regions are represented by P 1～Pn, the target color vector values
Step 207: and if the skin color category corresponding to the first skin color template is the target skin color category, determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image.
According to the embodiment of the invention, a plurality of target skin color candidate areas are determined according to branches of each skin color candidate area, a target color vector value is obtained according to the plurality of target skin color candidate areas, a target skin color template is determined according to the distance between the target color vector value and the color vector value of each template, and a skin color category corresponding to a face image to be identified is obtained according to the target skin color template. The target color vector in the embodiment of the invention is obtained according to a plurality of target skin color candidate areas, and compared with the target color vector obtained through one target skin color candidate area, the embodiment of the invention reduces the error of the target color vector and improves the accuracy of calculation.
In some embodiments, the color vector value for each skin tone template is calculated from a plurality of skin tone images for each skin tone template. Specifically, calculating a color vector value of each skin tone image in a plurality of skin tone images corresponding to each skin tone template; taking the average value of the color vector values of a plurality of skin color images corresponding to the same skin color template as the color vector value corresponding to the skin color template. Assuming that one skin tone image corresponding to one skin tone template is K skin tone images, the size of each skin tone image is m×n, taking color vector calculation under YCrCb color space as an example, the color vector t= [ C Y',CCr',CCb',SY',SCr',SCb ' ] of the skin tone template, wherein C Y',CCr',CCb ' respectively represents the average value of all pixels of the target skin tone candidate region on the Y channel, the Cr channel and the Cb channel, and S Y',SCr',SCb ' represents the variance of all pixels of the target skin tone candidate region on the Y channel, the Cr channel and the Cb channel.
Wherein C Yi ' represents the ith pixel of the skin tone image on the Y channel, C Cri ' represents the ith pixel of the skin tone image on the Cr channel, and C Cbi ' represents the ith pixel of the skin tone image on the Cb channel.
Fig. 4 shows a functional block diagram of a skin tone recognition device according to an embodiment of the invention. As shown in fig. 4, the apparatus includes: the acquisition module 310, the input module 320, the first determination module 330, and the second determination module 340. The acquiring module 310 is configured to acquire a face image to be identified. The input module 320 is configured to input the face image to be identified into a skin tone type detection model to obtain a target skin tone type and a target skin tone candidate region, where the target skin tone type is a skin tone type to which the skin corresponding to the face image determined by the skin tone type detection model belongs, the target skin tone candidate region is a skin tone candidate region obtained by processing the face image to be identified by the skin tone type detection model, and is configured to indicate that the skin tone type to which the skin corresponding to the face image belongs is the target skin tone type. The first determining module 330 is configured to determine, according to the target skin tone candidate region, a first skin tone template with highest similarity to the target skin tone candidate region from a plurality of preset skin tone templates, where one skin tone template corresponds to one skin tone in one skin tone category, and one skin tone category includes multiple skin tones. The second determining module 340 is configured to determine, when the skin tone category corresponding to the first skin tone template is the target skin tone category, the skin tone corresponding to the first skin tone template as the skin tone of the skin corresponding to the face image.
In an optional manner, the apparatus further includes a third determining module 350, configured to determine, when the skin tone category corresponding to the target skin tone is not the target skin tone category, a second skin tone template with the highest similarity to the target skin tone candidate area from among a plurality of skin tone templates corresponding to the target skin tone category, and determine the skin tone corresponding to the second skin tone template as the skin tone of the skin corresponding to the face image.
In an alternative approach, the input module 320 is further configured to:
Inputting the face image to be recognized into a skin color type detection model to obtain a plurality of skin color candidate areas obtained by processing the face image to be recognized by the skin color type detection model and score vectors corresponding to each skin color candidate area, wherein one score vector comprises a plurality of scores which are respectively used for indicating the probability that the skin color candidate area belongs to each skin color type;
According to the score vector corresponding to each skin color candidate region, determining a skin color category score corresponding to each skin color candidate region, wherein the skin color category score is the maximum score in the score vector corresponding to the skin color candidate region;
Determining a skin color category corresponding to a skin color candidate region corresponding to the maximum skin color category score as a target skin color category;
And determining the skin color candidate region corresponding to the target skin color category as a target skin color candidate region.
In an alternative approach, the input module 320 is further configured to:
and in the skin color candidate areas corresponding to the target skin color category, the first n skin color candidate areas with the highest skin color category score are determined as target skin color candidate areas, and n is a positive integer.
In an alternative manner, the first determining module 330 is further configured to:
Determining a target color vector corresponding to the target candidate region, wherein the target color vector is used for representing color characteristics of the target candidate region on different color channels;
calculating the similarity distance between the target color vector and the color vectors corresponding to the skin color templates respectively, wherein one color vector is used for indicating the color characteristics of one skin color template on different color channels;
And determining a skin color template corresponding to the minimum similarity distance as the first skin color template.
In an alternative manner, the first determining module 330 is further configured to:
restoring the target candidate region into a target image with the same size as the skin color template;
And calculating the characteristic value of the target image on each color channel to obtain a target color vector corresponding to the target candidate region.
In an alternative, the apparatus further comprises:
A first obtaining module 360, configured to obtain a plurality of skin color images corresponding to each skin color template;
A first calculating module 370, configured to calculate a feature value of each skin tone image on each color channel, so as to obtain a color vector of each skin tone image;
the second calculating module 380 is configured to calculate the average value of the color vectors of the plurality of skin color images corresponding to the same skin color template, so as to obtain the color vector corresponding to each skin color template.
According to the embodiment of the invention, the target skin tone candidate region is determined through the skin tone type detection model, the target skin tone template is determined according to the similarity between the target skin tone candidate region and each skin tone template, and when the type corresponding to the target skin tone template is consistent with the target skin tone type, the skin tone corresponding to the face image to be identified is the skin tone corresponding to the target skin tone template. By the method, the skin color category detection model can obtain the target skin color category corresponding to the target skin color candidate region, and the skin color corresponding to the face image to be identified is determined through the similarity between the target skin color candidate region and each skin color template on the basis of the target skin color category. Because the distances between different skin tone templates and the target skin tone candidate region can reflect the similarity between the target skin tone candidate region and each skin tone template, even if the skin tone templates with nuances exist, the skin tone template which is most similar to the target skin tone candidate region can be distinguished by calculating the distance between the skin tone template and the target skin tone candidate region. Therefore, the skin color corresponding to the face image to be recognized can be accurately recognized through the mode.
FIG. 5 illustrates a schematic diagram of a computing device in accordance with an embodiment of the invention, which is not limited to a particular implementation of the computing device.
As shown in fig. 5, the computing device may include: a processor 402, a communication interface (Communications Interface) 404, a memory 406, and a communication bus 408.
Wherein: processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. A communication interface 404 for communicating with network elements of other devices, such as clients or other servers. The processor 402 is configured to execute the program 410, and may specifically perform the relevant steps in the above-described embodiments of the skin tone recognition method.
In particular, program 410 may include program code including computer-executable instructions.
The processor 402 may be a central processing unit CPU, or an Application-specific integrated Circuit ASIC (Application SPECIFIC INTEGRATED Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included by the computing device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
Memory 406 for storing programs 410. Memory 406 may comprise high-speed RAM memory or may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
Program 410 may be specifically invoked by processor 402 to cause a computing device to perform steps 110-140 of fig. 1, steps 201-207 of fig. 3, or to implement the functions of modules 310-380 of fig. 4.
Embodiments of the present invention provide a computer readable storage medium storing at least one executable instruction that, when executed on a computing device/apparatus, cause the computing device/apparatus to perform a skin tone identification method as in any of the method embodiments described above.
Embodiments of the present invention provide a computer program that is callable by a processor to cause a computing device to perform a skin tone identification method as in any of the method embodiments described above.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when run on a computer, cause the computer to perform a skin tone identification method according to any of the method embodiments described above.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component, and they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specifically stated.
Claims (8)
1. A method of skin tone identification, the method comprising:
Acquiring a face image to be identified;
Inputting the face image to be identified into a skin color type detection model to obtain a target skin color type and a target skin color candidate region, wherein the target skin color type is the skin color type which the skin corresponding to the face image identified and determined by the skin color type detection model belongs to, the target skin color candidate region is the skin color candidate region which is obtained by processing the face image to be identified by the skin color type detection model, and the target skin color candidate region is used for indicating that the skin color type which the skin corresponding to the face image belongs to is the target skin color type; inputting the face image to be recognized into a skin color category detection model to obtain a target skin color category and a target skin color candidate region, wherein the method comprises the following steps: inputting the face image to be recognized into a skin color type detection model to obtain a plurality of skin color candidate areas obtained by processing the face image to be recognized by the skin color type detection model and score vectors corresponding to each skin color candidate area, wherein one score vector comprises a plurality of scores which are respectively used for indicating the probability that the skin color candidate area belongs to each skin color type; according to the score vector corresponding to each skin color candidate region, determining a skin color category score corresponding to each skin color candidate region, wherein the skin color category score is the maximum score in the score vector corresponding to the skin color candidate region; determining a skin color category corresponding to a skin color candidate region corresponding to the maximum skin color category score as a target skin color category; determining a skin color candidate region corresponding to the target skin color category as a target skin color candidate region;
According to the target skin color candidate region, determining a first skin color template with highest similarity with the target skin color candidate region from a plurality of preset skin color templates, wherein the method comprises the following steps: determining a target color vector corresponding to the target candidate region, wherein the target color vector is used for representing color characteristics of the target candidate region on different color channels; calculating the similarity distance between the target color vector and the color vectors corresponding to the skin color templates respectively, wherein one color vector is used for indicating the color characteristics of one skin color template on different color channels; determining a skin color template corresponding to the minimum similarity distance as the first skin color template; one skin tone template corresponds to one skin tone in one skin tone category, and one skin tone category comprises a plurality of skin tones;
And if the skin color category corresponding to the first skin color template is the target skin color category, determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image.
2. The method according to claim 1, wherein the method further comprises:
If the skin tone category corresponding to the target skin tone is not the target skin tone category, determining a second skin tone template with highest similarity with the target skin tone candidate area in a plurality of skin tone templates corresponding to the target skin tone category, and determining the skin tone corresponding to the second skin tone template as the skin tone of the skin corresponding to the face image.
3. The method of claim 1, wherein the determining a skin tone candidate region corresponding to the target skin tone category as the target skin tone candidate region comprises:
and in the skin color candidate areas corresponding to the target skin color category, the first n skin color candidate areas with the highest skin color category score are determined as target skin color candidate areas, and n is a positive integer.
4. The method of claim 1, wherein the determining the target color vector corresponding to the target candidate region comprises:
restoring the target candidate region into a target image with the same size as the skin color template;
And calculating the characteristic value of the target image on each color channel to obtain a target color vector corresponding to the target candidate region.
5. The method of claim 1, wherein prior to calculating the similarity distance between the target color vector and the respective color vectors of the plurality of skin tone templates, further comprising:
acquiring a plurality of skin color images corresponding to each skin color template;
calculating the characteristic value of each skin color image on each color channel to obtain the color vector of each skin color image;
And calculating the average value of the color vectors of the multiple skin color images corresponding to the same skin color template to obtain the color vector corresponding to each skin color template.
6. A skin tone identification device, the device comprising:
The acquisition module is used for acquiring the face image to be identified;
the input module is used for inputting the face image to be identified into a skin color type detection model to obtain a target skin color type and a target skin color candidate area, wherein the target skin color type is the skin color type of the skin corresponding to the face image identified and determined by the skin color type detection model, the target skin color candidate area is the skin color candidate area obtained by processing the face image to be identified by the skin color type detection model, and the target skin color candidate area is used for indicating that the skin color type of the skin corresponding to the face image is the target skin color type; inputting the face image to be recognized into a skin color category detection model to obtain a target skin color category and a target skin color candidate region, wherein the method comprises the following steps: inputting the face image to be recognized into a skin color type detection model to obtain a plurality of skin color candidate areas obtained by processing the face image to be recognized by the skin color type detection model and score vectors corresponding to each skin color candidate area, wherein one score vector comprises a plurality of scores which are respectively used for indicating the probability that the skin color candidate area belongs to each skin color type; according to the score vector corresponding to each skin color candidate region, determining a skin color category score corresponding to each skin color candidate region, wherein the skin color category score is the maximum score in the score vector corresponding to the skin color candidate region; determining a skin color category corresponding to a skin color candidate region corresponding to the maximum skin color category score as a target skin color category; determining a skin color candidate region corresponding to the target skin color category as a target skin color candidate region;
The first determining module is configured to determine, according to the target skin color candidate region, a first skin color template with highest similarity to the target skin color candidate region from among a plurality of preset skin color templates, where the first skin color template includes: determining a target color vector corresponding to the target candidate region, wherein the target color vector is used for representing color characteristics of the target candidate region on different color channels; calculating the similarity distance between the target color vector and the color vectors corresponding to the skin color templates respectively, wherein one color vector is used for indicating the color characteristics of one skin color template on different color channels; determining a skin color template corresponding to the minimum similarity distance as the first skin color template; one skin tone template corresponds to one skin tone in one skin tone category, and one skin tone category comprises a plurality of skin tones;
and the second determining module is used for determining the skin color corresponding to the first skin color template as the skin color of the skin corresponding to the face image when the skin color category corresponding to the first skin color template is the target skin color category.
7. A computing device, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
The memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the skin tone identification method according to any one of claims 1-5.
8. A computer readable storage medium having stored therein at least one executable instruction which, when executed on a computing device/apparatus, causes the computing device/apparatus to perform operations corresponding to the skin tone identification method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010676613.2A CN111881789B (en) | 2020-07-14 | 2020-07-14 | Skin color identification method, device, computing equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010676613.2A CN111881789B (en) | 2020-07-14 | 2020-07-14 | Skin color identification method, device, computing equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111881789A CN111881789A (en) | 2020-11-03 |
| CN111881789B true CN111881789B (en) | 2024-06-25 |
Family
ID=73151630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010676613.2A Active CN111881789B (en) | 2020-07-14 | 2020-07-14 | Skin color identification method, device, computing equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111881789B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112837304B (en) * | 2021-02-10 | 2024-03-12 | 姜京池 | Skin detection method, computer storage medium and computing device |
| CN113221695B (en) * | 2021-04-29 | 2023-12-12 | 深圳数联天下智能科技有限公司 | Method for training skin color recognition model, method for recognizing skin color and related device |
| CN113674366A (en) * | 2021-07-08 | 2021-11-19 | 北京旷视科技有限公司 | Skin color identification method and device and electronic equipment |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7376270B2 (en) * | 2003-12-29 | 2008-05-20 | Canon Kabushiki Kaisha | Detecting human faces and detecting red eyes |
| CN105893925A (en) * | 2015-12-01 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Human hand detection method based on complexion and device |
| CN107730446B (en) * | 2017-10-31 | 2022-02-18 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer equipment and computer readable storage medium |
| CN108701217A (en) * | 2017-11-23 | 2018-10-23 | 深圳和而泰智能控制股份有限公司 | A kind of face complexion recognition methods, device and intelligent terminal |
| CN108269290A (en) * | 2018-01-19 | 2018-07-10 | 厦门美图之家科技有限公司 | Skin complexion recognition methods and device |
| CN108665408A (en) * | 2018-05-21 | 2018-10-16 | 北京微播视界科技有限公司 | Method for regulating skin color, device and electronic equipment |
| CN109639982B (en) * | 2019-01-04 | 2020-06-30 | Oppo广东移动通信有限公司 | Image noise reduction method and device, storage medium and terminal |
| CN111062891A (en) * | 2019-12-16 | 2020-04-24 | Oppo广东移动通信有限公司 | Image processing method, device, terminal and computer readable storage medium |
-
2020
- 2020-07-14 CN CN202010676613.2A patent/CN111881789B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| 基于肤色分割的人脸检测算法研究;秦立峰;何东健;;计算机工程与设计(19);全文 * |
| 基于肤色模型和模板匹配的人脸检测方法研究;刘喜荣;田启川;;太原科技大学学报(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111881789A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10635890B2 (en) | Facial recognition method and apparatus, electronic device, and storage medium | |
| CN107944020B (en) | Face image searching method and device, computer device and storage medium | |
| CN111881789B (en) | Skin color identification method, device, computing equipment and computer storage medium | |
| CN109657554B (en) | Image identification method and device based on micro expression and related equipment | |
| CN111160269A (en) | Face key point detection method and device | |
| CN104123543B (en) | A kind of eye movement recognition methods based on recognition of face | |
| CN107463865B (en) | Face detection model training method, face detection method and device | |
| CN112115818A (en) | Mask wearing identification method | |
| CN109271930B (en) | Micro-expression recognition method, device and storage medium | |
| US11380010B2 (en) | Image processing device, image processing method, and image processing program | |
| CN113449704B (en) | Face recognition model training method and device, electronic equipment and storage medium | |
| JP6071002B2 (en) | Reliability acquisition device, reliability acquisition method, and reliability acquisition program | |
| CN110838119A (en) | Human face image quality evaluation method, computer device and computer readable storage medium | |
| CN113255561B (en) | Hair information identification method, device, equipment and storage medium | |
| CN112102348A (en) | Image processing apparatus | |
| CN112101127B (en) | Face shape recognition method and device, computing equipment and computer storage medium | |
| CN111209873A (en) | High-precision face key point positioning method and system based on deep learning | |
| CN111161281A (en) | Face region identification method and device and storage medium | |
| CN112541394A (en) | Black eye and rhinitis identification method, system and computer medium | |
| Putro et al. | Fast face-CPU: a real-time fast face detector on CPU using deep learning | |
| Bharadi et al. | Multi-instance iris recognition | |
| CN112825120B (en) | Face illumination evaluation method, device, computer readable storage medium and equipment | |
| Mangla et al. | Sketch-based facial recognition: a weighted component-based approach (WCBA) | |
| Akash et al. | Improvement of haar feature based face detection in OpenCV incorporating human skin color characteristic | |
| JP2007026308A (en) | Image processing method and image processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |