CN112528939A - Quality evaluation method and device for face image - Google Patents
Quality evaluation method and device for face image Download PDFInfo
- Publication number
- CN112528939A CN112528939A CN202011534287.8A CN202011534287A CN112528939A CN 112528939 A CN112528939 A CN 112528939A CN 202011534287 A CN202011534287 A CN 202011534287A CN 112528939 A CN112528939 A CN 112528939A
- Authority
- CN
- China
- Prior art keywords
- face
- face image
- key point
- image
- judgment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a device for evaluating the quality of a face image, which are used for separately processing the face images at different angles by aiming at the variability of the face images in a monitoring scene. For the frontal face image, replacing the quality of the frontal face image by using the pixel quality in a key point region consisting of sparse frontal face key points; for the side face image, replacing the quality of the side face image by using the pixel quality in a key point region consisting of dense side face key points; and then, the quality of the face image is evaluated by calculating the Laplacian variance, the gray average value and the brightness ratio difference between the left half face and the right half face in the key point area. By adopting the scheme, the quality of the face image can be more accurately evaluated, the classification accuracy of the face quality is improved, and more high-quality face images are provided for follow-up face recognition application.
Description
Technical Field
The invention relates to the field of face image processing, in particular to a method and a device for evaluating the quality of a face image.
Background
At present, under the background that hardware computing resources are greatly improved, along with the development of machine learning and deep learning, the recognition accuracy of many mainstream face recognition models on some face data sets can reach over 99% which is remarkable. However, in practical production applications, due to the limitations of real-time performance and the quality of the face images obtained in practical applications, the recognition accuracy of the face recognition models is greatly reduced.
Particularly, for face recognition in a video monitoring scene, a monitoring camera is usually relied on to continuously shoot a crowd area, and a video image is obtained, and then an intelligent face snapshot system is used to calculate a video frame image, so that a large amount of snapshot face images are output. However, because the position of the camera in the shooting process is relatively fixed, partial occlusion, too far distance or light variation exist in people, and the quality of continuous frame pictures of the same person in the video may be uneven. Meanwhile, low-quality face pictures caused by factors such as brightness, contrast, resolution, image blur and noise and the like, and faces at different angles all affect the recognition rate of the face recognition model. Therefore, in order to improve the recognition accuracy of the face recognition model and reduce the false recognition rate to improve the performance of the recognition system, a face image quality evaluation method needs to be constructed to classify the captured face images and reject low-quality face images. At present, one of the existing evaluation methods is to train an SVM or CNN classifier by means of a large amount of image data with perfect labels, and directly output the class probability of the corresponding quality of each picture, so as to classify various conditions of the quality of the face image. The method depends on the labeling quality, the labeling process is difficult to unify, the recognition rate is low, and the deep learning-based method needs to occupy a large amount of hardware resources and has certain delay. In addition, various characteristic indexes of the face image are directly calculated, such as: and evaluating the quality of the face image directly through indexes such as the field contrast of the image, the gradient difference of gray scale, frequency components, the Laplace variance value and the like. This method does not have objectivity by using a calculation index set by human experience as an evaluation index.
Therefore, in order to improve the quality of the face image and provide high-quality data for the subsequent face recognition process, a more accurate face image quality evaluation method is needed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for evaluating the quality of a face image, which can evaluate the quality of the face image more accurately and improve the classification accuracy of the quality of the face image, thereby improving the identification accuracy of subsequent face identification.
The embodiment of the invention provides a quality evaluation method of a face image, which comprises the following steps:
acquiring a face image to be evaluated, and judging the type of the face image; the categories of the face image include: a front face image and a side face image;
extracting face key points of the face image by selecting a corresponding key point extraction method according to the class judgment result of the face image, and dividing a face key point region in the face image according to the face key points; wherein the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image;
calculating a Laplace variance value and a gray level average value of pixel data consisting of all pixel points in the face key point region, and a difference value of brightness ratios between pixel data of a left face region and a right face region divided according to the face key point region, and performing fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image according to the Laplace variance value, the gray level average value and the brightness ratio difference value;
and classifying the quality of the face image according to the judging results of the fuzzy face judgment, the bright and dark face judgment and the yin and yang face judgment to obtain a quality evaluation result of the face image.
Further, the extracting, according to the category determination result of the face image, a corresponding key point extraction method is selected to extract the face key points of the face image, and a face key point region is partitioned in the face image according to the face key points, specifically:
if the type judgment result of the face image is the front face image, respectively calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the key points of the center by taking the nose of the face image as the key point of the center; expanding a corresponding range of the second distance towards the direction of each key point of the five sense organs by using a second distance generated by multiplying each distance by a preset proportion to generate a sparse key point region of the frontal face;
if the type judgment result of the face image is the side face image, extracting dense key points of the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
Further, the preset dense face key point model is formed by training face image data, and specifically comprises the following steps:
acquiring a face image to be recognized as data to be trained, and determining the number and positions of the marks of the dense key points; wherein the data to be trained comprises: a training set, a verification set and a test set;
marking the data to be trained according to the marked number and the marked positions of the dense key points;
and training the dense face key point model to be trained through the data to be trained so as to generate the dense face key point model which is well trained.
Further, the calculation process of the gray average value specifically includes:
converting the face image into a gray image;
and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value.
Further, the performing fuzzy face determination, bright-dark face determination, and yin-yang face determination on the face image according to the laplacian variance value, the gray average value, and the difference value of the brightness ratio specifically includes:
and judging whether the difference values of the Laplace variance value, the gray level average value and the brightness ratio are in the corresponding preset threshold value ranges or not so as to perform fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image.
Correspondingly, this embodiment further provides an evaluation apparatus for human face image quality, including: the device comprises a data acquisition unit, a key point area dividing unit, a judgment unit and a quality evaluation unit;
the data acquisition unit is used for acquiring a face image to be evaluated and judging the type of the face image; the categories of the face image include: a front face image and a side face image;
the key point region dividing unit is used for extracting the face key points of the face image by selecting a corresponding key point extraction method according to the class judgment result of the face image and dividing a face key point region in the face image according to the face key points; wherein the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image;
the judging unit is used for calculating a Laplace variance value and a gray level average value of pixel data consisting of all pixel points in the face key point region and a difference value of brightness ratios between pixel data of a left face region and a right face region divided according to the face key point region, and performing fuzzy face judgment, light and dark face judgment and yin and yang face judgment on the face image according to the Laplace variance value, the gray level average value and the difference value of the brightness ratios;
the quality evaluation unit is used for classifying the quality of the face image according to the judging results of the fuzzy face judgment, the bright and dark face judgment and the yin and yang face judgment to obtain the quality evaluation result of the face image.
Further, the key point region dividing unit includes a key point region calculating unit;
the key point region calculation unit is used for calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the central key point by taking the nose of the face image as the central key point if the category judgment result of the face image is the face image; expanding a corresponding range of the second distance towards the direction of each key point of the five sense organs by using a second distance generated by multiplying each distance by a preset proportion to generate a sparse key point region of the frontal face;
if the type judgment result of the face image is the side face image, extracting dense key points of the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
Further, the key point region calculation unit comprises a side face key point model training unit;
the side face key point model training unit is used for acquiring a face image to be recognized as data to be trained and determining the number and positions of the marks of dense key points; wherein the data to be trained comprises: a training set, a verification set and a test set;
marking the data to be trained according to the marked number and the marked positions of the dense key points;
and training the dense face key point model to be trained through the data to be trained so as to generate the dense face key point model which is well trained.
Further, the determination unit includes a gradation calculation unit;
the gray level calculation unit is used for converting the face image into a gray level image;
and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value.
Further, the determination unit includes a category determination unit;
the class determination unit is used for determining whether the difference values of the laplacian variance value, the gray level average value and the brightness ratio are within corresponding preset threshold ranges, so as to perform fuzzy face determination, bright-dark face determination and yin-yang face determination on the face image.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a method and a device for evaluating the quality of a face image, which are used for obtaining the face image to be evaluated and judging the type of the face image, wherein the type of the face image comprises the following steps: a front face image and a side face image. And selecting a corresponding key point extraction method according to the category judgment result, extracting the face key points of the corresponding face image, and dividing a face key point region in the face image. The method comprises the steps of calculating a Laplace variance value of pixels in a human face key point region, a gray average value of gray pixels in the human face key point region, calculating a difference value of brightness ratios between a left face region and a right face region of a human face image, digitizing the quality conditions of the fuzzy, bright and left and right half faces of the human face image, and carrying out fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the human face image, so that the quality of the human face image is classified, and the quality of the human face image is evaluated. By adopting the scheme of the embodiment, the quality of the face image can be evaluated more accurately, and the quality category of the face image can be classified more accurately, so that the evaluation accuracy of the quality of the face image is improved, and the identification accuracy of subsequent face identification is improved.
Further, if the type judgment result of the face image is a front face image, respectively calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the key points of the center by taking the nose of the face image as the key point of the center; expanding the range of the corresponding second distance towards the direction of each key point of the five sense organs by the second distance generated by multiplying each distance by the preset proportion to generate a sparse key point region of the front face; if the type judgment result of the face image is a side face image, performing dense key point extraction on the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data. By adopting the scheme of the embodiment, the front face image and the side face image can be further subdivided, different key point marking positions and different key point area generating modes are used for processing according to the difference of the definition of the front face image and the definition of the side face image, and the accuracy of subsequent quality evaluation calculation is improved.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a method for evaluating the quality of a human face image according to the present invention;
fig. 2 is a schematic structural diagram of an embodiment of an evaluation apparatus for human face image quality provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a method for evaluating the quality of a human face image according to the present invention; as shown in fig. 1, the specific steps of the method for evaluating the quality of a face image include steps 101 to 104:
step 101: acquiring a face image to be evaluated, and judging the type of the face image; the categories of the face image include: a front face image and a side face image.
In the embodiment, a large amount of monitoring video data can be acquired through the monitoring high-resolution camera, the monitoring video is acquired through the video decoding module, the face images of the continuous frames of the monitoring video are extracted through the intelligent face snapshot system, and a large amount of face images are output. These face images have uneven quality. Secondly, due to the change of the monitoring scene, the photos shot by the camera are often divided into front faces and side faces, and the number of the side faces is large, so that the front side face condition of the human face is firstly distinguished through the human face image angle distinguishing module, all the human face images are determined to belong to the front faces or the side faces, the corresponding rapid processing method is conveniently adopted for the front face images respectively in the follow-up process, another processing method based on deep learning is adopted for the side face images, and the pertinence of human image processing is improved.
Step 102: extracting face key points of the face image by selecting a corresponding key point extraction method according to the class judgment result of the face image, and dividing a face key point region in the face image according to the face key points; the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; the positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image.
In this embodiment, the face key point detection is a key step in the field of face recognition and analysis, which is a precondition and breakthrough for other face-related problems such as automatic face recognition, expression analysis, three-dimensional face reconstruction, and three-dimensional animation. The human face key point detection means that given human face images, key points of the human face are positioned, and the key points comprise eyebrow, eye, nose, mouth and points of facial contour regions. The key points of the face are important feature points of each part of the face, and are usually contour points and corner points. The number of face key points goes through the development process from the first 5 points to over 200 points today, and fewer key points means lower computational cost and faster speed, but it must also be ensured that key points can cover key organs and provide sufficient feature information for subsequent face calculation applications. Therefore, based on the judgment result of the face image, a sparse key point extraction method is adopted for the face to generate key point areas wrapping eyes, a nose, a mouth and eyebrows. And adopting a dense key point extraction method for the side face to generate key point areas wrapping the eyes, the nose, the mouth, the eyebrows and the face contour. The sparse key point extraction method of the front face only marks a single key point according to the positions of the five sense organs, and the side face wraps the part of each human face organ by using a plurality of key points, so that the number of the key points of the front face is far less than that of the key points of the side face. The design can process the human faces in different angles in a targeted manner, and the processing speed is improved. According to the areas surrounded by the key points, the face image can be further divided into key areas, the areas surrounded by the key points can replace the main information of the whole face, and analysis and calculation can be participated in subsequent calculation.
As another example of this embodiment, according to the class determination result of the face image, extracting the face key points of the face image by selecting a corresponding key point extraction method, and according to the face key points, dividing a face key point region in the face image, specifically: if the type judgment result of the face image is a front face image, respectively calculating the distances from the five sense organ key points of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the central key point by taking the nose of the face image as the central key point; expanding the range of the corresponding second distance towards the direction of each key point of the five sense organs by the second distance generated by multiplying each distance by the preset proportion to generate a sparse key point region of the front face; if the type judgment result of the face image is a side face image, performing dense key point extraction on the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
In this embodiment, whether an image belongs to a front face or a side face is determined based on the classification result obtained by classifying the face image in step 101, and different key point extraction methods are selected to extract key points. And when the judgment result is that the face image is a front face image, adopting a method for extracting sparse face key points by adopting a key point expansion area. By using the key points containing five sense organs, the remaining key points are expanded outwards based on the nose as the center, so that the area range formed in the key points is expanded, and the positions of the key five sense organs are wrapped. After the expansion, the region wrapped by the key points is the sparse key point region of the front face. Because the number of key points of the five sense organs is small relative to the number of the side faces, only a single key point at the position of the five sense organs is needed, and the face condition of the front face is relatively clear, the calculation speed for the front face is higher.
When the judgment result is that the face image is a side face image, most of the shot faces are displayed at the positions of the side faces due to the variability of the monitoring scene, so that the method for expanding key points of five sense organs cannot well perform self-adaptive processing on the side face image. Therefore, for the extraction of the key points of the side face image, a deep learning training method can be adopted, and the high adaptability advantage of the pre-trained deep learning model is utilized to extract relatively dense key points in the side face image. These key points are densely distributed on the contour of the face, the eyebrows, and the contour of the five sense organs, and a single key point is no longer used to represent one of the five sense organs. The dense key points are distributed and wrapped on the outline of each part, and the region wrapped by the key points is the dense key point region of the side face. Moreover, the five sense organs are wrapped in the two key point areas, the positions of the forehead, the hair and the background outside the outline of the face are not touched, the key characteristics of the face are reasonably represented by replacing the whole face image through the key areas of the face, and the accuracy of subsequent quality evaluation calculation is improved. Meanwhile, the front face and the side face are distinguished and processed, so that the subsequent calculation speed is increased.
As another example of this embodiment, the preset dense face key point model is formed by training face image data, and specifically includes: acquiring a face image to be recognized as data to be trained, and determining the number and positions of the marks of the dense key points; wherein, the data to be trained comprises: a training set, a verification set and a test set; labeling the data to be trained according to the number and the positions of the dense key points; and training the dense face key point model to be trained through the data to be trained so as to generate a perfectly trained dense face key point model.
In this embodiment, the number of dense key points is determined, and in this embodiment, 98 key points are preferably selected, a face image data set is selected and made into a training set, a verification set and a test set, so that each image set includes face images at various angles, and 98 key points of a face are labeled on each image by using a labeling tool. Selecting a proper CNN algorithm model as a dense face key point model to be trained, training a training set as the input of the algorithm model to obtain the output result of the model, comparing the output result with the manually marked result, calculating the gradient and performing back propagation calculation, adjusting the parameters of the model, continuously iterating, training for multiple times, and finishing the training after the model is converged; and finally, selecting a model with the best test effect on the test set as a finally perfect dense face key point model.
Step 103: and calculating a Laplace variance value and a gray level average value of pixel data consisting of all pixel points in the key point region of the human face and a difference value of brightness ratios between the pixel data of the left face region and the pixel data of the right face region which are divided according to the key point region of the human face, and performing fuzzy face judgment, bright and dark face judgment and yin-yang face judgment on the human face image according to the Laplace variance value, the gray level average value and the difference value of the brightness ratios.
In this embodiment, for the calculation of the laplacian variance value and the gray scale average value, the image key point region needs to be converted into a gray scale map. For the calculation of the laplacian variance value, a laplacian operator is needed to filter the image, and then the filtered region internal value is normalized and averaged to calculate the variance. The laplacian is a second derivative of the image, and can detect the rapid change of the gray value of the image, and the laplacian is often used for edge detection of the image. The boundary in the normal image is clear, and the variance is large after Laplace calculation; the fuzzy image boundary information is less, and the variance is small. Therefore, when the facial features are clearly visible, the laplacian variance value is large, and the facial features are blurred, the laplacian variance value is small. Therefore, the human face in each picture can be quantitatively evaluated only by manually finding the boundary between the clearness and the fuzziness of the facial features as a threshold value.
For the gray value average value, the calculation process of the gray value average value specifically includes: converting the face image into a gray image; and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value. In the calculation process, the pixel values of the gray level images in the key region of the human face are summed and then averaged. The formula for calculating the average value of the gray levels is as follows:
wherein, M is the height of the face picture, N is the width of the face picture, gray is a gray level image, mask is a key point mask, a face key region is 1, and a non-face key region is 0.
And for the calculation of the lightness difference value between the left half face and the right half face, dividing the lightness difference value into the left half face and the right half face according to a human face key region, then respectively converting RBGs of the left half face and the right half face into HSV (hue, saturation, value) to obtain respective lightness, finally dividing the summed lightness by the normalized half face pixel value to obtain a ratio value of each half face, and taking the ratio difference of the left half face and the right half face as a final yin-yang face judgment index. The ratio difference rate is calculated as follows:
wherein m represents the total pixel value, lv represents the lightness of the left half face, lface represents the RGB data of the left half face, rv represents the lightness of the right half face, rface represents the RGB data of the right half face, and lvrate and rvrate represent the lightness ratio values of the left half face and the right half face, respectively.
As another example of this embodiment, the fuzzy face determination, the bright-dark face determination, and the yin-yang face determination are performed on the face image according to the difference between the laplacian variance value, the gray scale average value, and the brightness ratio, specifically: and judging whether the difference values of the Laplace variance value, the gray level average value and the brightness ratio are in the corresponding preset threshold value ranges or not so as to perform fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image.
In this embodiment, a suitable laplacian variance threshold is set, and if the laplacian variance threshold is smaller than the threshold, the face is determined to be blurred; setting a proper gray average threshold value, and judging the human face to be a dark face if the gray average threshold value is smaller than the threshold value; then dividing the key region of the face into a left half face and a right half face, setting a threshold value of HSV brightness difference values of the left half face and the right half face, and judging the face to be a yin-yang face if the HSV brightness difference value is larger than the threshold value. For the threshold value selection, two types of picture sets are manually divided in advance, such as: the same number of sharp and blurred faces are chosen, as exemplified below by the laplacian variance threshold. Selecting a threshold range in advance, searching a threshold in the range by a computer, running a computer program for image evaluation and judgment based on the threshold to obtain an evaluation result, comparing the evaluation result with a result of manual calibration, and taking the threshold with high accuracy as a Laplace variance threshold.
Step 104: and classifying the quality of the face image according to the judging results of fuzzy face judgment, bright and dark face judgment and yin-yang face judgment to obtain the quality evaluation result of the face image.
In this embodiment, according to the judgment result of the threshold, the image of which quality class the current face image belongs to is judged. In application, the fuzzy face, the dark face and the yin-yang face all belong to the situation that the quality of the face image is poor, so that for the same face, only if the laplacian variance value, the gray level average value and the ratio difference value meet the requirements, subsequent face recognition is carried out, and any unsatisfied requirement directly gives up the subsequent recognition of the face of the frame.
The specific steps of the embodiment of the present invention can be, but are not limited to, refer to the quality evaluation method of the face image of the above embodiment; referring to fig. 2, fig. 2 is a schematic structural diagram of an embodiment of an evaluation apparatus for human face image quality provided by the present invention. The evaluation device of the human face image quality comprises: a data acquisition unit 201, a key point region dividing unit 202, a determination unit 203, and a quality evaluation unit 204;
the data acquisition unit 201 is configured to acquire a face image to be evaluated, and determine a category of the face image; the categories of the face image include: a front face image and a side face image;
the key point region dividing unit 202 is configured to extract a face key point of the face image by selecting a corresponding key point extraction method according to a category determination result of the face image, and divide a face key point region in the face image according to the face key point; the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; the positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image;
the determination unit 203 is configured to calculate a laplacian square difference value and a gray average value of pixel data composed of all pixel points in a face key point region, and a difference value of brightness ratios between pixel data of a left face region and a right face region divided according to the face key point region, and perform fuzzy face determination, bright-dark face determination, and yin-yang face determination on a face image according to the laplacian square difference value, the gray average value, and the difference value of the brightness ratios;
the quality evaluation unit 204 is configured to classify the quality of the face image according to the determination results of the blurred face determination, the bright-dark face determination, and the yin-yang face determination, and obtain a quality evaluation result of the face image.
In the present embodiment, the keypoint region division unit 202 includes a keypoint region calculation unit;
the key point region calculating unit is used for respectively calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the central key point by taking the nose of the face image as the central key point if the category judgment result of the face image is the front face image; expanding the range of the corresponding second distance towards the direction of each key point of the five sense organs by the second distance generated by multiplying each distance by the preset proportion to generate a sparse key point region of the front face;
if the type judgment result of the face image is a side face image, performing dense key point extraction on the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
In this embodiment, the key point region calculating unit includes a side face key point model training unit;
the side face key point model training unit is used for acquiring a face image to be recognized as data to be trained and determining the number and positions of the marks of the dense key points; wherein, the data to be trained comprises: a training set, a verification set and a test set;
labeling the data to be trained according to the number and the positions of the dense key points;
training the dense face key point model to be trained through the data to be trained to generate
And training a perfect dense face key point model.
In the present embodiment, the determination unit 203 includes a gradation calculation unit;
the gray level calculation unit is used for converting the face image into a gray level image;
and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value.
In the present embodiment, the determination unit 203 includes a category determination unit;
the type judging unit is used for judging whether the difference values of the Laplace variance value, the gray level average value and the brightness ratio are in the corresponding preset threshold value ranges or not so as to perform fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image.
The embodiment of the invention provides a method and a device for evaluating the quality of a face image, which are used for obtaining the face image to be evaluated and judging the type of the face image, wherein the type of the face image comprises the following steps: a front face image and a side face image. And selecting a corresponding key point extraction method according to the category judgment result, extracting the face key points of the corresponding face image, and dividing a face key point region in the face image. The method comprises the steps of calculating a Laplace variance value of pixels in a human face key point region, a gray average value of gray pixels in the human face key point region, calculating a difference value of brightness ratios between a left face region and a right face region of a human face image, digitizing the quality conditions of the fuzzy, bright and left and right half faces of the human face image, and carrying out fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the human face image, so that the quality of the human face image is classified, and the quality of the human face image is evaluated. By adopting the scheme of the embodiment, the quality of the face image can be evaluated more accurately, and the quality category of the face image can be classified more accurately, so that the evaluation accuracy of the quality of the face image is improved, and the identification accuracy of subsequent face identification is improved.
Further, if the type judgment result of the face image is a front face image, respectively calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the key points of the center by taking the nose of the face image as the key point of the center; expanding the range of the corresponding second distance towards the direction of each key point of the five sense organs by the second distance generated by multiplying each distance by the preset proportion to generate a sparse key point region of the front face; if the type judgment result of the face image is a side face image, performing dense key point extraction on the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data. By adopting the scheme of the embodiment, the front face image and the side face image can be further subdivided, different key point marking positions and different key point area generating modes are used for processing according to the difference of the definition of the front face image and the definition of the side face image, and the accuracy of subsequent quality evaluation calculation is improved.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (10)
1. A quality evaluation method of a face image is characterized by comprising the following steps:
acquiring a face image to be evaluated, and judging the type of the face image; the categories of the face image include: a front face image and a side face image;
extracting face key points of the face image by selecting a corresponding key point extraction method according to the class judgment result of the face image, and dividing a face key point region in the face image according to the face key points; wherein the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image;
calculating a Laplace variance value and a gray level average value of pixel data consisting of all pixel points in the face key point region, and a difference value of brightness ratios between pixel data of a left face region and a right face region divided according to the face key point region, and performing fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image according to the Laplace variance value, the gray level average value and the brightness ratio difference value;
and classifying the quality of the face image according to the judging results of the fuzzy face judgment, the bright and dark face judgment and the yin and yang face judgment to obtain a quality evaluation result of the face image.
2. The method for evaluating the quality of a face image according to claim 1, wherein the face key points of the face image are extracted by selecting a corresponding key point extraction method according to the category judgment result of the face image, and a face key point region is divided in the face image according to the face key points, specifically:
if the type judgment result of the face image is the front face image, respectively calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the key points of the center by taking the nose of the face image as the key point of the center; expanding a corresponding range of the second distance towards the direction of each key point of the five sense organs by using a second distance generated by multiplying each distance by a preset proportion to generate a sparse key point region of the frontal face;
if the type judgment result of the face image is the side face image, extracting dense key points of the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
3. The method for evaluating the quality of a face image according to claim 2, wherein the preset dense face key point model is trained by face image data, and specifically comprises:
acquiring a face image to be recognized as data to be trained, and determining the number and positions of the marks of the dense key points; wherein the data to be trained comprises: a training set, a verification set and a test set;
marking the data to be trained according to the marked number and the marked positions of the dense key points;
and training the dense face key point model to be trained through the data to be trained so as to generate the dense face key point model which is well trained.
4. The method for evaluating the quality of a human face image according to claim 1, wherein the calculation process of the gray level average value specifically comprises:
converting the face image into a gray image;
and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value.
5. The method for evaluating the quality of a human face image according to claim 1, wherein the fuzzy face determination, the bright-dark face determination, and the yin-yang face determination are performed on the human face image according to the laplacian variance value, the gray-scale average value, and the difference value of the brightness ratio, and specifically:
and judging whether the difference values of the Laplace variance value, the gray level average value and the brightness ratio are in the corresponding preset threshold value ranges or not so as to perform fuzzy face judgment, bright and dark face judgment and yin and yang face judgment on the face image.
6. An apparatus for evaluating the quality of a face image, comprising: the device comprises a data acquisition unit, a key point area dividing unit, a judgment unit and a quality evaluation unit;
the data acquisition unit is used for acquiring a face image to be evaluated and judging the type of the face image; the categories of the face image include: a front face image and a side face image;
the key point region dividing unit is used for extracting the face key points of the face image by selecting a corresponding key point extraction method according to the class judgment result of the face image and dividing a face key point region in the face image according to the face key points; wherein the positions of the key points of the human face extracted from the front face image comprise the positions of five sense organs of the human face image; positions of the key points of the human face extracted from the side face image comprise the contour positions of five sense organs and the eyebrow positions of the human face image;
the judging unit is used for calculating a Laplace variance value and a gray level average value of pixel data consisting of all pixel points in the face key point region and a difference value of brightness ratios between pixel data of a left face region and a right face region divided according to the face key point region, and performing fuzzy face judgment, light and dark face judgment and yin and yang face judgment on the face image according to the Laplace variance value, the gray level average value and the difference value of the brightness ratios;
the quality evaluation unit is used for classifying the quality of the face image according to the judging results of the fuzzy face judgment, the bright and dark face judgment and the yin and yang face judgment to obtain the quality evaluation result of the face image.
7. The apparatus for evaluating the quality of a face image according to claim 6, wherein said key point region dividing unit comprises a key point region calculating unit;
the key point region calculation unit is used for calculating the distances from the key points of five sense organs of the left eye, the right eye, the left mouth corner, the lip center and the right mouth corner in the face image to the central key point by taking the nose of the face image as the central key point if the category judgment result of the face image is the face image; expanding a corresponding range of the second distance towards the direction of each key point of the five sense organs by using a second distance generated by multiplying each distance by a preset proportion to generate a sparse key point region of the frontal face;
if the type judgment result of the face image is the side face image, extracting dense key points of the face image through a preset dense face key point model to generate a dense key point area of the side face; the preset dense face key point model is formed by training face picture data.
8. The apparatus for evaluating the quality of a face image according to claim 7, wherein said key point region calculating unit includes a side face key point model training unit;
the side face key point model training unit is used for acquiring a face image to be recognized as data to be trained and determining the number and positions of the marks of dense key points; wherein the data to be trained comprises: a training set, a verification set and a test set;
marking the data to be trained according to the marked number and the marked positions of the dense key points;
and training the dense face key point model to be trained through the data to be trained so as to generate the dense face key point model which is well trained.
9. The apparatus according to claim 6, wherein said determination unit comprises a gradation calculation unit;
the gray level calculation unit is used for converting the face image into a gray level image;
and accumulating all pixels in the human face key point area of the gray level image, and calculating an average value after accumulation to calculate a gray level average value.
10. The apparatus according to claim 6, wherein said determination means comprises a category determination means;
the class determination unit is used for determining whether the difference values of the laplacian variance value, the gray level average value and the brightness ratio are within corresponding preset threshold ranges, so as to perform fuzzy face determination, bright-dark face determination and yin-yang face determination on the face image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011534287.8A CN112528939A (en) | 2020-12-22 | 2020-12-22 | Quality evaluation method and device for face image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011534287.8A CN112528939A (en) | 2020-12-22 | 2020-12-22 | Quality evaluation method and device for face image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112528939A true CN112528939A (en) | 2021-03-19 |
Family
ID=74975821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011534287.8A Pending CN112528939A (en) | 2020-12-22 | 2020-12-22 | Quality evaluation method and device for face image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528939A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177917A (en) * | 2021-04-25 | 2021-07-27 | 重庆紫光华山智安科技有限公司 | Snapshot image optimization method, system, device and medium |
| CN113407774A (en) * | 2021-06-30 | 2021-09-17 | 广州酷狗计算机科技有限公司 | Cover determining method and device, computer equipment and storage medium |
| CN113506260A (en) * | 2021-07-05 | 2021-10-15 | 北京房江湖科技有限公司 | Face image quality evaluation method and device, electronic equipment and storage medium |
| CN113673466A (en) * | 2021-08-27 | 2021-11-19 | 深圳市爱深盈通信息技术有限公司 | Method for extracting photo stickers based on face key points, electronic equipment and storage medium |
-
2020
- 2020-12-22 CN CN202011534287.8A patent/CN112528939A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177917A (en) * | 2021-04-25 | 2021-07-27 | 重庆紫光华山智安科技有限公司 | Snapshot image optimization method, system, device and medium |
| CN113177917B (en) * | 2021-04-25 | 2023-10-13 | 重庆紫光华山智安科技有限公司 | Method, system, equipment and medium for optimizing snap shot image |
| CN113407774A (en) * | 2021-06-30 | 2021-09-17 | 广州酷狗计算机科技有限公司 | Cover determining method and device, computer equipment and storage medium |
| CN113506260A (en) * | 2021-07-05 | 2021-10-15 | 北京房江湖科技有限公司 | Face image quality evaluation method and device, electronic equipment and storage medium |
| CN113506260B (en) * | 2021-07-05 | 2023-08-29 | 贝壳找房(北京)科技有限公司 | Face image quality assessment method and device, electronic equipment and storage medium |
| CN113673466A (en) * | 2021-08-27 | 2021-11-19 | 深圳市爱深盈通信息技术有限公司 | Method for extracting photo stickers based on face key points, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Matern et al. | Exploiting visual artifacts to expose deepfakes and face manipulations | |
| CN109961049B (en) | Cigarette brand identification method under complex scene | |
| CN112528939A (en) | Quality evaluation method and device for face image | |
| Yao et al. | Oscar: On-site composition and aesthetics feedback through exemplars for photographers | |
| US8548257B2 (en) | Distinguishing between faces and non-faces | |
| US7715596B2 (en) | Method for controlling photographs of people | |
| CN104834898B (en) | A kind of quality classification method of personage's photographs | |
| US20230289979A1 (en) | A method for video moving object detection based on relative statistical characteristics of image pixels | |
| Ghimire et al. | A robust face detection method based on skin color and edges | |
| US7953253B2 (en) | Face detection on mobile devices | |
| CN104123543B (en) | A kind of eye movement recognition methods based on recognition of face | |
| CN107909081B (en) | Method for quickly acquiring and quickly calibrating image data set in deep learning | |
| KR102284096B1 (en) | System and method for estimating subject image quality using visual saliency and a recording medium having computer readable program for executing the method | |
| CN108647625A (en) | A kind of expression recognition method and device | |
| CN113592911B (en) | Apparent enhanced depth target tracking method | |
| CN110717896A (en) | Plate strip steel surface defect detection method based on saliency label information propagation model | |
| CN104794693A (en) | Human image optimization method capable of automatically detecting mask in human face key areas | |
| Xie et al. | A novel transient wrinkle detection algorithm and its application for expression synthesis | |
| CN111709305B (en) | Face age identification method based on local image block | |
| CN109725721A (en) | Human-eye positioning method and system for naked eye 3D display system | |
| CN113436735A (en) | Body weight index prediction method, device and storage medium based on face structure measurement | |
| CN112991159B (en) | Face illumination quality evaluation method, system, server and computer readable medium | |
| CN110188693B (en) | Improved complex environment vehicle feature extraction and parking discrimination method | |
| CN116386118A (en) | Drama matching cosmetic system and method based on human image recognition | |
| CN113807229A (en) | Non-contact attendance checking device, method, equipment and storage medium for intelligent classroom |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 510000 Room 601, building a, No. 23, Nanxiang Second Road, Huangpu District, Guangzhou, Guangdong Applicant after: GUANGZHOU HAIGE XINGHANG INFORMATION TECHNOLOGY CO.,LTD. Address before: 510000 room 3009, No.5, Wangjiang 2nd Street, Nansha District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU HAIGE XINGHANG INFORMATION TECHNOLOGY CO.,LTD. |