CN111667403A - Method and device for generating face image with shielding - Google Patents
Method and device for generating face image with shielding Download PDFInfo
- Publication number
- CN111667403A CN111667403A CN202010633219.0A CN202010633219A CN111667403A CN 111667403 A CN111667403 A CN 111667403A CN 202010633219 A CN202010633219 A CN 202010633219A CN 111667403 A CN111667403 A CN 111667403A
- Authority
- CN
- China
- Prior art keywords
- face
- coordinate value
- face image
- average
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 239000011159 matrix material Substances 0.000 claims abstract description 75
- 230000009466 transformation Effects 0.000 claims abstract description 61
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 230000001815 facial effect Effects 0.000 claims description 7
- 230000000875 corresponding effect Effects 0.000 description 23
- 238000001514 detection method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 6
- 239000000126 substance Substances 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a device for generating shielded face image data, wherein the method comprises the following steps: determining a target average face according to coordinate values of all key points in the face images with a first preset number; calculating the standard deviation between the first preset number of face images and the target average face; selecting at least one face image to be shielded based on the standard deviation; determining a first transformation matrix based on a face image to be shielded and a shielding object; covering the shelter to the target area in the face image to be sheltered according to the first transformation matrix to obtain the face image with the shelter. In the generation method, the key point coordinate values are only required to be calculated when the target average face is determined, and for the generation of a large number of face images with shielding, the shielding object can be covered in the face image to be shielded based on the standard deviation, the target average face and the first transformation matrix, so that the key point coordinates do not need to be calculated, and the calculation amount is reduced.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a device for generating a face image with shielding.
Background
In the process of training a face recognition model, in order to increase the accuracy of the face recognition model, not only a face image without shielding but also a face image with shielding needs to be present, and for the generation of the face image with shielding, it is a common practice at present to obtain key point coordinates of a face, perform translation rotation and scaling with a proper proportion on a shielding object (such as a mask image, an eyeglass image and the like) based on the key point coordinates, cover the shielding object on the face image without shielding, and simulate the face image with shielding.
In the prior art, the coordinates of the key points of the human face are obtained by detecting based on a detection model of the key points of the human face, the coordinates of the key points of the human face are detected once every time a human face image with shielding is generated, and when a large number of human face images with shielding need to be simulated, a large number of coordinates of the key points of the human face need to be detected, so that the calculation amount is large.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for generating a face image with an occlusion, so as to solve the problem that since coordinates of key points of a face are obtained by detection based on a face key point detection model, the coordinates of key points of the face need to be detected once every time a face image with an occlusion is generated, and when a large number of face key point images with an occlusion need to be simulated, a large number of coordinates of key points of the face need to be detected, which results in a large amount of calculation. The specific scheme is as follows:
a method for generating a face image with an occlusion comprises the following steps:
determining a target average face according to coordinate values of all key points in the face images with a first preset number;
calculating the standard deviation of the coordinate values of the corresponding key points in the first preset number of face images and the target average face;
selecting at least one face image to be shielded based on the standard deviation and the target average face;
determining a first transformation matrix based on the face image to be shielded and the shielding object;
and covering the shelter on a target area in the face image to be sheltered according to the first transformation matrix to obtain the sheltered face image.
Optionally, the method for determining the target average face according to the coordinate values of the key points in the first preset number of face images includes:
acquiring a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, wherein the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
determining a second transformation matrix based on the first coordinate value and the second coordinate value;
and determining the target average face according to the second transformation matrix.
Optionally, in the above method, selecting a facial image to be occluded based on the standard deviation and the target average face includes:
acquiring average coordinate values of key points of each target average human face in the target average face;
adding a disturbance interval for each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval;
and selecting a target coordinate value in each average coordinate disturbance interval, and constructing the face image to be shielded based on the target coordinate value.
The above method, optionally, determining a first transformation matrix based on the face image to be occluded and the occlusion object, includes:
acquiring a second preset number of third coordinate values in the shelter;
identifying the type of the shielding object and acquiring a fourth coordinate value of a second preset number in the face image to be shielded according to the type of the shielding object;
and determining the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
Optionally, in the method, the step of covering the object in the face image to be covered with the covering according to the first transformation matrix to obtain a face image with a covering includes:
determining a target area in the face image to be shielded corresponding to the shielding object;
obtaining coordinate values and pixel values of all pixel points in the shielding object;
and determining the projection coordinate value of each coordinate value in the target region based on the first transformation matrix, and adding the corresponding pixel value to the corresponding projection coordinate value.
An apparatus for generating an occluded face image, comprising:
the average face determining module is used for determining a target average face according to the coordinate values of all key points in the face images with the first preset number;
the calculation module is used for calculating the standard deviation of the coordinate values of the corresponding key points in the second preset number of face images and the target average face;
the selecting module is used for selecting at least one facial image to be occluded based on the standard deviation and the target average face;
the matrix determination module is used for determining a first transformation matrix based on the face image to be shielded and the shielding object;
and the covering module is used for covering the shelter to the target area in the face image to be sheltered according to the first transformation matrix to obtain the sheltered face image.
The above apparatus, optionally, the average face determining module includes:
a first coordinate value obtaining unit, configured to obtain a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, where the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
a first matrix determination unit configured to determine a second transformation matrix based on the first coordinate value and the second coordinate value;
and the average face determining unit is used for determining the target average face according to the second transformation matrix.
The above apparatus, optionally, the selecting module includes:
a second coordinate value obtaining unit, configured to obtain an average coordinate value of each target average face key point in the target average face;
the adding unit is used for adding disturbance intervals to each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval;
and the selecting unit is used for selecting a target coordinate value in each average coordinate disturbance interval and constructing the face image to be shielded based on the target coordinate value.
The above apparatus, optionally, the matrix determining module includes:
the third coordinate value acquisition unit is used for acquiring a second preset number of third coordinate values in the shelter;
the identification and acquisition unit is used for identifying the type of the shielding object and acquiring a fourth coordinate value of a second preset number in the face image to be shielded according to the type of the shielding object;
a second matrix determining unit, configured to determine the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
The above apparatus, optionally, the covering module includes:
the area determining unit is used for determining a target area in the face image to be shielded corresponding to the shielding object;
the acquisition unit is used for acquiring coordinate values and pixel values of all pixel points in the shelter;
and the determining and adding unit is used for determining the projection coordinate value of each coordinate value in the target area based on the first transformation matrix and adding the corresponding pixel value to the corresponding projection coordinate value.
Compared with the prior art, the invention has the following advantages:
the invention discloses a method and a device for generating shielded face image data, wherein the method comprises the following steps: determining a target average face according to coordinate values of all key points in the face images with a first preset number; calculating the standard deviation between the first preset number of face images and the target average face; selecting at least one face image to be shielded based on the standard deviation; determining a first transformation matrix based on the face image to be shielded and the shielding object; and covering the shelter on a target area in the face image to be sheltered according to the first transformation matrix to obtain the face image with the shelter. In the generation method, the key point coordinate values are only required to be calculated when the target average face is determined, and the occlusion object can be obtained by covering the occlusion object in the face image to be occluded based on the standard deviation, the target average face and the first transformation matrix aiming at a large number of face images with occlusion, so that the key point coordinates are not required to be calculated, and the calculation amount is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of a method for generating occluded face image data according to an embodiment of the present application;
FIG. 2 is a flowchart of another method for generating occluded face image data according to the embodiment of the present application;
FIG. 3 is another flowchart of a method for generating occluded face image data according to an embodiment of the present application;
fig. 4 is a block diagram of a device for generating facial image data with occlusion according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The invention discloses a method and a device for generating shielded face image data, which are applied to the generation of shielded face images, wherein the shielded face images are mainly used for training a face recognition model in a face recognition system, a common face recognition system usually comprises a plurality of neural network models, such as a face detection model for detecting the position of a face in an image, a face key point detection model for detecting facial features and chin contour coordinates, a face quality model for judging the quality of the face in the image, a living body model for judging whether the face appearing in the image is from a real person or a photo, a living body model and the like, and finally, a face recognition model for judging whether two images are of the same person is entered. The general flow is as follows: firstly, detecting and key points are made on a face picture, then filtering is carried out through a quality model (or living bodies), and finally, judgment is made on a face recognition model. To ensure the final result is reliable, high-quality face pictures with high resolution (e.g. the size of the face image is larger than 64 x 64), correct angle (e.g. the deflection angle of the face is smaller than 30 °), and small occlusion (e.g. the occlusion range does not exceed 20% of the face) need to be screened. The detection and quality model usually eliminates part of low-quality face pictures, and a shelter (such as a mask, glasses and the like) belongs to large-area face shelter. Under the background, the identification requirement of a sudden face image with a shielding object is met, a face identification model is trained based on a large number of face identification images with shielding objects, but aiming at the generation of the face image with shielding, the current common method is to obtain the coordinates of key points of the face, perform translation rotation and scaling with a proper proportion on the shielding object based on the coordinates of the key points, cover the shielding object on the face image without shielding and simulate the face image with shielding. The face key point coordinates are obtained by detecting based on a face key point detection model, the face key point coordinates need to be detected once every time a shielded face image is generated, when a large number of shielded face key point images need to be simulated, a large number of face key point coordinates need to be detected, and the calculation amount is large.
In order to solve the problem of large calculation amount of generating an occluded face image, the embodiment of the present invention provides a method for generating an occluded face image, where an execution flow of the generating method is shown in fig. 1, and the method includes the steps of:
s101, determining a target average face according to coordinate values of all key points in the face images with a first preset number;
in the embodiment of the present invention, the target average face is a front face image, and is obtained by mapping and averaging the coordinate values of the key points in the first preset number of face key faces, and the key points in the target average face are correlated with the key points in the first preset number of face images and have the same number. The coordinate values of the key points are obtained by a key point detector, the coordinate values of the key points can be the contours of key parts such as the face, the eyebrows, the eyes, the nose, the mouth and the like, and 68 points, 72 points, 108 points, 118 points and other key points can be selected optionally. In the embodiment of the present invention, taking 72 as an example for description, the number of the target average face key points in the target average face is also 72.
S102, calculating a standard deviation of coordinate values of corresponding key points in the first preset number of face images and the target average face;
in the embodiment of the invention, the coordinate values of the corresponding key points in the preset face image are obtained aiming at the average coordinate value of each target average face key point in the target average face, the standard deviation of the coordinate values of the corresponding key points relative to the preset face image is calculated, the calculation process is shown as formula (1),
wherein, s-standard deviation;
x1，x2…xn-corresponding keypoint coordinate values;
x-average coordinate value of current target average human face key point in the target average face;
further, according to the central limit theorem, when the data is large enough, most of the data can be considered to fall within 3 standard deviations from the mean value, namely 3 σ law, and therefore, random position disturbance with 3 times of variance is added near the mean area, and the true data can be approximately considered to be simulated.
S103, selecting at least one to-be-occluded face image based on the standard deviation and the target average face;
in the embodiment of the invention, the average coordinate value of each target average human face key point in the target average face is obtained, a disturbance interval is added to each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval, the target coordinate value is selected from each average coordinate disturbance interval, wherein the principle of selection is that the disturbance interval of the corresponding average value is not exceeded, for example, the disturbance interval of the average coordinate is 4.5-5.5, the corresponding target standard value can be any value in the interval, in the embodiment of the present invention, 72 target key points are taken as an example, because each average disturbance interval can have an infinite number of corresponding target coordinate values, each target coordinate value is arranged and combined to obtain an infinite number of combination forms of target key points, and at least one face image to be shielded can be constructed based on the combination forms.
S104, determining a first transformation matrix based on the face image to be shielded and the shielding object;
in the embodiment of the invention, the type of the shielding object is determined, the key point of the shielding object corresponding to the type of the shielding object and the key point to be shielded corresponding to the key point of the shielding object are obtained according to the type of the shielding object, and the first transformation matrix is determined according to the key point of the shielding object and the key point to be shielded.
S105, covering the shelter to a target area in the face image to be sheltered according to the first transformation matrix to obtain the sheltered face image.
In the embodiment of the present invention, based on the type of the obstruction, a target area in the face image to be obstructed corresponding to the obstruction is determined, for example: if the shelter is a mask, the target area is an oral-nasal area, the shelter is glasses, the target area is an eye area, each point in the shelter is covered in a corresponding point of the target area based on affine transformation, and when the pixel value of each point in the shelter is the same as the pixel value of the corresponding point in the face image to be sheltered, the affine transformation is completed, and the face image to be sheltered is obtained.
Furthermore, after the face image to be shielded is obtained, a face recognition model is trained based on the face image to be shielded, so that the diversity of face recognition model training data and the robustness of the face recognition model are improved.
Covering the shelter to a target area in the face image to be sheltered according to the first transformation matrix based on affine transformation
The invention discloses a method for generating shielded face image data, which comprises the following steps: determining a target average face according to coordinate values of all key points in the face images of a first preset number, wherein the target average face is a front face image; calculating the standard deviation between the first preset number of face images and the target average face; selecting at least one face image to be shielded based on the standard deviation; determining a first transformation matrix based on a face image to be shielded and a shielding object; covering the shelter to the target area in the face image to be sheltered according to the first transformation matrix to obtain the face image with the shelter. In the generation method, the key point coordinate values are only required to be calculated when the target average face is determined, and for the generation of a large number of face images with shielding, the shielding object can be covered in the face image to be shielded based on the standard deviation, the target average face and the first transformation matrix, so that the key point coordinates do not need to be calculated, and the calculation amount is reduced.
Based on the generation method, the face image can be acquired in real time, the shielded face image can be generated on line, the face image can also be acquired in advance, the shielded face image can be generated off line, and an off-line or on-line generation mode can be selected according to specific requirements.
In the embodiment of the present invention, an execution flow for determining a target average face according to coordinate values of each key point in a first preset number of face images is shown in fig. 2, and includes the steps of:
s201, acquiring a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, wherein the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
in the embodiment of the invention, the initial average face is a face image, an average value of coordinate values of all key points is obtained by carrying out statistical analysis based on big data, the average value is used as a second coordinate value of all average face key points of the initial average face and is used as a reference face image, wherein the number of the average face key points in the initial average face is the same as the number of all face key points in each face image, and a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in the initial average face are obtained.
S202, determining a second transformation matrix based on the first coordinate value and the second coordinate value;
in the embodiment of the present invention, assuming that the first coordinate value is a ═ x, y and the second coordinate value is B ═ u, v, and a is transformed into a point B satisfying an affine relationship through affine transformation by matrix operation, in this scheme, the existing coordinate point is the first coordinate value of each face key point of the face image, the desired coordinate point is the second coordinate value of each average face key point of the target average face, and the second matrix is H, then the formula (2) is followed
B＝H*A (2)
Fitting an H matrix through all the face key points and the average face key point
And S203, determining the target average face according to the second transformation matrix.
In the embodiment of the present invention, the first preset number of face images are mapped to the initial average face based on affine transformation according to the second transformation matrix, and an average value of the coordinate values of the first preset number of key points obtained by mapping is calculated based on each average face key point in the initial average face, so as to obtain the target average face.
In the embodiment of the present invention, an execution flow for determining a first transformation matrix based on the face image to be occluded and the occlusion object is shown in fig. 3, and includes the steps of:
s301, obtaining a second preset number of third coordinate values in the shelter;
in the embodiment of the present invention, for different types of shielding objects, a second preset number of third coordinate values in the shielding object may be selected in advance, where the selection process may be selected according to specific situations or experiences.
S302, identifying the type of the shielding object and acquiring a fourth coordinate value of a second preset number in the face image to be shielded according to the type of the shielding object;
in the embodiment of the invention, an incidence relation is established in advance based on the type of the shielding object and the coordinate values corresponding to the second preset number in the face image with the shielding object, the type of the shielding object is identified, and the fourth coordinate values of the second preset number in the face image to be shielded are obtained according to the type of the shielding object.
S303, determining the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
In the embodiment of the present invention, the first transformation matrix is determined based on the second preset number of third coordinate values and the second preset number of fourth coordinate values, and a specific determination process is the same as that of the second matrix, which is not described herein again.
Based on the above method for generating an occluded face image, in an embodiment of the present invention, a device for generating an occluded face image is further provided, and a structural block diagram of the device is shown in fig. 4, where the device includes:
an average face determination module 401, a calculation module 402, a selection module 403, a matrix determination module 404, and a coverage module 405.
Wherein the content of the first and second substances,
the average face determining module 401 is configured to determine a target average face according to coordinate values of key points in a first preset number of face images;
the calculating module 402 is configured to calculate a standard deviation between the first preset number of face images and the coordinate values of the corresponding key points in the target average face;
the selecting module 403 is configured to select at least one to-be-occluded face image based on the standard deviation and the target average face;
the matrix determining module 404 is configured to determine a first transformation matrix based on the face image to be occluded and the occlusion object;
the covering module 405 is configured to cover the blocking object to the target area in the face image to be blocked according to the first transformation matrix, so as to obtain a blocked face image.
The invention discloses a method and a device for generating shielded face image data, wherein the method comprises the following steps: determining a target average face according to the coordinate values of all key points in the face images with the first preset number; calculating the standard deviation between the first preset number of face images and the target average face; selecting at least one face image to be shielded based on the standard deviation; determining a first transformation matrix based on a face image to be shielded and a shielding object; covering the shelter to the target area in the face image to be sheltered according to the first transformation matrix to obtain the face image with the shelter. In the generation method, the key point coordinate values are only required to be calculated when the target average face is determined, and for the generation of a large number of face images with shielding, the shielding object can be covered in the face image to be shielded based on the standard deviation, the target average face and the first transformation matrix, so that the key point coordinates do not need to be calculated, and the calculation amount is reduced.
In this embodiment of the present invention, the average face determining module 401 includes:
a first coordinate value acquisition unit 406, a first matrix determination unit 407, and an average face determination unit 408.
Wherein the content of the first and second substances,
the first coordinate value obtaining unit 406 is configured to obtain a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, where the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
the first matrix determination unit 407 is configured to determine a second transformation matrix based on the first coordinate value and the second coordinate value;
the average face determining unit 408 is configured to determine the target average face according to the second transformation matrix.
In this embodiment of the present invention, the selecting module 403 includes:
a second coordinate value acquisition unit 409, an adding unit 410, and a selecting unit 411.
Wherein the content of the first and second substances,
the second coordinate value obtaining unit 409 is configured to obtain an average coordinate value of each target average face key point in the target average face;
the adding unit 410 is configured to add a disturbance interval to each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval;
the selecting unit 411 is configured to select a target coordinate value in each average coordinate disturbance interval, and construct the facial image to be occluded based on the target coordinate value.
In this embodiment of the present invention, the matrix determining module 404 includes:
a third coordinate value acquisition unit 412, a recognition and acquisition unit 413, and a second matrix determination unit 414.
Wherein the content of the first and second substances,
the third coordinate value obtaining unit 412 is configured to obtain a second preset number of third coordinate values in the obstruction;
the identifying and acquiring unit 413 is configured to identify a type of the blocking object and acquire a fourth coordinate value of a second preset number in the face image to be blocked according to the type of the blocking object;
the second matrix determining unit 414 is configured to determine the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
In this embodiment of the present invention, the covering module 405 includes:
an area determination unit 415, an acquisition unit 416, and a determination and addition unit 417.
Wherein the content of the first and second substances,
the region determining unit 415 is configured to determine a target region in the face image to be occluded, which corresponds to the occlusion;
the obtaining unit 416 is configured to obtain coordinate values and pixel values of each pixel point in the obstruction;
the determining and adding unit 417 is configured to determine a projection coordinate value of each coordinate value in the target region based on the first transformation matrix, and add a corresponding pixel value to the corresponding projection coordinate value.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functions of the units may be implemented in the same software and/or hardware or in a plurality of software and/or hardware when implementing the invention.
From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
The method and the device for generating the facial image data with the occlusion provided by the invention are described in detail above, a specific example is applied in the text to explain the principle and the implementation of the invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method for generating a face image with an occlusion is characterized by comprising the following steps:
determining a target average face according to coordinate values of all key points in the face images with a first preset number;
calculating the standard deviation of the coordinate values of the corresponding key points in the first preset number of face images and the target average face;
selecting at least one face image to be shielded based on the standard deviation and the target average face;
determining a first transformation matrix based on the face image to be shielded and the shielding object;
and covering the shelter on a target area in the face image to be sheltered according to the first transformation matrix to obtain the sheltered face image.
2. The method according to claim 1, wherein determining the target average face according to the coordinate values of the key points in the first preset number of face images comprises:
acquiring a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, wherein the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
determining a second transformation matrix based on the first coordinate value and the second coordinate value;
and determining the target average face according to the second transformation matrix.
3. The method according to claim 1, wherein selecting a face image to be occluded based on the standard deviation and the target average face comprises:
acquiring average coordinate values of key points of each target average human face in the target average face;
adding a disturbance interval for each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval;
and selecting a target coordinate value in each average coordinate disturbance interval, and constructing the face image to be shielded based on the target coordinate value.
4. The method according to claim 1, wherein determining a first transformation matrix based on the face image to be occluded and an occlusion comprises:
acquiring a second preset number of third coordinate values in the shelter;
identifying the type of the shielding object and acquiring a fourth coordinate value of a second preset number in the face image to be shielded according to the type of the shielding object;
and determining the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
5. The method according to claim 1, wherein covering the obstruction to the target area in the face image to be obstructed according to the first transformation matrix to obtain an obstructed face image comprises:
determining a target area in the face image to be shielded corresponding to the shielding object;
obtaining coordinate values and pixel values of all pixel points in the shielding object;
and determining the projection coordinate value of each coordinate value in the target region based on the first transformation matrix, and adding the corresponding pixel value to the corresponding projection coordinate value.
6. An apparatus for generating an occluded face image, comprising:
the average face determining module is used for determining a target average face according to the coordinate values of all key points in the face images with the first preset number;
the calculation module is used for calculating the standard deviation of the coordinate values of the corresponding key points in the first preset number of face images and the target average face;
the selecting module is used for selecting at least one facial image to be occluded based on the standard deviation and the target average face;
the matrix determination module is used for determining a first transformation matrix based on the face image to be shielded and the shielding object;
and the covering module is used for covering the shelter to the target area in the face image to be sheltered according to the first transformation matrix to obtain the sheltered face image.
7. The apparatus of claim 6, wherein the average face determination module comprises:
a first coordinate value obtaining unit, configured to obtain a first coordinate value of each face key point in each face image and a second coordinate value of each average face key point in an initial average face, where the first coordinate value and the second coordinate value are the same in quantity, and the initial average face is a reference face image;
a first matrix determination unit configured to determine a second transformation matrix based on the first coordinate value and the second coordinate value;
and the average face determining unit is used for determining the target average face according to the second transformation matrix.
8. The apparatus of claim 6, wherein the selecting module comprises:
a second coordinate value obtaining unit, configured to obtain an average coordinate value of each target average face key point in the target average face;
the adding unit is used for adding disturbance intervals to each average coordinate value based on the standard deviation to obtain each average coordinate disturbance interval;
and the selecting unit is used for selecting a target coordinate value in each average coordinate disturbance interval and constructing the face image to be shielded based on the target coordinate value.
9. The apparatus of claim 6, wherein the matrix determination module comprises:
the third coordinate value acquisition unit is used for acquiring a second preset number of third coordinate values in the shelter;
the identification and acquisition unit is used for identifying the type of the shielding object and acquiring a fourth coordinate value of a second preset number in the face image to be shielded according to the type of the shielding object;
a second matrix determining unit, configured to determine the first transformation matrix based on the second preset number of third coordinate values and the second preset number of fourth coordinate values.
10. The apparatus of claim 6, wherein the overlay module comprises:
the area determining unit is used for determining a target area in the face image to be shielded corresponding to the shielding object;
the acquisition unit is used for acquiring coordinate values and pixel values of all pixel points in the shelter;
and the determining and adding unit is used for determining the projection coordinate value of each coordinate value in the target area based on the first transformation matrix and adding the corresponding pixel value to the corresponding projection coordinate value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010633219.0A CN111667403B (en) | 2020-07-02 | 2020-07-02 | Method and device for generating human face image with shielding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010633219.0A CN111667403B (en) | 2020-07-02 | 2020-07-02 | Method and device for generating human face image with shielding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111667403A true CN111667403A (en) | 2020-09-15 |
| CN111667403B CN111667403B (en) | 2023-04-18 |
Family
ID=72390924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010633219.0A Active CN111667403B (en) | 2020-07-02 | 2020-07-02 | Method and device for generating human face image with shielding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111667403B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487923A (en) * | 2020-11-25 | 2021-03-12 | 奥比中光科技集团股份有限公司 | Method and system for acquiring training data of human face head posture |
| CN113327284A (en) * | 2021-05-27 | 2021-08-31 | 北京百度网讯科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN113421317A (en) * | 2021-06-10 | 2021-09-21 | 浙江大华技术股份有限公司 | Method and system for generating image and electronic equipment |
| CN115018696A (en) * | 2022-06-08 | 2022-09-06 | 东北师范大学 | Face mask data generation method based on OpenCV (open source/consumer computer vision library) affine transformation |
| RU2790018C1 (en) * | 2022-06-29 | 2023-02-14 | Федеральное государственное бюджетное учреждение науки "Санкт-Петербургский Федеральный исследовательский центр Российской академии наук" | Method for generating color protective mask on images of people's faces |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE59812694D1 (en) * | 1997-08-27 | 2005-05-04 | Cognitec Systems Gmbh | METHOD FOR LOCALIZING OBJECTS IN STILL IMAGES WITH APPLICATION TO FOCUS THE LOCALIZATION OF FACES |
| CN105224935A (en) * | 2015-10-28 | 2016-01-06 | 南京信息工程大学 | A kind of real-time face key point localization method based on Android platform |
| CN105760836A (en) * | 2016-02-17 | 2016-07-13 | 厦门美图之家科技有限公司 | Multi-angle face alignment method based on deep learning and system thereof and photographing terminal |
| US20160267320A1 (en) * | 2015-03-12 | 2016-09-15 | Omron Corporation | Image processing apparatus and image processing method |
| CN107463865A (en) * | 2016-06-02 | 2017-12-12 | 北京陌上花科技有限公司 | Face datection model training method, method for detecting human face and device |
| CN107679497A (en) * | 2017-10-11 | 2018-02-09 | 齐鲁工业大学 | Video face textures effect processing method and generation system |
| CN109508678A (en) * | 2018-11-16 | 2019-03-22 | 广州市百果园信息技术有限公司 | Training method, the detection method and device of face key point of Face datection model |
| CN109522841A (en) * | 2018-11-16 | 2019-03-26 | 重庆邮电大学 | A kind of face identification method restored based on group's rarefaction representation and low-rank matrix |
| CN109558837A (en) * | 2018-11-28 | 2019-04-02 | 北京达佳互联信息技术有限公司 | Face critical point detection method, apparatus and storage medium |
| CN109635659A (en) * | 2018-11-12 | 2019-04-16 | 东软集团股份有限公司 | Face key independent positioning method, device, storage medium and electronic equipment |
| CN109815813A (en) * | 2018-12-21 | 2019-05-28 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
| CN110020600A (en) * | 2019-03-05 | 2019-07-16 | 厦门美图之家科技有限公司 | Generate the method for training the data set of face alignment model |
| CN110175558A (en) * | 2019-05-24 | 2019-08-27 | 北京达佳互联信息技术有限公司 | A kind of detection method of face key point, calculates equipment and storage medium at device |
| CN110309815A (en) * | 2019-07-11 | 2019-10-08 | 广州华多网络科技有限公司 | A kind of processing method and system of facial recognition data |
| CN110909690A (en) * | 2019-11-26 | 2020-03-24 | 电子科技大学 | Method for detecting occluded face image based on region generation |
| CN111291678A (en) * | 2020-02-06 | 2020-06-16 | 北京爱笔科技有限公司 | Face image clustering method and device based on multi-feature fusion |
-
2020
- 2020-07-02 CN CN202010633219.0A patent/CN111667403B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE59812694D1 (en) * | 1997-08-27 | 2005-05-04 | Cognitec Systems Gmbh | METHOD FOR LOCALIZING OBJECTS IN STILL IMAGES WITH APPLICATION TO FOCUS THE LOCALIZATION OF FACES |
| US20160267320A1 (en) * | 2015-03-12 | 2016-09-15 | Omron Corporation | Image processing apparatus and image processing method |
| CN105224935A (en) * | 2015-10-28 | 2016-01-06 | 南京信息工程大学 | A kind of real-time face key point localization method based on Android platform |
| CN105760836A (en) * | 2016-02-17 | 2016-07-13 | 厦门美图之家科技有限公司 | Multi-angle face alignment method based on deep learning and system thereof and photographing terminal |
| CN107463865A (en) * | 2016-06-02 | 2017-12-12 | 北京陌上花科技有限公司 | Face datection model training method, method for detecting human face and device |
| CN107679497A (en) * | 2017-10-11 | 2018-02-09 | 齐鲁工业大学 | Video face textures effect processing method and generation system |
| CN109635659A (en) * | 2018-11-12 | 2019-04-16 | 东软集团股份有限公司 | Face key independent positioning method, device, storage medium and electronic equipment |
| CN109508678A (en) * | 2018-11-16 | 2019-03-22 | 广州市百果园信息技术有限公司 | Training method, the detection method and device of face key point of Face datection model |
| CN109522841A (en) * | 2018-11-16 | 2019-03-26 | 重庆邮电大学 | A kind of face identification method restored based on group's rarefaction representation and low-rank matrix |
| CN109558837A (en) * | 2018-11-28 | 2019-04-02 | 北京达佳互联信息技术有限公司 | Face critical point detection method, apparatus and storage medium |
| CN109815813A (en) * | 2018-12-21 | 2019-05-28 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
| CN110020600A (en) * | 2019-03-05 | 2019-07-16 | 厦门美图之家科技有限公司 | Generate the method for training the data set of face alignment model |
| CN110175558A (en) * | 2019-05-24 | 2019-08-27 | 北京达佳互联信息技术有限公司 | A kind of detection method of face key point, calculates equipment and storage medium at device |
| CN110309815A (en) * | 2019-07-11 | 2019-10-08 | 广州华多网络科技有限公司 | A kind of processing method and system of facial recognition data |
| CN110909690A (en) * | 2019-11-26 | 2020-03-24 | 电子科技大学 | Method for detecting occluded face image based on region generation |
| CN111291678A (en) * | 2020-02-06 | 2020-06-16 | 北京爱笔科技有限公司 | Face image clustering method and device based on multi-feature fusion |
Non-Patent Citations (4)
| Title |
|---|
| M. FEDIAS等: "A new approach based in mean and standard deviation for authentication system of face", 《INTERNATIONAL REVIEW ON COMPUTERS AND SOFTWARE》 * |
| 方三勇等: "基于姿态估计的正面人脸图像合成", 《计算机工程》 * |
| 李冬梅等: "基于异值区域消除的遮挡人脸识别", 《计算机科学》 * |
| 王志一等: "人脸识别中发型遮挡检测方法研究", 《微型机与应用》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487923A (en) * | 2020-11-25 | 2021-03-12 | 奥比中光科技集团股份有限公司 | Method and system for acquiring training data of human face head posture |
| CN113327284A (en) * | 2021-05-27 | 2021-08-31 | 北京百度网讯科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN113327284B (en) * | 2021-05-27 | 2022-08-26 | 北京百度网讯科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN113421317A (en) * | 2021-06-10 | 2021-09-21 | 浙江大华技术股份有限公司 | Method and system for generating image and electronic equipment |
| CN115018696A (en) * | 2022-06-08 | 2022-09-06 | 东北师范大学 | Face mask data generation method based on OpenCV (open source/consumer computer vision library) affine transformation |
| CN115018696B (en) * | 2022-06-08 | 2024-05-03 | 东北师范大学 | Face mask data generation method based on OpenCV affine transformation |
| RU2790018C1 (en) * | 2022-06-29 | 2023-02-14 | Федеральное государственное бюджетное учреждение науки "Санкт-Петербургский Федеральный исследовательский центр Российской академии наук" | Method for generating color protective mask on images of people's faces |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111667403B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111667403B (en) | Method and device for generating human face image with shielding | |
| Brauer-Burchardt et al. | A new algorithm to correct fish-eye-and strong wide-angle-lens-distortion from single images | |
| CN109376631B (en) | Loop detection method and device based on neural network | |
| EP2339507B1 (en) | Head detection and localisation method | |
| CN108875533B (en) | Face recognition method, device, system and computer storage medium | |
| JP2021531601A (en) | Neural network training, line-of-sight detection methods and devices, and electronic devices | |
| CN105989326A (en) | Method and device for determining three-dimensional position information of human eyes | |
| CN113240678B (en) | Plane information detection method and system | |
| CN112802208B (en) | Three-dimensional visualization method and device in terminal building | |
| CN111008935A (en) | Face image enhancement method, device, system and storage medium | |
| CN107368817B (en) | Face recognition method and device | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN113111827A (en) | Construction monitoring method and device, electronic equipment and storage medium | |
| CN112991159B (en) | Face illumination quality evaluation method, system, server and computer readable medium | |
| CN111767876A (en) | Method and device for generating face image with shielding | |
| US11080920B2 (en) | Method of displaying an object | |
| JP6773825B2 (en) | Learning device, learning method, learning program, and object recognition device | |
| CN117237409A (en) | Shooting game sight correction method and system based on Internet of things | |
| CN115661370B (en) | Infrared 3D human body model construction method and storage medium | |
| US11741621B2 (en) | Method and system for detecting plane information | |
| CN115272417A (en) | Image data processing method, image processing apparatus, and readable storage medium | |
| CN116086396A (en) | Method, device, system and storage medium for acquiring distance | |
| CN109690555A (en) | Face detector based on curvature | |
| JP7152651B2 (en) | Program, information processing device, and information processing method | |
| CN115527008A (en) | Safety simulation experience training system based on mixed reality technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |