CN110751078A - Method and equipment for determining non-skin color area of three-dimensional face - Google Patents
Method and equipment for determining non-skin color area of three-dimensional face Download PDFInfo
- Publication number
- CN110751078A CN110751078A CN201910978982.4A CN201910978982A CN110751078A CN 110751078 A CN110751078 A CN 110751078A CN 201910978982 A CN201910978982 A CN 201910978982A CN 110751078 A CN110751078 A CN 110751078A
- Authority
- CN
- China
- Prior art keywords
- skin color
- region
- determining
- face
- triangular surface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a method and equipment for determining a non-skin color area of a three-dimensional face, wherein the method comprises the following steps: scanning a target face in real time to generate a picture and a grid model corresponding to the target face; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image; determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for the facial features region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the user can determine the imaging quality of the three-dimensional face model in time by quickly and accurately identifying the non-skin color region in which the three-dimensional face appears, and the user experience is improved.
Description
Technical Field
The present application relates to the field of human image processing technologies, and in particular, to a method and an apparatus for determining a non-skin color region of a three-dimensional human face.
Background
Although the existing scanning imaging technology in the market can also generate a three-dimensional face model similar to a real user, if the face of the user is partially shielded (such as hair shielding), partially exposed and partially too dark, a non-skin color region, such as a local color block, appears on the face of the imaged three-dimensional face model. Because the non-skin color region cannot be accurately identified in the prior art, the user cannot determine the imaging quality of the three-dimensional face model, so that the quality of the shot and imaged three-dimensional face model is not high, and the user experience is influenced.
Disclosure of Invention
The invention provides a method for determining a non-skin color area of a three-dimensional face, which is used for solving the technical problem that a user cannot determine the imaging quality of a three-dimensional face model because the non-skin color area appearing in the three-dimensional face cannot be accurately identified in the prior art, and comprises the following steps:
scanning a target face in real time to generate a picture and a grid model corresponding to the target face;
acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image;
determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for the facial features region in the dynamic mask is the skin color region;
and determining the non-skin color area according to the triangular surface information corresponding to the area to be marked.
Preferably, the obtaining of the dynamic mask corresponding to the mesh model according to the photo and the preset face UV image specifically includes:
determining a non-skin color area of the face in the photo according to the preset skin color range;
determining a face range according to the face UV image;
determining key point coordinate information in the non-skin color region of the human face and non-key point coordinate information in the range of the human face, wherein the key point coordinate information is specifically key point coordinate information of a five sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV graph;
acquiring the dynamic mask based on the UV coordinate information.
Preferably, the determining the coordinate information of the key point in the non-skin color region of the face and the coordinate information of the non-key point in the face range specifically includes:
determining the coordinate information of the key points based on a neural network mode of machine learning;
and performing interpolation operation by adopting a prediction algorithm according to the coordinate information of the key points to determine the coordinate information of the non-key points.
Preferably, the area to be marked in the skin color area of the dynamic mask is determined based on a preset skin color range, specifically:
traversing the skin color area, and judging whether an area exceeding the preset skin color range exists in the skin color area;
if so, taking the area beyond the preset skin color range as the area to be marked.
Preferably, the determining the non-skin color region according to the triangular surface information corresponding to the region to be marked specifically includes:
generating a triangular surface queue according to the triangular surface where each vertex in the region to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining the triangular surface information according to the grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color area after performing highlight operation based on the triangular surface information.
Preferably, after generating the triangular surface queue according to the triangular surface on which each vertex in the to-be-marked area is located, the method further includes:
step a, selecting a first triangular surface from the triangular surface queue as a current search surface;
b, adding the current search surface into a search queue and a result queue;
step c, inquiring a triangular surface adjacent to the current search surface in the triangular surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b to d until the search queue is traversed.
Preferably, the triangular surface queues are classified to obtain a preset number of continuous triangular surface queues, specifically:
step A, determining a set A according to the triangular surface queue;
step B, determining a set B based on the steps a-e, and storing the set B into a queue DS;
c, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular surface queues according to the classified sets stored in the queue DS.
Correspondingly, the invention also provides equipment for determining the non-skin color area of the three-dimensional face, which comprises the following steps:
the scanning module is used for scanning a target face in real time to generate a picture and a grid model corresponding to the target face;
the acquisition module is used for acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image;
the first determination module is used for determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for a facial feature region in the dynamic mask is the skin color region;
and the second determining module is used for determining the non-skin color area according to the triangular surface information corresponding to the area to be marked.
Accordingly, the present invention also provides a computer-readable storage medium, which stores instructions that, when executed on a terminal device, cause the terminal device to execute the method for determining the non-skin color region of the three-dimensional face as described above.
Accordingly, the present invention also provides a computer program product, which when running on a terminal device, causes the terminal device to execute the method for determining the non-skin color region of the three-dimensional face as described above.
Compared with the prior art, the invention has the following beneficial effects:
the invention discloses a method and equipment for determining a non-skin color area of a three-dimensional face, wherein the method comprises the following steps: scanning a target face in real time to generate a picture and a grid model corresponding to the target face; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image; determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for the facial features region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the imaging quality of the three-dimensional face model can be determined in time by a user through rapidly and accurately identifying the non-skin color region in which the three-dimensional face appears, the three-dimensional face model with higher quality can be generated, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart illustrating a method for determining a non-skin color region of a three-dimensional face according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a photograph of a human face in an embodiment of the invention;
FIG. 3 is a schematic diagram illustrating the effect of determining a non-skin color region of a face photo according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of presetting a face UV picture in the embodiment of the invention;
FIG. 5 illustrates an original image without keypoint identification processing in an embodiment of the invention;
FIG. 6 is a diagram illustrating the effect of locating key points via FacePlusPlusPlusPlusExample in an embodiment of the present invention;
FIG. 7 is a diagram illustrating the effect of determining non-keypoints around an eye region by a difference algorithm in an embodiment of the present invention;
FIG. 8 is a diagram illustrating the effect of dynamic masking in an embodiment of the present invention;
FIG. 9 is a schematic diagram of a non-skin tone region in an embodiment of the present invention;
FIG. 10 is a schematic diagram of another non-skin tone region in an embodiment of the present invention;
FIG. 11 is a schematic diagram illustrating the effect of determining and marking a non-skin color region according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an apparatus for determining a non-skin color region of a three-dimensional human face according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As described in the background art, the non-skin color region appearing in the three-dimensional face cannot be accurately identified in the prior art, so that the user cannot determine the imaging quality of the three-dimensional face model.
In order to solve the above problems, an embodiment of the present application provides a method for determining a non-skin color region of a three-dimensional face, which determines a region to be marked in a skin color region of a dynamic mask of a three-dimensional face mesh model, and determines the non-skin color region of the three-dimensional face according to triangle surface information corresponding to the region to be marked, so as to quickly and accurately identify the non-skin color region where the three-dimensional face appears, and enable a user to determine imaging quality of the three-dimensional face model in time.
As shown in fig. 1, a schematic flow chart of a method for determining a non-skin color region of a three-dimensional face according to an embodiment of the present invention includes the following steps:
s101, scanning a target face in real time, and generating a picture and a grid model corresponding to the target face.
In order to generate a three-dimensional face model corresponding to a target face, the target face is scanned in real time to generate a corresponding photo and a grid model. The person skilled in the art can scan the target face in real time through a three-dimensional imaging device such as a three-dimensional camera and a three-dimensional imaging scanner, and different real-time scanning modes do not affect the protection scope of the present application.
And S102, acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image.
In a specific application scene of the application, a face can be shot by using an AR (Augmented Reality) development platform technology ARKit to obtain a grid model, and UV of the grid model is derived as a preset face UV picture by using three-dimensional animation rendering and manufacturing software 3DMAX, and technicians in the field can also flexibly select other modes to obtain the preset UV picture, wherein the preset UV picture is shown in figure 4. The dynamic mask can realize partial display effect, and can determine the five sense organ regions not participating in calculation, as shown in fig. 8, which is an effect diagram of the dynamic mask, wherein the dark color part is the five sense organ regions not participating in calculation.
In order to obtain a suitable dynamic mask, in a preferred embodiment of the present application, a dynamic mask corresponding to the mesh model is obtained according to the photo and a preset face UV image, specifically:
determining a non-skin color area of the face in the photo according to the preset skin color range;
determining a face range according to the face UV image;
determining key point coordinate information in the non-skin color region of the human face and non-key point coordinate information in the range of the human face, wherein the key point coordinate information is specifically key point coordinate information of a five sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV graph;
acquiring the dynamic mask based on the UV coordinate information.
Specifically, a non-skin color region of a human face in a photo is determined firstly, the non-skin color region of the human face in the photo can be determined through a preset skin color range, specifically, a skin detection based on RGB, a skin detection based on an elliptical skin model, YCrCb color space Cr component + Otsu method threshold segmentation, a non-skin color region determination based on YCrCb color space Cr, Cb range screening method, HSV color space H range screening method and opencv adaptive skin color detection type adaptive skin detector can be selected for determination, all the detection modes are based on the preset skin color range detection, and technicians in the field can flexibly select the detection modes according to actual needs without influencing the protection range of the application.
Since the picture contains regions (such as hair) outside the face contour and needs to be further filtered, the face range can be determined according to the preset UV map. The eyebrow area, the eyeball area and the lip area of the five sense organ area of the non-skin color area of the human face in the picture have larger difference with skin color, and the difference is required to be further filtered, so that the coordinate information of key points is determined, and the coordinate information of the key points is particularly the coordinate information of key points of the five sense organ area, so that the key points of the five sense organ area do not participate in calculation when the non-skin color area of the three-dimensional human face is judged. And after the key point coordinate information is determined, non-key point coordinate information in the human face range is also determined, and the dynamic mask is obtained according to the key point coordinate information and the UV coordinate information converted from the non-key point coordinate information.
It should be noted that the scheme of the above preferred embodiment is only a specific implementation scheme proposed in the present application, and other ways of obtaining the dynamic mask corresponding to the mesh model according to the photo and the preset face UV map all belong to the protection scope of the present application.
In order to obtain accurate coordinate information of key points and coordinate information of non-key points, in a preferred embodiment of the present application, the coordinate information of key points in the non-skin color region of the face and the coordinate information of non-key points in the face range are determined, specifically:
determining the coordinate information of the key points based on a neural network mode of machine learning;
and performing interpolation operation by adopting a prediction algorithm according to the coordinate information of the key points to determine the coordinate information of the non-key points.
The neural network mode of machine learning, such as the faceplus plus of the cloud-end visual service platform, can accurately locate the key points, thereby determining the coordinate information of the key points, and then, according to the coordinate information of the key points, interpolation operation is performed by adopting a prediction algorithm to determine the coordinate information of non-key points, the prediction algorithm is the prior art, and is not repeated herein, and a person skilled in the art can realize the method in different modes according to specific application scenes.
It should be noted that the scheme of the above preferred embodiment is only a specific implementation scheme proposed by the present application, and other ways of determining the coordinate information of the key point in the non-skin color region of the face and the coordinate information of the non-key point in the face range all belong to the protection scope of the present application.
S103, determining a region to be marked in the skin color region of the dynamic mask based on a preset skin color range, wherein the region except the facial features region in the dynamic mask is the skin color region.
And after the dynamic mask is obtained, determining a region to be marked in the skin color region of the dynamic mask according to a preset skin color range, wherein the region to be marked corresponds to a non-skin color region in the three-dimensional face.
In order to accurately determine the region to be marked, in a preferred embodiment of the present application, the region to be marked in the skin color region of the dynamic mask is determined based on a preset skin color range, specifically:
traversing the skin color area, and judging whether an area exceeding the preset skin color range exists in the skin color area;
if so, taking the area beyond the preset skin color range as the area to be marked.
Specifically, the region to be marked corresponds to a non-skin color region in the three-dimensional face, and whether a region beyond the preset skin color range exists in the skin color regions in the dynamic mask is traversed, if so, the region beyond the preset skin color range is used as the region to be marked. A person skilled in the art can flexibly preset a specific skin color range according to a specific application scene, different skin color ranges all belong to the protection range of the application, and the preset skin color range can be set as a skin color range corresponding to a non-skin color region, and the region in the preset skin color range is used as a region to be marked, which does not affect the protection range of the application.
And S104, determining the non-skin color area according to the triangular surface information corresponding to the area to be marked.
The area to be marked corresponds to a non-skin color area in the three-dimensional face, and the triangular surface is a plane formed by three vertexes and can be restored into lattice information, so that the non-skin color area of the three-dimensional face can be determined according to the triangular surface information corresponding to the area to be marked.
In order to accurately determine a non-skin color region in a three-dimensional face, in a preferred embodiment of the present application, the non-skin color region is determined according to triangle information corresponding to the region to be marked, specifically:
generating a triangular surface queue according to the triangular surface where each vertex in the region to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining the triangular surface information according to the grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color area after performing highlight operation based on the triangular surface information.
Specifically, the triangular surface queue is generated by the triangular surfaces where the vertexes of the areas to be marked are located, because the generated triangular surface queue cannot be completely continuous, for example, the vertexes of the three triangular surfaces a, b and c in fig. 9 are connected, so that the triangular surfaces are continuous, so that the three triangular surfaces are unified into a non-skin color area, the vertices of the triangular surfaces d and e in figure 10 are not connected with the vertices of the three triangular surfaces a, b and c in figure 9, the triangle surfaces in fig. 9 and 10 are not continuous and are two different non-skin color areas, so that the triangle surface queues need to be classified, a non-skin color area is determined according to the triangle surfaces of the same class, after a preset number of continuous triangular surface queues are obtained, triangular surface information of a non-skin color area can be determined according to the grid information corresponding to each continuous triangular surface, and the non-skin color area can be determined through highlight operation based on the triangular surface information. The highlighting operation may be to highlight the non-skin color region through the cg code, and a person skilled in the art may flexibly select another way to highlight the non-skin color region, which does not affect the protection scope of the present application.
It should be noted that the scheme of the above preferred embodiment is only a specific implementation scheme proposed by the present application, and other ways of determining the non-skin color region according to the triangle surface information corresponding to the region to be marked all belong to the protection scope of the present application.
In order to accurately obtain the triangular surface queue corresponding to the region to be marked, in a preferred embodiment of the present application, after the triangular surface queue is generated according to the triangular surface where each vertex in the region to be marked is located, the method further includes:
step a, selecting a first triangular surface from the triangular surface queue as a current search surface;
b, adding the current search surface into a search queue and a result queue;
step c, inquiring a triangular surface adjacent to the current search surface in the triangular surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b to d until the search queue is traversed.
Specifically, all triangular surfaces related to the area to be marked can be determined through the process, and an accurate triangular surface queue is generated.
It should be noted that the scheme of the above preferred embodiment is only a specific implementation scheme provided by the present application, and other ways of accurately acquiring the triangular surface queue corresponding to the area to be marked all belong to the protection scope of the present application.
In order to obtain the continuous triangular surface queues, in a preferred embodiment of the present application, the triangular surface queues are classified to obtain a preset number of continuous triangular surface queues, specifically:
step A, determining a set A according to the triangular surface queue;
step B, determining a set B based on the steps a-e, and storing the set B into a queue DS;
c, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular surface queues according to the classified sets stored in the queue DS.
Specifically, the queue DS is a queue marked as DS, and the DS is only one kind of mark or symbol, and a person skilled in the art can select other letters to mark the queue, and through the above process, the sorted sets are stored in the queue DS, so that a preset number of continuous triangular surface queues can be obtained.
It should be noted that the scheme of the above preferred embodiment is only a specific implementation scheme proposed in the present application, and other manners of classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues all belong to the protection scope of the present application.
By applying the technical scheme, the target face is scanned in real time, and a picture and a grid model corresponding to the target face are generated; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image; determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for the facial features region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the imaging quality of the three-dimensional face model can be determined in time by a user through rapidly and accurately identifying the non-skin color region in which the three-dimensional face appears, the three-dimensional face model with higher quality can be generated, and the user experience is improved.
In order to further illustrate the technical idea of the present invention, the technical solution of the present invention will now be described with reference to specific application scenarios.
The embodiment of the invention provides a method for determining a non-skin color region of a three-dimensional face, which comprises the steps of determining a region to be marked in a skin color region of a dynamic mask of a three-dimensional face grid model, and determining the non-skin color region of the three-dimensional face according to triangular face information corresponding to the region to be marked, so that a local color block of the skin of the face, which is caused by local occlusion of the face or shadow and highlight of light, is displayed to a user in time, the user can determine the imaging quality of the three-dimensional face model in time, and the user experience is improved.
The method comprises the following specific steps:
step one, scanning a target face in real time through a three-dimensional imaging device to obtain a picture of the target face and a Mesh model Mesh, wherein the three-dimensional imaging device can be a three-dimensional camera, a three-dimensional imaging scanner and the like.
Step two, determining a non-skin color region of the face in the photo based on the preset skin color range, specifically, after converting the RGB value of the photo into HSV, performing a filtering operation according to 7< H < 2028 < S < 25650 < V <256, for example, processing the face photo shown in fig. 2 through the steps to obtain the effect graph shown in fig. 3.
And step three, determining the face range according to the preset face UV image.
Because the photo contains the region (such as hair) outside the face outline and needs to be further filtered, the face range is determined, the face range can be judged according to the UV picture, because the UV pictures of all people are consistent, a default fixed UV picture is preset, the face can be shot by an AR (Augmented Reality) development platform technology ARKit to obtain a grid model, the UV of the grid model is derived as a preset face UV picture through three-dimensional animation rendering and manufacturing software 3DMAX, and technicians in the field can flexibly select other modes to obtain the preset UV picture, and the preset UV picture is shown in figure 4.
And step four, determining the coordinate information of key points in the non-skin color area of the face in the photo and the coordinate information of non-key points in the face range.
The difference between the eyebrow area, the eyeball area and the lip area of the five sense organ area of the non-skin color area of the human face in the picture and the skin color is large, and the differences are required to be further filtered, so that the coordinate information of key points is determined, so that the key points of the five sense organ area do not participate in calculation when the non-skin color area of the three-dimensional human face is judged, and the coordinate information of the key points is the coordinate information of the key points of the five sense organ area. The coordinate information of the key points can be determined through a neural network manner of machine learning, for example, the key points can be accurately located through a cloud visual service platform faceplus, for example, the key points in fig. 5 are located through faceplus, and an effect graph is shown in fig. 6.
The non-key points except the key points are determined by interpolation operation based on the key point coordinate information by using a prediction algorithm to determine the non-key point coordinate information in the face range, for example, the non-key points around the eye region are determined by using a difference algorithm, the effect graph is shown in fig. 7, the prediction algorithm is the prior art and is not described herein again, and those skilled in the art can implement the method in different ways according to specific application scenarios.
And step five, converting the coordinate information of the key points and the coordinate information of the non-key points in the step four into UV coordinate information in the UV image, and acquiring a dynamic mask corresponding to the grid model in the step one according to the UV coordinate information, wherein the dynamic mask is an effect image of the dynamic mask as shown in FIG. 8, and the dark color part is a five sense organ area which does not participate in calculation.
And step six, determining a region to be marked in the skin color region of the dynamic mask based on a preset skin color range.
Except for the five sense organ regions, other regions in the human face are skin color regions, the skin color regions are traversed, the regions which are outside a preset skin color range and appear in the skin color regions are used as regions to be marked, and the regions to be marked correspond to non-skin color regions of the three-dimensional human face.
And step seven, determining a non-skin color area of the three-dimensional face according to the triangular face information corresponding to the area to be marked.
The non-skin color area of the three-dimensional face can be marked by the result obtained in the sixth step and displayed on the three-dimensional face model, and the method specifically comprises the following steps:
1) generating a triangular surface queue according to a triangular surface where each vertex in the to-be-marked area is located, wherein the triangular surface is a plane formed by the three vertices and can be restored into grid information;
2) classifying the generated triangular surface queues to obtain a preset number of continuous triangular surface queues;
3) determining triangular face information of a non-skin color area according to the grid information corresponding to the continuous triangular face queue;
4) highlighting the non-skin color area through a cg code based on the triangle face information to identify the non-skin color area, the cg code being a high level shader language designed for GPU programming. Fig. 11 is a schematic diagram illustrating the effect of determining and marking a non-skin color region.
In order to obtain an accurate triangular surface queue corresponding to the area to be marked, the method further comprises, after the step 1):
step a, selecting a first triangular surface from the triangular surface queue as a current search surface;
b, adding the current search surface into a search queue and a result queue;
step c, inquiring a triangular surface adjacent to the current search surface in the triangular surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b to d until the search queue is traversed.
The results obtained through step 1) may not be completely continuous, and therefore, the results obtained in step one need to be classified into different triangular surface information, for example, the vertices of the three triangular surfaces a, b, and c in fig. 9 are connected, and therefore, are continuous triangular surfaces, and therefore, are unified into a non-skin color region, and the vertices of the triangular surfaces d and e in fig. 10 are not connected with the vertices of the three triangular surfaces a, b, and c in fig. 9, and therefore, the triangular surfaces in fig. 9 and 10 are not continuous, and are two different non-skin color regions.
The step 2) may specifically include:
step A, determining a set A according to the triangular surface queue;
step B, determining a set B based on the steps a-e, and storing the set B into a queue DS;
c, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular surface queues according to the classified sets stored in the queue DS.
By applying the technical scheme, after the three-dimensional face model is generated by scanning, the generation effect of the model can be immediately detected, if a user shoots the face, the user can be immediately marked and informed because of local shading or local color blocks of the skin of the face caused by light shadow and highlight, and under the condition of not influencing shooting and imaging, the user can be immediately judged and reminded, so that the user can obtain better shooting experience, and the three-dimensional face model with higher quality is generated.
In order to achieve the above technical object, an embodiment of the present application further provides an apparatus for determining a non-skin color region of a three-dimensional face, as shown in fig. 12, including:
the scanning module 201 is configured to scan a target face in real time, and generate a picture and a grid model corresponding to the target face;
an obtaining module 202, configured to obtain a dynamic mask corresponding to the mesh model according to the photo and a preset face UV map;
a first determining module 203, configured to determine, based on a preset skin color range, a region to be marked in a skin color region of the dynamic mask, where a region other than a facial region in the dynamic mask is the skin color region;
a second determining module 204, configured to determine the non-skin color region according to triangular surface information corresponding to the region to be marked.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A method of determining non-skin color regions of a three-dimensional face, the method comprising:
scanning a target face in real time to generate a picture and a grid model corresponding to the target face;
acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image;
determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for the facial features region in the dynamic mask is the skin color region;
and determining the non-skin color area according to the triangular surface information corresponding to the area to be marked.
2. The method according to claim 1, wherein the dynamic mask corresponding to the mesh model is obtained from the photo and a preset face UV map, specifically:
determining a non-skin color area of the face in the photo according to the preset skin color range;
determining a face range according to the face UV image;
determining key point coordinate information in the non-skin color region of the human face and non-key point coordinate information in the range of the human face, wherein the key point coordinate information is specifically key point coordinate information of a five sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV graph;
acquiring the dynamic mask based on the UV coordinate information.
3. The method according to claim 2, wherein determining the coordinate information of the key points in the non-skin color region of the human face and the coordinate information of the non-key points in the range of the human face specifically comprises:
determining the coordinate information of the key points based on a neural network mode of machine learning;
and performing interpolation operation by adopting a prediction algorithm according to the coordinate information of the key points to determine the coordinate information of the non-key points.
4. The method according to claim 1, wherein the region to be marked in the skin color region of the dynamic mask is determined based on a preset skin color range, specifically:
traversing the skin color area, and judging whether an area exceeding the preset skin color range exists in the skin color area;
if so, taking the area beyond the preset skin color range as the area to be marked.
5. The method according to claim 1, wherein determining the non-skin color region according to triangle information corresponding to the region to be marked specifically comprises:
generating a triangular surface queue according to the triangular surface where each vertex in the region to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining the triangular surface information according to the grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color area after performing highlight operation based on the triangular surface information.
6. The method of claim 5, after generating the triangular surface queue according to the triangular surfaces of the vertices in the to-be-marked region, further comprising:
step a, selecting a first triangular surface from the triangular surface queue as a current search surface;
b, adding the current search surface into a search queue and a result queue;
step c, inquiring a triangular surface adjacent to the current search surface in the triangular surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b to d until the search queue is traversed.
7. The method according to claim 6, wherein the triangular surface queues are classified to obtain a preset number of continuous triangular surface queues, specifically:
step A, determining a set A according to the triangular surface queue;
step B, determining a set B based on the steps a-e, and storing the set B into a queue DS;
c, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular surface queues according to the classified sets stored in the queue DS.
8. An apparatus for determining non-skin color regions of a three-dimensional face, comprising:
the scanning module is used for scanning a target face in real time to generate a picture and a grid model corresponding to the target face;
the acquisition module is used for acquiring a dynamic mask corresponding to the grid model according to the photo and a preset human face UV image;
the first determination module is used for determining a region to be marked in a skin color region of the dynamic mask based on a preset skin color range, wherein the region except for a facial feature region in the dynamic mask is the skin color region;
and the second determining module is used for determining the non-skin color area according to the triangular surface information corresponding to the area to be marked.
9. A computer-readable storage medium having stored therein instructions that, when executed on a terminal device, cause the terminal device to perform a method of determining non-skin color regions of a three-dimensional face as claimed in any one of claims 1-7.
10. A computer program product, characterized in that it, when run on a terminal device, causes the terminal device to execute the method of determining non-skin color regions of a three-dimensional face according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910978982.4A CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910978982.4A CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110751078A true CN110751078A (en) | 2020-02-04 |
| CN110751078B CN110751078B (en) | 2023-06-20 |
Family
ID=69278439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910978982.4A Active CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751078B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376594A (en) * | 2014-11-25 | 2015-02-25 | 福建天晴数码有限公司 | Three-dimensional face modeling method and device |
| WO2017026839A1 (en) * | 2015-08-12 | 2017-02-16 | 트라이큐빅스 인크. | 3d face model obtaining method and device using portable camera |
| CN106485222A (en) * | 2016-10-10 | 2017-03-08 | 上海电机学院 | A kind of method for detecting human face being layered based on the colour of skin |
| CN107506714A (en) * | 2017-08-16 | 2017-12-22 | 成都品果科技有限公司 | A kind of method of face image relighting |
| WO2018201551A1 (en) * | 2017-05-02 | 2018-11-08 | 厦门美图之家科技有限公司 | Facial image fusion method and apparatus and computing device |
-
2019
- 2019-10-15 CN CN201910978982.4A patent/CN110751078B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376594A (en) * | 2014-11-25 | 2015-02-25 | 福建天晴数码有限公司 | Three-dimensional face modeling method and device |
| WO2017026839A1 (en) * | 2015-08-12 | 2017-02-16 | 트라이큐빅스 인크. | 3d face model obtaining method and device using portable camera |
| CN106485222A (en) * | 2016-10-10 | 2017-03-08 | 上海电机学院 | A kind of method for detecting human face being layered based on the colour of skin |
| WO2018201551A1 (en) * | 2017-05-02 | 2018-11-08 | 厦门美图之家科技有限公司 | Facial image fusion method and apparatus and computing device |
| CN107506714A (en) * | 2017-08-16 | 2017-12-22 | 成都品果科技有限公司 | A kind of method of face image relighting |
Non-Patent Citations (1)
| Title |
|---|
| 阳策;李重;任义;刘恒;: "基于标准肤色的人脸图像纹理合成与三维重建应用" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110751078B (en) | 2023-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111445410B (en) | Texture enhancement method, device and equipment based on texture image and storage medium | |
| JP4452497B2 (en) | Method and system for modifying a digital image in consideration of noise | |
| CN112884637B (en) | Special effect generation method, device, equipment and storage medium | |
| US20170103510A1 (en) | Three-dimensional object model tagging | |
| CN112085673B (en) | Multi-exposure image fusion method for removing strong ghosts | |
| CN110390643B (en) | License plate enhancement method and device and electronic equipment | |
| CN112258440B (en) | Image processing method, device, electronic equipment and storage medium | |
| Sharma et al. | Estimating depth and global atmospheric light for image dehazing using type-2 fuzzy approach | |
| CN114170227B (en) | Product surface defect detection method, device, equipment and storage medium | |
| CN112106102A (en) | Image processing method, system, device, movable platform and storage medium | |
| CN111627076A (en) | Face changing method and device and electronic equipment | |
| CN114372931A (en) | Target object blurring method and device, storage medium and electronic equipment | |
| CN116612263B (en) | Method and device for sensing consistency dynamic fitting of latent vision synthesis | |
| KR100815209B1 (en) | The Apparatus and Method for Abstracting Peculiarity of Two-Dimensional Image ? The Apparatus and Method for Creating Three-Dimensional Image Using Them | |
| CN110458012B (en) | Multi-angle face recognition method and device, storage medium and terminal | |
| CN110751078B (en) | Method and equipment for determining non-skin color region of three-dimensional face | |
| CN109658360B (en) | Image processing method and device, electronic equipment and computer storage medium | |
| JP2003256834A (en) | Face area extracting and face configuring element position judging device | |
| CN114677393B (en) | Depth image processing method, depth image processing device, image pickup apparatus, conference system, and medium | |
| KR20010084996A (en) | Method for generating 3 dimension avatar using one face image and vending machine with the same | |
| CN114155569B (en) | Cosmetic progress detection method, device, equipment and storage medium | |
| CN116342519A (en) | Image processing method based on machine learning | |
| CN114596213A (en) | Image processing method and device | |
| KR100618493B1 (en) | The Apparatus and Method for Creating Three-Dimensional Image Automatically | |
| WO2023238702A1 (en) | Computer program, processing method, and processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |