CN110751078B - Method and equipment for determining non-skin color region of three-dimensional face - Google Patents
Method and equipment for determining non-skin color region of three-dimensional face Download PDFInfo
- Publication number
- CN110751078B CN110751078B CN201910978982.4A CN201910978982A CN110751078B CN 110751078 B CN110751078 B CN 110751078B CN 201910978982 A CN201910978982 A CN 201910978982A CN 110751078 B CN110751078 B CN 110751078B
- Authority
- CN
- China
- Prior art keywords
- face
- region
- skin color
- determining
- coordinate information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a method and equipment for determining a non-skin color region of a three-dimensional face, wherein the method comprises the following steps: scanning a target face in real time to generate a photo and a grid model corresponding to the target face; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image; determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the non-skin color region of the three-dimensional face is rapidly and accurately identified, the imaging quality of the three-dimensional face model is timely determined by a user, and the user experience is improved.
Description
Technical Field
The present disclosure relates to the field of portrait processing, and in particular, to a method and apparatus for determining a non-skin color region of a three-dimensional face.
Background
In the existing scanning imaging technology in the market, although a three-dimensional face model similar to a real user can be generated, if the face of the user is partially blocked (such as hair blocked), partially exposed and partially too dark, non-skin color areas, such as partial color blocks, of the face of the imaged three-dimensional face model can be caused. Because the non-skin color region cannot be accurately identified in the prior art, a user cannot determine the imaging quality of the three-dimensional face model, so that the quality of the three-dimensional face model imaged by shooting is low, and the user experience is affected.
Disclosure of Invention
The invention provides a method for determining a non-skin color region of a three-dimensional face, which is used for solving the technical problem that a user cannot determine imaging quality of a three-dimensional face model due to the fact that the non-skin color region in the three-dimensional face cannot be accurately identified in the prior art, and comprises the following steps:
scanning a target face in real time to generate a photo and a grid model corresponding to the target face;
acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image;
determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region;
and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked.
Preferably, the dynamic mask corresponding to the grid model is obtained according to the photo and a preset face UV map, specifically:
determining a face non-skin color area in the photo according to the preset skin color range;
determining a face range according to the face UV map;
determining key point coordinate information in the face non-skin color region and non-key point coordinate information in the face range, wherein the key point coordinate information is specifically key point coordinate information of a five-sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV map;
and acquiring the dynamic mask based on the UV coordinate information.
Preferably, the determining the key point coordinate information in the non-skin color region of the face and the non-key point coordinate information in the face range specifically includes:
determining the coordinate information of the key points based on a neural network mode of machine learning;
and carrying out interpolation operation by adopting a prediction algorithm according to the key point coordinate information to determine the non-key point coordinate information.
Preferably, the area to be marked in the skin area of the dynamic mask is determined based on a preset skin color range, specifically:
traversing the skin color region, and judging whether a region exceeding the preset skin color range exists in the skin color region;
if yes, taking the area exceeding the preset skin color range as the area to be marked.
Preferably, the non-skin color area is determined according to triangular surface information corresponding to the area to be marked, specifically:
generating a triangular surface queue according to the triangular surface where each vertex in the area to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining triangular surface information according to grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color region after performing highlighting operation based on the triangular surface information.
Preferably, after generating the triangular face queue according to the triangular face where each vertex in the area to be marked is located, the method further includes:
step a, selecting a first triangular face from the triangular face queue as a current search face;
b, adding the current search surface into a search queue and a result queue;
step c, querying a triangle surface adjacent to the current search surface in the triangle surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b-d until the search queue is traversed.
Preferably, the triangular face queues are classified to obtain a preset number of continuous triangular face queues, which specifically includes:
step A, determining a set A according to the triangular face queue;
step B, determining a set B based on the step a-step e, and storing the set B into a queue DS;
step C, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular face queues according to the classified set stored in the queue DS.
Correspondingly, the invention also provides a device for determining the non-skin color area of the three-dimensional face, which comprises:
the scanning module is used for scanning the target face in real time and generating a photo and a grid model corresponding to the target face;
the acquisition module is used for acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image;
the first determining module is used for determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except for a five sense organs region in the dynamic mask is the skin color region;
and the second determining module is used for determining the non-skin color region according to the triangular surface information corresponding to the region to be marked.
Accordingly, the present invention also proposes a computer readable storage medium having instructions stored therein, which when run on a terminal device, cause the terminal device to perform the method of determining a non-skin tone region of a three-dimensional face as described above.
Accordingly, the present invention also proposes a computer program product which, when run on a terminal device, causes the terminal device to perform the method of determining a non-skin tone region of a three-dimensional face as described above.
Compared with the prior art, the invention has the following beneficial effects:
the invention discloses a method and equipment for determining a non-skin color region of a three-dimensional face, wherein the method comprises the following steps: scanning a target face in real time to generate a photo and a grid model corresponding to the target face; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image; determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the non-skin color region of the three-dimensional face is rapidly and accurately identified, the imaging quality of the three-dimensional face model is timely determined by a user, a three-dimensional face model with higher quality can be generated, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for determining a non-skin color region of a three-dimensional face according to an embodiment of the present invention;
FIG. 2 is a schematic view of a face photo in an embodiment of the invention;
FIG. 3 is a schematic diagram showing the effect of non-skin color region determination on a face photo according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a preset face UV map in an embodiment of the present invention;
FIG. 5 shows an original image without key point recognition processing in an embodiment of the present invention;
FIG. 6 shows a schematic diagram of the effect of locating keypoints via facePlusPlus in an embodiment of the invention;
FIG. 7 is a schematic diagram showing the effect of determining non-key points around an eye region by a difference algorithm in an embodiment of the invention;
FIG. 8 illustrates an effect diagram of dynamic masking in an embodiment of the invention;
FIG. 9 is a schematic diagram of a non-skin tone region in accordance with an embodiment of the present invention;
FIG. 10 is a schematic diagram of another non-skin tone region in an embodiment of the present invention;
FIG. 11 is a schematic diagram showing the effect of determining and marking a non-skin color region according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an apparatus for determining a non-skin color region of a three-dimensional face according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
As described in the background art, in the prior art, non-skin color regions appearing in a three-dimensional face cannot be accurately identified, resulting in that a user cannot determine the imaging quality of a three-dimensional face model.
In order to solve the above problems, the embodiments of the present application provide a method for determining a non-skin color region of a three-dimensional face, by determining a region to be marked in a skin color region of a dynamic mask of a three-dimensional face mesh model, and determining the non-skin color region of the three-dimensional face according to triangular surface information corresponding to the region to be marked, thereby rapidly and accurately identifying the non-skin color region of the three-dimensional face, and enabling a user to determine imaging quality of the three-dimensional face model in time.
As shown in fig. 1, a flowchart of a method for determining a non-skin color region of a three-dimensional face according to an embodiment of the present invention includes the following steps:
s101, scanning a target face in real time, and generating a photo and a grid model corresponding to the target face.
In order to generate a three-dimensional face model corresponding to a target face, a corresponding photo and a grid model are generated by scanning the target face in real time. Those skilled in the art can scan the target face in real time through three-dimensional imaging devices such as three-dimensional cameras and three-dimensional imaging scanners, and different real-time scanning modes do not affect the protection scope of the application.
S102, acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image.
Because the UV graphs of all people are consistent, a default fixed UV graph is preset, in the specific application scene of the application, a grid model can be obtained by shooting a human face through an AR (Augmented Reality ) development platform technology ARkit, UV of the grid model is derived through three-dimensional animation rendering and manufacturing software 3DMAX to serve as the preset human face UV graph, and other modes can be flexibly selected by a person skilled in the art to obtain the preset UV graph, wherein the preset UV graph is shown in fig. 4. The dynamic mask can realize the effect of partial display, and can determine the five sense organ area not participating in calculation, as shown in fig. 8, which is an effect diagram of the dynamic mask, wherein the dark color part is the five sense organ area not participating in calculation.
In order to obtain a suitable dynamic mask, in a preferred embodiment of the present application, a dynamic mask corresponding to the grid model is obtained according to the photo and a preset face UV map, specifically:
determining a face non-skin color area in the photo according to the preset skin color range;
determining a face range according to the face UV map;
determining key point coordinate information in the face non-skin color region and non-key point coordinate information in the face range, wherein the key point coordinate information is specifically key point coordinate information of a five-sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV map;
and acquiring the dynamic mask based on the UV coordinate information.
Specifically, firstly, a face non-skin color area in a photo is judged, the face non-skin color area in the photo can be determined through a preset skin color range, and particularly, skin detection based on RGB (red, green and blue), skin detection based on an elliptic skin model, threshold segmentation of YCrCb color space Cr component and Otsu method, screening method based on YCrCb color space Cr and Cb range, screening method of HSV color space H range and opencv self-skin color detection class adaptation SkinDetector are selected to judge the non-skin color area, and all detection modes are based on the preset skin color range detection, so that a person skilled in the art can flexibly select the non-skin color area according to actual needs, and the protection range of the application is not affected.
Since the photo may contain an area (such as hair) outside the outline of the face, further filtering is required, and thus the face range is determined according to the preset UV map. The eyebrow area, eyeball area, lip area and skin color of the facial area in the non-skin color area of the face in the photo are large in difference, and the eyebrow area, eyeball area, lip area and skin color are required to be filtered out further, so that key point coordinate information is determined, the key point coordinate information is specifically the key point coordinate information of the facial area, and the key points of the facial area are not involved in calculation when the non-skin color area of the three-dimensional face is judged. And after the key point coordinate information is determined, non-key point coordinate information in the face range is also determined, and a dynamic mask is obtained according to the key point coordinate information and the UV coordinate information converted by the non-key point coordinate information.
It should be noted that, the solution of the above preferred embodiment is only one specific implementation solution provided in the present application, and other ways of obtaining the dynamic mask corresponding to the mesh model according to the photo and the preset face UV map all belong to the protection scope of the present application.
In order to obtain accurate key point coordinate information and non-key point coordinate information, in a preferred embodiment of the present application, the key point coordinate information in the non-skin color region of the face and the non-key point coordinate information in the face range are determined specifically:
determining the coordinate information of the key points based on a neural network mode of machine learning;
and carrying out interpolation operation by adopting a prediction algorithm according to the key point coordinate information to determine the non-key point coordinate information.
The machine learning neural network mode, such as the cloud vision service platform faceplus, can accurately position the key points, so as to determine the coordinate information of the key points, then, interpolation operation is carried out by adopting a prediction algorithm according to the coordinate information of the key points to determine the coordinate information of the non-key points, the prediction algorithm is the prior art, and the prediction algorithm is not repeated herein, and can be realized by adopting different modes according to specific application scenes by a person skilled in the art.
It should be noted that, the solution of the above preferred embodiment is only one specific implementation solution provided in the present application, and other ways of determining the key point coordinate information in the non-skin color area of the face and the non-key point coordinate information in the face range are all within the protection scope of the present application.
S103, determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region.
And after the dynamic mask is acquired, determining a region to be marked in the skin region of the dynamic mask according to a preset skin color range, wherein the region to be marked corresponds to a non-skin color region in the three-dimensional face.
In order to accurately determine the region to be marked, in a preferred embodiment of the present application, the region to be marked in the skin region of the dynamic mask is determined based on a preset skin color range, specifically:
traversing the skin color region, and judging whether a region exceeding the preset skin color range exists in the skin color region;
if yes, taking the area exceeding the preset skin color range as the area to be marked.
Specifically, since the region to be marked corresponds to the non-skin-color region in the three-dimensional face, traversing whether a region exceeding the preset skin-color range exists in the skin-color region in the dynamic mask, and if so, taking the region exceeding the preset skin-color range as the region to be marked. The specific skin color range can be flexibly preset according to specific application scenes by a person skilled in the art, different skin color ranges belong to the protection range of the application, and the area in the preset skin color range can be used as the area to be marked by setting the preset skin color range as the skin color range corresponding to the non-skin color area, so that the protection range of the application is not affected.
S104, determining the non-skin color region according to the triangular surface information corresponding to the region to be marked.
Because the area to be marked corresponds to the non-skin color area in the three-dimensional face, the triangular surface is a plane formed by three vertexes, and the triangular surface can be restored into grid information, the non-skin color area of the three-dimensional face can be determined according to the triangular surface information corresponding to the area to be marked.
In order to accurately determine a non-skin color region in a three-dimensional face, in a preferred embodiment of the present application, the non-skin color region is determined according to triangular surface information corresponding to the region to be marked, which specifically includes:
generating a triangular surface queue according to the triangular surface where each vertex in the area to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining triangular surface information according to grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color region after performing highlighting operation based on the triangular surface information.
Specifically, the triangular surface queue is generated by the triangular surface where each vertex in the area to be marked is located, and the triangular surface queue generated at this time is unlikely to be completely continuous, for example, the three triangular surface vertices of a, b and c in fig. 9 are connected and are continuous triangular surfaces, so that the triangular surface queue is unified into a non-skin color area, the triangular surface vertices of d and e in fig. 10 are not connected with the three triangular surface vertices of a, b and c in fig. 9, so that the triangular surface vertices in fig. 9 and fig. 10 are discontinuous and are two different non-skin color areas, therefore, the triangular surface queue needs to be classified, a non-skin color area is determined according to the triangular surface of the same type, triangular surface information of the non-skin color area can be determined according to grid information corresponding to each continuous triangular surface after a preset number of continuous triangular surface queues are acquired, and the non-skin color area can be determined through a highlighting operation based on the triangular surface information. The highlighting operation may be highlighting the non-skin color region by cg code, and those skilled in the art may flexibly select other ways to highlight the non-skin color region, which does not affect the protection scope of the present application.
It should be noted that, the solution of the above preferred embodiment is only one specific implementation solution provided in the present application, and other ways of determining the non-skin color area according to the triangular surface information corresponding to the area to be marked all belong to the protection scope of the present application.
In order to accurately obtain a triangular face queue corresponding to a region to be marked, in a preferred embodiment of the present application, after generating the triangular face queue according to a triangular face where each vertex in the region to be marked is located, the method further includes:
step a, selecting a first triangular face from the triangular face queue as a current search face;
b, adding the current search surface into a search queue and a result queue;
step c, querying a triangle surface adjacent to the current search surface in the triangle surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b-d until the search queue is traversed.
Specifically, all triangular faces related to the area to be marked can be determined through the flow, and an accurate triangular face queue is generated.
It should be noted that, the solution of the above preferred embodiment is only one specific implementation solution provided in the present application, and other ways of accurately obtaining the triangular face queue corresponding to the area to be marked all belong to the protection scope of the present application.
In order to obtain the continuous triangular surface queues, in a preferred embodiment of the present application, the triangular surface queues are classified to obtain a preset number of continuous triangular surface queues, which specifically includes:
step A, determining a set A according to the triangular face queue;
step B, determining a set B based on the step a-step e, and storing the set B into a queue DS;
step C, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular face queues according to the classified set stored in the queue DS.
Specifically, the queue DS is a queue marked as DS, the DS is only one mark or symbol, and a person skilled in the art can select other letters to mark the queue, and through the above process, the queue DS stores the classified set, so that a preset number of continuous triangular surface queues can be obtained.
It should be noted that, the solution of the above preferred embodiment is only one specific implementation solution provided in the present application, and other ways of classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues all belong to the protection scope of the present application.
By applying the technical scheme, the target face is scanned in real time, and a photo and a grid model corresponding to the target face are generated; acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image; determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region; and determining the non-skin color region according to the triangular surface information corresponding to the region to be marked, so that the non-skin color region of the three-dimensional face is rapidly and accurately identified, the imaging quality of the three-dimensional face model is timely determined by a user, a three-dimensional face model with higher quality can be generated, and the user experience is improved.
In order to further explain the technical idea of the invention, the technical scheme of the invention is described with specific application scenarios.
The embodiment of the invention provides a method for determining a non-skin color region of a three-dimensional face, which is characterized in that a region to be marked in a skin region of a dynamic mask of a three-dimensional face grid model is determined, and the non-skin color region of the three-dimensional face is determined according to triangular surface information corresponding to the region to be marked, so that local color blocks of the face skin caused by local face shielding or light shading and high light are timely displayed to a user, the imaging quality of the three-dimensional face model is timely determined by the user, and the user experience is improved.
The method comprises the following specific steps:
step one, scanning a target face in real time through three-dimensional imaging equipment, and obtaining a photo of the target face and a Mesh model Mesh, wherein the three-dimensional imaging equipment can be a three-dimensional camera, a three-dimensional imaging scanner and the like.
Step two, determining the non-skin color region of the face in the photo based on the preset skin color range may specifically be that after converting the RGB value of the photo into HSV, filtering according to 7<H<20 28<S<256 50<V<256, for example, processing the face photo shown in fig. 2 through step two to obtain the effect diagram shown in fig. 3.
And thirdly, determining a face range according to a preset face UV image.
Since the photo may include an area (such as hair) outside the outline of the face, further filtering is required, so that the face range is to be determined, the face range can be determined according to the UV map, and since the UV maps of all people are consistent, a default fixed UV map is preset, the photo of the face can be taken by an AR (Augmented Reality ) development platform technology ARKit to obtain a grid model, and UV of the grid model is derived through three-dimensional animation rendering and manufacturing software 3DMAX to be used as a preset UV map of the face, and a person skilled in the art can flexibly select other modes to obtain the preset UV map, wherein the preset UV map is shown in fig. 4.
And step four, determining key point coordinate information in a non-skin color region of the face in the photo and non-key point coordinate information in a face range.
The eyebrow area, eyeball area, lip area and skin color of the facial area in the non-skin color area of the face in the photo are large in difference, and therefore the eyebrow area, eyeball area, lip area and skin color are required to be filtered out further, and key point coordinate information is determined, so that key points of the facial area do not participate in calculation when the non-skin color area of the three-dimensional face is determined, and the key point coordinate information is specifically the key point coordinate information of the facial area. The key point coordinate information can be determined through a machine learning neural network mode, for example, key points can be precisely positioned through a cloud vision service platform facePlusPlus, for example, key points in a map 5 are positioned through facePlusPlus, and an effect map is shown in fig. 6.
The non-key points are other than key points, and the non-key point coordinate information in the face range can be determined by interpolation operation using a prediction algorithm based on the key point coordinate information, for example, the non-key points around the eye area are determined by a difference algorithm, the effect diagram is shown in fig. 7, the prediction algorithm is the prior art, and is not described herein, and the non-key point coordinate information can be realized in different ways according to specific application scenarios by those skilled in the art.
And fifthly, converting the key point coordinate information and the non-key point coordinate information in the fourth step into UV coordinate information in the UV map, and acquiring a dynamic mask corresponding to the grid model in the first step according to the UV coordinate information, wherein the dark color part is a five-sense organ area which does not participate in calculation, as shown in FIG. 8.
And step six, determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range.
The method comprises the steps that other areas except the facial areas in the face are skin color areas, the skin color areas are traversed, areas which appear in the skin color areas and are outside a preset skin color range are used as areas to be marked, and the areas to be marked correspond to non-skin color areas of the three-dimensional face.
And step seven, determining a non-skin color area of the three-dimensional face according to triangular surface information corresponding to the area to be marked.
The non-skin color area of the three-dimensional face can be marked through the result obtained in the step six and displayed on the three-dimensional face model, and the method specifically comprises the following steps:
1) Generating a triangular surface queue according to the triangular surface where each vertex is located in the area to be marked, wherein the triangular surface is a plane formed by three vertexes, and the triangular surface can be restored into grid information;
2) Classifying the generated triangular surface queues to obtain a preset number of continuous triangular surface queues;
3) Determining triangular surface information of a non-skin color area according to grid information corresponding to the continuous triangular surface queue;
4) The non-skin tone regions are highlighted based on the trigonometric plane information by cg code, a high-level shader language programmed for the GPU, to identify the non-skin tone regions. FIG. 11 is a schematic view showing the effect of determining and marking a non-skin tone region.
In order to obtain an accurate triangular face queue corresponding to the area to be marked, the step 1) further includes:
step a, selecting a first triangular face from the triangular face queue as a current search face;
b, adding the current search surface into a search queue and a result queue;
step c, querying a triangle surface adjacent to the current search surface in the triangle surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b-d until the search queue is traversed.
Wherein, the result obtained through step 1) cannot be completely continuous, so the result obtained in step one needs to be classified into different triangular surface information, for example, the vertexes of the three triangular surfaces a, b and c in fig. 9 are connected, thus being continuous triangular surfaces, so that the triangular surface vertexes of d and e in fig. 10 are not connected with the vertexes of the three triangular surfaces a, b and c in fig. 9, so that the triangular surfaces in fig. 9 and 10 are discontinuous, and are two different non-skin color areas.
The step 2) may specifically include:
step A, determining a set A according to the triangular face queue;
step B, determining a set B based on the step a-step e, and storing the set B into a queue DS;
step C, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular face queues according to the classified set stored in the queue DS.
By applying the technical scheme, after the three-dimensional face model is generated by scanning, the generation effect of the model can be immediately detected, if the local color lump of the face skin appears due to local shielding or light shadow and high light during shooting of a user, the user can be immediately marked and informed, and the instant judgment and reminding can be achieved under the condition that shooting and imaging are not affected, so that the user obtains better shooting experience, and the three-dimensional face model with higher quality is generated.
In order to achieve the above technical objective, an embodiment of the present application further provides an apparatus for determining a non-skin color region of a three-dimensional face, as shown in fig. 12, including:
the scanning module 201 is configured to scan a target face in real time, and generate a photo and a grid model corresponding to the target face;
an obtaining module 202, configured to obtain a dynamic mask corresponding to the grid model according to the photograph and a preset face UV map;
a first determining module 203, configured to determine a region to be marked in a skin region of the dynamic mask based on a preset skin color range, where a region in the dynamic mask other than a five-sense organ region is the skin color region;
the second determining module 204 is configured to determine the non-skin color region according to triangular surface information corresponding to the region to be marked.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, one of ordinary skill in the art will appreciate that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not drive the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (7)
1. A method of determining a non-skin tone region of a three-dimensional face, the method comprising: scanning a target face in real time to generate a photo and a grid model corresponding to the target face;
acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image;
determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except the five sense organs region in the dynamic mask is the skin color region;
determining the non-skin color region according to triangular surface information corresponding to the region to be marked;
the method comprises the steps of obtaining a dynamic mask corresponding to the grid model according to the photo and a preset face UV image, specifically: determining a face non-skin color area in the photo according to the preset skin color range;
determining a face range according to the face UV map;
determining key point coordinate information in the face non-skin color region and non-key point coordinate information in the face range, wherein the key point coordinate information is specifically key point coordinate information of a five-sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV map;
acquiring the dynamic mask based on the UV coordinate information;
the determining the non-skin color area according to the triangular surface information corresponding to the area to be marked specifically comprises: generating a triangular surface queue according to the triangular surface where each vertex in the area to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining triangular surface information according to grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color region after performing highlighting operation based on the triangular surface information.
2. The method according to claim 1, wherein the determining of the keypoint coordinate information in the non-skin color area of the face and the non-keypoint coordinate information in the face range is specifically: determining the coordinate information of the key points based on a neural network mode of machine learning;
and carrying out interpolation operation by adopting a prediction algorithm according to the key point coordinate information to determine the non-key point coordinate information.
3. The method according to claim 1, wherein the area to be marked in the skin area of the dynamic mask is determined based on a preset skin tone range, in particular: traversing the skin color region, and judging whether a region exceeding the preset skin color range exists in the skin color region;
if yes, taking the area exceeding the preset skin color range as the area to be marked.
4. The method of claim 1, further comprising, after generating a triangular face queue from the triangular faces where the vertices in the region to be marked are located:
step a, selecting a first triangular face from the triangular face queue as a current search face;
b, adding the current search surface into a search queue and a result queue;
step c, querying a triangle surface adjacent to the current search surface in the triangle surface queue;
step d, if the adjacent triangular surface is not in the result queue, adding the adjacent triangular surface into the search queue and the result queue;
and e, removing the head of the search queue, taking the new head of the search queue as the current search surface, and repeating the steps b-d until the search queue is traversed.
5. The method of claim 4, wherein classifying the triangular face queues to obtain a preset number of continuous triangular face queues, specifically:
step A, determining a set A according to the triangular face queue;
step B, determining a set B based on the step a-step e, and storing the set B into a queue DS;
step C, subtracting the set B from the set A to determine a set C;
step D, taking the set C as a new set A, and repeatedly executing the steps A-C until the new set A is an empty set;
and E, acquiring a preset number of continuous triangular face queues according to the classified set stored in the queue DS.
6. An apparatus for determining a non-skin tone region of a three-dimensional face, comprising: the scanning module is used for scanning the target face in real time and generating a photo and a grid model corresponding to the target face; the acquisition module is used for acquiring a dynamic mask corresponding to the grid model according to the photo and a preset face UV image;
the first determining module is used for determining a region to be marked in a skin region of the dynamic mask based on a preset skin color range, wherein the region except for a five sense organs region in the dynamic mask is the skin color region;
the second determining module is used for determining the non-skin color area according to triangular surface information corresponding to the area to be marked;
the method comprises the steps of obtaining a dynamic mask corresponding to the grid model according to the photo and a preset face UV image, specifically: determining a face non-skin color area in the photo according to the preset skin color range;
determining a face range according to the face UV map;
determining key point coordinate information in the face non-skin color region and non-key point coordinate information in the face range, wherein the key point coordinate information is specifically key point coordinate information of a five-sense organ region;
converting the key point coordinate information and the non-key point coordinate information into UV coordinate information in the UV map;
acquiring the dynamic mask based on the UV coordinate information;
the determining the non-skin color area according to the triangular surface information corresponding to the area to be marked specifically comprises: generating a triangular surface queue according to the triangular surface where each vertex in the area to be marked is located;
classifying the triangular surface queues to obtain a preset number of continuous triangular surface queues;
determining triangular surface information according to grid information corresponding to the continuous triangular surface queue;
and determining the non-skin color region after performing highlighting operation based on the triangular surface information.
7. A computer readable storage medium having instructions stored therein, which when run on a terminal device, cause the terminal device to perform the method of determining a non-skin tone region of a three-dimensional face as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910978982.4A CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910978982.4A CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110751078A CN110751078A (en) | 2020-02-04 |
| CN110751078B true CN110751078B (en) | 2023-06-20 |
Family
ID=69278439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910978982.4A Active CN110751078B (en) | 2019-10-15 | 2019-10-15 | Method and equipment for determining non-skin color region of three-dimensional face |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110751078B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376594A (en) * | 2014-11-25 | 2015-02-25 | 福建天晴数码有限公司 | Three-dimensional face modeling method and device |
| WO2017026839A1 (en) * | 2015-08-12 | 2017-02-16 | 트라이큐빅스 인크. | 3d face model obtaining method and device using portable camera |
| CN106485222A (en) * | 2016-10-10 | 2017-03-08 | 上海电机学院 | A kind of method for detecting human face being layered based on the colour of skin |
| CN107506714A (en) * | 2017-08-16 | 2017-12-22 | 成都品果科技有限公司 | A kind of method of face image relighting |
| WO2018201551A1 (en) * | 2017-05-02 | 2018-11-08 | 厦门美图之家科技有限公司 | Facial image fusion method and apparatus and computing device |
-
2019
- 2019-10-15 CN CN201910978982.4A patent/CN110751078B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376594A (en) * | 2014-11-25 | 2015-02-25 | 福建天晴数码有限公司 | Three-dimensional face modeling method and device |
| WO2017026839A1 (en) * | 2015-08-12 | 2017-02-16 | 트라이큐빅스 인크. | 3d face model obtaining method and device using portable camera |
| CN106485222A (en) * | 2016-10-10 | 2017-03-08 | 上海电机学院 | A kind of method for detecting human face being layered based on the colour of skin |
| WO2018201551A1 (en) * | 2017-05-02 | 2018-11-08 | 厦门美图之家科技有限公司 | Facial image fusion method and apparatus and computing device |
| CN107506714A (en) * | 2017-08-16 | 2017-12-22 | 成都品果科技有限公司 | A kind of method of face image relighting |
Non-Patent Citations (1)
| Title |
|---|
| 阳策 ; 李重 ; 任义 ; 刘恒 ; .基于标准肤色的人脸图像纹理合成与三维重建应用.计算机***应用.2019,(05),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110751078A (en) | 2020-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107771336B (en) | Feature detection and masking in images based on color distribution | |
| CN111445410B (en) | Texture enhancement method, device and equipment based on texture image and storage medium | |
| US9773302B2 (en) | Three-dimensional object model tagging | |
| CN109859098B (en) | Face image fusion method and device, computer equipment and readable storage medium | |
| CN108323204B (en) | Method for detecting face flaw point and intelligent terminal | |
| KR101446975B1 (en) | Automatic face and skin beautification using face detection | |
| KR100682889B1 (en) | Method and Apparatus for image-based photorealistic 3D face modeling | |
| JP4153108B2 (en) | Image processing method, image processing apparatus, and recording medium | |
| JP4519681B2 (en) | Method and apparatus for creating human lip area mask data | |
| JP2004265406A (en) | Method and system for improving portrait image processed in batch mode | |
| JP2007504719A (en) | System and method for detecting and correcting red eyes in an image suitable for embedded applications | |
| JP2007272435A (en) | Face feature extraction device and face feature extraction method | |
| CN112106102A (en) | Image processing method, system, device, movable platform and storage medium | |
| JP2004030007A (en) | Makeup simulation apparatus, makeup simulation method, makeup simulation program and recording medium with program recorded thereon | |
| CN111627076A (en) | Face changing method and device and electronic equipment | |
| CN110648336A (en) | Method and device for dividing tongue texture and tongue coating | |
| CN114372931A (en) | Target object blurring method and device, storage medium and electronic equipment | |
| CN110458012B (en) | Multi-angle face recognition method and device, storage medium and terminal | |
| CN110751078B (en) | Method and equipment for determining non-skin color region of three-dimensional face | |
| CN114155569B (en) | Cosmetic progress detection method, device, equipment and storage medium | |
| KR20010084996A (en) | Method for generating 3 dimension avatar using one face image and vending machine with the same | |
| KR20020085669A (en) | The Apparatus and Method for Abstracting Peculiarity of Two-Dimensional Image & The Apparatus and Method for Creating Three-Dimensional Image Using Them | |
| JP2021157705A (en) | Learned machine learning model, image generation device and training method of machine learning model | |
| JP5099434B2 (en) | Object region extraction system and method | |
| JP4831344B2 (en) | Eye position detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |