CN113971637A - Face image processing method and device, electronic equipment and storage medium - Google Patents
Face image processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113971637A CN113971637A CN202010722901.7A CN202010722901A CN113971637A CN 113971637 A CN113971637 A CN 113971637A CN 202010722901 A CN202010722901 A CN 202010722901A CN 113971637 A CN113971637 A CN 113971637A
- Authority
- CN
- China
- Prior art keywords
- face image
- target
- image
- intermediate frequency
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 230000001815 facial effect Effects 0.000 claims abstract description 99
- 210000000056 organ Anatomy 0.000 claims abstract description 62
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000001914 filtration Methods 0.000 claims description 54
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000009499 grossing Methods 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 3
- 230000036548 skin texture Effects 0.000 abstract description 9
- 238000005192 partition Methods 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 4
- 230000003796 beauty Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 210000004709 eyebrow Anatomy 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 208000007256 Nevus Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000000265 homogenisation Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20224—Image subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present disclosure provides a face image processing method, apparatus, electronic device and storage medium, the method comprising: frequency division and partition processing are carried out on the face image to be processed, a face organ area and a face skin area in the face image are determined, a target area which belongs to the face skin area but does not belong to the face organ area is determined from the face image, and an intermediate frequency component in the target area is removed to obtain a target face image. The face image comprises information of all frequency bands, namely low-frequency, intermediate-frequency and high-frequency information, wherein the intermediate-frequency component corresponds to information of a skin color uneven part in the face image, and the high-frequency component corresponds to skin texture in the face image. According to the scheme disclosed by the invention, only the intermediate frequency component in the target area is removed, namely, the facial flaws of the target area are weakened on the premise of keeping the skin texture, the sense of reality of the skin is kept to the maximum extent, and the face organ area is not subjected to fuzzy processing, so that the definition of the face organ area is ensured.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a method and an apparatus for processing a face image, an electronic device, and a storage medium.
Background
When a user shoots an image or a video of a face, due to illumination or the skin of the face, the skin of the real face is extremely uneven, for example, the face has stains, black nevi, pox and the like, which seriously affect the beauty of the face. Therefore, many applications (such as camera APP, live broadcast APP, and image correction APP) related to image or video processing include a beauty function, so that photos or videos can be beautified, and the beauty of the face is improved.
Most of the peeling algorithms in the related art can change facial skin into nearly pure color, so that the facial skin has serious smearing feeling and is extremely unreal.
Disclosure of Invention
The present disclosure provides a method and an apparatus for processing a face image, an electronic device, and a storage medium, so as to at least solve the problem of severe distortion in face skin color processing in the related art. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a face image processing method, including:
determining a facial organ area and a facial skin area from a face image to be processed;
determining a target area which belongs to the facial skin area and does not belong to the facial organ area in the face image to be processed;
and removing the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven skin color part of the face in the face image.
In a possible implementation manner of the first aspect, the step of removing the intermediate frequency component included in the target region to obtain a target face image includes:
acquiring intermediate frequency components contained in the face image to be processed;
and filtering the intermediate frequency component contained in the target area to obtain the target face image.
In another possible implementation manner of the first aspect, the step of removing the intermediate frequency component contained in the target region to obtain a target face image includes:
and acquiring intermediate frequency components contained in the target area, and filtering the intermediate frequency components in the target area to obtain the target face image.
In yet another possible implementation manner of the first aspect, the step of filtering out an intermediate frequency component contained in the target region to obtain the target face image includes:
obtaining a target pixel value of the target area according to the original pixel value in the target area and a pixel value corresponding to the intermediate frequency component of the face image to be processed;
and obtaining the target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area.
In another possible implementation manner of the first aspect, the obtaining the target face image according to an original pixel value of the to-be-processed face image and a target pixel value of the target region includes:
and replacing the original pixel value corresponding to the pixel point with the target pixel value aiming at any pixel point in the target area to obtain the target face image.
In another possible implementation manner of the first aspect, the obtaining the target face image according to the original pixel value of the to-be-processed face image and the target pixel value of the target region includes:
for any pixel point in the target area, replacing the original pixel value of the pixel point with the target pixel value to obtain a first face image;
and fusing pixel values corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion proportion to obtain the target face image.
In another possible implementation manner of the first aspect, the process of acquiring the intermediate frequency component included in the image includes:
down-sampling the image to obtain a first down-sampled image;
filtering the first downsampled image by utilizing an edge-preserving filtering algorithm to obtain a first filtered image;
the first downsampled image is downsampled to obtain a second downsampled image, and the second downsampled image is filtered by using a low-pass filtering algorithm to obtain a second filtered image;
and calculating the difference value between the pixel value of the first filtering image and the pixel value of the second filtering image to obtain the intermediate frequency component.
In yet another possible implementation manner of the first aspect, the step of determining a facial organ region from the face image to be processed includes:
carrying out face key point positioning on the face image to be processed;
and the face key points of the face image to be processed and the face key points of the standard face mask image are in one-to-one correspondence to obtain the face mask image corresponding to the face image to be processed, and the gray value of each pixel point in the face mask image represents the probability that the pixel point is a facial organ.
In another possible implementation manner of the first aspect, the step of determining a facial skin region from the face image to be processed includes:
and carrying out skin color detection on the face image to be processed to obtain a skin color mask image, wherein the gray value of each pixel point in the skin color mask image represents the probability that the pixel point is skin.
According to a second aspect of the embodiments of the present disclosure, there is provided a face image processing apparatus including:
a facial organ region determination module configured to perform determination of a facial organ region from a face image to be processed;
the skin area determining module is configured to determine a facial skin area from the face image to be processed;
a target region determination module configured to perform determination of a target region in the face image to be processed that belongs to the facial skin region and does not belong to the facial organ region;
and the skin smoothing processing module is configured to remove the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven part of the face skin color in the face image.
In one possible implementation manner of the second aspect, the skin smoothing processing module includes:
the first intermediate frequency acquisition sub-module is configured to acquire an intermediate frequency component contained in the face image to be processed;
and the intermediate frequency filtering submodule is configured to filter the intermediate frequency component in the target area to obtain the target face image.
In another possible implementation manner of the second aspect, the skin smoothing processing module includes:
a second intermediate frequency acquisition sub-module configured to perform acquisition of an intermediate frequency component contained in the target region;
and the intermediate frequency filtering submodule is configured to filter the intermediate frequency component in the target area to obtain the target face image.
In another possible implementation manner of the second aspect, the intermediate frequency filtering sub-module includes:
the target pixel determination submodule is configured to execute the step of obtaining a target pixel value of the target area according to an original pixel value in the target area and a pixel value corresponding to the intermediate frequency component of the face image to be processed;
and the target face image obtaining sub-module is configured to execute the operation of obtaining the target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area.
In yet another possible implementation manner of the second aspect, the target face image obtaining sub-module is specifically configured to:
and replacing the original pixel value corresponding to the pixel point with the target pixel value aiming at any pixel point in the target area to obtain the target face image.
In another possible implementation manner of the second aspect, the target face image obtaining sub-module includes:
a target area image determining submodule configured to execute replacing an original pixel value of any pixel point in the target area with the target pixel value to obtain a first face image;
and the image fusion submodule is configured to perform fusion on pixel values corresponding to the same pixel point in the to-be-processed face image and the first face image according to a set fusion proportion to obtain the target face image.
In another possible implementation manner of the second aspect, the first intermediate frequency obtaining sub-module includes:
the first downsampling submodule is configured to perform downsampling on the face image to be processed to obtain a first downsampled image;
a first filtering sub-module configured to perform filtering on the first downsampled image by using an edge-preserving filtering algorithm to obtain a first filtered image;
a second filtering sub-module configured to perform downsampling of the first downsampled image to obtain a second downsampled image, and filter the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image;
a first calculation sub-module configured to perform a calculation of a difference between pixel values of the first filtered image and pixel values of the second filtered image, resulting in the intermediate frequency component.
In another possible implementation manner of the second aspect, the facial organ region determination module includes:
the key point positioning sub-module is configured to perform face key point positioning on the face image to be processed;
and the face mask image acquisition submodule is configured to perform one-to-one correspondence between the face key points of the to-be-processed face image and the face key points of the standard face mask image to obtain a face mask image corresponding to the to-be-processed face image, wherein the gray value of each pixel point in the face mask image represents the probability that the pixel point is a face organ.
In another possible implementation manner of the second aspect, the skin region determining module is specifically configured to:
and carrying out skin color detection on the face image to be processed to obtain a skin color mask image, wherein the gray value of each pixel point in the skin color mask image represents the probability that the pixel point is skin.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the face image processing method according to any one of the first aspect.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium, wherein instructions of the storage medium, when executed by a processor of an electronic device, enable the electronic device to execute the face image processing method of any one of the first aspects.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product adapted to execute, when executed on a data processing device, a program for initializing the face image processing method according to any one of the possible implementations of the first aspect.
The face image processing method provided by the embodiment of the disclosure performs frequency division and partition processing on a face image to be processed, determines a face organ area and a face skin area in the face image, and determines a target area which belongs to the face skin area but not the face organ area from the face image. And removing the intermediate frequency component in the target area to obtain a target face image. The face image comprises information of all frequency bands, namely low-frequency, intermediate-frequency and high-frequency information, wherein the intermediate-frequency component corresponds to information of a skin color uneven part in the face image, and the high-frequency component corresponds to skin texture in the face image. Only the mid-frequency components in the target area are removed, i.e. facial imperfections of the target area are weakened while preserving the texture of the skin, thus preserving the sense of realism of the skin to the greatest extent. Also, the blurring process is not performed on the facial organ region, so that the clarity of the facial organ region is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a flow diagram illustrating a method of facial image processing according to an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a process for obtaining a mid-frequency component in a face image to be processed according to an exemplary embodiment;
FIG. 3 is a flow diagram illustrating another method of facial image processing according to an exemplary embodiment;
FIG. 4 is a schematic diagram illustrating a standard face mask image according to an exemplary embodiment;
FIG. 5 is a block diagram illustrating a face image processing apparatus according to an exemplary embodiment;
FIG. 6 is a block diagram illustrating a skin smoothing module according to an exemplary embodiment;
fig. 7 is a block diagram illustrating another face image processing apparatus according to an exemplary embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Fig. 1 is a flowchart illustrating a face image processing method according to an exemplary embodiment, as shown in fig. 1, the face image processing method is used in an APP (Application program) related to face image processing, wherein the APP related to face image processing may be run in a terminal device, such as a smartphone, a tablet computer, a smart camera, and the like, and the method includes the following steps.
In S110, a facial organ region and a facial skin region are determined from the face image to be processed.
The face image to be processed may be a picture or a video image shot by an APP related to face image processing, or may also be a picture or a video image shot by other loaded APPs.
After the face image to be processed is obtained, the region where the facial organ in the face image is located, namely the facial organ region, can be obtained by using the face key point detection model. Such as the eyes, eyebrows, nose wings, roots, mouth, chin line, fossa labialis, facial contours, etc. The facial organ region is a region that needs to be protected, i.e., a region that does not need skin color homogenization processing.
In an embodiment of the present disclosure, a skin color detection algorithm may be used to perform skin color detection on a to-be-processed face image to obtain a skin color mask image, where a gray value of each pixel point of the skin color mask image represents a probability that the pixel point is a face skin.
The skin color mask image is represented by gray values, the gray value corresponding to each pixel point represents the probability value of the pixel point position belonging to the skin color area, the probability value is a numerical value within the range of 0-1, for example, when the gray value is 0.2, the probability that the pixel point belongs to the skin color area is 20%, when the gray value is 0, the pixel point is not a point in the skin color area, and when the gray value is 1, the pixel point is a point in the skin color area.
In S120, a target region belonging to the facial skin region and not belonging to the facial organ region in the face image to be processed is determined.
Part of facial organ regions overlap with facial skin regions, for example, the wing of the nose, the mountain root, etc., and therefore, any one pixel point in the face image may have the following three conditions:
1) belonging to a facial organ region but not to a facial skin region;
2) both in the facial organ region and in the facial skin region;
3) belonging to the facial skin region but not to the facial organ region.
The above-described case 3) is required to perform the skin color equalization processing, and the above-described cases 1) and 2) are required to protect the region, that is, the region belonging to the facial skin region but not the facial organ region is the target region.
In S130, the intermediate frequency component included in the target region is removed to obtain a target face image.
In an image, a high-frequency component refers to a portion of the image where the brightness/gradation change is severe, for example, an edge, a contour; the low frequency component refers to a part of the image where the brightness/gray level change is gentle, such as a large patch; the other parts are intermediate frequency components, such as uneven skin color patches, color spots, moles or pox, or so forth, or referred to as facial blemishes.
If the high-frequency part in the target area is directly filtered, the skin texture can be removed, so that the skin has serious smearing feeling and is extremely unreal. In order to solve the technical problem, in the embodiment of the present disclosure, the region (i.e., the target region) corresponding to the case 3) is not directly subjected to the blurring process, but only the intermediate frequency component in the target region is removed, and the high frequency component and the low frequency component are retained to obtain the target face image.
The human face image to be processed contains low, medium and high frequency information, after the medium frequency component in the target area is removed, the high frequency and low frequency part information of the target area is reserved, and the area containing facial organs is not processed, so that the method weakens the facial flaws on the premise of reserving skin texture, and furthest reserves the skin reality and the definition of the facial organs.
In one possible implementation manner of the present disclosure, as shown in fig. 2, S130 may include the following steps:
in S131, the intermediate frequency component in the entire face image is acquired.
In S132, the intermediate frequency component in the target region is removed to obtain a target face image.
The realization method firstly obtains the intermediate frequency component contained in the whole face image, and then removes the intermediate frequency component contained in the target area, so that the following process of obtaining the intermediate frequency component can be directly executed in parallel on the pixels contained in the whole face image, and the image processing speed can be improved.
As shown in fig. 2, the process of acquiring the intermediate frequency component included in the whole face image to be processed may include the following steps:
in S1311, the face image to be processed is downsampled to obtain a first downsampled image.
The downsampling of the image, i.e. reducing the characteristic dimension and keeping the effective information, is equivalent to reducing the image and generating a thumbnail of the corresponding image. It follows that downsampling an image can reduce the feature dimension of the image and thus increase the image processing speed.
In S1312, the first downsampled image is filtered by using an edge-preserving filtering algorithm to obtain a first filtered image.
The edge-preserving filtering algorithm refers to a special filtering mode which can effectively preserve edge information in an image in the process of carrying out smooth filtering on the image. For example, both surface filtering and guided filtering belong to the edge-preserving filtering algorithms. The edge-preserving filtering algorithm mainly processes high-frequency components in the face image, namely filters partial high-frequency components and preserves medium-frequency and low-frequency components.
In the embodiment of the present disclosure, the blur radius of the edge-preserving filtering algorithm may be set to a larger value, such as 20 pixels. Certainly, the size of the image (e.g., the number of included pixels) is set, the larger the image is, the larger the corresponding blur radius is, and conversely, the smaller the image is, the smaller the corresponding blur radius can be set to be a smaller value. For example, for the 720 × 1080 image, the blur radius may be set to 20 pixels, and other units may be adopted, which are not described herein again.
The fuzzy radius refers to the magnitude of an outward fuzzy value of a center point of an object on a layer, namely, denoising and filtering a region in the fuzzy radius range, so that the filtered image is smoother and flatter.
In S1313, the first downsampled image is downsampled to obtain a second downsampled image, and the second downsampled image is filtered by a low-pass filter algorithm to obtain a second filtered image.
The characteristic dimensionality of the second downsampled image obtained by downsampling the first downsampled image again is lower, and the image processing efficiency is further improved.
The low-pass filtering of the image retains the low-frequency components in the image and filters the medium-frequency and high-frequency components in the image. For example, the average filtering is a common low-pass filtering method, and the average filtering is a method of averaging a target pixel and peripheral pixels and then filling the target pixel back to achieve the filtering purpose.
In S1314, the difference between the pixel value of the first filtered image and the pixel value of the second filtered image is calculated, resulting in an intermediate frequency component.
As already mentioned above, the first filtered image retains information of the intermediate frequency and the low frequency, and the second filtered image retains the low frequency component, so that the intermediate frequency component in the whole face image can be obtained by subtracting the pixel value of the second filtered image from the pixel value of the first filtered image. According to the method, the intermediate frequency component in the image can be obtained by edge-preserving filtering and low-pass filtering and subtracting the two filtering results, the processing process is simple, and the intermediate frequency component in the image can be accurately extracted.
In another possible implementation manner of the present disclosure, the intermediate frequency component of the target region in the face image may be obtained first, and then the intermediate frequency component of the target region is filtered to obtain the target face image. The difference from the implementation shown in fig. 2 is that this implementation processes only the image belonging to the target region to obtain the intermediate frequency component, and further filters the intermediate frequency component of the target region. The intermediate frequency components included in the target region need to be acquired through the processing procedures shown in S1311 to S1314, but the processing object is only an image of the target region and not the entire face image.
The realization method directly obtains the intermediate frequency component contained in the target area, but not the intermediate frequency component contained in the whole image, so the intermediate frequency component obtained by the method is more accurate, the filtered intermediate frequency component has pertinence, and the finally obtained image processing effect is better.
After the intermediate frequency component contained in the whole face image or the target area is obtained, the intermediate frequency component is converted into a time domain to obtain a pixel value corresponding to the intermediate frequency component, and the intermediate frequency component in the target area can be filtered out by subtracting the pixel value corresponding to the intermediate frequency component from the pixel value of the target area to obtain a target pixel value of the target area. And then, obtaining a target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area.
In a possible implementation manner, the original pixel values of the pixels in the target region of the whole face image can be directly replaced by the corresponding target pixel values, and the pixel values of the non-target region in the face image are kept unchanged, so that the target face image is obtained. According to the implementation mode, only the pixel points of the target area in the whole image are replaced by the target pixel values, other pixel points do not need to be processed, the processing process is simple, and the image processing speed is improved on the premise of ensuring the image processing effect.
In another possible implementation manner, each pixel point of a target area in a face image to be processed is replaced by a target pixel value, and a pixel value of a non-target area is kept unchanged to obtain a first face image; and then fusing the first face image and the face image to be processed according to a certain fusion proportion to obtain a target face image. The fusion proportion can be set by a user according to actual requirements. The realization mode can lead the user to adjust the skin homogenizing degree (namely the fusion proportion) according to the own requirements, and the skin homogenizing treatment result is more flexible and diversified.
The method for processing a face image according to this embodiment performs frequency division and partition processing on a face image to be processed, determines a facial organ region and a facial skin region in the face image, and determines a target region that belongs to the facial skin region but does not belong to the facial organ region from the face image. And removing the intermediate frequency component in the target area to obtain a target face image. The face image comprises information of all frequency bands, namely low-frequency, intermediate-frequency and high-frequency information, wherein the intermediate-frequency component corresponds to information of a skin color uneven part in the face image, and the high-frequency component corresponds to skin texture in the face image. Only the mid-frequency components in the target area are removed, i.e. facial imperfections of the target area are weakened while preserving the texture of the skin, thus preserving the sense of realism of the skin to the greatest extent. Also, the blurring process is not performed on the facial organ region, so that the clarity of the facial organ region is improved.
In order to more intuitively understand the implementation process of the facial image processing method provided by the present disclosure, which will be described in detail in conjunction with fig. 3, fig. 3 is a processing flow chart illustrating another facial image processing method according to an exemplary embodiment, and the method includes the following steps.
In S210, face key point positioning is performed on the input image.
And carrying out face key point positioning on the input image by using a face key point detection model to obtain position areas of eyes, eyebrows, nose wings, mountain roots, mouths, lower jaw lines, lip sockets, face contours and the like on the face image to be processed.
In S220, the face key points of the input image and the face key points of the standard face mask image are in one-to-one correspondence, so as to obtain a face mask image corresponding to the input image.
The key points of the face corresponding to the InputImage are in one-to-one correspondence with the key points of the standard face mask image, for example, each key point of the eyes in the standard face mask image is mapped to each key point of the eyes in the InputImage image. Further, a region (e.g., a triangular region formed by three key points) formed by a plurality of key points in the face key point image of the InputImage is replaced by pixel values of the same key point region in the standard face mask image, so as to obtain a face mask image corresponding to the InputImage.
The step completes warp mapping from the mask material image of the standard face to the InputImage, and the face mask image OrganMask of the InputImage is obtained. The gray value of each pixel point in the OrganMask represents the probability that the pixel point belongs to the facial organ. Therefore, the facial organ regions and non-facial organ regions of the face image can be recognized using the organn mask.
Fig. 4 is a schematic diagram showing a standard face mask image, in which black regions are non-facial organ regions of a standard face and white regions are facial organ regions of the standard face. The organn mask is an image similar to fig. 4, except that the position of the facial organ of the organn mask may be different from the organ position of the InputImage.
In S230, the skin color detection is performed on the input image to obtain a skin color mask image of the input image.
And performing facial skin detection on the InputImage by using a skin color detection algorithm to obtain a facial skin color area and obtain a skin color mask image SkinMask.
The SkinMask is represented by gray values, each gray value represents the probability that the point belongs to the facial skin region, the probability value is a numerical value in a range of 0-1, for example, when the gray value is 0.2, the probability that the pixel point belongs to the skin color region is 20%, when the gray value is 0, the pixel point is not a point in the skin color region, and when the gray value is 1, the pixel point is a point in the skin color region.
It should be noted that the present disclosure does not limit the order of acquiring the OrganMask and acquiring the SkinMask.
In S240, the input image is downsampled by a first preset coefficient to obtain DsImage, and the DsImage is subjected to surface filtering to obtain surface blurrimage.
In this embodiment, the first preset coefficient may be set according to the actual application requirement, for example, may be set to 2, that is, 2 times of down-sampling is performed. The edge-preserving filtering mode filters out high-frequency components in the image, such as skin texture.
In S250, the DsImage is downsampled by a second preset coefficient, and the sampled image is low-pass filtered to obtain blurrimage.
And low-pass filtering is carried out to filter out middle and high frequency components in the image and keep low frequency components in the image. The second predetermined coefficient may be set according to actual requirements, for example, 2 times. Blurrimage is the second filtered image described above.
In S260, the difference between the surfeitmage and the blurrimage is calculated to obtain DiffImage.
That is, DiffImage is a surface blurrimage, where DiffImage mainly contains intermediate frequency components to be removed, such as uneven skin color blocks, spots, moles, and the like.
In S270, the intermediate frequency component in the target region is removed according to the OrganMask and the skinnmask, and a target face image is obtained.
The target region is a region in the face image that belongs to the facial skin region and does not belong to the facial organ region.
In one embodiment of the present disclosure, the process of obtaining the first face image TempImage according to the following formula is as follows:
TempImage-DiffImage SkinMask (1.0-OrganMask) (formula 1)
In formula 1, DiffImage is the intermediate frequency component to be removed, SkinMask represents the probability of belonging to facial skin in InputImage, orgamsk represents the probability of belonging to facial organs in InputImage, (1.0-orgamsk) represents the probability of not belonging to facial organs in InputImage;
as can be seen from equation 1, if the SkinMask is 0, indicating that the region is not a facial skin region, then DiffImage SkinMask (1.0-organmark) is 0, and therefore, TempImage is InputImage, that is, the region is not processed.
If organmark is 1, it indicates that the region belongs to a facial organ region, 1.0-organmark is 0, DiffImage skinmmark (1.0-organmark) is 0, and therefore, TempImage is InputImage, i.e., the region is not processed.
Therefore, according to the above analysis, formula 1 means that the intermediate frequency component is removed from the input image, that is, the uneven skin color patch, spot, mole, and the like, which belong to the facial skin region and do not belong to the facial organ region (i.e., the target region), that is, the facial skin region and do not belong to the facial organ region.
And further fusing the TempImage and the InputImage according to a formula 2 to obtain final image output.
OutputImage＝mix(InputImage,TempImage,BlurAlpha)
(InputImage (1-BlurAlpha) + TempImage BlurAlpha (formula 2)
In formula 2, OutputImage represents the final output target face image, BlurAlpha represents the skin smoothing processing degree, which can be adjusted by the user, and BlurAlpha is the set fusion proportion.
For example, if BlurAlpha is 0, indicating that the input image is not subjected to the skin smoothing processing, OutputImage is InputImage; if BlurAlpha is 1, OutputImage is TempImage, that is, TempImage obtained in the previous step is directly used as the final target face image. If BlurAlpha is 0.5, OutputImage is InputImage 0.5+ TempImage 0.5, i.e. for any pixel, 50% of InputImage and 50% of TempImage are superimposed to form the sum.
The method for processing a face image provided by the embodiment removes the intermediate frequency component only for the region which belongs to the skin region and does not belong to the facial organ region in the face image. The facial flaws of the target area are weakened on the premise of keeping the skin texture in the face image, so that the sense of realism of the skin is kept to the maximum extent. In addition, the embodiment can also fuse the input image and the image after the skin smoothing processing according to the skin smoothing processing degree set by the user to obtain the final target face image so as to realize the skin smoothing processing with different skin smoothing degrees on the input image, and the skin smoothing processing is more flexible.
Fig. 5 is a block diagram illustrating a face image processing apparatus according to an exemplary embodiment. Referring to fig. 5, the apparatus includes a facial organ region determining module 110, a skin region determining module 120, a target region determining module 130, and a skin smoothing processing module 140.
The facial organ region determination module 110 is configured to perform the determination of the facial organ region from the face image to be processed.
In one possible implementation, the facial organ region determination module includes: a key point positioning sub-module and a face mask image acquisition sub-module.
The key point positioning submodule is configured to perform face key point positioning on a face image to be processed.
The face mask image acquisition submodule is configured to perform one-to-one correspondence between face key points of a to-be-processed face image and face key points of a standard face mask image to obtain a face mask image corresponding to the to-be-processed face image.
The gray value of each pixel point in the face mask image represents the probability that the pixel point is a facial organ.
The skin region determination module 120 is configured to perform the determination of the facial skin region from the face image to be processed.
The skin area determining module 120 is specifically configured to perform skin color detection on the face image to be processed to obtain a skin color mask image, where a gray value of each pixel point in the skin color mask image represents a probability that the pixel point is skin.
The target region determining module 130 is configured to perform determining a target region belonging to a facial skin region and not belonging to a facial organ region in the face image to be processed.
And the skin smoothing processing module 140 is configured to remove the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven facial skin color part in the face image.
In an embodiment of the present disclosure, the intermediate frequency component included in the whole face image is obtained first, and then the intermediate frequency component in the target region is filtered, as shown in fig. 6, the skin smoothing processing module 140 includes a first intermediate frequency obtaining submodule 141 and an intermediate frequency filtering submodule 142.
The first intermediate frequency obtaining sub-module 141 is configured to perform obtaining of an intermediate frequency component included in the face image to be processed.
In one possible implementation, as shown in fig. 6, the first intermediate frequency obtaining sub-module 141 includes:
and the first downsampling submodule 1411 is configured to perform downsampling on the face image to be processed to obtain a first downsampled image.
A first filtering sub-module 1412 configured to perform filtering of the first downsampled image using an edge-preserving filtering algorithm to obtain a first filtered image.
A second filtering sub-module 1413 configured to perform downsampling the first downsampled image to obtain a second downsampled image, and filter the second downsampled image using a low-pass filtering algorithm to obtain a second filtered image.
A first calculation sub-module 1414 configured to perform a calculation of differences between pixel values of the first filtered image and pixel values of the second filtered image, resulting in an intermediate frequency component.
The intermediate frequency filtering sub-module 142 is configured to perform filtering to remove intermediate frequency components in the target region to obtain a target face image.
In one possible implementation, as shown in fig. 6, the intermediate frequency filtering sub-module 142 includes:
the target pixel determining sub-module 1421 is configured to perform, according to the original pixel value in the target region and the pixel value corresponding to the intermediate frequency component of the face image to be processed, obtaining a target pixel value of the target region.
The target face image obtaining sub-module 1422 is configured to perform obtaining a target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area.
In a possible implementation manner, the target face image obtaining sub-module 1422 is specifically configured to: and replacing the original pixel value corresponding to the pixel point with a target pixel value aiming at any pixel point in the target area to obtain a target face image.
In another possible implementation, the target face image obtaining sub-module 1422 includes: a target area image determining submodule and an image fusion submodule;
the target area image determining submodule is configured to execute replacing an original pixel value of a pixel point with a target pixel value for any pixel point in a target area to obtain a first face image.
The image fusion submodule is configured to perform fusion on pixel values corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion proportion to obtain a target face image.
In another embodiment of the present disclosure, the intermediate frequency component of the target region in the face image may be obtained first, and then the intermediate frequency component of the target region is filtered to obtain the target face image. The process of acquiring the intermediate frequency component included in the target region is basically the same as the process of acquiring the intermediate frequency component of the whole image, but the processing object is only the image of the target region and not the whole face image, and the details are not repeated here.
The face image processing apparatus provided in this embodiment performs frequency division and partition processing on a face image to be processed, determines a facial organ region and a facial skin region in the face image, and determines a target region that belongs to the facial skin region but does not belong to the facial organ region from the face image. And removing the intermediate frequency component in the target area to obtain a target face image. The face image comprises information of all frequency bands, namely low-frequency, intermediate-frequency and high-frequency information, wherein the intermediate-frequency component corresponds to information of a skin color uneven part in the face image, and the high-frequency component corresponds to skin texture in the face image. Only the mid-frequency components in the target area are removed, i.e. facial imperfections of the target area are weakened while preserving the texture of the skin, thus preserving the sense of realism of the skin to the greatest extent. Also, the blurring process is not performed on the facial organ region, so that the clarity of the facial organ region is improved.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In another aspect, the present disclosure also provides an electronic device, including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement any one of the above-mentioned face image processing methods. As shown in fig. 7, the electronic device includes at least one memory 210, at least one processor 220, and a bus 230. Wherein the processor 220 communicates with the memory 210 over a bus 230.
The memory 210 stores instructions, and the processor 220 calls the instructions in the memory 210 to execute the above-mentioned face image processing method. The electronic device herein may be a mobile smart terminal (e.g., a smartphone, a tablet, a smart camera, etc.) or a PC, etc.
In an exemplary embodiment, the present disclosure also provides a storage medium comprising instructions, such as the memory 210 comprising instructions, which are executable by the processor 220 of the electronic device to perform the above-mentioned face image processing method.
Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A face image processing method is characterized by comprising the following steps:
determining a facial organ area and a facial skin area from a face image to be processed;
determining a target area which belongs to the facial skin area and does not belong to the facial organ area in the face image to be processed;
and removing the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven skin color part of the face in the face image.
2. The method of claim 1, wherein the step of removing the intermediate frequency component contained in the target region to obtain the target face image comprises:
acquiring intermediate frequency components contained in the face image to be processed;
and filtering the intermediate frequency component contained in the target area to obtain the target face image.
3. The method of claim 1, wherein the step of removing the intermediate frequency component contained in the target region to obtain the target face image comprises:
and acquiring intermediate frequency components contained in the target area, and filtering the intermediate frequency components in the target area to obtain the target face image.
4. The method according to claim 2 or 3, wherein the step of filtering out the intermediate frequency component contained in the target region to obtain the target face image comprises:
obtaining a target pixel value of the target area according to the original pixel value in the target area and a pixel value corresponding to the intermediate frequency component of the face image to be processed;
and obtaining the target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area.
5. The method for processing the face image according to claim 4, wherein the step of obtaining the target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area comprises:
and replacing the original pixel value corresponding to the pixel point with the target pixel value aiming at any pixel point in the target area to obtain the target face image.
6. The method for processing the face image according to claim 4, wherein the step of obtaining the target face image according to the original pixel value of the face image to be processed and the target pixel value of the target area comprises:
for any pixel point in the target area, replacing the original pixel value of the pixel point with the target pixel value to obtain a first face image;
and fusing pixel values corresponding to the same pixel point in the face image to be processed and the first face image according to a set fusion proportion to obtain the target face image.
7. The method according to claim 2 or 3, wherein the process of obtaining the intermediate frequency component contained in the image comprises:
down-sampling the image to obtain a first down-sampled image;
filtering the first downsampled image by utilizing an edge-preserving filtering algorithm to obtain a first filtered image;
the first downsampled image is downsampled to obtain a second downsampled image, and the second downsampled image is filtered by using a low-pass filtering algorithm to obtain a second filtered image;
and calculating the difference value between the pixel value of the first filtering image and the pixel value of the second filtering image to obtain the intermediate frequency component.
8. A face image processing apparatus, comprising:
a facial organ region determination module configured to perform determination of a facial organ region from a face image to be processed;
the skin area determining module is configured to determine a facial skin area from the face image to be processed;
a target region determination module configured to perform determination of a target region in the face image to be processed that belongs to the facial skin region and does not belong to the facial organ region;
and the skin smoothing processing module is configured to remove the intermediate frequency component contained in the target area to obtain a target face image, wherein the intermediate frequency component contains information representing the uneven part of the face skin color in the face image.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the face image processing method of any one of claims 1 to 7.
10. A storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the face image processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010722901.7A CN113971637A (en) | 2020-07-24 | 2020-07-24 | Face image processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010722901.7A CN113971637A (en) | 2020-07-24 | 2020-07-24 | Face image processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113971637A true CN113971637A (en) | 2022-01-25 |
Family
ID=79585552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010722901.7A Pending CN113971637A (en) | 2020-07-24 | 2020-07-24 | Face image processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113971637A (en) |
-
2020
- 2020-07-24 CN CN202010722901.7A patent/CN113971637A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112767289B (en) | Image fusion method, device, medium and electronic equipment | |
| KR102104403B1 (en) | Method and Apparatus for removing haze in a single image | |
| CN108205804B (en) | Image processing method and device and electronic equipment | |
| Pei et al. | Nighttime haze removal using color transfer pre-processing and dark channel prior | |
| CN107369133B (en) | Face image beautifying method and device | |
| CN107172354B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN107194869B (en) | Image processing method and terminal, computer storage medium and computer equipment | |
| Pei et al. | Effective image haze removal using dark channel prior and post-processing | |
| CN113808027B (en) | Human body image processing method and device, electronic equipment and storage medium | |
| Lee et al. | Correction of the overexposed region in digital color image | |
| CN112258440B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN111476735B (en) | Face image processing method and device, computer equipment and readable storage medium | |
| CN107564085B (en) | Image warping processing method and device, computing equipment and computer storage medium | |
| CN112837251A (en) | Image processing method and device | |
| CN113313645B (en) | Image processing method, image processing device, terminal and readable storage medium | |
| KR20140026747A (en) | Apparatus and method for image defogging in hsv color space and recording medium storing program for executing method of the same in computer | |
| CN111815729B (en) | Real-time skin beautifying method, device, equipment and computer storage medium | |
| CN114037641A (en) | Low-illumination image enhancement method, device, equipment and medium | |
| CN107316281B (en) | Image processing method and device and terminal equipment | |
| US20240013358A1 (en) | Method and device for processing portrait image, electronic equipment, and storage medium | |
| CN114862729A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN110136085B (en) | Image noise reduction method and device | |
| WO2023103813A1 (en) | Image processing method and apparatus, device, storage medium, and program product | |
| KR101089902B1 (en) | Apparatus and method for estimating edge of digital image | |
| CN111598794A (en) | Image imaging method and device for removing underwater overlapping condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |