CN110555812A - image adjusting method and device and computer equipment - Google Patents

image adjusting method and device and computer equipment Download PDF

Info

Publication number
CN110555812A
CN110555812A CN201910670790.7A CN201910670790A CN110555812A CN 110555812 A CN110555812 A CN 110555812A CN 201910670790 A CN201910670790 A CN 201910670790A CN 110555812 A CN110555812 A CN 110555812A
Authority
CN
China
Prior art keywords
image
points
face
triangular
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910670790.7A
Other languages
Chinese (zh)
Inventor
邹超洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority to CN201910670790.7A priority Critical patent/CN110555812A/en
Publication of CN110555812A publication Critical patent/CN110555812A/en
Priority to PCT/CN2019/126719 priority patent/WO2021012599A1/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/14Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • G06T5/94Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

the invention relates to an image adjusting method, an image adjusting device and computer equipment, wherein the method comprises the following steps: carrying out face detection on an input image to obtain a face feature point set; triangulating the input image from the set of facial feature points to obtain a plurality of original triangular images for the input image; adjusting the coordinates of part of the face characteristic points of the face characteristic point set to obtain a transition image; triangulating the transition image according to the face feature point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image; judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same; if not, affine transformation is carried out on the pixel points in the original triangular image, and transformation coordinates of the pixel points in the deformed triangular image are obtained. The method has the advantages of local deformation and small calculation amount, and has good effect of keeping a non-deformation area and an image background.

Description

Image adjusting method and device and computer equipment
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image adjustment method, an image adjustment device, and a computer apparatus.
background
with the development and popularization of portrait beautifying technology, users have not satisfied with the basic portrait beautifying technology, and the interests of the users in face deformation technology are increasing. The most typical application functions in the portrait beautifying technology are the functions of big eyes and thin faces of the portrait, big heads of the portrait and other local deformation functions.
the prior portrait beautifying technology generally carries out integral operation on the whole image, although the portrait beautifying effect reaches the standard, the calculation amount is very large, a plurality of calculation resources are occupied, and the processing speed is slow; moreover, when the face local area is beautified, the maintenance of the background is not considered, and the background at the edge of the area can be obviously deformed.
disclosure of Invention
Based on this, an object of the embodiments of the present invention is to provide an image adjusting method, an image adjusting apparatus, and a computer device, which have the advantages of local deformation and small calculation amount, and have a good effect of maintaining a non-deformation area and an image background.
In a first aspect, an embodiment of the present invention provides an image adjusting method, including the following steps:
Carrying out face detection on an input image to obtain a face feature point set;
Triangulating the input image from the set of facial feature points to obtain a plurality of original triangular images for the input image;
adjusting the coordinates of part of the face characteristic points of the face characteristic point set to obtain a transition image;
triangulating the transition image according to the face feature point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image;
Judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image;
and filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
further, the set of face feature points includes face contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
Further, the step of obtaining the face feature point set further includes: and selecting a plurality of face contour characteristic points to perform coordinate weighting calculation to obtain the cheek characteristic points positioned in the cheek area.
further, the step of selecting a plurality of the face contour feature points to perform coordinate weighting calculation to obtain cheek feature points located in a cheek region includes:
And respectively selecting a highest point and a lowest point on the left side and the right side of the face contour feature points, and respectively carrying out coordinate weighted calculation on the highest point and the lowest point selected on the left side and the right side to obtain cheek feature points positioned in cheek areas on the two sides.
The image adjusting method of the embodiment of the invention obtains the cheek feature points in the middle of the cheek of the human face by carrying out the coordinate weighting operation on the face contour feature points, so that more thin triangles are obtained by triangulation at the cheek of the human face, the adjustment of the cheek is smoother, and particularly, the face thinning effect or the face fat effect is more natural when the portrait of the face thinning and face fat is beautified.
Further, the coordinate adjustment of the partial human face feature points of the human face feature point set includes:
And adjusting the coordinates of the partial human face characteristic points of the human face characteristic point set according to the following formula:
fx=a1+b1cos(w1i)+c1sin(w1i)+d1cos(2w1i)+e1sin(2w1i)
fy=a2+b2cos(w2i)+c2sin(w2i)+d2cos(2w2i)+e2sin(2w2i)+fcos(3w2i)+gsin(3w2i)
In the above formula, i represents the number of the face feature point, fx and fy represent the displacement amounts of the abscissa and ordinate of the face feature point, respectively, and a1、b1、c1、d1、e1And a2、b2、c2、d2、e2F, g are constants, w1And w2the frequencies of the sine and cosine components.
The adjustment formula of the human face characteristic points provided by the image adjustment method of the embodiment of the invention is obtained by equation fitting based on multiple tests and statistical results, and the most smooth and natural image can be obtained by carrying out image deformation through the adjustment formula.
further, the triangulation of the input image according to the face feature point set and the triangulation of the transition image according to the coordinate-adjusted face feature point set include the following steps:
And connecting all human face characteristic points and four vertexes of the input image or the transition image into a plurality of triangles, and subdividing the image area.
Further, the step of performing affine transformation on the pixel points in the original triangular image to obtain transformed coordinates of the pixel points in the deformed triangular image includes:
Calculating an affine transformation matrix between the corresponding original triangular image and the corresponding deformed triangular image according to the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image;
And carrying out affine transformation on the pixel points in the original triangular image according to the affine transformation matrix to obtain transformation coordinates of the pixel points in the deformed triangular image.
Further, the filling of the pixel values of the deformed triangular image according to the transformed coordinates includes performing bilinear interpolation on the deformed triangular image according to the transformed coordinates.
in a second aspect, an embodiment of the present invention provides an image adjusting apparatus, where the apparatus includes:
A face detection module for detecting face of input image to obtain face feature point set
The first triangulation module is used for triangulating the input image according to the face characteristic point set to obtain a plurality of original triangular images related to the input image;
the coordinate adjusting module is used for adjusting the coordinates of part of the human face characteristic points of the human face characteristic point set to obtain a transition image;
The second triangulation module is used for triangulating the transition image according to the face characteristic point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image;
the coordinate transformation module is used for judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same or not, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image;
And the pixel value filling module is used for filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
A memory and a processor;
the memory for storing one or more programs;
When executed by the one or more processors, cause the one or more processors to implement the image adjustment method of the first aspect of the present invention.
According to the image adjusting method, the human face characteristic points are accurately positioned, the input image and the transition image are divided into a plurality of triangles through triangulation, and smooth local human face deformation can be achieved through affine transformation and pixel value filling modes aiming at the pixel points of which the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are changed. Because only the pixel points of the triangular area with the changed vertex coordinates are adjusted, rather than the whole image, the calculation amount is small, the processing speed is high, and the good effect of keeping the non-deformation area and the image background is achieved; furthermore, a plurality of local positions of the input image may be simultaneously deformed, and if the face feature points related to the simultaneously deformed local positions are different, the face feature points do not affect or interfere with each other.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
FIG. 1 is a flow chart of an image adjustment method of the present invention shown in one exemplary embodiment;
FIG. 2 is a facial feature point diagram illustrating the image adjustment method of the present invention in an exemplary embodiment;
FIG. 3 is a graph illustrating the displacement variation of the facial contour feature points of the image adjustment method of the present invention in an exemplary embodiment;
FIG. 4 is a triangulation diagram of the image adjustment method of the present invention shown in one exemplary embodiment;
FIG. 5 is a flow chart of affine transformation of the image adjustment method of the present invention shown in one exemplary embodiment;
FIG. 6 is a diagram of the effects obtained using the image adjustment method of the present invention in one exemplary embodiment;
Fig. 7 is a block diagram showing the configuration of an image adjusting apparatus of the present invention in an exemplary embodiment;
FIG. 8 is a block diagram of the architecture of the computer device of the present invention shown in an exemplary embodiment.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some but not all of the relevant aspects of the present invention are shown in the drawings.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
As shown in fig. 1, in an exemplary embodiment, the image adjusting method of the present embodiment includes the following steps:
Step S101: and carrying out face detection on the input image to obtain a face feature point set.
Step S102: triangulating the input image from the set of facial feature points to obtain a plurality of original triangular images for the input image.
Step S103: and adjusting the coordinates of part of the human face characteristic points of the human face characteristic point set to obtain a transition image.
Step S104: triangulating the transition image according to the face feature point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image; .
Step S105: judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, affine transformation is carried out on the pixel points in the original triangular image, and transformation coordinates of the pixel points in the deformed triangular image are obtained.
Step S106: and filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
The face feature point set according to the embodiment of the present invention includes a plurality of face feature points located at different positions of a face, and optionally, the face feature point set includes face contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points. The face characteristic points are selected to represent most characteristics of the face and effectively distinguish different faces. The face feature point set may detect and identify an input image through various existing face detection algorithms, and specifically, in the embodiment of the present invention, 68 individual face feature points are obtained by performing face detection on the input image through a Dlib face detection algorithm.
in an exemplary embodiment, considering that the facial feature points identified by the existing face detection algorithm hardly relate to the facial feature points in the middle of the cheek (colloquially called the position of the face) on the left and right sides of the nose below the eyes, the result is when triangulation is performed from the set of facial feature points. The middle part of the cheek is formed with a triangular area with a larger area, so that the middle part of the cheek cannot be effectively adjusted, for example, only the face contour is adjusted in a face thinning or face enlarging processing algorithm, after face thinning or face enlarging, the middle part of the cheek (a facial egg) is not changed, the image after face thinning or face enlarging is not smooth and natural enough, and the image deformation effect is poor. In order to overcome the above defect, in the embodiment of the present invention, the step of obtaining a face feature point set further includes: and selecting a plurality of face contour characteristic points to perform coordinate weighting calculation to obtain the cheek characteristic points positioned in the cheek area. The reason for selecting the face contour feature points is as follows: the distribution of the face contour feature points determines information such as the shape and size of the cheek, and the cheek feature points obtained by the weighting calculation of the face contour feature point coordinates can more accurately reflect the characteristics of the cheek. The number and the positions of the face contour feature points can be selected according to the requirements of experience and/or calculation amount.
preferably, the step of selecting a plurality of the face contour feature points to perform coordinate weighting calculation to obtain cheek feature points located in a cheek region includes:
And respectively selecting a highest point and a lowest point on the left side and the right side of the face contour feature points, and respectively carrying out coordinate weighted calculation on the highest point and the lowest point selected on the left side and the right side to obtain cheek feature points positioned in cheek areas on the two sides.
the highest point and the lowest point are selected for carrying out coordinate weighting calculation, the middle position of the cheek can be accurately positioned, and the obtained coordinate adjustment of the cheek feature points has a smoother and unobtrusive whole cheek adjustment effect.
Specifically, the cheek feature points are two points; selecting a face contour characteristic point positioned at the highest point of the left cheek and a face contour characteristic point positioned at the chin to perform coordinate weighting calculation to obtain a cheek characteristic point positioned at the left cheek; and selecting the face contour characteristic points at the highest point of the right cheek and the face contour characteristic points at the chin for coordinate weighting calculation to obtain the face contour characteristic points at the right cheek. When the cheek feature points of the left cheek and the right cheek are obtained, the selected face contour feature points at the chin are the same.
In a specific embodiment, referring to fig. 2, on the basis of obtaining 68 human face feature points by using a Dlib human face detection algorithm in step S101, the human face feature points in the human face feature point set are numbered according to a preset rule, and then a human face contour feature point (number 0) located at the highest point of the left cheek, a human face contour feature point (number 16) located at the highest point of the right cheek, and a human face contour feature point (number 8) located at the chin are selected to perform a coordinate weighting operation:
Cheek feature points of the left cheek:
cheek feature points of right cheek:
Wherein [0]. x and [0]. y are coordinates of the face contour feature point numbered 0, [8]. x and [8]. y are coordinates of the face contour feature point numbered 8, [16]. x and [16]. y are coordinates of the face contour feature point numbered 16, and x and y are coordinates of the cheek feature point. 0.4 and 0.6 are weight ratios determined over multiple tests, with a weight ratio of 0.4 and 0.6, no distortion is caused to the cheek adjustment, and the adjusted image is natural.
In an exemplary embodiment, in the step of performing coordinate adjustment on the partial facial feature points of the facial feature point set, the coordinate-adjusted displacement amount is obtained by superposition of fitted sine components and cosine components of different harmonics.
Specifically, coordinate adjustment is performed on part of the face feature points of the face feature point set according to the following formula:
fx=a1+b1cos(w1i)+c1sin(w1i)+d1cos(2w1i)+e1sin(2w1i)
fy=a2+b2cos(w2i)+c2sin(w2i)+d2cos(2w2i)+e2sin(2w2i)+fcos(3w2i)+gsin(3w2i)
In the above formula, i represents the number of the face feature points, and for the number of the face feature points, referring to the manner shown in fig. 2, the face contour points are numbered from left to right, and then the eyebrow feature points, the eye feature points, the nose feature points, and the mouth feature points are numbered in sequence. fx and fy represent the displacement of the horizontal and vertical coordinates of the characteristic points of the face, respectively, a1、b1、c1、d1、e1And a2、b2、c2、d2、e2F, g are constants, w1And w2the frequencies of the sine and cosine components.
The method for obtaining the displacement amount of the coordinate adjustment of the human face characteristic points, provided by the image adjusting method of the embodiment of the invention, is obtained by fitting an equation based on multiple tests and statistical results, and the image deformation is carried out according to the displacement amount calculated by the displacement amount obtaining method, so that the most smooth and natural image can be obtained.
through a portrait beautifying test, equation fitting is carried out based on statistical results to obtain a1、b1、c1、d1、e1And a2、b2、c2、d2、e2、f、g,w1And w2a specific reference design may be:
fx=-0.4856+1.725cos(w1i)+2.111sin(w1i)-0.8245cos(2w1i)-0.152sin(2w1i)
fy=-2.348+0.7059cos(w2i)-0.2756sin(w2i)+0.9887cos(2w2i)-0.1879sin(2w2i)
-0.3125cos(3w2i)+0.5605sin(3w2i)
wherein, w1=0.3181,w2=0.3612
taking the large-eye thin face or the small-eye fat face as an example, please refer to fig. 3, the adjustment formula controls the change of the face contour feature points in the X direction and the Y direction along with the number of the face feature points as shown in the figure. If the values of fx and fy are positive, a face thinning deformation operation is performed on the face, and if the values of fx and fy are negative, a fat face transformation is performed. The adjustment strategy of the eye part human face characteristic points is the same as that of the human face contour characteristic points, and the coordinate displacement of the eye part human face characteristic points is obtained by calculating the coordinate displacement of the human face contour characteristic points in proportion.
In an exemplary embodiment, referring to fig. 4, the triangulating the input image according to the face feature point set and triangulating the transition image according to the coordinate-adjusted face feature point set includes the following steps:
connecting all human face characteristic points and four vertexes of the input image or the transition image into a plurality of triangles, and subdividing an image area; when triangulation is performed, triangulation is performed according to the following characteristics:
a. except for the vertices, the triangles in the dissected image region do not contain any points in the set of human face feature points.
b. There are no intersecting edges.
c. all the figures obtained by splitting in the image area are triangles, and the collection of all the triangles is the convex hull of the face feature point set.
Meanwhile, the convex quadrangle formed by any two adjacent triangles obtained by subdivision has the following characteristics: after the diagonal lines are exchanged, the minimum angle of the two internal angles is not increased.
the image adjusting method of the embodiment of the invention adopts Delaunay triangulation for triangulation, and because the four vertexes of the input image or the transition image are included, the human face pixel points and the image background between the human face feature points and the four vertexes can be adjusted, which is beneficial to ensuring that the adjusted image is smoother and more natural.
for each pixel point of the input image, it corresponds to an original triangle image, and is located inside an original triangle formed by three specific human face feature points or image vertices, and the original triangle corresponds to a deformed triangle through the three specific human face feature points after coordinate adjustment, after moving the partial human face feature points to the target position through step S103, the original triangle image is changed into a deformed triangle image due to the movement of the partial human face feature points, and then the pixel points inside the original triangle are transformed into the deformed triangle image in the same deformation manner. This same deformation may be referred to as affine transformation between the vertex coordinates of the corresponding original triangular image and deformed triangular image. In an exemplary embodiment, referring to fig. 5, the step of performing affine transformation on the pixel points in the original triangular image to obtain transformed coordinates of the pixel points in the deformed triangular image includes:
step S501: calculating an affine transformation matrix between the corresponding original triangular image and the corresponding deformed triangular image according to the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image;
step S502: and carrying out affine transformation on the pixel points in the original triangular image according to the affine transformation matrix to obtain transformation coordinates of the pixel points in the deformed triangular image.
after the coordinate transformation of the pixel point is performed, the pixel value is also filled. In an exemplary embodiment, said filling in pixel values of said warped triangular image according to said transformed coordinates comprises bilinear interpolation of said warped triangular image according to said transformed coordinates. Since affine transformation involves transformation in both the X and Y directions, it is more appropriate to perform bilinear interpolation on this basis. Illustratively, the result of performing a large-eye face thinning or a small-eye face thinning on the human image through steps S101 to 106 is shown in FIG. 6.
the beneficial effects of the invention include:
1. The deformation control quantity of the areas such as cheeks, eyes, noses and the like can be obtained in a self-adaptive mode aiming at the faces with different sizes.
2. The calculated amount is small, only the adjusted triangular area is converted, and the processing speed is high.
3. multiple local positions can be adjusted simultaneously without mutual influence and interference. Such as treating the eyes, nose, cheeks, etc., can be performed simultaneously, and the treatment effect of other areas is not influenced by the adjustment of one area.
an image adjusting apparatus comprising:
a face detection module 701, configured to perform face detection on an input image to obtain a face feature point set
a first triangulation module 702, configured to triangulate the input image according to the set of facial feature points to obtain a plurality of original triangular images related to the input image;
A coordinate adjustment module 703, configured to perform coordinate adjustment on part of the face feature points in the face feature point set to obtain a transition image;
A second triangulation module 704, configured to triangulate the transition image according to the set of facial feature points after coordinate adjustment, so as to obtain a plurality of deformed triangle images related to the transition image;
A coordinate transformation module 705, configured to determine whether vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same, where the corresponding original triangular image and the corresponding deformed triangular image respectively use the same three human face feature points before and after coordinate adjustment as vertices; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image;
And a pixel value filling module 706, configured to fill the pixel values of the deformed triangular image according to the transformed coordinates, so as to obtain a target image.
In one exemplary embodiment, the set of face feature points includes face contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
In an exemplary embodiment, the face detection module 701 further includes:
Supplementary feature point generation module 7011: and the method is used for selecting a plurality of face contour characteristic points to perform coordinate weighting calculation so as to obtain the cheek characteristic points positioned in the cheek area.
in an exemplary embodiment, the cheek feature points generated by the auxiliary feature point generation module 7011 are two points, which include:
the left cheek feature point generating module is used for selecting a face contour feature point located at the highest point of a left cheek and a face contour feature point located at a chin to perform coordinate weighting calculation to obtain a cheek feature point located at the left cheek;
And the right cheek feature point generation module is further used for selecting the face contour feature point located at the highest point of the right cheek and the face contour feature point located at the chin to perform coordinate weighting calculation to obtain the cheek feature point located at the right cheek.
in an exemplary embodiment, in the coordinate adjustment module 703, the displacement amount of coordinate adjustment is obtained by superposition of fitted sine component and cosine component of different harmonics. The coordinate adjustment module 703 further includes a displacement amount calculation module 7031: the method is used for adjusting the coordinates of the partial human face characteristic points of the human face characteristic point set according to the following formula:
fx=a1+b1cos(w1i)+c1sin(w1i)+d1cos(2w1i)+e1sin(2w1i)
fy=a2+b2cos(w2i)+c2sin(w2i)+d2cos(2w2i)+e2sin(2w2i)+fcos(3w2i)+gsin(3w2i)
in the above formula, i represents the number of the face feature point, fx and fy represent the displacement amounts of the abscissa and ordinate of the face feature point, respectively, and a1、b1、c1、d1、e1And a2、b2、c2、d2、e2F, g are constants, w1And w2The frequencies of the sine and cosine components.
in an exemplary embodiment, the first triangulation module 702 and the second triangulation module 704 are configured to connect all human face feature points and four vertices of the input image or the transition image into a plurality of triangles, and subdivide an image area; wherein, two adjacent triangles formed form the diagonal of the convex quadrangle, and the minimum angle of two internal angles is not increased after mutual exchange.
In an exemplary embodiment, the coordinate transformation module 705 includes
Affine transformation matrix calculation module 7051: the affine transformation matrix is used for calculating the affine transformation matrix between the corresponding original triangular image and the corresponding deformed triangular image according to the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image;
pixel affine transformation module 7052: and carrying out affine transformation on the pixel points in the original triangular image according to the affine transformation matrix to obtain transformation coordinates of the pixel points in the deformed triangular image.
In an exemplary embodiment, the pixel value filling module 706 is configured to perform bilinear interpolation on the deformed triangle image according to the transformed coordinates.
The present invention also provides a computer apparatus comprising:
a processor;
a memory for storing a computer program executable by the processor;
Wherein the processor implements the image adjusting method described in any of the above embodiments when executing the program.
As shown in fig. 8, fig. 8 is a block diagram illustrating a computer device according to an exemplary embodiment of the present invention.
In an embodiment, the computer device may be a computer, a mobile phone, a tablet computer, an interactive smart tablet, a PDA (Personal Digital Assistant), an e-book reader, a multimedia player, and the like. In the embodiment, a computer device is taken as an example of an interactive smart tablet to describe.
the memory 801 is used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as the program of the image adjustment method according to any embodiment of the present invention, and program instructions/modules corresponding to the image adjustment method according to any embodiment of the present invention (for example, a human face detection module 701, a first triangulation module 702, a coordinate adjustment module 703, a second triangulation module 704, a coordinate transformation module 705, a pixel value filling module 706, and the like in the image adjustment apparatus). The memory 801 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the device, and the like. Further, the memory 801 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 801 may further include memory located remotely from processor 800, which may be connected to devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
the processor 800 executes various functional applications of the apparatus and data processing by running software programs, instructions, and modules stored in the memory 801, that is, implements the image adjustment method described above.
In particular, in an exemplary embodiment, the following operations are implemented when the processor 800 executes one or more programs stored in the memory 801: carrying out face detection on an input image to obtain a face feature point set; triangulating the input image from the set of facial feature points to obtain a plurality of original triangular images for the input image; adjusting the coordinates of part of the face characteristic points of the face characteristic point set to obtain a transition image; triangulating the transition image according to the face feature point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image; judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image; and filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
on the basis of the above embodiment, the face feature point set includes face contour feature points, eyebrow feature points, eye feature points, nose feature points, and mouth feature points.
on the basis of the above embodiment, the acquiring a face feature point set further includes: and selecting a plurality of face contour characteristic points to perform coordinate weighting calculation to obtain the cheek characteristic points positioned in the cheek area.
On the basis of the above embodiment, the selecting a plurality of the face contour feature points to perform coordinate weighting calculation to obtain cheek feature points located in a cheek region includes:
the cheek characteristic points are two points;
Selecting a face contour characteristic point positioned at the highest point of the left cheek and a face contour characteristic point positioned at the chin to perform coordinate weighting calculation to obtain a cheek characteristic point positioned at the left cheek;
And selecting the face contour characteristic points at the highest point of the right cheek and the face contour characteristic points at the chin for coordinate weighting calculation to obtain the face contour characteristic points at the right cheek.
on the basis of the foregoing embodiment, the coordinate adjustment of the partial facial feature points of the facial feature point set is performed, and a displacement amount of the coordinate adjustment is obtained by superimposing fitted sine components and cosine components of different harmonics, and includes:
numbering the human face characteristic points of the human face characteristic point set according to a preset rule;
and adjusting the coordinates of the partial human face characteristic points of the human face characteristic point set according to the serial numbers of the human face characteristic points and the following formula:
fx=a1+b1cos(w1i)+c1sin(w1i)+d1cos(2w1i)+e1sin(2w1i)
fy=a2+b2cos(w2i)+c2sin(w2i)+d2cos(2w2i)+e2sin(2w2i)+fcos(3w2i)+gsin(3w2i)
in the above formula, i represents the number of the face feature point, fx and fy represent the displacement amounts of the abscissa and ordinate of the face feature point, respectively, and a1、b1、c1、d1、e1And a2、b2、c2、d2、e2f, g are constants, w1And w2The frequencies of the sine and cosine components.
on the basis of the above embodiment, the triangulating the input image according to the face feature point set and triangulating the transition image according to the face feature point set after coordinate adjustment includes:
Connecting all human face characteristic points and four vertexes of the input image or the transition image into a plurality of triangles, and subdividing an image area; wherein, two adjacent triangles formed form the diagonal of the convex quadrangle, and the minimum angle of two internal angles is not increased after mutual exchange.
on the basis of the above embodiment, the performing affine transformation on the pixel points in the original triangular image to obtain transformed coordinates of the pixel points in the deformed triangular image includes:
calculating an affine transformation matrix between the corresponding original triangular image and the corresponding deformed triangular image according to the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image;
and carrying out affine transformation on the pixel points in the original triangular image according to the affine transformation matrix to obtain transformation coordinates of the pixel points in the deformed triangular image.
On the basis of the above embodiment, the filling of the pixel values of the deformed triangular image according to the transformed coordinates includes performing bilinear interpolation on the deformed triangular image according to the transformed coordinates.
the computer device provided above can be used to execute the image adjusting method provided in any of the above embodiments, and has corresponding functions and advantages. The implementation process of the functions and actions of each component in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the apparatus embodiment, since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment for relevant points. The above-described device embodiments are merely illustrative, wherein the components described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
it will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
the above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

Claims (10)

1. an image adjustment method is characterized by comprising the following steps:
Carrying out face detection on an input image to obtain a face feature point set;
Triangulating the input image from the set of facial feature points to obtain a plurality of original triangular images for the input image;
Adjusting the coordinates of part of the face characteristic points of the face characteristic point set to obtain a transition image;
Triangulating the transition image according to the face feature point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image;
Judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image;
and filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
2. The image adjustment method according to claim 1, characterized in that: the face feature point set comprises face contour feature points, eyebrow feature points, eye feature points, nose feature points and mouth feature points.
3. the image adjustment method according to claim 2, wherein the step of obtaining the face feature point set further comprises: and selecting a plurality of face contour characteristic points to perform coordinate weighting calculation to obtain the cheek characteristic points positioned in the cheek area.
4. the image adjustment method according to claim 3, wherein the step of selecting a plurality of the face contour feature points to perform coordinate weighting calculation to obtain cheek feature points located in a cheek region includes:
and respectively selecting a highest point and a lowest point on the left side and the right side of the face contour feature points, and respectively carrying out coordinate weighted calculation on the highest point and the lowest point selected on the left side and the right side to obtain cheek feature points positioned in cheek areas on the two sides.
5. the image adjustment method according to any one of claims 1 to 4, wherein in the step of performing coordinate adjustment on the partial human face feature points of the human face feature point set, displacement amounts of the coordinate adjustment are obtained by superposition of sine components and cosine components of different fitted harmonics.
6. the image adjustment method according to any one of claims 1 to 4, wherein the triangulation of the input image from the set of facial feature points and the triangulation of the transition image from the set of coordinate-adjusted facial feature points comprises:
And connecting all human face characteristic points and four vertexes of the input image or the transition image into a plurality of triangles, and subdividing the image area.
7. the image adjustment method according to claim 1, wherein the step of performing affine transformation on the pixel points in the original triangular image to obtain transformed coordinates of the pixel points in the deformed triangular image includes:
calculating an affine transformation matrix between the corresponding original triangular image and the corresponding deformed triangular image according to the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image;
And carrying out affine transformation on the pixel points in the original triangular image according to the affine transformation matrix to obtain transformation coordinates of the pixel points in the deformed triangular image.
8. The image adjustment method according to claim 1, characterized in that: and filling the pixel values of the deformed triangular image according to the transformed coordinates, wherein bilinear interpolation is carried out on the deformed triangular image according to the transformed coordinates.
9. An image adjusting apparatus, characterized in that the apparatus comprises:
A face detection module for detecting face of input image to obtain face feature point set
The first triangulation module is used for triangulating the input image according to the face characteristic point set to obtain a plurality of original triangular images related to the input image;
The coordinate adjusting module is used for adjusting the coordinates of part of the human face characteristic points of the human face characteristic point set to obtain a transition image;
The second triangulation module is used for triangulating the transition image according to the face characteristic point set after coordinate adjustment to obtain a plurality of deformed triangular images related to the transition image;
the coordinate transformation module is used for judging whether the vertex coordinates of the corresponding original triangular image and the corresponding deformed triangular image are the same or not, wherein the corresponding original triangular image and the corresponding deformed triangular image respectively take the same three human face characteristic points before and after coordinate adjustment as vertexes; if not, carrying out affine transformation on the pixel points in the original triangular image to obtain transformation coordinates of the pixel points in the deformed triangular image;
And the pixel value filling module is used for filling the pixel values of the deformed triangular image according to the transformed coordinates to obtain a target image.
10. a computer device, comprising:
A memory and a processor;
the memory for storing one or more programs;
When executed by the one or more processors, cause the one or more processors to implement the image adjustment method of any one of claims 1-8.
CN201910670790.7A 2019-07-24 2019-07-24 image adjusting method and device and computer equipment Pending CN110555812A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201910670790.7A CN110555812A (en) 2019-07-24 2019-07-24 image adjusting method and device and computer equipment
PCT/CN2019/126719 WO2021012599A1 (en) 2019-07-24 2019-12-19 Image adjustment method, apparatus, and computer device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910670790.7A CN110555812A (en) 2019-07-24 2019-07-24 image adjusting method and device and computer equipment

Publications (1)

Publication Number Publication Date
CN110555812A true CN110555812A (en) 2019-12-10

Family

ID=68735883

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910670790.7A Pending CN110555812A (en) 2019-07-24 2019-07-24 image adjusting method and device and computer equipment

Country Status (2)

Country Link
CN (1) CN110555812A (en)
WO (1) WO2021012599A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021012599A1 (en) * 2019-07-24 2021-01-28 广州视源电子科技股份有限公司 Image adjustment method, apparatus, and computer device
CN112488977A (en) * 2020-12-22 2021-03-12 北京达佳互联信息技术有限公司 Image processing method and device, electronic equipment and storage medium

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112884642B (en) * 2021-03-23 2022-07-08 福州大学 Real-time facial aging simulation method based on face recognition technology
CN113436063B (en) * 2021-07-30 2024-03-05 北京达佳互联信息技术有限公司 Image processing method, device, electronic equipment and storage medium
CN113961746B (en) * 2021-09-29 2023-11-21 北京百度网讯科技有限公司 Video generation method, device, electronic equipment and readable storage medium
CN115187822B (en) * 2022-07-28 2023-06-30 广州方硅信息技术有限公司 Face image dataset analysis method, live face image processing method and live face image processing device

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002003709A2 (en) * 2000-06-30 2002-01-10 Intel Corporation Method of video coding the movement of a human face from a sequence of images
CN101347332A (en) * 2008-08-22 2009-01-21 深圳先进技术研究院 Measurement method and equipment of digitized measurement system of human face three-dimensional surface shape
CN102663413A (en) * 2012-03-09 2012-09-12 中盾信安科技(江苏)有限公司 Multi-gesture and cross-age oriented face image authentication method
CN105894446A (en) * 2016-05-09 2016-08-24 西安北升信息科技有限公司 Automatic face outline modification method for video
CN106980809A (en) * 2016-01-19 2017-07-25 深圳市朗驰欣创科技股份有限公司 A kind of facial feature points detection method based on ASM
CN107527034A (en) * 2017-08-28 2017-12-29 维沃移动通信有限公司 A kind of face contour method of adjustment and mobile terminal
CN108537126A (en) * 2018-03-13 2018-09-14 东北大学 A kind of face image processing system and method
CN108876705A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 Image synthetic method, device and computer storage medium
CN109242905A (en) * 2018-08-31 2019-01-18 华中科技大学 A method of ohject displacement amount and direction are determined based on digital picture diffraction
CN109522910A (en) * 2018-12-25 2019-03-26 浙江商汤科技开发有限公司 Critical point detection method and device, electronic equipment and storage medium
CN110049351A (en) * 2019-05-23 2019-07-23 北京百度网讯科技有限公司 The method and apparatus of Facial metamorphosis, electronic equipment, computer-readable medium in video flowing

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109697688B (en) * 2017-10-20 2023-08-04 虹软科技股份有限公司 Method and device for image processing
CN110555812A (en) * 2019-07-24 2019-12-10 广州视源电子科技股份有限公司 image adjusting method and device and computer equipment

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002003709A2 (en) * 2000-06-30 2002-01-10 Intel Corporation Method of video coding the movement of a human face from a sequence of images
CN101347332A (en) * 2008-08-22 2009-01-21 深圳先进技术研究院 Measurement method and equipment of digitized measurement system of human face three-dimensional surface shape
CN102663413A (en) * 2012-03-09 2012-09-12 中盾信安科技(江苏)有限公司 Multi-gesture and cross-age oriented face image authentication method
CN106980809A (en) * 2016-01-19 2017-07-25 深圳市朗驰欣创科技股份有限公司 A kind of facial feature points detection method based on ASM
CN105894446A (en) * 2016-05-09 2016-08-24 西安北升信息科技有限公司 Automatic face outline modification method for video
CN107527034A (en) * 2017-08-28 2017-12-29 维沃移动通信有限公司 A kind of face contour method of adjustment and mobile terminal
CN108876705A (en) * 2017-11-23 2018-11-23 北京旷视科技有限公司 Image synthetic method, device and computer storage medium
CN108537126A (en) * 2018-03-13 2018-09-14 东北大学 A kind of face image processing system and method
CN109242905A (en) * 2018-08-31 2019-01-18 华中科技大学 A method of ohject displacement amount and direction are determined based on digital picture diffraction
CN109522910A (en) * 2018-12-25 2019-03-26 浙江商汤科技开发有限公司 Critical point detection method and device, electronic equipment and storage medium
CN110049351A (en) * 2019-05-23 2019-07-23 北京百度网讯科技有限公司 The method and apparatus of Facial metamorphosis, electronic equipment, computer-readable medium in video flowing

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
SHERIN F. ALY: "Techniques for Facial Expression Recognition using the Kinect", 《VTECHWORKS.LIB.VT.EDU》 *
郑青碧 等: "基于关键特征点的人脸纹理映射", 《计算机与数字工程》 *
陈鸿: "人脸图像变形技术研究", 《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021012599A1 (en) * 2019-07-24 2021-01-28 广州视源电子科技股份有限公司 Image adjustment method, apparatus, and computer device
CN112488977A (en) * 2020-12-22 2021-03-12 北京达佳互联信息技术有限公司 Image processing method and device, electronic equipment and storage medium
CN112488977B (en) * 2020-12-22 2024-06-11 北京达佳互联信息技术有限公司 Image processing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2021012599A1 (en) 2021-01-28

Similar Documents

Publication Publication Date Title
CN110555812A (en) image adjusting method and device and computer equipment
US20160328825A1 (en) Portrait deformation method and apparatus
CN106846241B (en) Image fusion method, device and equipment
CN105956997B (en) The method and apparatus of image deformation processing
WO2022001236A1 (en) Three-dimensional model generation method and apparatus, and computer device and storage medium
AU2018253460B2 (en) Framework for local parameterization of 3d meshes
US20160163083A1 (en) Real-time reconstruction of the human body and automated avatar synthesis
CN107507216A (en) The replacement method of regional area, device and storage medium in image
CN109711419A (en) Image processing method, device, computer equipment and storage medium
CN113111861A (en) Face texture feature extraction method, 3D face reconstruction method, device and storage medium
CN111951384A (en) Three-dimensional face reconstruction method and system based on single face picture
CN111951381A (en) Three-dimensional face reconstruction system based on single face picture
CN115239861A (en) Face data enhancement method and device, computer equipment and storage medium
CN105551001B (en) A kind of image rebuilding method, device and equipment
CN114424239A (en) De-noising technique for cyclic blur in ray tracing applications
CN107590858A (en) Medical sample methods of exhibiting and computer equipment, storage medium based on AR technologies
CN109035380B (en) Face modification method, device and equipment based on three-dimensional reconstruction and storage medium
JP6852224B2 (en) Sphere light field rendering method in all viewing angles
WO2018140223A1 (en) Stereo rendering
CN113223137B (en) Generation method and device of perspective projection human face point cloud image and electronic equipment
CN108876704B (en) Method and device for deforming human face image and computer storage medium
CN111275610B (en) Face aging image processing method and system
CN116109803B (en) Information construction method, device, equipment and storage medium
KR20180105210A (en) Image processing method and apparatus
CN116977539A (en) Image processing method, apparatus, computer device, storage medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20191210

RJ01 Rejection of invention patent application after publication