CN107977934B - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN107977934B CN107977934B CN201711107082.XA CN201711107082A CN107977934B CN 107977934 B CN107977934 B CN 107977934B CN 201711107082 A CN201711107082 A CN 201711107082A CN 107977934 B CN107977934 B CN 107977934B
- Authority
- CN
- China
- Prior art keywords
- offset
- grid
- vertex
- picture
- pixel point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 73
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 33
- 230000009466 transformation Effects 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims description 19
- 230000001815 facial effect Effects 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 16
- 238000005516 engineering process Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 230000003287 optical effect Effects 0.000 description 4
- 238000000638 solvent extraction Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 239000002537 cosmetic Substances 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002087 whitening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The present disclosure relates to an image processing method and apparatus. The method comprises the following steps: dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices; calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture; calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid; and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid. According to the method and the device, the face area of the picture is divided into the grids, the offset of the vertex of each grid is calculated by adopting an affine transformation algorithm, and the offset of each pixel point in each grid is calculated by adopting a bilinear interpolation algorithm, so that the calculated amount can be reduced, the image processing time can be saved, and the user experience can be improved.
Description
Technical Field
The present disclosure relates to the field of terminal technologies, and in particular, to an image processing method and apparatus.
Background
Generally, a terminal supports a function of beautifying a face in a picture or a video; the beautifying function has the functions of subverting the traditional photographing effect and instantly automatically beautifying the face, for example, the effects of face thinning, eye enlarging, whitening, three-dimensional five sense organs and the like can be realized.
Disclosure of Invention
To overcome the problems in the related art, embodiments of the present disclosure provide an image processing method and apparatus. The technical scheme is as follows:
according to a first aspect of embodiments of the present disclosure, there is provided an image processing method, including:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices;
calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture;
calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
In one embodiment, dividing the picture face region into equal sized grids comprises:
determining surrounding points in the picture; the enclosing line is composed of the enclosing points and is used for enclosing the human face characteristic points;
dividing the region enclosed by the surrounding lines in the picture into grids with equal size.
In one embodiment, calculating the offset of each vertex of the mesh by using an affine transformation algorithm according to the offset of the face feature point in the picture includes:
performing triangularization subdivision on the picture according to the face characteristic points and the surrounding points to obtain a subdivision triangle;
shifting the subdivision triangle according to the offset of the human face characteristic point;
determining the shifted subdivision triangles to which the vertexes of the mesh belong;
and respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
In one embodiment, the method further comprises:
and sequentially storing the data of each pixel point of the picture according to the grid as a unit.
In one embodiment, calculating the offset of each pixel point in the mesh by using a bilinear interpolation algorithm according to the offset of each vertex of the mesh includes:
when the offset of each vertex of the grid is smaller than a preset threshold value, determining the offset of each vertex of the grid and the offset of each pixel point in the grid to be zero; alternatively, the first and second electrodes may be,
and when the offset of at least one vertex of each vertex of the grid is greater than or equal to the preset threshold, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
In one embodiment, the grid includes a number of pixels that is a multiple of 8.
According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus including:
the dividing module is used for dividing the face area of the picture into grids with equal size; wherein each mesh comprises four vertices;
the first calculation module is used for calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture;
the second calculation module is used for calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
and the determining module is used for determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
In one embodiment, the partitioning module includes:
the first determining submodule is used for determining surrounding points in the picture; the enclosing line is composed of the enclosing points and is used for enclosing the human face characteristic points;
and the dividing submodule is used for dividing the area enclosed by the surrounding lines in the picture into grids with equal size.
In one embodiment, a first computing module includes:
the subdivision sub-module is used for carrying out triangular subdivision on the picture according to the human face characteristic points and the surrounding points to obtain a subdivision triangle;
the shifting submodule is used for shifting the subdivision triangle according to the shifting amount of the human face characteristic point;
the second determining submodule is used for determining the shifted subdivision triangles to which the vertexes of the mesh belong;
and the third determining submodule is used for respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
In one embodiment, the apparatus further comprises:
and the storage module is used for sequentially storing the data of each pixel point of the picture according to the grid as a unit.
In one embodiment, the second calculation module determines the offset of each vertex of the mesh and the offset of each pixel point in the mesh to be zero when the offsets of each vertex of the mesh are smaller than a preset threshold; and when the offset of at least one vertex of each vertex of the grid is greater than or equal to the preset threshold, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
According to a third aspect of the embodiments of the present disclosure, there is provided an image processing apparatus including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices;
calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture;
calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of the first aspect described above.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: according to the technical scheme, the image face area is divided into grids with the same size, the offset of the grid vertex is calculated by adopting an affine transformation algorithm, the offset of each pixel point in the grids is calculated by adopting a bilinear interpolation algorithm instead of the affine transformation algorithm, and compared with the related technology, the calculation amount can be reduced, the image processing time is saved, and the user experience is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow diagram illustrating an image processing method according to an exemplary embodiment.
FIG. 2 is a flow diagram illustrating an image processing method according to an exemplary embodiment.
Fig. 3a is a scene schematic diagram illustrating an image processing method according to an exemplary embodiment.
FIG. 3b is a scene schematic diagram illustrating an image processing method according to an exemplary embodiment.
Fig. 4 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 5 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 6 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 7 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 8 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 9 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Fig. 10 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Generally, a terminal, such as a smart phone, supports beauty treatment of a face in a picture or a video. In the related technology, a human face organ region in a picture is deformed according to feature point positioning; however, in the process of performing the beautifying processing on the picture, the affine transformation algorithm needs to be used to calculate the offset of each pixel in the picture after beautifying, which has a large calculation amount, is time-consuming, has low calculation efficiency, and affects user experience.
In order to solve the above problem, an embodiment of the present disclosure provides an image processing method, including: dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices; calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture; calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid; and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
According to the image processing method provided by the embodiment of the disclosure, the image face area is divided into the grids with the same size, the offset of the grid vertex is calculated by adopting the affine transformation algorithm, the offset of each pixel point in the grid is calculated by adopting the bilinear interpolation algorithm instead of adopting the affine transformation algorithm, and compared with the related technology, the calculated amount can be reduced, the image processing time is saved, and the user experience is improved.
It should be noted that, in the embodiment of the present disclosure, the terminal is, for example, a smart phone, a tablet computer, a desktop computer, a notebook computer, or a wearable device (such as a bracelet, smart glasses, and the like).
Based on the above analysis, the following specific examples are proposed.
Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment, where an executing subject of the method may be a terminal, as shown in fig. 1, the method includes the following steps 101 and 104:
in step 101, dividing a picture face region into grids with equal size; wherein each mesh comprises four vertices.
In an example, in a scene in which a face in a picture or a video is beautified, firstly, face feature points are positioned on the picture, and the face feature points and surrounding points in the picture are determined; a surrounding line composed of surrounding points for surrounding the human face feature points; the surrounding line cannot exceed the boundary of the picture, and when the face in the picture is beautified, the deformation range of the face is limited to be performed in the surrounding line; in the process of beautifying, the surrounding line is kept still, namely the offset of any point on the surrounding line is 0; determining the offset of each face characteristic point in the picture; and moving the face characteristic points to corresponding positions according to the offset of the face characteristic points.
In an example, a face key point positioning algorithm is used for carrying out face feature point positioning on a picture, and face feature points in the picture are positioned; the face keypoint localization algorithm may include, for example: active Appearance Models (AAM), supervised gradient descent methods (SDM), Convolutional Neural Networks (CNN), and the like.
For example, the area enclosed by the surrounding lines in the picture is divided into grids of equal size, for example, squares; illustratively, the grid includes a number of pixels that is a multiple of 8. For example, a single grid may include 8N pixel points, such as 8, 16, or 32, where N is a positive integer. Each grid has the same size, comprises 8 times of pixel points and can be aligned with a memory, the memory address is generally 8 times, data access is facilitated, and data access efficiency is improved.
It is common in the related art to store picture data by row or by column; however, when the offsets of the pixels of the grid need to be calculated respectively in a grid unit, in order to read the data of all the pixels of the grid from the memory, the data of the pixels of the adjacent grid is inevitably read, so that the cache resources are wasted, and the data reading efficiency is reduced. In order to solve the problem, after an area enclosed by surrounding lines in a picture is divided into grids with equal size, data of each pixel point of the picture can be sequentially stored by taking the grids as a unit; different from the method for storing the data of each pixel point of the picture in a centralized manner by taking a grid as a unit, taking an example that a single grid comprises 64 pixel points (8 rows by 8 columns of pixel points), the method comprises the steps of firstly storing the data of the 64 pixel points of the first grid, then storing the data of the 64 pixel points of the second grid until the data storage of the last grid is finished; the data of the pixel points include, for example, coordinates of the pixel points. When the offset of the pixel points of the grid is calculated, the data of the pixel points of the grid can be continuously read out due to the centralized storage of the data of the pixel points of the grid, so that the data does not need to be frequently read, the data reading is convenient, and the data reading efficiency is improved.
In step 102, the offset of each vertex of the mesh is calculated by an affine transformation algorithm according to the offset of the face feature point in the picture.
In an example, according to the characteristic points and the surrounding points of the human face, triangularly dividing a picture to obtain a division triangle; shifting the subdivision triangles according to the offset of the human face characteristic points; determining the shifted subdivision triangles to which the vertexes of the mesh belong; and respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
In step 103, the offset of each pixel point in the mesh is calculated by using a bilinear interpolation algorithm according to the offset of each vertex of the mesh.
For example, after obtaining the offset of each vertex of the mesh, respectively determining whether the offset of each vertex of the mesh is smaller than a preset threshold: when the offset of each vertex of the mesh is smaller than the preset threshold, the deformation amplitude of the face area corresponding to the mesh is little or unchanged, the face area corresponding to the mesh can not be deformed, the calculation amount can be reduced, and the image processing time can be saved; determining the offset of each vertex of the grid and the offset of each pixel point in the grid to be zero; and when the offset of at least one vertex of each vertex of the grid is greater than or equal to a preset threshold value, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
In step 104, the pixel value of each pixel point in the beautified picture is determined according to the offset of each vertex of the mesh and the offset of each pixel point in the mesh.
According to the technical scheme provided by the embodiment of the disclosure, the image face area is divided into the grids with the same size, the offset of the grid vertex is calculated by adopting an affine transformation algorithm, the offset of each pixel point in the grid is calculated by adopting a bilinear interpolation algorithm instead of adopting the affine transformation algorithm, and compared with the related technology, the calculation amount can be reduced, the image processing time is saved, and therefore, the user experience can be improved.
FIG. 2 is a flow diagram illustrating an image processing method according to an exemplary embodiment. As shown in fig. 2, on the basis of the embodiment shown in fig. 1, the image processing method according to the present disclosure may include the following steps 201 and 212:
in step 201, positioning face feature points of the picture, and determining the face feature points and surrounding points in the picture; wherein, the enclosing line formed by the enclosing points is used for enclosing the human face characteristic points.
For example, the bounding line itself cannot exceed the boundary of the picture, and when the face in the picture is beautified, the deformation range of the face is limited to be performed within the bounding line; during the cosmetic treatment, the bounding line remains stationary, i.e., the offset of any point on the bounding line is 0.
In step 202, the offset of each face feature point in the picture is determined.
In step 203, the picture is triangulated according to the face feature points and the surrounding points to obtain a subdivision triangle.
In step 204, the subdivision triangle is shifted according to the offset of the human face feature point, so as to obtain the shifted subdivision triangle.
For example, when the face feature point is moved to a position corresponding to the offset amount of the face feature point according to the offset amount of the face feature point, the vertex of the subdivision triangle corresponding to the face feature point is also offset; after the shifting of all the vertices except the surrounding points on the split triangle is completed, the shifted split triangle is obtained. Referring to the shifted split triangle shown in fig. 3a, the white thick line is a surrounding line, and the points on the surrounding line are surrounding points.
In step 205, dividing the region enclosed by the surrounding lines in the picture into grids of equal size; wherein each mesh comprises four vertices.
Referring to fig. 3b, the area enclosed by the enclosing lines in the picture is divided into equal sized grids.
Illustratively, the grid includes a number of pixels that is a multiple of 8.
In the related art, the picture data is usually stored in rows or columns, but when the offsets of the pixels of the grid need to be calculated respectively in a grid unit, in order to read the data of all the pixels of the grid from the memory, the data of the pixels of the adjacent grid is inevitably read, so that the cache resources are wasted, and the data reading efficiency is low. In order to solve the problem, the data of each pixel point of the picture can be sequentially stored by taking a grid as a unit; specifically, after dividing an area surrounded by surrounding lines in a picture into grids with equal size, recombining the storage sequence of data, and sequentially and intensively storing the data of each pixel point of the picture according to the grids as a unit; taking an example that a single grid includes 64 pixels (8 rows by 8 columns of pixels), first, data of 64 pixels of a first grid is stored, and then data of 64 pixels of a second grid is stored until the data of the last grid is stored. When the offset of the pixel points of the grid is calculated, the data of the pixel points of the grid can be continuously read out due to the centralized storage of the data of the pixel points of the grid, so that the data does not need to be frequently read, the data reading is convenient, the utilization rate of cache resources is improved, and the data reading efficiency is improved.
In step 206, the shifted split triangles to which the vertices of the mesh belong are determined.
In step 207, the offset of each vertex of the mesh is determined by an affine transformation algorithm based on the offset of the face feature point corresponding to the vertex of the split triangle after the offset.
In step 208, respectively determining whether the offset of each vertex of the mesh is smaller than a preset threshold, and turning to step 209 when the offsets of each vertex of the mesh are smaller than the preset threshold; when the offset of at least one vertex of each vertex of the mesh is greater than or equal to the preset threshold, go to step 210.
In step 209, the offset of each vertex of the mesh and the offset of each pixel point in the mesh are both determined to be zero; go to step 211.
In step 210, calculating the offset of each pixel point in the grid by using a bilinear interpolation algorithm; go to step 211.
In the related art, when the offset of each pixel needs to be calculated, the offset of the face feature point corresponding to the vertex of the split triangle after the offset is obtained first, and then the affine transformation algorithm is used to calculate the offset of each pixel point in the split triangle after the offset, which is large in calculation amount and time-consuming. When the offset of each pixel needs to be calculated, dividing an area surrounded by surrounding lines in a picture into grids with equal sizes, supposing that each grid comprises 156 pixel points (16 × 16 pixel points), calculating the offsets of four vertexes of the grid by using an affine transformation algorithm, and then respectively and approximately calculating the offset of each pixel point in the grid by using a bilinear interpolation algorithm; compared with the related art, the embodiment of the disclosure can improve the calculation speed by 256 times.
In step 211, the pixel value of each pixel point in the beautified picture is determined according to the offset of each vertex of the mesh and the offset of each pixel point in the mesh.
In step 212, the post-beauty picture is determined according to the offset of each vertex of the mesh, the offset of each pixel point in the mesh, and the pixel value of each pixel point in the post-beauty picture.
According to the technical scheme provided by the embodiment of the disclosure, the image face area is divided into the grids with the same size, only the offset of the grid vertex needs to be calculated by adopting an affine transformation algorithm, but the offset of each pixel point in the grid is calculated by adopting a bilinear interpolation algorithm, so that compared with the related technology, the calculation amount can be reduced, the image processing time can be saved, and the calculation efficiency can be improved; meanwhile, by judging whether the offset of each vertex of the mesh is smaller than a preset threshold value or not and when the offsets of all the vertices of the mesh are smaller than the preset threshold value, the deformation amplitude of the face area corresponding to the mesh is little or unchanged, the face area corresponding to the mesh can not be deformed, the calculation amount can be reduced, and the image processing time can be saved; thus, user experience can be improved.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
FIG. 4 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment; the apparatus may be implemented in various ways, for example, with all of the components of the apparatus being implemented in a terminal, or with components of the apparatus being implemented in a coupled manner on the terminal side; the apparatus may implement the method related to the present disclosure by software, hardware, or a combination of both, as shown in fig. 4, the image processing apparatus includes: a partitioning module 401, a first calculating module 402, a second calculating module 403, and a determining module 404, wherein:
the partitioning module 401 is configured to partition the picture face region into equal sized meshes; wherein each mesh comprises four vertices;
the first calculation module 402 is configured to calculate the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face feature point in the picture;
the second calculating module 403 is configured to calculate the offset of each pixel point in the mesh by using a bilinear interpolation algorithm according to the offset of each vertex of the mesh;
the determining module 404 is configured to determine a pixel value of each pixel point in the beautified picture according to the offset of each vertex of the mesh and the offset of each pixel point in the mesh.
The device provided by the embodiment of the disclosure can be used for executing the technical scheme of the embodiment shown in fig. 1, and the execution mode and the beneficial effect are similar, and are not described again here.
In one possible implementation, as shown in fig. 5, the image processing apparatus shown in fig. 4 may further include a dividing module 401 configured to include: a first determining sub-module 501 and a dividing sub-module 502, wherein:
the first determining submodule 501 is configured to determine surrounding points in the picture; wherein, the enclosing line formed by the enclosing points is used for enclosing the human face characteristic points;
the partitioning submodule 502 is configured to partition the area enclosed by the enclosing lines in the picture into equal sized grids.
In one possible implementation, as shown in fig. 6, the image processing apparatus shown in fig. 4 may further include a first calculation module 402 configured to include: a subdivision sub-module 601, an offset sub-module 602, a second determination sub-module 603, and a third determination sub-module 604, wherein:
the subdivision sub-module 601 is configured to triangulate a picture according to the human face feature points and the surrounding points to obtain a subdivision triangle;
the shift submodule 602 is configured to shift the segmentation triangle according to the shift amount of the face feature point;
the second determining submodule 603 is configured to determine the shifted subdivision triangle to which each vertex of the mesh belongs;
the third determining submodule 604 is configured to determine, according to the offset amounts of the face feature points corresponding to the vertices of the biased subdivision triangle, offset amounts of the vertices of the mesh by using an affine transformation algorithm, respectively.
In a possible implementation, as shown in fig. 7, the image processing apparatus shown in fig. 4 may further include a storage module 701 configured to sequentially store data of each pixel point of the picture in units of a grid.
In a possible implementation manner, when the offset of each vertex of the mesh is smaller than the preset threshold, the second calculation module 403 determines the offset of each vertex of the mesh and the offset of each pixel point in the mesh to be zero; the second calculating module 403 calculates the offset of each pixel point in the mesh by using a bilinear interpolation algorithm when the offset of at least one vertex of each vertex of the mesh is greater than or equal to a preset threshold.
Fig. 8 is a block diagram illustrating an image processing apparatus 800 according to an exemplary embodiment, where the image processing apparatus 800 may be implemented in various ways, such as implementing all components of the apparatus in a terminal or implementing components of the apparatus in a coupled manner on the terminal side; the image processing apparatus 800 includes:
a processor 801;
a memory 802 for storing processor-executable instructions;
wherein the processor 801 is configured to:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices;
calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture;
calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
In one embodiment, the processor 801 may be further configured to:
determining surrounding points in the picture; wherein, the enclosing line formed by the enclosing points is used for enclosing the human face characteristic points;
and dividing the area enclosed by the surrounding lines in the picture into grids with equal size.
In one embodiment, the processor 801 may be further configured to:
carrying out triangularization subdivision on the picture according to the characteristic points and the surrounding points of the human face to obtain a subdivision triangle;
shifting the subdivision triangles according to the offset of the human face characteristic points;
determining the shifted subdivision triangles to which the vertexes of the mesh belong;
and respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
In one embodiment, the processor 801 may be further configured to:
and sequentially storing the data of each pixel point of the picture according to the grid as a unit.
In one embodiment, the processor 801 may be further configured to:
when the offset of each vertex of the grid is smaller than a preset threshold, determining the offset of each vertex of the grid and the offset of each pixel point in the grid to be zero; alternatively, the first and second electrodes may be,
and when the offset of at least one vertex of each vertex of the grid is greater than or equal to a preset threshold value, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
In one embodiment, the grid includes a number of pixels that is a multiple of 8.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 9 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment; the image processing apparatus 900 is applicable to a terminal; image processing device 900 may include one or more of the following components: processing component 902, memory 904, power component 906, multimedia component 908, audio component 910, input/output (I/O) interface 912, sensor component 914, and communication component 916.
The processing component 902 generally controls the overall operation of the image processing apparatus 900, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. Processing component 902 may include one or more processors 920 to execute instructions to perform all or a portion of the steps of the methods described above. Further, processing component 902 can include one or more modules that facilitate interaction between processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.
The memory 904 is configured to store various types of data to support operations in the image processing apparatus 900. Examples of such data include instructions for any application or method operating on image processing device 900, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 904 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power component 906 provides power to the various components of the image processing device 900. The power components 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the image processing device 900.
The multimedia component 908 includes a screen that provides an output interface between the image processing apparatus 900 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front facing camera and/or a rear facing camera. When the image processing apparatus 900 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 910 is configured to output and/or input audio signals. For example, the audio component 910 includes a Microphone (MIC) configured to receive an external audio signal when the image processing apparatus 900 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 904 or transmitted via the communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.
I/O interface 912 provides an interface between processing component 902 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor component 914 includes one or more sensors for providing status evaluation of various aspects to the image processing apparatus 900. For example, the sensor assembly 914 may detect an open/closed state of the image processing apparatus 900, the relative positioning of components, such as a display and a keypad of the image processing apparatus 900, the sensor assembly 914 may also detect a change in position of the image processing apparatus 900 or a component of the image processing apparatus 900, the presence or absence of user contact with the image processing apparatus 900, orientation or acceleration/deceleration of the image processing apparatus 900, and a change in temperature of the image processing apparatus 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 916 is configured to facilitate communication between the image processing apparatus 900 and other devices in a wired or wireless manner. The image processing apparatus 900 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the image processing apparatus 900 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer readable storage medium is provided that includes instructions, such as the memory 904 including instructions, that are executable by the processor 920 of the image processing apparatus 900 to perform the above-described method. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Fig. 10 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. For example, the image processing apparatus 1000 may be provided as a server. The image processing device 1000 includes a processing component 1002 that further includes one or more processors, and memory resources, represented by memory 1003, for storing instructions, such as application programs, that are executable by the processing component 1002. The application programs stored in memory 1003 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1002 is configured to execute instructions to perform the above-described methods.
The image processing apparatus 1000 may further include a power supply component 1006 configured to perform power management of the image processing apparatus 1000, a wired or wireless network interface 1005 configured to connect the image processing apparatus 1000 to a network, and an input/output (I/O) interface 1008. The image processing apparatus 1000 may operate based on an operating system stored in the memory 1003, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
A non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an image processing apparatus 900 or an image processing apparatus 1000, enabling the image processing apparatus 900 or the image processing apparatus 1000 to perform an image processing method comprising:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices; calculating the offset of each vertex of the grid by adopting an affine transformation algorithm according to the offset of the human face characteristic points in the picture; calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid; and determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (11)
1. An image processing method, comprising:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices;
calculating the offset of each vertex of the grid corresponding to the face feature point by adopting an affine transformation algorithm according to the offset of the face feature point in the picture;
calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid;
the dividing of the picture face area into equal-sized grids comprises:
determining surrounding points in the picture; the enclosing line is composed of the enclosing points and is used for enclosing the human face characteristic points;
dividing the region enclosed by the surrounding lines in the picture into grids with equal size.
2. The method according to claim 1, wherein calculating the offset of each vertex of the mesh corresponding to the facial feature point by using an affine transformation algorithm according to the offset of the facial feature point in the picture comprises:
performing triangularization subdivision on the picture according to the face characteristic points and the surrounding points to obtain a subdivision triangle;
shifting the subdivision triangle according to the offset of the human face characteristic point;
determining the shifted subdivision triangles to which the vertexes of the mesh belong;
and respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
3. The method of claim 1, further comprising:
and sequentially storing the data of each pixel point of the picture according to the grid as a unit.
4. The method of claim 1, wherein calculating the offset of each pixel point in the mesh by using a bilinear interpolation algorithm according to the offset of each vertex of the mesh comprises:
when the offset of each vertex of the grid is smaller than a preset threshold value, determining the offset of each vertex of the grid and the offset of each pixel point in the grid to be zero; alternatively, the first and second electrodes may be,
and when the offset of at least one vertex of each vertex of the grid is greater than or equal to the preset threshold, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
5. The method of claim 1, wherein the grid comprises a number of pixels that is a multiple of 8.
6. An image processing apparatus characterized by comprising:
the dividing module is used for dividing the face area of the picture into grids with equal size; wherein each mesh comprises four vertices;
the first calculation module is used for calculating the offset of each vertex of the grid corresponding to the face feature point by adopting an affine transformation algorithm according to the offset of the face feature point in the picture;
the second calculation module is used for calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
the determining module is used for determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid;
the dividing module comprises:
the first determining submodule is used for determining surrounding points in the picture; the enclosing line is composed of the enclosing points and is used for enclosing the human face characteristic points;
and the dividing submodule is used for dividing the area enclosed by the surrounding lines in the picture into grids with equal size.
7. The apparatus of claim 6, wherein the first computing module comprises:
the subdivision sub-module is used for carrying out triangular subdivision on the picture according to the human face characteristic points and the surrounding points to obtain a subdivision triangle;
the shifting submodule is used for shifting the subdivision triangle according to the shifting amount of the human face characteristic point;
the second determining submodule is used for determining the shifted subdivision triangles to which the vertexes of the mesh belong;
and the third determining submodule is used for respectively determining the offset of each vertex of the mesh by adopting an affine transformation algorithm according to the offset of the human face characteristic point corresponding to the vertex of the split triangle after the offset.
8. The apparatus of claim 6, further comprising:
and the storage module is used for sequentially storing the data of each pixel point of the picture according to the grid as a unit.
9. The apparatus according to claim 6, wherein the second calculation module determines the offset of each vertex of the mesh and the offset of each pixel point in the mesh to be zero when the offset of each vertex of the mesh is smaller than a preset threshold; and when the offset of at least one vertex of each vertex of the grid is greater than or equal to the preset threshold, calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm.
10. An image processing apparatus characterized by comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
dividing the picture face area into grids with equal size; wherein each mesh comprises four vertices;
calculating the offset of each vertex of the grid corresponding to the face feature point by adopting an affine transformation algorithm according to the offset of the face feature point in the picture;
calculating the offset of each pixel point in the grid by adopting a bilinear interpolation algorithm according to the offset of each vertex of the grid;
determining the pixel value of each pixel point in the beautified picture according to the offset of each vertex of the grid and the offset of each pixel point in the grid;
the dividing of the picture face area into equal-sized grids comprises:
determining surrounding points in the picture; the enclosing line is composed of the enclosing points and is used for enclosing the human face characteristic points;
dividing the region enclosed by the surrounding lines in the picture into grids with equal size.
11. A computer-readable storage medium having stored thereon computer instructions, which, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711107082.XA CN107977934B (en) | 2017-11-10 | 2017-11-10 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711107082.XA CN107977934B (en) | 2017-11-10 | 2017-11-10 | Image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107977934A CN107977934A (en) | 2018-05-01 |
| CN107977934B true CN107977934B (en) | 2022-04-01 |
Family
ID=62013285
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711107082.XA Active CN107977934B (en) | 2017-11-10 | 2017-11-10 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107977934B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108921798B (en) * | 2018-06-14 | 2021-06-22 | 北京微播视界科技有限公司 | Image processing method and device and electronic equipment |
| CN109063560B (en) * | 2018-06-28 | 2022-04-05 | 北京微播视界科技有限公司 | Image processing method, image processing device, computer-readable storage medium and terminal |
| CN109741277A (en) * | 2018-12-29 | 2019-05-10 | 广州华多网络科技有限公司 | Image processing method, device, storage medium and server |
| CN110009559B (en) * | 2019-03-19 | 2023-07-07 | 北京迈格威科技有限公司 | Image processing method and device |
| CN114095646B (en) * | 2020-08-24 | 2022-08-26 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN114296622B (en) * | 2020-09-23 | 2023-08-08 | 北京达佳互联信息技术有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN113222844B (en) * | 2021-05-14 | 2023-11-24 | 上海绚显科技有限公司 | Image beautifying method and device, electronic equipment and medium |
| CN114862663B (en) * | 2022-05-17 | 2023-01-20 | 浙江省测绘科学技术研究院 | Dynamic adjustment method for grid tile data precision |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011210054A (en) * | 2010-03-30 | 2011-10-20 | Secom Co Ltd | Object detection device and learning device for the same |
| CN103824253A (en) * | 2014-02-19 | 2014-05-28 | 中山大学 | Figure five sense organ deformation method based on image local precise deformation |
| CN106648344A (en) * | 2015-11-02 | 2017-05-10 | 重庆邮电大学 | Screen content adjustment method and equipment |
| CN107330868A (en) * | 2017-06-26 | 2017-11-07 | 北京小米移动软件有限公司 | image processing method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5463866B2 (en) * | 2009-11-16 | 2014-04-09 | ソニー株式会社 | Image processing apparatus, image processing method, and program |
-
2017
- 2017-11-10 CN CN201711107082.XA patent/CN107977934B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011210054A (en) * | 2010-03-30 | 2011-10-20 | Secom Co Ltd | Object detection device and learning device for the same |
| CN103824253A (en) * | 2014-02-19 | 2014-05-28 | 中山大学 | Figure five sense organ deformation method based on image local precise deformation |
| CN106648344A (en) * | 2015-11-02 | 2017-05-10 | 重庆邮电大学 | Screen content adjustment method and equipment |
| CN107330868A (en) * | 2017-06-26 | 2017-11-07 | 北京小米移动软件有限公司 | image processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107977934A (en) | 2018-05-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107977934B (en) | Image processing method and device | |
| CN107680033B (en) | Picture processing method and device | |
| CN107818543B (en) | Image processing method and device | |
| CN107330868B (en) | Picture processing method and device | |
| CN107958439B (en) | Image processing method and device | |
| CN107657590B (en) | Picture processing method and device and storage medium | |
| CN107341777B (en) | Picture processing method and device | |
| EP3125158A2 (en) | Method and device for displaying images | |
| US9959484B2 (en) | Method and apparatus for generating image filter | |
| CN109087238B (en) | Image processing method and apparatus, electronic device, and computer-readable storage medium | |
| CN107464253B (en) | Eyebrow positioning method and device | |
| CN108470322B (en) | Method and device for processing face image and readable storage medium | |
| US11030733B2 (en) | Method, electronic device and storage medium for processing image | |
| CN107967459B (en) | Convolution processing method, convolution processing device and storage medium | |
| US20200312022A1 (en) | Method and device for processing image, and storage medium | |
| WO2015196715A1 (en) | Image retargeting method and device and terminal | |
| CN109784327B (en) | Boundary box determining method and device, electronic equipment and storage medium | |
| CN112767288A (en) | Image processing method and device, electronic equipment and storage medium | |
| US9665925B2 (en) | Method and terminal device for retargeting images | |
| CN108596957B (en) | Object tracking method and device | |
| CN107730443B (en) | Image processing method and device and user equipment | |
| CN110728621A (en) | Face changing method and device for face image, electronic equipment and storage medium | |
| CN107239758B (en) | Method and device for positioning key points of human face | |
| CN113920083A (en) | Image-based size measurement method and device, electronic equipment and storage medium | |
| CN107292901B (en) | Edge detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |