CN109685740B

CN109685740B - Face correction method and device, mobile terminal and computer readable storage medium

Info

Publication number: CN109685740B
Application number: CN201811591198.XA
Authority: CN
Inventors: 徐爱辉
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2018-12-25
Filing date: 2018-12-25
Publication date: 2023-08-11
Anticipated expiration: 2038-12-25
Also published as: CN109685740A

Abstract

The invention discloses a face correction method and device, a mobile terminal and a computer readable storage medium, and belongs to the field of mobile terminals. Comprising the following steps: obtaining face key point information obtained from a face key point detection stage, and obtaining a mapping transformation matrix map; transforming the image coordinates to be corrected through the map transformation matrix map to generate coordinates Warp corresponding to the new face image; re-determining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map; and correcting the new face image according to the new face frame bbox_new to obtain a corrected face image. According to the embodiment of the invention, the information of the relative positions of the facial features can be reserved, the separability of different facial features is improved, and the facial features of the same facial features can be well reserved at the same positions, so that the key parts of the facial features participating in the verification and the registration are kept at the same positions, the identification of the same facial features is simpler, and the user experience is improved.

Description

Face correction method and device, mobile terminal and computer readable storage medium

Technical Field

The present invention relates to the field of mobile terminals, and in particular, to a method and apparatus for face correction, a mobile terminal, and a computer readable storage medium.

Background

Currently, mobile terminals are increasingly popular, and users using the mobile terminals are more and more frequent, so that the mobile terminals are one of the indispensable mobile devices for the users.

The unlocking of the mobile terminal is performed through the relevant unlocking technologies such as sliding unlocking, password unlocking, gesture unlocking, fingerprint unlocking and the like, and a new unlocking technology is also developed currently, namely face unlocking, the face image of the user is obtained by utilizing the camera of the mobile terminal and is compared with the relevant characteristics of the face of the user which are input in advance, so that the purpose of unlocking the mobile terminal is achieved.

Face unlocking is a systematic scheme, and comprises face detection, face key point detection, face correction, face feature extraction, face verification comparison and the like. There are various technical studies on face correction, for example, a method of correcting the five sense organs of a face to the same position of an image by transformation using a bbox method. However, these face correction methods have the following problems in the face correction stage:

(1) According to the bbox correction method, after the face is rotated according to the bbox provided by the face detection, the face is scaled to the same uniform size (for example, the face is scaled to a square), so that the relative position information of the facial features of the face is changed, for example, a person with a long face type is changed into a face type, and erroneous recognition is easily caused.

(2) The bbox correction method of (1) above may make face correction not conducive to face unlocking in some cases. As in face images (a) and (b) of fig. 3, the five sense organs between the same person have been staggered after correction, which can lead to failure of face unlocking, especially in low light conditions, the disadvantages of the above method are very apparent.

(3) The correction method for correcting the facial features of the face to the same position of the image through transformation, such as face images (c) and (d) of fig. 3, that is, eyes, nose and mouth of different people are on the same coordinates of the image, can improve the error rate of face recognition.

Disclosure of Invention

In view of this, the method and apparatus for face correction, the mobile terminal and the computer readable storage medium provided by the embodiments of the present invention can retain information of relative positions between facial features of faces, and can well retain the same facial features of the same face in the same position while increasing the separability between different faces, so that key parts of the faces participating in the verification and the registration are retained in the same position, thereby making the identification of the same face simpler and improving the user experience.

The technical scheme adopted by the invention for solving the technical problems is as follows:

according to one aspect of the present invention, a face correction method is provided, which is applied to a mobile terminal having a face unlocking function, and includes:

acquiring a mapping transformation matrix map according to the face key point information obtained from the face key point detection stage;

transforming the image coordinates to be corrected through the map transformation matrix map to generate coordinates Warp corresponding to the new face image;

re-determining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map;

and correcting the new face image according to the new face frame bbox_new to obtain a corrected face image.

In one possible design, before the step of obtaining the mapping transformation matrix map according to the face key point information obtained from the face key point detection stage, the method further includes: and receiving the face key point information provided in the face key point detection stage, wherein the face key point information comprises face key point positions landmark and a face frame bbox.

In one possible design, the face keypoint locations landmark includes coordinate locations of 5 face keypoints: left eye coordinates landpack [0], right eye coordinates landpack [1], nose coordinates landpack [2], left mouth corner coordinates landpack [3] and right mouth corner coordinates landpack [4]; the face frame bbox records the information of the coordinates, width and height of the upper left corner of the face frame.

In one possible design, the step of obtaining the map matrix map according to the face key point information obtained from the face key point detection stage includes:

setting the distance between eyes of a standard face;

according to the coordinate position landmark of the face key points provided in the face key point detection stage, calculating the distance xDIs between eyes in the horizontal direction and the distance yddis between eyes in the vertical direction;

according to the distance xDIs in the horizontal direction and the distance yDis in the vertical direction between eyes, calculating a face rotation angle g_angle;

according to the face frame bbox information provided in the face key point detection stage, calculating the center coordinates of the face position;

calculating a face scaling factor Scale according to the distance g_norm dis between eyes of the standard face and the distance xDis between eyes in the horizontal direction and the distance yddis between eyes in the vertical direction;

and obtaining a mapping transformation matrix map according to the face position center coordinates, the face rotation angle g_angle and the face scaling factor Scale.

In one possible design, the coordinates of the image to be corrected are transformed by the map transformation matrix map to generate coordinates Warp corresponding to the new face image; comprising the following steps: the image coordinates to be corrected are subjected to map transformation by the mapping transformation matrix by adopting the following formula to generate coordinates Warp corresponding to the new face image:

Warp＝cv2.warpaffine(img,map,(img.shape[1],img.shape[0]))

Wherein, cv2.warp is the image coordinate, which is transformed by mapping transformation matrix map to generate new face image coordinate, img is the image to be corrected, (img. Shape [1], img. Shape [0 ]) is the size of the new face image, img. Shape [1] is the width of the new face image, img. Shape [0] is the height of the new face image.

In one possible design, the step of redefining the new key point position landmark_new and the new face box bbox_new according to the mapping transformation matrix map includes:

calculating a new key point position landmark_new according to the mapping transformation matrix map;

recalculating the distance dist_eyeeye between eyes and the distance dist_eyeomouth between eyes and mouth according to the new key point position landmark_new;

setting a face clipping position according to the recalculated distance dist_eyeeye between eyes and the distance dist_eyeomouth between eyes and a mouth;

and recalculating according to the new key point position landmark_new and the face clipping position to obtain a new face frame bbox_new.

In one possible design, the step of correcting the new face image according to the new face box bbox_new to obtain a corrected face image includes:

Cutting the new face image according to the new face frame bbox_new to obtain a corrected face image Ret, wherein the corrected face image Ret is as follows:

Ret＝warp[bbox_new[1]:bbox_new[3],bbox_new[0]:bbox_new[2]]

wherein Ret is a face image after correction obtained after clipping, warp represents clipping the new face image, bbox_new [1]: bbox_new [3] represents clipping from bbox_new [1] to bbox_new [3] on X horizontal axis; bbox_new [0]: bbox_new [2] denotes clipping on the Y vertical axis starting from bbox_new [0] to bbox_new [2] ending.

According to another aspect of the present invention, there is provided a face correction apparatus applied to a mobile terminal having a face unlocking function, comprising: the device comprises an acquisition module, a transformation module, a determination module and a correction module; wherein:

the acquisition module is used for acquiring a mapping transformation matrix map according to the face key point information acquired from the face key point detection stage;

the transformation module is used for generating a coordinate Warp corresponding to the new face image after the image coordinate to be corrected is transformed by the mapping transformation matrix map;

the determining module is used for redefining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map;

the correction module is configured to correct the new face image according to the new face frame bbox_new, and obtain a corrected face image.

According to another aspect of the present invention, there is provided a mobile terminal including: the face correction method comprises the steps of a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the computer program is executed by the processor to realize the face correction method provided by the embodiment of the invention.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored thereon a program of a face correction method, which when executed by a processor, implements the steps of the face correction method provided by the embodiment of the present invention.

Compared with the related art, the face correction method, the face correction device, the mobile terminal and the computer readable storage medium provided by the embodiment of the invention acquire the mapping transformation matrix map through the face key point information acquired from the face key point detection stage, and generate the coordinate Warp corresponding to the new face image after the image coordinate to be corrected is transformed by the mapping transformation matrix map; re-determining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map; and correcting the new face image according to the new face frame bbox_new to obtain a corrected face image. In this way, the information of the relative positions of the facial features of the faces can be reserved, the separability of different faces is improved, and the facial features of the same face can be well reserved at the same position, so that the key parts of the faces participating in the verification and the registration (recording) are kept at the same position, and the identification of the same face is simpler; in addition, the size of the face to be cut is normalized to a uniform size (for example, 160×160), and the scaling operation is not needed, so that the user experience is improved.

Drawings

Fig. 1 is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention;

fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention;

FIG. 3 is a graph of face image contrast before and after correction using a related face correction method;

fig. 4 is a flowchart of a face correction method according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a coordinate position landmark and a face frame bbox of a face key point provided in an embodiment of the present invention;

fig. 6 is a schematic diagram of a coordinate position landmark and a face frame bbox of a face key point determined by recalculation according to an embodiment of the present invention;

fig. 7 is a schematic diagram of setting a face clipping position according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a corrected face image according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a face correction device according to an embodiment of the present invention;

fig. 10 is a flowchart of a face correction method according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

In order to make the technical problems, technical schemes and beneficial effects to be solved more clear and obvious, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the particular embodiments described herein are illustrative only and are not limiting upon the invention.

In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present invention, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the foregoing drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.

The terminal may be implemented in various forms. For example, the terminals described in the present invention may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs, desktop computers, and the like.

The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention, the mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an a/V (audio/video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will appreciate that the mobile terminal structure shown in fig. 1 is not limiting of the mobile terminal and that the mobile terminal may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

The following describes the components of the mobile terminal in detail with reference to fig. 1:

the radio frequency unit 101 may be used for receiving and transmitting signals during the information receiving or communication process, specifically, after receiving downlink information of the base station, processing the downlink information by the processor 110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication, global System for Mobile communications), GPRS (General Packet Radio Service ), CDMA2000 (Code Division Multiple Access, CDMA 2000), WCDMA (Wideband Code Division Multiple Access ), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access, time Division synchronous code Division multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, frequency Division Duplex Long term evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution, time Division Duplex Long term evolution), etc.

WiFi belongs to a short-distance wireless transmission technology, and a mobile terminal can help a user to send and receive e-mails, browse web pages, access streaming media and the like through the WiFi module 102, so that wireless broadband Internet access is provided for the user. Although fig. 1 shows a WiFi module 102, it is understood that it does not belong to the necessary constitution of a mobile terminal, and can be omitted entirely as required within a range that does not change the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the mobile terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is used to receive an audio or video signal. The a/V input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, the graphics processor 1041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting the audio signal.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; as for other sensors such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured in the mobile phone, the detailed description thereof will be omitted.

The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 107 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile terminal. In particular, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1071 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 110, and can receive and execute commands sent from the processor 110. Further, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc., as specifically not limited herein.

Further, the touch panel 1071 may overlay the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or thereabout, the touch panel 1071 is transferred to the processor 110 to determine the type of touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components for implementing the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 may be integrated with the display panel 1061 to implement the input and output functions of the mobile terminal, which is not limited herein.

The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

Memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by running or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The mobile terminal 100 may further include a power source 111 (e.g., a battery) for supplying power to the respective components, and preferably, the power source 111 may be logically connected to the processor 110 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described herein.

In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based will be described below.

Referring to fig. 2, fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention, where the communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an e-UTRAN (Evolved UMTS Terrestrial Radio Access Network ) 202, an epc (Evolved Packet Core, evolved packet core) 203, and an IP service 204 of an operator that are sequentially connected in communication.

Specifically, the UE201 may be the terminal 100 described above, and will not be described herein.

The E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. The eNodeB2021 may be connected with other eNodeB2022 by a backhaul (e.g., an X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide access from the UE201 to the EPC 203.

EPC203 may include MME (Mobility Management Entity ) 2031, hss (Home Subscriber Server, home subscriber server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, pgw (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function, policy and tariff function entity) 2036, and so on. The MME2031 is a control node that handles signaling between the UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location registers (not shown) and to hold user specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034 and PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flows and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).

IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem ), or other IP services, etc.

Although the LTE system is described above as an example, it should be understood by those skilled in the art that the present invention is not limited to LTE systems, but may be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.

Based on the above mobile terminal hardware structure and the communication network system, various embodiments of the method of the present invention are provided.

Currently, a face unlocking technology is adopted for unlocking the mobile terminal, and the principle is that a camera of the mobile terminal is utilized to acquire a face image of a user, and the face image is compared with the face related features of the user which are input in advance, so that the purpose of unlocking the mobile terminal is achieved.

Face unlocking is a systematic scheme and comprises several stages of face detection, face key point detection, face correction, face feature extraction, face verification and the like. Face unlocking is divided into two processes:

1. face entry process

The face input process generally occurs in the first time of face unlocking, including face detection, face key point detection, face correction, face feature extraction and preservation, and the like.

2. Face verification process

The face verification process generally occurs when a mobile terminal is unlocked by using face unlocking, and comprises several stages of face detection, face key point detection, face correction, face feature extraction and face verification comparison.

The invention mainly provides a face correction method aiming at the face correction stage of face unlocking, and aims to ensure that key parts of the face to be verified and the registered (recorded) face are kept at the same position, so that the identification of the same face is simpler, and the user experience is improved.

Please refer to fig. 4. The embodiment of the invention provides a face correction method, which is applied to a mobile terminal with a face unlocking function and comprises the following steps:

step S1, acquiring a mapping transformation matrix map according to face key point information obtained from a face key point detection stage;

s2, generating a coordinate Warp corresponding to a new face image after map transformation of the image coordinate to be corrected by the map transformation matrix;

s3, redetermining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map;

and S4, correcting the new face image according to the new face frame bbox_new to obtain a corrected face image.

Further, in the step S1, before the mapping transformation matrix map is obtained according to the face key point information obtained from the face key point detection stage, the method further includes:

Receiving face key point information provided in a face key point detection stage, wherein the face key point information comprises face key point positions landmark and a face frame bbox; comprising the following steps:

and receiving face key point information provided in a face key point detection stage, wherein the face key point information comprises the coordinate positions landmark and face frames bbox of face key points including eyes, noses and mouths, wherein the coordinate positions landmark store the coordinate positions of 5 face key points including left eye coordinates landmark [0], right eye coordinates landmark [1], nose coordinates landmark [2], left mouth corner coordinates landmark [3] and right mouth corner coordinates landmark [4], and the face frames bbox record the information of the coordinates, width and height of the upper left corner of the face frames. As shown in fig. 5.

Further, in the step S1, a mapping transformation matrix map is obtained according to the face key point information obtained from the face key point detection stage; comprising the following steps:

step S11, setting a distance g_norm Dis between eyes of a standard face;

step S12, providing the coordinate positions landmark of the face key points including eyes, nose and mouth according to the face key point detection stage, and calculating the distance xDis in the horizontal direction and the distance yddis in the vertical direction between the eyes as follows:

xDis＝landmark[1][0]-landmark[0][0]

yDis＝landmark[1][1]-landmark[0][1]

Wherein, landmark [1] [0] represents the abscissa of the right eye, landmark [0] [0] represents the abscissa of the left eye, landmark [1] [1] represents the ordinate of the right eye, and landmark [0] [1] represents the ordinate of the left eye.

Step S13, calculating a face rotation angle g_angle according to the distance xDIs between eyes in the horizontal direction and the distance yDis between eyes in the vertical direction:

g_angle＝cv2.faceAtan2(yDis,xDis)。

step S14, calculating the Center coordinates (center_x, center_y) of the face position according to the face frame bbox information provided in the face key point detection stage, as follows:

Center_x＝bbox[0]+bbox[2]/2.0

Center_y＝bbox[1]+bbox[3]/2.0

wherein bbox [0] represents the abscissa of the bbox coordinate, bbox [1] represents the ordinate of the bbox coordinate, bbox [2] represents the width of the bbox, and bbox [3] represents the height of the bbox.

Step S15, calculating a face scaling factor Scale according to the distance g_norm Dis between the eyes of the standard face and the distance xDIs in the horizontal direction and the distance yDis in the vertical direction between the eyes:

Scale＝g_normDis/np.sqrt(xDis*xDis+yDis*yDis)

where np.sqrt (x) is the square root of the computation array x.

And S16, obtaining a mapping transformation matrix map according to the face position center coordinates, the face rotation angle g_angle and the face scaling factor Scale.

map＝cv2.getRotatinMatrix2D((Center_x,Center_y),g_angle,scale)

Wherein, the liquid crystal display device comprises a liquid crystal display device,

the cv2.Getrotatin matrix2d is a spatial transformation of image coordinates, in order to obtain a spatial transformation 2x 3-dimensional mapping transformation matrix map.

Further, in the step S2, coordinates Warp corresponding to the new face image are generated after the coordinates of the image to be corrected are map-transformed by the map transformation matrix; comprising the following steps:

the image coordinates to be corrected are subjected to map transformation by the mapping transformation matrix by adopting the following formula to generate coordinates Warp corresponding to the new face image:

Warp＝cv2.warpaffine(img,map,(img.shape[1],img.shape[0]))

wherein, cv2.warp affine is the new face image coordinate generated after the image coordinate is mapped by the mapping transformation matrix map, realizing the affine transformation of the coordinate system; wherein img is the image to be corrected, (img. Shape [1], img. Shape [0 ]) is the size of the new face image, img. Shape [1] is the width of the new face image, img. Shape [0] is the height of the new face image.

Further, in the step S3, the new key point position landmark_new and the new face frame bbox_new are redetermined according to the map transformation matrix map; as shown in fig. 6, includes:

step S31, calculating a new key point position landmark_new according to the mapping transformation matrix map; comprising the following steps:

the following formulas are used to calculate the left eye coordinates landmark_new [0], the right eye coordinates landmark_new [1], the nose coordinates landmark_new [2], the left mouth corner coordinates landmark_new [3], and the right mouth corner coordinates landmark_new [4] respectively, and a new key point landmark_new is obtained:

The formula for calculating the left eye coordinate landmark_new [0] is as follows:

landmark_new [0] [0] = map [0] [0] +map [0] [1] +map [0] [2] which is the abscissa in the left eye coordinate

landmark_new [0] [1] =map [1] [0] +map [1] [1] +map [1] [2] which is the ordinate in the left eye coordinate

The formula for calculating the right eye coordinates landmark_new [1] is as follows:

landmark_new [1] [0] =map [0] [ landmark [1] +map [0] [1] [ landmark [1] +map [0] [2], which is the abscissa in the right eye coordinate

landmark_new [1] [1] = map [1] [0] + map [1] [1] + landmark [1] + map [1] [2], which is the abscissa in the right eye coordinates

The formula for calculating the nose coordinates landmark_new [2] is as follows:

landmark_new [2] [0] =map [0] [0] +landmark [2] +map [0] [1] +landmark [2] +map [0] [2], which is the abscissa in the nasal coordinates

Landmark_new [2] [1] =map [1] [0] +map [2] +map [1] [1] +landmark [2] +map [1] [2], which is the abscissa in the nasal coordinates

The formula for calculating the left mouth angular position landmark_new [3] is as follows:

landmark_new [3] [0] =map [0] [3] +map [0] [1] [3] +map [0] [2] which is the abscissa in the left-hand angular position of the mouth

landmark_new [3] [1] =map [1] [0] +landmark [3] +map [1] [3] +map [1] [2], which is the abscissa in the left-hand angular position of the mouth

The formula for calculating the right mouth angular position landmark_new [4] is as follows:

landmark_new [4] [0] =map [0] [4] +map [0] [1] [4] +map [0] [2] that is the abscissa in right mouth angular position

landmark_new [4] [1] =map [1] [0] +map [4] +map [1] [4] +map [1] [2], which is the abscissa in the right mouth angular coordinate.

Step S32, recalculating the distance dist_eyespace between eyes and the distance dist_eyeshade between eyes and mouth according to the new key point position landmark_new:

dist_eyetoeye＝landmark_new[1][0]-landmark_new[0][0]

dist_eyetomouth＝landmark_new[3][1]-landmark_new[0][1]

dist_eyetoeye＝landmark_new[1][0]-landmark_new[0][0]

dist_eyetomouth＝landmark_new[3][1]-landmark_new[0][1]

wherein, landmark_new [1] [0] represents the abscissa in the right eye coordinate, landmark_new [0] [0] represents the abscissa in the left eye coordinate, landmark_new [3] [1] represents the abscissa in the left mouth corner coordinate, and landmark_new [0] [1] represents the ordinate in the left eye coordinate.

Step S33, setting a face clipping position x_cut, y_cut_up and y_cut_down according to the recalculated distance dist_eyeeye between eyes and the distance dist_eyepath between eyes and a mouth. As shown in fig. 7. The specific calculation formula is as follows:

x_cut＝(x-dist_eyetoeye)/2.0

y_cut_up＝y

y_cut_down＝h-y_cut_up-dist_eyetomouth。

And step S34, recalculating to obtain a new face frame bbox_new according to the new key point position landmark_new and the face clipping position. As shown in fig. 7.

bbox_new[0]＝landmark_new[0][0]-x_cut

bbox_new[1]＝landmark_new[0][1]-y_cut_up

bbox_new[2]＝min(landmark_new[1][0]+x_cut,img.shape[1])

bbox_new[3]＝min(landmark_new[3][1]+y_cut_down,img.shape[0])

Wherein bbox_new [0] represents the abscissa of the bbox_new coordinate, bbox_new [1] represents the ordinate of the bbox_new coordinate, bbox_new [2] represents the width of bbox_new, and bbox_new [3] represents the height of bbox_new.

Further, in the step S4, the new face image is corrected according to the new face frame bbox_new, and a corrected face image is obtained; comprising the following steps:

and cutting the new face image according to the new face frame bbox_new to obtain a corrected face image Ret as follows.

Ret＝warp[bbox_new[1]:bbox_new[3],bbox_new[0]:bbox_new[2]]

Wherein Ret is a face image after correction obtained after clipping, as shown in FIG. 8, warp represents clipping of a new face image, bbox_new [1]: bbox_new [3] represents clipping starting from bbox_new [1] to bbox_new [3] on the X horizontal axis; bbox_new [0]: bbox_new [2] denotes clipping on the Y vertical axis starting from bbox_new [0] to bbox_new [2] ending.

The face correction method is applied to a mobile terminal with a face unlocking function, and a mapping transformation matrix map is obtained through face key point information obtained from a face key point detection stage, and coordinates of an image to be corrected are transformed through the mapping transformation matrix map to generate coordinates Warp corresponding to a new face image; re-determining new key point positions landmark_new and new face frames bbox_new according to the mapping transformation matrix map; and correcting the new face image according to the new face frame bbox_new to obtain a corrected face image. In this way, the information of the relative positions of the facial features of the faces can be reserved, the separability of different faces is improved, and the facial features of the same face can be well reserved at the same position, so that the key parts of the faces participating in the verification and the registration (recording) are kept at the same position, and the identification of the same face is simpler; in addition, the size of the face to be cut is normalized to a uniform size (for example, 160×160), and the scaling operation is not needed, so that the user experience is improved.

Please refer to fig. 9. The embodiment of the invention provides a face correction device which is applied to a mobile terminal with a face unlocking function and comprises the following components: the system comprises an acquisition module 20, a transformation module 30, a determination module 40 and a correction module 50; wherein:

the acquiring module 20 is configured to acquire a mapping transformation matrix map according to the face key point information acquired from the face key point detection stage;

the transformation module 30 is configured to transform the coordinates of the image to be corrected by the map transformation matrix map to generate coordinates Warp corresponding to the new face image;

the determining module 40 is configured to redetermine a new key point position landmark_new and a new face frame bbox_new according to the map transformation matrix map;

the correction module 50 is configured to correct the new face image according to the new face frame bbox_new, and obtain a corrected face image.

Further, the apparatus further comprises: the receiving module 10 is configured to receive face key point information provided in a face key point detection stage, where the face key point information includes a face key point position landmark and a face frame bbox; comprising the following steps:

and receiving face key point information provided in a face key point detection stage, wherein the face key point information comprises the coordinate positions landmark and face frames bbox of face key points including eyes, noses and mouths, wherein the coordinate positions landmark store the coordinate positions of 5 face key points including left eye coordinates landmark [0], right eye coordinates landmark [1], nose coordinates landmark [2], left mouth corner coordinates landmark [3] and right mouth corner coordinates landmark [4], and the face frames bbox record the information of the coordinates, width and height of the upper left corner of the face frames.

Further, the acquiring module 20 is specifically configured to:

s11, setting a distance g_norm Dis between eyes of a standard face;

s12, providing the coordinate positions landmark of the face key points including eyes, nose and mouth according to the face key point detection stage, and calculating the distance xDIs between eyes in the horizontal direction and the distance yddis in the vertical direction as follows:

xDis＝landmark[1][0]-landmark[0][0]

yDis＝landmark[1][1]-landmark[0][1]

S13, calculating a face rotation angle g_angle according to the distance xDIs between eyes in the horizontal direction and the distance yDis between eyes in the vertical direction:

g_angle＝cv2.faceAtan2(yDis,xDis)。

s14, calculating the Center coordinates (center_x, center_y) of the face position according to the face frame bbox information provided in the face key point detection stage, wherein the center_x and center_y are as follows:

Center_x＝bbox[0]+bbox[2]/2.0

Center_y＝bbox[1]+bbox[3]/2.0

S15, calculating a face scaling factor Scale according to the distance g_norm Dis between the eyes of the standard face and the distance xDIs in the horizontal direction and the distance yDis in the vertical direction between the eyes:

Scale＝g_normDis/np.sqrt(xDis*xDis+yDis*yDis)

Where np.sqrt (x) is the square root of the computation array x.

S16, obtaining a mapping transformation matrix map according to the face position center coordinates, the face rotation angle g_angle and the face scaling factor Scale.

map＝cv2.getRotatinMatrix2D((Center_x,Center_y),g_angle,scale)

The transformation module 30 is specifically configured to: the image coordinates to be corrected are subjected to map transformation by the mapping transformation matrix by adopting the following formula to generate coordinates Warp corresponding to the new face image:

Warp＝cv2.warpaffine(img,map,(img.shape[1],img.shape[0]))

Further, the determining module 40 is specifically configured to:

s31, calculating a new key point position landmark_new according to the mapping transformation matrix map; comprising the following steps:

S32, recalculating the distance dist_eyespace between eyes and the distance dist_eyepath between eyes and mouth according to the new key point position landmark_new:

dist_eyetoeye＝landmark_new[1][0]-landmark_new[0][0]

dist_eyetomouth＝landmark_new[3][1]-landmark_new[0][1]

dist_eyetoeye＝landmark_new[1][0]-landmark_new[0][0]

dist_eyetomouth＝landmark_new[3][1]-landmark_new[0][1]

S33, setting a face clipping position x_cut, y_cut_up and y_cut_down according to the recalculation distance dist_eyeeye between eyes and the distance dist_eyeomouth between eyes and a mouth. The specific calculation formula is as follows:

x_cut＝(x-dist_eyetoeye)/2.0

y_cut_up＝y

y_cut_down＝h-y_cut_up-dist_eyetomouth。

s34, recalculating the new face frame bbox_new according to the new key point position landmark_new and the face clipping position.

bbox_new[0]＝landmark_new[0][0]-x_cut

bbox_new[1]＝landmark_new[0][1]-y_cut_up

bbox_new[2]＝min(landmark_new[1][0]+x_cut,img.shape[1])

bbox_new[3]＝min(landmark_new[3][1]+y_cut_down,img.shape[0])

Further, the correction module 50 is specifically configured to:

Ret＝warp[bbox_new[1]:bbox_new[3],bbox_new[0]:bbox_new[2]]

The device for correcting the human face is applied to a mobile terminal with a human face unlocking function, a mapping transformation matrix map is obtained through obtaining human face key point information obtained by an obtaining module from a human face key point detection stage, a transformation module transforms image coordinates to be corrected through the mapping transformation matrix map to generate coordinates Warp corresponding to a new human face image, a determining module redetermines new key points landmark_new and new human face frames bbox_new according to the mapping transformation matrix map, and a correcting module corrects the new human face image according to the new human face frames bbox_new to obtain a corrected human face image. In this way, the information of the relative positions of the facial features of the faces can be reserved, the separability of different faces is improved, and the facial features of the same face can be well reserved at the same position, so that the key parts of the faces participating in the verification and the registration (recording) are kept at the same position, and the identification of the same face is simpler; in addition, the size of the face to be cut is normalized to a uniform size (for example, 160×160), and the scaling operation is not needed, so that the user experience is improved.

It should be noted that the above device embodiments and method embodiments belong to the same concept, the specific implementation process of the device embodiments is detailed in the method embodiments, and technical features in the method embodiments are applicable correspondingly in the device embodiments, which are not repeated herein.

The technical scheme of the present invention is described in further detail below with reference to examples.

Please refer to fig. 10. The embodiment of the invention provides a face correction method, which is applied to a mobile terminal with a face unlocking function and comprises the following steps:

step S501, setting a distance g_norm Dis between eyes of a standard face. In this embodiment, g_norm dis=80.

Step 502, providing a coordinate position landmark of a face key point including eyes, nose and mouth according to a face key point detection stage, and calculating a distance xDis in a horizontal direction and a distance yddis in a vertical direction between eyes as follows:

xDis＝landmark[1][0]-landmark[0][0]

yDis＝landmark[1][1]-landmark[0][1]

For example, as shown in fig. 5, the key point coordinate positions landmark and the face frame bbox provided in the face key point detection stage are shown, wherein the coordinate positions landmark stores the coordinate positions of 5 face key points including the left eye coordinate landmark [0], the right eye coordinate landmark [1], the nose coordinate landmark [2], the left mouth corner coordinate landmark [3] and the right mouth corner coordinate landmark [4] in fig. 5, and the face frame bbox records the information of the coordinates, the width and the height of the upper left corner of the face frame.

Step 503, calculating a face rotation angle g_angle according to the distance xDis between eyes in the horizontal direction and the distance yddis in the vertical direction:

g_angle＝cv2.faceAtan2(yDis,xDis)。

where Atan2 is a function, atan2 (y, X) returns the azimuth angle from origin to point (X, y), i.e., the angle from the X-axis, with the result that positive indicates the angle of counterclockwise rotation from the X-axis and negative indicates the angle of clockwise rotation from the X-axis.

Step 504, calculating the Center coordinates (center_x, center_y) of the face position according to the face frame bbox information provided in the face key point detection stage, wherein the bbox records the information of the left upper corner coordinates, the width and the height of the face frame.

Center_x＝bbox[0]+bbox[2]/2.0

Center_y＝bbox[1]+bbox[3]/2.0

Step 505, calculating a face scaling factor Scale according to the distance g_norm dis between the eyes of the standard face and the distance xDis in the horizontal direction and the distance yddis in the vertical direction between the eyes:

Scale＝g_normDis/np.sqrt(xDis*xDis+yDis*yDis)

where np.sqrt (x) is the square root of the computation array x.

Step 506, obtaining a mapping transformation matrix map according to the face position center coordinates, the face rotation angle g_angle and the face scaling factor Scale.

map＝cv2.getRotatinMatrix2D((Center_x,Center_y),g_angle,scale)

And 507, transforming the image coordinates to be corrected through the mapping transformation matrix map to generate coordinates Warp corresponding to the new face image. Specifically, the following formula is adopted to generate a coordinate Warp corresponding to a new face image after the image coordinate to be corrected is subjected to map transformation by the mapping transformation matrix:

Warp＝cv2.warpaffine(img,map,(img.shape[1],img.shape[0]))

wherein, cv2.warp affine is the new face image coordinate generated after the image coordinate is mapped by the mapping transformation matrix map, realizing the affine transformation of the coordinate system; wherein img is the image to be corrected, (img. Shape [1], img. Shape [0 ]) is the size of the new face image, img. Shape [1] is the width of the new face image, img. Shape [0] is the height of the new face image. In this embodiment, the width img.shape [1] of the new face image and the height img.shape [0] of the new face image remain unchanged from the width and height of the image to be corrected.

Step 508, calculating a new key point position landmark_new according to the mapping transformation matrix map. Calculating the left eye coordinate landmark_new [0], the right eye coordinate landmark_new [1], the nose coordinate landmark_new [2], the left mouth angle coordinate landmark_new [3] and the right mouth angle coordinate landmark_new [4] respectively by adopting the following formulas to obtain a new key point position landmark_new; as shown in fig. 6:

landmark_new [4] [1] =map [1] [0] [4] +map [1] [1] +map [1] [2] which is the abscissa in right mouth angular position

Step 509, recalculate the distance dist_eyespace between eyes and the distance dist_eyepath between eyes and mouth according to the new keypoint location landmark_new as follows:

dist_eyetoeye＝landmark_new[1][0]-landmark_new[0][0]

dist_eyetomouth＝landmark_new[3][1]-landmark_new[0][1]

Step 510, setting a face clipping position x_cut, y_cut_up, y_cut_down according to the recalculated distance dist_eyespace between eyes and distance dist_eyepath between eyes and mouth. As shown in fig. 7. The specific calculation formula is as follows:

x_cut＝(x-dist_eyetoeye)/2.0

y_cut_up＝y

y_cut_down＝h-y_cut_up-dist_eyetomouth

Wherein, as shown in fig. 3, x_cut represents the abscissa x of the left eye, shifted to the left by x_cut units; y_cut_up represents the vertical y-axis of the left (right) eye offset y_cut_up units; y_cut_down represents the ordinate of the mouth shifted down by y_cut_down coordinates.

In this embodiment, y is 45, and the width x and height h are 160, namely:

x_cut＝(160-dist_eyetoeye)/2.0

y_cut_up＝45

y_cut_down＝160-y_cut_up-dist_eyetomouth

and 511, recalculating a new face frame bbox_new according to the new key point position landmark_new and the face clipping position, as shown in fig. 7.

bbox_new[0]＝landmark_new[0][0]-x_cut

bbox_new[1]＝landmark_new[0][1]-y_cut_up

bbox_new[2]＝min(landmark_new[1][0]+x_cut,img.shape[1])

bbox_new[3]＝min(landmark_new[3][1]+y_cut_down,img.shape[0])

Step 512, clipping the face image corresponding to the new face image coordinate Warp in step 507 according to the new face frame bbox_new to obtain a corrected face image Ret as follows.

Ret＝warp[bbox_new[1]:bbox_new[3],bbox_new[0]:bbox_new[2]]

Wherein Ret is a face image after correction obtained after clipping, as shown in FIG. 8, warp represents clipping the face image corresponding to the new face image coordinate Warp, bbox_new [1]: bbox_new [3] represents clipping from bbox_new [1] to bbox_new [3] on X horizontal axis; bbox_new [0]: bbox_new [2] denotes clipping on the Y vertical axis starting from bbox_new [0] to bbox_new [2] ending.

In addition, an embodiment of the present invention further provides a mobile terminal, as shown in fig. 11, where the mobile terminal 900 includes: a memory 902, a processor 901 and one or more computer programs stored in the memory 902 and executable on the processor 901, the memory 902 and the processor 901 being coupled together by a bus system 903, the one or more computer programs when executed by the processor 901 to implement the following steps of a method for face correction provided by an embodiment of the present invention:

The method disclosed in the above embodiment of the present invention may be applied to the processor 901 or implemented by the processor 901. The processor 901 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by instructions in the form of integrated logic circuits or software in hardware in the processor 901. The processor 901 may be a general purpose processor, DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 901 may implement or perform the methods, steps and logic blocks disclosed in embodiments of the present invention. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiment of the invention can be directly embodied in the hardware of the decoding processor or can be implemented by combining hardware and software modules in the decoding processor. The software modules may be located in a storage medium in a memory 902, and the processor 901 reads information in the memory 902, in combination with its hardware, to perform the steps of the method as described above.

It will be appreciated that the memory 902 of embodiments of the invention can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The nonvolatile Memory may be Read-Only Memory (ROM), programmable Read-Only Memory (PROM, programmable Read-Only Memory), erasable programmable Read-Only Memory (EPROM), electrically Erasable Read-Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), magnetic random access Memory (FRAM, ferromagnetic Random Access Memory), flash Memory (Flash Memory) or other Memory technology, compact disc Read-Only Memory (CD-ROM, compact Disk Read-Only Memory), digital versatile disc (DVD, digital Video Disk) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices; volatile memory can be random access memory (RAM, random Access Memory), many forms of RAM being available by way of example and not limitation, such as static random access memory (SRAM, static Random Access Memory), static random access memory (SSRAM, synchronous Static Random Access Memory), dynamic random access memory (DRAM, dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic Random Access Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, direct Rambus Random Access Memory). The memory described by embodiments of the present invention is intended to comprise, without being limited to, these and any other suitable types of memory.

It should be noted that the foregoing mobile terminal embodiments and the method embodiments belong to the same concept, the specific implementation process of the foregoing mobile terminal embodiments is detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable to the mobile terminal embodiments, which are not repeated herein.

In addition, in an exemplary embodiment, the present invention further provides a computer storage medium, specifically a computer readable storage medium, for example, including a memory 902 storing a computer program, where one or more programs of a face correction method are stored, where the one or more programs of the face correction method are executed by a processor 901 to implement the following steps of the face correction method provided in the embodiment of the present invention:

It should be noted that the foregoing method program embodiments and method embodiments of face correction on the computer readable storage medium belong to the same concept, the specific implementation process of the foregoing method embodiments is detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable to the foregoing embodiments of the computer readable storage medium, and are not repeated herein.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.

Claims

1. The face correction method is applied to a mobile terminal with a face unlocking function, and is characterized by comprising the following steps:

correcting the new face image according to the new face frame bbox_new to obtain a corrected face image;

the face key point positions landmark comprises coordinate positions of 5 face key points: left eye coordinates landpack [0], right eye coordinates landpack [1], nose coordinates landpack [2], left mouth corner coordinates landpack [3] and right mouth corner coordinates landpack [4]; the face frame bbox records the information of the coordinates, width and height of the upper left corner of the face frame;

The step of obtaining the mapping transformation matrix map according to the face key point information obtained from the face key point detection stage comprises the following steps:

setting the distance between eyes of a standard face;

2. The method according to claim 1, wherein before the step of acquiring the map matrix map based on the face key point information obtained from the face key point detection stage, the method further comprises: and receiving the face key point information provided in the face key point detection stage, wherein the face key point information comprises face key point positions landmark and a face frame bbox.

3. The method according to claim 1, wherein the image coordinates to be corrected are map-transformed by the mapping transformation matrix to generate coordinates Warp corresponding to the new face image; comprising the following steps: the image coordinates to be corrected are subjected to map transformation by the mapping transformation matrix by adopting the following formula to generate coordinates Warp corresponding to the new face image:

Warp＝cv2.warpaffine(img,map,(img.shape[1],img.shape[0]))

4. A method according to claim 3, wherein the step of redefining new keypoint locations landmark_new and new face box bbox_new from the mapping transformation matrix map comprises:

5. The method of claim 4, wherein the step of correcting the new face image according to the new face box bbox_new to obtain a corrected face image comprises:

Ret＝warp[bbox_new[1]:bbox_new[3],bbox_new[0]:bbox_new[2]]

6. The device for face correction is applied to a mobile terminal with a face unlocking function, and is characterized by comprising the following components: the device comprises an acquisition module, a transformation module, a determination module and a correction module; wherein:

7. A mobile terminal, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor performs the steps of a method of face correction as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, wherein a program of a method of face correction is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the method of face correction as claimed in any one of claims 1 to 5.