OA19067A - Face liveness detection method and apparatus, and electronic device. - Google Patents

Abstract

A first deep learning model is trained based on general facial images. A second deep learning model is trained based on extracted facial images cropped from the general facial images. Face liveness detection is performed based on the trained first deep learning model to obtain a first prediction score and the trained second deep learning model to obtain a second prediction score. A prediction score resuit is generated based on the first prediction score and the second prediction score, and the prediction score resuit is compared with a threshold to determine a face liveness detection resuit for the extracted facial images.

Description

FACE LIVENESS DETECTION METHOD AND APPARATUS, AND ELECTRONIC DEVICE

[0001] This application claims priority to Chinese Patent Application No.

201710421333.5, filed on June 7, 2017, which is hereby.incorporated by reference in its entirety.

TÊCHNICAL FIELD

[0002] The présent application relates to the field of computer software technologies^ and in particular, to a face liveness détection method, apparatus, arid electronic device.

BACKGROUND

[0003] A face liveness détection technology is used to déterminé whether the current user is the authentic user by using facial récognition techniques so as to intercept spoofing attacks such as a screen replay attack, a printed photo attack, and a three-dimensional modeling attack.

[0004] Currently, the face liveness détection technology can be classified into an intrusive face liveness détection technology and a non-intrusive face liveness détection technology. In the intrusive face liveness détection technology, a user needs to cooperatively complété some spécifie live actions such as blinking, head tuming, or mouth opening. When performing facial récognition based on the gîven instructions, the liveness détection module can détermine whether an operator accurately complétés the live operation and whether the operator is the authentic user. In the non-intrusive face liveness détection technology, a user does not need to cooperatively complété a live action, so that user expérience is better, but the technical complexity is higher. In addition, liveness détection is performed mainly depending on information about an input single frame image or information about other device sensors.

[0005] In the described non-intrusive face liveness détection technology in the existing technology, supervised training is usually performed on a single de'ep leaming model by using live and non-live facial images, and then face liveness prédiction is performed on the input single frame image by using the trained model.

[0006] However, such a technical solution heavily dépends on a spoofing face attack type of training data, and is limited by an objective condition of insufficient training data. It is dîfficult to fully extract a live face image feature. As a resuit, this model cannot fully express a live face feature, and accuracy of a face liveness détection resuit is reduced.

SUMMARY

[0007] Embodiments of the présent application provide a face liveness détection method, apparatus, and electronic device to résolve the following technical problems in the existing technology. In a technical solution based on a single deep leaming model, it is diffïcult to fully extract a live face image feature. As a resuit, this model cannot fully express a live face feature, and accuracy of a face liveness détection resuit is reduced.

[0008] To résolve the described technical problems, the embodiments of the présent application are implemented as follows :.....................................................................

[0009] An embodiment of the présent application provides a face liveness détection method, including: training a first deep leaming model based on the general facial images; training a second deep leaming model based on the extracted facial images cropped from the general facial images; and performing face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0010] An embodiment of the présent application provides a face liveness détection apparatus, including: a training module, configured to: train a first deep leaming model based on the general facial images; and train a second deep leaming model based on the extracted facial images cropped from the general facial images; and a détection module, configured to perform face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0011] At least one technical solution used in the embodiments of the présent application can achieve the following bénéficiai effects. One such benefit is more live face image features are extracted. Compared with a model in the existing technology, the trained first deep leaming model and the trained second deep leaming model jointly better express the live face feature, thereby improving the accuracy of the face liveness détection resuit. Therefore, a part or ail of problems in the existing technology can be resolved.

BRIEF DESCRIPTION OF DRAWINGS

[0012] To describe the technical solutions in embodiments of the présent application or in the existing technology more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments or the existing technology. Apparently, the accompanying drawings in the following description merely show some embodiments of the present application, and a person of ordinary skill in the art can still dérivé other drawings from these accompanying drawings without créative efforts.

[0013] FIG. 1 is a schematic flowchart illustrating an example of a model training stage in a solution of the present application;

[0014] FIG. 2 is a schematic flowchart illustrating an example of a liveness détection stage in a solution of the present application;

[0015] FIG. 3 is a schematic flowchart illustrating a face liveness détection method according to an embodiment ofthe present application; ........................................... ..........................

[0016] FIG. 4 is a schematic diagram illustrating comparison between a general facial image and an extracted facial image according to an embodiment of the present application;

[0017] FIG. 5 is a schematic structural diagram illustrating a face liveness détection apparatus corresponding to FIG. 3 according to an embodiment of the present application; and [0018] FIG. 6 is a flowchart illustrating an example of a computer-impiemented method for determining user authenticity with face liveness détection, according to an implémentation of the present disclosure.

DESCRIPTION OF EMBODIMENTS

[0019] Embodiments of the present application provide a face liveness détection method, apparatus, and electronic device.

[0020] To make a person skilled in the art better understand the technical solutions in the present application, the following clearly and completely describes the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are merely a part rather than ail of the embodiments of the present application. Ail other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without créative efforts shall fall within the protection scope of the present application.

[0021] Ail deep leaming models in a solution of the present application are based on a neural network. For ease of description, a core idea of the solution ofthe present application is first described based on an example and with reference to FIG. 1 and FIG. 2.

[0022] In this example, the solution of the present application can be classified into a model training stage and a liveness détection stage.

[0023] FIG. 1 is a schematic flowchart illustrating an example of a model training stage in a solution of the present application. In a model training stage, two independent deep 5 leaming models are trained by using live and non-live s amples (belonging to a training data set) in a facial image: a first deep leaming model and a second deep leaming model. An input image of the first deep leaming model is a collected general facial image, and an input image of the second deep leaming model can be an extracted facial image cropped from the general facial image. The first deep leaming model and the second deep leaming model can use 10 different deep leaming network structures (i.e. a structure of a neural network that a model is based on). Different network structures are differently sensitive to different image features. Live and non-live training data sets are used to complété training of the first deep leaming model and the second deep leaming model based on a deep leaming method.

[0024] FIG. 2 is a schematic flowchart illustrating an example of a liveness détection 15 stage in a solution of the present application. In a liveness détection stage, a face scanning image of a user is collected as a general facial image of the user, and a first deep leaming model is input to obtain a prédiction score PA. In addition, face détection is performed on the face scanning image of the user, an extracted facial image is cropped from the face scanning image of the user based on a détection resuit, and a second deep leaming model is input to the 20 extracted facial image, to obtain a prédiction score PB. Afterwards, for example, a prédiction score resuit of (PA+PB) can be compared with a determined threshold (e.g. the threshold can be 1), to make a joint decision to détermine a face liveness détection resuit for the face scanning image of the user.

[0025] Based on the described core idea, the following describes the solution of the 25 présent application in detail.

[0026] FIG. 3 is a schematic flowchart illustrating a face liveness détection method according to an embodiment of the present application. From a perspective of a program, the procedure can be executed by a program on a server or a terminal, for example, an identity authentication program or an e-commerce application. From a perspective of a device, the 30 procedure is executed by at least one of the following devices that can be used as a server or a terminal: an access control device, a personal computer, a medium computer, a computer cluster, a mobile phone, a tablet computer, an intelligent wearable device, a car machine, or a point of sale (POS),

[0027] The procedure in FIG. 3 can include the following steps.

[0028] S301. Train a first deep leaming model based on general facial images.

[0029] In this embodiment of the présent application, the general facial images used to train the first deep leaming model can include a plurality of samples. In the plurality of samples, some are live facial images that are collected by shooting a live face and that can be used as positive samples, and some are non-live facial images that are collected by shooting a non-live face such as a face picture or a face model and that can be used as négative samples.

[0030] ïn this embodiment of the présent application, the first deep leaming model is a classification model, and the general facial images are used as inputs of the classification model. After model processing, the general facial images can be classified into at least the live facial image category or the non-live facial image category. An objective of training the first deep leaming model is to improve classification accuracy of the first deep leaming model.

[0031] S302. Train a second deep leaming model based on extracted facial images cropped from the general facial images.

[0032] In this embodiment of the présent application, in addition to an entire facial région, the general facial image generally includes some unrelated régions, such as a background région and a human body except a face. The extracted facial image can exclude the unrelated régions, and can include at least an extracted facial région, for example, an entire facial région, an eye région, or a nasal région. There can be one or more second deep leaming models, and each second deep leaming model can correspond to a type of facial régions.

[0033] FIG. 4 is a schematic diagram illustrating comparison between a general facial image and an extracted facial image according to an embodiment of the présent application.

[0034] In FIG. 4, (a) is a general facial image. For ease of understanding, an extracted facial image is marked in (a) by using dashed lines, and (a) can be correspondingly cropped to obtain an extracted facial image shown in (b).

[0035] In addition, when the extracted facial image is an image încluding only a partial facial région, the general facial image can also be an image încluding an entire facial région and basically excluding an unrelated région.

[0036] In this embodiment of the présent application, the extracted facial image used to train the second deep leaming model can also include a variety of samples. In the variety of samples, some are live facial images that can be used as positive samples, and some are non-Iive facial images that can be used as négative samples.

[0037] In this embodiment of the présent application, the second deep leaming model is also a classification model, and the extracted facial images are used as input of the classification model. After model processing, the extracted facial images can be classified into at least the live facial image category or the non-live facial image category. An objective oftraining the second deep leaming model is to improve classification accuracy ofthe second 10 deep leaming model.

[0038] In addition to being cropped from the general facial image, the extracted facial image can be obtained by means of spécial collection without depending on the general facial image.

[0039] In this embodiment of the présent application, the first deep leaming model and the second deep leaming model can be different models or a same model before training.

[0040] An execution sequence of step S301 and step S302 is not limited in the présent application, and step S301 and step S302 can be simultaneously or successively performed.

[0041] S303. Perform face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0042] Each step in FIG. 3 can be performed by a same device or a same program, or can be performed by different devices or different programs. For example, step S301 to step S3O3 are performed by a device 1. For another example, both step S301 and step S302 are performed by a device 1, and step S303 is performed by a device 2; etc.

[0043] According to the method in FIG. 3, more live face image features are extracted.

Compared with a model in the existing technology, the trained first deep leaming model and the trained second deep leaming model jointly better express a live face feature, thereby improving accuracy of a face liveness détection resuit. Therefore, a part or ail of problème in the existing technology can be resolved.

[0044] Based on the method in FIG. 3, this embodiment of the présent application further provides some spécifie implémentation solutions of the method and an extension solution, which are described below.

[0045] In this embodiment of the présent application, to implement a différence between sensitivity of the first deep leaming model to an image feature and sensitivity of the second deep leaming model to an image feature, the first deep leaming model and the second deep leaming model can preferably use different deep leaming network structures.

[0046] Different network structures of two deep leaming models can indicate that the two deep leaming models include one or more different network structure parameters. The network structure parameter can include, for example, a quantity of hidden variable layers, a type of a hidden variable layer, a quantity of neuron nodes, a quantity of input layer nodes, or a quantity of output layer nodes.

[0047] Certainly, some spécifie deep leaming models can also include corresponding spécifie parameters. For example, for a deep leaming model based on a convolutional neural network widely used in the image field currently, a size of a convolution kernel of a convolution unit is also a spécifie network structure parameter of this deep leaming model.

[0048] For the solution of the présent application, generally, the different deep leaming network structures include at least one of the following parameters: a quantity of hidden variable layers, a type of a hidden variable layer, a quantity of neuron nodes, or a size of a convolution kernel of a convolution unit.

[0049J In this embodiment of the présent application, to improve model training efficiency and model training reliability, model training can be performed in a supervîsed training manner.

[0050] For example, in a supervîsed training manner, for step S301, the general facial image includes a first label, and the first label indicates whether a general facial image corresponding to the first label is a live facial image.

[0051] The training a first deep leaming model based on a general facial image can include: inputting the first deep leaming model to the general facial image, where the first deep leaming model extracts a feature of the general facial image, and predicts, based on the extracted feature, whether the general facial image is a live facial image; and adjusting the first deep leaming model based on a prédiction resuit and the first label of the general facial image. Generally, when the prédiction resuit is inconsistent with the first label, the first deep leaming model is adjusted, so that the adjusted first deep leaming model can obtain, by means of re-prediction, a prédiction resuit consistent with the first label.

[0052J The feature extracted by the first deep leaming model in a training process can preferably include an image structure feature of the general facial image, for example, a screen photo edge or face distortion in the general facial image.

[0053] For another example, similarly, in a supervised training manner, for step S302, the extracted facial image includes a second label, and the second label indicates whether an extracted facial image corresponding to the second label is a live facial image.

[0054] The training a second deep leaming model based on the extracted facial images cropped from the general facial images can include: obtaining the extracted facial images cropped from the general facial images; applying the second deep leaming model to the obtained extracted facial image, where the second deep leaming model extracts a feature of the extracted facial image, and predicts, based on the extracted feature, whether the extracted facial image is a live facial image; and adjusting the second deep leaming model based on a prédiction resuit and the second label of the extracted facial image. Generally, when the prédiction resuit is inconsistent with the second label, the second deep leaming model is adjusted, so that the adjusted second deep leaming model can obtain a prédiction resuit consistent with the second label by means of re-prediction.

[0055] The feature extracted by the second deep leaming model in a training process can preferably include an image material feature of the extracted facial image, for example, blurring, texture, or color distortion in the extracted facial image.

[0056] In the two examples described above, the first deep leaming model and the second deep leaming model are differently sensitive to different image features. The first deep leaming model is more sensitive to the image structure feature, and the second deep leaming model is more sensitive to the image material feature. For a face image, the image structure feature is relatively a global and generalized feature, and the image material feature is relatively a local and refined feature.

[0057] Therefore, the trained first deep leaming model and the trained second deep leaming model can jointly extract a face image feature more hierarchically and abundantly, so as to make a joint decision to obtain a more accurate face liveness détection resuit.

[0058] In this embodiment of the present application, corresponding training data sets and/or corresponding deep leaming network structures are different, so that the first deep leaming model and the second deep leaming model can be differently sensitive to different image features.

[0059] For example, if the first deep leaming model and the second deep leaming model are based on a convolutional neural network, a convolution kernel of a convolution unit in a convolutional neural network that the first deep leaming model is based on can be relatively large, so that the first deep leaming model extracts an image structure feature of the 5 general facial image. Correspondingly, a convolution kernel of a convolution unit in a convolutional neural network that the second deep leaming model is based on can be relatively small, so that the second deep leaming model extracts an image material feature of the extracted facial image. Therefore, in this example, the convolution kernel of the convolution unit in the convolutional neural network that the first deep leaming model is ^—1'θ based On is^_greater than^_the convolution kemel^offiie convolution unit in the convolutional neural network that the second deep leaming model is based on.

[0060] It should be noted that the size of the convolution kernel is merely an example of a parameter that can affect the sensitivity, and another network structure parameter can also affect the sensitivity.

[0061] In this embodiment of the present application, for step S303, the trained first deep leaming model and the trained second deep leaming model jointly make a decision to perform the face liveness détection. There are a variety of spécifie decision manners. For example, a separate decision is made by separately using the first deep leaming model and the second deep leaming model, and then a final decision resuit is determined by synthesizing 20 ail separate decision results. For another example, a separate decision can be first made by using either of the first deep leaming model and the second deep leaming model. When a separate decision resuit satisfîes a spécifie condition, the separate decision resuit can be directly used as a final decision resuit; otherwise, a decision is comprehensively made in combination with another remaining model, to obtain a final decision resuit; etc.

[0062] If a first manner described in the previous paragraph is used, an example is as follows:

[0063] For example, for step S303, the performing face liveness détection based on the trained first deep leaming model and the trained second deep leaming model can include: obtaining the general facial image (which is generally a face scanning image of a user) 30 collected for the face liveness détection; inputting the trained fïrst deep leaming model to the collected general facial image for processing, to obtain corresponding first prédiction data; obtaining the extracted facial image cropped from the collected general facial image, and inputting the trained second deep leaming model for processing, to obtain corresponding second prédiction data; and making a joint decision based on the first prédiction data and the second prédiction data, to obtain a face liveness détection resuit for the face scanning image of the user.

[0064] The first prédiction data can be, for example, the described prédiction score PA, and the second prédiction data can be, for example, the described prédiction score PB. Certainly, the prédiction score is merely an example of an expression form of the first prédiction data and the second prédiction data, or there can be another expression form, for example, a prob ability value or a Boolean value.

[0065] The above is the face liveness détection method provided in this embodiment of the présent application. As shown in FIG. 5, based on a same idea of the disclosure, an embodiment of the présent application further provides a corresponding apparatus.

[0066] FIG. 5 is a schematic structural diagram illustrating a face liveness détection apparatus corresponding to FIG. 3 according to an embodiment of the présent application. The apparatus can be located on an execution body of the procedure in FIG. 3, including: a training module 501, configured to: train a first deep leaming model based on the general facial images; and train a second deep leaming model based on the extracted facial images cropped from the general facial images; and a détection module 502, configured to perform face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0067] Optionally, the first deep leaming model and the second deep leaming model use different deep leaming network structures.

[0068] Optionally, the different deep leaming network structures include at least one of the following parameters: a quantity of hidden variable layers, a type of a hidden variable layer, a quantity of neuron nodes, or a size of a convolution kernel of a convolution unit.

[0069] Optionally, the general facial image includes a first label, and the first label indicates whether a general facial image corresponding to the first label is a live facial image. [0070] The training, by the training module 501, a first deep leaming model based on the general facial images includes: inputting, by the training module 501, the first deep leaming model to the general facial image, where the first deep leaming model predicts, based on an image structure feature of the general facial image, whether the general facial image is a live facial image; and adjusting the first deep leaming model based on a prédiction resuit and the first label of the general facial image.

[007IJ Optionally, the extracted facial image includes a second label, and the second label indicates whether an extracted facial image corresponding to the second label is a live facial image.

[0072] The training, by the training module 501, a second deep leaming model based on the extracted facial images cropped from the general facial images includes: obtaining, by the training module 501, the extracted facial image cropped from the general facial image; and inputting the second deep leaming model to the extracted facial image, where the second deep leaming model predicts, based on an image material feature of the extracted facial image, whether the extracted facial image is a live facial image; and adjusting the second deep leaming model based on a prédiction resuit and the second label of the extracted facial image.

[0073] Optionally, the first deep leaming model and the second deep leaming model are based on a convolutional neural network.

[0074] A convolution kernel of a convolution unit in a convolutional neural network that the first deep leaming model is based on is greater than a convolution kernel of a convolution unit in a convolutional neural network that the second deep leaming model is based on, so that the first deep leaming model extracts an image structure feature of the general facial image, and the second deep leaming model extracts an image material feature of the extracted facial image.

[0075] Optionally, the performing, by the détection module 502, face liveness détection based on the trained first deep leaming model and the trained second deep leaming model includes: obtaining, by the détection module 502, the general facial image collected for the face liveness détection; inputting the trained first deep leaming model to the collected general facial image for processing, to obtain corresponding first prédiction data; obtaining extracted facial image cropped from the collected general facial image, and inputting the trained second deep leaming model for processing, to obtain corresponding second prédiction data; and making a joint decision based on the first prédiction data and the second prédiction data, to obtain a face liveness détection resuit for a face scanning image of the user.

[0076] Based on a same idea of the disclosure, an embodiment of the present application further provides a corresponding electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor.

[0077] The memory stores an instruction that can be executed by the at least one processor, and the instruction is executed by the at least one processor, to enable the at least one processor to: train a first deep leaming model based on the general facial images; train a 5 second deep leaming model based on the extracted facial images cropped from the general facial images; and perform face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0078] Based on a same idea of the disclosure, an embodiment of the présent ___application further provides a corresponding non-volatile computer storage medium, where.................................

the non-volatile computer storage medium stores a computer exécutable instruction, and the computer exécutable instruction is set to: train a first deep leaming model based on the general facial images; train a second deep leaming model based on the extracted facial images cropped from the general facial images; and perform face liveness détection based on the trained first deep leaming model and the trained second deep leaming model.

[0079] The embodiments in this spécification are ail described in a progressive manner, for same or similar parts in the embodiments, reference can be made to these embodiments, and each embodiment focuses on a différence from other embodiments. Especially, an apparatus embodiment, an electronic device embodiment, a non-volatile computer storage medium embodiment are basically similar to a method embodiment, and therefore is described briefly; for related parts, reference is made to partial descriptions in the method embodiment.

[0080] The apparatus, the electronic device, and the non-volatile computer storage medium provided in the embodiments of the présent application correspond to the method. Therefore, the apparatus, the electronic device, and the non-volatile computer storage 25 medium also hâve bénéficiai technical effects similar to a bénéficiai technical effect of the corresponding method. The bénéficiai technical effect of the method is described in detail above, so that the bénéficiai technical effects of the corresponding apparatus, electronic device, and non-volatile computer storage medium are not described here again.

[0081] In the 1990s, whether technology improvement is hardware improvement (for example, improvement of a circuit structure, such as a diode, a transistor, or a switch) ori software improvement (improvement of a method procedure) can be obviously distinguished.>

However, as technologies develop, improvement of many current method procedures can be!

I 'j i

considered as direct improvement of a hardware circuit structure. A designer usually programs an improved method procedure to a hardware circuit to obtain a corresponding hardware circuit structure. Therefore, a method procedure can be improved by hardware entity modules. For example, a programmable logic device (PLD) (e.g. a field programmable 5 gâte array (FPGA)) is such an integrated circuit, and a logical function of the programmable logic device is determined by a user by means of device programming. The designer perforais programming to integrate a digital system to a PLD without requesting a chip manufacturer to design and produce an application-specific integrated circuit chip, fri addition, the programming is mostly implemented by modifying logic compiler software instead of -tO manuallymaking an integratedfoifcûirchipTThis is similar to a software compiler used to develop and compose a program. However, original code obtained before compilation is also written in a spécifie programming language, and this is referred to as hardware description language (Hardware Description Language, HDL). However, there are various HDLs, such as an ABEL (Advanced Boolean Expression Language), an AHDL (Altéra Hardware 15 Description Language), Confluence, a CUPL (Comell University Programming Language), HDCal, a JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and an RHDL (Ruby Hardware Description Language). Currently, a VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are most popular. A person skilled in the art should also understand that, only logic programming 20 needs to be performed on the method procedure by using the described several hardware description languages, and the several hardware description languages are programmed to an integrated circuit, so that a hardware circuit that implements the logical method procedure can be easily obtained.

[0082] A controller can be implemented in any appropriate manner. For example, the 25 controller can use a microprocessor or a processor, and can store forms of a computer readable medium, a logic gâte, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller that are of computer readable program code (e.g. software or hardware) that can be executed by the (micro) processor. The examples of controller include butare not limited to the following 30 microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, or Silicone Labs C8051F320. A memory controller can also be implemented as a part of control logic of the memory. A person skilled in the art also knows that, in addition to implementing the controller in a pure computer readable program code manner, logic programming can be completely performed by using the method step, so that the controller implements a same function in a form of a logical gâte, a switch, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. Therefore, the controller can be considered as a hardware component, and an apparatus for implementing various 5 functions in the controller can also be considered as a structure in a hardware component.

Altematively, an apparatus configured to implement various functions can be considered as a software module or a structure in a hardware component that can implement the method.

[0083] The system, apparatus, module, or unit described in the described embodiments can be implemented by a computer chip or an entity, or implemented by a 10 product with a function. A typical implémentation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a caméra phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, or a wearable device, or a combination of any of these devices.

[0084] For ease of description, the described apparatus is described by dividing functions into various units. Certainly, when the present application is implemented, the functions of each unit can be implemented in one or more pièces of software and/or hardware.

[0085] A person skilled in the art should understand that the embodiments of the 20 present disclosure can be provided as a method, a system, or a computer program product.

Therefore, the present disclosure can use a form of hardware only embodiments, software only embodiments, or embodiments with a combination of software and hardware. In addition, the present disclosure can use a form of a computer program product that is implemented on one or more computer-usable storage media (including but not limited to a 25 disk memory, a CD-ROM. an optical memory, etc.) that include computer-usable program code.

[0086] The present disclosure is described with reference to the flowcharts and/or block diagrams of the method, the device (system), and the computer program product according to the embodiments of the present disclosure. It should be understood that 30 computer program instructions can be used to implement each process and/or each block in the flowcharts and/or the block diagrams and a combination of a process and/or a block in the flowcharts and/or the block diagrams. These computer program instructions can be provided for a general-puipose computer, a dedicated computer, an embedded processor, or a processor of any other programmable data processing device to generate a machine, so that the instructions executed by a computer or a processor of any other programmable data processing device generate an apparatus for implementing a spécifie function in one or more 5 processes in the floweharts or in one or more blocks in the block diagrams.

[0087] These computer program instructions can be stored in a computer readable memory that can instruct the computer or any other programmable data processing device to work in a spécifie manner, so that the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus 10 implements a spécifie function in one or more processes in the floweharts and/or in one or more blocks in the block diagrams.

[0088] These computer program instructions can be loaded to a computer or another programmable data processing device, so that a sériés of operations and steps are performed on the computer or another programmable device, thereby generating computer-implemented 15 processing. Therefore, the instructions executed on the computer or another programmable device provide steps for implementing a spécifie function in one or more processes in the floweharts or in one or more blocks in the block diagrams.

[0089] In typical configuration, the computing device includes one or more processors (CPU), an input/output interface, a network interface, and a memory.

[0090] The memory can include a form of a volatile memory, a random access memory (RAM) and/or a non-volatile memory, etc. in a computer readable medium, such as a read-only memory (ROM) or a flash memory (flash RAM). The memory is an example of the computer readable medium.

[0091] The computer readable medium includes volatile and non-volatile, removable 25 and non-removable media, and can store information by using any method or technology. The information can be a computer readable instruction, a data structure, a program module, or other data. The examples of computer storage medium include but are not limited to a phase change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a random access memory (RAM) of another type, 30 a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory or another memory technology, a compact dise read-only memory (CD-ROM), a digital versatile dise (DVD) or another optical storage, a magnetic tape, a magnetic disk storage, another magnetic storage device, or any other non-transmission medium. The computer storage medium can be used to store information that can be accessed by the computing device. As described in this spécification, the computer readable medium does not include transitory media (transitory media), for example, a modulated data signal 5 and a carrier.

[0092] It should be further noted that, ternis include, contain, or their any other variant is intended to cover non-exclusive inclusion, so that a process, a method, an article, or a device that inciudes a sériés of éléments not only inciudes these very éléments, but also inciudes other éléments which are not expressly listed, or further inciudes éléments inhérent 10 to such process, method, article, or device. An element preceded by inciudes a ... does not, without more constraints, preclude the existence of additional identical éléments in the process, method, article, or device that inciudes the element.

[0093] The présent application can be described in common contexts of computer exécutable instructions executed by a computer, such as a program module. Generally, the 15 program module inciudes a routine, a program, an object, a component, a data structure, etc.

executing a spécifie task or implementing a spécifie abstract data type. The présent application can also be practiced in distributed computing environments. In these distributed computing environments, tasks are executed by remote processing devices that are connected by using a communications network. In the distributed computing environments, the program 20 module can be located in local and remote computer storage media that include storage devices.

[0094] The embodiments in this spécification are ail described in a progressive manner, for same or similar parts in the embodiments, reference can be made to these embodiments, and each embodiment focuses on a différence from other embodiments.

Especially, a system embodiment is basically similar to a method embodiment, and therefore is described briefly; for related parts, reference can be made to partial descriptions in the method embodiment.

[0095] The previous descriptions are merely embodiments of the présent application, and are not intended to limit the présent application. For a person skilled in the art, the 30 présent application can hâve various modifications and changes. Any modifications, équivalent replacements, improvements, etc. made within the spirit and principle of the présent application shall fall within the protection scope of the présent application.

[0096] FIG. 6 is a flowchart illustrating an example of a computer-implemented method 600 for determining user authenticity with face lîveness détection, according to an implémentation of the present disclosure. For clarity of présentation, the description that follows generally describes method 600 in the context ofthe other figures in this description.

However, it will be understood that method 600 can be performed, for example, by any system, environment, software, and hardware, or a combination of Systems, environments. software, and hardware, as appropriate. In some implémentations, various steps of method 600 can be run in parallel, in combination, in loops, or in any order.

[0097] At 602, a first deep leaming model is trained to classify general facial images.

The general facial images are classified into at least live facial images and non-live facial images. In some implémentations, the live facial images are considered to be positive samples and the non-live facial images are considered to be négative samples. In some implémentations, the first deep leaming model is a classification model and the general facial images are used as inputs of the first deep leaming model. Training the first deep leaming 15 model improves classification accuracy with respect to the general facial images.

[0098] In some implémentations, a particular general facial image includes a first label indicating whether the particular general facial image corresponding to the first label is a live facial image. In some implémentations, the training of the first deep leaming model includes: 1) inputting the particular general facial image to the first deep leaming model to 20 generate a first prédiction resuit, based on an image structure feature of the particular general facial image, of whether the particular general facial image is a live facial image and 2) adjusting the first deep leaming model based on the first prédiction resuit and the first label. From 602, method 600 proceeds to 604.

[0099] At 604, cropped facial images are extracted from the general facial images. In 25 some implémentations, a particular cropped facial image includes a second label, and the second label indicates whether the particular cropped facial image corresponding to the second label is a live facial image. In some implémentations, the training of the second deep leaming model based on the cropped facial image includes: 1) obtaining the particular cropped facial image; 2) inputting the particular cropped facial image to the second deep 30 leaming model to generate a second prédiction resuit, based on an image material feature of the particular cropped facial image, of whether particular cropped facial image is a live facial image; and 3) adjusting the second deep leaming model based on the second prédiction resuit and the second label. From 604, method 600 proceeds to 606.

[00100] At 606, a second deep leaming model is trained based on the cropped facial images. From 606, method 600 proceeds to 608.

[00101] At 608, a face liveness détection is performed based on the trained first deep 5 leaming model and the trained second deep leaming model. In some implémentations, the first deep leaming model and the second deep leaming model are based on a convolutional neural network, and wherein a convolution kernel of a convolution unit in a convolutional neural network of the first deep leaming model is greater than a convolution kernel of a .............................convolution unit in a convolutional neural network of the second deep leaming model. After 10 608, method 600 stops.

[00102] In some implémentations, the face liveness détection includes: 1) obtaining a general facial image; 2) inputting the general facial image into the trained first deep leaming model to obtain corresponding first prédiction data; 3) obtaining a cropped facial image from the general facial image; 4) inputting the cropped facial image into the trained second deep 15 leaming model to obtain corresponding second prédiction data; and 5) making a joint decision based on the first prédiction data and the second prédiction data to obtain a face liveness détection resuit. From 608, method 600 proceeds to 610.

[00103] Implémentations of the subject matter described in this spécification can be implemented so as to realize particular advantages or technical effects. The described face 20 liveness détection can be used to enhance authentication processes and to ensure data security.

For example, the described method can be used to distinguish between images of a lîve and non-live human face to help avoid fraud and malicious behavior with respect to secured data. The described method can be incorporated into computing devices (such as, mobile computing devices and digital imaging devices).

[00104] The face liveness resuit can be displayed on a graphical user interface. Based on the face liveness resuit, a détermination of whether to perform subséquent actions (for example, unlocking secured data, operating a software application, storing data, sending data across a network, or displaying data on a graphical user interface).

[00105] The described methodology permits enhancement of various mobile 30 computing device transactions and overall transaction/data security. Participants in transactions using mobile computing devices can be confident that facial images used to unlock a mobile computing device or to authorize a transaction are valid and that they will not be victims of fraud.

[00106] The described methodology can ensure the efficient usage of computer resources (for example, processing cycles, network bandwidth, and memory usage), through the efficient vérification of data/transactions. At least these actions can minimize or prevent waste of available computer resources with respect to multiple parties in a mobile computing transactions by preventing undesired/fraudulent transactions. Instead of users needing to verify data with additional research or transactions, transactions can be depended upon as valid.

[00107]______In some implémentations, a graphical user interface can be analyzed to ensure................... that graphical éléments used in face liveness détection operations (for example, scanning and vérification of the liveness of a human face with a mobile computing device) can be positioned on graphical user interfaces to be least obtrusive for a user (for example, to obscure the least amount of data and to avoid covering any critical or often-used graphical user interface éléments).

[00108] Embodiments and the operations described in this spécification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this spécification or in combinations of one or more of them. The operations can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources. A data processing apparatus, computer, or computing device may encompass apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include spécial purpose logic circuitry, for example, a central processing unit (CPU), a field programmable gâte array (FPGA) or an application-specific integrated circuit (ASIC). The apparatus can also include code that créâtes an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system (for example an operating system or a combination of operating Systems), a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.

[00109] A computer program (also known, for example, as a program, software, software application, software module, software unit, script, or code) can be written in any form of programming language, including compiled or interpreted languages, déclarative or procédural languages, and it can be deployed in any form, including as a stand-alone program 5 or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A program can be stored in a portion of a file that holds other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (for example, files that store one or more modules, sub-programs, or portions of code). A computer program can be —1Ό cxecuted^onmne^CŒrLputer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[00110] Processors for execution of a computer program include, by way of example, both general- and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a 15 read-only memory or a random-access memory or both. The essential éléments of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data. A computer can be embedded in another device, for ex ample 20 a mobile device, a personal digital assistant (PDA), a game console, a Global Positioning System (GPS) receiver, or a portable storage device. Devices suitable for storing computer program instructions and data include non-volatile memory, media and memory devices, including, by way of example, semiconductor memory devices, magnetic disks, and magneto-optical disks. The processor and the memory can be supplemented by, or 25 incorporated in, special-purpose logic circuitry.

[00111] Mobile devices can include handsets, user equipment (UE), mobile téléphonés (for example, smartphones), tablets, wearable devices (for example, smart watches and smart eyeglasses), implanted devices within the human body (for example, biosensors, cochlear implants), or other types of mobile devices. The mobile devices can communicate wirelessly 30 (for example, using radio frequency (RF) signais) to varions communication networks (described below). The mobile devices can include sensors for determining characteristics of the mobile device’s current environment. The sensors can include caméras, mi empli »n es proximity sensors, GPS sensors, motion sensors, accelerometers, ambient light sensors, moisture sensors, gyroscopes, compassés, barometers, fingerprint sensors, facial récognition Systems, RF sensors (for example, Wi-Fi and cellular radios), thermal sensors, or other types of sensors. For example, the caméras can include a forward- or rear-facing caméra with movable or fixed lenses, a flash, an image sensor, and an image processor. The caméra can be 5 a megapixel caméra capable of capturing details for facial and/or iris récognition. The caméra along with a data processor and authentication information stored in memory or accessed remotely can form a facial récognition System. The facial récognition system or one-or-more sensors, for example, microphones, motion sensors, accelerometers, GPS sensors, or RF sensors, can be used for user authentication.

[00112] To provide for interaction with a user, embodiments can be implemented on a computer having a display device and an input device, for example, a liquid crystal display (LCD) or organic light-emitting diode (OLED)/virtual-reality (VR)/augmented-reality (AR) display for displaying information to the user and a touchscreen, keyboard, and a pointing device by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, for example, visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, încluding acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending 20 web pages to a web browser on a user’s client device in response to requests received from the web browser.

[00113] Embodiments can be implemented using computing devices interconnected by any form or medium of wireline or wireless digital data communication (or combination thereof), for example, a communication network. Examples of interconnected devices are a 25 client and a server generally remote from each other that typically interact through a communication network. A client, for example, a mobile device, can carry out transactions itself, with a server, or through a server, for example, performing buy, sell, pay, give, send, or loan transactions, or authorizing the same. Such transactions may be in real time such that an action and a response are temporally proximate; for example an individual perceives the 30 action and the response occurring substantially simultaneously, the time différence for a response following the individual’s action is less than 1 millisecond (ms) or less than 1 second (s), or the response is without intentional delay taking into account processing limitations of the system.

[00114] Examples of communication networks include a local area network (LAN), a radio access network (RAN), a metropolitan area network (MAN), and a wide area network (WAN). The communication network can include ail or a portion of the Internet, another communication network, or a combination of communication networks. Information can be 5 transmitted on the communication network according to various protocols and standards, including Long Term Evolution (LTE), 5G IEEE 802, Internet Protocol (IP), or other protocols or combinations of protocols. The communication network can transmit voice, video, biométrie, or authentication data, or other information between the connected computing devices.

Features described as separate implémentations may be implemented, in combination, in a single implémentation, while features described as a single implémentation may be implemented in multiple implémentations, separately, or in any suitable sub-combination. Operations described and claimed in a particular order should not be understood as requiring that the particular order, nor that ail illustrated operations must be performed (some operations can be optional). As appropriate, multitasking or parallel-processing (or a combination of multitasking and parallel-processing) can be performed.

Claims (8)

1. AMENDED CLAIMS

   1. A face récognition method for determining whether an image that includes a face is a live image or a non-live image, the method comprising:

   training a first deep leaming model by supervised training on a plurality of general facial images (S301), the general facial images comprising live facial images collected by shooting a live face and labeled as positive samples, and non-live facial images collected by shooting a non-live face that is a face picture or a face model and labeled as négative samples;

   training a plurality of second deep leaming models by supervised training on a plurality of extracted facial images cropped from the general facial images (S302), the second deep leaming models comprising an eye deep leaming model and a nose deep leaming model corresponding to an eye and a nose type of facial région respectîvely, the extracted facial images comprising live facial images labeled as positive samples and non-live facial images and labeled as négative samples, wherein the first deep leaming model and each of the second deep leaming models are classification models and wherein after training the models classify facial images into a live facial image category or a non-live facial image category;

   performing face liveness détection on a first general facial image using the trained first deep leaming model to obtain a first prédiction score and the plurality of trained second deep leaming models to obtain second prédiction scores (S3O3), comprising:

   obtaining the first general facial image collected for the face liveness détection, inputting the first general facial image into the trained first deep leaming model for processing to obtain the first prédiction score, obtaining a plurality of extracted facial images cropped from the first general facial image, the extracted facial images comprising an eye image région image and a nose image région image, and inputting the extracted facial images into respective trained second deep leaming models for processing, the second deep leaming models comprising the eye deep leaming model and the nose deep leaming model, to obtain the second prédiction scores, generating a prédiction score resuit based on the first prédiction score and the second prédiction scores, and comparing the prédiction score resuit with a threshold to détermine whether the first general facial image is a live image or a non-live image.
2. 2. The method according to claim 1, wherein the first deep leaming model and a second deep leaming model use different deep leaming network structures.
3. 3. The method according to claim 2, wherein the different deep leaming network structures

   5 comprise at least one of the following parameters: a quantity of hidden variable layers, a type of a hidden variable layer, a quantity of neuron nodes, or a size of a convolution kemel of a convolution unit.
4. 4. The method according to any one of claims 1 to 3, wherein generating a prédiction score

   10 resuit based on the first prédiction score and the second prédiction scores comprises generating the prédiction score resuit as a sum of the first prédiction score and the second prédiction scores.
5. 5. The method according to any one of claims 1 to 3, wherein the first deep leaming model and the second deep leaming model are based on a convolutional neural network; and ¹⁵ a convolution kemel of a convolution unit in a convolutional neural network that the first deep leaming model is based on is relativeiy large so that the first deep leaming model extracts an image structure feature of a general facial image, and a convolution kemel of a convolution unit in a convolutional neural network that the second deep leaming model is based on is relativeiy small, so that the second deep leaming model extracts an image material feature of the 20 extracted facial image.
6. 6. The method according to any one of claims 1 to
7. 7, wherein the prédiction scores are ail a probability value or a Boolean value.

   25 7. An apparatus, comprising a plurality of modules confïgured to perform the method of any one of claims 1 to 6.
8. 8. An electronic device, comprising:

   at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores an instruction that can be executed by the at least one processor, and the instruction is 5 executed by the at least one processor, to enable the at least one processor to perform the method of any one of daims 1 to 6.