CN111967319B - Living body detection method, device, equipment and storage medium based on infrared and visible light - Google Patents
Living body detection method, device, equipment and storage medium based on infrared and visible light Download PDFInfo
- Publication number
- CN111967319B CN111967319B CN202010673553.9A CN202010673553A CN111967319B CN 111967319 B CN111967319 B CN 111967319B CN 202010673553 A CN202010673553 A CN 202010673553A CN 111967319 B CN111967319 B CN 111967319B
- Authority
- CN
- China
- Prior art keywords
- face
- detected
- detection
- living body
- pictures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 193
- 238000003860 storage Methods 0.000 title claims abstract description 18
- 230000009471 action Effects 0.000 claims abstract description 33
- 238000007781 pre-processing Methods 0.000 claims abstract description 28
- 238000000034 method Methods 0.000 claims description 28
- 238000012216 screening Methods 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 24
- 238000013145 classification model Methods 0.000 claims description 20
- 238000003708 edge detection Methods 0.000 claims description 20
- 238000001914 filtration Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 238000003331 infrared imaging Methods 0.000 claims description 4
- 210000001508 eye Anatomy 0.000 description 20
- 210000003128 head Anatomy 0.000 description 15
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 230000006872 improvement Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 210000000887 face Anatomy 0.000 description 3
- 238000001574 biopsy Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000007639 printing Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000010146 3D printing Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000000744 eyelid Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000001727 in vivo Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
- G06V40/45—Detection of the body part being alive
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a living body detection method based on infrared and visible light, which comprises the following steps: acquiring a face picture to be detected of a user to be detected; preprocessing a face picture to be detected; performing face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection; judging whether a preset number of face pictures to be detected in a plurality of face pictures to be detected meet preset living body detection conditions according to the face action detection result; if yes, judging that the user to be detected is a living body; if not, judging the user to be detected as a non-living body. The invention also discloses a living body detection device, equipment and a computer readable storage medium based on infrared and visible light. By adopting the embodiment of the invention, the accuracy of living body detection can be improved, the potential safety hazard is reduced, the cooperation of user actions is not needed, and the humanization degree is high.
Description
Technical Field
The invention relates to the field of face recognition, in particular to a living body detection method, device, equipment and storage medium based on infrared and visible light.
Background
With the wide application of face recognition technology, face attack modes of prostheses such as face photos, face videos, three-dimensional masks and the like are layered endlessly, and face living detection is increasingly focused by industry and academia. Face living body detection is also becoming an indispensable link in face recognition systems. The existing living body face detection algorithm generally has the characteristic that the accuracy rate is insufficient to meet the requirement in performance, and the great reason is that the information acquisition source is a visible light camera, and the algorithm is easily influenced by environment and illumination, so that the generalization of the algorithm is poor; for example, based on an optical flow detection mode, the method is suitable for a motion scene of a user, has poor detection effect on a single face picture, and leads to low living body detection accuracy. In addition, the existing living body face detection algorithm can be cracked by certain video synthesis software through synthesizing the face video conforming to the machine instruction to a great extent, and the hidden danger of unsafe exists.
Disclosure of Invention
The embodiment of the invention aims to provide a living body detection method, device, equipment and storage medium based on infrared and visible light, which can improve the accuracy of living body detection, reduce potential safety hazard, and have high humanization degree without cooperation of user actions.
In order to achieve the above object, an embodiment of the present invention provides a living body detection method based on infrared and visible light, including:
acquiring a face picture to be detected of a user to be detected;
preprocessing the face picture to be detected;
performing face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
judging whether a preset number of face pictures to be detected in a plurality of face pictures to be detected meet preset living body detection conditions according to the face action detection result;
if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body;
the preprocessing the face picture to be detected comprises the following steps: sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected; performing edge detection on the face picture to be detected after the preliminary screening is performed; inputting the face picture to be detected after edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics; and inputting the face picture to be detected with the face key point characteristics into a preset final classification model so as to carry out final screening on the face picture to be detected with the face key point characteristics.
As an improvement of the above-mentioned scheme, the initial positioning model includes an infrared positioning model and a visible light positioning model; and sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected, wherein the method comprises the following steps:
sequentially inputting the face pictures to be detected into the infrared positioning model and a preset initial classification model to screen the face pictures to be detected once;
after the face pictures to be detected are screened for the first time, the face pictures to be detected are sequentially input into the visible light positioning model and the preset confidence level filtering model, so that the face pictures to be detected are screened for the second time.
As an improvement of the above solution, after outputting the face picture to be detected with the face key point feature, before inputting the face picture to be detected with the face key point feature into a preset final classification model, the method further includes:
performing quality detection on the face picture to be detected carrying the key point characteristics of the face so as to output the face picture to be detected which accords with the preset quality detection standard;
and carrying out face correction on the face picture to be detected carrying the face key point characteristics after the quality detection by using a similar transformation matrix.
As an improvement of the above-described aspect, when the face motion detection is eye motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, one part of the face pictures to be detected is in an eye opening state, and the other part of the face pictures to be detected is in an eye closing state.
As an improvement of the above-described aspect, when the face motion detection is a mouth motion detection, the living body detection condition is: in the preset number of face pictures to be detected, the mouth opening of the user to be detected exceeds a preset threshold value.
As an improvement of the above-described aspect, when the face motion detection is head motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, the head steering angle of the user to be detected is within a preset angle range.
The embodiment of the invention also provides a living body detection device based on infrared and visible light, which comprises:
the face picture to be detected acquisition module is used for acquiring a face picture to be detected of a user to be detected;
the preprocessing module is used for preprocessing the face picture to be detected;
the face action detection module is used for carrying out face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
the living body judging module is used for judging whether a preset number of face pictures to be detected in the plurality of face pictures to be detected meet preset living body detection conditions; if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body;
the preprocessing the face picture to be detected comprises the following steps: sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected; performing edge detection on the face picture to be detected after the preliminary screening is performed; inputting the face picture to be detected after edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics; and inputting the face picture to be detected with the face key point characteristics into a preset final classification model so as to carry out final screening on the face picture to be detected with the face key point characteristics.
To achieve the above object, an embodiment of the present invention further provides an infrared and visible light-based living body detection device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the infrared and visible light-based living body detection method according to any one of the embodiments described above when executing the computer program.
To achieve the above object, an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium includes a stored computer program, where the computer program controls a device where the computer readable storage medium is located to execute the living body detection method based on infrared and visible light according to any one of the embodiments.
Compared with the prior art, the living body detection method, device, equipment and storage medium based on infrared and visible light disclosed by the embodiment of the invention are used for judging whether the face pictures to be detected meet the preset living body detection condition according to the face action detection result by acquiring the face pictures to be detected of the user to be detected, preprocessing the face pictures to be detected, and carrying out face action detection on a plurality of continuously acquired face pictures to be detected after preprocessing. In the living body detection process of the user, the fact that the eyes, the mouth or the head of the user slightly change due to the fact that the user keeps one gesture for a long time in the camera alignment process is considered, so that the system can automatically capture the face actions, whether the current user is living or not is confirmed according to the detected face action changes, the use experience of the user is fully considered, user action matching is not needed, humanization degree is high, accuracy of living body detection can be improved, and meanwhile potential safety hazards are reduced.
Drawings
FIG. 1 is a flow chart of a living body detection method based on infrared and visible light provided by an embodiment of the invention;
FIG. 2 is a flow chart of another method for detecting living body based on infrared and visible light according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a living body detection device based on infrared and visible light according to an embodiment of the present invention;
fig. 4 is a schematic structural view of an infrared and visible light-based living body detection apparatus according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart of a living body detection method based on infrared and visible light according to an embodiment of the present invention; the living body detection method based on infrared and visible light comprises the following steps:
s1, acquiring a face picture to be detected of a user to be detected;
s2, preprocessing the face picture to be detected;
s3, carrying out face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment;
s4, judging whether a preset number of face pictures to be detected in a plurality of face pictures to be detected meet preset living body detection conditions according to the face action detection result;
s5, if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body.
It should be noted that, the living body detection method based on infrared light and visible light according to the embodiment of the invention can be implemented by executing a processor in the face recognition device, wherein the processor is a front-end device of the face recognition device, integrates multiple functions such as data processing and data communication, and has a powerful service scheduling function and data processing capability.
In the process of training a model, a proper binocular camera is selected, training set samples of various angles and postures of a user are collected in a normal state, and the training set samples comprise real face picture videos shot under a near infrared camera, fake face picture videos shot in the near infrared, fake face picture videos shot in the visible light, real face picture videos shot under the visible light camera and the like. The types of the forged face pictures comprise: the mobile phone shoots the picture and attacks the embedded equipment under the case management scene; printing embedded equipment under a case management scene of the face information attack; an embedded device in a case management scene of mobile phone video playing attack; four types of attack photos such as attack embedded equipment under the 3D printed face picture and infrared and visible light pictures shot by the embedded equipment under the normal face.
All infrared and visible light pictures are classified, and the video is also subjected to frame cutting classification. The first class is mainly used for detecting and classifying RGB & YUV real human body prosthesis, and classifying normal face pictures and pseudo living body attack pictures which are shot under visible light, wherein the attack pictures mainly comprise mobile phone picture attacks shot by embedded equipment, face printing picture attacks shot by the embedded equipment, video playing attack pictures shot by the embedded equipment and 3D printing face attack pictures; the normal living body picture is a positive sample, and the abnormal attack counterfeiting face picture is a negative sample. The second type of normal face picture and pseudo attack picture shot under infrared and visible light double-shot is mainly used for face judgment and judging whether the pictures shot under double-shot are living bodies or not.
Specifically, in step S1, a face picture to be detected of a user to be detected is obtained, where the face picture to be detected may be a picture acquired by a camera of the face recognition device or a picture obtained by framing a video.
Specifically, in step S2, the preprocessing the face picture to be detected includes steps S21 to S24:
s21, sequentially inputting the face pictures to be detected into a preset initial positioning model to perform primary screening on the face pictures to be detected;
s22, carrying out edge detection on the face picture to be detected after the preliminary screening is carried out;
s23, inputting the face picture to be detected subjected to edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics;
s24, inputting the face picture to be detected with the face key point characteristics into a preset final classification model so as to carry out final screening on the face picture to be detected with the face key point characteristics.
Specifically, in step S21, the initial positioning model includes an infrared positioning model and a visible light positioning model; the face pictures to be detected are sequentially input into a preset initial positioning model to perform preliminary screening on the face pictures to be detected, including S211-S212:
s211, sequentially inputting the face pictures to be detected into the infrared positioning model and a preset initial classification model so as to screen the face pictures to be detected once.
The video and the mobile phone photo are collected according to the infrared light to form a black-white light spot, so that a living body and a prosthesis can be distinguished. An infrared positioning model and an initial classification model under an infrared camera are trained in advance, the infrared positioning model adopts a Mobilene_SSD as a backbone network, and the positioning speed and the accuracy are improved by optimizing a network structure; the initial classification model uses Resnet18 as a backbone network and cross entropy as a classification loss function. And detecting all the acquired infrared faces through the infrared positioning model, classifying the faces through the initial classification model, and judging whether the face picture to be detected is a prosthesis or not.
S212, after the face pictures to be detected are screened for the first time, the face pictures to be detected are sequentially input into the visible light positioning model and the preset confidence level filtering model, so that the face pictures to be detected are screened for the second time.
And (3) carrying out visible light RGB face positioning on the non-prosthesis picture separated in the step (S211) by adopting a libfacedetection library, wherein the visible light positioning model can reach 150fps, and the real-time requirement of living body detection can be met. And training an Onet network of the MTCNN in advance by using the acquired face image as a confidence filtering model, and enabling the face result output by the visible light positioning model to pass through the Onet model so as to reduce the false detection rate of face positioning.
Specifically, in step S22, edge detection is performed on the face image to be detected after the preliminary screening is performed. The main principle is that the false body edge texture of the print photo under infrared imaging is less than the normal color format texture and the gray image edge is not obvious. The Laplace detection operator and the canny detection operator are mainly used for further detection and judgment of the human face, whether the human face is a qualified human face area is judged, and abnormal human faces can be filtered, so that subsequent living body misjudgment is reduced.
In the traditional image processing method, the edge part of the image is directly detected, and the face area is determined according to the variance, because the larger the variance is, the richer the image color is, the smaller the variance is, and the less obvious the image is represented; based on the traditional image processing method, the false body edge texture of the print photo in the living body detection is less than the normal color format texture and the gray image edge is not obvious under the infrared imaging, so that the false body and the false body of the print photo can be judged by adopting the edge detection method. In the actual operation of edge detection, the preprocessing operation (detection of a face region, setting of a picture of the same size, graying of the picture, and the like) is required for the picture only so as not to be affected by the size of the image and factors other than the face.
The basic flow of face edge detection for the face picture to be detected in the embodiment of the invention comprises the following steps: firstly, preprocessing an image, reducing the influence of related parameters, and converting the image into a gray level image; and then using gradient operators (such as 3x3 channel and Laplacian operator), and calculating the mean value or mean square error between the pixel points to obtain the face edge information of the image. Taking the variance as the threshold for edge detection, a picture may be considered to be prosthetic if the picture variance is below a predefined threshold.
Specifically, in step S23, the face image to be detected after edge detection is input into a preset face key point positioning model, so as to output the face image to be detected carrying the face key point features. The method comprises the steps of training 68 a face key point positioning model by using a mobilet_v2 in advance, and positioning face key points under visible light, wherein the face key point positioning model uses the mobilet_v2 as a main network of key points, uses depth separable convolution as a feature extraction module, and distributes different weights according to a five-sense organ region and a contour region.
It is worth to say that after outputting the face picture to be detected carrying the key point feature of the face, the method further comprises S231-S232:
and S231, carrying out quality detection on the face picture to be detected carrying the key point characteristics of the face so as to output the face picture to be detected which accords with the preset quality detection standard. Because the face area after the key points of the five sense organs are positioned in the S23 has the problems of large angle, blurring, shielding and incomplete face, the problems affect the living body judgment, the face optimization model is developed, the whole scoring is mainly given out from the aspects of definition, illumination brightness, face size and gesture, shielding objects and the like, and then the scoring standard is analyzed according to the optimization algorithm.
For example, a preferred face procedure is to obtain a photo from a video stream or a photo stream, detect coordinates of five key points of five facial features and mark a size of a face frame by using a face key point detection method for the photo, such as Mtcnn/molblenet. The output information of the face detection is sent to a face preference model, the face preference model is mainly evaluated in terms of image quality to give overall scores, then a face living body detection module selects the picture suitable for being used as the face living body detection according to the scoring standard analyzed by a preference algorithm, if the picture with lower score is directly filtered out, living body judgment is not made, and therefore the accuracy of the face living body detection is improved.
S232, carrying out face correction on the face picture to be detected carrying the face key point characteristics after quality detection by using a similar transformation matrix. And (3) calculating a similarity transformation matrix between two points by using five key point positions such as five officials and the like of the qualified face after the face screening in the step S231, correcting the face by using the similarity transformation matrix, and acquiring the face after alignment.
Specifically, in step S24, the face image to be detected with the face key point features is input into a preset final classification model, so as to perform final screening on the face image to be detected with the face key point features.
Illustratively, a final classification model of the backbone network is pre-trained with mobilet_v1, which mainly further distinguishes between printed pictures and photographed false attack pictures. The final classification model adopts a network after the mobile_v1 clipping quantization, so that the classification speed is increased. The final classification model is trained, and the face classification result under visible light can be improved by using the model.
Specifically, in step S3, face motion detection is performed on a plurality of face images to be detected after pretreatment are continuously acquired; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection.
Specifically, in steps S4 to S5, it is determined, according to the result of face motion detection, whether a preset number of the face images to be detected in the plurality of face images to be detected satisfy a preset living body detection condition. When a preset number of face pictures to be detected meet living body detection conditions, judging that the user to be detected is a living body; and when the face pictures to be detected in the preset number meet living detection conditions, judging that the user to be detected is a non-living body.
In a first alternative embodiment, when the face motion detection is eye motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, one part of the face pictures to be detected is in an eye opening state, and the other part of the face pictures to be detected is in an eye closing state.
The method for determining the eye condition includes counting states of eyes between ten continuous frames of pictures, recording statistics values of ten continuous frames, and determining that the user to be detected is a living body if more than 6 continuous frames in ten continuous frames are open/closed states according to a distance ratio between upper and lower eyelid of pupil distance of eyes calculated by key points calculated by a single frame.
In a second alternative embodiment, when the face motion detection is a mouth motion detection, the living body detection condition is: in the preset number of face pictures to be detected, the mouth opening of the user to be detected exceeds a preset threshold value.
The method includes counting the mouth closing state between ten continuous frames of pictures, recording the continuous ten times of statistical data, and judging the mouth closing state according to the key points calculated in a single frame, wherein the judging method is to calculate the proportion between the central point of the mouth and the upper and lower mouth angles according to the key points calculated in a single frame, if more than 6 continuous frames exist, the mouth opening exceeds a preset threshold value, the user to be detected is judged to be a living body, and otherwise, the user to be detected is judged to be a non-living body.
In a third alternative embodiment, when the face motion detection is head motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, the head steering angle of the user to be detected is within a preset angle range.
For example, the head steering angle between ten consecutive frames of pictures is counted, the statistics data of ten consecutive frames is recorded, euler angle combined with threshold value judgment is adopted, if 6 consecutive frames are within a preset angle range (plus or minus 60 degrees), the user to be detected is judged to be a living body, otherwise, the user to be detected is judged to be a non-living body.
Further, the specific working procedure of the steps S1 to S4 described above may refer to fig. 2.
Compared with the prior art, the living body detection method based on infrared and visible light disclosed by the embodiment of the invention comprises the steps of obtaining the face picture to be detected of a user to be detected, preprocessing the face picture to be detected, and carrying out face action detection on a plurality of continuously obtained face pictures to be detected after preprocessing, so that whether preset living body detection conditions are met in the plurality of face pictures to be detected is judged according to the face action detection result. In the living body detection process of the user, the fact that the eyes, the mouth or the head of the user slightly change due to the fact that the user keeps one gesture for a long time in the camera alignment process is considered, so that the system can automatically capture the face actions, whether the current user is living or not is confirmed according to the detected face action changes, the use experience of the user is fully considered, user action matching is not needed, humanization degree is high, accuracy of living body detection can be improved, and meanwhile potential safety hazards are reduced.
Referring to fig. 3, fig. 3 is a schematic structural diagram of an infrared and visible light-based living body detection apparatus 10 according to an embodiment of the present invention; the infrared and visible light-based living body detection apparatus 10 includes:
the face picture to be detected acquisition module 11 is used for acquiring a face picture to be detected of a user to be detected;
the preprocessing module 12 is used for preprocessing the face picture to be detected;
the face action detection module 13 is used for carrying out face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
the living body judging module 14 is configured to judge whether a preset number of face pictures to be detected in the plurality of face pictures to be detected meet a preset living body detection condition; if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body.
Optionally, the preprocessing module 12 is configured to:
sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected;
performing edge detection on the face picture to be detected after the preliminary screening is performed;
inputting the face picture to be detected after edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics;
and inputting the face picture to be detected with the face key point characteristics into a preset final classification model so as to carry out final screening on the face picture to be detected with the face key point characteristics.
Optionally, the initial positioning model comprises an infrared positioning model and a visible light positioning model; and sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected, wherein the method comprises the following steps:
sequentially inputting the face pictures to be detected into the infrared positioning model and a preset initial classification model to screen the face pictures to be detected once;
after the face pictures to be detected are screened for the first time, the face pictures to be detected are sequentially input into the visible light positioning model and the preset confidence level filtering model, so that the face pictures to be detected are screened for the second time.
Optionally, the preprocessing module 12 is further configured to:
performing quality detection on the face picture to be detected carrying the key point characteristics of the face so as to output the face picture to be detected which accords with the preset quality detection standard;
and carrying out face correction on the face picture to be detected carrying the face key point characteristics after the quality detection by using a similar transformation matrix.
Optionally, when the face motion detection is eye motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, one part of the face pictures to be detected is in an eye opening state, and the other part of the face pictures to be detected is in an eye closing state.
Optionally, when the face motion detection is a mouth motion detection, the living body detection condition is: in the preset number of face pictures to be detected, the mouth opening of the user to be detected exceeds a preset threshold value.
Optionally, when the face motion detection is head motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, the head steering angle of the user to be detected is within a preset angle range.
It should be noted that, the specific working process of each module in the living body detection device 10 based on infrared light and visible light according to the embodiment of the present invention may refer to the working process of the living body detection method, which is not described herein.
Compared with the prior art, the living body detection device 10 based on infrared and visible light disclosed by the embodiment of the invention is used for judging whether the preset living body detection condition is met in a plurality of face pictures to be detected according to the result of the face action detection by acquiring the face pictures to be detected of the user to be detected, preprocessing the face pictures to be detected, and carrying out face action detection on a plurality of face pictures to be detected after preprocessing which are continuously acquired. In the living body detection process of the user, the fact that the eyes, the mouth or the head of the user slightly change due to the fact that the user keeps one gesture for a long time in the camera alignment process is considered, so that the system can automatically capture the face actions, whether the current user is living or not is confirmed according to the detected face action changes, the use experience of the user is fully considered, user action matching is not needed, humanization degree is high, accuracy of living body detection can be improved, and meanwhile potential safety hazards are reduced.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an infrared and visible light-based living body detecting device 20 according to an embodiment of the present invention. The infrared and visible light-based living body detection device 20 includes: a processor 21, a memory 22 and a computer program, such as a travel control program, stored in the memory and executable on the processor. The processor 21, when executing the computer program, implements the steps of the above-described embodiment of the living body detection method based on infrared and visible light, such as steps S1 to S5 shown in fig. 1. Alternatively, the processor may implement the functions of the modules in the above embodiments of the apparatus when executing the computer program, for example, the face image obtaining module 11 to be detected.
Illustratively, the computer program may be split into one or more modules that are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in the infrared and visible light based living body detection device 20. For example, the computer program may be divided into a face picture acquisition module 11 to be detected, a preprocessing module 12, a face motion detection module 13, and a living body judgment module 14, each of which specifically functions as follows:
the face picture to be detected acquisition module 11 is used for acquiring a face picture to be detected of a user to be detected;
the preprocessing module 12 is used for preprocessing the face picture to be detected;
the face action detection module 13 is used for carrying out face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
the living body judging module 14 is configured to judge whether a preset number of face pictures to be detected in the plurality of face pictures to be detected meet a preset living body detection condition; if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body.
The operation of each module may refer to the operation of the living body detection apparatus 10 based on infrared light and visible light described in the above embodiments, and will not be described herein.
The living body detection device 20 based on infrared light and visible light can be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The infrared and visible light based living body detection device 20 may include, but is not limited to, a processor 21, a memory 22. It will be appreciated by those skilled in the art that the schematic diagram is merely an example of an image enhancement device and does not constitute a limitation of the infrared and visible light based biopsy device 20, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the infrared and visible light based biopsy device 20 may also include an input output device, a network access device, a bus, etc.
The processor 21 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 21 is a control center of the infrared and visible light based living body detection device 20, and connects the respective parts of the entire infrared and visible light based living body detection device 20 using various interfaces and lines.
The memory 22 may be used to store the computer program and/or module, and the processor 21 may implement various functions of the infrared and visible light based in vivo detection device 20 by executing or executing the computer program and/or module stored in the memory 22 and invoking data stored in the memory 22. The memory 22 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
Wherein the integrated module of the infrared and visible light based living body detecting device 20 may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that changes and modifications may be made without departing from the principles of the invention, such changes and modifications are also intended to be within the scope of the invention.
Claims (8)
1. An infrared and visible light-based living body detection method, characterized by comprising:
acquiring a face picture to be detected of a user to be detected;
preprocessing the face picture to be detected;
performing face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
judging whether a preset number of face pictures to be detected in a plurality of face pictures to be detected meet preset living body detection conditions according to the face action detection result;
if yes, judging that the user to be detected is a living body;
if not, judging that the user to be detected is a non-living body;
the preprocessing the face picture to be detected comprises the following steps: sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected; performing edge detection on the face picture to be detected after the preliminary screening is performed; inputting the face picture to be detected after edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics; inputting the face picture to be detected with the face key point characteristics into a preset final classification model to carry out final screening on the face picture to be detected with the face key point characteristics;
the initial positioning model comprises an infrared positioning model and a visible light positioning model; and sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected, wherein the method comprises the following steps: sequentially inputting the face pictures to be detected into the infrared positioning model and a preset initial classification model to screen the face pictures to be detected once; after the face picture to be detected is screened for the first time, the face picture to be detected is sequentially input into the visible light positioning model and a preset confidence level filtering model, so that the face picture to be detected is screened for the second time;
the edge detection is carried out on the face picture to be detected after the preliminary screening is carried out, and the basis is that the false body edge texture of the printed photo under infrared imaging is less than the normal color format texture and the gray level image edge is not obvious; the Laplace detection operator and the canny detection operator are mainly used for further detection and judgment of the human face, judging whether the human face is a qualified human face area or not, filtering out an abnormal human face and reducing subsequent living body misjudgment.
2. The method for detecting living body based on infrared and visible light according to claim 1, wherein after outputting the face picture to be detected carrying the face key point feature, before inputting the face picture to be detected carrying the face key point feature into a preset final classification model, the method further comprises:
performing quality detection on the face picture to be detected carrying the key point characteristics of the face so as to output the face picture to be detected which accords with the preset quality detection standard;
and carrying out face correction on the face picture to be detected carrying the face key point characteristics after the quality detection by using a similar transformation matrix.
3. The infrared and visible light-based living body detection method according to claim 1, wherein when the face motion detection is eye motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, one part of the face pictures to be detected is in an eye opening state, and the other part of the face pictures to be detected is in an eye closing state.
4. The infrared and visible light-based living body detection method according to claim 1, wherein when the face motion detection is a mouth motion detection, the living body detection condition is: in the preset number of face pictures to be detected, the mouth opening of the user to be detected exceeds a preset threshold value.
5. The infrared and visible light-based living body detection method according to claim 1, wherein when the face motion detection is head motion detection, the living body detection condition is: and in the preset number of face pictures to be detected, the head steering angle of the user to be detected is within a preset angle range.
6. An infrared and visible light-based living body detection device, comprising:
the face picture to be detected acquisition module is used for acquiring a face picture to be detected of a user to be detected;
the preprocessing module is used for preprocessing the face picture to be detected;
the face action detection module is used for carrying out face action detection on a plurality of continuously acquired face pictures to be detected after pretreatment; wherein the face motion detection includes at least one of eye motion detection, mouth motion detection, and head motion detection;
the living body judging module is used for judging whether a preset number of face pictures to be detected in the plurality of face pictures to be detected meet preset living body detection conditions; if yes, judging that the user to be detected is a living body; if not, judging that the user to be detected is a non-living body;
the preprocessing the face picture to be detected comprises the following steps: sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected; performing edge detection on the face picture to be detected after the preliminary screening is performed; inputting the face picture to be detected after edge detection into a preset face key point positioning model to output the face picture to be detected carrying the face key point characteristics; inputting the face picture to be detected with the face key point characteristics into a preset final classification model to carry out final screening on the face picture to be detected with the face key point characteristics;
the initial positioning model comprises an infrared positioning model and a visible light positioning model; and sequentially inputting the face pictures to be detected into a preset initial positioning model to perform preliminary screening on the face pictures to be detected, wherein the method comprises the following steps: sequentially inputting the face pictures to be detected into the infrared positioning model and a preset initial classification model to screen the face pictures to be detected once; after the face picture to be detected is screened for the first time, the face picture to be detected is sequentially input into the visible light positioning model and a preset confidence level filtering model, so that the face picture to be detected is screened for the second time;
the edge detection is carried out on the face picture to be detected after the preliminary screening is carried out, and the basis is that the false body edge texture of the printed photo under infrared imaging is less than the normal color format texture and the gray level image edge is not obvious; the Laplace detection operator and the canny detection operator are mainly used for further detection and judgment of the human face, judging whether the human face is a qualified human face area or not, filtering out an abnormal human face and reducing subsequent living body misjudgment.
7. An infrared and visible light-based living body detection apparatus, characterized by comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the infrared and visible light-based living body detection method according to any one of claims 1 to 5 when the computer program is executed.
8. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run, controls a device in which the computer readable storage medium is located to perform the infrared and visible light based living body detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010673553.9A CN111967319B (en) | 2020-07-14 | 2020-07-14 | Living body detection method, device, equipment and storage medium based on infrared and visible light |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010673553.9A CN111967319B (en) | 2020-07-14 | 2020-07-14 | Living body detection method, device, equipment and storage medium based on infrared and visible light |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111967319A CN111967319A (en) | 2020-11-20 |
| CN111967319B true CN111967319B (en) | 2024-04-12 |
Family
ID=73362222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010673553.9A Active CN111967319B (en) | 2020-07-14 | 2020-07-14 | Living body detection method, device, equipment and storage medium based on infrared and visible light |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111967319B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699857A (en) * | 2021-03-24 | 2021-04-23 | 北京远鉴信息技术有限公司 | Living body verification method and device based on human face posture and electronic equipment |
| CN113095180B (en) * | 2021-03-31 | 2024-06-11 | 上海商汤智能科技有限公司 | Living body detection method and device, equipment and computer storage medium |
| CN113011385A (en) * | 2021-04-13 | 2021-06-22 | 深圳市赛为智能股份有限公司 | Face silence living body detection method and device, computer equipment and storage medium |
| CN116259091B (en) * | 2023-01-18 | 2023-11-10 | 北京飞腾时光信息科技有限公司 | Method and device for detecting silent living body |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874861A (en) * | 2017-01-22 | 2017-06-20 | 北京飞搜科技有限公司 | A kind of face antidote and system |
| CN106909882A (en) * | 2017-01-16 | 2017-06-30 | 广东工业大学 | A kind of face identification system and method for being applied to security robot |
| CN107862299A (en) * | 2017-11-28 | 2018-03-30 | 电子科技大学 | A kind of living body faces detection method based on near-infrared Yu visible ray binocular camera |
| CN109740472A (en) * | 2018-12-25 | 2019-05-10 | 武汉纺织大学 | A kind of photographic method of anti-eye closing |
| CN110032970A (en) * | 2019-04-11 | 2019-07-19 | 深圳市华付信息技术有限公司 | Biopsy method, device, computer equipment and the storage medium of high-accuracy |
| CN111079688A (en) * | 2019-12-27 | 2020-04-28 | 中国电子科技集团公司第十五研究所 | Living body detection method based on infrared image in face recognition |
| CN111126366A (en) * | 2020-04-01 | 2020-05-08 | 湖南极点智能科技有限公司 | Method, device, equipment and storage medium for distinguishing living human face |
-
2020
- 2020-07-14 CN CN202010673553.9A patent/CN111967319B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106909882A (en) * | 2017-01-16 | 2017-06-30 | 广东工业大学 | A kind of face identification system and method for being applied to security robot |
| CN106874861A (en) * | 2017-01-22 | 2017-06-20 | 北京飞搜科技有限公司 | A kind of face antidote and system |
| CN107862299A (en) * | 2017-11-28 | 2018-03-30 | 电子科技大学 | A kind of living body faces detection method based on near-infrared Yu visible ray binocular camera |
| CN109740472A (en) * | 2018-12-25 | 2019-05-10 | 武汉纺织大学 | A kind of photographic method of anti-eye closing |
| CN110032970A (en) * | 2019-04-11 | 2019-07-19 | 深圳市华付信息技术有限公司 | Biopsy method, device, computer equipment and the storage medium of high-accuracy |
| CN111079688A (en) * | 2019-12-27 | 2020-04-28 | 中国电子科技集团公司第十五研究所 | Living body detection method based on infrared image in face recognition |
| CN111126366A (en) * | 2020-04-01 | 2020-05-08 | 湖南极点智能科技有限公司 | Method, device, equipment and storage medium for distinguishing living human face |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111967319A (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111967319B (en) | Living body detection method, device, equipment and storage medium based on infrared and visible light | |
| EP3611915B1 (en) | Method and apparatus for image processing | |
| Matern et al. | Exploiting visual artifacts to expose deepfakes and face manipulations | |
| CN108229369B (en) | Image shooting method and device, storage medium and electronic equipment | |
| EP3916627A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| CN110084135B (en) | Face recognition method, device, computer equipment and storage medium | |
| CN109697416B (en) | Video data processing method and related device | |
| CN109657554B (en) | Image identification method and device based on micro expression and related equipment | |
| CN109284738B (en) | Irregular face correction method and system | |
| CN108717524B (en) | Gesture recognition system based on double-camera mobile phone and artificial intelligence system | |
| CN108537749B (en) | Image processing method, image processing device, mobile terminal and computer readable storage medium | |
| KR101303877B1 (en) | Method and apparatus for serving prefer color conversion of skin color applying face detection and skin area detection | |
| CN108198130B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| JP4597391B2 (en) | Facial region detection apparatus and method, and computer-readable recording medium | |
| CN109190456B (en) | Multi-feature fusion overlook pedestrian detection method based on aggregated channel features and gray level co-occurrence matrix | |
| CN110543848B (en) | Driver action recognition method and device based on three-dimensional convolutional neural network | |
| CN111652082A (en) | Face living body detection method and device | |
| CN112069887A (en) | Face recognition method, face recognition device, terminal equipment and storage medium | |
| Kim et al. | Exposing fake faces through deep neural networks combining content and trace feature extractors | |
| CN112633221A (en) | Face direction detection method and related device | |
| CN111860316A (en) | Driving behavior recognition method and device and storage medium | |
| CN113781421A (en) | Underwater-based target identification method, device and system | |
| CN112434647A (en) | Human face living body detection method | |
| CN110738607A (en) | Method, device and equipment for shooting driving license based on artificial intelligence and storage medium | |
| CN110610131A (en) | Method and device for detecting face motion unit, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |