CN113673378A - Face recognition method and device based on binocular camera and storage medium - Google Patents
Face recognition method and device based on binocular camera and storage medium Download PDFInfo
- Publication number
- CN113673378A CN113673378A CN202110889007.3A CN202110889007A CN113673378A CN 113673378 A CN113673378 A CN 113673378A CN 202110889007 A CN202110889007 A CN 202110889007A CN 113673378 A CN113673378 A CN 113673378A
- Authority
- CN
- China
- Prior art keywords
- face
- image
- camera
- acquiring
- face recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The application discloses a face recognition method, a face recognition device and a storage medium based on a binocular camera, wherein the face recognition method based on the binocular camera comprises the following steps: acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera; and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information. Through adopting the binocular camera to carry out face recognition, attacks such as photos, videos and 3D printing can be effectively resisted, and therefore safety of face recognition is improved.
Description
Technical Field
The present application relates to the field of face recognition technologies, and in particular, to a face recognition method and apparatus based on a binocular camera, and a storage medium.
Background
At present, a monocular camera is usually adopted for face recognition, but the recognition effect of the monocular camera is poor, and attacks such as photos, videos and 3D printing cannot be effectively resisted, so that the problem of low recognition safety exists when the monocular camera is adopted for face recognition.
Disclosure of Invention
The embodiment of the application aims to solve the problem of low recognition safety when a monocular camera is adopted to recognize the human face by providing a human face recognition method, a human face recognition device and a storage medium based on the binocular camera.
In order to achieve the above object, an aspect of the present application provides a face recognition method based on a binocular camera, where the method includes:
acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera;
and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information.
Optionally, the step of obtaining the face key point information of the second image and performing face recognition according to the face key point information includes:
acquiring face key point information of the second image in a face detection frame of the second image, and aligning face images according to the face key point information to obtain a target face image;
and carrying out face recognition on the target face image.
Optionally, the step of performing face recognition on the target face image includes:
acquiring facial features according to the face key point information in the target face image, and judging whether the face of the target face image is shielded or not according to the facial features;
when shielding exists, acquiring shielding area;
and when the shielding area is smaller than a preset value, carrying out face recognition according to the facial features.
Optionally, before the step of obtaining the face key point information of the second image, the method includes:
acquiring the face size in the detected face image according to the first image;
acquiring relative position information between the infrared camera and the image camera, and acquiring the resolution ratios of the infrared camera and the image camera;
acquiring position information of a face detection frame of the first image;
and determining target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution and the position information of the face detection frame of the first image, and displaying the face detection frame in the second image according to the target position information.
Optionally, the step of determining the target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution, and the position information of the face detection frame of the first image includes:
acquiring a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution;
acquiring a length value and a height value in the face size;
multiplying the first horizontal position offset by the length value to obtain a second horizontal position offset between the infrared camera and the image camera, and multiplying the first vertical position offset by the height value to obtain a second vertical position offset between the infrared camera and the image camera;
and adjusting the position information of the face detection frame of the first image according to the second horizontal position offset and the second vertical position offset to obtain the target position information of the face detection frame of the second image.
Optionally, the step of obtaining a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution includes:
acquiring left and right position offset and up and down position offset between the infrared camera and the image camera according to the relative position information;
acquiring a pixel value in a horizontal direction and a pixel value in a vertical direction in the resolution;
and dividing the left and right position offset by the pixel value in the horizontal direction to obtain the first horizontal position offset, and dividing the up and down position offset by the pixel value in the vertical direction to obtain the first vertical position offset.
Optionally, the steps of obtaining a first image of an infrared camera in a binocular camera and a second image of an image camera, and obtaining face key point information of the second image when it is determined that a detected face is a living body according to the first image, and performing face recognition according to the face key point information further include:
carrying out face detection on the first image by adopting a preset face detection model;
when a human face is detected, inputting the first image into a preset living body detection model to obtain a living body detection result;
and when the living body detection result is a preset result, determining the detected human face as a living body.
In addition, in order to achieve the above object, another aspect of the present application further provides a face recognition device based on a binocular camera, the device includes a memory, a processor, and a face recognition program stored in the memory and running on the processor, wherein the processor implements the steps of the face recognition method based on the binocular camera when executing the face recognition program based on the binocular camera.
In addition, in order to achieve the above object, another aspect of the present application further provides a face recognition apparatus based on a binocular camera, the apparatus includes an obtaining module and a recognition module, wherein:
the acquisition module is used for acquiring a first image of an infrared camera in the binocular camera and a second image of the image camera;
and the identification module is used for acquiring the face key point information of the second image when the detected face is determined to be a living body according to the first image, and carrying out face identification according to the face key point information.
In addition, in order to achieve the above object, another aspect of the present application further provides a terminal, where the terminal includes a memory, a processor, and a face recognition program stored in the memory and running on the processor, and the processor implements the steps of the face recognition method based on the binocular camera when executing the face recognition program based on the binocular camera.
In addition, in order to achieve the above object, another aspect of the present application further provides a storage medium, on which a face recognition program based on a binocular camera is stored, and when the face recognition program based on the binocular camera is executed by a processor, the steps of the face recognition method based on the binocular camera are implemented as described above.
The application provides a face recognition method based on a binocular camera, which comprises the steps of obtaining a first image of an infrared camera in the binocular camera and a second image of an image camera; and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information. Through adopting the binocular camera to carry out face recognition, attacks such as photos, videos and 3D printing can be effectively resisted, and therefore safety of face recognition is improved.
Drawings
Fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a first embodiment of a face recognition method based on a binocular camera according to the present application;
fig. 3 is a schematic flow chart of the binocular camera-based face recognition method of the present application before the step of obtaining face key point information of the second image;
fig. 4 is a schematic view of a process of performing face recognition on the target face image in the face recognition method based on the binocular camera according to the present application;
FIG. 5 is a diagram of a face recognition model architecture of the present application;
fig. 6 is a schematic view of an operation flow of the binocular camera-based face recognition method according to the present application;
fig. 7 is a schematic module diagram of the face recognition device based on the binocular camera according to the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The main solution of the embodiment of the application is as follows: acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera; and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information.
Due to the fact that the monocular camera is adopted for face recognition, the recognition effect is poor, attacks such as photos, videos and 3D printing cannot be effectively resisted, and the problem of low recognition safety exists. According to the method, a first image of an infrared camera in a binocular camera and a second image of an image camera are obtained; and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information. Through adopting the binocular camera to carry out face recognition, attacks such as photos, videos and 3D printing can be effectively resisted, and therefore safety of face recognition is improved.
As shown in fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present application.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 does not constitute a limitation of the terminal device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a binocular camera-based face recognition program may be included in a memory 1005, which is a kind of computer-readable storage medium.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for data communication with the background server; the user interface 1003 is mainly used for data communication with a client (user side); the processor 1001 may be configured to invoke a binocular camera based face recognition program in the memory 1005 and perform the following operations:
acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera;
and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information.
Referring to fig. 2, fig. 2 is a schematic flowchart of a first embodiment of a face recognition method based on a binocular camera according to the present application.
The embodiments of the present application provide a binocular camera based face recognition method, and it should be noted that, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different sequence from that here.
The face recognition method based on the binocular camera comprises the following steps:
step S10, acquiring a first image of an infrared camera in the binocular camera and a second image of the image camera;
it should be noted that, the binocular camera is adopted for face recognition in the application, wherein the binocular camera includes an RGB camera and an NIR (modern Near Infrared spectroscopy) camera, the RGB camera is used for collecting visible light photos, and the NIR camera is used for collecting Near external light photos. Compared with a monocular face recognition scheme, the binocular face recognition can effectively resist cheating of photos, videos and the like, has stronger adaptability to factors such as light change, complex background environment and the like, and is suitable for entrance guard management scenes, face recognition gate scenes, face detection tracking scenes and the like. The infrared camera is an NIR camera, the first image is an NIR image, the image camera is an RGB camera, and the second image is an RGB image. Meanwhile, the face recognition method based on the binocular camera is analyzed and explained by taking the application to an access control management scene as an example.
In this embodiment, install two mesh cameras on the access control equipment, carry out image acquisition to the specified area through two mesh cameras, for example, when the user opens the door, the people's face can be in two mesh cameras's collection within range, at this moment, read the video stream of RGB camera and NIR camera, acquire RGB image and NIR image at the same moment in step, keep in the RGB image, NIR image transmission to the treater with gathering through wired or wireless data transmission mode, carry out analysis processes by the treater to the facial image information of gathering.
And step S20, when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information.
It should be noted that, in the current scheme of performing face recognition by using a binocular camera, face detection is performed twice on an RGB image and an NIR image, and after a face is detected, living body detection is performed on the face, so that performing face detection twice on the RGB image and the NIR image causes time-consuming superposition of detection algorithms, and meanwhile, invalid face recognition operations are performed for many times under the condition that the living body detection is not passed, so that real-time experience of a user is seriously affected under the condition that the quality of the acquired face image is not high. Based on the problems, the face detection prejudgment and the living body detection are directly carried out based on the NIR image, the face key points of the RGB image are regressed after the translation of the detection result based on the NIR image, and then the face alignment and the face recognition are carried out based on the key point information.
In this embodiment, the processor of the access control device includes a face detection module and a living body detection module, and when receiving the NIR image, the NIR image is input into a pre-trained face detection model, and a detection result output by the face detection model is obtained. For example, face detection training data is constructed, the training data includes NIR images, a face detection model is constructed according to the face detection training data, wherein the face detection model is not limited to known Adaboost, MTCNN, Cascade CNN, RCNN series, SSD, and YOLO series models, and then the trained face detection model is used to perform face detection on the NIR images. As the NIR image is filtered based on the infrared camera, the background is relatively simple, the influence on the face imaging in dark light and strong light scenes is small, the detection rate can be improved, the complexity of a detection network can be reduced, and the time consumption is reduced. The installation environment of the binocular camera may be in a strong backlight or a scene with low ambient brightness, and when the binocular camera is in a backlight or a dark background environment, the accuracy of the living body detection is affected. Therefore, when a human face is detected, the ambient brightness of the current scene is obtained, then the exposure parameters of the infrared camera are adjusted according to the ambient brightness, for example, the ambient brightness of the current scene is detected through the brightness sensor, if the current ambient brightness is darker, the binocular camera is currently in backlight or at night, and at the moment, the white light supplement lamp and the infrared supplement lamp are automatically turned on. Therefore, the exposure effect of the binocular camera in the dark backlight and environment can be improved, and the living body detection recognition rate is improved.
As the NIR camera has the characteristic of no imaging on the electronic screens, the NIR image is used for in vivo detection, so that the attacks of electronic screen pictures and dynamic video prostheses can be directly filtered out. Therefore, when a face is detected in the NIR image, the NIR image is used for live body detection, for example, the NIR image is input to a live body detection model to obtain a live body detection result, and when the live body detection result is a preset result, the detected face is determined to be a live body. In one embodiment, the paper picture prosthesis, the mask prosthesis and the real person picture are adopted to construct living body detection training data, and the training data is input into an SVM classifier to be trained to obtain an SVM classifier training model. During the process of living body detection, firstly calculating a characteristic vector of a gray level image of an NIR image, then inputting the characteristic vector of the gray level image of the NIR image into an SVM classifier training model, if the SVM classifier training model outputs +1, indicating that the source of the NIR image is a living body face, and the face verification is successful; if the training model of the SVM classifier outputs-1, the NIR image is from the photo face, face verification is rejected, and face verification fails. The NIR images can be classified based on deep learning and a classification model developed in the future, so that the human face NIR images can be classified as false bodies or living bodies.
And when the detected face is a living body, extracting key point information in the RGB image, and then carrying out face recognition according to the key point information. In an embodiment, a face detection frame corresponding to an NIR image is obtained, the face detection frame is mapped to an RGB image, and key point information of the RGB image, such as eyes, nose tip, mouth corner points, eyebrows, and contour points of each part of a face, is extracted from the mapped face detection frame. And then, carrying out face recognition according to the facial features corresponding to the key point information, and determining that the face recognition is successful when the facial features are matched with the preset facial features.
In the embodiment, a first image of an infrared camera in a binocular camera and a second image of an image camera are obtained; and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information. Through adopting the binocular camera to carry out face recognition, attacks such as photos, videos and 3D printing can be effectively resisted, and therefore safety of face recognition is improved. Meanwhile, the face detection and the living body detection are directly carried out on the NIR image, the face detection is not required to be respectively carried out on the RGB image and the NIR image, and therefore the face detection accuracy rate under dark light and strong light and the whole system time consumption can be improved.
Further, referring to fig. 3, a second embodiment of the face recognition method based on a binocular camera is provided.
The binocular camera-based face recognition method in the second embodiment is different from the first embodiment in that the step of obtaining the face key point information of the second image comprises the following steps:
step S21, acquiring the face size in the detected face image according to the first image;
step S22, acquiring relative position information between the infrared camera and the image camera, and acquiring the resolution of the infrared camera and the image camera;
step S23, obtaining the position information of the face detection frame of the first image
Step S24, determining target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution, and the position information of the face detection frame of the first image, and displaying the face detection frame in the second image according to the target position information.
It should be noted that, in the prior art, a method for sharing face detection results of images acquired by a binocular camera is provided, in which a second face detection frame of a second image is obtained according to an upper, lower, left and right region extension rule, so that the obtained second face detection frame contains more redundant backgrounds and further affects the accuracy of key points, and a method for directly generating a third face detection frame from key point results lacks robustness and is susceptible to the influence of the accuracy of key point detection and the size of a face. Based on the problems, the method and the device provide a face detection frame sharing mode based on translation transformation, and support that the key points of the face and the fine-tuned face detection frame are output simultaneously according to the face detection frame after translation transformation.
In this embodiment, when initializing the binocular face recognition access control device, relative position information between the NIR camera and the RGB camera is acquired, where the relative position information refers to NIR camera shootingRelative positional offset information between the head and the RGB camera, which can be preset in the hardware ROM at the time of equipment shipment, where the left-right positional offset is set as wbThe offset of the upper and lower positions is hbAnd simultaneously acquiring the resolution ratio w of the image acquired by the camerac*hc，wcIs the pixel value in the horizontal direction, hcThe pixel values in the vertical direction, wherein the resolutions of the NIR camera and the RGB camera are the same. Then, the amount of the lateral position deviation w is calculatedbOffset h of up-down positionbAnd resolution wc*hcCalculating a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera, wherein the first horizontal position offset between the infrared camera and the image camera is as follows:
the first vertical position offset between infrared camera and the image camera is:
meanwhile, a face size (i.e., face size) is obtained from a face image detected from the NIR image, assuming that the detected face size is wf*hf，wfIs a length value, hfFor the height value, then the second horizontal position offset between infrared camera and the image camera is:
the second vertical position offset between infrared camera and the image camera is:
wherein, the upper, lower, left and right regions are expanded by a wf(0 ≦ a ≦ 1), assuming that the coordinates of the upper left corner and the lower right corner of the face frame detected in the NIR image are (x) respectivelylt,ylt)、(xrb,yrb) Then, preliminarily obtaining the coordinate point of the face detection frame of the RGB image by the following formula:
and after calculating to obtain the coordinate point of the face detection frame of the RGB image, displaying the face detection frame in the RGB image according to the left point.
In the embodiment, through a face region sharing mechanism based on a binocular camera picture, hardware information and face region size information are comprehensively utilized to perform mapping transformation on the face detection frame, redundant backgrounds contained in the face detection frame obtained through mapping are reduced, the precision of key point detection is further improved, and the key points and the detection frame can be supported to return at the same time.
Further, referring to fig. 4, a third embodiment of the face recognition method based on a binocular camera is provided.
The binocular camera-based face recognition method in the third embodiment differs from the second embodiment in that the step of performing face recognition on the target face image comprises:
step S25, obtaining facial features according to the face key point information in the target face image, and judging whether the face of the target face image is shielded or not according to the facial features;
step S26, when there is shielding, the shielding area is obtained;
and step S27, when the shielding area is smaller than a preset value, carrying out face recognition according to the facial features.
It should be noted that, in the face recognition, there are some factors, such as a face has a shelter (a hat, a mask, etc.), and once the face is sheltered, some features disappear or are wrong, and the face image features are incomplete, the recognition will fail.
In one embodiment, in order to obtain the shape change of the face image, the face image needs to be aligned with the face image first by comparing corresponding points in different shapes. The face key point information is adopted to carry out face alignment on the RGB image, and the face alignment means that key feature points of the face, such as eyes, nose tips, mouth corner points, eyebrows, contour points of each part of the face and the like, are automatically positioned according to the input face image, and corresponding part features are extracted. For example, affine transformation is performed on the face image according to 5 key points of the face, and the face image is changed to a relatively standard size position through image changes such as scaling, rotation, stretching and the like, for example, the size of the face image is unified to a size (where a is 112), so that the face region to be recognized is more regular, and subsequent matching is facilitated. The affine transformation refers to linear transformation from two-dimensional coordinates to two-dimensional coordinates, and maintains the "straightness" and the "parallelism" of the two-dimensional graph.
Then inputting the aligned face image into a face recognition model, which supports outputting face features and face occlusion simultaneously, and the extracted feature values can be adaptive to the occlusion, and can effectively solve the problem of face brushing in a mask or sunglasses wearing scene, referring to fig. 5, fig. 5 is a face recognition model structure diagram of the present application, and it can be known from fig. 5 that the input face image is divided into 4 parts from top to bottom, and the 4 parts are respectively input into 5 feature extraction modules of the model together with the whole face image, the feature extraction method of each module is not limited to the known LBP (Local Binary Pattern), Gabor (features for describing image texture information), PCA (Principal component analysis), LDA (topic dictionary Allocation, topic model) and the existing deep learning method, finally the features of each channel are fused, and the feature fusion layer can reduce the proportion of the face features having the occlusion part, for example, in the case of wearing a mask, the model focuses more on the characteristics of the eyes and forehead of the face after combining the occlusion information.
The face recognition model extracts facial features from the RGB image, wherein the extracted facial features are facial features corresponding to exposed parts (i.e. parts without occlusion) of the face, so that whether the face of the face image is occluded or not can be known according to the extracted facial features, for example, the extracted facial features are compared with preset extracted facial features to determine unextracted facial features, the parts corresponding to the facial features are occlusion parts, and if mouth features are not extracted, the user wears the mask. And then acquiring a shielding area corresponding to the shielding part, if the shielding area is smaller than a preset shielding area (if the shielding area is smaller than the face 3/4), carrying out face recognition according to the facial features of the naked part, and when the facial features of the naked part are matched with the preset facial features, confirming that the face recognition is successful. Meanwhile, in order to ensure the accuracy of face recognition, a user who has a face seriously shielded to influence recognition is prompted, for example, in the case that the face 3/4 or above has a seriously shielded area, the user is prompted to remove glasses or take off a mask or take off a hat and then recognize the face again.
The feature fusion layer is not limited to the existing multi-strategy feature fusion method developed in the future, and can also be combined with the feature extraction method to carry out end-to-end multi-task training. bi(i belongs to {0,1,2,3}) respectively represents forehead occlusion condition, eye occlusion condition, nose occlusion condition and mouth occlusion condition predicted by the model, wherein b is more than or equal to 0 and less than or equal to bi≤1(biE.g. R), 1 denotes complete occlusion, ci(i ∈ {0,1,2,3}) represents the expected output value of each part, and the occlusion regression task is computed using L2 penalties:
and the method can also be used as secondary classification, the subarea with the area larger than 50% of the face shielding area is judged to be shielded, and a secondary classification task is calculated by using the cross entropy:
combining the two cases can be systematically expressed as follows, where λ is the control switch:
the face feature extraction part adopts ArcFace Softmax loss:
the method maps the face features to a spherical surface, wherein s is the radius of the sphere, cos theta is the cosine value of an included angle between the last layer of weight vector w of the constructed neural network and the corresponding feature vector x, and an angle interval control factor m is added for enabling the interval between different face features to be larger and the distribution between the same face features to be more compact. In summary, the overall loss function can be expressed as:
where N is the number of samples to which training is added per batch.
In order to solve the problem of shielding face images, after the face is subjected to regional extraction of features, face features with shielding self-adaption are output by fusing face shielding information, and meanwhile, the face shielding information is output for face quality judgment, so that the face recognition problem under the shielding condition of a wearing mask or other faces can be effectively solved.
In order to better explain the face recognition method based on the binocular camera of the present application, refer to fig. 6, and fig. 6 is a schematic view of an operation flow of the face recognition method based on the binocular camera of the present application.
In this embodiment, video streams of two cameras are read, an RGB image and an NIR image at the same time are synchronously obtained, the RGB image is temporarily stored, and the NIR image is input to the face detection module. And acquiring a detection result output by the face detection module, acquiring the ambient brightness of the current scene if the detection result is that someone exists, and automatically starting a white light supplement lamp and an infrared supplement lamp at the moment if the current ambient brightness is darker and indicates that the binocular camera is currently in backlight or at night. And meanwhile, performing living body detection on the NIR image, acquiring a face detection frame corresponding to the NIR image when the face is detected to be a living body, mapping the face detection frame to the RGB image, and extracting key point information of the RGB image, such as eyes, a nose tip, a mouth corner point, eyebrow, contour points of all parts of the face and the like, from the mapped face detection frame. Then, face alignment and recognition are performed according to the key points, for example, affine transformation is performed on the face image according to 5 key points of the face, and the face image is changed to a standard size position through image changes such as scaling, rotation and stretching. And then inputting the aligned face image into a face recognition model, wherein the model supports the simultaneous output of face features and the shielding condition of the face, and the extracted feature value can be adaptive to the shielding condition, so that the face brushing problem in a mask wearing scene or a sunglasses wearing scene can be effectively solved. If the shielding area of the user face is smaller than the preset shielding area, the situation that the face is seriously shielded does not exist, face recognition is carried out according to the facial features of the exposed part of the face, and when the facial features are matched with the preset facial features, the face recognition is determined to be successful. If the face is seriously shielded to influence the identification, prompt information needs to be output, for example, in the case that the face 3/4 and above areas are seriously shielded, a user is prompted to remove glasses or remove a mask or remove a hat and then the recognition is carried out.
According to the embodiment, the binocular camera is adopted for face recognition, attacks such as photos, videos and 3D printing can be effectively resisted, and therefore the safety of face recognition is improved. Meanwhile, the human face detection and the living body detection are directly carried out on the NIR image, so that the whole human face recognition scheme only needing one-step human face detection is realized, the single human face recognition time consumption of the whole embedded terminal can be controlled within 200ms, and the human face detection accuracy rate and the whole system time consumption under the dark light and the strong light are improved simultaneously.
In addition, the application still provides a face recognition device based on binocular camera, the device includes memory, treater and storage on the memory and on the treater runs face identification program based on binocular camera, realize as above when treater execution face identification program based on binocular camera the step of face recognition method based on binocular camera.
Referring to fig. 7, the binocular camera-based face recognition apparatus 100 includes an acquisition module 10 and a recognition module 20, wherein:
the acquisition module 10 is configured to acquire a first image of an infrared camera in a binocular camera and a second image of an image camera;
the recognition module 20 is configured to, when it is determined that the detected face is a living body according to the first image, acquire face key point information of the second image, and perform face recognition according to the face key point information.
In addition, the application also provides a storage medium, wherein the storage medium is stored with a face recognition program based on the binocular camera, and the steps of the face recognition method based on the binocular camera are realized when the face recognition program based on the binocular camera is executed by a processor.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
While alternative embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following appended claims be interpreted as including alternative embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A face recognition method based on a binocular camera is characterized by comprising the following steps:
acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera;
and when the detected face is determined to be a living body according to the first image, acquiring face key point information of the second image, and performing face recognition according to the face key point information.
2. The binocular camera-based face recognition method of claim 1, wherein the step of obtaining face key point information of the second image and performing face recognition according to the face key point information comprises:
acquiring face key point information of the second image in a face detection frame of the second image, and aligning face images according to the face key point information to obtain a target face image;
and carrying out face recognition on the target face image.
3. The binocular camera based face recognition method of claim 2, wherein the step of performing face recognition on the target face image comprises:
acquiring facial features according to the face key point information in the target face image, and judging whether the face of the target face image is shielded or not according to the facial features;
when shielding exists, acquiring shielding area;
and when the shielding area is smaller than a preset value, carrying out face recognition according to the facial features.
4. The binocular camera based face recognition method of claim 2, wherein the step of obtaining the face keypoint information of the second image is preceded by:
acquiring the face size in the detected face image according to the first image;
acquiring relative position information between the infrared camera and the image camera, and acquiring the resolution ratios of the infrared camera and the image camera;
acquiring position information of a face detection frame of the first image;
and determining target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution and the position information of the face detection frame of the first image, and displaying the face detection frame in the second image according to the target position information.
5. The binocular camera based face recognition method of claim 4, wherein the determining of the target position information of the face detection frame of the second image based on the face size, the relative position information, the resolution, and the position information of the face detection frame of the first image comprises:
acquiring a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution;
acquiring a length value and a height value in the face size;
multiplying the first horizontal position offset by the length value to obtain a second horizontal position offset between the infrared camera and the image camera, and multiplying the first vertical position offset by the height value to obtain a second vertical position offset between the infrared camera and the image camera;
and adjusting the position information of the face detection frame of the first image according to the second horizontal position offset and the second vertical position offset to obtain the target position information of the face detection frame of the second image.
6. The binocular camera based face recognition method of claim 5, wherein the step of acquiring a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution comprises:
acquiring left and right position offset and up and down position offset between the infrared camera and the image camera according to the relative position information;
acquiring a pixel value in a horizontal direction and a pixel value in a vertical direction in the resolution;
and dividing the left and right position offset by the pixel value in the horizontal direction to obtain the first horizontal position offset, and dividing the up and down position offset by the pixel value in the vertical direction to obtain the first vertical position offset.
7. The binocular camera based face recognition method of claim 1, wherein between the steps of acquiring a first image of an infrared camera in a binocular camera and a second image of an image camera and acquiring face key point information of the second image when it is determined that the detected face is a living body according to the first image, and performing face recognition according to the face key point information, further comprising:
carrying out face detection on the first image by adopting a preset face detection model;
when a human face is detected, inputting the first image into a preset living body detection model to obtain a living body detection result;
and when the living body detection result is a preset result, determining the detected human face as a living body.
8. A binocular camera based face recognition apparatus, comprising a memory, a processor and a face recognition program stored on the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the face recognition program.
9. The utility model provides a face identification device based on binocular camera which characterized in that, the device is including obtaining module and identification module, wherein:
the acquisition module is used for acquiring a first image of an infrared camera in the binocular camera and a second image of the image camera;
and the identification module is used for acquiring the face key point information of the second image when the detected face is determined to be a living body according to the first image, and carrying out face identification according to the face key point information.
10. A storage medium having stored thereon a binocular camera based face recognition program, which when executed by a processor implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110889007.3A CN113673378A (en) | 2021-08-02 | 2021-08-02 | Face recognition method and device based on binocular camera and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110889007.3A CN113673378A (en) | 2021-08-02 | 2021-08-02 | Face recognition method and device based on binocular camera and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113673378A true CN113673378A (en) | 2021-11-19 |
Family
ID=78541290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110889007.3A Pending CN113673378A (en) | 2021-08-02 | 2021-08-02 | Face recognition method and device based on binocular camera and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673378A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114519888A (en) * | 2022-02-22 | 2022-05-20 | 平安科技（深圳）有限公司 | Binocular camera-based face frame acquisition method, system, device and medium |
-
2021
- 2021-08-02 CN CN202110889007.3A patent/CN113673378A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114519888A (en) * | 2022-02-22 | 2022-05-20 | 平安科技（深圳）有限公司 | Binocular camera-based face frame acquisition method, system, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107818305B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN104951773B (en) | A kind of real-time face recognition monitoring system | |
| CN105138954B (en) | A kind of image automatic screening inquiry identifying system | |
| JP5629803B2 (en) | Image processing apparatus, imaging apparatus, and image processing method | |
| CN110807364B (en) | Modeling and capturing method and system for three-dimensional face and eyeball motion | |
| EP1650711B1 (en) | Image processing device, imaging device, image processing method | |
| JP4743823B2 (en) | Image processing apparatus, imaging apparatus, and image processing method | |
| CN108764058B (en) | Double-camera face in-vivo detection method based on thermal imaging effect | |
| CN112215180B (en) | Living body detection method and device | |
| RU2431190C2 (en) | Facial prominence recognition method and device | |
| CN109858439A (en) | A kind of biopsy method and device based on face | |
| Guo et al. | Improving face anti-spoofing by 3D virtual synthesis | |
| JP2006293644A (en) | Information processing device and information processing method | |
| CN106529494A (en) | Human face recognition method based on multi-camera model | |
| CN109858375A (en) | Living body faces detection method, terminal and computer readable storage medium | |
| CN109325408A (en) | A kind of gesture judging method and storage medium | |
| WO2023279799A1 (en) | Object identification method and apparatus, and electronic system | |
| Yu et al. | A video-based facial motion tracking and expression recognition system | |
| CN113673378A (en) | Face recognition method and device based on binocular camera and storage medium | |
| CN112488165A (en) | Infrared pedestrian identification method and system based on deep learning model | |
| Moreira et al. | Eyes and eyebrows detection for performance driven animation | |
| Zhang et al. | An approach of region of interest detection based on visual attention and gaze tracking | |
| Chintalapati et al. | Illumination, expression and occlusion invariant pose-adaptive face recognition system for real-time applications | |
| Kalavdekar Prakash | Face detection using neural network | |
| CN113901423B (en) | Intelligent security equipment control method and system based on face recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |