CN113705460A - Method, device and equipment for detecting opening and closing of eyes of human face in image and storage medium - Google Patents
Method, device and equipment for detecting opening and closing of eyes of human face in image and storage medium Download PDFInfo
- Publication number
- CN113705460A CN113705460A CN202111003044.6A CN202111003044A CN113705460A CN 113705460 A CN113705460 A CN 113705460A CN 202111003044 A CN202111003044 A CN 202111003044A CN 113705460 A CN113705460 A CN 113705460A
- Authority
- CN
- China
- Prior art keywords
- image
- eye
- face
- detected
- human face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 claims abstract description 100
- 238000010586 diagram Methods 0.000 claims abstract description 51
- 238000013145 classification model Methods 0.000 claims abstract description 27
- 238000012216 screening Methods 0.000 claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 210000002569 neuron Anatomy 0.000 claims description 9
- 210000004279 orbit Anatomy 0.000 claims description 8
- 230000003213 activating effect Effects 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000012549 training Methods 0.000 description 23
- 238000013528 artificial neural network Methods 0.000 description 17
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to artificial intelligence and digital medical technology, and discloses a method for detecting the opening and closing of eyes of a human face in an image, which comprises the following steps: executing face detection on an image to be detected by using a face detection model to obtain the face size, the face center offset and the heat of each pixel point; screening out a target image according to the heat, and calculating a human face block diagram in the target image according to the human face size and the human face center offset in the target image; detecting the coordinates of key points of the face in a face frame diagram by using a key point detection model; extracting coordinates of left and right eye corner points from the coordinates of the key points of the human face to obtain left and right eye images; and (4) carrying out classification detection on the left eye image and the right eye image by using an eye state classification model to obtain eye state classes. In addition, the invention also relates to a block chain technology, such as a face detection model which can be stored in the nodes of the block chain. The invention also provides a device, equipment and medium for detecting the eyes of the human face open and the eyes of the human face closed in the image. The invention can improve the accuracy rate of eye state detection in the image.
Description
Technical Field
The invention relates to the technical field of artificial intelligence and digital medical treatment, in particular to a method and a device for detecting the eyes opening and closing of a human face in an image, electronic equipment and a computer-readable storage medium.
Background
In the human image acquisition process, the acquired human face image is unqualified due to the condition that the user is not concentrated and the like, for example, the user closes the eye, which brings difficulty to the subsequent human face recognition. Therefore, it is important to effectively determine the eye opening and closing state of the acquired face image under complicated conditions.
Methods commonly used in the industry today, such as calculating eye-opening and eye-closing states based on distances of key points of a human face and eye-opening and eye-closing classification and judgment based on deep learning, are easily affected by inaccurate positioning of a human face frame and the key points, and bring errors to subsequent eye state classification.
Disclosure of Invention
The invention provides a method and a device for detecting the eyes of a human face in an image and a computer-readable storage medium, and mainly aims to solve the problem of inaccurate detection of eye states in the image.
In order to achieve the above object, the present invention provides a method for detecting the opening and closing of eyes of a human face in an image, comprising:
acquiring an image set to be detected, and calculating image characteristics in each image to be detected in the image set to be detected by using a pre-constructed face detection model;
screening out a target image from the image set to be detected according to the image characteristics and calculating a human face block diagram in the target image;
detecting the coordinates of the key points of the face in the face block diagram by using a pre-constructed key point detection model;
extracting coordinates of left and right canthus points from the coordinates of the key points of the human face, and performing outward expansion on the coordinates of the left and right canthus points to obtain left and right eye frames;
and cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and carrying out classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
Optionally, the method for calculating the image characteristics of each image to be detected in the image set to be detected by using a pre-constructed face detection model to obtain the face size, the face center offset and the heat of each pixel point in each image to be detected includes:
performing data enhancement processing on each image to be detected in the image set to be detected to obtain a standard image set to be detected;
counting pixel values of all pixel points of each standard image to be detected in the standard image set to be detected one by one to obtain a pixel matrix of each standard image to be detected;
performing convolution, pooling and activation processing on the pixel matrix by using the face detection model to obtain the heat of each pixel point in the standard image to be detected;
selecting pixel points with the heat degree of each pixel point in the standard image to be detected larger than a preset threshold value as face pixel points, and calculating the face size of the standard image to be detected according to the face pixel points;
calculating to obtain the human face center offset in the standard image to be detected according to the human face pixel points;
and summarizing the heat degree, the face size and the face center offset of each pixel point to obtain the image characteristics of each image to be detected.
Optionally, the performing data enhancement processing on each image to be detected in the image set to be detected to obtain a standard image set to be detected includes:
randomly cutting the images to be detected of the image set to be detected one by one;
and carrying out random color dithering on the cut image to be detected, and summarizing the cut and dithered image to obtain a standard image set to be detected.
Optionally, the screening out a target image from the to-be-detected image set according to the image features includes:
averaging the heat degrees of all pixel points of each image to be detected in the image set to be detected to obtain a heat degree average value;
and screening the image with the average heat value larger than a preset threshold value as a target image.
Optionally, the detecting, by using a pre-constructed key point detection model, the coordinates of the key points of the face in the face frame diagram includes:
extracting features of the face frame diagram by using each convolution layer of the key point detection model to obtain key point feature information;
and activating the key point feature information by using a regressor to obtain the coordinates of the key points of the human face.
Optionally, the extracting coordinates of left and right eye corner points from the coordinates of the face key points includes:
extracting the position information of each key point from the key point feature information corresponding to the face key point coordinates, and generating a key point data table according to the position information;
constructing an index of the key point data table by using a preset index function;
and searching in the index by using a preset left eye corner point coordinate label and a preset right eye corner point coordinate label, and using the position information obtained by searching as left and right eye point coordinates.
Optionally, the classifying and detecting the left-eye image and the right-eye image by using the pre-constructed eye state classification model to obtain the eye state classification includes:
counting the pixel values of all pixel points in the left eye image and the right eye image to respectively obtain a pixel matrix of the left eye image and a pixel matrix of the right eye image;
converting the pixel matrix of the left eye image and the pixel matrix of the right eye image into one-dimensional matrixes respectively by using the eye state classification model to obtain the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image;
performing convolution and full connection operation on the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image for preset times by using a plurality of neurons of a hidden layer in the eye state classification model to obtain left and right eye state information;
activating the left and right eye state information by using two classifiers to obtain the eye opening state probability and the eye closing state probability of the left and right eyes;
if the eye opening state probability is larger than the eye closing state probability, determining the eye state categories of the left eye and the right eye to be eye opening;
and if the eye opening state probability is smaller than or equal to the eye closing state probability, determining the eye state categories of the left eye and the right eye as eye closing.
In order to solve the above problem, the present invention also provides an apparatus for detecting an open eye and a closed eye of a human face in an image, the apparatus comprising:
the image detection module is used for calculating the image characteristics of each image to be detected in the image set to be detected by utilizing a pre-constructed human face detection model;
the human face block diagram acquisition module is used for screening out a target image from the image set to be detected according to the image characteristics and calculating a human face block diagram in the target image;
the face key point acquisition module is used for detecting the coordinates of the face key points in the face frame diagram by utilizing a pre-constructed key point detection model;
the left and right eye socket acquisition module is used for extracting left and right eye corner point coordinates from the key point coordinates of the human face and obtaining left and right eye frames by performing external expansion on the left and right eye corner point coordinates;
and the eye state acquisition module is used for cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and classifying and detecting the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the above-described method of detecting an open eye or closed eye of a human face in an image.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, the at least one computer program being executed by a processor in an electronic device to implement the method for detecting the eyes open and closed of the human face in the image.
The embodiment of the invention utilizes the face detection model to detect the image to be detected to obtain the face frame diagram, and then utilizes the key point detection model to position the face and extract key points of the face from the face frame diagram, so that the eye area of the face can be more accurately obtained and the false detection rate is lower; the eye state classification model is adopted to realize eye image state classification of the eye expanding region, so that the detection is more accurate and the robustness is high. Therefore, the method, the device, the electronic equipment and the computer-readable storage medium for detecting the eyes of the human face in the image can solve the problem of inaccurate detection of the eye state in the image.
Drawings
Fig. 1 is a schematic flow chart of a method for detecting an open eye and a closed eye of a human face in an image according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a process of calculating image features in an image to be detected according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart illustrating a method for determining eye state category according to an embodiment of the present invention;
fig. 4 is a functional block diagram of an apparatus for detecting open eyes and closed eyes of a human face in an image according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device for implementing the method for detecting whether the eyes of the human face are open or closed in the image according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a method for detecting the opening and closing of eyes of a human face in an image. The execution subject of the method for detecting the open eyes and the closed eyes of the human face in the image includes, but is not limited to, at least one of a server, a terminal and other electronic devices that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for detecting the eyes open and eyes closed of the face in the image may be performed by software or hardware installed in the terminal device or the server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
Referring to fig. 1, a schematic flow chart of a method for detecting an open eye and a closed eye of a human face in an image according to an embodiment of the present invention is shown. In this embodiment, the method for detecting the opening and closing of the eyes of the human face in the image includes:
s1, acquiring an image set to be detected, and calculating the image characteristics of each image to be detected in the image set to be detected by using a pre-constructed human face detection model;
in the embodiment of the invention, the image characteristics comprise the face size, the face center offset and the heat of each pixel point, and the face size comprises the width and the height of a face area and is the size of a face frame of the image to be detected; the heat degree of each pixel point is the heat degree of each pixel point of the image to be detected, which is a face pixel point; and the human face center offset is the offset generated between the heat mapping of the human face center pixel point of the image to be detected and the real human face center pixel point of the image to be detected.
In the embodiment of the invention, the pre-constructed face detection model can be composed of a MobileNet V2 neural network and a UNet neural network with improved structures.
In detail, the structure-improved MobileNetV2 neural network is a lightweight convolutional neural network, the rear three-layer network structure of the complete MobileNetV2 neural network is removed from the network, a Linear Bottleneck Block (Linear bottle) and an Inverted Residual error Block (Inverted Residual) of the complete MobileNetV2 neural network are reserved, and the two modules can improve the feature expression capability of the network, so that the accuracy of a face detection model is improved.
Optionally, the UNet neural network adopts a full convolution neural network, and the left convolution network is a feature extraction network: using convolution (conv) and pooling (pooling), the right convolutional network is a feature fusion network: the right convolution network uses the feature map generated by the up-sampling to carry out the layer jump connection (closure) operation with the feature map obtained by the convolution of the left convolution network, and the network is favorable for improving the image processing speed and better retaining the image features.
In the embodiment of the present invention, before the image features in each image to be detected in the image set to be detected are calculated by using the pre-constructed face detection model, the method further includes:
acquiring a preset training image set, and a real face center offset, a real face size and a real heat of each pixel point corresponding to the training image set;
generating a predicted face center offset, a predicted face size and a predicted heat of each pixel point of each training image in the training image set by using a pre-constructed face detection model;
calculating a loss value between the predicted heat of each pixel point and the real heat of each pixel point as a first loss value, calculating a loss value between the real face center offset and the predicted face center offset as a second loss value, and calculating a loss value between the predicted face size and the real face size as a third loss value;
and optimizing the face detection model by using the first loss value, the second loss value and the third loss value to obtain a pre-constructed face detection model.
In the embodiment of the invention, the training image set is images collected in advance before the model is trained. The real face center offset, the real face size and the real heat of each pixel point corresponding to the training image set are manually calibrated by service personnel for training the model.
In this embodiment of the present invention, the optimizing the face detection model by using the first loss value, the second loss value, and the third loss value to obtain a pre-constructed face detection model includes:
calculating a series loss value of the first, second, and third loss values using a series loss function;
when the series loss value is larger than a preset loss threshold value, optimizing the face detection model by using a preset optimization algorithm to obtain an optimized face detection model;
calculating a predicted face center offset, a predicted face size and a predicted heat of each pixel point of each training image in the training image set by using the optimized face detection model, calculating a loss value between the predicted heat of each pixel point and the real heat of each pixel point as a first loss value, calculating a loss value between the real face center offset and the predicted face center offset as a second loss value, calculating a loss value between the predicted face size and the real face size as a third loss value, and returning to the step of calculating a series loss value of the first loss value, the second loss value and the third loss value by using a series loss function;
and when the series loss value is less than or equal to a preset loss threshold value, obtaining a standard face detection model.
In an optional embodiment of the present invention, a minimum loss allocation strategy may be used for model training, that is, for the face real frame of each image, for all output predicted face center offsets, predicted face sizes, and predicted heat of each pixel, only one predicted face center offset, predicted face size, and predicted heat of each pixel with the minimum series loss is selected as a positive sample, and others are all selected as positive samplesIs negative sample, and the iterative training is performed 80 times by using the positive sample and the negative sample until the learning rate is reduced to a predetermined learning rate (e.g. 5 e)-5) And continuously repeating the iteration for 80 times until the parameters of the face detection model are converged to obtain the standard face detection model.
In another optional embodiment of the present invention, when the series loss value is greater than the preset loss threshold, the parameter of the face detection model is optimized by using the Adam optimization algorithm, and the Adam optimization algorithm can adaptively adjust the learning rate in the training process of the face detection model, so that the face detection model is more accurate, and the performance of the face detection model is improved, for example, when the learning rate is reduced to the preset learning rate 5e-5And then, finishing the training of the face detection model to obtain the trained face detection model.
In the embodiment of the present invention, referring to fig. 2, the calculating, by using a pre-constructed face detection model, image features in each to-be-detected image in the to-be-detected image set includes:
s11, performing data enhancement processing on each image to be detected in the image set to be detected to obtain a standard image set to be detected;
s12, counting pixel values of all pixel points of each standard image to be detected in the standard image set to be detected one by one to obtain a pixel matrix of each standard image to be detected;
s13, carrying out convolution, pooling and activation processing on the pixel matrix by using the face detection model to obtain the heat of each pixel point in the standard image to be detected;
s14, counting the pixel points with the heat degree of each pixel point in the standard image to be detected being larger than a preset threshold value as face pixel points, and calculating the face size of the standard image to be detected according to the face pixel points;
s15, calculating to obtain the human face center offset in the standard image to be detected according to the human face pixel points;
and S16, summarizing the heat degree, the face size and the face center offset of each pixel point to obtain the image characteristics of each image to be detected.
Further, the data enhancement processing is performed on each image to be detected in the image set to be detected to obtain a standard image set to be detected, and the method includes:
randomly cutting the images to be detected of the image set to be detected one by one;
and carrying out random color dithering on the cut image to be detected, and summarizing the cut and dithered image to obtain a standard image set to be detected.
In the embodiment of the invention, the random cutting is to cut a plurality of images from one image at random, for example, cutting by a python technology; the random color dithering is a color cross effect which causes adjacent point-like difference by generating displacement on hues forming an image, and comprises random color dithering, random brightness dithering, random saturation dithering, random contrast dithering and the like; the random brightness dithering is an effect of causing brightness light and shade crossing on an image; the random saturation dithering is to generate a saturation difference-like cross effect on the image; the random contrast dithering is a cross effect that produces contrast differences in the contrast of the image.
In an optional embodiment of the invention, because of numerous parameters of the neural network, if training data is not rich enough, the neural network is often over-fitted, the model generalization capability is seriously influenced, the diversity of images can be improved and the data of the images can be enhanced through random clipping processing and random color dithering, and the neural network can also detect the images more accurately, so that the model generalization capability is improved.
S2, screening out a target image from the image set to be detected according to the image characteristics, and calculating a human face block diagram in the target image according to the human face center offset in the target image, the heat of each pixel point and the human face size;
in the embodiment of the invention, the target image is an image containing a human face; the human face block diagram is an image of a framed human face obtained by removing a complex background.
In the embodiment of the present invention, the step of screening out the target image from the to-be-detected image set according to the image characteristics includes:
averaging the heat of all pixel points of each image to be detected in the image set to be detected;
and screening the image with the average value larger than a preset threshold value as a target image.
In the embodiment of the present invention, the calculating a face frame diagram in the target image according to the face center offset, the heat of each pixel point and the face size in the target image includes:
selecting the face pixel points with the heat degree larger than a preset threshold value, and screening the face pixel points to obtain center pixel points;
obtaining a face central point according to the central pixel point and the face central offset;
calculating according to the width and height of the face center point and the face size to obtain a face frame;
and clipping the target image according to the face frame to obtain a face frame diagram.
Further, in an optional embodiment of the present invention, the screening the face pixel to obtain a center pixel includes:
screening out extreme value pixel points of the abscissa and extreme value pixel points of the ordinate from the face pixel points to obtain a first pixel point with the largest abscissa, a second pixel point with the largest ordinate, a third pixel point with the smallest abscissa and a fourth pixel point with the smallest ordinate;
connecting the first pixel point with the third pixel point to obtain a first straight line, and connecting the second pixel point with the fourth pixel point to obtain a second straight line;
and determining a central pixel point of the face pixel point according to the intersection point of the first straight line and the second straight line.
In the embodiment of the present invention, a point of the heat degree of the pixel point, which is greater than a preset threshold (for example, 0.35), may be regarded as a face pixel point.
S3, detecting the coordinates of the key points of the face in the face block diagram by using a pre-constructed key point detection model;
in the embodiment of the invention, the structure of the key point detection model is similar to that of the face detection model, and the key point detection model can be composed of a MobileNet V2 neural network and a UNet neural network with improved structures. And (3) carrying out regression on the input picture through the key point detection model to obtain a plurality of face key points, wherein for example, the key points can be output as 486 key points.
In the embodiment of the present invention, before the detecting the coordinates of the key points of the face in the face frame diagram by using the pre-constructed key point detection model, the method further includes:
acquiring a preset training image set and preset key point coordinates corresponding to the training image set;
generating a predicted key point coordinate of the training image by using a pre-constructed key point detection model;
calculating a loss value between the predicted key point coordinate and the preset key point coordinate to be a key point coordinate loss value;
and optimizing the key point detection model by using the key point coordinate loss value to obtain a pre-constructed key point detection model.
Further, the calculation formula of the coordinate loss value of the key point is as follows:
wherein L isoffIs the loss value of the key point, okIn order to preset the horizontal/vertical coordinate,
the predicted horizontal/vertical coordinates are obtained, N is the number of training image sets, and x is the difference value between the preset horizontal/vertical coordinate true value of the key point and the predicted horizontal/vertical coordinates of the key point.
In the embodiment of the present invention, the step of optimizing the pre-constructed key point detection model is similar to the step and the embodiment of the pre-constructed face detection model, and is not described in detail herein.
In the embodiment of the present invention, the detecting coordinates of the key points of the face in the face frame diagram by using the pre-constructed key point detection model includes:
extracting features of the face frame diagram by using each convolution layer of the key point detection model to obtain key point feature information;
and activating the key point feature information by using a regressor to obtain the coordinates of the key points of the human face.
In the embodiment of the invention, before the feature extraction is carried out on the face frame diagram, the face frame diagram can be expanded, and the coordinate [ x ] of the upper left point of the face frame is used1,y1]And coordinates of lower right point [ x ]2,y2]The face block diagram is expanded for example. Horizontal coordinates of upper left point and lower right point of face frame
By flaring of one quarter of the width w of the detection frame, i.e.
Longitudinal coordinates of upper left point and lower right point of face frame
Extending the face frame by one fourth of the height h of the face frame, i.e.
The detection range of the face is expanded by external expansion, the problem that the extraction of key points is incomplete when the key points of the face are detected due to the fact that the detection result of a face detection model is too small can be solved, and the accuracy of extracting the key points is improved.
Further, the keypoint detection model converts the pixel matrix into a one-dimensional by multi-dimensional matrix; the neurons of the first convolutional layer perform convolution on the one-dimensional multi-dimensional matrix to obtain input of a second convolutional layer, the neurons of the second convolutional layer perform convolution operation on the input of the second convolutional layer, and the neurons of the nth layer (N may be a preset number of convolution layers, for example, 4) perform convolution operation as described above to finally obtain face key point information, wherein the convolution kernel dimension of each convolutional layer is different according to different feature extraction; and activating the key point information through a regressor of the key point detection model to obtain a plurality of key point coordinates, wherein the regressor comprises, but is not limited to Mean-Square-Error and MSE.
In the embodiment of the invention, when the deep neural network detects key points, each layer of neural network can be constructed according to the requirement of key point extraction, and a plurality of neurons are arranged to detect different positions of a human face, so that the accuracy of the detection result is improved.
S4, extracting coordinates of left and right canthus points from the coordinates of the key points of the human face, and performing outward expansion on the coordinates of the left and right canthus points to obtain left and right eye frames;
in the embodiment of the present invention, the left and right eye corner point coordinates may include left eye corner point coordinates and right eye corner point coordinates of a left eye, and left eye corner point coordinates and right eye corner point coordinates of a right eye. The left and right frames obtained by external expansion are image frames containing eyes.
Further, the extracting left and right eye corner point coordinates from the face key point coordinates includes:
extracting the position information of each key point from the key point feature information corresponding to the face key point coordinates, and generating a key point data table according to the position information;
constructing an index of the key point data table by using a preset index function;
and searching in the index by using a preset left eye corner point coordinate label and a preset right eye corner point coordinate label, and using the position information obtained by searching as left and right eye point coordinates.
In the embodiment of the invention, the INDEX can select a CREATE INDEX function to construct the INDEX of the key point data table.
In the embodiment of the invention, taking the left eye as an example, the coordinates [ p ] of the left eye corner point of the left eye are determined1,q1]And the right eye angular point coordinate [ p ]2,q2]Calculating to obtain an external expansion weight l ═ p1-p2The formula for flaring the eye region is as follows:
wherein the content of the first and second substances,
is the abscissa point of the left eye frame after external expansion,
is the vertical coordinate point of the left eye frame after external expansion.
The coordinates of a right lower angular point, a left lower angular point, a right upper angular point and a right upper angular point of the left eye socket are respectively, and the left eye socket can be obtained through the coordinate points. The right eye frame is obtained as described in the above example, and is not described herein again.
S5, cutting left and right eye images from the human face frame diagram according to the left and right eye frames, and carrying out classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
In the embodiment of the invention, the pre-constructed eye state classification model is similar to the construction and training process of the human face detection model and the key point detection model, and can be composed of a MobileNet V2 neural network and a UNet neural network with improved structures.
In an embodiment of the present invention, before the classifying and detecting the left-eye image and the right-eye image by using the pre-constructed eye state classification model, the method further includes:
acquiring a preset training image set and a real eye state corresponding to the training image set;
generating a predicted eye state of the training image by using a pre-constructed eye state classification model;
calculating a loss value between the predicted eye state and the true eye state as an eye state loss value;
and optimizing the eye state classification model by using the eye state loss value to obtain a pre-constructed eye state classification model.
Further, the calculation formula of the key point loss value is as follows:
wherein
As the value of eye state loss, e(i)In order to be the true eye state,
to predict the eye state, N is the number of training images.
In the embodiment of the present invention, the step of optimizing the pre-constructed eye state classification model is similar to the steps and embodiments of the pre-constructed face detection model and the pre-constructed key point detection model, and thus, redundant description is omitted here.
In the embodiment of the invention, computer sentences with data grabbing functions, such as java sentences, python sentences and the like, can be utilized. And grabbing a pre-stored face detection model, a key point detection model and an eye state classification model from a pre-constructed storage area, wherein the storage area comprises but is not limited to a database, a block chain node and a network cache.
In the embodiment of the invention, the regions of the left eye socket and the right eye socket can be cut through the image to be detected, so as to obtain the left eye image and the right eye image.
In the embodiment of the present invention, referring to fig. 3, the classifying and detecting the left-eye image and the right-eye image by using the pre-constructed eye state classification model to obtain the eye state classification includes:
s51, counting pixel values of all pixel points in the left-eye image and the right-eye image to respectively obtain a pixel matrix of the left-eye image and a pixel matrix of the right-eye image;
s52, converting the pixel matrix of the left eye image and the pixel matrix of the right eye image into one-dimensional matrixes respectively by using the eye state classification model to obtain the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image;
s53, performing convolution and full connection operation on the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image for preset times respectively by using a plurality of neurons of a hidden layer in the eye state classification model to obtain left and right eye state information;
s54, activating the left and right eye state information by using a two-classifier to obtain the eye opening state probability and the eye closing state probability of the left and right eyes;
s55, judging whether the eye opening state probability is larger than the eye closing state probability;
if the eye opening state probability is greater than the eye closing state probability, executing S56 and determining the eye state categories of the left eye and the right eye to be eye opening;
if the eye-open state probability is less than or equal to the eye-closing state probability, S57 is executed to determine the eye state categories of the left and right eyes as eye-closing.
In the embodiment of the invention, the eye state detection is carried out through the depth network, two classification results of the open eyes and the closed eyes are finally obtained, the neural network activation function can adopt two classifier functions to calculate the output of the full-connection layer, the probability values of the open eyes and the closed eyes are respectively obtained, and the eye state can be obtained. For example, the first neuron of the full connection layer outputs an eye-open probability value of 0.8 through the two-classifier function, and the second neuron of the full connection layer outputs an eye-closing probability value of 0.2 through the two-classifier function, so that the eye state is obtained as eye-closing.
The embodiment of the invention utilizes the face detection model to detect the image to be detected to obtain the face frame diagram, and then utilizes the key point detection model to position the face and extract key points of the face from the face frame diagram, so that the eye area of the face can be more accurately obtained and the false detection rate is lower; the eye state classification model is adopted to realize eye image state classification of the eye expanding region, so that the detection is more accurate and the robustness is high. Therefore, the method for detecting the eyes of the human face in the image can solve the problem of inaccurate eye state detection in the image.
Fig. 4 is a functional block diagram of an apparatus for detecting open eyes and closed eyes of a human face in an image according to an embodiment of the present invention.
The device 100 for detecting the eyes open and eyes closed of the human face in the image according to the present invention may be installed in an electronic apparatus. According to the implemented functions, the device 100 for detecting the eyes of the human face open and closed in the image may include an image detection module 101, a human face frame diagram acquisition module 102, a human face key point acquisition module 103, a left and right eye socket acquisition module 104, and an eye state acquisition module 105. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the image detection module 101 is configured to acquire an image set to be detected, and calculate image features in each image to be detected in the image set to be detected by using a pre-constructed face detection model;
a face block diagram obtaining module 102, which screens out a target image from the image set to be detected according to the image characteristics and calculates a face block diagram in the target image;
a face key point obtaining module 103, configured to detect coordinates of face key points in the face frame diagram by using a pre-constructed key point detection model;
a left and right eye socket acquisition module 104, configured to extract left and right eye corner point coordinates from the face key point coordinates, and obtain left and right eye frames by performing external expansion on the left and right eye corner point coordinates;
the eye state obtaining module 105 is configured to cut out left and right eye images from the face frame diagram according to the left and right eye frames, and perform classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state categories.
In detail, when the modules in the device 100 for detecting the eyes open and close of the face in the image according to the embodiment of the present invention are used, the same technical means as the method for detecting the eyes open and close of the face in the image described in fig. 1 to fig. 3 are adopted, and the same technical effects can be produced, which is not described herein again.
Fig. 5 is a schematic structural diagram of an electronic device for implementing a method for detecting an open eye and a closed eye of a human face in an image according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a human face open-eye and closed-eye detection program in an image, stored in the memory 11 and executable on the processor 10.
In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions, and includes one or more Central Processing Units (CPUs), a microprocessor, a digital Processing chip, a graphics processor, a combination of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device by running or executing a program or a module (for example, executing a program for detecting whether a person's face is open or closed in an image) stored in the memory 11 and calling data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only to store application software installed in the electronic device and various types of data, such as codes of a program for detecting the opening and closing of the eyes of a person's face in an image, etc., but also to temporarily store data that has been output or is to be output.
The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
The communication interface 13 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.
Fig. 5 only shows an electronic device with components, and it will be understood by a person skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management and the like are realized through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The program for detecting the eyes open and close of the human face in the image stored in the memory 11 of the electronic device 1 is a combination of a plurality of computer programs, and when running in the processor 10, can realize:
obtaining an image set to be detected, and calculating the face size, the face center offset and the heat of each pixel point in each image to be detected in the image set to be detected by using a pre-constructed face detection model;
screening out a target image from the image set to be detected according to the heat, and calculating a face frame diagram in the target image according to the face center offset in the target image, the heat of each pixel point and the face size;
detecting the coordinates of the key points of the face in the face block diagram by using a pre-constructed key point detection model;
extracting coordinates of left and right canthus points from the coordinates of the key points of the human face, and performing outward expansion on the coordinates of the left and right canthus points to obtain left and right eye frames;
and cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and carrying out classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to the drawings, and is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
obtaining an image set to be detected, and calculating the face size, the face center offset and the heat of each pixel point in each image to be detected in the image set to be detected by using a pre-constructed face detection model;
screening out a target image from the image set to be detected according to the heat, and calculating a face frame diagram in the target image according to the face center offset in the target image, the heat of each pixel point and the face size;
detecting the coordinates of the key points of the face in the face block diagram by using a pre-constructed key point detection model;
extracting coordinates of left and right canthus points from the coordinates of the key points of the human face, and performing outward expansion on the coordinates of the left and right canthus points to obtain left and right eye frames;
and cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and carrying out classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A method for detecting an opening and closing of a human face in an image, the method comprising:
acquiring an image set to be detected, and calculating image characteristics in each image to be detected in the image set to be detected by using a pre-constructed face detection model;
screening out a target image from the image set to be detected according to the image characteristics and calculating a human face block diagram in the target image;
detecting the coordinates of the key points of the face in the face block diagram by using a pre-constructed key point detection model;
extracting coordinates of left and right canthus points from the coordinates of the key points of the human face, and performing outward expansion on the coordinates of the left and right canthus points to obtain left and right eye frames;
and cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and carrying out classification detection on the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
2. The method for detecting the eyes open and closed of the human face in the image according to claim 1, wherein the calculating the image characteristics in each image to be detected in the image set to be detected by using the pre-constructed human face detection model comprises:
performing data enhancement processing on each image to be detected in the image set to be detected to obtain a standard image set to be detected;
counting pixel values of all pixel points of each standard image to be detected in the standard image set to be detected one by one to obtain a pixel matrix of each standard image to be detected;
performing convolution, pooling and activation processing on the pixel matrix by using the face detection model to obtain the heat of each pixel point in the standard image to be detected;
selecting pixel points with the heat degree of each pixel point in the standard image to be detected larger than a preset threshold value as face pixel points, and calculating the face size of the standard image to be detected according to the face pixel points;
calculating to obtain the human face center offset in the standard image to be detected according to the human face pixel points;
and summarizing the heat degree, the face size and the face center offset of each pixel point to obtain the image characteristics of each image to be detected.
3. The method for detecting the opening and closing of the eyes of the human face in the image according to claim 2, wherein the step of performing data enhancement processing on each image to be detected in the image set to be detected to obtain a standard image set to be detected comprises the following steps:
randomly cutting the images to be detected of the image set to be detected one by one;
and carrying out random color dithering on the cut image to be detected, and summarizing the cut and dithered image to obtain a standard image set to be detected.
4. The method for detecting the opening and closing of the eyes of the human face in the image according to claim 2, wherein the step of screening out the target image from the image set to be detected according to the image characteristics comprises the following steps:
averaging the heat degrees of all pixel points of each image to be detected in the image set to be detected to obtain a heat degree average value;
and screening the image with the average heat value larger than a preset threshold value as a target image.
5. The method for detecting the eyes open and closed of the human face in the image according to claim 1, wherein the detecting the coordinates of the key points of the human face in the frame diagram of the human face by using the pre-constructed key point detection model comprises:
extracting features of the face frame diagram by using each convolution layer of the key point detection model to obtain key point feature information;
and activating the key point feature information by using a regressor to obtain the coordinates of the key points of the human face.
6. The method for detecting the eyes open and closed of the human face in the image according to claim 5, wherein the extracting the coordinates of the left and right eye corner points from the coordinates of the key points of the human face comprises:
extracting the position information of each key point from the key point feature information corresponding to the face key point coordinates, and generating a key point data table according to the position information;
constructing an index of the key point data table by using a preset index function;
and searching in the index by using a preset left eye corner point coordinate label and a preset right eye corner point coordinate label, and using the position information obtained by searching as left and right eye point coordinates.
7. The method for detecting the eyes of a human face open and close in an image according to any one of claims 1 to 6, wherein the classifying and detecting the left and right eye images by using the pre-constructed eye state classification model to obtain the eye state classes comprises:
counting the pixel values of all pixel points in the left eye image and the right eye image to respectively obtain a pixel matrix of the left eye image and a pixel matrix of the right eye image;
converting the pixel matrix of the left eye image and the pixel matrix of the right eye image into one-dimensional matrixes respectively by using the eye state classification model to obtain the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image;
performing convolution and full connection operation on the one-dimensional matrix of the left eye image and the one-dimensional matrix of the right eye image for preset times by using a plurality of neurons of a hidden layer in the eye state classification model to obtain left and right eye state information;
activating the left and right eye state information by using two classifiers to obtain the eye opening state probability and the eye closing state probability of the left and right eyes;
if the eye opening state probability is larger than the eye closing state probability, determining the eye state categories of the left eye and the right eye to be eye opening;
and if the eye opening state probability is smaller than or equal to the eye closing state probability, determining the eye state categories of the left eye and the right eye as eye closing.
8. An apparatus for detecting the opening of a face and the closing of an eye in an image, the apparatus comprising:
the image detection module is used for acquiring an image set to be detected and calculating image characteristics in each image to be detected in the image set to be detected by utilizing a pre-constructed face detection model;
the human face block diagram acquisition module is used for screening out a target image from the image set to be detected according to the image characteristics and calculating a human face block diagram in the target image;
the face key point acquisition module is used for detecting the coordinates of the face key points in the face frame diagram by utilizing a pre-constructed key point detection model;
the left and right eye socket acquisition module is used for extracting left and right eye corner point coordinates from the key point coordinates of the human face and obtaining left and right eye frames by performing external expansion on the left and right eye corner point coordinates;
and the eye state acquisition module is used for cutting out left and right eye images from the human face frame diagram according to the left and right eye frames, and classifying and detecting the left and right eye images by using a pre-constructed eye state classification model to obtain eye state classes.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform a method of detecting an open eye or closed eye of a human face in an image as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, implements a method for detecting the opening and closing of a human face in an image according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003044.6A CN113705460B (en) | 2021-08-30 | 2021-08-30 | Method, device, equipment and storage medium for detecting open and closed eyes of face in image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003044.6A CN113705460B (en) | 2021-08-30 | 2021-08-30 | Method, device, equipment and storage medium for detecting open and closed eyes of face in image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113705460A true CN113705460A (en) | 2021-11-26 |
| CN113705460B CN113705460B (en) | 2024-03-15 |
Family
ID=78656652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111003044.6A Active CN113705460B (en) | 2021-08-30 | 2021-08-30 | Method, device, equipment and storage medium for detecting open and closed eyes of face in image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113705460B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114283488A (en) * | 2022-03-08 | 2022-04-05 | 北京万里红科技有限公司 | Method for generating detection model and method for detecting eye state by using detection model |
| CN114359645A (en) * | 2022-01-12 | 2022-04-15 | 中国平安人寿保险股份有限公司 | Image expansion method, device, equipment and storage medium based on characteristic region |
| CN115546884A (en) * | 2022-12-01 | 2022-12-30 | 广州视景医疗软件有限公司 | Eyeball tremor identification method, device and equipment |
| CN115690892A (en) * | 2023-01-03 | 2023-02-03 | 京东方艺云(杭州)科技有限公司 | Squinting recognition method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704805A (en) * | 2017-09-01 | 2018-02-16 | 深圳市爱培科技术股份有限公司 | method for detecting fatigue driving, drive recorder and storage device |
| CN108614999A (en) * | 2018-04-16 | 2018-10-02 | 贵州大学 | Eyes based on deep learning open closed state detection method |
| CN110462632A (en) * | 2019-06-27 | 2019-11-15 | 深圳市汇顶科技股份有限公司 | The method, apparatus and electronic equipment of recognition of face |
| CN112541484A (en) * | 2020-12-28 | 2021-03-23 | 平安银行股份有限公司 | Face matting method, system, electronic device and storage medium |
| CN113177515A (en) * | 2021-05-20 | 2021-07-27 | 罗普特科技集团股份有限公司 | Eye movement tracking method and system based on image |
-
2021
- 2021-08-30 CN CN202111003044.6A patent/CN113705460B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704805A (en) * | 2017-09-01 | 2018-02-16 | 深圳市爱培科技术股份有限公司 | method for detecting fatigue driving, drive recorder and storage device |
| CN108614999A (en) * | 2018-04-16 | 2018-10-02 | 贵州大学 | Eyes based on deep learning open closed state detection method |
| CN110462632A (en) * | 2019-06-27 | 2019-11-15 | 深圳市汇顶科技股份有限公司 | The method, apparatus and electronic equipment of recognition of face |
| CN112541484A (en) * | 2020-12-28 | 2021-03-23 | 平安银行股份有限公司 | Face matting method, system, electronic device and storage medium |
| CN113177515A (en) * | 2021-05-20 | 2021-07-27 | 罗普特科技集团股份有限公司 | Eye movement tracking method and system based on image |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359645A (en) * | 2022-01-12 | 2022-04-15 | 中国平安人寿保险股份有限公司 | Image expansion method, device, equipment and storage medium based on characteristic region |
| CN114359645B (en) * | 2022-01-12 | 2024-05-21 | 中国平安人寿保险股份有限公司 | Image expansion method, device, equipment and storage medium based on characteristic area |
| CN114283488A (en) * | 2022-03-08 | 2022-04-05 | 北京万里红科技有限公司 | Method for generating detection model and method for detecting eye state by using detection model |
| CN115546884A (en) * | 2022-12-01 | 2022-12-30 | 广州视景医疗软件有限公司 | Eyeball tremor identification method, device and equipment |
| CN115546884B (en) * | 2022-12-01 | 2023-03-14 | 广州视景医疗软件有限公司 | Eyeball tremor identification method, device and equipment |
| CN115690892A (en) * | 2023-01-03 | 2023-02-03 | 京东方艺云(杭州)科技有限公司 | Squinting recognition method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113705460B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113705460A (en) | Method, device and equipment for detecting opening and closing of eyes of human face in image and storage medium | |
| CN112446919B (en) | Object pose estimation method and device, electronic equipment and computer storage medium | |
| WO2022247005A1 (en) | Method and apparatus for identifying target object in image, electronic device and storage medium | |
| CN112465071A (en) | Image multi-label classification method and device, electronic equipment and medium | |
| WO2022105179A1 (en) | Biological feature image recognition method and apparatus, and electronic device and readable storage medium | |
| WO2021151277A1 (en) | Method and apparatus for determining severity of damage on target object, electronic device, and storage medium | |
| CN112699775A (en) | Certificate identification method, device and equipment based on deep learning and storage medium | |
| CN113033543B (en) | Curve text recognition method, device, equipment and medium | |
| CN113705461B (en) | Face definition detection method, device, equipment and storage medium | |
| CN112528908B (en) | Living body detection method, living body detection device, electronic equipment and storage medium | |
| CN113887408B (en) | Method, device, equipment and storage medium for detecting activated face video | |
| CN111898538A (en) | Certificate authentication method and device, electronic equipment and storage medium | |
| CN112528909A (en) | Living body detection method, living body detection device, electronic apparatus, and computer-readable storage medium | |
| CN113869219A (en) | Face living body detection method, device, equipment and storage medium | |
| CN115471775A (en) | Information verification method, device and equipment based on screen recording video and storage medium | |
| CN113723280A (en) | Static face-based confrontation sample detection method, device, equipment and medium | |
| CN112528903B (en) | Face image acquisition method and device, electronic equipment and medium | |
| CN113792801B (en) | Method, device, equipment and storage medium for detecting face dazzling degree | |
| CN113888086B (en) | Article signing method, device, equipment and storage medium based on image recognition | |
| CN113627394B (en) | Face extraction method and device, electronic equipment and readable storage medium | |
| CN114049676A (en) | Fatigue state detection method, device, equipment and storage medium | |
| CN115114992A (en) | Method, device and equipment for training classification model and storage medium | |
| CN114385815A (en) | News screening method, device, equipment and storage medium based on business requirements | |
| CN114463685A (en) | Behavior recognition method and device, electronic equipment and storage medium | |
| CN113869269A (en) | Activity site congestion degree detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |