CN110175509B - All-weather eye circumference identification method based on cascade super-resolution - Google Patents
All-weather eye circumference identification method based on cascade super-resolution Download PDFInfo
- Publication number
- CN110175509B CN110175509B CN201910281741.4A CN201910281741A CN110175509B CN 110175509 B CN110175509 B CN 110175509B CN 201910281741 A CN201910281741 A CN 201910281741A CN 110175509 B CN110175509 B CN 110175509B
- Authority
- CN
- China
- Prior art keywords
- periocular
- image
- resolution
- super
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Ophthalmology & Optometry (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of pattern recognition and digital image processing, and particularly relates to an all-weather eye circumference recognition method based on cascade super-resolution. Using a multispectral camera to acquire periocular images, preprocessing a limited periocular data set, and expanding samples; performing image super-resolution operation on the periocular data set by adopting a convolutional neural network based on deep learning to amplify the periocular image; performing image restoration on the processed periocular data set by using an image deconvolution technology based on deep learning; based on the periocular data obtained by image restoration, adopting Laplace image sharpening enhancement; constructing a novel stacked neural network model by adopting a deep learning theory based on the obtained periocular data set; dividing the periocular data set into a training set and a test set; and calculating the test set obtained by division through the trained convolutional neural network model to obtain the characteristics. The method has stronger robustness and good generalization performance.
Description
Technical Field
The invention belongs to the technical field of pattern recognition and digital image processing, and particularly relates to an all-weather eye circumference recognition method based on cascade super-resolution.
Background
Periocular identification is a new and particularly advantageous biometric modality that has emerged in recent years. It may for example be used as a complement to face recognition and has been shown to have the most significant information of face parts in case of face occlusion. Furthermore, periocular recognition may also be considered as an alternative to failure of iris recognition, since iris recognition requires the acquisition of iris images of very high quality, which is often not met at long distances.
However, researchers have heretofore focused only on visible light-based periocular identification techniques, which generally do not perform well in harsh climates and environments, such as unconstrained situations and harsh climates with uneven lighting, nighttime, rain and snow. With the emergence of applications in various complex environments in the real world, the eye periphery recognition technology based on visible light is more and more difficult to meet the requirements, for example, the monitoring task in the real world often occurs in poor atmospheric environments such as night, rainy and snowy days, and the acquisition of high-definition face images through visible light in the environment is a task which is difficult to complete. Therefore, new algorithms and systems need to be developed to improve the universality and robustness of eye contour identification. Based on this, the patent proposes in particular a multispectral periocular identification technique using a combination of visible and infrared light. The technology has the advantage of working in all weather, and is suitable for various environments such as daytime, night, sunny days, rain and snow and the like.
On the other hand, since the periocular image is usually captured from the face image, the corresponding area is small, and the image size is small (i.e., the resolution is small). The existing eye contour identification technology usually directly extracts and identifies features of eye contour images with original sizes, and influence of the eye contour image sizes on final identification performance is not considered. Because the high-resolution and high-quality image has significant meaning for improving the identification performance, the patent particularly provides a cascade super-resolution periocular identification technology, and the small-size periocular image is successfully amplified by a plurality of times by introducing the super-resolution, so that the effective area of feature extraction during periocular identification is increased. In addition, the simple super-resolution increases the area around the eye, but has the side effect of reducing the image quality. In order to solve the problem, the eye periphery super-resolution method is further perfected: after super-resolution, a depth learning-based periocular image restoration technology is introduced, and an image enhancement step based on Laplace sharpening is combined. This complete process is called: and (4) cascading eye periphery super-resolution. Finally, the cascade eye periphery super-resolution technology combines the multispectral all-weather characteristics, and the overall name is as follows: an all-weather eye circumference identification method based on cascade super-resolution.
Finally, as the feature extraction algorithm for eye contour identification is the traditional way so far, such as PCA, LBP, WLD, SIFT, etc. The traditional algorithms are all designed manually, are generally complex and complex in design, have good performance but poor robustness for specific situations, and have poor solution to the problems of illumination change and the like. With the further maturity of deep learning, the problem needs to be solved urgently. Automatic feature extraction methods such as convolutional neural networks are simpler and more robust. Therefore, the invention provides a convolutional neural network StackConvNet for extracting the periocular features with high robustness aiming at the problem.
In conclusion, the complete eye contour identification technology with all-weather advantage provided by the invention can overcome the defects of narrow application range, low identification performance, poor feature extraction robustness and the like of the traditional eye contour identification technology. The invention provides a new theory and a new algorithm support for the practicability of the eye contour identification technology, so that the eye contour identification technology becomes more practical, reliable and popularized. The invention can be widely applied to the application occasions of attendance checking, civil monitoring, public security law enforcement, access control, community entrance and the like in outdoor, night, rain and snow and other complex environments.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an all-weather eye periphery identification method based on cascade super-resolution.
The invention is realized in such a way that the all-weather eye circumference identification method based on the cascade super-resolution comprises the following steps:
the method comprises the following steps that firstly, a multispectral camera is used for simultaneously acquiring periocular images of a certain body, necessary preprocessing is carried out on a limited periocular data set, and sample expansion is carried out on the limited periocular data set;
step two, amplifying the periocular data set processed in the step one by using a super-resolution technology based on deep learning;
thirdly, performing image restoration on the periocular data set processed in the second step by using an image deconvolution technology based on deep learning;
step four, based on the periocular data set obtained by processing in the step three, adopting Laplace image sharpening enhancement;
step five, constructing a novel stacked neural network model by adopting a deep learning theory based on the periocular data set obtained in the step four;
dividing the periocular data set into a training set and a test set, and training the training set by using a triple loss function triple loss and training a convolutional neural network by adopting a back propagation algorithm to obtain and store a model;
and step seven, calculating the test set obtained by the division in the step six through the convolutional neural network model trained in the step six to obtain characteristics, calculating by using Euclidean distance to obtain a matching score matrix, and calculating GAR and FAR values according to the matching score matrix.
Further, the image preprocessing and sample expansion of the first step include:
(1) and acquiring the periocular image by using a multispectral camera, wherein the specific electromagnetic wave band of the multispectral camera is a visible light wave band and an infrared wave band. Wherein the infrared band consists of two sub-bands of 980nm near infrared band (NIR) and 1550nm near infrared band (SWIR);
(2) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
i is a periocular image ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs normalized output;
(3) the infrared periocular image is enhanced using a log operator, as follows:
I=log(1+X);
then normalizing the standard value to [0,255] by using the same method in the step (2);
(4) and (3) performing data expansion on the eye circumference image in the step (2) by using different color space representations (RGB, HSV) and rotation operation by 6 times.
And further, reconstructing the small-size periocular image by adopting a super-resolution technology method based on deep learning, wherein the method takes the relation between the deep learning method and the traditional sparse coding-based method as a basis, and divides the three-layer convolutional neural network into three parts, namely image block extraction, nonlinear mapping and reconstruction.
Further, in the third step, the image deconvolution technology based on the convolutional neural network is used for deblurring and restoring the eye circumference data set reconstructed by the super-resolution technology, and the network consists of a deconvolution module, an artifact removal module and a reconstruction module.
Further, the fourth step of enhancing the eye image after the image deconvolution technology is restored by adopting a Laplace image sharpening technology comprises the following specific steps:
(1) the second derivative is found using the laplacian:
wherein I (x, y) is a periocular image restored by an image deconvolution technique,andthe directional derivatives along the x-axis and y-axis, respectively;
(2) add the original periocular image and the result after laplacian processing:
wherein Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
Further, in the fifth step, a novel stacked neural network model stackconvNet is constructed by adopting a deep learning theory, wherein the network architecture of the novel stacked neural network model stackconvNet comprises 10 convolutional layers, 6 maximum pooling layers and 2 full-connection layers, and a Drapout layer is introduced into the 2 full-connection layers.
Further, the step six of dividing the data set into a training set and a test set and training the model by using the triple loss function Triplet loss comprises the following specific steps:
(1) dividing a data set into a training set and a testing set, wherein the ratio is 7: 3;
(2) aiming at a training set, a triple loss function triplets loss is used, a back propagation algorithm is adopted to train a convolution neural network to obtain a model, and the model is stored, wherein the triple loss function formula is as follows:
in the above formulaAndthree image inputs representing a network model, whereinAndare two images from the same category and,andis two images from different types, alpha is an interval parameter; andare the output features of the network model, which make up the triplets.
Another object of the present invention is to provide a digital image processing system applying the eye periphery recognition method based on the cascaded super-resolution technique.
Another object of the present invention is to provide an image pattern recognition system applying the eye periphery recognition method based on the cascaded super-resolution technique.
In summary, the advantages and positive effects of the invention are as follows: the introduction of the cascade periocular super-resolution technology gradually improves the final identification accuracy rate through the cascade resolution technology. Specifically, the method comprises the following steps: firstly, performing super-resolution amplification on a small-size periocular image before periocular identification to increase the effective area of the periocular image, thereby being beneficial to improving the periocular identification performance; in addition, a periocular image enhancement technology based on image restoration is added after super-resolution to relieve the problem of image quality reduction caused by the periocular super-resolution; and finally, adding an image sharpening step based on Laplace operator to further improve the image quality.
Compared with the prior art, the invention has the following advantages:
(1) the invention provides a cascading eye periphery super-resolution technology aiming at the eye periphery identification problem, and the eye periphery identification accuracy is gradually improved by amplifying a cascading eye periphery area. The periocular identification is preceded by super-resolution eye magnification, and image restoration and sharpening to enhance periocular quality. Experiments show that the method is superior to the traditional enhancement algorithm.
(2) The invention designs a novel stack convolutional neural network StackConvNet aiming at the problem of feature extraction in periocular recognition. Experiments show that compared with the traditional feature extraction operator, the network architecture can extract eye features with higher robustness and higher identification performance.
(3) Aiming at the difficulty that the effective area of the periocular image is small, the super-resolution reconstruction technical method based on deep learning is applied to the amplification of the periocular image, and experiments show that the method can successfully realize the function of amplifying the periocular area.
(4) The method organically combines the deblurring of the periocular image, the Laplacian (Laplace) sharpening and the super-resolution technology, provides a framework of the cascade super-resolution technology, can successfully solve the problem that the image quality after super-resolution is not high, and experiments show that the use of the method can obviously improve the final recognition rate.
Drawings
Fig. 1 is a flowchart of an all-weather eye periphery identification method based on cascaded super-resolution according to an embodiment of the present invention.
Fig. 2 is a block diagram of an eye contour recognition system according to an embodiment of the present invention.
Fig. 3 is an example of a periocular data set used by an embodiment of the present invention, which is a schematic diagram of images taken in different scenes.
Fig. 4 is a schematic diagram of an example of a periocular image after super-resolution magnification based on fig. 3 according to an embodiment of the present invention.
Fig. 5 is a block diagram of a super-resolution reconstruction technique SRCNN network used in an embodiment of the present invention.
FIG. 6 is a graph of the comparison of sharpness averages using super-resolution amplification and interpolation amplification provided by embodiments of the present invention.
Fig. 7 is a network block diagram of a deep learning based deconvolution technique used according to an embodiment of the present invention.
Fig. 8 is a specific parameter diagram of a network block of a deep learning based deconvolution technique used according to an embodiment of the present invention.
Fig. 9 is a block diagram of a novel stacked neural network (StackConvNet) based on construction provided by an embodiment of the present invention.
FIG. 10 is a graphical representation comparing results provided by embodiments of the present invention with some conventional methods.
FIG. 11 is a graph comparing GAR and EER results of the present method and other methods provided by embodiments of the present invention, including results before and after use of super resolution techniques.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of the principles of the invention is provided in connection with the accompanying drawings.
As shown in fig. 1, the all-weather eye periphery identification method based on the cascaded super-resolution provided by the embodiment of the present invention includes the following steps:
s101: acquiring periocular images by using a multispectral camera, performing necessary preprocessing on a limited periocular data set, and performing sample expansion on the limited periocular data set;
s102: for the periocular data set after processing, magnifying it using a super-resolution technique based on deep learning;
s103: performing image restoration on the processed periocular data set by using an image deconvolution technology based on deep learning;
s104: based on the periocular data set obtained by processing, adopting Laplace image sharpening enhancement;
s105: constructing a novel stacked neural network model (named as StackConvNet) by adopting a deep learning theory based on the obtained periocular data set;
s106: dividing the periocular data set into a training set and a testing set, and as for the training set, using a triple loss function triple loss, training a convolutional neural network by adopting a back propagation algorithm to obtain a model and storing the model;
s107: for the test set obtained by the division in the step S106, the features are calculated by the convolutional neural network model trained in the step S106, a matching score matrix is calculated by using the euclidean distance, and the GAR and FAR values are calculated according to the matching score matrix.
The application of the principles of the present invention will now be described in further detail with reference to the accompanying drawings.
The eye periphery identification method based on the cascade super-resolution technology provided by the embodiment of the invention has a processing flow frame as shown in fig. 2, and comprises image preprocessing, super-resolution technology amplification, deconvolution image restoration, Laplace image sharpening, StackConvNet feature extraction and final training and testing parts, wherein in order to make the expression clearer, each part is explained in the following:
(1) image preprocessing: the image preprocessing of the invention comprises the following steps:
(I) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
where I is the periocular image, ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs a normalized output.
(II) enhancing the infrared periocular image using a log operator, the formula being:
I=log(1+X);
then, it was normalized to [0,255] again using the same method as in step (1).
And (III) performing data expansion on the eye periphery image obtained in the step (II) by 6 times by using different color space representations (RGB, HSV) and a rotation operation.
The periocular data set used in the present invention is shown in fig. 3, (a) is a periocular image obtained by shooting near infrared at 1.5m, (b) is a periocular image obtained by shooting visible light at 1.5m, (c) is a periocular image at 50m near infrared, (d) is a periocular image at 106m near infrared, and (e) is a periocular image at 50m short wave infrared.
(2) Super-resolution technology amplification: after the eye periphery image is preprocessed in the step (1), the eye periphery image is amplified by using a super-resolution technology method based on deep learning. The network frame divides a 3-layer network into image block extraction (Patch extraction and representation), Non-linear mapping (Non-linear mapping) and final Reconstruction (Reconstruction) according to a relation between deep learning and traditional sparse coding.
In which the periocular image of figure 3, enlarged on the basis of super-resolution techniques, is shown in figure 4. The super-resolution technology and the traditional interpolation algorithm are used for respectively amplifying the image in the figure 3, and then the result of calculating the mean value of the sharpness is shown in the figure 6.
(3) And (3) restoration of a deconvolved image: after the eye periphery image is amplified by the super-resolution Reconstruction technology in the step (2), the eye periphery data set reconstructed by the super-resolution technology is deblurred and restored by using an image Deconvolution technology based on a convolutional neural network, wherein the network consists of a Deconvolution module (Deconvolution module), an Artifact removal module (Artifact removal module) and a Reconstruction module (Reconstruction module). Specific parameters are shown in fig. 8, there are 11 layers, wherein the first 3 layers are deconvolution module parts, the 4 layers to 8 layers are artifact removal module parts, and the last 3 layers are image reconstruction module parts. In the deconvolution module, the first layer uses a convolution kernel of 1 × 45 size, the second layer uses a convolution kernel of 41 × 1 size, and the third layer uses a convolution kernel of 15 × 15 size. The artifact removal section employs a 1 × 1 convolution kernel. The reconstruction part uses a 2 x 2 deconvolution structure first, followed by a 2 x 2 convolution kernel.
(4) And (3) sharpening the Laplace image: after the eye image is enhanced by the deconvolution technology in the step (3), enhancing the eye image restored by the image deconvolution technology by using a Laplace image sharpening technology, wherein the specific operations are as follows:
(I) first find the second derivative using the laplacian:
wherein I (x, y) is a periocular image restored by an image deconvolution technique,andthe directional derivatives along the x-axis and y-axis, respectively.
(II) to obtain a sharpened periocular image, the original periocular image and the result after Laplacian processing are added:
in which Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
(5) And (3) extracting the characteristics of the StackConvNet: after the Laplace image sharpening enhancement is carried out in the step (4), a novel stacked neural network model stackconvNet extraction feature is constructed by using the method based on the deep learning theory, a stackconvNet network framework is shown in fig. 9, as shown in the figure, the stackconvNet network comprises 10 convolutional layers, 6 maximum pooling layers and 2 full-connection layers, and in order to prevent overfitting, a Dropout layer is introduced into the 2 full-connection layers.
As shown in fig. 9, the variables beginning with C represent convolutional layers, the variables beginning with MP represent maximum pooling layer operations, and the variables beginning with FC represent fully-connected layers. Wherein, all convolution layers use convolution kernels with the size of 3 × 3, the step number is 1, and padding is 0 filling in the convolution process; all pooling layers are maximum pooling layers with the size of 2 x 2 and the step number of 2, padding is 0 in the pooling process, the number of convolution kernels used in each layer is 16, 32, 64 and 64 in sequence, and the number of neurons in the last two fully-connected layers is 512 and 128 respectively.
(6) Training and testing: the periocular data set is first divided into training set test sets in a 7:3 ratio. In the training process, a triple loss function Triplet loss training model is used, and the used triple loss function is as follows:
in the above formulaAndthree image inputs representing a network model, whereinAndare two images from the same category and,andare from two images of a different type, and α is the interval parameter. Accordingly, the number of the first and second electrodes,andare the output features of the network model, which make up the triplets.
Therefore, the triple loss function needs to receive the characteristics of the three images and corresponding labels as input, the aim is to enable the intra-class distance to be smaller than the inter-class distance through a large amount of triple training, an optimizer selected in the training process is Adam, the learning rate is 0.001, the batch-size is 128, and a proper threshold value is set through a Tensor Board plug-in to view a curve of the loss value and the iteration times so that the model stops training and is stored.
The effect of the present invention will be described in detail with reference to the test.
In the test process, the Euclidean distance is used as a measurement function, after a test set extracts features through a trained model, the features of the test set and the input of a label of the test set to the most matched function are input to obtain a matching score matrix, GAR and FAR are calculated through matching scores, and the method specifically comprises the following steps:
(I) minimum value S according to matching scoreminAnd a maximum value SmaxIs the interval [ Smin,Smax]Obtaining a series of threshold values T with fixed step sizeiIn which S ismin≤Ti≤SmaxIs greater than the threshold value TiIs a true match, less than the threshold TiIs a false match.
(II) by step (I) according to different threshold values TiObtaining a series of true matching rates GARiFAR ratio of sum and falsei。
(III) passing the true match Rate GARiFAR ratio of sum and falseiThe Receiver Operating Curve (ROC) was plotted and the results are shown in FIG. 10.
The following description will be made in detail with reference to the effects of the present invention.
In order to prove the superiority of the cascade super-resolution network and the StackConvNet network introduced by the invention, a design experiment is contrasted and demonstrated from the following three aspects: first, to prove that the eye recognition performance after the reduction by the cascade super-resolution technique is better than the recognition performance without the cascade super-resolution technique, eye recognition experiments are performed on the cases with and without the cascade super-resolution technique, and the recognition results are respectively displayed by ROC curves and calculation of correct acceptance rate (GAR) and Equal Error Rate (EER) values, as shown in fig. 10 and fig. 11. From the first two ROC curves of FIG. 10 and the last two rows of FIG. 11, it can be seen that the recognition result using the cascaded super-resolution technique is better than the result without the cascaded super-resolution technique; secondly, to illustrate the advantages of the super-resolution based on deep learning, it is compared with the well-known conventional interpolation method: bilinear and bicubic interpolations and computes sharpness values to quantitatively measure contrast effect, the results of which are shown in fig. 6. It can be seen from the figure that the sharpness mean value obtained by the super-resolution technology based on deep learning is higher than that obtained by the traditional interpolation method, namely the super-resolution technology method based on deep learning is proved to be superior to the traditional method; finally, to demonstrate the superiority of the StackConvNet network proposed by the present invention for feature extraction, it is compared with the Local Binary Pattern (LBP) and Principal Component Analysis (PCA) of the typical conventional face recognition method, respectively, as shown in fig. 10 and fig. 11. As can be seen from the ROC curves of fig. 10, the two ROC curves of the StackConvNet network proposed by the present invention are significantly higher than PCA and LBP, and the EER value of the StackConvNet network proposed by the present invention is significantly lower than LBP and PCA, which is obtained from the EER column of fig. 11. In conclusion, the eye circumference identification method based on deep learning cascade super-resolution provided by the invention is superior to other methods and has good robustness.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. An all-weather eye circumference identification method based on cascade super-resolution is characterized by comprising the following steps:
the method comprises the following steps that firstly, a multispectral camera is used for simultaneously acquiring an eye image of a certain body, limited eye data sets are subjected to gray level conversion, normalization and log enhancement preprocessing, and samples are expanded;
step two, amplifying the periocular data set processed in the step one by using a super-resolution technology based on deep learning;
thirdly, performing image restoration on the periocular data set processed in the second step by using an image deconvolution technology based on deep learning;
step four, based on the periocular data set obtained by processing in the step three, adopting Laplace image sharpening enhancement;
step five, constructing a novel stacked neural network model by adopting a deep learning theory based on the periocular data set obtained in the step four;
dividing the periocular data set into a training set and a test set, and training the convolutional neural network by using a triple loss function Tripletloss and adopting a back propagation algorithm to obtain a model and store the model for the training set;
and step seven, calculating the test set obtained by the division in the step six through the convolutional neural network model trained in the step six to obtain characteristics, calculating by using Euclidean distance to obtain a matching score matrix, and calculating GAR and FAR values according to the matching score matrix.
2. The all-weather periocular identification method based on cascaded super-resolution as claimed in claim 1, wherein the image preprocessing and sample expansion of the first step comprises:
(1) using a multispectral camera to collect the periocular image, wherein the specific electromagnetic wave band of the multispectral camera is a visible light wave band and an infrared wave band, and the infrared wave band is composed of two sub-wave bands of a 980nm near infrared wave band (NIR) and a 1550nm near infrared wave band (SWIR);
(2) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
i is a periocular image ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs normalized output;
(3) the infrared periocular image is enhanced using a log operator, as follows:
I=log(1+X);
then normalizing the standard value to [0,255] by using the same method in the step (2);
(4) and (3) performing data expansion on the eye periphery image in the step (2) by using different color space representations (RGB, HSV) and rotation operation by 6 times.
3. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein the step two adopts a super-resolution technique based on deep learning to reconstruct the small-size periocular image, and the method uses the relationship between the deep learning method and the traditional method based on sparse coding as a basis to divide the three-layer convolutional neural network into three parts of image block extraction, nonlinear mapping and reconstruction.
4. The all-weather periocular identification method based on cascaded super-resolution as claimed in claim 1, wherein, in step three, the image deconvolution technique based on convolutional neural network is used to deblur the reconstructed periocular data set by super-resolution technique, and the network consists of three modules of deconvolution module, artifact removal module and reconstruction module.
5. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein the enhancement of the periocular image after the image deconvolution technology is restored by adopting a Laplace image sharpening technology in the fourth step comprises the following specific steps:
(1) the second derivative is found using the laplacian:
wherein I (x, y) is a periocular image restored by an image deconvolution technique,andthe directional derivatives along the x-axis and y-axis, respectively;
(2) the original periocular image and the result after laplacian processing are added:
in which Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
6. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein in step five, a deep learning theory is adopted to construct a novel stacked neural network model StackConvNet, and the network architecture thereof comprises 10 convolutional layers, 6 maximum pooling layers, 2 full-link layers, and a Dropout layer is introduced into the 2 full-link layers.
7. The all-weather periocular recognition method based on cascade super-resolution as claimed in claim 1, wherein the step six of dividing the data set into a training set and a testing set and training the model using triple loss function Tripletloss comprises the following specific steps:
(1) dividing a data set into a training set and a testing set, wherein the ratio is 7: 3;
(2) aiming at a training set, a triple loss function Tripletloss is used, a back propagation algorithm is adopted to train a convolutional neural network to obtain a model, and the model is stored, wherein the triple loss function formula is as follows:
8. A digital image processing system applying the all-weather eye periphery identification method based on cascaded super-resolution as claimed in any one of claims 1-7.
9. An image pattern recognition system applying the all-weather eye periphery recognition method based on cascade super-resolution as claimed in any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910281741.4A CN110175509B (en) | 2019-04-09 | 2019-04-09 | All-weather eye circumference identification method based on cascade super-resolution |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910281741.4A CN110175509B (en) | 2019-04-09 | 2019-04-09 | All-weather eye circumference identification method based on cascade super-resolution |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110175509A CN110175509A (en) | 2019-08-27 |
CN110175509B true CN110175509B (en) | 2022-07-12 |
Family
ID=67689669
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910281741.4A Active CN110175509B (en) | 2019-04-09 | 2019-04-09 | All-weather eye circumference identification method based on cascade super-resolution |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110175509B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111079624B (en) * | 2019-12-11 | 2023-09-01 | 北京金山云网络技术有限公司 | Sample information acquisition method and device, electronic equipment and medium |
CN114998976A (en) * | 2022-07-27 | 2022-09-02 | 江西农业大学 | Face key attribute identification method, system, storage medium and computer equipment |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107944379A (en) * | 2017-11-20 | 2018-04-20 | 中国科学院自动化研究所 | White of the eye image super-resolution rebuilding and image enchancing method based on deep learning |
CN109345449A (en) * | 2018-07-17 | 2019-02-15 | 西安交通大学 | A kind of image super-resolution based on converged network and remove non-homogeneous blur method |
-
2019
- 2019-04-09 CN CN201910281741.4A patent/CN110175509B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107944379A (en) * | 2017-11-20 | 2018-04-20 | 中国科学院自动化研究所 | White of the eye image super-resolution rebuilding and image enchancing method based on deep learning |
CN109345449A (en) * | 2018-07-17 | 2019-02-15 | 西安交通大学 | A kind of image super-resolution based on converged network and remove non-homogeneous blur method |
Non-Patent Citations (5)
Title |
---|
Accurate Periocular Recognition Under Less Constrained Environment Using Semantics-Assisted Convolutional Neural Network;Zhao Zijing 等;《IEEE Transactions on Information Forensics and Security》;20161206;全文 * |
Fusion of operators for heterogeneous periocular recognition at varying ranges;Cao zhicheng 等;《Pattern Recognition Letters》;20161015;全文 * |
Image Super-Resolution Using Deep Convolutional Networks;Dong Chao 等;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;20150601;第38卷(第2期);全文 * |
Improving Periocular Recognition by Explicit Attention to Critical Regions in Deep Neural Network;Zhao Zijing 等;《IEEE Transactions on Information Forensics and Security》;20180503;第13卷(第12期);全文 * |
On cross spectral periocular recognition;Anjali Sharma 等;《2014 IEEE International Conference on Image Processing》;20150129;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN110175509A (en) | 2019-08-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109886986B (en) | Dermatoscope image segmentation method based on multi-branch convolutional neural network | |
CN112949565B (en) | Single-sample partially-shielded face recognition method and system based on attention mechanism | |
Wang et al. | End-to-end image super-resolution via deep and shallow convolutional networks | |
CN109978848B (en) | Method for detecting hard exudation in fundus image based on multi-light-source color constancy model | |
CN102063708B (en) | Image denoising method based on Treelet and non-local means | |
CN110084238B (en) | Finger vein image segmentation method and device based on LadderNet network and storage medium | |
CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
CN110458792B (en) | Method and device for evaluating quality of face image | |
CN111539246B (en) | Cross-spectrum face recognition method and device, electronic equipment and storage medium thereof | |
CN110674824A (en) | Finger vein segmentation method and device based on R2U-Net and storage medium | |
CN110175509B (en) | All-weather eye circumference identification method based on cascade super-resolution | |
Juefei-Xu et al. | Pokerface: partial order keeping and energy repressing method for extreme face illumination normalization | |
Li et al. | Densely connected network for impulse noise removal | |
Velliangira et al. | A novel forgery detection in image frames of the videos using enhanced convolutional neural network in face images | |
CN111027564A (en) | Low-illumination imaging license plate recognition method and device based on deep learning integration | |
CN115862121B (en) | Face quick matching method based on multimedia resource library | |
Singh et al. | Multiscale reflection component based weakly illuminated nighttime image enhancement | |
CN114926348B (en) | Device and method for removing low-illumination video noise | |
CN115797205A (en) | Unsupervised single image enhancement method and system based on Retinex fractional order variation network | |
CN113269684A (en) | Hyperspectral image restoration method based on single RGB image and unsupervised learning | |
CN114140361A (en) | Generation type anti-network image defogging method fusing multi-stage features | |
CN109785253B (en) | Panchromatic sharpening post-processing method based on enhanced back projection | |
CN113095185A (en) | Facial expression recognition method, device, equipment and storage medium | |
Yadav et al. | Statistical measures for Palmprint image enhancement | |
CN112837293A (en) | Hyperspectral image change detection method based on Gaussian function typical correlation analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20230224 Address after: 400031 unit 1, building 1, phase 3, R & D building, Xiyong micro power park, Shapingba District, Chongqing Patentee after: Chongqing Institute of integrated circuit innovation Xi'an University of Electronic Science and technology Address before: 710071 Xi'an Electronic and Science University, 2 Taibai South Road, Shaanxi, Xi'an Patentee before: XIDIAN University |