CN110175509B - All-weather eye circumference identification method based on cascade super-resolution - Google Patents

All-weather eye circumference identification method based on cascade super-resolution Download PDF

Info

Publication number
CN110175509B
CN110175509B CN201910281741.4A CN201910281741A CN110175509B CN 110175509 B CN110175509 B CN 110175509B CN 201910281741 A CN201910281741 A CN 201910281741A CN 110175509 B CN110175509 B CN 110175509B
Authority
CN
China
Prior art keywords
periocular
image
resolution
super
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910281741.4A
Other languages
Chinese (zh)
Other versions
CN110175509A (en
Inventor
曹志诚
庞辽军
赵恒�
赵远明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Institute Of Integrated Circuit Innovation Xi'an University Of Electronic Science And Technology
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201910281741.4A priority Critical patent/CN110175509B/en
Publication of CN110175509A publication Critical patent/CN110175509A/en
Application granted granted Critical
Publication of CN110175509B publication Critical patent/CN110175509B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • G06V40/166Detection; Localisation; Normalisation using acquisition arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • G06V40/171Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • G06V40/19Sensors therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • G06V40/193Preprocessing; Feature extraction
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of pattern recognition and digital image processing, and particularly relates to an all-weather eye circumference recognition method based on cascade super-resolution. Using a multispectral camera to acquire periocular images, preprocessing a limited periocular data set, and expanding samples; performing image super-resolution operation on the periocular data set by adopting a convolutional neural network based on deep learning to amplify the periocular image; performing image restoration on the processed periocular data set by using an image deconvolution technology based on deep learning; based on the periocular data obtained by image restoration, adopting Laplace image sharpening enhancement; constructing a novel stacked neural network model by adopting a deep learning theory based on the obtained periocular data set; dividing the periocular data set into a training set and a test set; and calculating the test set obtained by division through the trained convolutional neural network model to obtain the characteristics. The method has stronger robustness and good generalization performance.

Description

All-weather eye circumference identification method based on cascade super-resolution
Technical Field
The invention belongs to the technical field of pattern recognition and digital image processing, and particularly relates to an all-weather eye circumference recognition method based on cascade super-resolution.
Background
Periocular identification is a new and particularly advantageous biometric modality that has emerged in recent years. It may for example be used as a complement to face recognition and has been shown to have the most significant information of face parts in case of face occlusion. Furthermore, periocular recognition may also be considered as an alternative to failure of iris recognition, since iris recognition requires the acquisition of iris images of very high quality, which is often not met at long distances.
However, researchers have heretofore focused only on visible light-based periocular identification techniques, which generally do not perform well in harsh climates and environments, such as unconstrained situations and harsh climates with uneven lighting, nighttime, rain and snow. With the emergence of applications in various complex environments in the real world, the eye periphery recognition technology based on visible light is more and more difficult to meet the requirements, for example, the monitoring task in the real world often occurs in poor atmospheric environments such as night, rainy and snowy days, and the acquisition of high-definition face images through visible light in the environment is a task which is difficult to complete. Therefore, new algorithms and systems need to be developed to improve the universality and robustness of eye contour identification. Based on this, the patent proposes in particular a multispectral periocular identification technique using a combination of visible and infrared light. The technology has the advantage of working in all weather, and is suitable for various environments such as daytime, night, sunny days, rain and snow and the like.
On the other hand, since the periocular image is usually captured from the face image, the corresponding area is small, and the image size is small (i.e., the resolution is small). The existing eye contour identification technology usually directly extracts and identifies features of eye contour images with original sizes, and influence of the eye contour image sizes on final identification performance is not considered. Because the high-resolution and high-quality image has significant meaning for improving the identification performance, the patent particularly provides a cascade super-resolution periocular identification technology, and the small-size periocular image is successfully amplified by a plurality of times by introducing the super-resolution, so that the effective area of feature extraction during periocular identification is increased. In addition, the simple super-resolution increases the area around the eye, but has the side effect of reducing the image quality. In order to solve the problem, the eye periphery super-resolution method is further perfected: after super-resolution, a depth learning-based periocular image restoration technology is introduced, and an image enhancement step based on Laplace sharpening is combined. This complete process is called: and (4) cascading eye periphery super-resolution. Finally, the cascade eye periphery super-resolution technology combines the multispectral all-weather characteristics, and the overall name is as follows: an all-weather eye circumference identification method based on cascade super-resolution.
Finally, as the feature extraction algorithm for eye contour identification is the traditional way so far, such as PCA, LBP, WLD, SIFT, etc. The traditional algorithms are all designed manually, are generally complex and complex in design, have good performance but poor robustness for specific situations, and have poor solution to the problems of illumination change and the like. With the further maturity of deep learning, the problem needs to be solved urgently. Automatic feature extraction methods such as convolutional neural networks are simpler and more robust. Therefore, the invention provides a convolutional neural network StackConvNet for extracting the periocular features with high robustness aiming at the problem.
In conclusion, the complete eye contour identification technology with all-weather advantage provided by the invention can overcome the defects of narrow application range, low identification performance, poor feature extraction robustness and the like of the traditional eye contour identification technology. The invention provides a new theory and a new algorithm support for the practicability of the eye contour identification technology, so that the eye contour identification technology becomes more practical, reliable and popularized. The invention can be widely applied to the application occasions of attendance checking, civil monitoring, public security law enforcement, access control, community entrance and the like in outdoor, night, rain and snow and other complex environments.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an all-weather eye periphery identification method based on cascade super-resolution.
The invention is realized in such a way that the all-weather eye circumference identification method based on the cascade super-resolution comprises the following steps:
the method comprises the following steps that firstly, a multispectral camera is used for simultaneously acquiring periocular images of a certain body, necessary preprocessing is carried out on a limited periocular data set, and sample expansion is carried out on the limited periocular data set;
step two, amplifying the periocular data set processed in the step one by using a super-resolution technology based on deep learning;
thirdly, performing image restoration on the periocular data set processed in the second step by using an image deconvolution technology based on deep learning;
step four, based on the periocular data set obtained by processing in the step three, adopting Laplace image sharpening enhancement;
step five, constructing a novel stacked neural network model by adopting a deep learning theory based on the periocular data set obtained in the step four;
dividing the periocular data set into a training set and a test set, and training the training set by using a triple loss function triple loss and training a convolutional neural network by adopting a back propagation algorithm to obtain and store a model;
and step seven, calculating the test set obtained by the division in the step six through the convolutional neural network model trained in the step six to obtain characteristics, calculating by using Euclidean distance to obtain a matching score matrix, and calculating GAR and FAR values according to the matching score matrix.
Further, the image preprocessing and sample expansion of the first step include:
(1) and acquiring the periocular image by using a multispectral camera, wherein the specific electromagnetic wave band of the multispectral camera is a visible light wave band and an infrared wave band. Wherein the infrared band consists of two sub-bands of 980nm near infrared band (NIR) and 1550nm near infrared band (SWIR);
(2) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
Figure BDA0002021888040000031
i is a periocular image ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs normalized output;
(3) the infrared periocular image is enhanced using a log operator, as follows:
I=log(1+X);
then normalizing the standard value to [0,255] by using the same method in the step (2);
(4) and (3) performing data expansion on the eye circumference image in the step (2) by using different color space representations (RGB, HSV) and rotation operation by 6 times.
And further, reconstructing the small-size periocular image by adopting a super-resolution technology method based on deep learning, wherein the method takes the relation between the deep learning method and the traditional sparse coding-based method as a basis, and divides the three-layer convolutional neural network into three parts, namely image block extraction, nonlinear mapping and reconstruction.
Further, in the third step, the image deconvolution technology based on the convolutional neural network is used for deblurring and restoring the eye circumference data set reconstructed by the super-resolution technology, and the network consists of a deconvolution module, an artifact removal module and a reconstruction module.
Further, the fourth step of enhancing the eye image after the image deconvolution technology is restored by adopting a Laplace image sharpening technology comprises the following specific steps:
(1) the second derivative is found using the laplacian:
Figure BDA0002021888040000041
wherein
Figure BDA0002021888040000042
And
Figure BDA0002021888040000043
respectively as follows:
Figure BDA0002021888040000044
wherein I (x, y) is a periocular image restored by an image deconvolution technique,
Figure BDA0002021888040000045
and
Figure BDA0002021888040000046
the directional derivatives along the x-axis and y-axis, respectively;
(2) add the original periocular image and the result after laplacian processing:
Figure BDA0002021888040000047
wherein Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
Further, in the fifth step, a novel stacked neural network model stackconvNet is constructed by adopting a deep learning theory, wherein the network architecture of the novel stacked neural network model stackconvNet comprises 10 convolutional layers, 6 maximum pooling layers and 2 full-connection layers, and a Drapout layer is introduced into the 2 full-connection layers.
Further, the step six of dividing the data set into a training set and a test set and training the model by using the triple loss function Triplet loss comprises the following specific steps:
(1) dividing a data set into a training set and a testing set, wherein the ratio is 7: 3;
(2) aiming at a training set, a triple loss function triplets loss is used, a back propagation algorithm is adopted to train a convolution neural network to obtain a model, and the model is stored, wherein the triple loss function formula is as follows:
Figure BDA0002021888040000051
in the above formula
Figure BDA0002021888040000052
And
Figure BDA0002021888040000053
three image inputs representing a network model, wherein
Figure BDA0002021888040000054
And
Figure BDA0002021888040000055
are two images from the same category and,
Figure BDA0002021888040000056
and
Figure BDA0002021888040000057
is two images from different types, alpha is an interval parameter;
Figure BDA0002021888040000058
Figure BDA0002021888040000059
and
Figure BDA00020218880400000510
are the output features of the network model, which make up the triplets.
Another object of the present invention is to provide a digital image processing system applying the eye periphery recognition method based on the cascaded super-resolution technique.
Another object of the present invention is to provide an image pattern recognition system applying the eye periphery recognition method based on the cascaded super-resolution technique.
In summary, the advantages and positive effects of the invention are as follows: the introduction of the cascade periocular super-resolution technology gradually improves the final identification accuracy rate through the cascade resolution technology. Specifically, the method comprises the following steps: firstly, performing super-resolution amplification on a small-size periocular image before periocular identification to increase the effective area of the periocular image, thereby being beneficial to improving the periocular identification performance; in addition, a periocular image enhancement technology based on image restoration is added after super-resolution to relieve the problem of image quality reduction caused by the periocular super-resolution; and finally, adding an image sharpening step based on Laplace operator to further improve the image quality.
Compared with the prior art, the invention has the following advantages:
(1) the invention provides a cascading eye periphery super-resolution technology aiming at the eye periphery identification problem, and the eye periphery identification accuracy is gradually improved by amplifying a cascading eye periphery area. The periocular identification is preceded by super-resolution eye magnification, and image restoration and sharpening to enhance periocular quality. Experiments show that the method is superior to the traditional enhancement algorithm.
(2) The invention designs a novel stack convolutional neural network StackConvNet aiming at the problem of feature extraction in periocular recognition. Experiments show that compared with the traditional feature extraction operator, the network architecture can extract eye features with higher robustness and higher identification performance.
(3) Aiming at the difficulty that the effective area of the periocular image is small, the super-resolution reconstruction technical method based on deep learning is applied to the amplification of the periocular image, and experiments show that the method can successfully realize the function of amplifying the periocular area.
(4) The method organically combines the deblurring of the periocular image, the Laplacian (Laplace) sharpening and the super-resolution technology, provides a framework of the cascade super-resolution technology, can successfully solve the problem that the image quality after super-resolution is not high, and experiments show that the use of the method can obviously improve the final recognition rate.
Drawings
Fig. 1 is a flowchart of an all-weather eye periphery identification method based on cascaded super-resolution according to an embodiment of the present invention.
Fig. 2 is a block diagram of an eye contour recognition system according to an embodiment of the present invention.
Fig. 3 is an example of a periocular data set used by an embodiment of the present invention, which is a schematic diagram of images taken in different scenes.
Fig. 4 is a schematic diagram of an example of a periocular image after super-resolution magnification based on fig. 3 according to an embodiment of the present invention.
Fig. 5 is a block diagram of a super-resolution reconstruction technique SRCNN network used in an embodiment of the present invention.
FIG. 6 is a graph of the comparison of sharpness averages using super-resolution amplification and interpolation amplification provided by embodiments of the present invention.
Fig. 7 is a network block diagram of a deep learning based deconvolution technique used according to an embodiment of the present invention.
Fig. 8 is a specific parameter diagram of a network block of a deep learning based deconvolution technique used according to an embodiment of the present invention.
Fig. 9 is a block diagram of a novel stacked neural network (StackConvNet) based on construction provided by an embodiment of the present invention.
FIG. 10 is a graphical representation comparing results provided by embodiments of the present invention with some conventional methods.
FIG. 11 is a graph comparing GAR and EER results of the present method and other methods provided by embodiments of the present invention, including results before and after use of super resolution techniques.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of the principles of the invention is provided in connection with the accompanying drawings.
As shown in fig. 1, the all-weather eye periphery identification method based on the cascaded super-resolution provided by the embodiment of the present invention includes the following steps:
s101: acquiring periocular images by using a multispectral camera, performing necessary preprocessing on a limited periocular data set, and performing sample expansion on the limited periocular data set;
s102: for the periocular data set after processing, magnifying it using a super-resolution technique based on deep learning;
s103: performing image restoration on the processed periocular data set by using an image deconvolution technology based on deep learning;
s104: based on the periocular data set obtained by processing, adopting Laplace image sharpening enhancement;
s105: constructing a novel stacked neural network model (named as StackConvNet) by adopting a deep learning theory based on the obtained periocular data set;
s106: dividing the periocular data set into a training set and a testing set, and as for the training set, using a triple loss function triple loss, training a convolutional neural network by adopting a back propagation algorithm to obtain a model and storing the model;
s107: for the test set obtained by the division in the step S106, the features are calculated by the convolutional neural network model trained in the step S106, a matching score matrix is calculated by using the euclidean distance, and the GAR and FAR values are calculated according to the matching score matrix.
The application of the principles of the present invention will now be described in further detail with reference to the accompanying drawings.
The eye periphery identification method based on the cascade super-resolution technology provided by the embodiment of the invention has a processing flow frame as shown in fig. 2, and comprises image preprocessing, super-resolution technology amplification, deconvolution image restoration, Laplace image sharpening, StackConvNet feature extraction and final training and testing parts, wherein in order to make the expression clearer, each part is explained in the following:
(1) image preprocessing: the image preprocessing of the invention comprises the following steps:
(I) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
Figure BDA0002021888040000081
where I is the periocular image, ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs a normalized output.
(II) enhancing the infrared periocular image using a log operator, the formula being:
I=log(1+X);
then, it was normalized to [0,255] again using the same method as in step (1).
And (III) performing data expansion on the eye periphery image obtained in the step (II) by 6 times by using different color space representations (RGB, HSV) and a rotation operation.
The periocular data set used in the present invention is shown in fig. 3, (a) is a periocular image obtained by shooting near infrared at 1.5m, (b) is a periocular image obtained by shooting visible light at 1.5m, (c) is a periocular image at 50m near infrared, (d) is a periocular image at 106m near infrared, and (e) is a periocular image at 50m short wave infrared.
(2) Super-resolution technology amplification: after the eye periphery image is preprocessed in the step (1), the eye periphery image is amplified by using a super-resolution technology method based on deep learning. The network frame divides a 3-layer network into image block extraction (Patch extraction and representation), Non-linear mapping (Non-linear mapping) and final Reconstruction (Reconstruction) according to a relation between deep learning and traditional sparse coding.
In which the periocular image of figure 3, enlarged on the basis of super-resolution techniques, is shown in figure 4. The super-resolution technology and the traditional interpolation algorithm are used for respectively amplifying the image in the figure 3, and then the result of calculating the mean value of the sharpness is shown in the figure 6.
(3) And (3) restoration of a deconvolved image: after the eye periphery image is amplified by the super-resolution Reconstruction technology in the step (2), the eye periphery data set reconstructed by the super-resolution technology is deblurred and restored by using an image Deconvolution technology based on a convolutional neural network, wherein the network consists of a Deconvolution module (Deconvolution module), an Artifact removal module (Artifact removal module) and a Reconstruction module (Reconstruction module). Specific parameters are shown in fig. 8, there are 11 layers, wherein the first 3 layers are deconvolution module parts, the 4 layers to 8 layers are artifact removal module parts, and the last 3 layers are image reconstruction module parts. In the deconvolution module, the first layer uses a convolution kernel of 1 × 45 size, the second layer uses a convolution kernel of 41 × 1 size, and the third layer uses a convolution kernel of 15 × 15 size. The artifact removal section employs a 1 × 1 convolution kernel. The reconstruction part uses a 2 x 2 deconvolution structure first, followed by a 2 x 2 convolution kernel.
(4) And (3) sharpening the Laplace image: after the eye image is enhanced by the deconvolution technology in the step (3), enhancing the eye image restored by the image deconvolution technology by using a Laplace image sharpening technology, wherein the specific operations are as follows:
(I) first find the second derivative using the laplacian:
Figure BDA0002021888040000091
wherein
Figure BDA0002021888040000092
And
Figure BDA0002021888040000093
respectively as follows:
Figure BDA0002021888040000101
wherein I (x, y) is a periocular image restored by an image deconvolution technique,
Figure BDA0002021888040000102
and
Figure BDA0002021888040000103
the directional derivatives along the x-axis and y-axis, respectively.
(II) to obtain a sharpened periocular image, the original periocular image and the result after Laplacian processing are added:
Figure BDA0002021888040000104
in which Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
(5) And (3) extracting the characteristics of the StackConvNet: after the Laplace image sharpening enhancement is carried out in the step (4), a novel stacked neural network model stackconvNet extraction feature is constructed by using the method based on the deep learning theory, a stackconvNet network framework is shown in fig. 9, as shown in the figure, the stackconvNet network comprises 10 convolutional layers, 6 maximum pooling layers and 2 full-connection layers, and in order to prevent overfitting, a Dropout layer is introduced into the 2 full-connection layers.
As shown in fig. 9, the variables beginning with C represent convolutional layers, the variables beginning with MP represent maximum pooling layer operations, and the variables beginning with FC represent fully-connected layers. Wherein, all convolution layers use convolution kernels with the size of 3 × 3, the step number is 1, and padding is 0 filling in the convolution process; all pooling layers are maximum pooling layers with the size of 2 x 2 and the step number of 2, padding is 0 in the pooling process, the number of convolution kernels used in each layer is 16, 32, 64 and 64 in sequence, and the number of neurons in the last two fully-connected layers is 512 and 128 respectively.
(6) Training and testing: the periocular data set is first divided into training set test sets in a 7:3 ratio. In the training process, a triple loss function Triplet loss training model is used, and the used triple loss function is as follows:
Figure BDA0002021888040000105
in the above formula
Figure BDA0002021888040000111
And
Figure BDA0002021888040000112
three image inputs representing a network model, wherein
Figure BDA0002021888040000113
And
Figure BDA0002021888040000114
are two images from the same category and,
Figure BDA0002021888040000115
and
Figure BDA0002021888040000116
are from two images of a different type, and α is the interval parameter. Accordingly, the number of the first and second electrodes,
Figure BDA0002021888040000117
and
Figure BDA0002021888040000118
are the output features of the network model, which make up the triplets.
Therefore, the triple loss function needs to receive the characteristics of the three images and corresponding labels as input, the aim is to enable the intra-class distance to be smaller than the inter-class distance through a large amount of triple training, an optimizer selected in the training process is Adam, the learning rate is 0.001, the batch-size is 128, and a proper threshold value is set through a Tensor Board plug-in to view a curve of the loss value and the iteration times so that the model stops training and is stored.
The effect of the present invention will be described in detail with reference to the test.
In the test process, the Euclidean distance is used as a measurement function, after a test set extracts features through a trained model, the features of the test set and the input of a label of the test set to the most matched function are input to obtain a matching score matrix, GAR and FAR are calculated through matching scores, and the method specifically comprises the following steps:
(I) minimum value S according to matching scoreminAnd a maximum value SmaxIs the interval [ Smin,Smax]Obtaining a series of threshold values T with fixed step sizeiIn which S ismin≤Ti≤SmaxIs greater than the threshold value TiIs a true match, less than the threshold TiIs a false match.
(II) by step (I) according to different threshold values TiObtaining a series of true matching rates GARiFAR ratio of sum and falsei
(III) passing the true match Rate GARiFAR ratio of sum and falseiThe Receiver Operating Curve (ROC) was plotted and the results are shown in FIG. 10.
The following description will be made in detail with reference to the effects of the present invention.
In order to prove the superiority of the cascade super-resolution network and the StackConvNet network introduced by the invention, a design experiment is contrasted and demonstrated from the following three aspects: first, to prove that the eye recognition performance after the reduction by the cascade super-resolution technique is better than the recognition performance without the cascade super-resolution technique, eye recognition experiments are performed on the cases with and without the cascade super-resolution technique, and the recognition results are respectively displayed by ROC curves and calculation of correct acceptance rate (GAR) and Equal Error Rate (EER) values, as shown in fig. 10 and fig. 11. From the first two ROC curves of FIG. 10 and the last two rows of FIG. 11, it can be seen that the recognition result using the cascaded super-resolution technique is better than the result without the cascaded super-resolution technique; secondly, to illustrate the advantages of the super-resolution based on deep learning, it is compared with the well-known conventional interpolation method: bilinear and bicubic interpolations and computes sharpness values to quantitatively measure contrast effect, the results of which are shown in fig. 6. It can be seen from the figure that the sharpness mean value obtained by the super-resolution technology based on deep learning is higher than that obtained by the traditional interpolation method, namely the super-resolution technology method based on deep learning is proved to be superior to the traditional method; finally, to demonstrate the superiority of the StackConvNet network proposed by the present invention for feature extraction, it is compared with the Local Binary Pattern (LBP) and Principal Component Analysis (PCA) of the typical conventional face recognition method, respectively, as shown in fig. 10 and fig. 11. As can be seen from the ROC curves of fig. 10, the two ROC curves of the StackConvNet network proposed by the present invention are significantly higher than PCA and LBP, and the EER value of the StackConvNet network proposed by the present invention is significantly lower than LBP and PCA, which is obtained from the EER column of fig. 11. In conclusion, the eye circumference identification method based on deep learning cascade super-resolution provided by the invention is superior to other methods and has good robustness.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (9)

1. An all-weather eye circumference identification method based on cascade super-resolution is characterized by comprising the following steps:
the method comprises the following steps that firstly, a multispectral camera is used for simultaneously acquiring an eye image of a certain body, limited eye data sets are subjected to gray level conversion, normalization and log enhancement preprocessing, and samples are expanded;
step two, amplifying the periocular data set processed in the step one by using a super-resolution technology based on deep learning;
thirdly, performing image restoration on the periocular data set processed in the second step by using an image deconvolution technology based on deep learning;
step four, based on the periocular data set obtained by processing in the step three, adopting Laplace image sharpening enhancement;
step five, constructing a novel stacked neural network model by adopting a deep learning theory based on the periocular data set obtained in the step four;
dividing the periocular data set into a training set and a test set, and training the convolutional neural network by using a triple loss function Tripletloss and adopting a back propagation algorithm to obtain a model and store the model for the training set;
and step seven, calculating the test set obtained by the division in the step six through the convolutional neural network model trained in the step six to obtain characteristics, calculating by using Euclidean distance to obtain a matching score matrix, and calculating GAR and FAR values according to the matching score matrix.
2. The all-weather periocular identification method based on cascaded super-resolution as claimed in claim 1, wherein the image preprocessing and sample expansion of the first step comprises:
(1) using a multispectral camera to collect the periocular image, wherein the specific electromagnetic wave band of the multispectral camera is a visible light wave band and an infrared wave band, and the infrared wave band is composed of two sub-wave bands of a 980nm near infrared wave band (NIR) and a 1550nm near infrared wave band (SWIR);
(2) the visible light periocular image is converted to a grayscale image using the following formula:
Igray=0.2989×R+0.5870×G+0.1140×B;
it is then normalized to [0,255] using the following equation:
Figure FDA0003685511500000011
i is a periocular image ImaxAnd IminMaximum and minimum gray values in the periocular image I, respectivelynIs normalized output;
(3) the infrared periocular image is enhanced using a log operator, as follows:
I=log(1+X);
then normalizing the standard value to [0,255] by using the same method in the step (2);
(4) and (3) performing data expansion on the eye periphery image in the step (2) by using different color space representations (RGB, HSV) and rotation operation by 6 times.
3. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein the step two adopts a super-resolution technique based on deep learning to reconstruct the small-size periocular image, and the method uses the relationship between the deep learning method and the traditional method based on sparse coding as a basis to divide the three-layer convolutional neural network into three parts of image block extraction, nonlinear mapping and reconstruction.
4. The all-weather periocular identification method based on cascaded super-resolution as claimed in claim 1, wherein, in step three, the image deconvolution technique based on convolutional neural network is used to deblur the reconstructed periocular data set by super-resolution technique, and the network consists of three modules of deconvolution module, artifact removal module and reconstruction module.
5. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein the enhancement of the periocular image after the image deconvolution technology is restored by adopting a Laplace image sharpening technology in the fourth step comprises the following specific steps:
(1) the second derivative is found using the laplacian:
Figure FDA0003685511500000021
wherein
Figure FDA0003685511500000022
And
Figure FDA0003685511500000023
respectively as follows:
Figure FDA0003685511500000024
wherein I (x, y) is a periocular image restored by an image deconvolution technique,
Figure FDA0003685511500000031
and
Figure FDA0003685511500000032
the directional derivatives along the x-axis and y-axis, respectively;
(2) the original periocular image and the result after laplacian processing are added:
Figure FDA0003685511500000033
in which Ish(x, y) is the sharpened image of the periphery of the eye, and c is a weight for adjusting the degree of sharpening.
6. The all-weather periocular recognition method based on cascaded super-resolution as claimed in claim 1, wherein in step five, a deep learning theory is adopted to construct a novel stacked neural network model StackConvNet, and the network architecture thereof comprises 10 convolutional layers, 6 maximum pooling layers, 2 full-link layers, and a Dropout layer is introduced into the 2 full-link layers.
7. The all-weather periocular recognition method based on cascade super-resolution as claimed in claim 1, wherein the step six of dividing the data set into a training set and a testing set and training the model using triple loss function Tripletloss comprises the following specific steps:
(1) dividing a data set into a training set and a testing set, wherein the ratio is 7: 3;
(2) aiming at a training set, a triple loss function Tripletloss is used, a back propagation algorithm is adopted to train a convolutional neural network to obtain a model, and the model is stored, wherein the triple loss function formula is as follows:
Figure FDA0003685511500000034
in the above formula
Figure FDA0003685511500000035
And
Figure FDA0003685511500000036
three image inputs representing a network model, wherein
Figure FDA0003685511500000037
And
Figure FDA0003685511500000038
are two images from the same category and,
Figure FDA0003685511500000039
and
Figure FDA00036855115000000310
is two images from different types, alpha is an interval parameter;
Figure FDA00036855115000000311
Figure FDA00036855115000000312
and
Figure FDA00036855115000000313
are the output features of the network model, which make up the triplets.
8. A digital image processing system applying the all-weather eye periphery identification method based on cascaded super-resolution as claimed in any one of claims 1-7.
9. An image pattern recognition system applying the all-weather eye periphery recognition method based on cascade super-resolution as claimed in any one of claims 1 to 7.
CN201910281741.4A 2019-04-09 2019-04-09 All-weather eye circumference identification method based on cascade super-resolution Active CN110175509B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910281741.4A CN110175509B (en) 2019-04-09 2019-04-09 All-weather eye circumference identification method based on cascade super-resolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910281741.4A CN110175509B (en) 2019-04-09 2019-04-09 All-weather eye circumference identification method based on cascade super-resolution

Publications (2)

Publication Number Publication Date
CN110175509A CN110175509A (en) 2019-08-27
CN110175509B true CN110175509B (en) 2022-07-12

Family

ID=67689669

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910281741.4A Active CN110175509B (en) 2019-04-09 2019-04-09 All-weather eye circumference identification method based on cascade super-resolution

Country Status (1)

Country Link
CN (1) CN110175509B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111079624B (en) * 2019-12-11 2023-09-01 北京金山云网络技术有限公司 Sample information acquisition method and device, electronic equipment and medium
CN114998976A (en) * 2022-07-27 2022-09-02 江西农业大学 Face key attribute identification method, system, storage medium and computer equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107944379A (en) * 2017-11-20 2018-04-20 中国科学院自动化研究所 White of the eye image super-resolution rebuilding and image enchancing method based on deep learning
CN109345449A (en) * 2018-07-17 2019-02-15 西安交通大学 A kind of image super-resolution based on converged network and remove non-homogeneous blur method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107944379A (en) * 2017-11-20 2018-04-20 中国科学院自动化研究所 White of the eye image super-resolution rebuilding and image enchancing method based on deep learning
CN109345449A (en) * 2018-07-17 2019-02-15 西安交通大学 A kind of image super-resolution based on converged network and remove non-homogeneous blur method

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Accurate Periocular Recognition Under Less Constrained Environment Using Semantics-Assisted Convolutional Neural Network;Zhao Zijing 等;《IEEE Transactions on Information Forensics and Security》;20161206;全文 *
Fusion of operators for heterogeneous periocular recognition at varying ranges;Cao zhicheng 等;《Pattern Recognition Letters》;20161015;全文 *
Image Super-Resolution Using Deep Convolutional Networks;Dong Chao 等;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;20150601;第38卷(第2期);全文 *
Improving Periocular Recognition by Explicit Attention to Critical Regions in Deep Neural Network;Zhao Zijing 等;《IEEE Transactions on Information Forensics and Security》;20180503;第13卷(第12期);全文 *
On cross spectral periocular recognition;Anjali Sharma 等;《2014 IEEE International Conference on Image Processing》;20150129;全文 *

Also Published As

Publication number Publication date
CN110175509A (en) 2019-08-27

Similar Documents

Publication Publication Date Title
CN109886986B (en) Dermatoscope image segmentation method based on multi-branch convolutional neural network
CN112949565B (en) Single-sample partially-shielded face recognition method and system based on attention mechanism
Wang et al. End-to-end image super-resolution via deep and shallow convolutional networks
CN109978848B (en) Method for detecting hard exudation in fundus image based on multi-light-source color constancy model
CN102063708B (en) Image denoising method based on Treelet and non-local means
CN110084238B (en) Finger vein image segmentation method and device based on LadderNet network and storage medium
CN111079764A (en) Low-illumination license plate image recognition method and device based on deep learning
CN110458792B (en) Method and device for evaluating quality of face image
CN111539246B (en) Cross-spectrum face recognition method and device, electronic equipment and storage medium thereof
CN110674824A (en) Finger vein segmentation method and device based on R2U-Net and storage medium
CN110175509B (en) All-weather eye circumference identification method based on cascade super-resolution
Juefei-Xu et al. Pokerface: partial order keeping and energy repressing method for extreme face illumination normalization
Li et al. Densely connected network for impulse noise removal
Velliangira et al. A novel forgery detection in image frames of the videos using enhanced convolutional neural network in face images
CN111027564A (en) Low-illumination imaging license plate recognition method and device based on deep learning integration
CN115862121B (en) Face quick matching method based on multimedia resource library
Singh et al. Multiscale reflection component based weakly illuminated nighttime image enhancement
CN114926348B (en) Device and method for removing low-illumination video noise
CN115797205A (en) Unsupervised single image enhancement method and system based on Retinex fractional order variation network
CN113269684A (en) Hyperspectral image restoration method based on single RGB image and unsupervised learning
CN114140361A (en) Generation type anti-network image defogging method fusing multi-stage features
CN109785253B (en) Panchromatic sharpening post-processing method based on enhanced back projection
CN113095185A (en) Facial expression recognition method, device, equipment and storage medium
Yadav et al. Statistical measures for Palmprint image enhancement
CN112837293A (en) Hyperspectral image change detection method based on Gaussian function typical correlation analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230224

Address after: 400031 unit 1, building 1, phase 3, R & D building, Xiyong micro power park, Shapingba District, Chongqing

Patentee after: Chongqing Institute of integrated circuit innovation Xi'an University of Electronic Science and technology

Address before: 710071 Xi'an Electronic and Science University, 2 Taibai South Road, Shaanxi, Xi'an

Patentee before: XIDIAN University