US8194942B2 - Authentication device and authentication method - Google Patents
Authentication device and authentication method Download PDFInfo
- Publication number
- US8194942B2 US8194942B2 US12/675,207 US67520708A US8194942B2 US 8194942 B2 US8194942 B2 US 8194942B2 US 67520708 A US67520708 A US 67520708A US 8194942 B2 US8194942 B2 US 8194942B2
- Authority
- US
- United States
- Prior art keywords
- quaternary
- valued
- frequency
- image
- brightness component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4887—Locating particular structures in or on the body
- A61B5/489—Blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/117—Identification of persons
- A61B5/1171—Identification of persons based on the shapes or appearances of their bodies or parts thereof
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/6813—Specially adapted to be attached to a specific body part
- A61B5/6825—Hand
- A61B5/6826—Finger
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
- A61B5/683—Means for maintaining contact with the body
- A61B5/6838—Clamps or clips
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/13—Sensors therefor
- G06V40/1312—Sensors therefor direct reading, e.g. contactless acquisition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/117—Identification of persons
- A61B5/1171—Identification of persons based on the shapes or appearances of their bodies or parts thereof
- A61B5/1172—Identification of persons based on the shapes or appearances of their bodies or parts thereof using fingerprinting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/141—Control of illumination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/1341—Sensing with light passing through the finger
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/14—Vascular patterns
Definitions
- the present invention relates to an authentication apparatus for personal identification based on finger vein patterns, and an authentication method thereof.
- the feature point extraction method is generally considered invulnerable to rotation or displacement. In reality, however, when rotation or displacement actually occurs, the feature point extraction method has a problem that the brightness of an image changes so greatly that important feature points are lost in many cases.
- the brightness of the image also changes greatly due to scattered light or indirect reflection light of the sunlight, or due to change in the amount of blood flow depending on an outdoor air temperature.
- feature points at a finger joint and an edge portion of the finger are lost, and thereby the authentication score decreases.
- the problem to be solved by the invention is to achieve highly accurate authentication even if some feature points are lost as a result of rotation or displacement of a finger when the finger is placed on a scanner, or as a result of change in the brightness of the image.
- a multiple-valued pattern matching method is adopted so that such changes can be dealt with by converting a data map itself into a multiple-valued map.
- a first aspect of the present invention includes an authentication apparatus comprising: a light source that emits an infrared light beam onto a finger; image pick-up means that picks up an image of the finger by using a transmitted light beam, and thus outputs an image signal; low-frequency brightness component extracting means that extracts a low-frequency brightness component signal from the image signal by using a low-pass filter; middle-frequency brightness component extracting means that extracts a middle-frequency brightness component signal from the image signal by using a middle-pass filter; correction means that corrects the middle-frequency brightness component signal by using the low-frequency brightness component signal; quaternary-value converting means that converts the corrected middle-frequency brightness component signal into quaternary values such that the quaternary values are distributed at a predetermined ratio; storage means that stores a quaternary-valued image obtained from the quaternary-value conversion by the quaternary-value converting means; correlation coefficient calculating means that calculates a correlation coefficient between a quaternary-valued evaluation image as an
- the middle-frequency brightness component signal is extracted by using a middle-pass filter (band-pass filter) which removes low frequencies by using a matrix in which weights assigned to the respective pixels decrease by 2 ⁇ 3 in geometric series as the pixels become farther from the center, and then removes high-frequencies by the moving average method.
- a middle-pass filter band-pass filter
- “To correct the middle-frequency brightness component signal using the low-frequency brightness component signal” means, for example, that (middle-frequency brightness component signal) ⁇ (low-frequency brightness component signal).
- “To convert the corrected middle-frequency brightness component signal into a quaternary value” means, for example, that the corrected middle-frequency brightness component signal is converted into a quaternary value “0” when the signal is less than a first threshold, another quaternary value “1” when the signal is not less than the first threshold but less than a second threshold, yet another quaternary value “2” when the signal is not less than the second threshold but less than a third threshold, and still another quaternary value “3” when the signal is not less than the third threshold.
- “So that quaternary values of the middle-frequency brightness component signal are distributed at a predetermined ratio” means, for example, that the relationship in quantity between the quaternary value “0” and the whole quaternary values is 80%; the relationship in quantity between the quaternary value “1” and the whole quaternary values is 5%; the relationship in quantity between the quaternary value “2” and the whole quaternary values is 5%; and the relationship in quantity between the quaternary value “3” and the whole quaternary values is 10%. It is desirable that, whenever deemed necessary, the ratio among the quaternary values “0”, “1”, “2”, and “3” should be adjusted depending on the brightness of the light source, the sensitivity of image-pickup means, and the like.
- the quaternary-valued evaluation image as the object of evaluation means a quaternary-valued image which is intended to be evaluated in terms of whether the quaternary-valued image coincides with any one of multiple quaternary-valued registered images.
- the “correlation coefficient” is found by, for example,
- the “pattern matching method based on a normalized correlation method” is a method in which:
- “To judge whether or not the quaternary-valued evaluation image and the quaternary-valued registered image are obtained by picking up the respective images of the same finger on a basis of the correlation coefficient calculated by the correlation coefficient calculating means” means, for example, that: when the correlation coefficient is equal to or more than a predetermined value, the images are judged as being obtained by photographing the same finger; and when the correlation coefficient is less than the predetermined value, the images are judged as being obtained by photographing the respective different fingers.
- a third aspect of the present invention is one which further comprises quaternary-valued reduced image generating means that generates a quaternary-valued reduced image by convolving a plurality of cells included in the quaternary-valued image obtained from the quaternary-value conversion by the quaternary-value converting means, wherein: the correlation coefficient calculating means calculates a primary correlation coefficient between a quaternary-valued reduced evaluation image generated from the quaternary-valued evaluation image, and a quaternary-valued reduced registered image beforehand stored in the storage means, by using the pattern matching method based on the normalized correlation method; and only when the primary correlation coefficient is not less than a predetermined value, the correlation coefficient calculating means calculates the correlation coefficient between the quaternary-valued evaluation image associated with the quaternary-valued reduced evaluation image, and the quaternary-valued registered image associated with the quaternary-valued reduced registered image, by using the pattern matching method based on the normalized correlation method.
- a fourth aspect of the present invention is to one which further comprises a high-frequency brightness component extracting means that extracts a high-frequency brightness component signal from the image signal by using a high-pass filter, wherein: the correction means corrects the middle-frequency brightness component signal and the high-frequency brightness component signal by using the low-frequency brightness component signal; the quaternary-value converting means converts the corrected middle-frequency brightness component signal into quaternary values such that the quaternary values are distributed at a first predetermined ratio, and converts the corrected high-frequency brightness component signal into quaternary values such that the quaternary values are distributed at a second predetermined ratio; the correlation coefficient calculating means calculates (1) a middle-frequency correlation coefficient between a quaternary-valued middle-frequency evaluation image obtained from the middle-frequency brightness component signal, and a quaternary-valued middle-frequency registered image beforehand stored, and (2) a high-frequency correlation coefficient between a quaternary-valued high-frequency evaluation image obtained from the high-frequency brightness component signal, and
- the high-pass filter will be described later.
- the authenticity is determined as NG (i.e., it is judged that the quaternary-valued evaluation image and the quaternary-valued registered image are not obtained by picking up the respective images of the same finger);
- the authenticity is determined as OK (i.e., it is judged that the quaternary-valued evaluation image and the quaternary-valued registered image are obtained by picking up the respective images of the same finger);
- authentication with high environmental resistance can be achieved, with high reproducibility, by converting each bit into a value represented by a multiple-valued number.
- authentication dealing with change in the amount of blood flow can also be achieved.
- both the quaternary-valued image and the quaternary-valued reduced image are produced so that authentication is carried out by two steps. This makes it possible to perform evaluation with a large number of object images at high speed.
- both a vein and an inter-joint dermal print of a finger are simultaneously evaluated by transmitted light from the same light source.
- an authentication method can be achieved in which an image of a finger is difficult to forge.
- evaluation can be performed more accurately than in the case where only the vein is used.
- FIG. 1 is a diagram showing an overall configuration of an authentication apparatus according to an embodiment of the present invention.
- FIG. 2 is a diagram showing a configuration of an authentication executing unit according to the embodiment of the present invention.
- FIG. 3 is a diagram showing an overall flow of an authentication process according to the embodiment of the present invention.
- FIG. 4 is a diagram showing a flow of a map generating process according to the embodiment of the present invention.
- FIG. 5 is a diagram showing a flow of a matching process according to the embodiment of the present invention.
- FIG. 6 is a diagram showing a relationship between a location of an LED and an emitted area according to the embodiment of the present invention.
- FIGS. 1 to 5 descriptions will be hereinbelow provided for a finger vein authentication apparatus according to a first embodiment of the present invention.
- FIG. 1 shows an overall configuration of the finger vein authentication apparatus according to the first embodiment.
- the finger vein authentication apparatus includes: a light source 1 including 4 infrared LED (light-emitting diode) systems; a first user interface 2 including a 10-key and a LCD (liquid crystal display); a second user interface 3 including an LED and a buzzer; a finger detector 4 including a touch sensor; an infrared LED driving circuit capable of controlling electric currents of the respective 4 systems; a CPU (central processing unit) 6 that controls operations of various parts; a memory 7 used as a work memory for the CPU, and also used to retain data on veins; a camera 8 that picks up an image of veins; a video decoder 9 that converts an camera image into a set of digital data; and an FPGA (field programmable gate array) 10 that converts the set of digital video data into a set of data on brightness.
- the memory 7 includes SDRAM (synchronous dynamic random access memory), a flash memory and the like.
- FIG. 2 shows a configuration of an authentication executing unit according to the first embodiment.
- the CPU 6 functions as the authentication executing unit under control of software.
- the CPU 6 function as: low-frequency brightness component extracting means 61 that extracts a low-frequency brightness component signal from an image signal using a low-pass filter; middle-frequency brightness component extracting means 62 that extracts a middle-frequency brightness component signal from the image signal using a middle-pass filter; correction means that corrects the middle-frequency brightness component signal using the low-frequency brightness component signal; quaternary-value converting means 64 that converts the corrected middle-frequency brightness component signal into a quaternary value so that quaternary values are distributed at a predetermined ratio; correlation coefficient calculating means 65 that calculates a correlation coefficient between a quaternary-valued evaluation image as an object of evaluation, the quaternary-valued evaluation image being converted into a quaternary value by the quaternary-value converting means, and a quaternary-valued registered image, the quaternary-valued registered image being beforehand stored in the storage means, by using a pattern matching method based on a normalized correlation method; and judgment means
- FIG. 3 shows an overall flow of an authentication process according to the first embodiment.
- maps quadternary-valued images
- step S 101 maps (quaternary-valued images) are generated in step S 101 .
- a primary registered map, a secondary registered map, a primary evaluation map and a secondary evaluation map are generated.
- the primary registered map and the secondary registered map are beforehand generated and stored in the memory 7 .
- step S 102 a screening (primary authentication) is carried out using primary maps (thumbnail dictionaries) Specifically, a matching is performed between each of primary maps stored in the memory 7 and a primary map as an object of evaluation generated from an image of veins captured by the camera 8 . Primary maps, whose matching result evaluation values (similarity evaluation values) are higher than a limit value, are selected as primary candidates.
- the number of primary maps, whose matching result evaluation values (similarity evaluation values) are higher than the limit value is larger than a predetermined number
- the number of primary maps, whose matching result evaluation values are higher than the limit value is equal to or more than 10
- step S 104 If any primary candidate exists (or if YES in step S 103 ), the process proceeds to step S 104 . If no primary candidate exits (or if NO in step S 103 ), the process proceeds to step S 107 .
- a secondary authentication process is carried out using secondary maps. Specifically, a matching is performed between each of the stored secondary maps which are associated with the respective primary candidates selected by the primary screening (primary authentication), and a secondary map of the object of evaluation, which is generated from the image of the veins captured by the camera 8 . Secondary maps, whose matching result evaluation values (similarity evaluation values) is higher than a limit value, are selected as secondary candidates.
- step S 105 If any secondary candidate exists (or if YES in step S 105 ), the process proceeds to step S 106 . If no secondary candidate exits (or if NO in step S 105 ), the process proceeds to step S 107 .
- step S 106 in a case where a score (description of the score will be given later) of a secondary map having the highest matching result evaluation values (similarity evaluation values) is higher than a threshold value, authentication is determined as OK.
- step S 107 authentication is determined as NG.
- FIG. 4 shows a flow of a map generating process according to the first embodiment.
- a basic quantity of light is measured with the IR-LEDs (infrared light-emitting diodes) being turned off in step S 201 , and a brightness gain of an image is adjusted in step S 202 .
- IR-LEDs infrared light-emitting diodes
- step S 203 light from an IR-LED is adjusted. Specifically, an optimal quantity of light is emitted onto an object of evaluation from the IR-LED, and the brightness gain of the image is finely tuned.
- step S 204 8 frames of image are captured. Thus, for each pixel in the same coordinates throughout all the frames, a simple sum of brightness is calculated.
- step S 205 a camera noise is removed, and a centering process is carried out.
- the centering process means a process of making the center of the image and the center of each frame coincide with each other.
- step S 206 separation is performed on a background image and a vein image. Specifically, the background image (represented by a low-frequency brightness component signal) is separated from the 8 frames using a low-pass filter, and the vein image (represented by a middle-frequency brightness component signal) is separated from the same 8 frames using a band-pass filter.
- the background image represented by a low-frequency brightness component signal
- the vein image represented by a middle-frequency brightness component signal
- the separation of the low-frequency brightness component signal from the 8 frames is achieved using the moving average method. Specifically, the separation of the low-frequency brightness component is achieved by setting the brightness of the center pixel (namely, a pixel at coordinates (11, 11) at an arithmetic mean of brightnesses of the respective 21 ⁇ 21 pixels.
- the separation of the middle-frequency brightness component signal from the 8 frames is achieved as follows. To begin with, the low-frequency brightness component signal is separated using a filter which decreases the weights assigned to the respective 21 ⁇ 21 pixels by 2 ⁇ 3 in geometric series as the pixels become farther from the center. Subsequently, for each 3 ⁇ 3 pixels, the brightness of a pixel at the center (or at coordinates (2, 2)) is set at an arithmetic mean of the brightnesses of the 3 ⁇ 3 pixels.
- step S 207 an area outside the edge of the finger is located and removed from each frame.
- step S 208 the vein image is corrected using the background image. Specifically, the middle-frequency brightness component signal is divided by the low-frequency brightness component signal.
- step S 209 isolated vein portions whose reproducibility is low are located and removed.
- isolated vein portions mean portions of a vein which are photographed as isolated pieces but not reproduced as a single vein. The removal of the isolated vein portions enhances the reproducibility for recognition.
- the “vicinity” includes coordinates (x, y+1), (x, y ⁇ 1), (x+1, y), (x+1, y+1) and (x+1, y ⁇ 1) surrounding coordinates (x, y).
- step S 210 a histogram is made.
- step S 211 secondary maps (quaternary-valued maps) are generated.
- the brightness of each pixel is represented by a value in a range of 0 to 255.
- the brightnesses of each pixel in the 8 frames are summed up.
- the total brightness of each pixel takes a value in a range of 0 to 2040.
- the value in the range of 0 to 2040 is converted into quaternary values so that the ratio among 0s, 1s, 2s and 3s can be a predetermined value.
- the quaternary value takes “0” when the total brightness is less than a first threshold; the quaternary value takes “1” when the total brightness is not less than the first threshold but less than a second threshold; the quaternary value takes “2” when the total brightness is not less than the second threshold but less than a third threshold; and the quaternary value takes “3” when the total brightness is not less than the third threshold.
- the first threshold, the second threshold and the third threshold are determined so that: the ratio of “0” can be 80%; the ratio of “1” can be 5%; and the ratio of “2” can be 5%.
- a clear image is not always acquired from a vein.
- a vein image is not clear, if the image is represented by binary values and thus processed as if a clear image were instead acquired, it is difficult to reflect the unclearness of the image on the evaluation.
- the image is represented by quaternary values, it is possible to reflect the unclearness of the image on the evaluation.
- step S 212 the primary maps (thumbnail dictionaries) are generated.
- the generation of the primary maps is carried out in order to make the operation faster, and to make the operation meet the cache capacity of the CPU.
- step S 213 the primary maps and the secondary maps are outputted.
- primary maps and secondary maps are generated. It is desirable that multiple primary maps should be generated for each finger of each person who is a subject of authentication evaluation. Similarly, it is desirable that multiple secondary maps should be generated for each finger of each person who is a subject of authentication evaluation. For example, let us assume a case where persons who are subjects of authentication evaluation are Messrs. A, B and C, i.e., a case where an authentication process is performed so as to evaluate who among Messrs. A, B and C places a finger on the authentication apparatus. For a middle finger of Mr. A, three primary maps and three secondary maps are generated. Similarly, three primary maps and three secondary maps are generated for a middle finger of each of Messrs. B and C. In this respect, it does not matter whether or not the number of generated primary maps is equal to the number of generated secondary maps. All the primary and secondary maps thus generated are stored in the memory 7 .
- the total brightness of each pixel is converted into a quaternary value so that the ratios of the numbers of 0s, 1s, 2s and 3s to the number of quaternary values are all 25%.
- the ratios of the numbers of 0s, 1s, 2s and 3s to the number of quaternary values are equal to one another.
- the ratios of the numbers of 0s, 1s, 2s and 3s to the number of quaternary values may be arbitrarily determined. It does not matter if the ratios of the numbers of 0s, 1s, 2s and 3s to the number of quaternary values are not equal to one another.
- the quaternary value takes “0” when the total brightness of a pixel in the 8 combined frame is less than 400; the quaternary value takes “1” when the total brightness thereof is not less than 400 but less than 800; the quaternary value takes “2” when the total brightness thereof is not less than 800 but less than 1200; and the quaternary value takes “3” when the total brightness thereof is not less than 1200.
- the total brightness of each pixel is converted into a quaternary value as shown in Table 4.
- a brightness at (1, 2) in the primary map is calculated on the basis of brightnesses at (1, 3), (1, 4), (2, 3) and (2, 4) in the secondary map;
- a brightness at (2, 1) in the primary map is calculated on the basis of brightnesses at (3, 1), (3, 2), (4, 1) and (4, 2) in the secondary map;
- a brightness at (2, 2) in the primary map is calculated on the basis of brightnesses at (3, 3), (3, 4), (4, 3) and (4, 4) in the secondary map.
- coordinates (X, Y) means the X-th row and the Y-th column in a matrix.
- the brightness values in the primary map are calculated from the brightness values in the secondary map in the following manner.
- the ratio among the numbers of 0s, 1s, 2s and 3s are set at a predetermined ratio as is the case with the generation of the secondary map.
- the median of the thresholds (the lower threshold is 0, whereas the upper threshold is 400) for a value “0” in the secondary map is “200”; the median of the thresholds (the lower threshold is 400, whereas the upper threshold is 800) for a value “1” in the secondary map is “600”; and the median of the thresholds (the lower threshold is 800, whereas the upper threshold is 1200) for a value “2” in the secondary map is “1000”.
- the median of the threshold (in a range of 1200 to 2047) for a value “3” in the secondary map is set at “1600”.
- FIG. 7 Detailed descriptions will be provided using FIG. 7 .
- a value at (1, 1) is “0”, and the median corresponding to “0” is “200”;
- a value at (1, 2) is “0”, and the median corresponding to “0” is “200”;
- a value at (2, 1) is “0”, and the median corresponding to “0” is “200”;
- a value at (2, 2) is “1”, and the median corresponding to “1” is “600”.
- a value at (1, 3) is “0”, and the median corresponding to “0” is “200”;
- a value at (1, 4) is “1”, and the median corresponding to “1” is “600”;
- a value at (2, 3) is “1”, and the median corresponding to “1” is “600”;
- a value at (2, 4) is “2”, and the median corresponding to “2” is “1000”.
- FIG. 5 shows a flow of a matching process according to the first embodiment.
- step S 302 a loop of the matching process starts to be applied to all the search locations.
- step S 303 as an initial value, 0 is substituted for X and Y.
- step S 304 a loop of the matching process starts to be applied to all the sets of data.
- all the sets of data means all the registered primary maps (thumbnail dictionaries) when the matching process is executed using the primary maps
- all the sets of data means all the secondary maps associated with each primary candidate when the matching process is executed using the secondary maps
- step S 305 a cross-correlation between X and Y is calculated, and the thus-calculated value is substituted for F.
- step S 306 an autocorrelation of X is calculated, and the thus-calculated value is substituted for G.
- step S 307 the loop of the matching process to all the sets of data is terminated.
- step S 308 F/G is calculated for each set of data. Among the values obtained by the calculation, the 10 largest values are selected as candidate sets of data.
- step s 309 the loop of the matching process to all the search locations is terminated.
- step S 310 the candidate sets of data are outputted. Detailed contents of each step will be described later.
- the first stage primary evaluation
- the process speed can be made faster by two to three digits.
- the evaluation image and each registered image are fitted to each other in a fine tune, and how much the evaluation image and each registered image coincide together is evaluated with high accuracy.
- the time needed for the process can be reduced without sacrificing the quality of the evaluation, when whether or not the evaluation image and each registered image match together in position is checked in one stage, and how much the evaluation image and each registered image coincide together is evaluated in the other stage.
- Table 8 shows an example of a set of registered data.
- T denotes a quaternary value into which a brightness of the registered image is converted.
- Table 10 shows an example of a set of evaluation image.
- I denotes a quaternary value into which a brightness of the evaluation image is converted.
- the correlation coefficient between the set of registered data and the set of evaluation data is equal to 0.875 which results from the division of 49 by 56.
- a correlation coefficient F/G between the set of evaluation data and all sets of registered data is calculated.
- the set of evaluation data (evaluation map) and the set of registered data (registered map) are relatively shifted vertically or horizontally by one cell, and then the correlation coefficients for all the sets of registered data are again calculated.
- the candidate sets of data which are outputted in step S 310 , are a result of the matching process shown as step S 102 in FIG. 3 .
- a registered primary map corresponding to the correlation coefficient is selected as a primary candidate in step S 103 shown in FIG. 3 .
- a matching is performed between a secondary registered map associated with the primary candidate and the secondary evaluation map (step S 104 shown in FIG. 3 ).
- the contents of the matching process are the same as those shown in FIG. 5 .
- the matching is performed with respect to only the secondary registered map associated with the primary registered map which is judged as the primary candidate in step S 103 , but not with respect to all the secondary registered maps which are stored in the memory 7 .
- a matching is performed using the secondary evaluation map and the secondary registered map.
- the 10 largest correlation coefficients F/G are outputted as a set of evaluation data (step S 310 shown in FIG. 5 ).
- the set of evaluation data on correlation, which is outputted in step S 310 is a result of the matching in step S 104 shown in FIG. 3 .
- a secondary registered map corresponding to the correlation coefficient is selected as a secondary candidate in step S 105 shown in FIG. 3 .
- the original image represented by the set of evaluation data is authenticated as an image of the vein of a person whose data is registered, the image corresponding to the registered secondary map selected as the secondary candidate whose correlation coefficient is the largest among those of the secondary candidates (step S 106 ).
- a set of data on the inter-joint dermal print and a set of data on the vein are separated from the data of the picked-up image.
- an authentication is executed on the basis of the inter-joint dermal print, while another authentication is executed on the basis of the vein. Because the set of data on the inter-joint dermal print and the set of data on the vein are acquired from the single set of data on the pick-up image, an authentication apparatus can be achieved in which data is extremely difficult to forge, while the authentication apparatus requires no specific reflection light source or the like.
- an authentication on a finger print is performed on the basis of an image which is captured from the finger print independently, whereas an authentication on a vein is performed on the basis of an image which is captured from the vein independently.
- a second embodiment makes it possible to simultaneously perform an authentication on the basis of an inter-joint dermal print (which is a dermal print between the first and second joints from the fingertip on a finger) and an authentication on the basis of a vein with a set of data on the inter-joint dermal print, a set of data on the vein and a set of data on the background being separated from a set of data on an image which is captured from the finger by a single photographic shot.
- an inter-joint dermal print which is a dermal print between the first and second joints from the fingertip on a finger
- the image of the background is used for normalizing the gray scales of each of the acquired images.
- a background image (represented by a low-frequency brightness component signal) is separated using a low-pass filter.
- a vein image (represented by a middle-frequency brightness component signal) is separated using a band-pass filter.
- an inter-joint dermal print image (represented by a high-frequency brightness component signal) is separated from the resultant 8 frames using a high-pass filter.
- the high-pass filter in use is a 21 ⁇ 21, 1 ⁇ 2 geometric series filter.
- FIG. 12 shows a 5 ⁇ 5, 1 ⁇ 2 geometric series filter.
- the vein image and the inter-joint dermal print image, the vein image and the inter-joint dermal print image are corrected using the background image.
- the resultant vein image and the resultant inter-joint dermal print image are converted into quaternary values; for each of the vein image and the inter-joint dermal print image, secondary maps and primary maps (thumbnail images for screening) are generated; and for each of the vein image and the inter-joint dermal print image, match checks are performed using correlation coefficients.
- the brightness should be adjusted in the following manner.
- the finger is discriminated from anything else on the basis of how much an image obtained with no light being emitted from each near infrared LED is changed into an image obtained with light being emitted from the near infrared LED due to only the transmission of the light emitted from the near infrared LED by evaluating the difference between the two images.
- An amount of light emitted from each near infrared LED and a camera gain, which are used to obtain a target brightness, are controlled by: obtaining an image with no light being emitted from the near infrared LED; and measuring how much bright the thus-obtained image is due to external light.
- Light is emitted from each of the four near infrared LED so that the amount of light changes with the difference in location among the four near infrared LEDs. Thereby, the transmission brightness changes with the difference in location. From the thus-obtained transmission brightnesses different from one another, one transmission brightness most suitable for obtaining a target brightness is selected so that an amount of light emitted from each near infrared LED is set up for obtaining the target brightness. Thus, the amount of light can be controlled within a time used to capture one image.
- FIG. 6 shows a relationship between the locations of the LEDs and the respective emitted areas. Using FIG. 6 , descriptions will be provided for how the amount of light emitted from each LED is adjusted on the basis of the gradient in the amount of light. For example, the amount of light emitted from the LED 1 is set smaller than the amount of light emitted from the LED 2 , which is set smaller than the amount of light emitted from the LED 3 , which is set smaller than the amount of light emitted from the LED 4 .
- the area illuminated by light from the LED 1 is denoted by A 11 ; the area illuminated by light from the LED 2 is denoted by A 12 ; the area illuminated by light from the LED 3 is denoted by A 13 ; and the area illuminated by light from the LED 4 is denoted by A 14 .
- the transmission brightness of the area A 12 is the closest to the target brightness among the transmission brightnesses of the respective areas A 11 , A 12 , A 13 and A 14 .
- the amount of light emitted from each of the LED 1 , the LED 3 and the LED 4 is set equal to the amount of light emitted from the LED 2 .
- the brightness is automatically adjusted in the above-described manner when an image is captured, no optimal image may be acquired, in some cases, due to individual difference or environmental change.
- some finger which is thick enough to decrease the transmittance of light, makes it difficult for its vein to be photographed under a standard brightness.
- some other finger which is slender with a slender vein, makes the image of the vein indistinct or vague under the standard brightness because the vein allows some portion of light to reach the back of the vein.
- an optimal image is selected from the captured images in the following manner.
- 8 brightness averages are obtained.
- the variance is calculated from the 8 brightness averages in the following manner.
- an average brightness Lm1 is calculated for 64 points including a point at coordinate (16,25) to a point at coordinates (16+7,25+7), i.e., (16,25), (16,26), (16,27), (16,28), (16,29), (16,30), (16,31), (16,32), (17,25), . . . , (17,32), (18,25), . . . , (18,32), (19,25), . . . , (19,32), (20,25), . . . , (20,32), (21,25), . . . , (21,32), (22, 25), . . . , (22,32), (23,25), . . .
- an amount of light used to capture an image is gradually increased so that: an amount of light used to capture an image 01 is the smallest; an amount of light used to capture an image 02 is larger than the amount light used to capture the image 01; an amount of light used to capture an image 03 is larger than the amount light used to capture the image 02; an amount of light used to capture an image 04 is larger than the amount of light used to capture the image 03; . . . ; and an amount of light used to capture an image 20 is the largest.
- the variance of the image 07 is the largest.
- the image 07, 3 images captured using amounts of light, which are immediately larger than the amount of light used to capture the image 07, and other 3 images captured using amounts of light, which are immediately smaller than the amount of light used to capture the image 07, are selected.
- the image 04, the image 05, the image 06, the image 07, the image 08, the image 09 and the image 10 are selected.
- multiple vein patterns are stored in the memory 7 ; and each time a finger is evaluated in terms of its authenticity, the vein pattern of the finger is compared with stored multiple vein patterns. For example, when it is evaluated whether or not a vein pattern is that of Mr. A, the evaluation is achieved on the basis of the correlation coefficient which is calculated between the vein pattern and each of the stored multiple vein patterns of Mr. A with the vein pattern being compared with each stored vein pattern. It is ideal that the multiple vein patterns should be different from one another to a reasonable extent.
- the authentication score (synonymous with the correlation coefficient obtained by dividing the value of cross-correlation by the value of autocorrelation) concerning some vein pattern may be low.
- the vein pattern which has just been scanned should be stored in the memory 7 in exchange for deleting a vein pattern which has the lowest authentication score among the vein patterns stored in the memory 7 .
- a subject of evaluation is a child
- the child's growth changes the size of the finger.
- the change in the size of the finger is likely to subtly change the characteristics of the vein pattern of the finger. If the vein pattern database is not updated despite the change, the authentication score may decrease gradually. If the database is updated each time a predetermined condition is satisfied, the authentication performance can be kept optimal.
- a habitual manner of placing the finger on the authentication apparatus for the evaluation is likely to become different from a manner of placing the finger on the apparatus during the registration.
- a habitual manner of placing the finger on the apparatus is likely to change gradually.
- the authentication score is likely to decrease to an extent that the vein pattern needs to be registered again. If the database is updated each time a predetermined condition is satisfied, the authentication performance can be kept optimal.
- vein patterns P11, P12 and P13 3 vein patterns (patterns P11, P12 and P13) are registered; and when Mr. A places the same finger on the authentication apparatus, a vein pattern P14 of the finger of Mr. A is scanned by the camera 8 .
- the authentication score of the vein pattern P11 is smaller than the authentication score of the vein pattern P12 which is smaller than the authentication score of the vein pattern P13, the number of times the vein pattern 13 is judged as suitable increases by one.
- the process of scanning a vein pattern of the same finger of Mr. A, which is placed on the authentication apparatus, is repeated 9 times using the camera 8 ; the number of times the vein pattern P13 is judged as suitable increases by 4; the number of times the vein pattern P12 is judged as suitable increases by 5; and the number of times the vein pattern P11 is judged as suitable does not increase at all.
- vein pattern P11 is deleted from the memory 7 , and the new vein pattern scanned by the camera 8 is instead stored in the memory 7 .
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- Pathology (AREA)
- Public Health (AREA)
- Vascular Medicine (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Collating Specific Patterns (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Input (AREA)
- Image Analysis (AREA)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2007227124A JP4996394B2 (ja) | 2007-08-31 | 2007-08-31 | 認証装置及び認証方法 |
JP2007-227124 | 2007-08-31 | ||
PCT/JP2008/065297 WO2009028551A1 (ja) | 2007-08-31 | 2008-08-27 | 認証装置及び認証方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20100278397A1 US20100278397A1 (en) | 2010-11-04 |
US8194942B2 true US8194942B2 (en) | 2012-06-05 |
Family
ID=40387268
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/675,207 Expired - Fee Related US8194942B2 (en) | 2007-08-31 | 2008-08-27 | Authentication device and authentication method |
Country Status (3)
Country | Link |
---|---|
US (1) | US8194942B2 (ja) |
JP (1) | JP4996394B2 (ja) |
WO (1) | WO2009028551A1 (ja) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103778423A (zh) * | 2013-11-25 | 2014-05-07 | 中国人民解放军国防科学技术大学 | 基于灰度近邻和方差约束的手指静脉纹路修复方法 |
US10485431B1 (en) | 2018-05-21 | 2019-11-26 | ARC Devices Ltd. | Glucose multi-vital-sign system in an electronic medical records system |
US10492684B2 (en) | 2017-02-21 | 2019-12-03 | Arc Devices Limited | Multi-vital-sign smartphone system in an electronic medical records system |
US10506926B2 (en) | 2017-02-18 | 2019-12-17 | Arc Devices Limited | Multi-vital sign detector in an electronic medical records system |
US10602987B2 (en) | 2017-08-10 | 2020-03-31 | Arc Devices Limited | Multi-vital-sign smartphone system in an electronic medical records system |
US11504014B2 (en) | 2020-06-01 | 2022-11-22 | Arc Devices Limited | Apparatus and methods for measuring blood pressure and other vital signs via a finger |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2410485B1 (en) * | 2009-03-17 | 2019-01-09 | Fujitsu Limited | Data collating device and program |
WO2012020718A1 (ja) * | 2010-08-12 | 2012-02-16 | 日本電気株式会社 | 画像処理装置、画像処理方法及び画像処理プログラム |
JP5685272B2 (ja) * | 2011-02-15 | 2015-03-18 | 富士通フロンテック株式会社 | 認証装置、認証プログラム、および認証方法 |
WO2012114474A1 (ja) * | 2011-02-23 | 2012-08-30 | 富士通フロンテック株式会社 | 生体情報取得装置、および生体情報取得方法 |
WO2013088566A1 (ja) * | 2011-12-15 | 2013-06-20 | 富士通株式会社 | 静脈認証方法、画像処理方法及び静脈認証装置 |
CN111246621B (zh) * | 2020-01-10 | 2022-03-15 | 江门市征极光兆科技有限公司 | 一种基于WiFi技术的幻彩灯带控制*** |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020054694A1 (en) * | 1999-03-26 | 2002-05-09 | George J. Vachtsevanos | Method and apparatus for analyzing an image to direct and identify patterns |
US6526160B1 (en) * | 1998-07-17 | 2003-02-25 | Media Technology Corporation | Iris information acquisition apparatus and iris identification apparatus |
US6545715B1 (en) * | 1997-05-21 | 2003-04-08 | Samsung Electronics Co., Ltd. | Apparatus and method for controlling focus using adaptive filter |
JP2003150962A (ja) | 2001-11-08 | 2003-05-23 | Sony Corp | 個人認証装置、個人認証方法、プログラム |
US20040208395A1 (en) * | 2003-04-18 | 2004-10-21 | Konica Minolta Photo Imaging, Inc. | Image processing method, image processing apparatus, and image processing program |
JP2005346654A (ja) | 2004-06-07 | 2005-12-15 | Glory Ltd | 画像認識装置、画像認識方法およびその方法をコンピュータに実行させるプログラム |
US7310434B2 (en) * | 2001-07-16 | 2007-12-18 | Canon Kabushiki Kaisha | Image processing apparatus and method |
US20080025580A1 (en) * | 2006-07-31 | 2008-01-31 | Lumidigm, Inc. | Biometrics with spatiospectral spoof detection |
US7804984B2 (en) * | 2006-07-31 | 2010-09-28 | Lumidigm, Inc. | Spatial-spectral fingerprint spoof detection |
US7889885B2 (en) * | 2005-11-23 | 2011-02-15 | Pitney Bowes Inc. | Method for detecting perforations on the edge of an image of a form |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4344986B2 (ja) * | 2002-05-13 | 2009-10-14 | ソニー株式会社 | 認証方法及び認証装置 |
JP4315665B2 (ja) * | 2002-10-30 | 2009-08-19 | シャープ株式会社 | 太陽電池モジュールの端面封止部材及びそれを用いた太陽電池モジュール |
-
2007
- 2007-08-31 JP JP2007227124A patent/JP4996394B2/ja not_active Expired - Fee Related
-
2008
- 2008-08-27 US US12/675,207 patent/US8194942B2/en not_active Expired - Fee Related
- 2008-08-27 WO PCT/JP2008/065297 patent/WO2009028551A1/ja active Application Filing
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6545715B1 (en) * | 1997-05-21 | 2003-04-08 | Samsung Electronics Co., Ltd. | Apparatus and method for controlling focus using adaptive filter |
US6526160B1 (en) * | 1998-07-17 | 2003-02-25 | Media Technology Corporation | Iris information acquisition apparatus and iris identification apparatus |
US6650779B2 (en) * | 1999-03-26 | 2003-11-18 | Georgia Tech Research Corp. | Method and apparatus for analyzing an image to detect and identify patterns |
US20020054694A1 (en) * | 1999-03-26 | 2002-05-09 | George J. Vachtsevanos | Method and apparatus for analyzing an image to direct and identify patterns |
US7310434B2 (en) * | 2001-07-16 | 2007-12-18 | Canon Kabushiki Kaisha | Image processing apparatus and method |
JP2003150962A (ja) | 2001-11-08 | 2003-05-23 | Sony Corp | 個人認証装置、個人認証方法、プログラム |
US7174032B2 (en) | 2001-11-08 | 2007-02-06 | Sony Corporation | Apparatus, method, and program for personal identification |
US20040208395A1 (en) * | 2003-04-18 | 2004-10-21 | Konica Minolta Photo Imaging, Inc. | Image processing method, image processing apparatus, and image processing program |
JP2005346654A (ja) | 2004-06-07 | 2005-12-15 | Glory Ltd | 画像認識装置、画像認識方法およびその方法をコンピュータに実行させるプログラム |
US7835547B2 (en) | 2004-06-07 | 2010-11-16 | Glory Ltd. | Image recognition device, image recognition method, and program for causing computer to execute the method |
US7889885B2 (en) * | 2005-11-23 | 2011-02-15 | Pitney Bowes Inc. | Method for detecting perforations on the edge of an image of a form |
US20080025580A1 (en) * | 2006-07-31 | 2008-01-31 | Lumidigm, Inc. | Biometrics with spatiospectral spoof detection |
US7804984B2 (en) * | 2006-07-31 | 2010-09-28 | Lumidigm, Inc. | Spatial-spectral fingerprint spoof detection |
Non-Patent Citations (3)
Title |
---|
English abstract of JP2003-150962 A published May 23, 2003. |
English abstract of JP2005-346654 A published Dec. 15, 2005. |
Office action issued Jan. 24, 2012 in a counterpart application JP2007-227124. |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103778423A (zh) * | 2013-11-25 | 2014-05-07 | 中国人民解放军国防科学技术大学 | 基于灰度近邻和方差约束的手指静脉纹路修复方法 |
US10506926B2 (en) | 2017-02-18 | 2019-12-17 | Arc Devices Limited | Multi-vital sign detector in an electronic medical records system |
US10492684B2 (en) | 2017-02-21 | 2019-12-03 | Arc Devices Limited | Multi-vital-sign smartphone system in an electronic medical records system |
US10667688B2 (en) | 2017-02-21 | 2020-06-02 | ARC Devices Ltd. | Multi-vital sign detector of SpO2 blood oxygenation and heart rate from a photoplethysmogram sensor and respiration rate, heart rate variability and blood pressure from a micro dynamic light scattering sensor in an electronic medical records system |
US10602987B2 (en) | 2017-08-10 | 2020-03-31 | Arc Devices Limited | Multi-vital-sign smartphone system in an electronic medical records system |
US10485431B1 (en) | 2018-05-21 | 2019-11-26 | ARC Devices Ltd. | Glucose multi-vital-sign system in an electronic medical records system |
US11504014B2 (en) | 2020-06-01 | 2022-11-22 | Arc Devices Limited | Apparatus and methods for measuring blood pressure and other vital signs via a finger |
US12036005B2 (en) | 2020-06-01 | 2024-07-16 | Arc Devices Limited | Apparatus and methods for measuring blood pressure and other vital signs via a finger |
Also Published As
Publication number | Publication date |
---|---|
WO2009028551A1 (ja) | 2009-03-05 |
JP2009059249A (ja) | 2009-03-19 |
US20100278397A1 (en) | 2010-11-04 |
JP4996394B2 (ja) | 2012-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8194942B2 (en) | Authentication device and authentication method | |
US7778483B2 (en) | Digital image processing method having an exposure correction based on recognition of areas corresponding to the skin of the photographed subject | |
JP7304082B2 (ja) | 三次元実物体を実物体の二次元のスプーフと区別するための方法 | |
US8666121B2 (en) | Vein authentication device which verifies a target that is compared to the extracted piece of vein information of a finger | |
US7925093B2 (en) | Image recognition apparatus | |
US8229179B2 (en) | Finger vein authentication apparatus and finger vein authentication method | |
EP1918872B1 (en) | Image segmentation method and system | |
JP4078334B2 (ja) | 画像処理装置および画像処理方法 | |
WO2006036027A1 (ja) | 画像処理装置および方法、ならびに画像処理プログラム | |
US8280122B2 (en) | Registration device, collation device, extraction method, and program | |
CN110807759B (zh) | 照片质量的评价方法及装置、电子设备、可读存储介质 | |
CN101165706B (zh) | 图像处理装置和图像获取方法 | |
CN110728201B (zh) | 用于指纹识别的图像处理方法及装置 | |
US7174032B2 (en) | Apparatus, method, and program for personal identification | |
TW201822708A (zh) | 動態影像之心率活動偵測系統與其方法 | |
JP3241028B2 (ja) | 画像撮像装置及びその画像撮像方法 | |
JP2004005384A (ja) | 画像処理方法、画像処理装置、プログラム及び記録媒体、自動トリミング装置、並びに肖像写真撮影装置 | |
KR101769741B1 (ko) | 동공 탐지를 통한 홍채 인식 방법 및 홍채 인식 장치 | |
CN107145741A (zh) | 基于图像分析的耳诊数据采集方法及装置 | |
CN110705409A (zh) | 指纹采集*** | |
KR100888674B1 (ko) | 주파수영역을 이용한 지폐의 유사도 측정방법 | |
KR20180012638A (ko) | 누적 채널 특징점을 이용한 영상 인식에서의 객체 검출 방법 및 장치 | |
WO2021220444A1 (ja) | 肌評価係数の学習装置、肌評価指標の推定装置、肌評価係数の学習方法、肌評価指標の推定方法、フォーカス値の取得方法および肌滑らかさの取得方法 | |
JPH11283036A (ja) | 対象物検出装置及び対象物検出方法 | |
CN113361506B (zh) | 一种面向移动端的人脸识别方法与*** |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FIT DESIGN SYSTEM CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TOBE, TAKEO;MIZUNO, KENZO;TSUJI, TETSUJI;AND OTHERS;REEL/FRAME:024665/0878 Effective date: 20100708 |
|
AS | Assignment |
Owner name: SHOBAYASHI, MASAYUKI, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FIT DESIGN SYSTEM CO. LTD.;REEL/FRAME:033167/0973 Effective date: 20140624 |
|
AS | Assignment |
Owner name: MANAGEMENT SERVICE CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHOBAYASHI, MASAYUKI;REEL/FRAME:034008/0512 Effective date: 20141020 |
|
AS | Assignment |
Owner name: FIT DESIGN SYSTEM CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MANAGEMENT SERVICE CORPORATION;REEL/FRAME:035853/0737 Effective date: 20150615 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20160605 |