US20200387688A1 - Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same - Google Patents
Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same Download PDFInfo
- Publication number
- US20200387688A1 US20200387688A1 US16/799,175 US202016799175A US2020387688A1 US 20200387688 A1 US20200387688 A1 US 20200387688A1 US 202016799175 A US202016799175 A US 202016799175A US 2020387688 A1 US2020387688 A1 US 2020387688A1
- Authority
- US
- United States
- Prior art keywords
- light sources
- fingerprint recognition
- setting values
- light source
- source setting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G06K9/00033—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/13—Sensors therefor
- G06V40/1312—Sensors therefor direct reading, e.g. contactless acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/13—Sensors therefor
- G06V40/1318—Sensors therefor using electro-optical elements or layers, e.g. electroluminescent sensing
-
- G06K9/00026—
-
- G06K9/0004—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/1335—Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/12—Fingerprints or palmprints
- G06V40/1347—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03547—Touch pads, in which fingers can move on a surface
Definitions
- Example embodiments relate generally to semiconductor integrated circuits, and more particularly to methods of setting light sources in display panels for optical fingerprint recognition, and methods of performing optical fingerprint recognition using the methods of setting the light sources.
- Biometric information is widely used in personal authentication because of its invariability and uniqueness.
- One type of biometric information is a fingerprint.
- Fingerprint recognition may be performed conveniently and serves as an excellent way of determining the identity of a person.
- Optical fingerprint recognition obtains a fingerprint image based on differences in light reflected by ridges and valleys of a finger.
- obtaining an accurate fingerprint image has proven to be difficult because the differences in reflected light tend to be very small.
- a method of setting light sources in a display panel for optical fingerprint recognition comprising driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data.
- a method of setting light sources in a display panel for optical fingerprint recognition comprising driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and determining final light source setting values such that powers for driving light sources at different positions among the portion of light sources are different, the final light source setting values being determined to optimize a magnitude of an interference signal included in the initial calibration data.
- a method of performing optical fingerprint recognition comprising obtaining light source setting values and calibration data; displaying, in a fingerprint recognition mode, a fingerprint recognition window on a partial region of a display panel based on the light source setting values; and recognizing, in the fingerprint recognition mode, a fingerprint based on reflected light of the fingerprint received through the fingerprint recognition window and based on the calibration data, wherein obtaining the light source setting values and the calibration data includes driving a portion of light sources disposed to correspond to the fingerprint recognition window among a plurality of light sources included in the display panel based on initial light source setting values; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different and/or powers for driving light sources at different positions among the some light sources are different, the final light source setting values
- FIG. 1 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments
- FIG. 2 is a plan view of an electronic device according to example embodiments
- FIG. 3 is a cross-sectional view of an example of the electronic device taken along a line A-A′ of FIG. 2 , according to example embodiments;
- FIG. 4 is a block diagram illustrating an example of the electronic device of FIG. 2 , according to example embodiments;
- FIGS. 5A, 5B, 5C and 5D are diagrams for describing a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments;
- FIGS. 6, 7A, 7B and 7C are diagrams for describing an example of driving some light sources based on initial light source setting values and obtaining initial calibration data in the method of FIG. 1 , according to example embodiments;
- FIG. 8 is a flowchart illustrating an example of determining final light source setting values in the method of FIG. 1 , according to example embodiments;
- FIGS. 9, 10A, 10B and 10C are diagrams for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 1 , according to example embodiments;
- FIG. 11 is a flowchart illustrating another example of determining final light source setting values in the method of FIG. 1 , according to example embodiments;
- FIG. 12 is a diagram for describing another example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 1 , according to example embodiments;
- FIG. 13 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments
- FIG. 14 is a flowchart illustrating an example of determining final light source setting values in the method of FIG. 13 , according to example embodiments;
- FIG. 15 is a diagram for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 13 , according to example embodiments;
- FIG. 16 is a diagram for describing an example of determining final light source setting values in the method of FIG. 13 , according to example embodiments;
- FIG. 17 is a flowchart illustrating another example of determining final light source setting values in the method of FIG. 13 , according to example embodiments;
- FIGS. 18 and 19 are diagrams for describing another example of determining final light source setting values in the method of FIG. 13 , according to example embodiments;
- FIG. 20 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments
- FIG. 21 is a diagram for describing an example of determining final light source setting values in the method of FIG. 20 , according to example embodiments;
- FIG. 22 is a flowchart illustrating a method of performing optical fingerprint recognition according to example embodiments.
- FIG. 23 is a flowchart illustrating an example of recognizing a fingerprint in the method of FIG. 22 , according to example embodiments;
- FIGS. 24 and 25 are flowcharts illustrating a method of performing optical fingerprint recognition according to example embodiments
- FIG. 26 is a diagram illustrating an example process for a method of performing optical fingerprint recognition according to example embodiments.
- FIG. 27 is a block diagram illustrating an electronic device according to example embodiments.
- light source setting values may be changed such that powers for driving light sources of different colors within a fingerprint recognition window are different and/or such that the powers for driving the light sources at different positions within the fingerprint recognition window are different, thereby minimizing the contrast of an interference signal included in initial calibration data and/or optimizing a magnitude of the interference signal included in the initial calibration data.
- Final calibration data may then be obtained based on the changed light source setting values. Accordingly, robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to external temperature of the display panel and pressure applied to the display panel and efficient optical fingerprint recognition may be supported.
- FIG. 1 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments.
- a method of setting light sources in a display panel for optical fingerprint recognition is performed by an electronic device that includes the display panel and a fingerprint recognition sensor.
- the display panel includes a plurality of light sources, and the fingerprint recognition sensor performs the optical fingerprint recognition using light provided from the plurality of light sources.
- FIGS. 2 through 4 A detailed configuration of the electronic device will be described with reference to FIGS. 2 through 4 .
- some light sources among the plurality of light sources included in the display panel are driven based on initial light source setting values (step S 100 ).
- the light sources that are driven are light sources disposed to correspond to a fingerprint recognition window which is a partial region of the display panel.
- the initial light source setting values may be the same value for all of the driven light sources.
- all of the driven light sources may emit light to have the same grayscale value based on the initial light source setting values.
- the initial light source setting values may be the same for a portion of the driven light sources.
- initial calibration data is obtained based on reflected light received through the fingerprint recognition window (step S 200 ).
- the initial calibration data may be obtained from a fingerprint recognition sensor.
- all of the driven light sources may be substantially simultaneously or concurrently turned on based on the initial light source setting values.
- Light generated from the driven light sources may be reflected by an object on the fingerprint recognition window, the reflected light of the object may be provided to the fingerprint recognition sensor, and thus the fingerprint recognition sensor may obtain the initial calibration data based on the reflected light of the object.
- the object may be a specific object used for obtaining calibration data, rather than a finger of a user.
- a portion of the driven light sources may be substantially simultaneously or concurrently turned on based on the initial light source setting values.
- final light source setting values are determined such that powers for driving light sources of different colors among the driven light sources are different (step S 300 ).
- the contrast of the interference signal may be minimized by the final light source setting values.
- the final light source setting values may be different values for the light sources of different colors among the driven light sources.
- the light sources of different colors may emit light to have different grayscale values based on the final light source setting values.
- the initial calibration data may include a low frequency component and a high frequency component
- the interference signal may indicate the high frequency component included in the initial calibration data
- some light sources may be driven based on the final light source setting values (step S 400 ), and while driving the light sources based on the final light source setting values, final calibration data may be obtained based on reflected light received through the fingerprint recognition window (step S 500 ).
- all of the driven light sources may be substantially simultaneously or concurrently turned on based on the final light source setting values.
- the reflected light of the object which is a specific object used for obtaining calibration data, may be provided to the fingerprint recognition sensor.
- a portion of the driven light sources may be substantially simultaneously or concurrently turned on based on the final light source setting values.
- a display panel has a complicated internal structure including patterns of multi-layered wirings, electrodes, and the like.
- reflected light received through the fingerprint recognition window includes information of a fingerprint that exists on the user's finger and information of the internal structure of the display panel.
- the internal structure may be a bottom view of the display panel.
- an interference component In order to obtain only the information of the fingerprint, only information on the internal structure of the display panel, which is an interference component, should be obtained first, and compensation for removing the interference component from an image signal that includes both the interference component and the information of the fingerprint obtained by the user's finger should be performed.
- Such interference component may be referred to as calibration data.
- the compensation may be performed by removing the calibration data from the read data from the user's finger, or may be performed by determining compensation data that when added to the read data from the user's finger negates the calibration data.
- the initial calibration data may be obtained while the light sources within the fingerprint recognition window emit light at the same grayscale, and the light source setting values may be changed such that the powers for driving the light sources of different colors within the fingerprint recognition window are different, thereby minimizing the contrast of the interference signal included in the initial calibration data.
- the final calibration data may then be obtained while the light sources of different colors within the fingerprint recognition window emit light with different grayscales. Accordingly, a robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to external temperature of the display device, pressure applied to the display panel, etc., and the efficient optical fingerprint recognition may be supported.
- step S 300 of FIG. 1 the final light source setting values are determined at once or at one time
- example embodiments are not limited thereto, and an operation of determining the final light source setting values may be repeatedly performed until the contrast of the interference signal included in the initial calibration data is minimized, e.g., until the final calibration data has an optimal value for fingerprint image restoration.
- FIG. 2 is a plan view of an electronic device according to example embodiments.
- an electronic device 100 includes a display panel 110 to interface with a user.
- the display panel 110 outputs visual information to the user.
- the electronic device 100 may further include a touch sensor panel for sensing a touch input of the user, and the user may input a signal to the electronic device 100 through the touch sensor panel.
- a fingerprint recognition window FRW may be provided on the display panel 110 .
- a fingerprint recognition sensor for fingerprint detection may be disposed to spatially correspond to a location of the fingerprint recognition window FRW.
- the fingerprint recognition window FRW is illustrated as a rectangle in FIG. 2 and subsequent figures, a shape and/or a size of the fingerprint recognition window FRW may be changed according to example embodiments.
- the electronic device 100 may be or may include any mobile system, such as a mobile phone, a smart phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a portable game console, a music player, a camcorder, a video player, a navigation device, a wearable device, an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR) device, an augmented reality (AR) device, a robotic device, etc.
- PDA personal digital assistant
- PMP portable multimedia player
- digital camera a portable game console
- music player a camcorder
- video player a navigation device
- a wearable device an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR) device, an augmented reality (AR) device, a robotic device, etc.
- IoT internet of things
- the present disclosure may provide an interface used to detect a fingerprint. For example, a function of fingerprint detection may be performed when the user contacts or approaches the touch sensor panel or the display panel 110 .
- the interface and the fingerprint recognition sensor used for fingerprint detection may share an area on the electronic device 100 with the touch sensor panel and the display panel 110 , and thus the interface and the fingerprint recognition sensor may not require an additional area on the electronic device 100 . Accordingly, it may be possible to reduce the size of the electronic device 100 , or a spare area outside of the display panel 110 that would otherwise be used for the interface may be used for other purpose(s).
- FIG. 3 is a cross-sectional view of an example of the electronic device taken along a line A-A′ of FIG. 2 , according to example embodiments.
- the fingerprint recognition window FRW may be provided on a partial region (or portion) of the display panel 110 .
- the FRW may be displayed on the partial region (or portion) of the display panel 110 in a fingerprint recognition mode.
- the display panel 110 may include a plurality of light sources 112 .
- the plurality of light sources 112 may be included in a plurality of pixels included in the display panel 110 .
- the light sources 114 a and 114 b that are disposed to correspond to the fingerprint recognition window FRW may be driven substantially simultaneously in the fingerprint recognition mode.
- the light sources 114 a and 114 b that are driven and emit light are hatched.
- the powers (or power sources) for driving the light sources of different colors may be differently set.
- the light sources 114 a and 114 b with different powers are indicated by different hatched lines.
- FIG. 3 shows light sources 114 a and 114 b in an alternating pattern.
- a fingerprint recognition sensor 130 may be disposed under the display panel 110 such that the fingerprint recognition sensor 130 may at least partially overlap the fingerprint recognition window FRW in a vertical direction.
- the display panel 110 may include a first surface 110 a on which an image is displayed and a second surface 110 b opposite to the first surface, and the fingerprint recognition sensor 130 may be disposed under the second surface 110 b of the display panel 110 .
- the fingerprint recognition sensor 130 may include a lens 132 and an image sensor 134 .
- the lens 132 may be disposed under the display panel 110 (e.g., disposed between the display panel 110 and the image sensor 134 ), and may concentrate reflected light received through the fingerprint recognition window FRW on the image sensor 134 .
- the image sensor 134 may be disposed under the lens 132 , and may generate an image signal for an object on a partial region of the FRW based on the reflected light concentrated by the lens 132 .
- the fingerprint recognition sensor 130 may be implemented in the form of a compact camera module (CCM) including the lens 132 and the image sensor 134 .
- CCM compact camera module
- the fingerprint recognition sensor 130 may capture an image signal for the fingerprint or information associated with a shape of the fingerprint (e.g., a fingerprint image) based on the reflected light of the fingerprint received through the fingerprint recognition window FRW.
- the fingerprint recognition sensor 130 may capture an image signal for the calibration data (e.g., an image of the internal structure of the display panel 110 ) based on the reflected light of the object 20 received through the fingerprint recognition window FRW.
- the fingerprint recognition sensor 130 may further include a filter for adjusting a frequency characteristic and/or a polarization characteristic of the reflected light which is to be provided to the image sensor 134 .
- FIG. 4 is a block diagram illustrating an example of the electronic device of FIG. 2 , according to example embodiments.
- the electronic device 100 may include the display panel 110 and the fingerprint recognition sensor 130 .
- the electronic device 100 may further include a display driver 120 , a processor 140 and a memory 150 .
- the display panel 110 outputs visual information to the user.
- the display panel 110 may include a plurality of pixels which are arranged along rows and columns to display an image.
- FIG. 4 illustrates one pixel PX as an example.
- Each pixel may be configured to emit light of a specific color which forms the image.
- the display panel 110 may display a desired or intended image.
- the display panel 110 may be an electroluminescent display panel.
- the electroluminescent display panel may be driven with rapid response speed and low power consumption using a light emitting diode (LED) or an organic light emitting diode (OLED) that generates light by recombination of electrons and holes.
- LED light emitting diode
- OLED organic light emitting diode
- the pixels PX of the electroluminescent display panel may emit light by themselves, and the reflected light received through the fingerprint recognition window FRW may be provided to the fingerprint recognition sensor 130 under the display panel 110 through a space (gap) between the pixels PX.
- light emitting diodes or organic light emitting diodes included in the pixels PX may correspond to the light sources included in the display panel according to example embodiments.
- the display panel 110 may be any display panel having a structure in which the reflected light received through the fingerprint recognition window FRW may be provided to the fingerprint recognition sensor 130 .
- the display driver 120 may control operations of the display panel 110 and may drive the display panel 110 .
- the display driver 120 may suitably drive each pixel of the display panel 110 in response to a command of the processor 140 such that the desired or intended image is displayed on the display panel 110 .
- the display driver 120 may partially drive the display panel 110 such that pixels corresponding to the fingerprint recognition window FRW emit light.
- the display driver 120 may include a data driver, a scan driver, a timing controller, a gamma circuit, etc.
- the fingerprint recognition sensor 130 may be used to detect a fingerprint.
- the fingerprint recognition sensor 130 may generate/output an image signal associated with an object which is on the fingerprint recognition window FRW.
- the fingerprint recognition sensor 130 may operate to obtain an image signal associated with a fingerprint of a finger which contacts or approaches the fingerprint recognition window FRW.
- the fingerprint recognition sensor 130 may include the lens 132 and the image sensor 134 .
- the fingerprint recognition sensor 130 may provide a function of optical fingerprint recognition or optics-based fingerprint detection.
- the image sensor 134 included in the fingerprint recognition sensor 130 may include photo-diode(s) which is capable of generating current in response to light.
- the processor 140 may control overall operations of the electronic device 100 .
- the processor 140 may process/perform various arithmetic/logical operations based on program code to provide functions of the electronic device 100 .
- the processor 140 may communicate with the display driver 120 , the fingerprint recognition sensor 130 and the memory 150 .
- the processor 140 may control operations of the display driver 120 , the fingerprint recognition sensor 130 and the memory 150 .
- the processor 140 may process program code including commands, requests, responses, and/or the like, which are associated with operations of the display driver 120 , the fingerprint recognition sensor 130 and the memory 150 .
- the processor 140 may provide a variety of information to the display driver 120 , to display the desired or intended image on the display panel 110 .
- the processor 140 may control an operation timing/sequence of the display panel 110 and the fingerprint recognition sensor 130 such that the fingerprint recognition sensor 130 generates the image signals associated with the fingerprint and/or the calibration data.
- the processor 140 may generate and analyze image information associated with the fingerprint and/or the calibration data based on the image signals output from the fingerprint recognition sensor 130 .
- the processor 140 may store associated data in the memory 150 or may load the associated data from the memory 150 .
- the processor 140 may include one or more special-purpose circuits (e.g., a field programmable gate array (FPGA), application specific integrated chips (ASICs), and/or the like) to perform various operations.
- the processor 140 may include one or more processor cores which are capable of performing various operations.
- the processor 140 may be implemented with a general-purpose processor, a special-purpose processor, or an application processor.
- the memory 150 may store data related to the operation of the electronic device 100 .
- the memory 150 may store the program code mentioned above, and may store the initial light source setting values, the initial calibration data, the final light source setting values, the final calibration data, etc. for performing the method of setting the light sources in the display panel according to example embodiments.
- the memory 150 may include at least one of various volatile memories such as a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like, and/or at least one of various nonvolatile memories such as a flash memory, a phase change random access memory (PRAM), a resistance random access memory (RRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a nano floating gate memory (NFGM), a polymer random access memory (PoRAM), or the like.
- various volatile memories such as a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like
- nonvolatile memories such as a flash memory, a phase change random access memory (PRAM), a resistance random access memory (RRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a nano floating gate memory (NFGM), a polymer random access memory (PoRAM), or the like.
- DRAM dynamic random access memory
- SRAM static random
- the display driver 120 , the fingerprint recognition sensor 130 , the processor 140 and the memory 150 may be respectively implemented with separate circuits/modules/chips. In other example embodiments, on the basis of a function, some of the display driver 120 , the fingerprint recognition sensor 130 , the processor 140 and the memory 150 may be combined into one circuit/module/chip, or may be further separated into a plurality of circuits/modules/chips.
- the electronic device 100 may perform the method of setting the light sources in the display panel described with reference to FIG. 1 .
- the display panel 110 and the display driver 120 under control of the processor 140 may perform steps S 100 and S 400 in FIG. 1
- the fingerprint recognition sensor 130 under control of the processor 140 may perform steps S 200 and S 500 in FIG. 1
- the processor 140 may perform step S 300 in FIG. 1 .
- the electronic device 100 may perform a method of setting light sources which will be described with reference to FIGS. 13 and 20 , and/or a method of performing optical fingerprint recognition which will be described with reference to FIG. 22 .
- FIGS. 5A, 5B, 5C and 5D are diagrams for describing a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments.
- FIG. 5A illustrates an image obtained based on reflected light by an object used for obtaining calibration data, without a fingerprint (e.g., without a finger of a user).
- the image of FIG. 5A may correspond to calibration data (e.g., the initial calibration data) obtained by a fingerprint recognition sensor, and may include only information of an internal structure of a display panel.
- FIG. 5B illustrates an image obtained based on reflected light by the fingerprint.
- the image of FIG. 5B may also be obtained by the fingerprint recognition sensor, and may include both information of the fingerprint and the information of the internal structure of the display panel.
- FIGS. 5C and 5D illustrate examples of a pure fingerprint image obtained based on the images of FIGS. 5A and 5B .
- a first image e.g., the image of FIG. 5A
- a second image e.g., the image of FIG. 5B
- a pure fingerprint image may be obtained by performing compensation to remove the interference component.
- the pure fingerprint image may be obtained by performing a simple face subtraction to obtain a difference between the second image and the first image.
- a pure fingerprint image in which the interference component is completely removed may be obtained.
- a residual component RES may remain or may be left without the interference component being completely removed.
- the interference component may vary depending on temperature of the display panel and/or pressure applied to the display panel. Particularly, deformation (e.g., rotation, scaling, translation, etc.) may occur on the interference component in the second image (i.e., FIG. 5B ), and thus the residual component RES may occur because there is a spatial mismatch between the interference component in the first image ( FIG. 5A ) and the interference component in the second image ( FIG. 5B ).
- the powers of the light sources within the fingerprint recognition window may be adjusted to minimize the contrast of the interference signal included in the initial calibration data as described above with reference to FIG. 1 , or the powers of the light sources within the fingerprint recognition window may be adjusted to optimize a magnitude of the interference signal included in the initial calibration data as will be described above with reference to FIG. 13 .
- FIGS. 6, 7A, 7B and 7C are diagrams for describing an example of driving some light sources based on initial light source setting values and an example of obtaining initial calibration data in the method of FIG. 1 , according to example embodiments.
- the some light sources 114 a and 114 b that are disposed to correspond to the fingerprint recognition window FRW may be substantially simultaneously turned on based on initial light source setting values ILSET, and initial calibration data ICDAT may be obtained based on light emitted from the some light sources 114 a and 114 b and reflected by an object 20 .
- the initial light source setting values ILSET may be the same value for all of the turned on light sources 114 a and 114 b, and all of the turned on light sources 114 a and 114 b may emit light to have the same grayscale value based on the initial light source setting values ILSET, and thus all of the light sources 114 a and 114 b that emit light are indicated by the same hatched line in FIG. 6 .
- the initial light source setting values ILSET may be stored in the memory 150 in FIG. 4 and provided from the memory 150 , or may be provided from the processor 140 in FIG. 4 .
- the object 20 may be a specific object used for obtaining calibration data, rather than a finger of the user.
- the object 20 may be any object of white flatness and high reflectivity without bending.
- the initial calibration data ICDAT may be provided to and stored in the memory 150 , or may be provided to the processor 140 to be used to determine the final light source setting values.
- FIG. 7A illustrates an image corresponding to the initial calibration data ICDAT of FIG. 6 obtained by the fingerprint recognition sensor 130 .
- FIG. 7B illustrates a luminance change occurring along a line B-B′ in the image of FIG. 7A .
- an X-axis represents a position along the line B-B′ in FIG. 7A
- a Y-axis represents luminance at a corresponding position.
- the initial calibration data ICDAT may include a low frequency component and a high frequency component.
- the low frequency component may represent an overall change in luminance by a relatively bright center and a relatively dark edge in the image of FIG. 7A .
- the high frequency component may represent a local luminance change (e.g., fluctuation) where light and dark portions are alternately repeated at a center of the image of FIG. 7A .
- the high frequency component may be generated by the internal structure of the display panel 110 (e.g., reflected light may or may not pass through the patterns). Since the high frequency component has a frequency similar to that of the fingerprint signal by ridges and valleys of the fingerprint, it may be difficult to remove only the high frequency component.
- FIG. 7C illustrates that the residual component RES in FIG. 5D may occur when the initial calibration data ICDAT is used as it is.
- a solid line represents the interference signal included in the initial calibration data ICDAT illustrated in FIG. 7B
- a dotted line represents the interference signal included in the image including both the fingerprint and the interference component.
- FIG. 8 is a flowchart illustrating an example of determining final light source setting values in the method of FIG. 1 , according to example embodiments.
- luminance of first light sources among the some light sources may be determined as a first value (step S 310 ), and luminance of second light sources among the some light sources may be determined as a second value different from the first value (step S 320 ).
- the first light sources may have a first color
- the second light sources may have a second color different from the first color.
- the first color of the first light sources may be blue, and the second color of the second light sources may be green.
- example embodiments are not limited thereto, and any two of different colors of the plurality of light sources included in the display panel may be selected as the first and second colors, respectively.
- FIGS. 9, 10A, 10B and 10C are diagrams for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 1 , according to example embodiments. The descriptions repeated with FIG. 6 will be omitted.
- the some light sources 114 a and 114 b that are disposed to correspond to the fingerprint recognition window FRW may be substantially simultaneously turned on based on final light source setting values FLSET, and final calibration data FCDAT may be obtained based on light emitted from the some light sources 114 a and 114 b and reflected by the object 20 .
- the final light source setting values FLSET may be different values for light sources of different colors among the some light sources 114 a and 114 b.
- the light sources 114 a and 114 b may emit light to have different brightness or grayscale values for each color, and thus the light sources 114 a and 114 b that emit light with different brightness or grayscale values are indicated by different hatched lines in FIG. 9 .
- the final light source setting values FLSET may be stored in the memory 150 in FIG. 4 and provided from the memory 150 , or may be provided from the processor 140 in FIG. 4 .
- FIG. 10A illustrates an image corresponding to the final calibration data FCDAT of FIG. 9 obtained by the fingerprint recognition sensor 130 .
- FIG. 10B illustrates luminance change occurring along a line C-C′ in the image of FIG. 10A .
- an X-axis represents a position along the line C-C′ in FIG. 10A and a Y-axis represents luminance at a corresponding position.
- the final calibration data FCDAT may also include a low frequency component and a high frequency component.
- the low frequency component included in the final calibration data FCDAT is similar to the low frequency component included in the initial calibration data ICDAT, and the high frequency component included in the final calibration data FCDAT is reduced compared with the high frequency component included in the initial calibration data ICDAT.
- FIG. 10C illustrates that the residual component RES in FIG. 5D may be reduced or eliminated when the final calibration data FCDAT is used.
- a solid line represents the interference signal included in the final calibration data FCDAT illustrated in FIG. 10B
- a dotted line represents the interference signal included in the image including both the fingerprint and the interference component.
- differences between the two interference signals may be relatively small and overall more uniform as illustrated by double-headed arrows in FIG. 10C , even if two interference signals are spatially mismatched or inconsistent.
- the residual component RES may be efficiently removed from the fingerprint image when the final calibration data FCDAT is used.
- FIG. 11 is a flowchart illustrating another example of determining final light source setting values in FIG. 1 , according to example embodiments. The descriptions repeated with FIG. 8 will be omitted.
- steps S 310 and S 320 in FIG. 11 may be substantially the same as steps S 310 and S 320 in FIG. 8 , respectively.
- Luminance of third light sources among the turned on light sources may be determined as a third value (step S 330 ).
- the third value may be different from the first value and the second value.
- the third light sources may have third color different from the first color and the second color.
- the first color of the first light sources may be blue
- the second color of the second light sources may be green
- the third color of the third light sources may be red
- example embodiments are not limited thereto, and any three of different colors of the plurality of light sources included in the display panel may be selected as the first, second and third colors, respectively.
- FIG. 12 is a diagram for describing another example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 1 . The descriptions repeated with FIGS. 6 and 9 will be omitted.
- light sources 114 a, 114 b and 114 c that are disposed to correspond to the fingerprint recognition window FRW may be substantially simultaneously turned on based on final light source setting values FLSET′, and final calibration data FCDAT′ may be obtained based on light emitted from the turned on light sources 114 a, 114 b and 114 c and reflected by the object 20 .
- the light sources 114 a, 114 b and 114 c that have different colors and emit light with different brightness or grayscale values are indicated by different hatched lines in FIG. 12 .
- the first, second and third values that represent the luminance of the first, second and third light sources, respectively may be obtained by Equation 1.
- Equation 1 ⁇ , ⁇ and ⁇ denote the first, second and third values, respectively, cost fn denotes a cost function, and argmin denotes a function of obtaining indices ⁇ , ⁇ and ⁇ for the cost function to have a minimum value.
- each of ⁇ , ⁇ and ⁇ may be a grayscale value of one of grayscale ranges (e.g., 0 to 255 grayscales) of the display panel.
- the cost function may be a function representing the contrast of the interference signal included in the initial calibration data, and may be a standard deviation or a difference from a low pass filtered signal (e.g., a signal from which the interference signal is removed).
- the power of each of the R, G and B channels may be set based on the ⁇ , ⁇ and ⁇ ratio calculated by Equation 1, and then the set power may be used to drive the light sources in the fingerprint recognition window.
- example embodiments have been described that the light sources of two or three different colors are set to have different grayscale values for each color, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which light sources of at least two different colors of any of a plurality of different colors included in the display panel are set to have different grayscale values for each color.
- example embodiments have been described that all the light sources in the fingerprint recognition window are set to have different grayscale values for each color, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which only a portion of the light sources in the fingerprint recognition window are set to have different grayscale values for each color.
- FIG. 13 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. The descriptions repeated with FIG. 1 will be omitted.
- steps S 100 , S 200 , S 400 and S 500 in FIG. 13 may be substantially the same as steps S 100 , S 200 , S 400 and S 500 in FIG. 1 , respectively.
- the final light source setting values are determined such that powers for driving light sources at different positions among the driven light sources are different (step S 600 ).
- the magnitude of the interference signal included in the initial calibration data is optimized by the final light source setting values.
- the final light source setting values may be different values for the light sources at different positions among the driven light sources.
- the light sources at different positions may emit light to have different grayscale values based on the final light source setting values.
- the initial calibration data may be obtained while emitting the light sources within the fingerprint recognition window at the same grayscale, and the light source setting values may be changed such that the powers for driving the light sources at different positions within the fingerprint recognition window are different, thereby optimizing the magnitude of the interference signal included in the initial calibration data.
- the final calibration data may be obtained while emitting the light sources at different positions within the fingerprint recognition window with different grayscales. Accordingly, the robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to temperature of the display panel, pressure applied to the display panel, etc., and efficient optical fingerprint recognition may be supported.
- FIG. 14 is a flowchart illustrating an example of determining final light source setting values in the method of FIG. 13 , according to example embodiments.
- luminance of first light sources among the some light sources may be determined as a first value (step S 610 ), and luminance of second light sources among the some light sources may be determined as a second value (step S 620 ).
- the second value may be different from the first value.
- the first light sources may be disposed in a first region of the partial region, and the second light sources may be disposed in a second region of the partial region different from the first region.
- FIG. 15 is a diagram for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method of FIG. 13 , according to example embodiments. The descriptions repeated with FIGS. 6, 9 and 12 will be omitted.
- the some light sources 114 a and 114 c that are disposed to correspond to the fingerprint recognition window FRW may be substantially simultaneously turned on based on final light source setting values FLSET′′, and final calibration data FCDAT′′ may be obtained based on light emitted from the some light sources 114 a and 114 c and reflected by the object 20 .
- the final light source setting values FLSET′′ may be different values for light sources at different positions among the some light sources 114 a and 114 c.
- the light sources 114 a and 114 c may emit light to have different brightness or grayscale values for each position, and thus the light sources 114 a and 114 c that emit light with different brightness or grayscale values are indicated by different hatched lines in FIG. 15 .
- FIG. 16 is a diagram for describing an example of determining final light source setting values in the method of FIG. 13 , according to example embodiments.
- a first region R 1 in which the first light sources are disposed may be a central portion of the fingerprint recognition window FRW, and a second region R 2 in which the second light sources are disposed may be an edge portion (or peripheral portion) of the fingerprint recognition window FRW that surrounds the central portion.
- the first light sources may be set to have a relatively low grayscale value or luminance (e.g., set to be relatively dark).
- FIG. 17 is a flowchart illustrating another example of determining final light source setting values in the method of FIG. 13 , according to example embodiments. The descriptions repeated with FIG. 14 will be omitted.
- steps S 610 and S 620 in FIG. 17 may be substantially the same as steps S 610 and S 620 in FIG. 14 , respectively.
- Luminance of third light sources among the turned on light sources may be determined as a third value (step S 630 ).
- the third value may be different from the first value and the second value.
- the third light sources may be disposed in a third region of the partial region different from the first region and the second region.
- An operation of obtaining the final calibration data by driving the first, second and third light sources may be similar to that described with reference to FIG. 12 .
- FIGS. 18 and 19 are diagrams for describing another example of determining final light source setting values in the method of FIG. 13 , according to example embodiments.
- a first region R 1 ′ in which the first light sources are disposed may be a central portion of the fingerprint recognition window FRW
- a second region R 2 ′ in which the second light sources are disposed may be a middle portion of the fingerprint recognition window FRW that surrounds the central portion
- a third region R 3 ′ in which the third light sources are disposed may be an edge portion of the fingerprint recognition window FRW that surrounds the middle portion.
- example embodiments have been described that the fingerprint recognition window is divided into two or three regions and the light sources in the fingerprint recognition window are set to have different grayscale values for each region
- example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which the fingerprint recognition window is divided into any number of different regions and the light sources in the fingerprint recognition window are set to have different grayscale values for each region.
- example embodiments may be employed or extended to examples in which all the light sources in the fingerprint recognition window are set to have different grayscale values for each position.
- all the light sources in the fingerprint recognition window are set to have different grayscale values based on position.
- each individual turned on light source may have a different grayscale value.
- FIG. 19 illustrates a difference in luminance according to positions of light sources in the fingerprint recognition window FRW when all the light sources in the fingerprint recognition window are set to have different grayscale values for each position.
- an X′-axis represents a position of the light sources in the fingerprint recognition window FRW along the line A-A′ in FIG. 2
- a Y′-axis represents luminance or grayscale generated by the light source disposed at corresponding position.
- luminance of the light sources in the fingerprint recognition window may be determined to be lower as a light source becomes closer to a center (or central portion).
- the interference signal may appear relatively large in the center portion
- the interference signal may appear relatively small in the edge portion
- the fingerprint signal may also appear to be weak due to vignetting of the lens.
- the powers for driving the light sources may be adjusted according to the position of each of the light sources so that the center portion is relatively dark and the edge portion is relatively bright.
- light sources at respective edge positions may be the same grayscale value, next light sources adjacent to the edge positions may have another grayscale value, etc.
- each individual light source may have a different grayscale value for each position.
- each individual light source may have a different grayscale value based on position.
- each individual turned on light source may have a different grayscale value.
- example embodiments have been described that all the light sources in the fingerprint recognition window are set to have different grayscale values for each position, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which only a portion of the light sources in the fingerprint recognition window are set to have different grayscale values for each position.
- FIG. 20 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. The descriptions repeated with FIGS. 1 and 13 will be omitted.
- steps S 100 , S 200 , S 300 , S 400 and S 500 in FIG. 20 may be substantially the same as steps S 100 , S 200 , S 300 , S 400 and S 500 in FIG. 1 , respectively, and step S 600 in FIG. 20 may be substantially the same as step S 600 in FIG. 13 .
- an order of performing steps S 300 and S 600 may be changed, and steps S 300 and S 600 may be substantially simultaneously performed.
- FIG. 21 is a diagram for describing an example of determining final light source setting values in the method of FIG. 20 , according to example embodiments.
- FIG. 21 illustrates a difference in luminance according to positions of light sources in the fingerprint recognition window FRW when all the light sources in the fingerprint recognition window are set to have different grayscale values for each color and each position.
- Luminance L 1 of the first light sources having the first color and luminance L 2 of the second light sources having the second color may be set differently as described with reference to FIGS. 1 through 12
- luminance of the light sources may be set differently for each position of the light sources as described with reference to FIGS. 13 through 19 .
- green and blue light sources disposed in the center portion may be set to have 230 and 216 grayscales, respectively
- green and blue light sources disposed in the edge portion may be set to have 254 and 255 grayscales, respectively.
- the brightness and hue of the light sources for each location may be changed together.
- FIG. 22 is a flowchart illustrating a method of performing optical fingerprint recognition according to example embodiments.
- Step S 1100 light source setting values and calibration data are obtained (step S 1100 ).
- Step S 1100 may be performed based on the method of setting the light sources in the display panel according to example embodiments described with reference to FIGS. 1 through 21 .
- the final light source setting values may be determined such that the powers for driving the light sources of different colors among the driven light sources within the fingerprint recognition window are different, such that the powers for driving the light sources at different positions among the driven light sources are different, or such that both the powers for driving the light sources of different colors and the powers for driving the light sources at different positions are different.
- the final calibration data may be obtained based on the final light source setting values, and the final light source setting values and the final calibration data may be stored as the light source setting values and the calibration data, respectively.
- the light source setting values and the calibration data may be stored in the memory 150 in FIG. 4 .
- a fingerprint recognition mode is entered (step S 1200 ).
- the fingerprint recognition mode may be entered based on a request.
- the request may be associated with a signal or an action which directs to detect a fingerprint.
- the request may occur when any input is received from a user by the electronic device 100 , when a signal/command is generated in the electronic device 100 based on the received input, and/or the like.
- the request may occur when the user contacts or approaches any area on the touch sensor panel or the display panel 110 through the object 10 (e.g., the request may occur in response to a touch of the object 10 ).
- the request may occur when the object 10 acts specific motion or gesture in the vicinity of the electronic device 100 .
- the request may occur when the electronic device 100 moves in a specific manner.
- an application running on the electronic device 100 may initiate the request.
- example embodiments are not limited thereto, and the request may be changed or modified to recognize directing of fingerprint detection.
- the request may occur while the electronic device 100 is in an idle state or the display panel 110 is not driven.
- the request may occur while the display panel 110 is in a stand-by mode.
- the stand-by mode may denote an operation mode where the display panel 110 displays a reduced amount of information or a minimal amount of information (e.g., a current time, a date, and/or the like), and may be also called as an “Always ON display (AOD)” mode, an “active display mode”, and/or the like.
- AOD Automatically ON display
- the request may occur while the display panel 110 is in a normal mode.
- the normal mode may denote an operation mode where the display panel 110 displays a variety of information according to intention of the user.
- the fingerprint recognition window is displayed on a partial region of the display panel based on the light source setting values (step S 1300 ), and the fingerprint is recognized based on reflected light of the fingerprint received through the fingerprint recognition window, and the calibration data (step S 1400 ).
- the fingerprint recognition mode may then be existed once the fingerprint is recognized.
- the fingerprint may be recognized based on the reflected light and the calibration data when the powers for driving the light sources are set to be different from each other by color and/or position. Accordingly, the robust fingerprint image restoration may be performed even if changes occur due to temperature of the display panel, pressure applied to the display panel, etc., and efficient optical fingerprint recognition may be supported.
- an operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference to FIGS. 1 through 21 may be performed when an electronic device including the display panel and the fingerprint recognition sensor is manufactured.
- the method of setting the light sources in the display panel according to example embodiments may be performed once at the time of manufacturing the electronic device, and the final light source setting values and the final calibration data may be obtained and stored in the electronic device.
- an operation of loading the final light source setting values and the final calibration data already stored in the memory 150 may be performed in step S 1100 .
- a fingerprint recognition operation of steps S 1200 , S 1300 and S 1400 may be performed by loading the stored final light source setting values and the stored final calibration data.
- the operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference to FIGS. 1 through 21 may be periodically performed.
- the operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference to FIGS. 1 through 21 may be repeatedly performed whenever use time of the display panel exceeds a threshold reference time.
- FIG. 23 is a flowchart illustrating an example of recognizing a fingerprint in the method of FIG. 22 , according to example embodiments.
- an image signal for the fingerprint may be obtained based on the reflected light (step S 1410 ).
- the image signal for the fingerprint may be obtained based on the reflected light reflected by the fingerprint and received through the fingerprint recognition window.
- Final information for the fingerprint may be obtained by subtracting the calibration data from the image signal (step S 1420 ).
- the first image e.g., the image of FIG. 5A
- the second image e.g., the image of FIG. 5B
- both the fingerprint and the interference component may be obtained next
- the pure fingerprint image may be obtained by performing the compensation to remove the interference component.
- FIGS. 24 and 25 are flowcharts illustrating a method of performing optical fingerprint recognition according to example embodiments. The descriptions repeated with FIG. 22 will be omitted.
- step S 1100 , S 1200 , S 1300 and S 1400 in FIG. 24 may be substantially the same as step S 1100 , S 1200 , S 1300 and S 1400 in FIG. 22 , respectively.
- the fingerprint recognized in the fingerprint recognition mode i.e., in step 1400
- the fingerprint of the authenticated user stored in advance and the fingerprint recognized in the fingerprint recognition mode may be compared to determine whether two fingerprints are substantially identical or the same.
- step S 1500 When the fingerprint recognized in the fingerprint recognition mode is the fingerprint of the authenticated user (step S 1500 : YES), it may be determined that the fingerprint recognition is successful (step S 1510 ), and appropriate subsequent operations may be performed.
- step S 1500 When the fingerprint recognized in the fingerprint recognition mode is not the fingerprint of the authenticated user (step S 1500 : NO), it may be determined that the fingerprint recognition has failed (step S 1520 ), and the operation may be terminated. In some example embodiments, the fingerprint recognition operation may be repeated until the number of times that the fingerprint recognition fails exceeds a threshold number.
- step S 1100 , S 1200 , S 1300 and S 1400 in FIG. 25 may be substantially the same as step S 1100 , S 1200 , S 1300 and S 1400 in FIG. 22 , respectively.
- An image may be displayed on at least a portion of remaining regions of the display panel other than the partial region in the fingerprint recognition mode (step S 1600 ).
- Step S 1600 may be performed simultaneously with at least one of steps S 1300 and S 1400 .
- an image may also be displayed on the partial region where the fingerprint recognition window is disposed.
- FIG. 26 is a diagram illustrating an example process for a method of performing optical fingerprint recognition according to example embodiments.
- FIG. 26 will be described with reference to the method of FIG. 24 as an example.
- the request may be generated or the request may occur to enter the fingerprint recognition mode (step S 1200 ).
- the request may occur in response to contact or proximity of the object 10 on or to any area on the display panel 110 .
- the request may occur while the display panel 110 is in a stand-by mode (e.g., while the display panel 110 displays the reduced or the minimal amount of information such as a current time).
- the user may not know a location at which the fingerprint recognition sensor 130 is disposed.
- the user may contact or approach a region other than the fingerprint recognition window FRW through the object 10 .
- the electronic device 100 may determine that a touched area does not coincide with the fingerprint recognition window FRW, and may display a reference image RI by partially driving the display panel 110 under control of the display driver 120 .
- the reference image RI may be displayed to inform the user of the location at which the fingerprint recognition sensor 130 is disposed.
- the reference image RI may be displayed on some or all portions of the fingerprint recognition window FRW.
- the electronic device 100 may determine that a touched area coincides with the fingerprint recognition window FRW, and may emit light by partially driving the display panel 110 under control of the display driver 120 (step S 1300 ).
- the electronic device 100 may generate an image signal associated with the object 10 which is on the fingerprint recognition window FRW, based on the emitted light.
- the electronic device 100 may recognize the fingerprint based on the reflected light of the fingerprint received through the fingerprint recognition window FRW and the calibration data (step S 1400 ). The electronic device 100 may determine whether the fingerprint recognized in the fingerprint recognition mode is the fingerprint of the authenticated user (step S 1500 ).
- the reference image RI may be provided in association with a function of fingerprint detection.
- the electronic device 100 may suitably drive the display panel 110 under control of the display driver 120 , such that an interface (e.g., the contact or proximity of the object 10 ) associated with the reference image RI is processed prior to an interface (e.g., a time setting) associated with the stand-by mode.
- an interface e.g., the contact or proximity of the object 10
- the electronic device 100 may display an error response to inform the user that a touched area does not coincide with the fingerprint recognition window FRW.
- FIG. 27 is a block diagram illustrating an electronic device according to example embodiments.
- an electronic device 1000 may include a processor 1010 , a memory device 1020 , a fingerprint recognition sensor 1030 , an input/output (I/O) device 1040 , a power supply 1050 and a display device 1060 .
- the electronic device 100 may further include a plurality of ports for communicating a video card, a sound card, a memory card, a universal serial bus (USB) device, other electronic devices, etc.
- USB universal serial bus
- the processor 1010 controls operations of the electronic device 1000 .
- the processor 1010 may execute an operating system and at least one application to provide an internet browser, games, videos, or the like.
- the memory device 1020 may store data for operations of the electronic device 1000 .
- the I/O device 1040 may include an input device such as a keyboard, a keypad, a mouse, a touchpad, a touch-screen, a remote controller, etc., and an output device such as a printer, a speaker, etc.
- the power supply 1050 may provide a power for operations of the electronic device 1000 .
- the display device 1060 includes a display panel.
- the display panel, the fingerprint recognition sensor 1030 , the processor 1010 and the memory device 1020 in FIG. 27 may correspond to the display panel 110 , the fingerprint recognition sensor 130 , the processor 140 and the memory 150 in FIG. 4 , respectively, and may perform the method of setting the light sources in the display panel according to example embodiments and the method of performing the optical fingerprint recognition according to example embodiments.
- the inventive concept may be applied to various electronic devices and systems that include the display panels and the fingerprint recognition sensors and perform the optical fingerprint recognition.
- the inventive concept may be applied to systems such as a mobile phone, a smart phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a portable game console, a music player, a camcorder, a video player, a navigation device, a wearable device, an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR) device, an augmented reality (AR) device, a robotic device, etc.
- PDA personal digital assistant
- PMP portable multimedia player
- digital camera a portable game console
- music player a camcorder
- video player a navigation device
- a wearable device an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR)
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Input (AREA)
- Collating Specific Patterns (AREA)
Abstract
A method of setting light sources in a display panel for optical fingerprint recognition is provided. The method includes driving a portion of light sources disposed to correspond to a fingerprint recognition window which is a partial region of a display panel based on initial light source setting values, obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources; and determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data.
Description
- This application claims priority under 35 USC § 119 to Korean Patent Application No. 10-2019-0066482, filed on Jun. 5, 2019 in the Korean Intellectual Property Office (KIPO), the contents of which are herein incorporated by reference in their entirety.
- Example embodiments relate generally to semiconductor integrated circuits, and more particularly to methods of setting light sources in display panels for optical fingerprint recognition, and methods of performing optical fingerprint recognition using the methods of setting the light sources.
- Biometric information is widely used in personal authentication because of its invariability and uniqueness. One type of biometric information is a fingerprint. Fingerprint recognition may be performed conveniently and serves as an excellent way of determining the identity of a person. Optical fingerprint recognition obtains a fingerprint image based on differences in light reflected by ridges and valleys of a finger. However, obtaining an accurate fingerprint image has proven to be difficult because the differences in reflected light tend to be very small.
- It is an aspect to provide a method of setting light sources in a display panel for optical fingerprint recognition capable of supporting efficient optical fingerprint recognition.
- It is another aspect to provide a method of performing optical fingerprint recognition using the method of setting the light sources.
- According to an aspect of one or more example embodiments, there is provided a method of setting light sources in a display panel for optical fingerprint recognition, the method comprising driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data.
- According to another aspect of one or more example embodiments, there is provided a method of setting light sources in a display panel for optical fingerprint recognition, the method comprising driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and determining final light source setting values such that powers for driving light sources at different positions among the portion of light sources are different, the final light source setting values being determined to optimize a magnitude of an interference signal included in the initial calibration data.
- According to another aspect of one or more example embodiments, there is provided a method of performing optical fingerprint recognition, the method comprising obtaining light source setting values and calibration data; displaying, in a fingerprint recognition mode, a fingerprint recognition window on a partial region of a display panel based on the light source setting values; and recognizing, in the fingerprint recognition mode, a fingerprint based on reflected light of the fingerprint received through the fingerprint recognition window and based on the calibration data, wherein obtaining the light source setting values and the calibration data includes driving a portion of light sources disposed to correspond to the fingerprint recognition window among a plurality of light sources included in the display panel based on initial light source setting values; obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different and/or powers for driving light sources at different positions among the some light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data and/or to optimize a magnitude of the interference signal included in the initial calibration data, respectively; driving the portion of the light sources based on the final light source setting values; obtaining final calibration data based on reflected light of the object received by the fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the final light source setting values; and storing the final light source setting values and the final calibration data as the light source setting values and the calibration data.
- Illustrative, non-limiting example embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
-
FIG. 1 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments; -
FIG. 2 is a plan view of an electronic device according to example embodiments; -
FIG. 3 is a cross-sectional view of an example of the electronic device taken along a line A-A′ ofFIG. 2 , according to example embodiments; -
FIG. 4 is a block diagram illustrating an example of the electronic device ofFIG. 2 , according to example embodiments; -
FIGS. 5A, 5B, 5C and 5D are diagrams for describing a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments; -
FIGS. 6, 7A, 7B and 7C are diagrams for describing an example of driving some light sources based on initial light source setting values and obtaining initial calibration data in the method ofFIG. 1 , according to example embodiments; -
FIG. 8 is a flowchart illustrating an example of determining final light source setting values in the method ofFIG. 1 , according to example embodiments; -
FIGS. 9, 10A, 10B and 10C are diagrams for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 1 , according to example embodiments; -
FIG. 11 is a flowchart illustrating another example of determining final light source setting values in the method ofFIG. 1 , according to example embodiments; -
FIG. 12 is a diagram for describing another example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 1 , according to example embodiments; -
FIG. 13 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments; -
FIG. 14 is a flowchart illustrating an example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments; -
FIG. 15 is a diagram for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 13 , according to example embodiments; -
FIG. 16 is a diagram for describing an example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments; -
FIG. 17 is a flowchart illustrating another example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments; -
FIGS. 18 and 19 are diagrams for describing another example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments; -
FIG. 20 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments; -
FIG. 21 is a diagram for describing an example of determining final light source setting values in the method ofFIG. 20 , according to example embodiments; -
FIG. 22 is a flowchart illustrating a method of performing optical fingerprint recognition according to example embodiments; -
FIG. 23 is a flowchart illustrating an example of recognizing a fingerprint in the method ofFIG. 22 , according to example embodiments; -
FIGS. 24 and 25 are flowcharts illustrating a method of performing optical fingerprint recognition according to example embodiments; -
FIG. 26 is a diagram illustrating an example process for a method of performing optical fingerprint recognition according to example embodiments; and -
FIG. 27 is a block diagram illustrating an electronic device according to example embodiments. - Various example embodiments will be described more fully with reference to the accompanying drawings, in which embodiments are shown. Embodiments consistent with the present disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Like reference numerals refer to like elements throughout this application.
- In a method of setting the light sources in the display panel for the optical fingerprint recognition and a method of performing optical fingerprint recognition according to example embodiments, light source setting values may be changed such that powers for driving light sources of different colors within a fingerprint recognition window are different and/or such that the powers for driving the light sources at different positions within the fingerprint recognition window are different, thereby minimizing the contrast of an interference signal included in initial calibration data and/or optimizing a magnitude of the interference signal included in the initial calibration data. Final calibration data may then be obtained based on the changed light source setting values. Accordingly, robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to external temperature of the display panel and pressure applied to the display panel and efficient optical fingerprint recognition may be supported.
-
FIG. 1 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. - Referring to
FIG. 1 , a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments is performed by an electronic device that includes the display panel and a fingerprint recognition sensor. The display panel includes a plurality of light sources, and the fingerprint recognition sensor performs the optical fingerprint recognition using light provided from the plurality of light sources. A detailed configuration of the electronic device will be described with reference toFIGS. 2 through 4 . - In the method of setting the light sources in the display panel for the optical fingerprint recognition according to example embodiments, some light sources among the plurality of light sources included in the display panel are driven based on initial light source setting values (step S100). The light sources that are driven are light sources disposed to correspond to a fingerprint recognition window which is a partial region of the display panel. For example, the initial light source setting values may be the same value for all of the driven light sources. In other words, all of the driven light sources may emit light to have the same grayscale value based on the initial light source setting values. In some example embodiments, the initial light source setting values may be the same for a portion of the driven light sources.
- While driving the light sources based on the initial light source setting values, initial calibration data is obtained based on reflected light received through the fingerprint recognition window (step S200). The initial calibration data may be obtained from a fingerprint recognition sensor. For example, all of the driven light sources may be substantially simultaneously or concurrently turned on based on the initial light source setting values. Light generated from the driven light sources may be reflected by an object on the fingerprint recognition window, the reflected light of the object may be provided to the fingerprint recognition sensor, and thus the fingerprint recognition sensor may obtain the initial calibration data based on the reflected light of the object. In some example embodiments, the object may be a specific object used for obtaining calibration data, rather than a finger of a user. In some example embodiments, a portion of the driven light sources may be substantially simultaneously or concurrently turned on based on the initial light source setting values.
- To minimize contrast of an interference signal included in the initial calibration data, final light source setting values are determined such that powers for driving light sources of different colors among the driven light sources are different (step S300). The contrast of the interference signal may be minimized by the final light source setting values. For example, the final light source setting values may be different values for the light sources of different colors among the driven light sources. In other words, the light sources of different colors may emit light to have different grayscale values based on the final light source setting values.
- In some example embodiments, as will be described with reference to
FIGS. 7A and 7B , the initial calibration data may include a low frequency component and a high frequency component, and the interference signal may indicate the high frequency component included in the initial calibration data. - Returning to
FIG. 1 , some light sources may be driven based on the final light source setting values (step S400), and while driving the light sources based on the final light source setting values, final calibration data may be obtained based on reflected light received through the fingerprint recognition window (step S500). For example, all of the driven light sources may be substantially simultaneously or concurrently turned on based on the final light source setting values. As with step S200, the reflected light of the object, which is a specific object used for obtaining calibration data, may be provided to the fingerprint recognition sensor. In some example embodiments, a portion of the driven light sources may be substantially simultaneously or concurrently turned on based on the final light source setting values. - Typically, a display panel has a complicated internal structure including patterns of multi-layered wirings, electrodes, and the like. When a user's finger is placed on a fingerprint recognition window as an object while light sources in the fingerprint recognition window are illuminated, reflected light received through the fingerprint recognition window includes information of a fingerprint that exists on the user's finger and information of the internal structure of the display panel. For example, the internal structure may be a bottom view of the display panel. Thus, in order to obtain only the information of the fingerprint, only information on the internal structure of the display panel, which is an interference component, should be obtained first, and compensation for removing the interference component from an image signal that includes both the interference component and the information of the fingerprint obtained by the user's finger should be performed. Such interference component may be referred to as calibration data. For example, the compensation may be performed by removing the calibration data from the read data from the user's finger, or may be performed by determining compensation data that when added to the read data from the user's finger negates the calibration data.
- In the method of setting the light sources in the display panel according to example embodiments, the initial calibration data may be obtained while the light sources within the fingerprint recognition window emit light at the same grayscale, and the light source setting values may be changed such that the powers for driving the light sources of different colors within the fingerprint recognition window are different, thereby minimizing the contrast of the interference signal included in the initial calibration data. The final calibration data may then be obtained while the light sources of different colors within the fingerprint recognition window emit light with different grayscales. Accordingly, a robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to external temperature of the display device, pressure applied to the display panel, etc., and the efficient optical fingerprint recognition may be supported.
- Although it is illustrated in step S300 of
FIG. 1 that the final light source setting values are determined at once or at one time, example embodiments are not limited thereto, and an operation of determining the final light source setting values may be repeatedly performed until the contrast of the interference signal included in the initial calibration data is minimized, e.g., until the final calibration data has an optimal value for fingerprint image restoration. -
FIG. 2 is a plan view of an electronic device according to example embodiments. - Referring to
FIG. 2 , anelectronic device 100 includes adisplay panel 110 to interface with a user. Thedisplay panel 110 outputs visual information to the user. Although not illustrated inFIG. 2 , theelectronic device 100 may further include a touch sensor panel for sensing a touch input of the user, and the user may input a signal to theelectronic device 100 through the touch sensor panel. - A fingerprint recognition window FRW may be provided on the
display panel 110. As will be described with reference toFIG. 3 , a fingerprint recognition sensor for fingerprint detection may be disposed to spatially correspond to a location of the fingerprint recognition window FRW. Although the fingerprint recognition window FRW is illustrated as a rectangle inFIG. 2 and subsequent figures, a shape and/or a size of the fingerprint recognition window FRW may be changed according to example embodiments. - In some example embodiments, the
electronic device 100 may be or may include any mobile system, such as a mobile phone, a smart phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a portable game console, a music player, a camcorder, a video player, a navigation device, a wearable device, an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR) device, an augmented reality (AR) device, a robotic device, etc. - The present disclosure may provide an interface used to detect a fingerprint. For example, a function of fingerprint detection may be performed when the user contacts or approaches the touch sensor panel or the
display panel 110. According to the present disclosure, the interface and the fingerprint recognition sensor used for fingerprint detection may share an area on theelectronic device 100 with the touch sensor panel and thedisplay panel 110, and thus the interface and the fingerprint recognition sensor may not require an additional area on theelectronic device 100. Accordingly, it may be possible to reduce the size of theelectronic device 100, or a spare area outside of thedisplay panel 110 that would otherwise be used for the interface may be used for other purpose(s). -
FIG. 3 is a cross-sectional view of an example of the electronic device taken along a line A-A′ ofFIG. 2 , according to example embodiments. - Referring to
FIG. 3 , the fingerprint recognition window FRW may be provided on a partial region (or portion) of thedisplay panel 110. In some example embodiments the FRW may be displayed on the partial region (or portion) of thedisplay panel 110 in a fingerprint recognition mode. Thedisplay panel 110 may include a plurality oflight sources 112. For example, as will be described with reference toFIG. 4 , the plurality oflight sources 112 may be included in a plurality of pixels included in thedisplay panel 110. - Among the plurality of
light sources 112, only somelight sources FIG. 3 , thelight sources FIG. 1 , the powers (or power sources) for driving the light sources of different colors may be differently set. InFIG. 3 , thelight sources FIG. 3 showslight sources - A
fingerprint recognition sensor 130 may be disposed under thedisplay panel 110 such that thefingerprint recognition sensor 130 may at least partially overlap the fingerprint recognition window FRW in a vertical direction. In other words, thedisplay panel 110 may include afirst surface 110 a on which an image is displayed and asecond surface 110 b opposite to the first surface, and thefingerprint recognition sensor 130 may be disposed under thesecond surface 110 b of thedisplay panel 110. - The
fingerprint recognition sensor 130 may include alens 132 and animage sensor 134. Thelens 132 may be disposed under the display panel 110 (e.g., disposed between thedisplay panel 110 and the image sensor 134), and may concentrate reflected light received through the fingerprint recognition window FRW on theimage sensor 134. Theimage sensor 134 may be disposed under thelens 132, and may generate an image signal for an object on a partial region of the FRW based on the reflected light concentrated by thelens 132. Thefingerprint recognition sensor 130 may be implemented in the form of a compact camera module (CCM) including thelens 132 and theimage sensor 134. - For example, as illustrated in
FIG. 3 , when a user puts afinger 10 on the fingerprint recognition window FRW, the light generated from thelight sources finger 10, and the reflected light of the fingerprint may be provided to thefingerprint recognition sensor 130. Thefingerprint recognition sensor 130 may capture an image signal for the fingerprint or information associated with a shape of the fingerprint (e.g., a fingerprint image) based on the reflected light of the fingerprint received through the fingerprint recognition window FRW. - For another example, as will be described with reference to
FIG. 6 , when anobject 20 used for obtaining calibration data is placed on the fingerprint recognition window FRW, the light generated from thelight sources object 20, and the reflected light of theobject 20 may be provided to thefingerprint recognition sensor 130. Thefingerprint recognition sensor 130 may capture an image signal for the calibration data (e.g., an image of the internal structure of the display panel 110) based on the reflected light of theobject 20 received through the fingerprint recognition window FRW. - Although not illustrated in
FIG. 3 , thefingerprint recognition sensor 130 may further include a filter for adjusting a frequency characteristic and/or a polarization characteristic of the reflected light which is to be provided to theimage sensor 134. -
FIG. 4 is a block diagram illustrating an example of the electronic device ofFIG. 2 , according to example embodiments. - Referring to
FIG. 4 , theelectronic device 100 may include thedisplay panel 110 and thefingerprint recognition sensor 130. Theelectronic device 100 may further include adisplay driver 120, aprocessor 140 and amemory 150. - The
display panel 110 outputs visual information to the user. Thedisplay panel 110 may include a plurality of pixels which are arranged along rows and columns to display an image.FIG. 4 illustrates one pixel PX as an example. Each pixel may be configured to emit light of a specific color which forms the image. As the plurality of pixels emit light together, thedisplay panel 110 may display a desired or intended image. - In some example embodiments, the
display panel 110 may be an electroluminescent display panel. The electroluminescent display panel may be driven with rapid response speed and low power consumption using a light emitting diode (LED) or an organic light emitting diode (OLED) that generates light by recombination of electrons and holes. In comparison with a liquid crystal display panel using a backlight unit, the pixels PX of the electroluminescent display panel may emit light by themselves, and the reflected light received through the fingerprint recognition window FRW may be provided to thefingerprint recognition sensor 130 under thedisplay panel 110 through a space (gap) between the pixels PX. Thus, light emitting diodes or organic light emitting diodes included in the pixels PX may correspond to the light sources included in the display panel according to example embodiments. However, example embodiments are not limited thereto, and thedisplay panel 110 may be any display panel having a structure in which the reflected light received through the fingerprint recognition window FRW may be provided to thefingerprint recognition sensor 130. - The
display driver 120 may control operations of thedisplay panel 110 and may drive thedisplay panel 110. For example, thedisplay driver 120 may suitably drive each pixel of thedisplay panel 110 in response to a command of theprocessor 140 such that the desired or intended image is displayed on thedisplay panel 110. For example, thedisplay driver 120 may partially drive thedisplay panel 110 such that pixels corresponding to the fingerprint recognition window FRW emit light. Although not illustrated inFIG. 4 , thedisplay driver 120 may include a data driver, a scan driver, a timing controller, a gamma circuit, etc. - The
fingerprint recognition sensor 130 may be used to detect a fingerprint. Thefingerprint recognition sensor 130 may generate/output an image signal associated with an object which is on the fingerprint recognition window FRW. For example, thefingerprint recognition sensor 130 may operate to obtain an image signal associated with a fingerprint of a finger which contacts or approaches the fingerprint recognition window FRW. As described with reference toFIG. 3 , thefingerprint recognition sensor 130 may include thelens 132 and theimage sensor 134. - The
fingerprint recognition sensor 130 may provide a function of optical fingerprint recognition or optics-based fingerprint detection. For example, theimage sensor 134 included in thefingerprint recognition sensor 130 may include photo-diode(s) which is capable of generating current in response to light. - The
processor 140 may control overall operations of theelectronic device 100. Theprocessor 140 may process/perform various arithmetic/logical operations based on program code to provide functions of theelectronic device 100. - The
processor 140 may communicate with thedisplay driver 120, thefingerprint recognition sensor 130 and thememory 150. Theprocessor 140 may control operations of thedisplay driver 120, thefingerprint recognition sensor 130 and thememory 150. Theprocessor 140 may process program code including commands, requests, responses, and/or the like, which are associated with operations of thedisplay driver 120, thefingerprint recognition sensor 130 and thememory 150. For example, theprocessor 140 may provide a variety of information to thedisplay driver 120, to display the desired or intended image on thedisplay panel 110. For example, theprocessor 140 may control an operation timing/sequence of thedisplay panel 110 and thefingerprint recognition sensor 130 such that thefingerprint recognition sensor 130 generates the image signals associated with the fingerprint and/or the calibration data. For example, theprocessor 140 may generate and analyze image information associated with the fingerprint and/or the calibration data based on the image signals output from thefingerprint recognition sensor 130. For example, theprocessor 140 may store associated data in thememory 150 or may load the associated data from thememory 150. - In some example embodiments, the
processor 140 may include one or more special-purpose circuits (e.g., a field programmable gate array (FPGA), application specific integrated chips (ASICs), and/or the like) to perform various operations. For example, theprocessor 140 may include one or more processor cores which are capable of performing various operations. For example, theprocessor 140 may be implemented with a general-purpose processor, a special-purpose processor, or an application processor. - The
memory 150 may store data related to the operation of theelectronic device 100. For example, thememory 150 may store the program code mentioned above, and may store the initial light source setting values, the initial calibration data, the final light source setting values, the final calibration data, etc. for performing the method of setting the light sources in the display panel according to example embodiments. - In some example embodiments, the
memory 150 may include at least one of various volatile memories such as a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like, and/or at least one of various nonvolatile memories such as a flash memory, a phase change random access memory (PRAM), a resistance random access memory (RRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a nano floating gate memory (NFGM), a polymer random access memory (PoRAM), or the like. - In some example embodiments, the
display driver 120, thefingerprint recognition sensor 130, theprocessor 140 and thememory 150 may be respectively implemented with separate circuits/modules/chips. In other example embodiments, on the basis of a function, some of thedisplay driver 120, thefingerprint recognition sensor 130, theprocessor 140 and thememory 150 may be combined into one circuit/module/chip, or may be further separated into a plurality of circuits/modules/chips. - The
electronic device 100 may perform the method of setting the light sources in the display panel described with reference toFIG. 1 . For example, thedisplay panel 110 and thedisplay driver 120 under control of theprocessor 140 may perform steps S100 and S400 inFIG. 1 , thefingerprint recognition sensor 130 under control of theprocessor 140 may perform steps S200 and S500 inFIG. 1 , and theprocessor 140 may perform step S300 inFIG. 1 . In addition, theelectronic device 100 may perform a method of setting light sources which will be described with reference toFIGS. 13 and 20 , and/or a method of performing optical fingerprint recognition which will be described with reference toFIG. 22 . -
FIGS. 5A, 5B, 5C and 5D are diagrams for describing a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. -
FIG. 5A illustrates an image obtained based on reflected light by an object used for obtaining calibration data, without a fingerprint (e.g., without a finger of a user). The image ofFIG. 5A may correspond to calibration data (e.g., the initial calibration data) obtained by a fingerprint recognition sensor, and may include only information of an internal structure of a display panel. -
FIG. 5B illustrates an image obtained based on reflected light by the fingerprint. The image ofFIG. 5B may also be obtained by the fingerprint recognition sensor, and may include both information of the fingerprint and the information of the internal structure of the display panel. -
FIGS. 5C and 5D illustrate examples of a pure fingerprint image obtained based on the images ofFIGS. 5A and 5B . As described with reference toFIG. 1 , a first image (e.g., the image ofFIG. 5A ) including only a pure interference component without a fingerprint may be obtained first, a second image (e.g., the image ofFIG. 5B ) including both the fingerprint and the interference component may be obtained next, and then a pure fingerprint image may be obtained by performing compensation to remove the interference component. For example, the pure fingerprint image may be obtained by performing a simple face subtraction to obtain a difference between the second image and the first image. - As illustrated in
FIG. 5C , a pure fingerprint image in which the interference component is completely removed may be obtained. However, as illustrated inFIG. 5D , a residual component RES may remain or may be left without the interference component being completely removed. For example, when the finger of the user is placed on the fingerprint recognition window, the interference component may vary depending on temperature of the display panel and/or pressure applied to the display panel. Particularly, deformation (e.g., rotation, scaling, translation, etc.) may occur on the interference component in the second image (i.e.,FIG. 5B ), and thus the residual component RES may occur because there is a spatial mismatch between the interference component in the first image (FIG. 5A ) and the interference component in the second image (FIG. 5B ). - To minimize such residual component RES, the powers of the light sources within the fingerprint recognition window may be adjusted to minimize the contrast of the interference signal included in the initial calibration data as described above with reference to
FIG. 1 , or the powers of the light sources within the fingerprint recognition window may be adjusted to optimize a magnitude of the interference signal included in the initial calibration data as will be described above with reference toFIG. 13 . -
FIGS. 6, 7A, 7B and 7C are diagrams for describing an example of driving some light sources based on initial light source setting values and an example of obtaining initial calibration data in the method ofFIG. 1 , according to example embodiments. - Referring to
FIGS. 1 and 6 , the somelight sources light sources object 20. - As described with reference to
FIG. 1 , the initial light source setting values ILSET may be the same value for all of the turned onlight sources light sources light sources FIG. 6 . According to example embodiments, the initial light source setting values ILSET may be stored in thememory 150 inFIG. 4 and provided from thememory 150, or may be provided from theprocessor 140 inFIG. 4 . - The
object 20 may be a specific object used for obtaining calibration data, rather than a finger of the user. For example, to facilitate acquisition of the initial calibration data ICDAT associated with the internal structure of the display panel, theobject 20 may be any object of white flatness and high reflectivity without bending. According to example embodiments, the initial calibration data ICDAT may be provided to and stored in thememory 150, or may be provided to theprocessor 140 to be used to determine the final light source setting values. -
FIG. 7A illustrates an image corresponding to the initial calibration data ICDAT ofFIG. 6 obtained by thefingerprint recognition sensor 130.FIG. 7B illustrates a luminance change occurring along a line B-B′ in the image ofFIG. 7A . InFIG. 7B , an X-axis represents a position along the line B-B′ inFIG. 7A , and a Y-axis represents luminance at a corresponding position. - As illustrated in
FIGS. 7A and 7B , the initial calibration data ICDAT may include a low frequency component and a high frequency component. The low frequency component may represent an overall change in luminance by a relatively bright center and a relatively dark edge in the image ofFIG. 7A . The high frequency component may represent a local luminance change (e.g., fluctuation) where light and dark portions are alternately repeated at a center of the image ofFIG. 7A . The high frequency component may be generated by the internal structure of the display panel 110 (e.g., reflected light may or may not pass through the patterns). Since the high frequency component has a frequency similar to that of the fingerprint signal by ridges and valleys of the fingerprint, it may be difficult to remove only the high frequency component. -
FIG. 7C illustrates that the residual component RES inFIG. 5D may occur when the initial calibration data ICDAT is used as it is. InFIG. 7C , a solid line represents the interference signal included in the initial calibration data ICDAT illustrated inFIG. 7B , and a dotted line represents the interference signal included in the image including both the fingerprint and the interference component. When two interference signals are spatially mismatched or inconsistent, differences between the two interference signals may be relatively large as illustrated by double-headed arrows inFIG. 7C , and the residual component RES may occur as illustrated inFIG. 5D by the differences inFIG. 7C . -
FIG. 8 is a flowchart illustrating an example of determining final light source setting values in the method ofFIG. 1 , according to example embodiments. - Referring to
FIGS. 1 and 8 , when determining the final light source setting values such that the powers for driving the light sources of different colors are different (step S300), luminance of first light sources among the some light sources may be determined as a first value (step S310), and luminance of second light sources among the some light sources may be determined as a second value different from the first value (step S320). The first light sources may have a first color, and the second light sources may have a second color different from the first color. - For example, when the display panel includes red, green and blue (RGB) light sources, the first color of the first light sources may be blue, and the second color of the second light sources may be green. However, example embodiments are not limited thereto, and any two of different colors of the plurality of light sources included in the display panel may be selected as the first and second colors, respectively.
-
FIGS. 9, 10A, 10B and 10C are diagrams for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 1 , according to example embodiments. The descriptions repeated withFIG. 6 will be omitted. - Referring to
FIGS. 1 and 9 , the somelight sources light sources object 20. - As described with reference to
FIG. 1 , the final light source setting values FLSET may be different values for light sources of different colors among the somelight sources light sources light sources FIG. 9 . According to example embodiments, the final light source setting values FLSET may be stored in thememory 150 inFIG. 4 and provided from thememory 150, or may be provided from theprocessor 140 inFIG. 4 . -
FIG. 10A illustrates an image corresponding to the final calibration data FCDAT ofFIG. 9 obtained by thefingerprint recognition sensor 130.FIG. 10B illustrates luminance change occurring along a line C-C′ in the image ofFIG. 10A . InFIG. 10B , an X-axis represents a position along the line C-C′ inFIG. 10A and a Y-axis represents luminance at a corresponding position. - As illustrated in
FIGS. 10A and 10B , the final calibration data FCDAT may also include a low frequency component and a high frequency component. In comparison withFIGS. 7A and 7B (e.g., in comparison with the initial calibration data ICDAT), respectively, the low frequency component included in the final calibration data FCDAT is similar to the low frequency component included in the initial calibration data ICDAT, and the high frequency component included in the final calibration data FCDAT is reduced compared with the high frequency component included in the initial calibration data ICDAT. -
FIG. 10C illustrates that the residual component RES inFIG. 5D may be reduced or eliminated when the final calibration data FCDAT is used. InFIG. 10C , a solid line represents the interference signal included in the final calibration data FCDAT illustrated inFIG. 10B , and a dotted line represents the interference signal included in the image including both the fingerprint and the interference component. In comparison withFIG. 7C , differences between the two interference signals may be relatively small and overall more uniform as illustrated by double-headed arrows inFIG. 10C , even if two interference signals are spatially mismatched or inconsistent. Thus, the residual component RES may be efficiently removed from the fingerprint image when the final calibration data FCDAT is used. -
FIG. 11 is a flowchart illustrating another example of determining final light source setting values inFIG. 1 , according to example embodiments. The descriptions repeated withFIG. 8 will be omitted. - Referring to
FIGS. 1 and 11 , when determining the final light source setting values such that the powers for driving the light sources of different colors are different (step S300), steps S310 and S320 inFIG. 11 may be substantially the same as steps S310 and S320 inFIG. 8 , respectively. Luminance of third light sources among the turned on light sources may be determined as a third value (step S330). The third value may be different from the first value and the second value. For example, the third light sources may have third color different from the first color and the second color. - For example, when the display panel includes red, green and blue light sources, the first color of the first light sources may be blue, the second color of the second light sources may be green, and the third color of the third light sources may be red. However, example embodiments are not limited thereto, and any three of different colors of the plurality of light sources included in the display panel may be selected as the first, second and third colors, respectively.
-
FIG. 12 is a diagram for describing another example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 1 . The descriptions repeated withFIGS. 6 and 9 will be omitted. - Referring to
FIGS. 1 and 12 ,light sources light sources object 20. Thelight sources FIG. 12 . - In some example embodiments, when the display panel includes red, green and blue light sources, the first, second and third values that represent the luminance of the first, second and third light sources, respectively, may be obtained by Equation 1.
-
(α,β,γ)=argmin(α,β,γ)(costfn(α*Blue+β*Green+γ*Red)) [Equation 1] - In Equation 1, α, β and γ denote the first, second and third values, respectively, costfn denotes a cost function, and argmin denotes a function of obtaining indices α, β and γ for the cost function to have a minimum value. For example, each of α, β and γ may be a grayscale value of one of grayscale ranges (e.g., 0 to 255 grayscales) of the display panel. For example, the cost function may be a function representing the contrast of the interference signal included in the initial calibration data, and may be a standard deviation or a difference from a low pass filtered signal (e.g., a signal from which the interference signal is removed).
- The power of each of the R, G and B channels may be set based on the α, β and γ ratio calculated by Equation 1, and then the set power may be used to drive the light sources in the fingerprint recognition window.
- Although example embodiments have been described that the light sources of two or three different colors are set to have different grayscale values for each color, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which light sources of at least two different colors of any of a plurality of different colors included in the display panel are set to have different grayscale values for each color. In addition, although example embodiments have been described that all the light sources in the fingerprint recognition window are set to have different grayscale values for each color, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which only a portion of the light sources in the fingerprint recognition window are set to have different grayscale values for each color.
-
FIG. 13 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. The descriptions repeated withFIG. 1 will be omitted. - Referring to
FIG. 13 , in a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments, steps S100, S200, S400 and S500 inFIG. 13 may be substantially the same as steps S100, S200, S400 and S500 inFIG. 1 , respectively. - To optimize a magnitude of the interference signal included in the initial calibration data, the final light source setting values are determined such that powers for driving light sources at different positions among the driven light sources are different (step S600). The magnitude of the interference signal included in the initial calibration data is optimized by the final light source setting values. For example, the final light source setting values may be different values for the light sources at different positions among the driven light sources. In other words, the light sources at different positions may emit light to have different grayscale values based on the final light source setting values.
- In the method of setting the light sources in the display panel according to example embodiments, the initial calibration data may be obtained while emitting the light sources within the fingerprint recognition window at the same grayscale, and the light source setting values may be changed such that the powers for driving the light sources at different positions within the fingerprint recognition window are different, thereby optimizing the magnitude of the interference signal included in the initial calibration data. After that, the final calibration data may be obtained while emitting the light sources at different positions within the fingerprint recognition window with different grayscales. Accordingly, the robust fingerprint image restoration may be performed using the final calibration data even if changes occur due to temperature of the display panel, pressure applied to the display panel, etc., and efficient optical fingerprint recognition may be supported.
-
FIG. 14 is a flowchart illustrating an example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments. - Referring to
FIGS. 13 and 14 , when determining the final light source setting values such that the powers for driving the light sources at different positions are different (step S600), luminance of first light sources among the some light sources may be determined as a first value (step S610), and luminance of second light sources among the some light sources may be determined as a second value (step S620). The second value may be different from the first value. The first light sources may be disposed in a first region of the partial region, and the second light sources may be disposed in a second region of the partial region different from the first region. -
FIG. 15 is a diagram for describing an example of driving some light sources based on final light source setting values and obtaining final calibration data in the method ofFIG. 13 , according to example embodiments. The descriptions repeated withFIGS. 6, 9 and 12 will be omitted. - Referring to
FIGS. 13 and 15 , the somelight sources light sources object 20. - As described with reference to
FIG. 13 , the final light source setting values FLSET″ may be different values for light sources at different positions among the somelight sources light sources light sources FIG. 15 . -
FIG. 16 is a diagram for describing an example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments. - Referring to
FIGS. 14 and 16 , a first region R1 in which the first light sources are disposed may be a central portion of the fingerprint recognition window FRW, and a second region R2 in which the second light sources are disposed may be an edge portion (or peripheral portion) of the fingerprint recognition window FRW that surrounds the central portion. For example, the first light sources may be set to have a relatively low grayscale value or luminance (e.g., set to be relatively dark). -
FIG. 17 is a flowchart illustrating another example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments. The descriptions repeated withFIG. 14 will be omitted. - Referring to
FIGS. 13 and 17 , when determining the final light source setting values such that the powers for driving the light sources at different positions are different (step S600), steps S610 and S620 inFIG. 17 may be substantially the same as steps S610 and S620 inFIG. 14 , respectively. Luminance of third light sources among the turned on light sources may be determined as a third value (step S630). The third value may be different from the first value and the second value. For example, the third light sources may be disposed in a third region of the partial region different from the first region and the second region. An operation of obtaining the final calibration data by driving the first, second and third light sources may be similar to that described with reference toFIG. 12 . -
FIGS. 18 and 19 are diagrams for describing another example of determining final light source setting values in the method ofFIG. 13 , according to example embodiments. - Referring to
FIGS. 17 and 18 , a first region R1′ in which the first light sources are disposed may be a central portion of the fingerprint recognition window FRW, a second region R2′ in which the second light sources are disposed may be a middle portion of the fingerprint recognition window FRW that surrounds the central portion, and a third region R3′ in which the third light sources are disposed may be an edge portion of the fingerprint recognition window FRW that surrounds the middle portion. - Although example embodiments have been described that the fingerprint recognition window is divided into two or three regions and the light sources in the fingerprint recognition window are set to have different grayscale values for each region, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which the fingerprint recognition window is divided into any number of different regions and the light sources in the fingerprint recognition window are set to have different grayscale values for each region. In addition, example embodiments may be employed or extended to examples in which all the light sources in the fingerprint recognition window are set to have different grayscale values for each position. In some example embodiments, all the light sources in the fingerprint recognition window are set to have different grayscale values based on position. In some other example embodiments, each individual turned on light source may have a different grayscale value.
-
FIG. 19 illustrates a difference in luminance according to positions of light sources in the fingerprint recognition window FRW when all the light sources in the fingerprint recognition window are set to have different grayscale values for each position. InFIG. 19 , an X′-axis represents a position of the light sources in the fingerprint recognition window FRW along the line A-A′ inFIG. 2 , and a Y′-axis represents luminance or grayscale generated by the light source disposed at corresponding position. - As illustrated in
FIG. 19 , luminance of the light sources in the fingerprint recognition window may be determined to be lower as a light source becomes closer to a center (or central portion). As illustrated inFIG. 7A and other figures, the interference signal may appear relatively large in the center portion, the interference signal may appear relatively small in the edge portion, and the fingerprint signal may also appear to be weak due to vignetting of the lens. To improve this, the powers for driving the light sources may be adjusted according to the position of each of the light sources so that the center portion is relatively dark and the edge portion is relatively bright. For example, in some example embodiments, light sources at respective edge positions may be the same grayscale value, next light sources adjacent to the edge positions may have another grayscale value, etc. In other example embodiments, each individual light source may have a different grayscale value for each position. In still other example embodiments, each individual light source may have a different grayscale value based on position. In still other example embodiments, each individual turned on light source may have a different grayscale value. Thus, it is possible to enhance the fingerprint signal in the edge portion while reducing the magnitude of the interference signal in the center portion. - Although example embodiments have been described that all the light sources in the fingerprint recognition window are set to have different grayscale values for each position, example embodiments are not limited thereto, and example embodiments may be employed or applied to examples in which only a portion of the light sources in the fingerprint recognition window are set to have different grayscale values for each position.
-
FIG. 20 is a flowchart illustrating a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments. The descriptions repeated withFIGS. 1 and 13 will be omitted. - Referring to
FIG. 20 , in a method of setting light sources in a display panel for optical fingerprint recognition according to example embodiments, steps S100, S200, S300, S400 and S500 inFIG. 20 may be substantially the same as steps S100, S200, S300, S400 and S500 inFIG. 1 , respectively, and step S600 inFIG. 20 may be substantially the same as step S600 inFIG. 13 . According to example embodiments, an order of performing steps S300 and S600 may be changed, and steps S300 and S600 may be substantially simultaneously performed. -
FIG. 21 is a diagram for describing an example of determining final light source setting values in the method ofFIG. 20 , according to example embodiments. -
FIG. 21 illustrates a difference in luminance according to positions of light sources in the fingerprint recognition window FRW when all the light sources in the fingerprint recognition window are set to have different grayscale values for each color and each position. Luminance L1 of the first light sources having the first color and luminance L2 of the second light sources having the second color may be set differently as described with reference toFIGS. 1 through 12 , and luminance of the light sources may be set differently for each position of the light sources as described with reference toFIGS. 13 through 19 . For example, green and blue light sources disposed in the center portion may be set to have 230 and 216 grayscales, respectively, and green and blue light sources disposed in the edge portion may be set to have 254 and 255 grayscales, respectively. The brightness and hue of the light sources for each location may be changed together. -
FIG. 22 is a flowchart illustrating a method of performing optical fingerprint recognition according to example embodiments. - Referring to
FIG. 22 , in a method of performing optical fingerprint recognition according to example embodiments, light source setting values and calibration data are obtained (step S1100). Step S1100 may be performed based on the method of setting the light sources in the display panel according to example embodiments described with reference toFIGS. 1 through 21 . For example, the final light source setting values may be determined such that the powers for driving the light sources of different colors among the driven light sources within the fingerprint recognition window are different, such that the powers for driving the light sources at different positions among the driven light sources are different, or such that both the powers for driving the light sources of different colors and the powers for driving the light sources at different positions are different. For example, the final calibration data may be obtained based on the final light source setting values, and the final light source setting values and the final calibration data may be stored as the light source setting values and the calibration data, respectively. For example, the light source setting values and the calibration data may be stored in thememory 150 inFIG. 4 . - A fingerprint recognition mode is entered (step S1200). The fingerprint recognition mode may be entered based on a request. For example, the request may be associated with a signal or an action which directs to detect a fingerprint.
- For example, the request may occur when any input is received from a user by the
electronic device 100, when a signal/command is generated in theelectronic device 100 based on the received input, and/or the like. For example, the request may occur when the user contacts or approaches any area on the touch sensor panel or thedisplay panel 110 through the object 10 (e.g., the request may occur in response to a touch of the object 10). For example, the request may occur when theobject 10 acts specific motion or gesture in the vicinity of theelectronic device 100. For example, the request may occur when theelectronic device 100 moves in a specific manner. As another example, an application running on theelectronic device 100 may initiate the request. However, example embodiments are not limited thereto, and the request may be changed or modified to recognize directing of fingerprint detection. - For example, the request may occur while the
electronic device 100 is in an idle state or thedisplay panel 110 is not driven. For example, the request may occur while thedisplay panel 110 is in a stand-by mode. Herein, the stand-by mode may denote an operation mode where thedisplay panel 110 displays a reduced amount of information or a minimal amount of information (e.g., a current time, a date, and/or the like), and may be also called as an “Always ON display (AOD)” mode, an “active display mode”, and/or the like. For example, the request may occur while thedisplay panel 110 is in a normal mode. Herein, the normal mode may denote an operation mode where thedisplay panel 110 displays a variety of information according to intention of the user. - The fingerprint recognition window is displayed on a partial region of the display panel based on the light source setting values (step S1300), and the fingerprint is recognized based on reflected light of the fingerprint received through the fingerprint recognition window, and the calibration data (step S1400). The fingerprint recognition mode may then be existed once the fingerprint is recognized. As described above, the fingerprint may be recognized based on the reflected light and the calibration data when the powers for driving the light sources are set to be different from each other by color and/or position. Accordingly, the robust fingerprint image restoration may be performed even if changes occur due to temperature of the display panel, pressure applied to the display panel, etc., and efficient optical fingerprint recognition may be supported.
- In some example embodiments, an operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference to
FIGS. 1 through 21 may be performed when an electronic device including the display panel and the fingerprint recognition sensor is manufactured. For example, the method of setting the light sources in the display panel according to example embodiments may be performed once at the time of manufacturing the electronic device, and the final light source setting values and the final calibration data may be obtained and stored in the electronic device. After that, an operation of loading the final light source setting values and the final calibration data already stored in thememory 150 may be performed in step S1100. A fingerprint recognition operation of steps S1200, S1300 and S1400 may be performed by loading the stored final light source setting values and the stored final calibration data. - In other example embodiments, since characteristics of the light sources are changed, degraded or deteriorated by usage of the display panel, the operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference to
FIGS. 1 through 21 may be periodically performed. For example, the operation of obtaining the final light source setting values and the final calibration data based on the method of setting the light sources in the display panel according to example embodiments described with reference toFIGS. 1 through 21 may be repeatedly performed whenever use time of the display panel exceeds a threshold reference time. -
FIG. 23 is a flowchart illustrating an example of recognizing a fingerprint in the method ofFIG. 22 , according to example embodiments. - Referring to
FIGS. 22 and 23 , when recognizing the fingerprint based on the reflected light of the fingerprint received through the fingerprint recognition window and the calibration data (step S1400), an image signal for the fingerprint may be obtained based on the reflected light (step S1410). For example, the image signal for the fingerprint may be obtained based on the reflected light reflected by the fingerprint and received through the fingerprint recognition window. Final information for the fingerprint may be obtained by subtracting the calibration data from the image signal (step S1420). As described with reference toFIGS. 5C and 5D , the first image (e.g., the image ofFIG. 5A ) including only the pure interference component without the fingerprint may be obtained first, the second image (e.g., the image ofFIG. 5B ) including both the fingerprint and the interference component may be obtained next, and then the pure fingerprint image may be obtained by performing the compensation to remove the interference component. -
FIGS. 24 and 25 are flowcharts illustrating a method of performing optical fingerprint recognition according to example embodiments. The descriptions repeated withFIG. 22 will be omitted. - Referring to
FIG. 24 , in a method of performing optical fingerprint recognition according to example embodiments, step S1100, S1200, S1300 and S1400 inFIG. 24 may be substantially the same as step S1100, S1200, S1300 and S1400 inFIG. 22 , respectively. - It may be determined whether the fingerprint recognized in the fingerprint recognition mode (i.e., in step 1400) is a fingerprint of an authenticated user (step S1500). For example, the fingerprint of the authenticated user stored in advance and the fingerprint recognized in the fingerprint recognition mode may be compared to determine whether two fingerprints are substantially identical or the same.
- When the fingerprint recognized in the fingerprint recognition mode is the fingerprint of the authenticated user (step S1500: YES), it may be determined that the fingerprint recognition is successful (step S1510), and appropriate subsequent operations may be performed.
- When the fingerprint recognized in the fingerprint recognition mode is not the fingerprint of the authenticated user (step S1500: NO), it may be determined that the fingerprint recognition has failed (step S1520), and the operation may be terminated. In some example embodiments, the fingerprint recognition operation may be repeated until the number of times that the fingerprint recognition fails exceeds a threshold number.
- Referring to
FIG. 25 , in a method of performing optical fingerprint recognition according to example embodiments, step S1100, S1200, S1300 and S1400 inFIG. 25 may be substantially the same as step S1100, S1200, S1300 and S1400 inFIG. 22 , respectively. - An image may be displayed on at least a portion of remaining regions of the display panel other than the partial region in the fingerprint recognition mode (step S1600). Step S1600 may be performed simultaneously with at least one of steps S1300 and S1400. In some example embodiments, an image may also be displayed on the partial region where the fingerprint recognition window is disposed.
-
FIG. 26 is a diagram illustrating an example process for a method of performing optical fingerprint recognition according to example embodiments.FIG. 26 will be described with reference to the method ofFIG. 24 as an example. - Referring to
FIG. 26 , after obtaining the light source setting values and the calibration data (step S1100), the request may be generated or the request may occur to enter the fingerprint recognition mode (step S1200). For example, the request may occur in response to contact or proximity of theobject 10 on or to any area on thedisplay panel 110. For example, the request may occur while thedisplay panel 110 is in a stand-by mode (e.g., while thedisplay panel 110 displays the reduced or the minimal amount of information such as a current time). - For example, the user may not know a location at which the
fingerprint recognition sensor 130 is disposed. Thus, in some example embodiments, the user may contact or approach a region other than the fingerprint recognition window FRW through theobject 10. Theelectronic device 100 may determine that a touched area does not coincide with the fingerprint recognition window FRW, and may display a reference image RI by partially driving thedisplay panel 110 under control of thedisplay driver 120. The reference image RI may be displayed to inform the user of the location at which thefingerprint recognition sensor 130 is disposed. The reference image RI may be displayed on some or all portions of the fingerprint recognition window FRW. - After that, the user may contact or approach, through the
object 10, the fingerprint recognition window FRW in which the reference image RI is displayed. Theelectronic device 100 may determine that a touched area coincides with the fingerprint recognition window FRW, and may emit light by partially driving thedisplay panel 110 under control of the display driver 120 (step S1300). Theelectronic device 100 may generate an image signal associated with theobject 10 which is on the fingerprint recognition window FRW, based on the emitted light. - The
electronic device 100 may recognize the fingerprint based on the reflected light of the fingerprint received through the fingerprint recognition window FRW and the calibration data (step S1400). Theelectronic device 100 may determine whether the fingerprint recognized in the fingerprint recognition mode is the fingerprint of the authenticated user (step S1500). - In some example embodiments, the reference image RI may be provided in association with a function of fingerprint detection. For example, since the function of fingerprint detection is associated with an issue of user authentication and security, the function of fingerprint detection may be processed with the highest priority. In some example embodiments, the
electronic device 100 may suitably drive thedisplay panel 110 under control of thedisplay driver 120, such that an interface (e.g., the contact or proximity of the object 10) associated with the reference image RI is processed prior to an interface (e.g., a time setting) associated with the stand-by mode. In some cases, the user may contact or approach an area other than the fingerprint recognition window FRW again, even if the reference image RI is displayed. In this case, theelectronic device 100 may display an error response to inform the user that a touched area does not coincide with the fingerprint recognition window FRW. -
FIG. 27 is a block diagram illustrating an electronic device according to example embodiments. - Referring to
FIG. 27 , anelectronic device 1000 may include aprocessor 1010, amemory device 1020, afingerprint recognition sensor 1030, an input/output (I/O)device 1040, apower supply 1050 and adisplay device 1060. Theelectronic device 100 may further include a plurality of ports for communicating a video card, a sound card, a memory card, a universal serial bus (USB) device, other electronic devices, etc. - The
processor 1010 controls operations of theelectronic device 1000. Theprocessor 1010 may execute an operating system and at least one application to provide an internet browser, games, videos, or the like. Thememory device 1020 may store data for operations of theelectronic device 1000. The I/O device 1040 may include an input device such as a keyboard, a keypad, a mouse, a touchpad, a touch-screen, a remote controller, etc., and an output device such as a printer, a speaker, etc. Thepower supply 1050 may provide a power for operations of theelectronic device 1000. - The
display device 1060 includes a display panel. The display panel, thefingerprint recognition sensor 1030, theprocessor 1010 and thememory device 1020 inFIG. 27 may correspond to thedisplay panel 110, thefingerprint recognition sensor 130, theprocessor 140 and thememory 150 inFIG. 4 , respectively, and may perform the method of setting the light sources in the display panel according to example embodiments and the method of performing the optical fingerprint recognition according to example embodiments. - The inventive concept may be applied to various electronic devices and systems that include the display panels and the fingerprint recognition sensors and perform the optical fingerprint recognition. For example, the inventive concept may be applied to systems such as a mobile phone, a smart phone, a tablet computer, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a portable game console, a music player, a camcorder, a video player, a navigation device, a wearable device, an internet of things (IoT) device, an internet of everything (IoE) device, an e-book reader, a virtual reality (VR) device, an augmented reality (AR) device, a robotic device, etc.
- The foregoing is illustrative of example embodiments and is not to be construed as limiting thereof. Although some example embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible in the example embodiments without materially departing from the novel teachings and advantages of the example embodiments. Accordingly, all such modifications are intended to be included within the scope of the example embodiments as defined in the claims. Therefore, it is to be understood that the foregoing is illustrative of various example embodiments and is not to be construed as limited to the specific example embodiments disclosed, and that modifications to the disclosed example embodiments, as well as other example embodiments, are intended to be included within the scope of the appended claims.
Claims (20)
1. A method of setting light sources in a display panel for optical fingerprint recognition, the method comprising:
driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel;
obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and
determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data.
2. The method of claim 1 , wherein:
the initial calibration data includes a low frequency component and a high frequency component, and
the interference signal corresponds to the high frequency component included in the initial calibration data.
3. The method of claim 1 , wherein:
the initial light source setting values are a same value for all of the portion of the light sources, and
the final light source setting values are different values for the light sources of different colors among the portion of the light sources.
4. The method of claim 1 , wherein determining the final light source setting values includes:
determining a luminance of first light sources among the portion of the light sources as a first value, the first light sources having first color; and
determining a luminance of second light sources among the portion of the light sources as a second value different from the first value, the second light sources having second color different from the first color.
5. The method of claim 4 , wherein determining the final light source setting values further includes:
determining a luminance of third light sources among the portion of the light sources as a third value different from the first value and the second value, the third light sources having third color different from the first color and the second color.
6. The method of claim 1 , further comprising:
determining the final light source setting values such that powers for driving light sources at different positions among the portion of the light sources are different, the final light source setting values being determined to optimize a magnitude of the interference signal included in the initial calibration data.
7. The method of claim 6 , wherein the determining the final light source setting values such that the powers for driving the light sources at different positions are different includes:
determining a luminance of first light sources among the portion of the light sources as a first value, the first light sources being disposed in a first region of the partial region; and
determining a luminance of second light sources among the portion of the light sources as a second value different from the first value, the second light sources being disposed in a second region of the partial region different from the first region.
8. The method of claim 7 , wherein the determining the final light source setting values such that the powers for driving the light sources at different positions are different further includes:
determining a luminance of third light sources among the some light sources as a third value different from the first value and the second value, the third light sources being disposed in a third region of the partial region different from the first region and the second region.
9. The method of claim 1 , further comprising:
driving the portion of the light sources based on the final light source setting values; and
obtaining final calibration data based on reflected light of the object received by the fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the final light source setting values.
10. The method of claim 9 , wherein the portion of the light sources are simultaneously turned on based on the initial light source setting values or the final light source setting values.
11. The method of claim 1 , wherein:
the display panel includes a first surface on which an image is displayed and a second surface opposite to the first surface, and
the fingerprint recognition sensor is disposed under the second surface of the display panel.
12. The method of claim 11 , wherein the fingerprint recognition sensor includes:
a lens configured to concentrate the reflected light received through the fingerprint recognition window; and
an image sensor configured to generate an image signal corresponding to the object on the partial region based on the reflected light concentrated by the lens.
13. A method of setting light sources in a display panel for optical fingerprint recognition, the method comprising:
driving a portion of light sources among a plurality of light sources included in the display panel based on initial light source setting values, the portion of the light sources being disposed to correspond to a fingerprint recognition window which is a partial region of the display panel;
obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values; and
determining final light source setting values such that powers for driving light sources at different positions among the portion of light sources are different, the final light source setting values being determined to optimize a magnitude of an interference signal included in the initial calibration data.
14. The method of claim 13 , wherein the powers for driving the light sources are determined such that a luminance of the portion of the light sources becomes lower as a light source becomes closer to a center of the partial region.
15. A method of performing optical fingerprint recognition, the method comprising:
obtaining light source setting values and calibration data;
displaying, in a fingerprint recognition mode, a fingerprint recognition window on a partial region of a display panel based on the light source setting values; and
recognizing, in the fingerprint recognition mode, a fingerprint based on reflected light of the fingerprint received through the fingerprint recognition window and based on the calibration data,
wherein obtaining the light source setting values and the calibration data includes:
driving a portion of light sources disposed to correspond to the fingerprint recognition window among a plurality of light sources included in the display panel based on initial light source setting values;
obtaining initial calibration data based on reflected light of an object received by a fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the initial light source setting values;
determining final light source setting values such that powers for driving light sources of different colors among the portion of the light sources are different and/or powers for driving light sources at different positions among the some light sources are different, the final light source setting values being determined to minimize contrast of an interference signal included in the initial calibration data and/or to optimize a magnitude of the interference signal included in the initial calibration data, respectively;
driving the portion of the light sources based on the final light source setting values;
obtaining final calibration data based on reflected light of the object received by the fingerprint recognition sensor through the fingerprint recognition window while driving the portion of the light sources based on the final light source setting values; and
storing the final light source setting values and the final calibration data as the light source setting values and the calibration data.
16. The method of claim 15 , wherein, after the final light source setting values and the final calibration data are obtained and stored, a fingerprint recognition operation is performed by loading the stored final light source setting values and the stored final calibration data.
17. The method of claim 16 , wherein the final light source setting values and the final calibration data are obtained and stored when an electronic device including the display panel and the fingerprint recognition sensor is manufactured.
18. The method of claim 15 , wherein recognizing the fingerprint includes:
obtaining an image signal for the fingerprint based on the reflected light of the fingerprint received through the fingerprint recognition window; and
obtaining final information for the fingerprint by subtracting the calibration data from the image signal.
19. The method of claim 15 , further comprising:
determining whether the fingerprint recognized in the fingerprint recognition mode is a fingerprint of an authenticated user.
20. The method of claim 15 , further comprising:
displaying, in the fingerprint recognition mode, an image on at least a portion of remaining regions of the display panel other than the partial region.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020190066482A KR20200139941A (en) | 2019-06-05 | 2019-06-05 | Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same |
KR10-2019-0066482 | 2019-06-05 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200387688A1 true US20200387688A1 (en) | 2020-12-10 |
US10963665B2 US10963665B2 (en) | 2021-03-30 |
Family
ID=73609374
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/799,175 Active US10963665B2 (en) | 2019-06-05 | 2020-02-24 | Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same |
Country Status (3)
Country | Link |
---|---|
US (1) | US10963665B2 (en) |
KR (1) | KR20200139941A (en) |
CN (1) | CN112052715A (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210142085A1 (en) * | 2019-11-07 | 2021-05-13 | Novatek Microelectronics Corp. | Image processing method and related image processing system |
US20220207674A1 (en) * | 2020-12-29 | 2022-06-30 | Motorola Mobility Llc | Mitigating Display Interference With Captured Image Using Calibration Images |
US11514872B2 (en) | 2020-12-29 | 2022-11-29 | Motorola Mobility Llc | Mitigating display interference with captured image by controlling display brightness |
US20230043020A1 (en) * | 2020-01-21 | 2023-02-09 | Egis Technology Inc. | Image scanning device and image scanning method |
WO2023063862A1 (en) * | 2021-10-15 | 2023-04-20 | Fingerprint Cards Anacatum Ip Ab | Shading compensated readout |
US11749014B2 (en) * | 2020-08-13 | 2023-09-05 | Novatek Microelectronics Corp. | Optical fingerprint sensing apparatus, driver apparatus thereof and operation method |
US11900579B2 (en) | 2020-12-29 | 2024-02-13 | Motorola Mobility Llc | Generating calibration images by merging captured recalibration images with previously captured calibration images |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20240039254A (en) | 2022-09-19 | 2024-03-26 | 경희대학교 산학협력단 | Display apparatus for recognizing finger print and operating method thereof |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9342733B2 (en) | 2015-11-24 | 2016-05-17 | Secugen Corporation | Fingerprint sensing and calibration apparatus |
KR102401009B1 (en) | 2015-11-27 | 2022-05-24 | 엘지전자 주식회사 | Mobile terminal |
US20180074627A1 (en) | 2016-09-13 | 2018-03-15 | Samsung Electronics Co., Ltd. | Electronic device including display emitting light to provide optics-based fingerprint detection |
EP3383006A1 (en) | 2017-01-24 | 2018-10-03 | Shenzhen Goodix Technology Co., Ltd. | Electronic device |
KR101886035B1 (en) | 2017-06-08 | 2018-08-08 | 주식회사 센소니아 | Optical fingerprint sensing method reducing reducing the effect of ambient light |
CN107680988B (en) | 2017-09-15 | 2020-03-13 | 上海天马微电子有限公司 | Display panel and electronic equipment |
US10216975B1 (en) | 2018-02-23 | 2019-02-26 | Shenzhen GOODIX Technology Co., Ltd. | Optical imaging via imaging lens and imaging pinhole in under-screen optical sensor module for on-screen fingerprint sensing in devices having organic light emitting diode (OLED) screens or other screens |
-
2019
- 2019-06-05 KR KR1020190066482A patent/KR20200139941A/en active Search and Examination
-
2020
- 2020-02-24 US US16/799,175 patent/US10963665B2/en active Active
- 2020-05-19 CN CN202010428016.8A patent/CN112052715A/en active Pending
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210142085A1 (en) * | 2019-11-07 | 2021-05-13 | Novatek Microelectronics Corp. | Image processing method and related image processing system |
US20230043020A1 (en) * | 2020-01-21 | 2023-02-09 | Egis Technology Inc. | Image scanning device and image scanning method |
US11749014B2 (en) * | 2020-08-13 | 2023-09-05 | Novatek Microelectronics Corp. | Optical fingerprint sensing apparatus, driver apparatus thereof and operation method |
US20220207674A1 (en) * | 2020-12-29 | 2022-06-30 | Motorola Mobility Llc | Mitigating Display Interference With Captured Image Using Calibration Images |
US11514872B2 (en) | 2020-12-29 | 2022-11-29 | Motorola Mobility Llc | Mitigating display interference with captured image by controlling display brightness |
US11615517B2 (en) * | 2020-12-29 | 2023-03-28 | Motorola Mobility Llc | Mitigating display interference with captured image using calibration images |
US11900579B2 (en) | 2020-12-29 | 2024-02-13 | Motorola Mobility Llc | Generating calibration images by merging captured recalibration images with previously captured calibration images |
WO2023063862A1 (en) * | 2021-10-15 | 2023-04-20 | Fingerprint Cards Anacatum Ip Ab | Shading compensated readout |
Also Published As
Publication number | Publication date |
---|---|
KR20200139941A (en) | 2020-12-15 |
CN112052715A (en) | 2020-12-08 |
US10963665B2 (en) | 2021-03-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10963665B2 (en) | Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same | |
US20180349669A1 (en) | Operating method of optical fingerprint sensor, operating method of electronic device including the optical fingerprint sensor, and display device including the optical fingerprint sensor | |
US10909916B2 (en) | OLED array substrate, OLED display panel, pixel circuit, driving method and method for fingerprint recognition using OLED display panel | |
TWI696059B (en) | Electronic device and fingerprint sensing method | |
US20170371213A1 (en) | Array substrate and drive method thereof, display panel and display device | |
WO2017012259A1 (en) | Array substrate, driving method therefor, display panel, and display device | |
US11908227B2 (en) | Method and device for reference imaging and optical object recognition | |
CN110941981B (en) | Mobile fingerprint identification method and apparatus using display | |
US11610424B2 (en) | Systems and methods for registering a fingerprint based on an optical fingerprint recognition | |
US20100002008A1 (en) | Image input/output device and method of correcting photo-reception level in image input/output device, and method of inputting image | |
TWI404999B (en) | Input device and display | |
US11249598B2 (en) | Integrated fingerprint detection touch control display apparatus and integrated method of fingerprint detection, touch control, and image display | |
US11348376B2 (en) | Display device and method of anti-spoofing of the display device | |
US20220086378A1 (en) | Electronic device and imaging method thereof | |
US11556195B2 (en) | Input-display device with shared memory | |
US20220012452A1 (en) | Fingerprint identification method, fingerprint identification device, display panel and readable storage medium | |
KR20180132496A (en) | Operating method of optical fingerprint sensor and operating method of electronic device and display device including thereof | |
US11551467B2 (en) | Method of obtaining reference image for optical object recognition and method of performing optical object recognition using the same | |
CN109492520B (en) | Display device and biological feature detection method thereof | |
US11347353B2 (en) | Method of performing half-shutter function using optical object recognition and method of capturing image using the same | |
KR102335556B1 (en) | Touch sensing method and apparatus for identifying large object | |
US11816288B2 (en) | Dynamic noise sampling for unspecified display noise | |
CN118244921A (en) | Display driving method and module for touch display panel and touch display device | |
CN112750873A (en) | Display device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PARK, DONG JIN;KIM, DUCKSOO;MIN, DONGKI;AND OTHERS;SIGNING DATES FROM 20191209 TO 20191211;REEL/FRAME:051907/0368 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |