EP3246801A1 - Combined fingerprint recognition touch screen device, method of driving the touch screen device, and electronic device including the touch screen device - Google Patents

Combined fingerprint recognition touch screen device, method of driving the touch screen device, and electronic device including the touch screen device Download PDF

Info

Publication number
EP3246801A1
EP3246801A1 EP17151846.7A EP17151846A EP3246801A1 EP 3246801 A1 EP3246801 A1 EP 3246801A1 EP 17151846 A EP17151846 A EP 17151846A EP 3246801 A1 EP3246801 A1 EP 3246801A1
Authority
EP
European Patent Office
Prior art keywords
driving
lines
touch
driving lines
touch screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17151846.7A
Other languages
German (de)
French (fr)
Other versions
EP3246801B1 (en
Inventor
Daekun Yoon
Byungkyu Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of EP3246801A1 publication Critical patent/EP3246801A1/en
Application granted granted Critical
Publication of EP3246801B1 publication Critical patent/EP3246801B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416Control or interface arrangements specially adapted for digitisers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416Control or interface arrangements specially adapted for digitisers
    • G06F3/04166Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving
    • G06F3/041661Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving using detection at multiple resolutions, e.g. coarse and fine scanning; using detection within a limited area, e.g. object tracking window
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • G06F3/0446Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • G06V40/13Sensors therefor
    • G06V40/1306Sensors therefor non-optical, e.g. ultrasonic or capacitive sensing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04106Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection

Definitions

  • Apparatuses and methods consistent with exemplary embodiments relate to a combined fingerprint recognition touch screen device, and more particularly, to a combined fingerprint recognition on-screen touch screen device capable of performing both fingerprint recognition and touch sensing on a display panel, a method of driving the touch screen device, and an electronic device including the touch screen device.
  • Personal authentication using personal unique characteristics such as fingerprints, voice, face, hands, irises, and the like is gradually increasing.
  • Personal authentication is usually used in banking devices, access control systems, mobile devices, laptop computers, etc., and recently, with the wide spread of mobile devices like smartphones, a fingerprint recognition device for personal authentication is being adopted to protect security information stored in the smartphones.
  • a touch screen device is attached onto a display panel to provide an intuitive input interface to a user, and generally, a fingerprint recognition device is provided separately from the touch screen device to recognize a fingerprint only when a position is touched thereon.
  • Exemplary embodiments may address at least the above problems and/or disadvantages and other disadvantages not described above. Also, the exemplary embodiments are not required to overcome the disadvantages described above, and may not overcome any of the problems described above.
  • a touch screen device a touch pad including driving lines and sensing lines intersecting the driving lines, a driver configured to apply a driving signal to the driving lines, a signal output collector configured to receive a touch signal from the sensing lines, and a controller configured to control the driver and the signal output collector.
  • the controller is further configured to control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • Each of the predetermined intervals may be of n driving lines, the predetermined intervals may be over an entire region of the touch pad, and n may be a natural number that is greater than 1.
  • the controller may be further configured to control the driver to perform the primary scanning to sequentially apply the driving signal to (kn+m) th driving lines at the predetermined intervals, starting from an m th driving line, among the driving lines, n may be a natural number that is greater than 1, m may be a natural number that is greater than or equal to 1, and k may be an integer that is greater than or equal to 0.
  • the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before a third driving line where the touch is not sensed after the first driving line, among the first driving lines.
  • the controller may be further configured to determine whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more, and the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the determination that the first contact strength is less than the second contact strength by the reference value or more.
  • the controller may be further configured to determine the first contact strength and the second contact strength from capacitance variations in the respective third driving lines.
  • the touch panel may further include driving island sets that are spaced apart from each other, and each of the driving island sets may include a respective portion of the driving lines.
  • a width of each of the driving island sets may be about 4 mm to about 5 mm, and an interval between each adjacent pair of the driving island sets may be about 4 mm to about 5 mm.
  • the controller may be further configured to perform the primary scanning to sequentially apply the driving signal to one driving line among each of the driving island sets.
  • the controller may be further configured to determine the scanning region as a driving island set including third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  • a pitch between the driving lines may be about 70 ⁇ m.
  • the controller may be further configured to operate in a fingerprint recognition mode for recognizing a fingerprint or in a touch sensing mode for sensing a touch, and repeat the primary scanning in the touch sensing mode.
  • the controller may be further configured to, in the fingerprint recognition mode, after the primary scanning, determine whether a region that is touched on the touch pad is maintained in a same position on the touch pad for a period of time or longer, and perform the secondary scanning in response to the determination that the touched region touched is maintained in the same position on the touch pad for the period of time or longer.
  • an electronic device including a display panel, and a touch screen device including a touch pad including driving lines and sensing lines intersecting the driving lines, a driver configured to apply a driving signal to the driving lines, a signal output collector configured to receive a touch signal from the sensing lines, and a controller configured to control the driver and the signal output collector.
  • the controller is further configured to control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • a method of driving a touch screen device including a touch pad including driving lines and sensing lines intersecting the sensing lines, the method including performing primary scanning to sequentially apply a driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determining a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and performing secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • the performing of the primary scanning may further include sequentially applying the driving signal to (kn+m) th driving lines at the predetermined intervals, starting from an m th driving line, among the driving lines, n may be a natural number that is greater than 1, m may be a natural number that is greater than or equal to 1, and k may be an integer that is greater than or equal to 0.
  • the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before a third driving line where the touch is not sensed after the first driving line, among the first driving lines.
  • the method may further include determining whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more, and the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the determination that the first contact strength is less than the second contact strength by the reference value or more.
  • the touch panel may further include driving island sets that are spaced apart from each other, each of the driving island sets may include a respective portion of the driving lines, the performing of the primary scanning may include sequentially applying the driving signal to one driving line among each of the driving island sets, and the determining of the scanning region may include determining the scanning region as a driving island set including third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  • the method may further include operating in a fingerprint recognition mode for recognizing a fingerprint or in a touch sensing mode for sensing a touch, and repeating the primary scanning in the touch sensing mode.
  • the method may further include, in the fingerprint recognition mode, after the primary scanning, determining whether a region that is touched on the touch pad is maintained in a same position on the touch pad for a first period of time or longer, and the performing of the secondary scanning may include performing the secondary scanning in response to the determination that the touched region touched is maintained in the same position on the touch pad for the first period of time or longer.
  • the method may further include performing a user authentication to compare an image that is obtained through the secondary scanning with a previously-registered fingerprint image.
  • the method may further include determining whether there is no effective touch on the touch pad for a second period of time or longer in response to the determination that the touched region touched is not maintained in the same position on the touch pad for the first period of time or longer, and ending the fingerprint recognition mode in response to the determination that there is no effective touch on the touch pad for the second period time or longer.
  • a touch screen device including a touch pad including driving lines and sensing lines intersecting the driving lines, a driver, and a controller configured to control the driver to sequentially apply a driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • each block, unit and/or module may be implemented by dedicated hardware, or as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.
  • each block, unit and/or module of the embodiments may be physically separated into two or more interacting and discrete blocks, units and/or modules without departing from the scope of the inventive concepts. Further, the blocks, units and/or modules of the embodiments may be physically combined into more complex blocks, units and/or modules without departing from the scope of the inventive concepts.
  • the term "and/or" includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.
  • an expression such as “above” or “on” may include not only the meaning of “immediately on/under/to the left/to the right in a contact manner,” but also the meaning of “on/under/to the left/to the right in a noncontact manner.
  • FIG. 1 is a block diagram illustrating a structure of an electronic device including a combined fingerprint recognition and touch screen device, according to an exemplary embodiment.
  • an electronic device 100 may be a small-size mobile device including a smartphone, a smart watch, a tablet personal computer (PC), laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), or the like.
  • the electronic device 100 includes a central processing unit 110, a mobile communication interface 111, a short-range communication interface 112, a broadcasting receiver 113, a camera 114, a sensor 115, a global positioning system (GPS) receiver 116, an input/output interface 120, a power supply 130, a memory 140, a touch screen device 160.
  • GPS global positioning system
  • the communication interface 111 transmits and receives a radio signal to and from any one or any combination of a base station, an external terminal, and a server over a mobile communication network.
  • the radio signal may include various forms of data corresponding to transmission/reception of a voice call signal, a video communication call signal, or a text/multimedia message.
  • the short-range communication interface 112 performs a short-range wireless communication function.
  • the communication interface 150 may include, but is not limited to, a Bluetooth Low Energy (BLE) communication interface, a near field communication (NFC) unit, a wireless local area network (WLAN) (WiFi) communication interface, a ZigBee communication interface, an infrared Data Association (IrDA) communication interface, a WiFi Direct (WFD) communication interface, an ultra wideband (UWB) communication interface, and an Ant+ communication interface.
  • BLE Bluetooth Low Energy
  • NFC near field communication
  • WiFi wireless local area network
  • ZigBee ZigBee communication interface
  • IrDA infrared Data Association
  • WFD WiFi Direct
  • UWB ultra wideband
  • the broadcasting receiver 113 receives a digital multimedia broadcasting (DMB) broadcast signal.
  • the camera 114 may include a lens and optical devices for capturing a still image or a moving image, and includes a first camera and a second camera.
  • the sensor 115 may include a gravity sensor for sensing motion of the electronic device 100, an illumination sensor for sensing a brightness of light, a proximity sensor for sensing proximity of a person, a motion sensor for sensing motion of a person, etc.
  • the GPS receiver 116 receives a GPS signal from an artificial satellite. By using the GPS signal, various services may be provided to a user.
  • the input/output interface 120 provides an interface with an external device or the user, and includes a button 121, a microphone 122, a speaker 123, a vibration motor 124, a connector 125, and a display panel 126.
  • the power supply 130 is connected with a battery or an external power source to supply power for the electronic device 100.
  • the memory 140 stores various programs executed by the central processing unit 110.
  • the central processing unit 110 controls programs stored in the memory 140 and operations of the above-described components.
  • the programs stored in the memory 140 may be classified depending on their functions into a plurality of modules, e.g., a mobile communication module 141, a WiFi module 142, a Bluetooth module 143, a DMB module 142, a camera module 145, a sensor module 146, a GPS module 147, a video play module 148, an audio play module 149, a power module 150, a touch screen module 151, a UI module 152, and a fingerprint recognition module 153.
  • modules e.g., a mobile communication module 141, a WiFi module 142, a Bluetooth module 143, a DMB module 142, a camera module 145, a sensor module 146, a GPS module 147, a video play module 148, an audio play module 149, a power module 150, a touch screen module 151,
  • the touch screen device 160 may be a combined fingerprint recognition touch screen device 160.
  • the combined fingerprint recognition touch screen device 160 is configured to perform both fingerprint recognition for user authentication and touch sensing for inputting various commands and information to the electronic device 100 with a touch of a finger.
  • a touch screen controller 161 delivers fingerprint information or a touch input, which is input through the combined fingerprint recognition touch screen device 160, to the central processing unit 110.
  • FIG. 2 is a block diagram illustrating a structure of the combined fingerprint recognition touch screen device 160 according to an exemplary embodiment.
  • the combined fingerprint recognition touch screen device 160 includes a touch panel 162 in which a plurality of driving lines 164 and a plurality of sensing lines 166 are arranged to intersect with one another, a driver 163 that sequentially provides a driving signal to the plurality of driving lines 164, and a signal output collector 165 that collects a touch signal from the plurality of sensing lines 166 and outputs the touch signal as a digital signal.
  • the touch screen controller 161 controls operations of the driver 163 and the signal output collector 165 in cooperation with the touch screen module 151 and the central processing unit 110.
  • the touch panel 162 may be, for example, arranged on the display panel 126 of the electronic device 100.
  • the combined fingerprint recognition touch screen device 160 may be, for example, of a capacitance type.
  • a self capacitance or a mutual capacitance change occurs due to a touch input or a fingerprint input, and coordinates of a touch input or an image of a touched fingerprint may be calculated from an electric capacitance change at a plurality of touched nodes.
  • a node at which the driving lines 164 and the sensing lines 166 intersect may serve as a pixel for sensing a touch input or a fingerprint input.
  • the driving lines 164 and the sensing lines 166 are marked with thin solid lines for convenience in the drawing, in practice, a plurality of transparent electrode patterns may be arranged along the driving lines 164 and the sensing lines 166.
  • the driving lines 164 and the sensing lines 166 of the touch panel 162 may be arranged at shorter intervals than driving lines and sensing lines of touch panels that sense only a touch, and sizes of electrode patterns may also be small.
  • a pitch between the driving lines 164 may be about 70 ⁇ m
  • a pitch between the sensing lines 166 may be about 70 ⁇ m.
  • about 60 through 70 driving lines 164 and sensing lines 166 may be arranged in an area of about 4mm through about 5mm.
  • the driver 163 sequentially applies a driving signal to the driving lines 164 from the first line to the last line. For example, when a driving signal is applied to the first driving line 164, the signal output collector 165 may simultaneously receive an output signal from the plurality of sensing lines 166. Then, it may be determined at which node a touch is generated among nodes at which the first driving line 164 intersects the plurality of sensing lines 166. The driver 163 then applies a driving signal to the second driving line 164. In this way, scanning for the entire region of the touch panel 162 may be performed.
  • a time for performing scanning with respect to the entire region of the touch panel 162 may increase. For example, if a driving signal is applied to all the driving lines 164 from the first line to the last line one-by-one during fingerprint recognition, a time for fingerprint recognition may increase. An area of a fingerprint of a finger is much smaller than an area of the display panel 126 and the touch panel 162, and thus, if only a region of the touch panel 162 the finger touches is scanned, a time for fingerprint recognition may be reduced. In other words, by previously recognizing a position at which the finger is situated on the touch panel 162 and obtaining a fingerprint image of the position, speedy fingerprint recognition becomes possible.
  • FIG. 3 is a block diagram illustrating a process of searching for a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment
  • FIG. 4 is a graph illustrating a driving signal applied to the driving lines 164 in the process of FIG. 3 .
  • the combined fingerprint recognition touch screen device 160 may primarily search for a region where a finger is situated on the touch panel 162 by performing primary scanning before obtaining a fingerprint image. In this case, speedy scanning may be performed with respect to the entire region of the touch panel 162 by applying a driving signal (V) to some of the driving lines 164.
  • V driving signal
  • a driving signal may be sequentially applied to the entire region of the touch panel 162 at intervals of n lines.
  • n is a natural number greater than 1. That is, as marked with thick solid lines in FIG. 3 , the driving signal is applied to the first driving line 164, and then the driving signal is applied to a (n+1) th driving line 164. Then, the driving signals are sequentially applied to a (2n+1) th driving line 164, a (3n+1) th driving line 164, and a (4n+1) th driving line 164.
  • primary scanning starts from the first driving line 164 in FIG. 3
  • primary scanning may start from an m th driving line 164 by considering that a user may contact a finger around the center of the touch panel 162.
  • m is a natural number greater than 1.
  • a driving signal may be sequentially applied to the driving lines 164 in an order of the m th driving line 164, an (n+m) th driving line 164, a (2n+m) th driving line 164, a (3n+m) th driving line 164, and a (4n+m) th driving line 164.
  • the driving signal is sequentially applied to (kn+m) th driving lines, where n and m are natural numbers greater than 1 and k is an integer greater than or equal to 0.
  • a time for primary scanning may be reduced to about 1/n of a time for sequentially applying a driving signal to all the driving lines 164 one-by-one.
  • a driving signal may be applied at intervals of about 50 through about 150 lines.
  • a time for primary scanning may be about 1/50 through about 1/150 of a time for sequentially applying a driving signal to all the driving lines 164 of the touch panel 162 one-by-one.
  • a pitch between the driving lines 164 is about 70 ⁇ m, and thus if a driving signal is applied at intervals of about 50 through about 150 lines, a finger's position is searched for on the touch panel 162 at intervals of about 3mm through about 10mm.
  • a size of a fingerprint is about 2cm through about 4cm, and thus the position of the finger may be accurately recognized by searching for the position of the finger at intervals of about 3mm through about 10mm.
  • the combined fingerprint recognition touch screen device 160 then precisely performs secondary scanning with respect to the region where the finger is situated, recognized during primary scanning. For example, assuming that a region marked with "A" in an example of FIG. 3 is a region A where a finger contacts on the touch panel 162, a touch is sensed in the (n+1) th , (2n+1) th , and (3n+1) th driving lines 164. If the touch is fixed without moving for a predetermined time or more, it is determined that a finger contacts the touch panel 162 for fingerprint recognition, and secondary scanning is performed.
  • FIG. 5 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment
  • FIG. 6 is a graph illustrating a driving signal applied to the driving lines 164 in the process of FIG. 5 .
  • secondary scanning starts from the driving line 164 where a touch is sensed, that is, the (n+1) th driving line 164.
  • the driver 163 sequentially applies a driving signal (V) to all the driving lines 164 one-by-one from the (n+1) th driving line 164 to the 4n th driving line 164 during secondary scanning.
  • a secondary scanning region B determined in this way is marked with thick solid lines.
  • the secondary scanning region B may be determined based on the driving line 164 where the touch is sensed during primary scanning.
  • the secondary scanning region B may not completely match the region A where the finger is actually situated.
  • the region A where the finger is situated in FIG. 5 may protrude forward with respect to the secondary scanning region B.
  • the secondary scanning region B may extend to a portion where the finger is not situated.
  • the secondary scanning region B may overlap with a most part of the region A where the finger is situated, and thus the fingerprint may be sufficiently recognized from the secondary scanning region B.
  • fingerprint recognition is performed, the region A where the finger is situated is rapidly searched for by primary scanning, and precise scanning is performed with respect to the secondary scanning region B determined by primary scanning, thereby reducing a time for fingerprint recognition.
  • the combined fingerprint recognition touch screen device 160 may repeat the above-described primary scanning operation.
  • the combined fingerprint recognition touch screen device 160 may operate in a touch sensing mode to repeat primary scanning, and may operate in a fingerprint recognition mode to perform primary scanning and then secondary scanning if fingerprint recognition is needed.
  • scanning with respect to a boundary of a finger may be skipped in some cases, thus reducing the size of the secondary scanning region B.
  • FIG. 7 is a diagram illustrating a state in which a finger contacts the touch panel 162, according to an exemplary embodiment. As illustrated in FIG. 7 , if a position of the (3n+1) th driving line 164 is in a boundary of a region where the finger contacts the touch panel 162, secondary scanning may be omitted from the (3n+1) th driving line 164 to reduce a fingerprint recognition time.
  • a contact strength in the driving line 164 that is the last line of the driving lines 164 in which the touch is sensed may be compared with a contact strength in the other driving lines 164. If the contact strength in the last driving line 164 of the driving lines 164 where the touch is sensed is much smaller than contact strengths in the other driving lines 164, the position of the last driving line 164 where the touch is sensed may be determined as a boundary of the region the finger contacts on the touch panel 162. Herein, the contact strength may be determined as a capacitance variation in each driving line 164.
  • the position of the last driving line 164 where the touch is sensed may be determined as the boundary of the region the finger contacts on the touch panel 162.
  • the secondary scanning region B may be determined from the first driving line 164 where the touch is sensed to the driving line 164 immediately before the last driving line 164 where the touch is sensed.
  • the touch may be sensed in the (n+1) th , (2n+1) th , and (3n+1) th driving lines 164. Then, the touch screen controller 161 compares a capacitance variation in the (3n+1) th driving line 164 with a capacitance variation in the (n+1) th and (2n+1) th driving lines 164. If the capacitance variation in the (3n+1) th driving line 164 is nearly similar to the capacitance variation in the (n+1) th and (2n+1) th driving lines 164, the secondary scanning region B may be determined from the (n+1) th driving line 164 to the 4n th driving line 164 as illustrated in FIG. 5 .
  • a position of the (3n+1) th driving line 164 may be determined to be in a boundary of a region the finger contacts on the touch panel 162, and secondary scanning may not be performed from the (3n+1) th driving line 164. That is, the secondary scanning region B may be determined from the (n+1) th driving line 164 to the 3n th driving line 164.
  • FIG. 8 is a graph illustrating a driving signal applied to the driving lines 164 in the state of FIG. 7 .
  • the secondary scanning is performed from the (n+1) th driving line 164 to the 3n th driving line 164. Then, the number of driving lines 164 in the secondary scanning region B is reduced, thereby more rapidly completing secondary scanning. Hence, a time for fingerprint recognition may be further reduced.
  • FIG. 9 is a diagram illustrating a process of searching for a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to another exemplary embodiment.
  • the plurality of driving lines 164 and the plurality of sensing lines 166 are arranged uniformly over the entire region of the touch panel 162.
  • a manufacturing cost and power consumption for the combined fingerprint recognition touch screen device 160 may also increase.
  • fingerprint recognition may be possible merely with a partial region of a fingerprint rather than the entire region of the fingerprint.
  • the driving lines 164 and the sensing lines 166 may be distributed in the form of an island on the touch panel 162.
  • the plurality of driving lines 164 may be arranged along a plurality of driving island sets 164a, 164b, 164c, and 164d
  • the plurality of sensing lines 166 may be arranged along a plurality of sensing island sets 166a, 166b, 166c, and 166d.
  • the plurality of driving lines 164 are arranged in each of the driving island sets 164a, 164b, 164c, and 164d
  • the plurality of sensing lines 166 are arranged in each of the sensing island sets 166a, 166b, 166c, and 166d.
  • one driving island set 164a, 164b, 164c, or 164d and one sensing island set 166a, 166b, 166c, or 166d may have a width of about 4 mm to about 5 mm.
  • a pitch between the driving lines 164 may be about 70 ⁇ m
  • a pitch between the sensing lines 166 may be about 70 ⁇ m
  • about 60 to 70 driving lines and sensing lines 166 may be arranged in one driving island set 164a, 164b, 164c, or 164d and one sensing island set 166a, 166b, 166c, or 166d, respectively.
  • Each of an interval between adjacent driving island sets 164a, 164b, 164c, and 164d and an interval between adjacent sensing island sets 166a, 166b, 166c, and 166d may be about 4 mm to about 5 mm.
  • a driving signal is sequentially applied to the first driving lines 164 among the driving island sets 164a, 164b, 164c, and 164d as indicated by solid lines in FIG. 9 .
  • a driving signal may be applied to the first driving line 164 in the first driving island set 164a
  • a driving signal may be applied to the first driving line 164 in the second driving island set 164b
  • a driving signal may be applied to the first driving line 164 in the third driving island set 164c.
  • rough scanning may be performed over the entire region of the touch panel 162.
  • the above description is merely an example, and another one may be selected from each driving island set 164a, 164b, 164c, or 164d to apply a driving signal to the selected driving line.
  • secondary scanning may be performed precisely with respect to the recognized region where the finger is situated.
  • the region marked with "A" is a region A the finger contacts on the touch panel 162
  • a touch is sensed in the second driving island set 164b and the third driving island set 164c.
  • a driving signal is sequentially applied to all the driving lines 164 of the second driving island set 164b and the third driving island set 164c.
  • FIG. 10 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated in the combined fingerprint recognition touch screen device 160, according to another exemplary embodiment.
  • the driver 163 sequentially applies a driving signal to the driving lines 164 of the second driving island set 164b and the third driving island set 164c.
  • the secondary scanning region B may include the second driving island set 164b and the third driving island set 164c. That is, the second driving island set 164b and the third driving island set 164c including the driving lines 164 in which a touch is sensed are determined as the secondary scanning region B, and secondary scanning is performed with respect to all the driving lines 164 of the second driving island set 164b and the third driving island set 164c.
  • a fingerprint image may be substantially obtained only in regions B1, B2, B3, and B4 of the secondary scanning region B where the second and third driving island sets 164b and 164c and the second and third sensing island sets 166b and 166c intersect each other. Fingerprint recognition may be sufficiently possible from fingerprint images obtained from the partial regions B1, B2, B3, and B4.
  • a capacitance variation in the last third driving island set 164c is compared with a capacitance variation in the second driving island set 164b. If a capacitance change in the third driving island set 164c is much smaller than a capacitance change in the second driving island set 164b, secondary scanning may be performed only for the second driving island set 164b except for the third driving island set 164c. That is, the secondary scanning region B may be determined as the second driving island set 164b.
  • the foregoing description has been made of a case in which primary scanning and secondary scanning are performed only for the driving lines 164.
  • the signal output collector 165 receives output signals from the plurality of sensing lines 166 at the same time, and thus for the sensing lines 166, scanning of the same type as for the driving lines 164 is not necessary. However, if scanning of the same type as for the driving lines 164 is performed for outputting a touch sensing signal, primary scanning and secondary scanning may be performed with respect to the sensing lines 166 in the same manner as described for the driving lines 164.
  • the combined fingerprint recognition touch screen device 160 is applicable to various electric devices 100, e.g., smartphones, smart watches, etc.
  • the touch panel 162 of the combined fingerprint recognition touch screen device 160 may be mounted on the display panel 126 of the electronic device 100.
  • the electronic device 100 including the disclosed combined fingerprint recognition touch screen device 160 then may perform fingerprint recognition for user authentication merely with a user's touch on the display panel 126 by a finger. Therefore, there is no need to mount a separate fingerprint sensor, saving a space for the fingerprint sensor.
  • FIG. 11 is a flowchart illustrating a process of recognizing a fingerprint and performing user authentication in the electronic device 100 including the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment.
  • the electronic device 100 starts a fingerprint recognition mode. If user authentication is performed in the electronic device 100 when the user presets user authentication to be performed with fingerprint recognition, the fingerprint recognition mode may start. For example, if user authentication is performed for unlocking a screen in a screen lock state of the electronic device 100, the fingerprint recognition mode may start. When user authentication is performed for performing online banking, online payment, or the like by using the electronic device 100, the fingerprint recognition mode may start.
  • the central processing unit 110 executes, for example, the fingerprint recognition module 153 that then performs primary scanning by controlling the driver 163 and the signal output collector 165 through the touch screen controller 161.
  • a driving signal may be applied to only some of the plurality of driving lines 164, thereby rapidly and roughly performing primary scanning with respect to the entire region of the touch panel 162.
  • the electronic device 100 determines whether there is a touch by a finger for fingerprint recognition on the touch panel 162 of the electronic device 100, i.e., whether the touched region is maintained in the same position on the touch panel 162 for a first predetermined time or longer. If so, it may be determined that there is a contact by the finger for fingerprint recognition. For example, if the touched region is maintained in the same position for at least 100 ms, it may be determined that there is a contact by the finger.
  • the fingerprint recognition module 153 performs secondary scanning by controlling the driver 163 and the signal output collector 165 through the touch screen controller 161. Secondary scanning may be performed by sequentially applying a driving voltage to all the driving lines 164 of the secondary scanning region B determined in the primary scanning operation S11. For example, secondary scanning may be performed in the manner described with reference to FIG. 5 , 8 , or 10 . Through such secondary scanning, a high-resolution image for the touched region may be obtained.
  • the fingerprint recognition module 153 compares the image obtained in the secondary scanning operation S13 with a user's fingerprint image previously registered in the electronic device 100 . If the image obtained in the secondary scanning operation S13 completely matches the user's fingerprint image, in operation S15, the electronic device 100 determines that user authentication is successful. If user authentication is successful, the screen of the electronic device 100 may be unlocked, or online banking or online payment may be performed normally. However, if the image obtained in the secondary scanning operation S13 does not match the user's fingerprint image, in operation S16, the electronic device 100 determines whether user authentication fails consecutive times or more, e.g., consecutive five times.
  • an alert message may be displayed on the display panel 126 of the electronic device 100, and the fingerprint recognition mode ends.
  • the process returns to a touch sensing mode in which only primary scanning is repeatedly performed. If user authentication does not fail consecutive times or more, the process returns to operation S11.
  • the secondary scanning operation S13 is not performed. Instead, in operation S17, the electronic device 100 determines whether there is no effective touch for a second predetermined time or longer. If there is no effective touch for the second predetermined time or longer, for example, if there is no effective touch for fingerprint recognition for one second, two seconds, or longer depending on a user's setting, in operation S18, an alert message may be displayed on the display panel 126 of the electronic device 100, and the fingerprint recognition mode ends. In operation S19, the process returns to a touch sensing mode in which only primary scanning is repeatedly performed. If there is an effective touch for the second predetermined time, the process returns to operation S11.
  • the exemplary embodiments may also be implemented through computer-readable code and/or instructions on a medium, e.g., a computer-readable medium, to control at least one processing element to implement any of the above-described exemplary embodiments.
  • the medium may correspond to any medium or media that may serve as a storage and/or perform transmission of the computer-readable code.
  • the computer-readable code may be recorded and/or transferred on a medium in a variety of ways, and examples of the medium include recording media, such as magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) and optical recording media (e.g., compact disc read only memories (CD-ROMs) or digital versatile discs (DVDs)), and transmission media such as Internet transmission media.
  • the medium may have a structure suitable for storing or carrying a signal or information, such as a device carrying a bitstream according to exemplary embodiments.
  • the medium may also be on a distributed network, so that the computer-readable code is stored and/or transferred on the medium and executed in a distributed fashion.
  • the processing element may include a processor or a computer processor, and the processing element may be distributed and/or included in a single device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Image Input (AREA)
  • Position Input By Displaying (AREA)

Abstract

A touch screen device includes a touch pad including driving lines and sensing lines intersecting the driving lines, a driver configured to apply a driving signal to the driving lines, a signal output collector configured to receive a touch signal from the sensing lines, and a controller configured to control the driver and the signal output collector. The controller is further configured to control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.

Description

    FIELD OF THE INVENTION
  • Apparatuses and methods consistent with exemplary embodiments relate to a combined fingerprint recognition touch screen device, and more particularly, to a combined fingerprint recognition on-screen touch screen device capable of performing both fingerprint recognition and touch sensing on a display panel, a method of driving the touch screen device, and an electronic device including the touch screen device.
  • BACKGROUND OF THE INVENTION
  • Personal authentication using personal unique characteristics such as fingerprints, voice, face, hands, irises, and the like is gradually increasing. Personal authentication is usually used in banking devices, access control systems, mobile devices, laptop computers, etc., and recently, with the wide spread of mobile devices like smartphones, a fingerprint recognition device for personal authentication is being adopted to protect security information stored in the smartphones.
  • In the smartphone, a touch screen device is attached onto a display panel to provide an intuitive input interface to a user, and generally, a fingerprint recognition device is provided separately from the touch screen device to recognize a fingerprint only when a position is touched thereon.
  • SUMMARY OF THE INVENTION
  • Exemplary embodiments may address at least the above problems and/or disadvantages and other disadvantages not described above. Also, the exemplary embodiments are not required to overcome the disadvantages described above, and may not overcome any of the problems described above.
  • According to an aspect of an exemplary embodiment, there is provided a touch screen device a touch pad including driving lines and sensing lines intersecting the driving lines, a driver configured to apply a driving signal to the driving lines, a signal output collector configured to receive a touch signal from the sensing lines, and a controller configured to control the driver and the signal output collector. The controller is further configured to control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • Each of the predetermined intervals may be of n driving lines, the predetermined intervals may be over an entire region of the touch pad, and n may be a natural number that is greater than 1.
  • The controller may be further configured to control the driver to perform the primary scanning to sequentially apply the driving signal to (kn+m)th driving lines at the predetermined intervals, starting from an mth driving line, among the driving lines, n may be a natural number that is greater than 1, m may be a natural number that is greater than or equal to 1, and k may be an integer that is greater than or equal to 0.
  • The determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before a third driving line where the touch is not sensed after the first driving line, among the first driving lines.
  • The controller may be further configured to determine whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more, and the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the determination that the first contact strength is less than the second contact strength by the reference value or more.
  • The controller may be further configured to determine the first contact strength and the second contact strength from capacitance variations in the respective third driving lines.
  • The touch panel may further include driving island sets that are spaced apart from each other, and each of the driving island sets may include a respective portion of the driving lines.
  • A width of each of the driving island sets may be about 4 mm to about 5 mm, and an interval between each adjacent pair of the driving island sets may be about 4 mm to about 5 mm.
  • The controller may be further configured to perform the primary scanning to sequentially apply the driving signal to one driving line among each of the driving island sets.
  • The controller may be further configured to determine the scanning region as a driving island set including third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  • A pitch between the driving lines may be about 70 µm.
  • The controller may be further configured to operate in a fingerprint recognition mode for recognizing a fingerprint or in a touch sensing mode for sensing a touch, and repeat the primary scanning in the touch sensing mode.
  • The controller may be further configured to, in the fingerprint recognition mode, after the primary scanning, determine whether a region that is touched on the touch pad is maintained in a same position on the touch pad for a period of time or longer, and perform the secondary scanning in response to the determination that the touched region touched is maintained in the same position on the touch pad for the period of time or longer.
  • According to an aspect of another exemplary embodiment, there is provided an electronic device including a display panel, and a touch screen device including a touch pad including driving lines and sensing lines intersecting the driving lines, a driver configured to apply a driving signal to the driving lines, a signal output collector configured to receive a touch signal from the sensing lines, and a controller configured to control the driver and the signal output collector. The controller is further configured to control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • According to an aspect of another exemplary embodiment, there is provided a method of driving a touch screen device including a touch pad including driving lines and sensing lines intersecting the sensing lines, the method including performing primary scanning to sequentially apply a driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determining a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and performing secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • The performing of the primary scanning may further include sequentially applying the driving signal to (kn+m)th driving lines at the predetermined intervals, starting from an mth driving line, among the driving lines, n may be a natural number that is greater than 1, m may be a natural number that is greater than or equal to 1, and k may be an integer that is greater than or equal to 0.
  • The determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before a third driving line where the touch is not sensed after the first driving line, among the first driving lines.
  • The method may further include determining whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more, and the determined scanning region may be from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the determination that the first contact strength is less than the second contact strength by the reference value or more.
  • The touch panel may further include driving island sets that are spaced apart from each other, each of the driving island sets may include a respective portion of the driving lines, the performing of the primary scanning may include sequentially applying the driving signal to one driving line among each of the driving island sets, and the determining of the scanning region may include determining the scanning region as a driving island set including third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  • The method may further include operating in a fingerprint recognition mode for recognizing a fingerprint or in a touch sensing mode for sensing a touch, and repeating the primary scanning in the touch sensing mode.
  • The method may further include, in the fingerprint recognition mode, after the primary scanning, determining whether a region that is touched on the touch pad is maintained in a same position on the touch pad for a first period of time or longer, and the performing of the secondary scanning may include performing the secondary scanning in response to the determination that the touched region touched is maintained in the same position on the touch pad for the first period of time or longer.
  • The method may further include performing a user authentication to compare an image that is obtained through the secondary scanning with a previously-registered fingerprint image.
  • The method may further include determining whether there is no effective touch on the touch pad for a second period of time or longer in response to the determination that the touched region touched is not maintained in the same position on the touch pad for the first period of time or longer, and ending the fingerprint recognition mode in response to the determination that there is no effective touch on the touch pad for the second period time or longer.
  • According to an aspect of another exemplary embodiment, there is provided a touch screen device including a touch pad including driving lines and sensing lines intersecting the driving lines, a driver, and a controller configured to control the driver to sequentially apply a driving signal to first driving lines disposed at predetermined intervals, among the driving lines, determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied, and control the driver to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The above and/or other aspects will be more apparent by describing exemplary embodiments with reference to the accompanying drawings, in which:
    • FIG. 1 is a block diagram illustrating a structure of an electronic device including a combined fingerprint recognition touch screen device, according to an exemplary embodiment;
    • FIG. 2 is a block diagram illustrating a structure of a combined fingerprint recognition touch screen device according to an exemplary embodiment;
    • FIG. 3 is a diagram illustrating a process of searching for a region where a finger is situated on a combined fingerprint recognition touch screen device, according to an exemplary embodiment;
    • FIG. 4 is a graph illustrating a driving signal applied to driving lines in the process of FIG. 3;
    • FIG. 5 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated on a combined fingerprint recognition touch screen device, according to an exemplary embodiment;
    • FIG. 6 is a graph illustrating a driving signal applied to driving lines in the process of FIG. 5;
    • FIG. 7 is a diagram illustrating a state in which a finger contacts a touch panel, according to an exemplary embodiment;
    • FIG. 8 is a graph illustrating a driving signal applied to driving lines during the state of FIG. 7;
    • FIG. 9 is a diagram illustrating a process of searching for a region where a finger is situated on a combined fingerprint recognition touch screen device, according to another exemplary embodiment;
    • FIG. 10 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated on a combined fingerprint recognition touch screen device, according to another exemplary embodiment; and
    • FIG. 11 is a flowchart illustrating a process of recognizing a fingerprint and performing user authentication in an electronic device including a combined fingerprint recognition touch screen device, according to an exemplary embodiment.
    DETAILED DESCRIPTION
  • Exemplary embodiments are described in greater detail below with reference to the accompanying drawings.
  • In the following description, like drawing reference numerals are used for like elements, even in different drawings. The matters defined in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the exemplary embodiments. However, it is apparent that the exemplary embodiments can be practiced without those specifically defined matters. Also, well-known functions or constructions may not be described in detail because they would obscure the description with unnecessary detail.
  • It will be understood that the terms "comprises" and/or "comprising" used herein specify the presence of stated features or components, but do not preclude the presence or addition of one or more other features or components.
  • As is traditional in the field of the inventive concepts, embodiments are described, and illustrated in the drawings, in terms of functional blocks, units and/or modules. Those skilled in the art will appreciate that these blocks, units and/or modules are physically implemented by electronic (or optical) circuits such as logic circuits, discrete components, microprocessors, hard-wired circuits, memory elements, wiring connections, and the like, which may be formed using semiconductor-based fabrication techniques or other manufacturing technologies. In the case of the blocks, units and/or modules being implemented by microprocessors or similar, they may be programmed using software (e.g., microcode) to perform various functions discussed herein and may optionally be driven by firmware and/or software. Alternatively, each block, unit and/or module may be implemented by dedicated hardware, or as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions. Also, each block, unit and/or module of the embodiments may be physically separated into two or more interacting and discrete blocks, units and/or modules without departing from the scope of the inventive concepts. Further, the blocks, units and/or modules of the embodiments may be physically combined into more complex blocks, units and/or modules without departing from the scope of the inventive concepts.
  • As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of," when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.
  • In a layer structure described below, an expression such as "above" or "on" may include not only the meaning of "immediately on/under/to the left/to the right in a contact manner," but also the meaning of "on/under/to the left/to the right in a noncontact manner.
  • FIG. 1 is a block diagram illustrating a structure of an electronic device including a combined fingerprint recognition and touch screen device, according to an exemplary embodiment. For example, an electronic device 100 may be a small-size mobile device including a smartphone, a smart watch, a tablet personal computer (PC), laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), or the like. Referring to FIG. 1, the electronic device 100 includes a central processing unit 110, a mobile communication interface 111, a short-range communication interface 112, a broadcasting receiver 113, a camera 114, a sensor 115, a global positioning system (GPS) receiver 116, an input/output interface 120, a power supply 130, a memory 140, a touch screen device 160.
  • The communication interface 111 transmits and receives a radio signal to and from any one or any combination of a base station, an external terminal, and a server over a mobile communication network. Herein, the radio signal may include various forms of data corresponding to transmission/reception of a voice call signal, a video communication call signal, or a text/multimedia message. The short-range communication interface 112 performs a short-range wireless communication function. The communication interface 150 may include, but is not limited to, a Bluetooth Low Energy (BLE) communication interface, a near field communication (NFC) unit, a wireless local area network (WLAN) (WiFi) communication interface, a ZigBee communication interface, an infrared Data Association (IrDA) communication interface, a WiFi Direct (WFD) communication interface, an ultra wideband (UWB) communication interface, and an Ant+ communication interface.
  • The broadcasting receiver 113 receives a digital multimedia broadcasting (DMB) broadcast signal. The camera 114 may include a lens and optical devices for capturing a still image or a moving image, and includes a first camera and a second camera. The sensor 115 may include a gravity sensor for sensing motion of the electronic device 100, an illumination sensor for sensing a brightness of light, a proximity sensor for sensing proximity of a person, a motion sensor for sensing motion of a person, etc. The GPS receiver 116 receives a GPS signal from an artificial satellite. By using the GPS signal, various services may be provided to a user.
  • The input/output interface 120 provides an interface with an external device or the user, and includes a button 121, a microphone 122, a speaker 123, a vibration motor 124, a connector 125, and a display panel 126. The power supply 130 is connected with a battery or an external power source to supply power for the electronic device 100.
  • The memory 140 stores various programs executed by the central processing unit 110. The central processing unit 110 controls programs stored in the memory 140 and operations of the above-described components. The programs stored in the memory 140 may be classified depending on their functions into a plurality of modules, e.g., a mobile communication module 141, a WiFi module 142, a Bluetooth module 143, a DMB module 142, a camera module 145, a sensor module 146, a GPS module 147, a video play module 148, an audio play module 149, a power module 150, a touch screen module 151, a UI module 152, and a fingerprint recognition module 153.
  • The touch screen device 160 may be a combined fingerprint recognition touch screen device 160. The combined fingerprint recognition touch screen device 160 is configured to perform both fingerprint recognition for user authentication and touch sensing for inputting various commands and information to the electronic device 100 with a touch of a finger. A touch screen controller 161 delivers fingerprint information or a touch input, which is input through the combined fingerprint recognition touch screen device 160, to the central processing unit 110.
  • FIG. 2 is a block diagram illustrating a structure of the combined fingerprint recognition touch screen device 160 according to an exemplary embodiment. Referring to FIG. 2, the combined fingerprint recognition touch screen device 160 includes a touch panel 162 in which a plurality of driving lines 164 and a plurality of sensing lines 166 are arranged to intersect with one another, a driver 163 that sequentially provides a driving signal to the plurality of driving lines 164, and a signal output collector 165 that collects a touch signal from the plurality of sensing lines 166 and outputs the touch signal as a digital signal. The touch screen controller 161 controls operations of the driver 163 and the signal output collector 165 in cooperation with the touch screen module 151 and the central processing unit 110. The touch panel 162 may be, for example, arranged on the display panel 126 of the electronic device 100.
  • The combined fingerprint recognition touch screen device 160 may be, for example, of a capacitance type. In this case, at a node at which the plurality of driving lines 164 and the plurality of sensing lines 166 intersect, a self capacitance or a mutual capacitance change occurs due to a touch input or a fingerprint input, and coordinates of a touch input or an image of a touched fingerprint may be calculated from an electric capacitance change at a plurality of touched nodes. Thus, a node at which the driving lines 164 and the sensing lines 166 intersect may serve as a pixel for sensing a touch input or a fingerprint input. While the driving lines 164 and the sensing lines 166 are marked with thin solid lines for convenience in the drawing, in practice, a plurality of transparent electrode patterns may be arranged along the driving lines 164 and the sensing lines 166.
  • To obtain a resolution that is high enough for fingerprint recognition, the driving lines 164 and the sensing lines 166 of the touch panel 162 may be arranged at shorter intervals than driving lines and sensing lines of touch panels that sense only a touch, and sizes of electrode patterns may also be small. For example, a pitch between the driving lines 164 may be about 70 µm, and a pitch between the sensing lines 166 may be about 70 µm. In this case, about 60 through 70 driving lines 164 and sensing lines 166 may be arranged in an area of about 4mm through about 5mm.
  • When the combined fingerprint recognition touch screen device 160 senses a fingerprint or a touch, the driver 163 sequentially applies a driving signal to the driving lines 164 from the first line to the last line. For example, when a driving signal is applied to the first driving line 164, the signal output collector 165 may simultaneously receive an output signal from the plurality of sensing lines 166. Then, it may be determined at which node a touch is generated among nodes at which the first driving line 164 intersects the plurality of sensing lines 166. The driver 163 then applies a driving signal to the second driving line 164. In this way, scanning for the entire region of the touch panel 162 may be performed.
  • Because the combined fingerprint recognition touch screen device 160 has more driving lines 164 than touch panels that sense only a touch, a time for performing scanning with respect to the entire region of the touch panel 162 may increase. For example, if a driving signal is applied to all the driving lines 164 from the first line to the last line one-by-one during fingerprint recognition, a time for fingerprint recognition may increase. An area of a fingerprint of a finger is much smaller than an area of the display panel 126 and the touch panel 162, and thus, if only a region of the touch panel 162 the finger touches is scanned, a time for fingerprint recognition may be reduced. In other words, by previously recognizing a position at which the finger is situated on the touch panel 162 and obtaining a fingerprint image of the position, speedy fingerprint recognition becomes possible.
  • FIG. 3 is a block diagram illustrating a process of searching for a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment, and FIG. 4 is a graph illustrating a driving signal applied to the driving lines 164 in the process of FIG. 3. Referring to FIGS. 3 and 4, the combined fingerprint recognition touch screen device 160 may primarily search for a region where a finger is situated on the touch panel 162 by performing primary scanning before obtaining a fingerprint image. In this case, speedy scanning may be performed with respect to the entire region of the touch panel 162 by applying a driving signal (V) to some of the driving lines 164. For example, during primary scanning, a driving signal may be sequentially applied to the entire region of the touch panel 162 at intervals of n lines. Herein, n is a natural number greater than 1. That is, as marked with thick solid lines in FIG. 3, the driving signal is applied to the first driving line 164, and then the driving signal is applied to a (n+1)th driving line 164. Then, the driving signals are sequentially applied to a (2n+1)th driving line 164, a (3n+1)th driving line 164, and a (4n+1)th driving line 164.
  • Although it has been described that primary scanning starts from the first driving line 164 in FIG. 3, primary scanning may start from an mth driving line 164 by considering that a user may contact a finger around the center of the touch panel 162. Herein, m is a natural number greater than 1. For example, a driving signal may be sequentially applied to the driving lines 164 in an order of the mth driving line 164, an (n+m)th driving line 164, a (2n+m)th driving line 164, a (3n+m)th driving line 164, and a (4n+m)th driving line 164. Hereinbelow, for ease of a description, it will be described that primary scanning starts from the first driving line 164. The driving signal is sequentially applied to (kn+m)th driving lines, where n and m are natural numbers greater than 1 and k is an integer greater than or equal to 0.
  • By scanning the entire region of the touch panel 162 in this way, a time for primary scanning may be reduced to about 1/n of a time for sequentially applying a driving signal to all the driving lines 164 one-by-one. For example, a driving signal may be applied at intervals of about 50 through about 150 lines. Then, a time for primary scanning may be about 1/50 through about 1/150 of a time for sequentially applying a driving signal to all the driving lines 164 of the touch panel 162 one-by-one. As described above, a pitch between the driving lines 164 is about 70µm, and thus if a driving signal is applied at intervals of about 50 through about 150 lines, a finger's position is searched for on the touch panel 162 at intervals of about 3mm through about 10mm. A size of a fingerprint is about 2cm through about 4cm, and thus the position of the finger may be accurately recognized by searching for the position of the finger at intervals of about 3mm through about 10mm.
  • The combined fingerprint recognition touch screen device 160 then precisely performs secondary scanning with respect to the region where the finger is situated, recognized during primary scanning. For example, assuming that a region marked with "A" in an example of FIG. 3 is a region A where a finger contacts on the touch panel 162, a touch is sensed in the (n+1)th, (2n+1)th, and (3n+1)th driving lines 164. If the touch is fixed without moving for a predetermined time or more, it is determined that a finger contacts the touch panel 162 for fingerprint recognition, and secondary scanning is performed.
  • FIG. 5 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment, and FIG. 6 is a graph illustrating a driving signal applied to the driving lines 164 in the process of FIG. 5. Referring to FIGS. 5 and 6, secondary scanning starts from the driving line 164 where a touch is sensed, that is, the (n+1)th driving line 164. After scanning in the (n+1)th driving line 164, scanning is performed to the driving line 164 in which a touch is not sensed, e.g., the driving line 164 immediately before the (4n+1)th driving line 164, i.e., the 4nth driving line 164. For example, the driver 163 sequentially applies a driving signal (V) to all the driving lines 164 one-by-one from the (n+1)th driving line 164 to the 4nth driving line 164 during secondary scanning.
  • In FIG. 5, a secondary scanning region B determined in this way is marked with thick solid lines. The secondary scanning region B may be determined based on the driving line 164 where the touch is sensed during primary scanning. The secondary scanning region B may not completely match the region A where the finger is actually situated. For example, the region A where the finger is situated in FIG. 5 may protrude forward with respect to the secondary scanning region B. The secondary scanning region B may extend to a portion where the finger is not situated. However, the secondary scanning region B may overlap with a most part of the region A where the finger is situated, and thus the fingerprint may be sufficiently recognized from the secondary scanning region B. As such, when fingerprint recognition is performed, the region A where the finger is situated is rapidly searched for by primary scanning, and precise scanning is performed with respect to the secondary scanning region B determined by primary scanning, thereby reducing a time for fingerprint recognition.
  • Meanwhile, in a touch sensing mode in which a touch sensing function is performed without fingerprint recognition, precise sensing is not required, and thus the combined fingerprint recognition touch screen device 160 may repeat the above-described primary scanning operation. For example, the combined fingerprint recognition touch screen device 160 may operate in a touch sensing mode to repeat primary scanning, and may operate in a fingerprint recognition mode to perform primary scanning and then secondary scanning if fingerprint recognition is needed.
  • To further reduce a time for secondary scanning, scanning with respect to a boundary of a finger may be skipped in some cases, thus reducing the size of the secondary scanning region B.
  • FIG. 7 is a diagram illustrating a state in which a finger contacts the touch panel 162, according to an exemplary embodiment. As illustrated in FIG. 7, if a position of the (3n+1)th driving line 164 is in a boundary of a region where the finger contacts the touch panel 162, secondary scanning may be omitted from the (3n+1)th driving line 164 to reduce a fingerprint recognition time.
  • To this end, before second scanning starts after completion of primary scanning, a contact strength in the driving line 164 that is the last line of the driving lines 164 in which the touch is sensed may be compared with a contact strength in the other driving lines 164. If the contact strength in the last driving line 164 of the driving lines 164 where the touch is sensed is much smaller than contact strengths in the other driving lines 164, the position of the last driving line 164 where the touch is sensed may be determined as a boundary of the region the finger contacts on the touch panel 162. Herein, the contact strength may be determined as a capacitance variation in each driving line 164. For example, if a capacitance variation in the last driving line 164 of the driving lines 164 where the touch is sensed is less than capacitance variations in the other driving lines 164 by a reference value (e.g., 1/2) or more, the position of the last driving line 164 where the touch is sensed may be determined as the boundary of the region the finger contacts on the touch panel 162. The secondary scanning region B may be determined from the first driving line 164 where the touch is sensed to the driving line 164 immediately before the last driving line 164 where the touch is sensed.
  • In the example illustrated in FIG. 3, the touch may be sensed in the (n+1)th, (2n+1)th, and (3n+1)th driving lines 164. Then, the touch screen controller 161 compares a capacitance variation in the (3n+1)th driving line 164 with a capacitance variation in the (n+1)th and (2n+1)th driving lines 164. If the capacitance variation in the (3n+1)th driving line 164 is nearly similar to the capacitance variation in the (n+1)th and (2n+1)th driving lines 164, the secondary scanning region B may be determined from the (n+1)th driving line 164 to the 4nth driving line 164 as illustrated in FIG. 5. However, if the capacitance variation in the (3n+1)th driving line 164 is less than the capacitance variations in the (n+1)th driving line 164 and the (2n+1)th driving line 164 by a reference value or more (e.g., less than 1/2 of the capacitance variations in the (n+1)th driving line 164 and the (2n+1)th driving line 164), a position of the (3n+1)th driving line 164 may be determined to be in a boundary of a region the finger contacts on the touch panel 162, and secondary scanning may not be performed from the (3n+1)th driving line 164. That is, the secondary scanning region B may be determined from the (n+1)th driving line 164 to the 3nth driving line 164.
  • FIG. 8 is a graph illustrating a driving signal applied to the driving lines 164 in the state of FIG. 7. Referring to FIG. 8, the secondary scanning is performed from the (n+1)th driving line 164 to the 3nth driving line 164. Then, the number of driving lines 164 in the secondary scanning region B is reduced, thereby more rapidly completing secondary scanning. Hence, a time for fingerprint recognition may be further reduced.
  • FIG. 9 is a diagram illustrating a process of searching for a region where a finger is situated on the combined fingerprint recognition touch screen device 160, according to another exemplary embodiment. In the example illustrated in FIG. 3, the plurality of driving lines 164 and the plurality of sensing lines 166 are arranged uniformly over the entire region of the touch panel 162. However, as described before, if there are excessively many driving lines 164 and sensing lines 166, much time is used for touch sensing and fingerprint recognition, and a manufacturing cost and power consumption for the combined fingerprint recognition touch screen device 160 may also increase. When fingerprint recognition is performed, fingerprint recognition may be possible merely with a partial region of a fingerprint rather than the entire region of the fingerprint.
  • Thus, as shown in FIG. 9, the driving lines 164 and the sensing lines 166 may be distributed in the form of an island on the touch panel 162. For example, the plurality of driving lines 164 may be arranged along a plurality of driving island sets 164a, 164b, 164c, and 164d, and the plurality of sensing lines 166 may be arranged along a plurality of sensing island sets 166a, 166b, 166c, and 166d. The plurality of driving lines 164 are arranged in each of the driving island sets 164a, 164b, 164c, and 164d, and the plurality of sensing lines 166 are arranged in each of the sensing island sets 166a, 166b, 166c, and 166d. For example, one driving island set 164a, 164b, 164c, or 164d and one sensing island set 166a, 166b, 166c, or 166d may have a width of about 4 mm to about 5 mm. A pitch between the driving lines 164 may be about 70 µm, a pitch between the sensing lines 166 may be about 70 µm, and about 60 to 70 driving lines and sensing lines 166 may be arranged in one driving island set 164a, 164b, 164c, or 164d and one sensing island set 166a, 166b, 166c, or 166d, respectively. Each of an interval between adjacent driving island sets 164a, 164b, 164c, and 164d and an interval between adjacent sensing island sets 166a, 166b, 166c, and 166d may be about 4 mm to about 5 mm.
  • When primary scanning is performed, a driving signal is sequentially applied to the first driving lines 164 among the driving island sets 164a, 164b, 164c, and 164d as indicated by solid lines in FIG. 9. For example, after a driving signal is applied to the first driving line 164 in the first driving island set 164a, a driving signal may be applied to the first driving line 164 in the second driving island set 164b, and a driving signal may be applied to the first driving line 164 in the third driving island set 164c. In this way, rough scanning may be performed over the entire region of the touch panel 162. However, the above description is merely an example, and another one may be selected from each driving island set 164a, 164b, 164c, or 164d to apply a driving signal to the selected driving line.
  • After primary scanning is finished, secondary scanning may be performed precisely with respect to the recognized region where the finger is situated. In the example illustrated in FIG. 9, assuming that the region marked with "A" is a region A the finger contacts on the touch panel 162, a touch is sensed in the second driving island set 164b and the third driving island set 164c. Then, in secondary scanning, a driving signal is sequentially applied to all the driving lines 164 of the second driving island set 164b and the third driving island set 164c.
  • FIG. 10 is a diagram illustrating a process of performing fingerprint recognition with respect to a region where a finger is situated in the combined fingerprint recognition touch screen device 160, according to another exemplary embodiment. As indicated with thick solid lines in FIG. 10, the driver 163 sequentially applies a driving signal to the driving lines 164 of the second driving island set 164b and the third driving island set 164c.
  • In this case, the secondary scanning region B may include the second driving island set 164b and the third driving island set 164c. That is, the second driving island set 164b and the third driving island set 164c including the driving lines 164 in which a touch is sensed are determined as the secondary scanning region B, and secondary scanning is performed with respect to all the driving lines 164 of the second driving island set 164b and the third driving island set 164c. Because the plurality of driving lines 164 and the plurality of sensing lines 166 are distributed along the plurality of driving island sets 164a, 164b, 164c, and 164d and the plurality of sensing island sets 166a, 166b, 166c, and 166d that are spaced apart from each other, a fingerprint image may be substantially obtained only in regions B1, B2, B3, and B4 of the secondary scanning region B where the second and third driving island sets 164b and 164c and the second and third sensing island sets 166b and 166c intersect each other. Fingerprint recognition may be sufficiently possible from fingerprint images obtained from the partial regions B1, B2, B3, and B4.
  • As in the example illustrated in FIG. 8, in FIG. 10, among the second and third driving island sets 164b and 164c where a touch is sensed, a capacitance variation in the last third driving island set 164c is compared with a capacitance variation in the second driving island set 164b. If a capacitance change in the third driving island set 164c is much smaller than a capacitance change in the second driving island set 164b, secondary scanning may be performed only for the second driving island set 164b except for the third driving island set 164c. That is, the secondary scanning region B may be determined as the second driving island set 164b.
  • The foregoing description has been made of a case in which primary scanning and secondary scanning are performed only for the driving lines 164. The signal output collector 165 receives output signals from the plurality of sensing lines 166 at the same time, and thus for the sensing lines 166, scanning of the same type as for the driving lines 164 is not necessary. However, if scanning of the same type as for the driving lines 164 is performed for outputting a touch sensing signal, primary scanning and secondary scanning may be performed with respect to the sensing lines 166 in the same manner as described for the driving lines 164.
  • The combined fingerprint recognition touch screen device 160 is applicable to various electric devices 100, e.g., smartphones, smart watches, etc. The touch panel 162 of the combined fingerprint recognition touch screen device 160 may be mounted on the display panel 126 of the electronic device 100. The electronic device 100 including the disclosed combined fingerprint recognition touch screen device 160 then may perform fingerprint recognition for user authentication merely with a user's touch on the display panel 126 by a finger. Therefore, there is no need to mount a separate fingerprint sensor, saving a space for the fingerprint sensor.
  • FIG. 11 is a flowchart illustrating a process of recognizing a fingerprint and performing user authentication in the electronic device 100 including the combined fingerprint recognition touch screen device 160, according to an exemplary embodiment. Referring to FIG. 11, in operation S10, the electronic device 100 starts a fingerprint recognition mode. If user authentication is performed in the electronic device 100 when the user presets user authentication to be performed with fingerprint recognition, the fingerprint recognition mode may start. For example, if user authentication is performed for unlocking a screen in a screen lock state of the electronic device 100, the fingerprint recognition mode may start. When user authentication is performed for performing online banking, online payment, or the like by using the electronic device 100, the fingerprint recognition mode may start.
  • In operation S11, the central processing unit 110 executes, for example, the fingerprint recognition module 153 that then performs primary scanning by controlling the driver 163 and the signal output collector 165 through the touch screen controller 161. For example, as described with reference to FIG. 3 or 9, a driving signal may be applied to only some of the plurality of driving lines 164, thereby rapidly and roughly performing primary scanning with respect to the entire region of the touch panel 162.
  • In operation S12, the electronic device 100 determines whether there is a touch by a finger for fingerprint recognition on the touch panel 162 of the electronic device 100, i.e., whether the touched region is maintained in the same position on the touch panel 162 for a first predetermined time or longer. If so, it may be determined that there is a contact by the finger for fingerprint recognition. For example, if the touched region is maintained in the same position for at least 100 ms, it may be determined that there is a contact by the finger.
  • If it is determined that there is the contact by the finger for fingerprint recognition on the touch panel 162, in operation S13, the fingerprint recognition module 153 performs secondary scanning by controlling the driver 163 and the signal output collector 165 through the touch screen controller 161. Secondary scanning may be performed by sequentially applying a driving voltage to all the driving lines 164 of the secondary scanning region B determined in the primary scanning operation S11. For example, secondary scanning may be performed in the manner described with reference to FIG. 5, 8, or 10. Through such secondary scanning, a high-resolution image for the touched region may be obtained.
  • In operation S14, the fingerprint recognition module 153 compares the image obtained in the secondary scanning operation S13 with a user's fingerprint image previously registered in the electronic device 100 . If the image obtained in the secondary scanning operation S13 completely matches the user's fingerprint image, in operation S15, the electronic device 100 determines that user authentication is successful. If user authentication is successful, the screen of the electronic device 100 may be unlocked, or online banking or online payment may be performed normally. However, if the image obtained in the secondary scanning operation S13 does not match the user's fingerprint image, in operation S16, the electronic device 100 determines whether user authentication fails consecutive times or more, e.g., consecutive five times.
  • If user authentication fails consecutive times or more, in operation S18, an alert message may be displayed on the display panel 126 of the electronic device 100, and the fingerprint recognition mode ends. In operation S19, the process returns to a touch sensing mode in which only primary scanning is repeatedly performed. If user authentication does not fail consecutive times or more, the process returns to operation S11.
  • If the touched region on the touch panel 162 moves instead of being maintained in the same position, or the touch is repeated at short intervals, and thus the touch is not regarded as an effective touch for fingerprint recognition in operation S12, then the secondary scanning operation S13 is not performed. Instead, in operation S17, the electronic device 100 determines whether there is no effective touch for a second predetermined time or longer. If there is no effective touch for the second predetermined time or longer, for example, if there is no effective touch for fingerprint recognition for one second, two seconds, or longer depending on a user's setting, in operation S18, an alert message may be displayed on the display panel 126 of the electronic device 100, and the fingerprint recognition mode ends. In operation S19, the process returns to a touch sensing mode in which only primary scanning is repeatedly performed. If there is an effective touch for the second predetermined time, the process returns to operation S11.
  • In addition, the exemplary embodiments may also be implemented through computer-readable code and/or instructions on a medium, e.g., a computer-readable medium, to control at least one processing element to implement any of the above-described exemplary embodiments. The medium may correspond to any medium or media that may serve as a storage and/or perform transmission of the computer-readable code.
  • The computer-readable code may be recorded and/or transferred on a medium in a variety of ways, and examples of the medium include recording media, such as magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) and optical recording media (e.g., compact disc read only memories (CD-ROMs) or digital versatile discs (DVDs)), and transmission media such as Internet transmission media. Thus, the medium may have a structure suitable for storing or carrying a signal or information, such as a device carrying a bitstream according to exemplary embodiments. The medium may also be on a distributed network, so that the computer-readable code is stored and/or transferred on the medium and executed in a distributed fashion. Furthermore, the processing element may include a processor or a computer processor, and the processing element may be distributed and/or included in a single device.
  • The foregoing exemplary embodiments are examples and are not to be construed as limiting. The present teaching can be readily applied to other types of apparatuses. Also, the description of the exemplary embodiments is intended to be illustrative, and not to limit the scope of the claims, and many alternatives, modifications, and variations will be apparent to those skilled in the art.

Claims (15)

  1. A touch screen device comprising:
    a touch pad comprising driving lines and sensing lines intersecting the driving lines;
    a driver configured to apply a driving signal to the driving lines;
    a signal output collector configured to receive a touch signal from the sensing lines; and
    a controller configured to control the driver and the signal output collector,
    wherein the controller is further configured to:
    control the driver to perform primary scanning to sequentially apply the driving signal to first driving lines disposed at predetermined intervals, among the driving lines;
    determine a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied; and
    control the driver to perform secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  2. The touch screen device of claim 1, wherein each of the predetermined intervals is of n driving lines,
    the predetermined intervals are over an entire region of the touch pad, and
    n is a natural number that is greater than 1.
  3. The touch screen device of claim 1 or 2, wherein the controller is further configured to control the driver to perform the primary scanning to sequentially apply the driving signal to (kn+m)th driving lines at the predetermined intervals, starting from an mth driving line, among the driving lines,
    n is a natural number that is greater than 1,
    m is a natural number that is greater than or equal to 1, and
    k is an integer that is greater than or equal to 0.
  4. The touch screen device of any preceding claim, wherein the determined scanning region is from the first driving line where the touch is sensed to a second driving line immediately before a third driving line where the touch is not sensed after the first driving line, among the first driving lines.
  5. The touch screen device of any preceding claim, wherein the controller is further configured to determine whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more, and
    the determined scanning region is from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the the determination that the first contact strength is less than the second contact strength by the reference value or more, and optionally wherein the controller is further configured to determine the first contact strength and the second contact strength from capacitance variations in the respective third driving lines.
  6. The touch screen device of any preceding claim, wherein the touch panel further comprises driving island sets that are spaced apart from each other, and
    each of the driving island sets comprises a respective portion of the driving lines, and preferably wherein a width of each of the driving island sets is 4 mm to 5 mm, and
    an interval between each adjacent pair of the driving island sets is 4 mm to 5 mm.
  7. The touch screen device of claim 6, wherein the controller is further configured to perform the primary scanning to sequentially apply the driving signal to one driving line among each of the driving island sets, and optionally wherein the controller is further configured to determine the scanning region as a driving island set comprising third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  8. The touch screen device of any preceding claim, wherein a pitch between the driving lines is 70 µm.
  9. The touch screen device of any preceding claim, wherein the controller is further configured to:
    operate in a fingerprint recognition mode for recognizing a fingerprint or in a touch sensing mode for sensing a touch; and
    repeat the primary scanning in the touch sensing mode, and optionally wherein the controller is further configured to, in the fingerprint recognition mode, after the primary scanning:
    determine whether a region that is touched on the touch pad is maintained in a same position on the touch pad for a period of time or longer; and
    perform the secondary scanning in response to the determination that the touched region touched is maintained in the same position on the touch pad for the period of time or longer.
  10. An electronic device comprising:
    a display panel; and
    a touch screen device according to any preceding claim.
  11. A method of driving a touch screen device comprising a touch pad comprising driving lines and sensing lines intersecting the sensing lines, the method comprising:
    performing primary scanning to sequentially apply a driving signal to first driving lines disposed at predetermined intervals, among the driving lines;
    determining a scanning region, based on a first driving line where a touch is sensed among the first driving lines to which the driving signal is applied; and
    performing secondary scanning to sequentially apply the driving signal to second driving lines included in the determined scanning region, among the driving lines.
  12. The method of claim 11, wherein the performing of the primary scanning further comprises sequentially applying the driving signal to (kn+m)th driving lines at the predetermined intervals, starting from an mth driving line, among the driving lines,
    n is a natural number that is greater than 1,
    m is a natural number that is greater than or equal to 1, and
    k is an integer that is greater than or equal to 0.
  13. The method of claim 11 or 12, further comprising determining whether a first contact strength in a last driving line disposed in a last position among third driving lines where the touch is sensed among the first driving lines, is less than a second contact strength in fourth driving lines among the third driving lines, by a reference value or more,
    wherein the determined scanning region is from the first driving line where the touch is sensed to a second driving line immediately before the last driving line where the touch is sensed, in response to the the determination that the first contact strength is less than the second contact strength by the reference value or more.
  14. The method of any of claims 11 to 13, wherein the touch panel further comprises driving island sets that are spaced apart from each other,
    each of the driving island sets comprises a respective portion of the driving lines,
    the performing of the primary scanning comprises sequentially applying the driving signal to one driving line among each of the driving island sets, and
    the determining of the scanning region comprises determining the scanning region as a driving island set comprising third driving lines where the touch is sensed, among the driving island sets to which the driving signal is applied.
  15. A computer program comprising computer program code means adapted to perform all of the steps of any of claims 11 to 14 when said program is run on a computer.
EP17151846.7A 2016-05-17 2017-01-17 Combined fingerprint recognition touch screen device, method of driving the touch screen device, and electronic device including the touch screen device Active EP3246801B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160060212A KR102558004B1 (en) 2016-05-17 2016-05-17 Touch screen apparatus capable of touch sensing and fingerprint sensing and method of driving the touch screen apparatus, and electronic apparatus including the same

Publications (2)

Publication Number Publication Date
EP3246801A1 true EP3246801A1 (en) 2017-11-22
EP3246801B1 EP3246801B1 (en) 2019-04-17

Family

ID=57850923

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17151846.7A Active EP3246801B1 (en) 2016-05-17 2017-01-17 Combined fingerprint recognition touch screen device, method of driving the touch screen device, and electronic device including the touch screen device

Country Status (4)

Country Link
US (1) US10345950B2 (en)
EP (1) EP3246801B1 (en)
KR (1) KR102558004B1 (en)
CN (1) CN107402661B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2559846A (en) * 2016-12-16 2018-08-22 Lg Display Co Ltd Display device, display panel, fingerprint-sensing method, and circuit for sensing fingerprint
EP3832446A1 (en) * 2019-12-02 2021-06-09 Beijing Xiaomi Mobile Software Co., Ltd. Method and device for fingerprint acquisition, and touchpad

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102384014B1 (en) * 2017-03-14 2022-04-08 삼성전자주식회사 Method and electronic device for detecting fingerprint
KR102364855B1 (en) 2017-07-26 2022-02-18 삼성전자주식회사 Fingerprint/touch sensor and electronic apparatus including the same
US11092998B1 (en) * 2018-01-12 2021-08-17 Snap Inc. Eyewear device with fingerprint sensor for user input
KR102507281B1 (en) 2018-03-23 2023-03-08 삼성전자주식회사 Electornic device comprising fingerprint sensor and method for operating thereof
CN108647569B (en) 2018-03-30 2019-11-19 维沃移动通信有限公司 A kind of finger scan method, mobile terminal
US11106883B2 (en) 2018-08-02 2021-08-31 Samsung Electronics Co., Ltd. Method and apparatus for processing data corresponding to fingerprint image
KR102581472B1 (en) 2018-08-10 2023-09-21 삼성전자주식회사 Touch-fingerprint complex sensor and electronic apparatus including the same
CN111259690B (en) * 2018-11-30 2024-01-19 上海耕岩智能科技有限公司 Biological feature recognition driving method, storage medium and apparatus
KR20200073696A (en) 2018-12-14 2020-06-24 삼성전자주식회사 Method for driving fingerprint sensing system and fingerprint sensing system
TWI757198B (en) 2018-12-27 2022-03-01 聯詠科技股份有限公司 Electronic device, fingerprint sensing control method and fingerprint scanning control method
CN111209889A (en) * 2019-05-16 2020-05-29 神盾股份有限公司 Fingerprint sensor
EP3980875A1 (en) * 2019-06-05 2022-04-13 Touch Biometrix Limited Pointing device with biometric skin-contact sensor, and corresponding method
KR20210068769A (en) * 2019-12-02 2021-06-10 엘지디스플레이 주식회사 Ultrasonic sensor and display device
KR20210069483A (en) 2019-12-03 2021-06-11 삼성전자주식회사 Combined fingerprint recognition touch sensor, electronic apparatus including the same, method for enrolling and recognizing fingerprint employing the same
JP2022115333A (en) * 2021-01-28 2022-08-09 株式会社ジャパンディスプレイ detector

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110074731A1 (en) * 2009-09-30 2011-03-31 Fujitsu Limited Input device, input processing program, and input control method
EP2629183A2 (en) * 2012-02-16 2013-08-21 Samsung Display Co., Ltd. Method of operating a touch panel, touch panel and display device
EP2772838A1 (en) * 2013-02-27 2014-09-03 Samsung Display Co., Ltd. Display device comprising a touch panel and method of driving the same
US20140341446A1 (en) * 2013-05-16 2014-11-20 International Business Machines Corporation Utilizing a touch screen as a biometric device
US20150077370A1 (en) * 2013-09-17 2015-03-19 Lg Display Co., Ltd. Display device integrated with touch screen and driving method thereof
US20150109217A1 (en) * 2013-10-21 2015-04-23 Tianma Micro-Electronics Co., Ltd. Touch scanning method for touch screen, touch scanning control circuit and display device

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5963679A (en) 1996-01-26 1999-10-05 Harris Corporation Electric field fingerprint sensor apparatus and related methods
US20110032206A1 (en) * 2008-04-24 2011-02-10 Kyocera Corporation Mobile electronic device
US8866347B2 (en) 2010-01-15 2014-10-21 Idex Asa Biometric image sensing
US20130265282A1 (en) * 2010-12-08 2013-10-10 Sharp Kabushiki Kaisha Touch panel and display device with touch panel
KR20120119966A (en) * 2011-04-21 2012-11-01 신텐 산교 가부시키가이샤 Authentication device and authentication method of portable information terminal
JP2012226720A (en) * 2011-04-21 2012-11-15 Shinten Sangyo Co Ltd Authentication apparatus for personal digital assistant and authentication method thereof
US20130026528A1 (en) * 2011-07-27 2013-01-31 Safety Traffic Equipment Co., Ltd. Waterproof transparent led shield structure
KR101398253B1 (en) * 2011-12-02 2014-05-23 엘지디스플레이 주식회사 Apparatus and method for driving touch screen
US9886116B2 (en) * 2012-07-26 2018-02-06 Apple Inc. Gesture and touch input detection through force sensing
US10203816B2 (en) 2013-05-07 2019-02-12 Egis Technology Inc. Apparatus and method for TFT fingerprint sensor
US9261991B2 (en) * 2013-05-28 2016-02-16 Google Technology Holdings LLC Multi-layered sensing with multiple resolutions
KR102140197B1 (en) * 2013-10-23 2020-08-03 삼성디스플레이 주식회사 Capacitive type touch sensor
KR101432988B1 (en) * 2014-04-02 2014-08-29 (주)이미지스테크놀로지 A capacitive touch screen for integrated of fingerprint recognition
KR102212632B1 (en) * 2014-05-12 2021-02-08 삼성전자주식회사 Fingerprint Recognition method and electronic device performing thereof
KR20150139104A (en) * 2014-06-02 2015-12-11 한국전자통신연구원 Touch screen panel
KR102236279B1 (en) 2014-06-17 2021-04-02 엘지전자 주식회사 Mobile terminal and method for controlling the same
KR20150087802A (en) 2015-01-16 2015-07-30 크루셜텍 (주) Electronic device for recognizing fingerprints and multi-touching
CN104966061B (en) * 2015-06-16 2018-07-10 京东方科技集团股份有限公司 A kind of palm grain identification method and device, intelligent safety door

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110074731A1 (en) * 2009-09-30 2011-03-31 Fujitsu Limited Input device, input processing program, and input control method
EP2629183A2 (en) * 2012-02-16 2013-08-21 Samsung Display Co., Ltd. Method of operating a touch panel, touch panel and display device
EP2772838A1 (en) * 2013-02-27 2014-09-03 Samsung Display Co., Ltd. Display device comprising a touch panel and method of driving the same
US20140341446A1 (en) * 2013-05-16 2014-11-20 International Business Machines Corporation Utilizing a touch screen as a biometric device
US20150077370A1 (en) * 2013-09-17 2015-03-19 Lg Display Co., Ltd. Display device integrated with touch screen and driving method thereof
US20150109217A1 (en) * 2013-10-21 2015-04-23 Tianma Micro-Electronics Co., Ltd. Touch scanning method for touch screen, touch scanning control circuit and display device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHRIS STEIN ET AL: "Fingerphoto recognition with smartphone cameras", BIOMETRICS SPECIAL INTEREST GROUP (BIOSIG), 2012 BIOSIG - PROCEEDINGS OF THE INTERNATIONAL CONFERENCE OF THE, IEEE, 6 September 2012 (2012-09-06), pages 1 - 12, XP032243117, ISBN: 978-1-4673-1010-9 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2559846A (en) * 2016-12-16 2018-08-22 Lg Display Co Ltd Display device, display panel, fingerprint-sensing method, and circuit for sensing fingerprint
US10664677B2 (en) 2016-12-16 2020-05-26 Lg Display Co., Ltd. Display device, display panel, fingerprint-sensing method, and circuit for sensing fingerprint
GB2559846B (en) * 2016-12-16 2020-09-09 Lg Display Co Ltd Display device, display panel, fingerprint-sensing method, and circuit for sensing fingerprint
DE102017130321B4 (en) 2016-12-16 2024-06-27 Lg Display Co., Ltd. Display device, display panel, fingerprint detection method and circuit for detecting a fingerprint
EP3832446A1 (en) * 2019-12-02 2021-06-09 Beijing Xiaomi Mobile Software Co., Ltd. Method and device for fingerprint acquisition, and touchpad
US11455819B2 (en) 2019-12-02 2022-09-27 Beijing Xiaomi Mobile Software Co., Ltd. Method and device for fingerprint acquisition, and touchpad

Also Published As

Publication number Publication date
US10345950B2 (en) 2019-07-09
CN107402661A (en) 2017-11-28
KR20170129476A (en) 2017-11-27
KR102558004B1 (en) 2023-07-20
US20170336906A1 (en) 2017-11-23
CN107402661B (en) 2022-03-29
EP3246801B1 (en) 2019-04-17

Similar Documents

Publication Publication Date Title
EP3246801B1 (en) Combined fingerprint recognition touch screen device, method of driving the touch screen device, and electronic device including the touch screen device
US10936100B2 (en) Electronic device including flexible display capable of changing size of display area and method for controlling same
CN105704297B (en) Mobile terminal and control method thereof
KR102135160B1 (en) Information processing apparatus, information processing method and storage medium
KR101695818B1 (en) Mobile terminal and Method for controlling virtual key pad thereof
EP3989043A1 (en) Device and method for activating with voice input
US20200160025A1 (en) Electronic Device
EP3144784B1 (en) Mobile terminal and control method for the mobile terminal
EP3444752A2 (en) Fingerprint recognition process
CN112567321A (en) Electronic device comprising an electronic pen and method of controlling a communication connection between an electronic device and an electronic pen
EP3407256A1 (en) Recognizing biological feature
CN107305605B (en) Terminal equipment and application starting method
US10827424B2 (en) Mobile terminal and method for controlling the same
JP6719995B2 (en) Electronic device, control method, and program
US10162898B2 (en) Method and apparatus for searching
CN114489449B (en) Display method and electronic equipment
CN108182020A (en) screen display processing method, device and storage medium
CN110889102B (en) Image unlocking method and device, computer readable storage medium and terminal
CN106227505A (en) Image detecting method, device and the device for image detection
KR101862216B1 (en) Mobile terminal and control method therof
CN106780513B (en) The method and apparatus of picture conspicuousness detection
KR101292050B1 (en) Mobile terminal and method of controlling operation thereof
US20240028768A1 (en) Controlling access to restricted and unrestricted software functionality
US20240078847A1 (en) Controlling an active fingerprint sensor area
US10908816B2 (en) Electronic device for providing predictive word and operating method thereof

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180522

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 3/044 20060101AFI20181120BHEP

Ipc: G06F 3/041 20060101ALI20181120BHEP

INTG Intention to grant announced

Effective date: 20181217

RIN1 Information on inventor provided before grant (corrected)

Inventor name: YOON, DAEKUN

Inventor name: LEE, BYUNGKYU

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017003261

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1122293

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190515

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190417

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190717

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190718

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190717

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1122293

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190417

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190817

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017003261

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

26N No opposition filed

Effective date: 20200120

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200131

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200131

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190417

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230520

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231214

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231222

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20231220

Year of fee payment: 8