WO2022126653A1 - 心率检测装置及方法、电子设备 - Google Patents

心率检测装置及方法、电子设备 Download PDF

Info

Publication number
WO2022126653A1
WO2022126653A1 PCT/CN2020/137769 CN2020137769W WO2022126653A1 WO 2022126653 A1 WO2022126653 A1 WO 2022126653A1 CN 2020137769 W CN2020137769 W CN 2020137769W WO 2022126653 A1 WO2022126653 A1 WO 2022126653A1
Authority
WO
WIPO (PCT)
Prior art keywords
fingerprint
frame
heart rate
fingerprint image
display screen
Prior art date
Application number
PCT/CN2020/137769
Other languages
English (en)
French (fr)
Inventor
杨小强
青小刚
张珂
Original Assignee
深圳市汇顶科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市汇顶科技股份有限公司 filed Critical 深圳市汇顶科技股份有限公司
Priority to PCT/CN2020/137769 priority Critical patent/WO2022126653A1/zh
Publication of WO2022126653A1 publication Critical patent/WO2022126653A1/zh

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate

Definitions

  • the embodiments of the present application relate to the field of biometric identification, and more particularly, to a heart rate detection apparatus and method, and electronic equipment.
  • Under-screen fingerprint recognition devices have also become a trend, and are gradually popularized in smart terminals.
  • the under-screen fingerprint recognition devices are based on optical system imaging, making In addition to acquiring fingerprint signals, there are more possibilities.
  • Embodiments of the present application provide a heart rate detection apparatus and method, and electronic equipment, which can realize heart rate detection based on an under-screen optical fingerprint system.
  • a heart rate detection device which is applied to an electronic device with a display screen.
  • the heart rate detection device includes: a photosensitive component disposed below the display screen, and the photosensitive component is used to collect user-generated information from the display screen.
  • the light signal reflected or scattered by the finger to obtain a multi-frame fingerprint image
  • the multi-frame fingerprint image is used to obtain a photoplethysmography PPG signal
  • the standard deviation of the interval is used to obtain HRV information of the user's heart rate variability; wherein, the data amount of each frame of fingerprint images in the multi-frame fingerprint images is smaller than the data amount of fingerprint images used for fingerprint identification.
  • the photosensitive component is a fingerprint sensor chip
  • the fingerprint sensor chip is used for: continuously collecting multiple frames of the first fingerprint image, and performing each frame of the first fingerprint image in the multiple frames of the first fingerprint image. Compression processing is performed to obtain multiple frames of second fingerprint images respectively, and the multiple frames of second fingerprint images are used to obtain the PPG signal.
  • the fingerprint sensor chip is configured to: divide the first fingerprint image of each frame into at least one sub-region; each sub-region in the at least one sub-region of the first fingerprint image of each frame includes The plurality of pixel data are processed into one data, so as to obtain the second fingerprint images of the plurality of frames respectively.
  • the fingerprint sensor chip is used for: performing summation processing or averaging processing on a plurality of pixel data included in each sub-area.
  • the fingerprint sensor chip is configured to: acquire the multiple frames of the second fingerprint image according to part of the pixel data in the first fingerprint image of each frame.
  • the photosensitive component is formed by splicing at least part of the pixels in the fingerprint sensor chip in the electronic device.
  • the photosensitive component is provided independently of the fingerprint sensor chip in the electronic device.
  • the initial moment of the exposure time of the first frame of fingerprint images in the multi-frame fingerprint images is synchronized with the initial moment of the refresh period of the display screen.
  • the exposure time of each frame of fingerprint images in the multi-frame fingerprint images is an integer multiple of the refresh period of the display screen.
  • the exposure time of each frame of fingerprint image is equal to the refresh period of the display screen.
  • the exposure time of each frame of fingerprint images in the multi-frame fingerprint images is a non-integer multiple of the refresh period of the display screen
  • the heart rate detection device further includes: a processor, configured to Exposure energy deviation sequence, performing energy compensation on the fingerprint image after the first frame fingerprint image in the multi-frame fingerprint image, and the exposure energy deviation sequence includes the difference between each frame fingerprint image after the first frame fingerprint image and the first frame fingerprint image. Energy deviation value.
  • the photosensitive component is used for acquiring the multi-frame fingerprint images in the non-highlight mode HBM mode of the display screen.
  • the sampling rate of the multi-frame fingerprint images is greater than or equal to 100 Hz.
  • a heart rate detection method which is applied to an electronic device with a display screen, the heart rate detection method comprising: obtaining the user's multiple data according to the light signal reflected or scattered by the user's finger and returned by the display screen. frame fingerprint image; extract the photoplethysmography PPG signal of each frame of fingerprint image in the multi-frame fingerprint image; The heart rate variability HRV information; wherein, the data volume of each fingerprint image in the multi-frame fingerprint image is smaller than the data volume of the fingerprint image used for fingerprint identification.
  • acquiring the multi-frame fingerprint images of the user includes: continuously collecting the multi-frame first fingerprint images of the user; Compression processing is performed to obtain multiple frames of second fingerprint images respectively, and the multiple frames of second fingerprint images are used to obtain the PPG signal.
  • performing compression processing on each frame of the first fingerprint image in the multiple frames of the first fingerprint image to obtain the multiple frames of the second fingerprint image includes: compressing each frame of the first fingerprint image Dividing into at least one sub-area; processing multiple pixel data included in each sub-area in the at least one sub-area of each frame of the first fingerprint image into one data, so as to obtain the multiple frames of the second fingerprint image respectively.
  • the processing of multiple pixel data included in each sub-region in the at least one sub-region into one piece of data includes: performing a summation process on the multiple pixel data included in each sub-region or Find the average treatment.
  • compressing each frame of the first fingerprint image in the multiple frames of the first fingerprint image to obtain the multiple frames of the second fingerprint image includes: according to the first fingerprint image of each frame Part of the pixel data in the multi-frame second fingerprint image is obtained.
  • the initial moment of the exposure time of the first frame of fingerprint images in the multi-frame fingerprint images is synchronized with the initial moment of the refresh period of the display screen.
  • the exposure time of each frame of fingerprint images in the multi-frame fingerprint images is an integer multiple of the refresh period of the display screen.
  • the exposure time of each frame of fingerprint image is equal to the refresh period of the display screen.
  • the exposure time of each frame of fingerprint images in the multi-frame fingerprint images is a non-integer multiple of the refresh period of the display screen
  • the heart rate detection method further includes: according to a preset exposure energy deviation sequence, Energy compensation is performed on the fingerprint images after the first frame fingerprint image in the multi-frame fingerprint images, and the exposure energy deviation sequence includes the energy deviation value between each fingerprint image after the first frame fingerprint image and the first frame fingerprint image.
  • the acquiring the multi-frame fingerprint images of the user includes: acquiring the multi-frame fingerprint images in a non-highlight mode HBM mode of the display screen.
  • the sampling rate of the multi-frame fingerprint images is greater than or equal to 100 Hz.
  • an electronic device including a display screen and the first aspect or the heart rate detection apparatus in any possible implementation manner of the first aspect.
  • the technical solutions of the embodiments of the present application acquire multiple frames of fingerprint images based on the optical signal emitted by the display screen, and perform heart rate detection, so that the heart rate detection under the screen can be realized. Further, by calculating the PPG signal of the multi-frame fingerprint image, and obtaining the user's HRV information based on the standard deviation of the adjacent peak interval or adjacent wave trough interval of the PPG signal in the time domain, the user's heart rate information can be obtained more accurately. .
  • FIG. 1A is an oriented view of an electronic device according to an embodiment of the present application.
  • Fig. 1B is a schematic diagram of a partial cross-sectional structure of the electronic device shown in Fig. 1A along A-A'.
  • FIG. 2 is a schematic block diagram of a heart rate detection apparatus according to an embodiment of the present application.
  • FIG. 3 is an amplitude-time schematic diagram of the conversion of an optical signal of a finger to an electrical signal.
  • Fig. 4 is a PPG signal diagram extracted from a fingerprint image reflecting heart rate information.
  • 5 to 8 are system block diagrams of sampling heart rate information according to an embodiment of the present application.
  • Figure 9 is a simplified model of the noise caused by the drop in brightness of the screen spot.
  • FIG. 10 is a schematic diagram illustrating that the initial moment of the exposure time of each frame of fingerprint image is synchronized with the initial moment of the refresh cycle of the display screen.
  • Figure 11 is a system diagram of the timing synchronization between the photosensitive component and the display screen.
  • FIG. 12 is a timing diagram of the refresh cycle N equally dividing the AC energy model.
  • FIG. 13 is a schematic block diagram of a heart rate detection method according to an embodiment of the present application.
  • portable or mobile computing devices such as smartphones, laptops, tablets, gaming devices, and other electronic devices such as electronic databases, automobiles, and bank automated teller machines (ATMs).
  • ATMs bank automated teller machines
  • the embodiments of the present application do not limit this.
  • the technical solutions of the embodiments of the present application can be used for the under-screen fingerprint identification technology.
  • the under-screen fingerprint recognition technology refers to installing the fingerprint recognition module under the display screen, so as to realize the fingerprint recognition operation in the display area of the display screen, and it is not necessary to set a fingerprint collection area on the front of the electronic device except the display area.
  • the fingerprint recognition module uses light returned from the top surface of the display assembly of the electronic device for fingerprint sensing and other sensing operations. This returned light carries the information of the object (such as a finger) in contact with the top surface of the display assembly, and the fingerprint recognition module located under the display assembly realizes off-screen fingerprint recognition by collecting and detecting this returned light.
  • the fingerprint recognition module can be designed to achieve the desired optical imaging by properly configuring the optical elements for collecting and detecting the returned light.
  • FIG. 1A and 1B show schematic diagrams of an electronic device 100 to which the under-screen fingerprint recognition technology can be applied, wherein FIG. 1A is a schematic front view of the electronic device 100 , and FIG. 1B is a schematic view of the electronic device 100 shown in FIG. 1A along AA' Schematic diagram of part of the cross-section structure.
  • the electronic device 100 may include a display screen 120 and an optical fingerprint device 140 .
  • the display screen 120 may be a self-luminous display screen, which uses display units having self-luminescence as display pixels.
  • the display screen 120 may be an organic light-emitting diode (Organic Light-Emitting Diode, OLED) display screen or a micro light-emitting diode (Micro-LED) display screen.
  • the display screen 120 may also be a liquid crystal display (Liquid Crystal Display, LCD) or other passive light-emitting display screens, which are not limited in this embodiment of the present application.
  • the display screen 120 can also be specifically a touch display screen, which can not only display a picture, but also detect a user's touch or pressing operation, thereby providing a human-computer interaction interface for the user.
  • the electronic device 100 may include a touch sensor, and the touch sensor may specifically be a touch panel (Touch Panel, TP), which may be disposed on the surface of the display screen 120, or may be partially integrated or The whole is integrated into the display screen 120 to form the touch display screen.
  • Touch Panel Touch Panel
  • the optical fingerprint device 140 may include a fingerprint sensor chip having an optical sensing array (hereinafter also referred to as an optical fingerprint sensor or an optical fingerprint chip).
  • the optical sensing array includes a plurality of optical sensing units, and each optical sensing unit may specifically include a photodetector or a photoelectric sensor.
  • the optical fingerprint device 140 may include a photodetector (PD) array (or referred to as a photodetector array, a photosensor array, an optical sensor array, a sensing array, and a pixel array), which includes a plurality of arrays in an array. distributed light detectors.
  • PD photodetector
  • the optical fingerprint device 140 may be disposed in a partial area below the display screen 120 , so that the fingerprint collection area (or fingerprint detection area) 130 of the optical fingerprint device 140 is at least partially located on the display screen 120 . in the display area 102 .
  • the optical fingerprint device 140 may also be disposed at other positions, such as the side surface of the display screen 120 or the non-light-transmitting area of the edge of the electronic device 100 .
  • the optical signal of at least part of the display area of the display screen 120 can be guided to the optical fingerprint device 140 through the optical path design, so that the fingerprint collection area 130 is actually located in the display area of the display screen 120.
  • the optical fingerprint device 140 may only include one fingerprint sensor chip.
  • the fingerprint collection area 130 of the optical fingerprint device 140 has a small area and a fixed position. Therefore, the user needs to put his finger on the finger when inputting a fingerprint. Press to a specific position of the fingerprint collection area 130, otherwise the optical fingerprint device 140 may not be able to collect the fingerprint image, resulting in poor user experience.
  • the optical fingerprint device 140 may specifically include multiple fingerprint sensor chips; the multiple fingerprint sensor chips may be arranged side by side under the display screen 120 by splicing, and the multiple fingerprint sensor chips
  • the sensing areas of the fingerprint sensor chip together constitute the fingerprint collection area 130 of the optical fingerprint device 140 .
  • the fingerprint collection area 130 of the optical fingerprint device 140 may include a plurality of sub-areas, each sub-area corresponds to the sensing area of one of the fingerprint sensor chips, so that the fingerprint collection area 130 of the optical fingerprint module 130 It can be extended to the main area of the lower half of the display screen, that is, to the area where the finger is habitually pressed, so as to realize the blind-pressing fingerprint input operation.
  • the fingerprint detection area 130 can also be extended to half the display area or even the entire display area, so as to realize half-screen or full-screen fingerprint detection.
  • the embodiments of the present application do not limit the specific forms of the plurality of fingerprint sensor chips.
  • the plurality of fingerprint sensor chips may be individually packaged fingerprint sensor chips, or may be multiple chips (Dies) packaged in the same chip package.
  • the plurality of fingerprint sensor chips can also be fabricated on different regions of the same chip (Die) through a semiconductor process.
  • the area where the optical sensing array of the optical fingerprint device 140 is located or the light sensing range corresponds to the fingerprint collection area 130 of the optical fingerprint device 140 .
  • the fingerprint collection area 130 of the optical fingerprint device 140 may or may not be equal to the area or light sensing range of the area where the optical sensing array of the optical fingerprint device 140 is located, which is not specifically limited in this embodiment of the present application.
  • the fingerprint collection area 130 of the optical fingerprint device 140 can be designed to be substantially the same as the area of the sensing array of the optical fingerprint device 140 .
  • the area of the fingerprint collection area 130 of the optical fingerprint device 140 can be made larger than the area of the sensing array of the optical fingerprint device 140 through the optical path design of converging light or the optical path design of reflected light.
  • the optical fingerprint device 140 may further include an optical component, and the optical component may be disposed above the sensing array, which may specifically include a filter layer (Filter), a light guide layer or an optical path Guide structures and other optical elements, the filter layer can be used to filter out ambient light that penetrates the finger, for example, infrared light that interferes with imaging, and the light guide layer or light path guide structure is mainly used to reflect back from the surface of the finger The reflected light is directed to the sensing array for optical detection.
  • a filter layer Finter
  • the light guide layer or light path guide structure is mainly used to reflect back from the surface of the finger The reflected light is directed to the sensing array for optical detection.
  • optical path design of the optical fingerprint device 140 is exemplarily described below.
  • the optical fingerprint device 140 may use an optical collimator having a through-hole array with a high aspect ratio, and the optical collimator may specifically be a collimator fabricated on a semiconductor silicon wafer.
  • layer which has a plurality of collimation units or micro-holes, the collimation units can be specifically small holes, and in the reflected light reflected from the finger, the light perpendicularly incident to the collimation unit can pass through and be passed by the lower part of the The light with an excessively large incident angle is attenuated after multiple reflections inside the collimation unit, so each fingerprint sensor chip can basically only receive the reflected light from the fingerprint lines directly above it. , which can effectively improve the image resolution, thereby improving the fingerprint recognition effect.
  • an alignment unit may be configured for one optical sensing unit in the optical sensing array of each fingerprint sensor chip, and the collimation unit may be attached to the corresponding optical sensing unit.
  • the multiple optical sensing units may also share one collimating unit, that is, the one collimating unit has an aperture large enough to cover the multiple optical sensing units. Since one collimation unit can correspond to multiple optical sensing units, the correspondence between the space period of the display screen 120 and the space period of the fingerprint sensor chip is destroyed.
  • the optical sensing array has a similar spatial structure, which can effectively prevent the optical fingerprint device 140 from using the optical signal passing through the display screen 120 to perform fingerprint imaging to generate Moiré fringes, thereby effectively improving the fingerprint recognition effect of the optical fingerprint device 140 .
  • the optical fingerprint device 140 may adopt an optical path design based on an optical lens
  • the optical lens may include an optical lens (Lens) layer having one or more lens units, such as one or more aspherical surfaces
  • a lens group composed of lenses is used for converging the reflected light from the finger to the sensing array of the fingerprint sensor chip below it, so that the sensing array can perform imaging based on the reflected light, so as to obtain the image of the finger.
  • Fingerprint image The optical lens layer can also be formed with pinholes in the optical path of the lens unit, and the pinholes can cooperate with the optical lens layer to expand the field of view of the optical fingerprint device 140 to improve the fingerprint of the optical fingerprint device 140 . Imaging effect.
  • each fingerprint sensor chip may be configured with an optical lens to perform fingerprint imaging, or multiple fingerprint sensor chips may be configured with one optical lens to achieve light convergence and fingerprint imaging. Even when a fingerprint sensor chip has two sensing arrays (Dual Array) or multiple sensing arrays (Multi-Array), the fingerprint sensor chip can also be configured with two or more optical lenses to cooperate with the two sensing arrays. The array or multiple sensing arrays perform optical imaging, thereby reducing the imaging distance and enhancing the imaging effect.
  • the optical fingerprint device 140 may adopt the optical path design of a micro-lens (Micro-Lens) layer, and the micro-lens layer may have a micro-lens array formed by a plurality of micro-lenses, which may be formed by a semiconductor growth process. Or other processes are formed above the sensing array of the fingerprint sensor chip, and each microlens may respectively correspond to one of the sensing units of the sensing array.
  • Other optical film layers such as a dielectric layer or a passivation layer, may also be formed between the microlens layer and the sensing unit.
  • the light-blocking layer wherein the micro-holes are formed between the corresponding micro-lenses and the sensing units, the light-blocking layer can block the optical interference between the adjacent micro-lenses and the sensing units, and allow the light to pass through the micro-lenses and the sensing units.
  • the lens converges inside the micro-hole and is transmitted to the sensing unit corresponding to the micro-lens via the micro-hole, so as to perform optical fingerprint imaging.
  • a microlens layer may be further provided under the collimator layer or the optical lens layer.
  • a microlens layer may be further provided under the collimator layer or the optical lens layer.
  • the collimator layer or the optical lens layer is used in combination with the microlens layer, its specific stack structure or optical path may need to be adjusted according to actual needs.
  • the optical fingerprint device 140 may be used to collect fingerprint information (such as fingerprint image information) of the user.
  • the display screen 120 may adopt a display screen having a self-luminous display unit, such as an organic light-emitting diode (Organic Light-Emitting Diode, OLED) display screen or a micro-light-emitting diode (Micro-LED) display screen Screen.
  • a self-luminous display unit such as an organic light-emitting diode (Organic Light-Emitting Diode, OLED) display screen or a micro-light-emitting diode (Micro-LED) display screen Screen.
  • OLED Organic Light-Emitting Diode
  • Micro-LED micro-light-emitting diode
  • the display screen 120 When a finger touches, presses or approaches (for the convenience of description, collectively referred to as pressing in this application) on the fingerprint collection area 130, the display screen 120 emits a beam of light to the finger above the fingerprint collection area 130, and this beam of light is on the finger's surface. The surface is reflected to form reflected light, or scattered light is formed by internal scattering of the finger. In related patent applications, for the convenience of description, the above-mentioned reflected light and scattered light are collectively referred to as reflected light. Since the ridges and valleys of the fingerprint have different reflection capabilities for light, the reflected light from the fingerprint ridge and the occurrence from the fingerprint valley have different light intensities.
  • the fingerprint sensor chip in the fingerprint device 140 receives and converts it into a corresponding electrical signal, that is, a fingerprint detection signal; based on the fingerprint detection signal, fingerprint image data can be obtained, and fingerprint matching verification can be further performed, so that the electronic device can perform fingerprint matching verification. 100 realizes the optical fingerprint recognition function.
  • the user when the user needs to perform fingerprint unlocking or other fingerprint verification on the electronic device 100 , the user only needs to press the finger on the fingerprint collection area 130 located on the display screen 120 to realize the input operation of the fingerprint feature. Since the collection of fingerprint features can be implemented in the display area 102 of the display screen 120 , the electronic device 100 with the above structure does not need to reserve a space on the front to set the fingerprint buttons (such as the Home button), so a full-screen solution can be adopted. Therefore, the display area 102 of the display screen 120 may extend to substantially the entire front surface of the electronic device 100 .
  • the optical fingerprint device 140 may also use a built-in light source or an external light source to provide an optical signal for fingerprint detection and identification.
  • the optical fingerprint device 140 can be applied not only to self-luminous display screens such as OLED display screens, but also to non-self-luminous display screens, such as liquid crystal display screens or other passive light-emitting display screens.
  • the optical fingerprint system of the electronic device 100 may further include an excitation light source for optical fingerprint detection, the excitation The light source can be specifically an infrared light source or a light source of non-visible light with a specific wavelength, which can be arranged under the backlight module of the liquid crystal display or in the edge area under the protective cover of the electronic device 100, and the optical fingerprint device 140 can be arranged Under the edge area of the liquid crystal panel or the protective cover plate and guided by the optical path, the fingerprint detection light can reach the optical fingerprint device 140;
  • the group allows the fingerprint detection light to pass through the liquid crystal panel and the backlight module and reach the optical fingerprint device 140 by making holes or other optical designs on the film layers such as the diffusion sheet, the brightness enhancement sheet, and the reflective sheet.
  • the optical fingerprint device 140 uses a built-in light source or an external light source to provide an optical signal for fingerprint detection, the detection principle can be the same.
  • the electronic device 100 may further include a transparent protective cover plate 110 , such as a glass cover plate or a sapphire cover plate, which is located above the display screen 120 and covers the front surface of the electronic device 100 , and the surface of the cover plate 110 also A protective layer may be provided. Therefore, in the embodiment of the present application, the so-called finger pressing on the display screen 120 may actually refer to the finger pressing on the cover plate 110 above the display screen 120 or the surface of the protective layer covering the cover plate 110 .
  • a transparent protective cover plate 110 such as a glass cover plate or a sapphire cover plate
  • a circuit board 150 such as a flexible printed circuit (Flexible Printed Circuit, FPC), may also be disposed below the optical fingerprint device 140 .
  • FPC Flexible Printed Circuit
  • the optical fingerprint device 140 can be soldered to the circuit board 150 through the pads. Specifically, the fingerprint sensor chip in the optical fingerprint device 140 can be connected to the circuit board 150 through the pads, and can be connected with other peripheral circuits or electronic devices through the circuit board 150 . Electrical interconnection and signal transmission of other elements of device 100 . For example, the optical fingerprint device 140 can receive the control signal of the processing unit of the electronic device 100 through the circuit board 150 , and can also output the fingerprint detection signal from the optical fingerprint device 140 to the processing unit or the control unit of the electronic device 100 through the circuit board 150 . Wait.
  • the optical fingerprint device 140 may also be considered to include the circuit board 150 .
  • the fingerprint image obtained by the fingerprint identification device can be used for other function detection besides fingerprint identification.
  • other function detection such as heart rate detection. That is, when the user presses the finger to the fingerprint collection area, the fingerprint identification device can continuously collect multiple frames of fingerprint images at a fixed frame rate, calculate the average optical signal of each frame of fingerprint image, and the average optical signal of multiple consecutive fingerprint images within a period of time. It showed strong heart rate photoplethysmography (PPG) signal characteristics.
  • PPG heart rate photoplethysmography
  • the embodiments of the present application provide a heart rate detection device, which is suitable for an electronic device with a display screen, acquires multiple frames of fingerprint images based on optical signals emitted by the display screen, and obtains multiple frames of fingerprint images based on the light signals emitted by the display screen, and uses the PPG signals extracted from the multiple frames of fingerprint images.
  • the standard deviation of the interval between adjacent peaks or adjacent troughs in the time domain obtains the user's heart rate variability (HRV), so that the user's heart rate information can be obtained more accurately.
  • HRV heart rate variability
  • FIG. 2 shows a schematic block diagram of a heart rate detection apparatus 200 according to an embodiment of the present application.
  • the heart rate detection apparatus 200 is suitable for an electronic device with a display screen, and the electronic device can adopt the structure shown in FIG. 1 .
  • the heart rate detection apparatus 200 may include some or all of the following contents:
  • the photosensitive component 210 is arranged below the display screen, and the photosensitive component is used to collect the light signal reflected or scattered by the user's finger and returned by the display screen, so as to obtain a multi-frame fingerprint image, the multi-frame fingerprint image
  • the image is used to obtain a photoplethysmography PPG signal, and the standard deviation of the adjacent peak interval or adjacent trough interval of the PPG signal in the time domain is used to obtain the heart rate variability HRV information of the user;
  • the data amount of each frame of fingerprint images in the multi-frame fingerprint images is smaller than the data amount of fingerprint images used for fingerprint identification.
  • the photosensitive component 210 may be a fingerprint sensor chip including a PD array of photodetectors (one PD may also be referred to as one pixel). It can also be set independently of the fingerprint sensor chip and include one or more PDs dedicated to heart rate detection.
  • the photosensitive component 210 may also be formed by splicing at least part of PDs in a fingerprint sensor chip including a PD array. In other words, the photosensitive component may be formed by splicing at least part of the pixels in the fingerprint sensor chip including the pixel array.
  • This embodiment of the present application should not limit the number of PDs included in the photosensitive assembly 210, as long as the number of PDs included in the photosensitive assembly 210 is smaller than the number of PDs included in the fingerprint sensor used for fingerprint identification.
  • FIG. 3 is an amplitude-time schematic diagram of the conversion of a finger optical signal into an electrical signal.
  • Light is attenuated to a certain extent when it passes through the skin tissue and is then reflected back to the photosensitive component.
  • the absorption of light by muscles, bones, veins and other connecting tissues is basically unchanged, but blood is different. Due to the flow of blood in the arteries, the absorption of light will naturally change.
  • the converted electrical signal is a DC component
  • the AC component is the AC component
  • the AC component can be extracted from the electrical signal converted from the optical signal received by the photosensitive component to reflect the characteristics of blood flow. That is to say, the PPG signal can be extracted from the multi-frame fingerprint images collected continuously over a period of time. Then, the HRV information of the user can be obtained by calculating the standard deviation of the interval between adjacent peaks or adjacent troughs of the PPG signal in the time domain.
  • Figure 4 shows an amplitude-time schematic of the PPG signal extracted from the fingerprint image.
  • the standard deviation of the interval between adjacent peaks or adjacent troughs of the PPG signal in the time domain can be calculated.
  • the interval between adjacent peaks can be represented by ti
  • the third set may also be formed by the interval between adjacent peaks and the interval between adjacent troughs, and the HRV information of the user is obtained by calculating the standard deviation of the elements in the third set.
  • the embodiment of the present application is not limited to the interval between adjacent peaks or the interval between adjacent troughs, and may also be the interval between other adjacent specific points, for example, the interval between adjacent sub-peaks or adjacent sub-troughs.
  • a fingerprint image is 6mm*6mm, and the data size of a fingerprint image with a resolution of 500dpi (dots per inch) is dozens of K, and it needs to be transmitted for 20ms to 40ms, which takes a lot of time. This imposes a relatively large limitation on the sampling rate.
  • the general requirement for HRV detection is that the sampling rate is above 100 Hz. Therefore, the data volume of a fingerprint image used for HRV detection is naturally smaller than that of a fingerprint image used for fingerprint identification.
  • the photosensitive component 210 is a fingerprint sensor chip, and the fingerprint sensor chip can be used to continuously collect multiple frames of the first fingerprint image, and respectively acquire multiple frames of the second fingerprint image according to the multiple frames of the first fingerprint image. fingerprint image, so that the data amount of one frame of the second fingerprint image is smaller than that of one frame of the first fingerprint image.
  • the multi-frame second fingerprint images are used to acquire PPG signals.
  • the multiple frames of the first fingerprint image and the multiple frames of the second fingerprint image are in a one-to-one correspondence, that is, one frame of the first fingerprint image is correspondingly used to obtain one frame of the second fingerprint image.
  • an arithmetic unit may be added at the chip end of the fingerprint sensor to compress pixel data in a frame of the first fingerprint image to reduce the amount of data, thereby obtaining a frame of the second fingerprint image.
  • a frame of the first fingerprint image can be divided into at least one sub-region, and multiple pixel data of each sub-region can be processed separately, and the amount of data obtained by each sub-region can be less than the multiple pixel data included in the sub-region , and optionally, the pixel data in each sub-region can be summed or averaged.
  • one pixel data in each sub-region may be selected, and the remaining pixel data may be discarded.
  • pixel data at the center of each sub-region can also be selected.
  • all or part of the pixel data included in a frame of the first fingerprint image may be processed together and processed into one data, that is, one frame of the second fingerprint image includes one data.
  • each frame of the first fingerprint image may be the same or different.
  • both the first fingerprint image 1 and the first fingerprint image 2 are divided into 4 sub-areas, each sub-area includes 4 pixel data, and the 4 pixel data of each sub-area is processed into one data to form the second fingerprint image 1 and the second fingerprint image 2, that is, the second fingerprint image 1 and the second fingerprint image 2 each include 4 pieces of data.
  • the first fingerprint image 1 is divided into 2 sub-areas
  • the second fingerprint image is divided into 4 sub-areas
  • the 8 pixel data included in each sub-area in the first fingerprint image 1 is processed into one data
  • the The 4 pixel data included in each sub-area in a fingerprint image 2 is processed into one data to form a second fingerprint image 1 and a second fingerprint image 2, that is to say, the second fingerprint image 1 includes 2 data
  • the second fingerprint image Image 2 includes 4 pieces of data.
  • serial Peripheral Interface Serial Peripheral Interface
  • MCU Microcontroller Unit
  • the size of the pixel array of the fingerprint sensor chip is M*N, and when collecting the fingerprint image, M*N pixels are all exposed, and each pixel point is X after an analog-to-digital converter (ADC).
  • ADC analog-to-digital converter
  • some data in the pixel array is selected for transmission, which reduces the amount of transmitted data and increases the sampling rate.
  • the size of the pixel array of the fingerprint sensor chip is M*N, and when collecting the fingerprint image, M*N pixels are all exposed, and each pixel point is X bits after ADC.
  • the master communication rate is Y bits per second
  • a photosensitive component can be set separately outside the fingerprint sensor chip, which is dedicated to heart rate detection.
  • the number of PDs included in the photosensitive component is less than the number of PDs included in the fingerprint sensor chip.
  • the included data is transferred to the MCU, which can reduce the data transfer amount of the SPI.
  • the size of the pixel array of the fingerprint sensor chip is M*N, and when collecting the fingerprint image, M*N pixels are all exposed, and each pixel point is X bits after ADC.
  • the master communication rate is Y bits per second
  • a photosensitive component may not be provided separately outside the fingerprint sensor chip, but some or all of the pixels in the fingerprint sensor chip may be spliced into a large-area photosensitive PD.
  • the hardware gating method can be configured. The data of the fingerprint image collected by the spliced photosensitive PD after one conversion by the ADC is transmitted to the MCU, which can reduce the data transmission amount of the SPI.
  • the size of the pixel array of the fingerprint sensor chip is M*N, and when collecting the fingerprint image, M*N pixels are all exposed, and each pixel point is X bits after ADC.
  • the master communication rate is Y bits per second
  • the initial moment of the exposure time of the first frame fingerprint image in the multi-frame fingerprint images is synchronized with the initial moment of the refresh period of the display screen.
  • FIG. 9 shows a simplified model of the noise caused by the drop in brightness of the screen spot.
  • the spot drop cycle in the figure is the screen refresh cycle.
  • the light energy emitted by the display screen can be simplified into a direct current (DC) component and an alternating current (AC) component.
  • FIG. 11 shows a block diagram of a communication system in an electronic device for realizing synchronization between the initial moment of the exposure time of the fingerprint image and the initial moment of the refresh cycle of the display screen.
  • the fingerprint sensor chip and the processor of the electronic device can communicate based on SPI, such as receiving commands from the processor to perform operations related to fingerprint detection, and the acquired fingerprint
  • SPI system for communicating based on SPI, such as receiving commands from the processor to perform operations related to fingerprint detection, and the acquired fingerprint
  • the data of the image is uploaded to the processor.
  • the processor may also provide power (POWER) to the fingerprint sensor chip, send a reset signal (RST) to the fingerprint sensor chip, and the like.
  • POWER power
  • RST reset signal
  • a communication connection may be added between the photosensitive component and the display screen, so as to synchronize the initial moment of the exposure time of the fingerprint image with the initial moment of the refresh period of the display screen.
  • the photosensitive element may receive a synchronization signal (SYNC) sent by the display screen for triggering the pixel array to perform exposure, and then the photosensitive element performs exposure based on the synchronization signal.
  • SYNC synchronization signal
  • the exposure time of the fingerprint image is a non-integer multiple of the refresh period of the display screen, it can be
  • the set exposure energy deviation sequence performs energy compensation on the fingerprint image after the first frame fingerprint image in the multi-frame fingerprint image, and the exposure energy deviation sequence includes each frame fingerprint image after the first frame fingerprint image and the first frame fingerprint image. Energy offset value between frame fingerprint images.
  • the preset exposure energy deviation sequence may be a corresponding relationship, that is, the corresponding relationship between the energy deviation value and the fingerprint image, and the energy deviation value may be the energy deviation between each frame of fingerprint image and the first frame of fingerprint image by default value.
  • the energy deviation value can also be the energy deviation value between each frame of fingerprint image and any one of the fingerprint images, then in practical application, the initial moment of the exposure time of the any fingerprint image and the The initial moment of the refresh cycle of the display is synchronized.
  • the exposure energy deviation sequence can be obtained through theoretical derivation. Assuming that the photosensitive element is in the continuous exposure mode, there is no time pause in the exposure of two adjacent frames, that is, the exposure time of the previous frame is the start time of the exposure of the next frame.
  • X is a positive real number
  • X can be divided into integer x1 and fractional part x2.
  • the exposure light energy represented by integer x1 is the same, and the exposure energy difference comes from x2.
  • the exposure energy difference of x2 can be theoretically calculated.
  • E1-EN (N-1)*Eac/N*N
  • corresponding compensation can be performed on the fingerprint images from the 2nd to the Nth frame, so that the noise of the brightness drop of the light spot can be minimized.
  • the exposure energy deviation sequence can also be obtained in the whole machine calibration stage.
  • the photosensitive component in the calibration stage of the whole machine, can be in continuous exposure mode to ensure that the initial moment of the exposure time of the first frame of fingerprint image is synchronized with the initial moment of the refresh cycle of the display screen, and then it can actually measure the difference between each frame of fingerprint image and the For the exposure energy difference of the first frame of fingerprint image, as long as a partial or complete exposure energy deviation sequence is obtained during calibration, the algorithmic compensation can be performed using the exposure energy deviation sequence obtained by calibration during actual sampling.
  • the exposure energy deviation value in the exposure energy deviation sequence can be obtained by combining theoretical derivation and calibration.
  • the fingerprint image is compensated according to the exposure energy deviation sequence, which can reduce the noise caused by the spot period drop.
  • the multi-frame fingerprint images may be acquired in a non-highlight mode (High Brightness Mode, HBM) of the display screen.
  • HBM High Brightness Mode
  • brightness in highlight mode can be considered to be higher than 450 nits (nit), while brightness in non-highlight mode can be considered to be lower than 450 nits.
  • brightness in the highlight mode may be considered to be higher than 1000 lux (lux)
  • the brightness in the non-highlight mode may be considered to be lower than 600 lux.
  • the sampling rate of the multi-frame fingerprint images may be between 100 Hz and 1 KHZ.
  • the heart rate detection apparatus of the embodiment of the present application is described in detail above, and the heart rate detection method of the present application will be described in detail below with reference to FIG. 13 .
  • FIG. 13 shows a schematic block diagram of a heart rate detection method 300 according to an embodiment of the present application.
  • the heart rate detection method 300 may be performed by a heart rate detection apparatus 200, and the heart rate detection apparatus 200 is disposed below the display screen of the electronic device.
  • the heart rate detection method 300 may include some or all of the following contents:
  • the data amount of each frame of fingerprint images in the multi-frame fingerprint images is smaller than the data amount of fingerprint images used for fingerprint identification.
  • acquiring the multi-frame fingerprint images of the user includes: continuously collecting the multi-frame first fingerprint images of the user; each frame of the first fingerprint in the multi-frame first fingerprint images The images are compressed to obtain the multiple frames of the second fingerprint images respectively, and the multiple frames of the second fingerprint images are used to obtain the PPG signal.
  • performing compression processing on each frame of the first fingerprint image in the multiple frames of the first fingerprint image to obtain the multiple frames of the second fingerprint image includes: compressing each frame of the first fingerprint image The fingerprint image is divided into at least one sub-area; the plurality of pixel data included in each sub-area of the at least one sub-area of the first fingerprint image of each frame is processed into one data to obtain the multiple frames of the second fingerprint image respectively.
  • the processing of multiple pixel data included in each sub-region in the at least one sub-region into one data includes: summing multiple pixel data included in each sub-region Processing or averaging processing.
  • performing compression processing on each frame of the first fingerprint image in the multiple frames of the first fingerprint image to obtain the multiple frames of the second fingerprint image includes: according to the first fingerprint image of each frame Part of the pixel data in the fingerprint image forms the multi-frame second fingerprint image.
  • the initial moment of the exposure time of the first frame of the fingerprint image in the multi-frame fingerprint image is synchronized with the initial moment of the refresh period of the display screen.
  • the exposure time of each frame of fingerprint images in the multi-frame fingerprint images is an integer multiple of the refresh period of the display screen.
  • the exposure time of each frame of the fingerprint image is equal to the refresh period of the display screen.
  • the exposure time of each frame of fingerprint image in the multi-frame fingerprint image is a non-integer multiple of the refresh period of the display screen
  • the method further includes: according to a preset exposure energy deviation sequence, Energy compensation is performed on the fingerprint images after the first frame fingerprint image in the multi-frame fingerprint images, and the exposure energy deviation sequence includes the exposure energy deviation value between each fingerprint image after the first frame fingerprint image and the first frame fingerprint image.
  • the acquiring the multi-frame fingerprint images of the user includes: in the non-highlight mode HBM mode of the display screen, acquiring the multi-frame fingerprint images.
  • the sampling rate of the multi-frame fingerprint images is greater than or equal to 100 Hz.
  • an embodiment of the present application further provides an electronic device, where the electronic device includes the heart rate detection device in the above-mentioned various embodiments and a display screen, and the heart rate detection device is arranged below the display screen.
  • the electronic equipment includes but is not limited to mobile phones, computers, multimedia machines and game machines.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Cardiology (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Engineering & Computer Science (AREA)
  • Physiology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Image Input (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种心率检测装置(200)及方法(300)、电子设备(100)，心率检测装置(200)应用于具有显示屏(120)的电子设备(100)，其特征在于，包括：感光组件(210)，设置于显示屏(120)的下方，感光组件(210)用于采集显示屏(120)发出的经用户的手指反射或散射返回的光信号，以获取多帧指纹图像，多帧指纹图像用于获取光电容积脉搏波描记法PPG信号，PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差用于得到用户的心率变异性HRV信息，其中，多帧指纹图像中每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。

Description

心率检测装置及方法、电子设备 技术领域
本申请实施例涉及生物识别领域，并且更具体地，涉及一种心率检测装置及方法、电子设备。
背景技术
“全面屏”作为当下智能终端的流行词，越来越被用户接受和喜爱，屏下指纹识别装置也成为一种趋势，逐渐在智能终端中普及，屏下指纹识别装置基于光学***成像，使得其除了获取指纹信号外，还有更多可能。
如何在屏下指纹识别***中实现心率检测，是值得研究的问题。
发明内容
本申请实施例提供一种心率检测装置及方法、电子设备，能够基于屏下光学指纹***实现心率检测。
一方面，提供了一种心率检测装置，应用于具有显示屏的电子设备，该心率检测装置包括：感光组件，设置于该显示屏的下方，该感光组件用于采集该显示屏发出的经用户的手指反射或散射返回的光信号，以获取多帧指纹图像，该多帧指纹图像用于获取光电容积脉搏波描记法PPG信号，该PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差用于得到该用户的心率变异性HRV信息；其中，该多帧指纹图像中每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
在一种可能的实现方式中，该感光组件为指纹传感器芯片，该指纹传感器芯片用于：连续采集多帧第一指纹图像，对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以分别获取多帧第二指纹图像，该多帧第二指纹图像用于获取该PPG信号。
在一种可能的实现方式中，该指纹传感器芯片用于：将该每帧第一指纹图像划分为至少一个子区域；将该每帧第一指纹图像的至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，以分别获取该多帧第二指纹图像。
在一种可能的实现方式中，该指纹传感器芯片用于：对该每个子区域包 括的多个像素数据进行求和处理或求平均处理。
在一种可能的实现方式中，该指纹传感器芯片用于：根据该每帧第一指纹图像中的部分像素数据，获取该多帧第二指纹图像。
在一种可能的实现方式中，该感光组件是由该电子设备中的指纹传感器芯片中的至少部分像素拼接形成的。
在一种可能的实现方式中，该感光组件独立于该电子设备中的指纹传感器芯片设置。
在一种可能的实现方式中，该多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与该显示屏的刷新周期的初始时刻同步。
在一种可能的实现方式中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的整数倍。
在一种可能的实现方式中，该每帧指纹图像的曝光时间等于该显示屏的刷新周期。
在一种可能的实现方式中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的非整数倍，该心率检测装置还包括：处理器，用于根据预设的曝光能量偏差序列，对该多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，该曝光能量偏差序列包括该首帧指纹图像之后的每帧指纹图像与该首帧指纹图像之间的能量偏差值。
在一种可能的实现方式中，该感光组件用于：在该显示屏的非高亮模式HBM模式下，获取该多帧指纹图像。
在一种可能的实现方式中，该多帧指纹图像的采样率大于或等于100Hz。
另一方面，提供了一种心率检测方法，应用于具有显示屏的电子设备，该心率检测方法包括：根据该显示屏发出的经用户的手指反射或散射返回的光信号，获取该用户的多帧指纹图像；提取该多帧指纹图像中每帧指纹图像的光电容积脉搏波描记法PPG信号；计算该PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差，得到该用户的心率变异性HRV信息；其中，该多帧指纹图像中的每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
在一种可能的实现方式中，该获取该用户的多帧指纹图像，包括：连续采集该用户的多帧第一指纹图像；对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以分别获取多帧第二指纹图像，该多帧第二指纹图像用 于获取该PPG信号。
在一种可能的实现方式中，该对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取该多帧第二指纹图像，包括：将该每帧第一指纹图像划分为至少一个子区域；将该每帧第一指纹图像的至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，以分别获取该多帧第二指纹图像。
在一种可能的实现方式中，该将该至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，包括：对该每个子区域包括的多个像素数据进行求和处理或求平均处理。
在一种可能的实现方式中，该对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取该多帧第二指纹图像，包括：根据该每帧第一指纹图像中的部分像素数据，获取该多帧第二指纹图像。
在一种可能的实现方式中，该多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与该显示屏的刷新周期的初始时刻同步。
在一种可能的实现方式中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的整数倍。
在一种可能的实现方式中，该每帧指纹图像的曝光时间等于该显示屏的刷新周期。
在一种可能的实现方式中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的非整数倍，该心率检测方法还包括：根据预设的曝光能量偏差序列，对该多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，该曝光能量偏差序列包括该首帧指纹图像之后的每帧指纹图像与该首帧指纹图像之间的能量偏差值。
在一种可能的实现方式中，该获取该用户的多帧指纹图像，包括：在该显示屏的非高亮模式HBM模式下，获取该多帧指纹图像。
在一种可能的实现方式中，该多帧指纹图像的采样率大于或等于100Hz。
第三方面，提供了一种电子设备，包括显示屏和第一方面或第一方面中任一可能的实现方式中该的心率检测装置。
本申请实施例的技术方案基于显示屏发出的光信号获取多帧指纹图像，并进行心率检测，从而可以实现屏下心率检测。进一步地通过计算该多帧指纹图像的PPG信号，并基于该PPG信号在时域上的相邻波峰间隔或相邻波 谷间隔的标准差得到用户的HRV信息，可以更精确地获得用户的心率信息。
附图说明
图1A是根据本申请一实施例的电子设备的定向视图。
图1B是图1A所示的电子设备沿A-A’的部分剖面结构示意图。
图2是本申请实施例的心率检测装置的示意性框图。
图3是手指光学信号转换为电信号的幅度-时间示意图。
图4是反应心率信息的指纹图像提取PPG信号图。
图5至图8是本申请实施例的采样心率信息的***框图。
图9是由屏幕光斑亮度跌落所引起的噪声的模型简化图。
图10是每一帧指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步的示意图。
图11是感光组件与显示屏时序同步的***图。
图12是刷新周期N等分交流能量模型时序图。
图13是本申请实施例的心率检测方法的示意性框图。
具体实施方式
为了使本领域的人员更好地理解本申请实施例中的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请实施例的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员所获得的所有其他实施例，都应当属于本申请实施例保护的范围。
本申请实施例的技术方案可以应用于各种电子设备。
例如，智能手机、笔记本电脑、平板电脑、游戏设备等便携式或移动计算设备，以及电子数据库、汽车、银行自动柜员机(Automated Teller Machine，ATM)等其他电子设备。但本申请实施例对此并不限定。
本申请实施例的技术方案可以用于屏下指纹识别技术。屏下指纹识别技术是指将指纹识别模组安装在显示屏下方，从而实现在显示屏的显示区域内进行指纹识别操作，不需要在电子设备正面除显示区域外的区域设置指纹采集区域。具体地，指纹识别模组使用从电子设备的显示组件的顶面返回的光来进行指纹感应和其他感应操作。这种返回的光携带与显示组件的顶面接触 的物体(例如手指)的信息，位于显示组件下方的指纹识别模组通过采集和检测这种返回的光以实现屏下指纹识别。其中，指纹识别模组的设计可以为通过恰当地配置用于采集和检测返回的光的光学元件来实现期望的光学成像。
图1A和图1B示出了屏下指纹识别技术可以适用的电子设备100的示意图，其中图1A为电子设备100的正面示意图，图1B为图1A所示的电子设备100沿A-A’的部分剖面结构示意图。
如图1A和图1B所示，电子设备100可以包括显示屏120和光学指纹装置140。
显示屏120可以为自发光显示屏，其采用具有自发光的显示单元作为显示像素。比如显示屏120可以为有机发光二极管(Organic Light-Emitting Diode，OLED)显示屏或者微型发光二极管(Micro-LED)显示屏。在其他可替代实施例中，显示屏120也可以为液晶显示屏(Liquid Crystal Display,LCD)或者其他被动发光显示屏，本申请实施例对此不做限制。
此外，显示屏120还可以具体为触控显示屏，其不仅可以进行画面显示，还可以检测用户的触摸或者按压操作，从而为用户提供一个人机交互界面。比如，在一种实施例中，电子设备100可以包括触摸传感器，所述触摸传感器可以具体为触控面板(Touch Panel,TP)，其可以设置在所述显示屏120表面，也可以部分集成或者整体集成到所述显示屏120内部，从而形成所述触控显示屏。
具体来说，光学指纹装置140可以包括具有光学感应阵列的指纹传感器芯片(后面也称为光学指纹传感器或光学指纹芯片)。其中，光学感应阵列包括多个光学感应单元，每个光学感应单元可以具体包括光探测器或者光电传感器。或者说，光学指纹装置140可以包括光探测器(Photo detector，PD)阵列(或称为光电探测器阵列、光电传感器阵列、光学传感器阵列，感应阵列、像素阵列)，其包括多个呈阵列式分布的光探测器。
如图1A所示，光学指纹装置140可以设置在所述显示屏120的下方的局部区域，从而使得光学指纹装置140的指纹采集区域(或指纹检测区域)130至少部分位于所述显示屏120的显示区域102内。
当然，在其他可替代实施例中，光学指纹装置140也可以设置在其他位置，比如显示屏120的侧面或者电子设备100的边缘非透光区域。这种情况 下，可以通过光路设计将显示屏120的至少部分显示区域的光信号导引到光学指纹装置140，从而使得所述指纹采集区域130实际上位于所述显示屏120的显示区域内。
在本申请的一些实施例中，光学指纹装置140可以仅包括一个指纹传感器芯片，此时光学指纹装置140的指纹采集区域130的面积较小且位置固定，因此用户在进行指纹输入时需要将手指按压到所述指纹采集区域130的特定位置，否则光学指纹装置140可能无法采集到指纹图像而造成用户体验不佳。
在本申请的另一些实施例中，光学指纹装置140可以具体包括多个指纹传感器芯片；所述多个指纹传感器芯片可以通过拼接方式并排设置在所述显示屏120的下方，且所述多个指纹传感器芯片的感应区域共同构成所述光学指纹装置140的指纹采集区域130。也就是说，所述光学指纹装置140的指纹采集区域130可以包括多个子区域，每个子区域分别对应于其中一个指纹传感器芯片的感应区域，从而将所述光学指纹模组130的指纹采集区域130可以扩展到所述显示屏的下半部分的主要区域，即扩展到手指惯常按压区域，从而实现盲按式指纹输入操作。可替代地，当所述指纹传感器芯片数量足够时，所述指纹检测区域130还可以扩展到半个显示区域甚至整个显示区域，从而实现半屏或者全屏指纹检测。
应理解，本申请实施例对所述多个指纹传感器芯片的具体形式不做限定。例如，所述多个指纹传感器芯片可以分别是独立封装的指纹传感器芯片，也可以是封装在同一个芯片封装体内的多个芯片(Die)。又例如，还可以通过半导体工艺在同一个芯片(Die)的不同区域上制作形成所述多个指纹传感器芯片。
如图1B所示，光学指纹装置140的光学感应阵列的所在区域或者光感应范围对应所述光学指纹装置140的指纹采集区域130。其中，光学指纹装置140的指纹采集区域130可以等于或不等于光学指纹装置140的光学感应阵列的所在区域的面积或者光感应范围，本申请实施例对此不做具体限定。
例如，通过光线准直的光路设计，光学指纹装置140的指纹采集区域130可以设计成与所述光学指纹装置140的感应阵列的面积基本一致。
又例如，通过汇聚光线的光路设计或者反射光线的光路设计，可以使得所述光学指纹装置140的指纹采集区域130的面积大于所述光学指纹装置140感应阵列的面积。
在本申请一些实施例中，所述光学指纹装置140还可以包括光学组件，所述光学组件可以设置在所述感应阵列的上方，其可以具体包括滤光层(Filter)、导光层或光路引导结构以及其他光学元件，所述滤光层可以用于滤除穿透手指的环境光，例如，干扰成像的红外光，而所述导光层或光路引导结构主要用于从手指表面反射回来的反射光引导至所述感应阵列进行光学检测。
下面对光学指纹装置140的光路设计进行示例性说明。
作为一个实施例，所述光学指纹装置140可以采用具有高深宽比的通孔阵列的光学准直器，所述光学准直器可以具体为在半导体硅片制作而成的准直器(Collimator)层，其具有多个准直单元或者微孔，所述准直单元可以具体为小孔，从手指反射回来的反射光中，垂直入射到所述准直单元的光线可以穿过并被其下方的指纹传感器芯片接收，而入射角度过大的光线在所述准直单元内部经过多次反射被衰减掉，因此每一个指纹传感器芯片基本只能接收到其正上方的指纹纹路反射回来的反射光，能够有效提高图像分辨率，进而提高指纹识别效果。
进一步地，当光学指纹装置140包括多个指纹传感器芯片时，可以为每个指纹传感器芯片的光学感应阵列中的一个光学感应单元配置一个准直单元，并贴合设置在其对应的光学感应单元的上方。当然，所述多个光学感应单元也可以共享一个准直单元，即所述一个准直单元具有足够大的孔径以覆盖多个光学感应单元。由于一个准直单元可以对应多个光学感应单元，破坏了显示屏120的空间周期和指纹传感器芯片的空间周期的对应性，因此，即使显示屏120的发光显示阵列的空间结构和指纹传感器芯片的光学感应阵列的空间结构类似，也能够有效避免光学指纹装置140利用经过显示屏120的光信号进行指纹成像生成莫尔条纹，有效提高了光学指纹装置140的指纹识别效果。
作为另一个实施例，所述光学指纹装置140可以采用基于光学镜头的光路设计，所述光学镜头可以包括光学透镜(Lens)层，其具有一个或多个透镜单元，比如一个或多个非球面透镜组成的透镜组，其用于将从手指反射回来的反射光汇聚到其下方的指纹传感器芯片的感应阵列，以使得所述感应阵列可以基于所述反射光进行成像，从而得到所述手指的指纹图像。所述光学透镜层在所述透镜单元的光路中还可以形成有针孔，所述针孔可以配合所述 光学透镜层扩大光学指纹装置140的视场，以提高所述光学指纹装置140的指纹成像效果。
进一步地，当光学指纹装置140包括多个指纹传感器芯片时，可以为每一个指纹传感器芯片配置一个光学镜头进行指纹成像，或者为多个指纹传感器芯片配置一个光学镜头来实现光线汇聚和指纹成像。甚至于，当一个指纹传感器芯片具有两个感应阵列(Dual Array)或多个感应阵列(Multi-Array)时，也可以为这个指纹传感器芯片配置两个或多个光学镜头配合所述两个感应阵列或多个感应阵列进行光学成像，从而减小成像距离并增强成像效果。
作为再一个实施例，所述光学指纹装置140可以采用微透镜(Micro-Lens)层的光路设计，所述微透镜层可以具有由多个微透镜形成的微透镜阵列，其可以通过半导体生长工艺或者其他工艺形成在所述指纹传感器芯片的感应阵列上方，并且每一个微透镜可以分别对应于所述感应阵列的其中一个感应单元。所述微透镜层和所述感应单元之间还可以形成其他光学膜层，比如介质层或者钝化层，更具体地，所述微透镜层和所述感应单元之间还可以包括具有微孔的挡光层，其中所述微孔形成在其对应的微透镜和感应单元之间，所述挡光层可以阻挡相邻微透镜和感应单元之间的光学干扰，并使光线通过所述微透镜汇聚到所述微孔内部并经由所述微孔传输到所述微透镜对应的感应单元，以进行光学指纹成像。
应当理解，上述光路引导结构的几种实现方案可以单独使用也可以结合使用，比如，可以在所述准直器层或者所述光学透镜层下方进一步设置微透镜层。当然，在所述准直器层或者所述光学透镜层与所述微透镜层结合使用时，其具体叠层结构或者光路可能需要按照实际需要进行调整。
所述光学指纹装置140可以用于采集用户的指纹信息(比如指纹图像信息)。
作为一种可选的实施例，所述显示屏120可以采用具有自发光显示单元的显示屏，比如有机发光二极管(Organic Light-Emitting Diode，OLED)显示屏或者微型发光二极管(Micro-LED)显示屏。以采用OLED显示屏为例，光学指纹装置140可以利用OLED显示屏的位于指纹采集区域130的显示单元(即OLED光源)来作为光学指纹检测的激励光源。
当手指触摸、按压或者接近(为便于描述，在本申请中统称为按压)在指纹采集区域130时，显示屏120向指纹采集区域130上方的手指发出一束 光，这一束光在手指的表面发生反射形成反射光或者经过手指的内部散射后而形成散射光，在相关专利申请中，为便于描述，上述反射光和散射光统称为反射光。由于指纹的嵴(ridge)与峪(vally)对于光的反射能力不同，因此，来自指纹嵴的反射光和来自指纹峪的发生过具有不同的光强，反射光经过显示屏120后，被光学指纹装置140中的指纹传感器芯片所接收并转换为相应的电信号，即指纹检测信号；基于所述指纹检测信号便可以获得指纹图像数据，并且可以进一步进行指纹匹配验证，从而在所述电子设备100实现光学指纹识别功能。
由此可见，用户需要对电子设备100进行指纹解锁或者其他指纹验证的时候，只需要将手指按压在位于显示屏120的指纹采集区域130，便可以实现指纹特征的输入操作。由于指纹特征的采集可以在显示屏120的显示区域102的内部实现，采用上述结构的电子设备100无需其正面专门预留空间来设置指纹按键(比如Home键)，因而可以采用全面屏方案。因此，所述显示屏120的显示区域102可以基本扩展到所述电子设备100的整个正面。
在其他替代实施例中，所述光学指纹装置140也可以采用内置光源或者外置光源来提供用于进行指纹检测识别的光信号。在这种情况下，光学指纹装置140不仅可以适用于如OLED显示屏等自发光显示屏，还可以适用于非自发光显示屏，比如液晶显示屏或者其他的被动发光显示屏。
以应用在具有背光模组和液晶面板的液晶显示屏为例，为支持液晶显示屏的屏下指纹检测，电子设备100的光学指纹***还可以包括用于光学指纹检测的激励光源，所述激励光源可以具体为红外光源或者特定波长非可见光的光源，其可以设置在所述液晶显示屏的背光模组下方或者设置在电子设备100的保护盖板下方的边缘区域，而光学指纹装置140可以设置液晶面板或者保护盖板的边缘区域下方并通过光路引导以使得指纹检测光可以到达所述光学指纹装置140；或者，光学指纹装置140也可以设置在所述背光模组下方，且所述背光模组通过对扩散片、增亮片、反射片等膜层进行开孔或者其他光学设计以允许指纹检测光穿过液晶面板和背光模组并到达光学指纹装置140。当采用所述光学指纹装置140采用内置光源或者外置光源来提供用于进行指纹检测的光信号时，其检测原理可以相同。
如图1A所示，电子设备100还可以包括透明保护盖板110，比如玻璃盖板或者蓝宝石盖板，其位于显示屏120的上方并覆盖所述电子设备100的 正面，且盖板110表面还可以设置有保护层。因此，本申请实施例中，所谓的手指按压显示屏120实际上可以是指手指按压在显示屏120上方的盖板110或者覆盖所述盖板110的保护层表面。
如图1B所示，光学指纹装置140的下方还可以设置有电路板150，比如软性电路板(Flexible Printed Circuit，FPC)。
光学指纹装置140可以通过焊盘焊接到电路板150，具体地，所述光学指纹装置140中的指纹传感器芯片可以通过焊盘连接到电路板150，并通过电路板150实现与其他***电路或者电子设备100的其他元件的电性互连和信号传输。比如，光学指纹装置140可以通过电路板150接收电子设备100的处理单元的控制信号，并且还可以通过电路板150将来自光学指纹装置140的指纹检测信号输出给电子设备100的处理单元或者控制单元等。
在一些实施例中，也可以认为所述光学指纹装置140包括所述电路板150。
随着屏下指纹识别技术的发展，通过指纹识别装置获取到的指纹图像除了可以进行指纹识别之外，还可以进行其他功能检测。例如心率检测。即当用户将手指按压到指纹采集区域时，指纹识别装置可以按照固定帧率连续采集多帧指纹图像，计算每帧指纹图像的平均光学信号量，一段时间内连续多帧指纹图像的平均光学信号量呈现较强的心率光电容积脉搏波描记法(photo plethysmography，PPG)信号特征。
因此，本申请实施例提供了一种心率检测装置，其适用于具有显示屏的电子设备，基于显示屏发出的光信号获取多帧指纹图像，并通过从多帧指纹图像中所提取的PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差得到用户的心率变异性(heart rate variability,HRV)，从而可以比较精确地获取到用户的心率信息。
图2示出了本申请实施例的心率检测装置200的示意性框图。该心率检测装置200适用于具有显示屏的电子设备，该电子设备可以采用如图1所示的结构。具体地，该心率检测装置200可以包括以下部分或全部内容：
感光组件210，设置于所述显示屏的下方，所述感光组件用于采集所述显示屏发出的经用户的手指反射或散射返回的光信号，以获取多帧指纹图像，所述多帧指纹图像用于获取光电容积脉搏波描记法PPG信号，所述PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差用于得到所述用户的 心率变异性HRV信息；其中，所述多帧指纹图像中每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
首先需要说明的是，该感光组件210可以是包括光探测器PD阵列(一个PD也可以称为一个像素)的指纹传感器芯片。也可以是独立于指纹传感器芯片设置且包括一个或多个PD，专用于心率检测。该感光组件210还可以是由包括PD阵列的指纹传感器芯片中的至少部分PD拼接而成的。换句话说，该感光组件可以是由包括像素阵列的指纹传感器芯片中的至少部分像素拼接而成的。本申请实施例应对感光组件210所包括的PD数量不作限定，只要感光组件210包括的PD数量小于用于指纹识别的指纹传感器所包括的PD数量即可。
图3是手指光信号转换为电信号的幅度-时间示意图。当光照透过皮肤组织然后再反射到感光组件时光照是有一定的衰减的。像肌肉、骨骼、静脉和其他连接组织等等对光的吸收是基本不变的，但是血液不同，由于动脉里有血液的流动，那么对光的吸收自然也有所变化。从图3中可知，由于手指静脉血或手指深层组织对光的吸收基本不变，所转换的电信号则为直流分量，而手指动脉血对光的吸收是有变化的，所转换的电信号则为交流分量，从感光组件接收到的光信号转换的电信号中提取出交流分量就可以反映出血液流动的特点。也就是说可以从一段时间内连续采集的多帧指纹图像中提取出PPG信号。再通过计算PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差就可以得到用户的HRV信息。图4示出了从指纹图像中提取的PPG信号的幅度-时间示意图。如图4所示，可以计算PPG信号在时域上相邻波峰间隔或相邻波谷间隔的标准差。例如，相邻波峰间隔可以用t i表示，相邻波谷间隔可以用t i’表示(i＝1,2,3……)，其中，相邻波峰间隔(t 1，t 2，t 3……)构成第一集合，相邻波谷间隔(t 1’，t 2’，t 3’……)构成第二集合，通过计算第一集合或第二集合内元素的标准差，得到用户的HRV信息。
可选地，也可以由相邻波峰间隔和相邻波谷间隔共同构成第三集合，通过计算第三集合内元素的标准差得到用户的HRV信息。需要说明的是，本申请实施例并不限于相邻波峰间隔或相邻波谷间隔，也可以是其他相邻特定点之间的间隔，例如，相邻次波峰间隔或相邻次波谷间隔。
通常一张指纹图像为6mm*6mm，分辨率为500dpi(点每英寸)的指纹图像的数据大小为几十K，需要传输20ms～40ms，耗时较大。这对于采样率 的限制相对较大。而HRV检测的一般要求是采样率在100Hz以上，因此，用于HRV检测的一张指纹图像的数据量自然要小于用于指纹识别的一张指纹图像的数据量。
在本申请一种实现方式中，该感光组件210是指纹传感器芯片，该指纹传感器芯片可以用于连续采集多帧第一指纹图像，以及根据该多帧第一指纹图像，分别获取多帧第二指纹图像，以使得一帧第二指纹图像的数据量是小于一帧第一指纹图像的数据量。该多帧第二指纹图像用于获取PPG信号。
需要说明的是，该多帧第一指纹图像与多帧第二指纹图像是一一对应的，也就是说，一帧第一指纹图像对应用于获取一帧第二指纹图像。
具体地，可以在指纹传感器芯片端增加运算单元，对一帧第一指纹图像中的像素(pixel)数据进行压缩处理，以减少数据量，从而获取一帧第二指纹图像。例如，可以将一帧第一指纹图像分割成至少一个子区域，并单独对每个子区域的多个像素数据进行处理，每个子区域得到的数据量可以少于该子区域包括的多个像素数据的数据量，可选地，可以对每个子区域内的像素数据进行求和或者求平均值。可替代的，可以选取每个子区域中的一个像素数据，舍弃其余的像素数据。或者也可以选择每个子区域处于中心位置的像素数据等。可选地，还可以将一帧第一指纹图像所包括的所有或部分像素数据一并处理，并处理成一个数据，也就是说，一帧第二指纹图像包括一个数据。
需要说明的是，对每一帧第一指纹图像所作的数据处理可以是相同的，也可以是不同的。例如，将第一指纹图像1和第一指纹图像2均划分为4个子区域，每个子区域包括4个像素数据，将每个子区域的4个像素数据处理成一个数据，形成第二指纹图像1和第二指纹图像2，也就是说，第二指纹图像1和第二指纹图像2均包括4个数据。再例如，将第一指纹图像1划分成2个子区域，将第二指纹图像划分为4个子区域，将第一指纹图像1中的每个子区域包括的8个像素数据处理成一个数据，将第一指纹图像2中的每个子区域包括的4个像素数据处理成一个数据，形成第二指纹图像1和第二指纹图像2，也就是说，第二指纹图像1包括2个数据，第二指纹图像2包括4个数据。
如图5所示，通过改进指纹传感器芯片，在其内部增加运算单元对所有或者部分区域的像素数据求和或者求均值，由多个数据得到一个数据从而减 少通过串行外设接口(Serial Peripheral Interface,SPI)传输到电子设备的处理器，例如微控制单元(Microcontroller Unit,MCU)的数据量，降低传输耗时，提升采样率。
假设指纹传感器芯片的pixel阵列的大小为M*N，而采集指纹图像时需要M*N个pixel全部曝光，每个pixel点经过模数转换器(analog-to-digital converter，ADC)之后为X比特，假设主控通信速率为每秒Y比特，那么指纹传感器芯片获取的一帧指纹图像通过原始的数据路径(即路径1)传输所需要的时间为t1＝M*N*X/Y；在增加运算单元之后，指纹传感器芯片获取的一帧指纹图像可以在经过运算单元处理之后得到Z个数据(Z<M*N)再传输，即通过路径2传输，所需要的时间为t2＝Z*X/Y，由此可知，t2小于t1，即采用路径2传输的采样率大于采用路径1传输的采样率。
如图6所示，在不改变现有指纹传感器芯片的方式下，选择pixel阵列中的部分数据进行传输，降低传输数据量，提高采样率。
假设指纹传感器芯片的pixel阵列的大小为M*N，而采集指纹图像时需要M*N个pixel全部曝光，每个pixel点经过ADC之后为X比特，假设主控通信速率为每秒Y比特，那么传输一帧指纹图像所需要的时间为t1＝M*N*X/Y；在进行心率检测时，可以配置指纹传感器芯片的部分pixel曝光，即曝光pixel的个数为a*b，其中，a<M，b<N，因此，传输一帧指纹图像所需要的时间为t2＝a*b*X/Y，由此可知，t2小于t1，采样率提高。
如图7所示，可以在指纹传感器芯片的外部单独设置一个感光组件，专用于心率检测，该感光组件包括的PD数量小于指纹传感器芯片包括的PD数量，将该感光组件检测到的指纹图像所包括的数据传输给MCU，可以减少SPI的数据传输量。
假设指纹传感器芯片的pixel阵列的大小为M*N，而采集指纹图像时需要M*N个pixel全部曝光，每个pixel点经过ADC之后为X比特，假设主控通信速率为每秒Y比特，那么传输一帧指纹图像所需要的时间为t1＝M*N*X/Y；在进行心率检测时，通过感光组件采集到的一帧指纹图像的数据量为Z*X比特(Z<M*N)，那么传输由感光组件采集到一帧指纹图像所需要的时间为t2＝Z*X/Y。由此可知，t2小于t1，采样率提高。
可选地，也可以不在指纹传感器芯片外部单独设置一个感光组件，而是可以将指纹传感器芯片中的部分或全部pixel拼接成一个大面积的感光PD。 如图8所示，可以通过配置硬件选通的方式。将由该拼接而成的感光PD采集到的指纹图像经过ADC一次转换之后的数据传输给MCU，可以减少SPI的数据传输量。
假设指纹传感器芯片的pixel阵列的大小为M*N，而采集指纹图像时需要M*N个pixel全部曝光，每个pixel点经过ADC之后为X比特，假设主控通信速率为每秒Y比特，那么传输一帧指纹图像所需要的时间为t1＝M*N*X/Y；在进行心率检测时，可将所有的pixel硬件连通形成一个大的PD，那么传输由该PD采集到一帧指纹图像所需要的时间为t2＝X/Y。由此可知，t2小于t1，采样率提高。
进一步地，在本申请实施例中，所述多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与所述显示屏的刷新周期的初始时刻同步。
由于本申请中的心率检测装置是将显示屏作为光源，而屏幕光斑刷新会引起光斑亮度跌落，若光斑亮度跌落与心率检测装置中的感光组件的曝光不同步，则帧与帧之间在曝光期间存在光强度差异，对PPG信号的采集会造成噪声干扰。图9示出了由屏幕光斑亮度跌落所引起的噪声的模型简化图。图中的光斑跌落周期即屏幕刷新周期。由图9所示，显示屏发出的光能量可以简化为直流(DC)分量以及交流(AC)分量组成。
为了降低该屏幕光斑亮度跌落所引起的噪声，可选地，如图10所示，可以将每一帧指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步。即曝光时间t＝X*T(刷新周期)，其中，X为正整数，曝光时间可以是相邻两帧曝光开始的间隔或相邻两帧曝光结束的间隔。由此可以确保采集每一帧指纹图像的曝光pixel接收到的光强相同，从而降低***噪声。进一步地，可以将每一帧指纹图像的曝光时间设置为等于显示屏的刷新周期。
图11示出了电子设备中实现指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步的通信***框图。如图11所示，在传统的通信***中，指纹传感器芯片与电子设备的处理器之间可以基于SPI进行通信，例如接收处理器的命令以执行与指纹检测相关的操作，以及将获取的指纹图像的数据上传至处理器。处理器还可以向指纹传感器芯片提供电源(POWER)以及向指纹传感器芯片发送复位信号(RST)等。而在本申请实施例中，可以在感光组件和显示屏之间增加通信连接，以实现指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步。例如，感光元件可以接收显示 屏发送的用于触发像素阵列进行曝光的同步信号(SYNC)，进而感光元件基于该同步信号进行曝光。
可选地，在确保首帧指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步的情况下，如果指纹图像的曝光时间是显示屏的刷新周期的非整数倍，可以根据预设的曝光能量偏差序列，对所述多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，所述曝光能量偏差序列包括所述首帧指纹图像之后的每帧指纹图像与所述首帧指纹图像之间的能量偏差值。
该预设的曝光能量偏差序列可以是一种对应关系，即能量偏差值与指纹图像之间的对应关系，该能量偏差值可以默认是每一帧指纹图像与首帧指纹图像之间的能量偏差值。可选地，该能量偏差值也可以是每一帧指纹图像与其中任一帧指纹图像之间的能量偏差值，那么在实际应用时，则需要该任一指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步。
在确定的显示屏的刷新周期与指纹图像的曝光时间之下，该曝光能量偏差序列包括的能量偏差值的个数是确定的。例如，显示屏的刷新周期为10ms，指纹图像的曝光时间为10.2ms，那么该曝光能量偏差序列包括的能量偏差值的个数为10/(10.2-10)＝50。
可选地，在本申请实施例中，该曝光能量偏差序列可以通过理论推导获得。假设感光组件处于连续曝光模式，相邻两帧曝光没有时间停顿，即上一帧曝光接收时刻就是下一帧曝光开始时刻。
假设在一个完整的刷新周期中，感光组件积分得到的能量为Et＝Eac(交流部分能量)+Edc(直流部分能量)；假设曝光时间t＝X*T(刷新周期)，X为正实数，X可分为整数x1和小数部分x2，对于每一帧指纹图像来说，整数x1代表的曝光光能量是相同的，曝光能量差来源于x2，连续曝光模式下只要第一帧曝光时间的初始时刻和显示屏的刷新周期同步，则可以理论推算出x2的曝光能量差。
假设刷新周期被N等分，N＝T/(x2*T)，图12示出了刷新周期N等分交流能量模型时序图：
第1帧曝光收集的光能量为E1＝X1*Et+Edc/N+Eac*(2N-1)/(2*N*N)
第2帧曝光收集的光能量为E2＝X1*Et+Edc/N+Eac*(2N-3)/(2*N*N)
第3帧曝光收集的光能量为E3＝X1*Et+Edc/N+Eac*(2N-5)/(2*N*N)
…..
第N帧曝光收集的光能量为EN＝X1*Et+Edc/N+Eac*/(2*N*N)
每一帧相对于首帧的曝光能量差进行理论推算如下：
E1-E2＝Eac/N*N
E1-E3＝2*Eac/N*N
…..
E1-EN＝(N-1)*Eac/N*N
在实际应用中可以对第2帧至第N帧指纹图像进行相应的补偿，从而可以将光斑亮度跌落的噪声降至最低。
可选地，该曝光能量偏差序列也可以通过在整机校准阶段获得。具体地，可以在整机校准阶段，使得感光组件处于连续曝光模式，确保首帧指纹图像的曝光时间的初始时刻与显示屏的刷新周期的初始时刻同步，然后可以实际测量出每帧指纹图像与首帧指纹图像的曝光能量差值，只要在校准过中获取部分或完整的曝光能量偏差序列，在实际采样时就可以利用校准获得的曝光能量偏差序列进行算法补偿。
可选地，该曝光能量偏差序列中的曝光能量偏差值可以结合理论推导和校准获得，在实际应用中依据曝光能量偏差序列对指纹图像进行补偿，就可以降低光斑周期跌落引入的噪声。
可选地，在本申请实施例中，可以在显示屏的非高亮模式(High Brightness Mode，HBM)下获取该多帧指纹图像。在进行心率检测时，可以减少长时间显示屏的高亮对显示屏寿命造成的影响。
例如，高亮模式下的亮度可以认为是高于450尼特(nit)，而非高亮模式下的亮度则可以认为是低于450nit。再例如，高亮模式下的亮度可以认为是高于1000勒克斯(lux)，非高亮模式下的亮度可以认为是低于600lux。
可选地，在本申请实施例，该多帧指纹图像的采样率可以在100Hz到1KHZ之间。
上文详细描述本申请实施例的心率检测装置，下面将结合图13详细描述本申请的心率检测方法。
图13示出了本申请实施例的心率检测方法300的示意性框图。该心率检测方法300可以由心率检测装置200来执行，该心率检测装置200设置于电子设备的显示屏的下方，具体地，该心率检测方法300可以包括以下部分或全部内容：
S310，根据该显示屏发出的经用户的手指反射或散射返回的光信号，获取该用户的多帧指纹图像；
S320，提取该多帧指纹图像中每帧指纹图像的光电容积脉搏波描记法PPG信号；
S330，计算所述PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差，得到所述用户的心率变异性HRV信息；
其中，该多帧指纹图像中的每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
可选地，在本申请实施例中，该获取该用户的多帧指纹图像，包括：连续采集该用户的多帧第一指纹图像；对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以分别获取该多帧第二指纹图像，该多帧第二指纹图像用于获取该PPG信号。
可选地，在本申请实施例中，该对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取该多帧第二指纹图像，包括：将该每帧第一指纹图像划分为至少一个子区域；将该每帧第一指纹图像的至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，以分别获取该多帧第二指纹图像。
可选地，在本申请实施例中，该将该至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，包括：对该每个子区域包括的多个像素数据进行求和处理或求平均处理。
可选地，在本申请实施例中，该对该多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取该多帧第二指纹图像，包括：根据该每帧第一指纹图像中的部分像素数据，形成该多帧第二指纹图像。
可选地，在本申请实施例中，该多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与该显示屏的刷新周期的初始时刻同步。
可选地，在本申请实施例中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的整数倍。
可选地，在本申请实施例中，该每帧指纹图像的曝光时间等于该显示屏的刷新周期。
可选地，在本申请实施例中，该多帧指纹图像中每帧指纹图像的曝光时间是该显示屏的刷新周期的非整数倍，该方法还包括：根据预设的曝光能量 偏差序列，对该多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，该曝光能量偏差序列包括该首帧指纹图像之后的每帧指纹图像与该首帧指纹图像之间的曝光能量偏差值。
可选地，在本申请实施例中，该获取该用户的多帧指纹图像，包括：在该显示屏的非高亮模式HBM模式下，获取该多帧指纹图像。
可选地，在本申请实施例中，该多帧指纹图像的采样率大于或等于100Hz。
应理解，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
可选地，本申请实施例还提供了一种电子设备，该电子设备包括上述各种实施例中的心率检测装置和显示屏，该心率检测装置设置于该显示屏的下方。
可选地，该电子设备包括但不限于手机、电脑、多媒体机和游戏机。
应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间 的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。

Claims (25)

  1. 一种心率检测装置，应用于具有显示屏的电子设备，其特征在于，包括：
    感光组件，设置于所述显示屏的下方，所述感光组件用于采集所述显示屏发出的经用户的手指反射或散射返回的光信号，以获取多帧指纹图像，所述多帧指纹图像用于获取光电容积脉搏波描记法PPG信号，所述PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差用于得到所述用户的心率变异性HRV信息；
    其中，所述多帧指纹图像中每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
  2. 根据权利要求1所述的心率检测装置，其特征在于，所述感光组件为指纹传感器芯片，所述指纹传感器芯片用于：
    连续采集多帧第一指纹图像，以及
    对所述多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以分别获取多帧第二指纹图像，所述多帧第二指纹图像用于获取所述PPG信号。
  3. 根据权利要求2所述的心率检测装置，其特征在于，所述指纹传感器芯片用于：
    将所述每帧第一指纹图像划分为至少一个子区域；
    将所述每帧第一指纹图像的所述至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，以分别获取所述多帧第二指纹图像。
  4. 根据权利要求3所述的心率检测装置，其特征在于，所述指纹传感器芯片用于：
    对所述每个子区域包括的多个像素数据进行求和处理或求平均处理。
  5. 根据权利要求2所述的心率检测装置，其特征在于，所述指纹传感器芯片用于：
    根据所述每帧第一指纹图像中的部分像素数据，获取所述多帧第二指纹图像。
  6. 根据权利要求1所述的心率检测装置，其特征在于，所述感光组件是由所述电子设备中的指纹传感器芯片中的至少部分像素拼接形成的。
  7. 根据权利要求1所述的心率检测装置，其特征在于，所述感光组件独立于所述电子设备中的指纹传感器芯片设置。
  8. 根据权利要求1至7中任一项所述的心率检测装置，其特征在于，所述多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与所述显示屏的刷新周期的初始时刻同步。
  9. 根据权利要求8所述的心率检测装置，其特征在于，所述多帧指纹图像中每帧指纹图像的曝光时间是所述显示屏的刷新周期的整数倍。
  10. 根据权利要求9所述的心率检测装置，其特征在于，所述每帧指纹图像的曝光时间等于所述显示屏的刷新周期。
  11. 根据权利要求8所述的心率检测装置，其特征在于，所述多帧指纹图像中每帧指纹图像的曝光时间是所述显示屏的刷新周期的非整数倍，所述心率检测装置还包括：
    处理器，用于根据预设的曝光能量偏差序列，对所述多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，所述曝光能量偏差序列包括所述首帧指纹图像之后的每帧指纹图像与所述首帧指纹图像之间的曝光能量偏差值。
  12. 根据权利要求1至11中任一项所述的心率检测装置，其特征在于，所述感光组件用于：
    在所述显示屏的非高亮模式HBM模式下，获取所述多帧指纹图像。
  13. 根据权利要求1至12中任一项所述的心率检测装置，其特征在于，所述多帧指纹图像的采样率大于或等于100Hz。
  14. 一种心率检测方法，应用于具有显示屏的电子设备，其特征在于，包括：
    根据所述显示屏发出的经用户的手指反射或散射返回的光信号，获取所述用户的多帧指纹图像；
    提取所述多帧指纹图像中每帧指纹图像的光电容积脉搏波描记法PPG信号；
    计算所述PPG信号在时域上的相邻波峰间隔或相邻波谷间隔的标准差，得到所述用户的心率变异性HRV信息；
    其中，所述多帧指纹图像中的每帧指纹图像的数据量小于用于指纹识别的指纹图像的数据量。
  15. 根据权利要求14所述的心率检测方法，其特征在于，所述获取所述用户的多帧指纹图像，包括：
    连续采集所述用户的多帧第一指纹图像；
    对所述多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以分别获取多帧第二指纹图像，所述多帧第二指纹图像用于获取所述PPG信号。
  16. 根据权利要求15所述的心率检测方法，其特征在于，所述对所述多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取所述多帧第二指纹图像，包括：
    将所述每帧第一指纹图像划分为至少一个子区域；
    将所述每帧第一指纹图像的所述至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，以分别获取所述多帧第二指纹图像。
  17. 根据权利要求16所述的心率检测方法，其特征在于，所述将所述至少一个子区域中的每个子区域包括的多个像素数据处理成一个数据，包括：
    对所述每个子区域包括的多个像素数据进行求和处理或求平均处理。
  18. 根据权利要求15所述的心率检测方法，其特征在于，所述对所述多帧第一指纹图像中的每帧第一指纹图像进行压缩处理，以获取所述多帧第二指纹图像，包括：
    根据所述每帧第一指纹图像中的部分像素数据，获取所述多帧第二指纹图像。
  19. 根据权利要求14至18中任一项所述的心率检测方法，其特征在于，所述多帧指纹图像中的首帧指纹图像的曝光时间的初始时刻与所述显示屏的刷新周期的初始时刻同步。
  20. 根据权利要求19所述的心率检测方法，其特征在于，所述多帧指纹图像中每帧指纹图像的曝光时间是所述显示屏的刷新周期的整数倍。
  21. 根据权利要求20所述的心率检测方法，其特征在于，所述每帧指纹图像的曝光时间等于所述显示屏的刷新周期。
  22. 根据权利要求19所述的心率检测方法，其特征在于，所述多帧指纹图像中每帧指纹图像的曝光时间是所述显示屏的刷新周期的非整数倍，所述心率检测方法还包括：
    根据预设的曝光能量偏差序列，对所述多帧指纹图像中首帧指纹图像之后的指纹图像进行能量补偿，所述曝光能量偏差序列包括所述首帧指纹图像之后的每帧指纹图像与所述首帧指纹图像之间的曝光能量偏差值。
  23. 根据权利要求14至22中任一项所述的心率检测方法，其特征在于， 所述获取所述用户的多帧指纹图像，包括：
    在所述显示屏的非高亮模式HBM模式下，获取所述多帧指纹图像。
  24. 根据权利要求14至23中任一项所述的心率检测方法，其特征在于，所述多帧指纹图像的采样率大于或等于100Hz。
  25. 一种电子设备，其特征在于，包括所述显示屏和如权利要求1至13中任一项所述的心率检测装置，所述心率检测装置设置于所述显示屏的下方。
PCT/CN2020/137769 2020-12-18 2020-12-18 心率检测装置及方法、电子设备 WO2022126653A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/137769 WO2022126653A1 (zh) 2020-12-18 2020-12-18 心率检测装置及方法、电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/137769 WO2022126653A1 (zh) 2020-12-18 2020-12-18 心率检测装置及方法、电子设备

Publications (1)

Publication Number Publication Date
WO2022126653A1 true WO2022126653A1 (zh) 2022-06-23

Family

ID=82060005

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/137769 WO2022126653A1 (zh) 2020-12-18 2020-12-18 心率检测装置及方法、电子设备

Country Status (1)

Country Link
WO (1) WO2022126653A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830064A (zh) * 2018-06-06 2018-11-16 Oppo（重庆）智能科技有限公司 电子装置以及电子装置的控制方法
CN109558804A (zh) * 2018-11-05 2019-04-02 Oppo广东移动通信有限公司 指纹采集方法及相关产品
US20190139341A1 (en) * 2018-12-28 2019-05-09 Intel Corporation Multi-factor biometric authentication
CN110383286A (zh) * 2019-05-22 2019-10-25 深圳市汇顶科技股份有限公司 用于生物识别的方法、指纹识别装置和电子设备
US20190370518A1 (en) * 2018-05-31 2019-12-05 Qualcomm Incorporated Heart rate and respiration rate measurement using a fingerprint sensor
CN209949215U (zh) * 2019-08-06 2020-01-14 广东以诺通讯有限公司 一种具有心率测试功能的手机

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190370518A1 (en) * 2018-05-31 2019-12-05 Qualcomm Incorporated Heart rate and respiration rate measurement using a fingerprint sensor
CN108830064A (zh) * 2018-06-06 2018-11-16 Oppo（重庆）智能科技有限公司 电子装置以及电子装置的控制方法
CN109558804A (zh) * 2018-11-05 2019-04-02 Oppo广东移动通信有限公司 指纹采集方法及相关产品
US20190139341A1 (en) * 2018-12-28 2019-05-09 Intel Corporation Multi-factor biometric authentication
CN110383286A (zh) * 2019-05-22 2019-10-25 深圳市汇顶科技股份有限公司 用于生物识别的方法、指纹识别装置和电子设备
CN209949215U (zh) * 2019-08-06 2020-01-14 广东以诺通讯有限公司 一种具有心率测试功能的手机

Similar Documents

Publication Publication Date Title
CN111316287B (zh) 用于屏上指纹感测的超薄屏下光学传感器中的透镜-针孔阵列设计
CN111448570B (zh) 利用集成到显示屏的光学检测器对显示屏上或接近显示屏的指纹或其他图案进行光学感测
WO2020133378A1 (zh) 指纹识别装置和电子设备
US10410037B2 (en) Under-screen optical sensor module for on-screen fingerprint sensing implementing imaging lens, extra illumination or optical collimator array
US11663846B2 (en) Fingerprint identification apparatus and electronic device
CN211349388U (zh) 指纹识别装置和电子设备
US11804509B2 (en) Fingerprint detection apparatus and electronic device
CN112674746B (zh) 心率检测装置及方法、电子设备
CN109690567B (zh) 指纹识别装置和电子设备
US11210491B2 (en) Fingerprint sensor under a display module with tilted receiving optics
KR102374723B1 (ko) 광학 지문 장치 및 전자 기기
WO2019178793A1 (zh) 屏下生物特征识别装置和电子设备
WO2021077259A1 (zh) 识别指纹的方法、指纹识别装置和电子设备
WO2020168495A1 (zh) 用于指纹识别的方法、装置和终端设备
WO2020154955A1 (zh) 心率检测的方法、装置和电子设备
CN111108511A (zh) 指纹检测装置和电子设备
CN111108509B (zh) 指纹检测装置和电子设备
CN210605739U (zh) 指纹检测装置和电子设备
WO2020168496A1 (zh) 用于指纹识别的方法、装置和终端设备
WO2021007730A1 (zh) 指纹检测装置和电子设备
CN209911987U (zh) 指纹识别装置和电子设备
WO2022126653A1 (zh) 心率检测装置及方法、电子设备
WO2020061734A1 (zh) 注册指纹的方法、指纹注册装置和电子设备
CN111095287B (zh) 光学指纹装置和电子设备
CN111788574B (zh) 具有集成有光电检测器的tft的oled显示屏

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20965667

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20965667

Country of ref document: EP

Kind code of ref document: A1