WO2020238491A1 - 电子装置的控制方法及电子装置 - Google Patents
电子装置的控制方法及电子装置 Download PDFInfo
- Publication number
- WO2020238491A1 WO2020238491A1 PCT/CN2020/085973 CN2020085973W WO2020238491A1 WO 2020238491 A1 WO2020238491 A1 WO 2020238491A1 CN 2020085973 W CN2020085973 W CN 2020085973W WO 2020238491 A1 WO2020238491 A1 WO 2020238491A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- electronic device
- laser
- display screen
- use state
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000005484 gravity Effects 0.000 claims description 26
- 238000003491 array Methods 0.000 claims description 13
- 230000003287 optical effect Effects 0.000 description 25
- 238000012545 processing Methods 0.000 description 14
- 238000004364 calculation method Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000003860 storage Methods 0.000 description 8
- 238000002834 transmittance Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000002829 reductive effect Effects 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 229910052594 sapphire Inorganic materials 0.000 description 1
- 239000010980 sapphire Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/02—Constructional features of telephone sets
- H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
- H04M1/026—Details of the structure or mounting of specific components
- H04M1/0272—Details of the structure or mounting of specific components for a projector or beamer module assembly
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/02—Constructional features of telephone sets
- H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
- H04M1/026—Details of the structure or mounting of specific components
- H04M1/0266—Details of the structure or mounting of specific components for a display module assembly
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/02—Constructional features of telephone sets
- H04M1/22—Illumination; Arrangements for improving the visibility of characters on dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/02—Constructional features of telephone sets
- H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
- H04M1/0206—Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings
- H04M1/0208—Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings characterized by the relative motions of the body parts
- H04M1/0214—Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/02—Constructional features of telephone sets
- H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
- H04M1/0206—Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings
- H04M1/0241—Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings using relative motion of the body parts to change the operational status of the telephone set, e.g. switching on/off, answering incoming call
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
- H04M1/72454—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/16—Details of telephonic subscriber devices including more than one display unit
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/54—Details of telephonic subscriber devices including functional features of a projector or beamer module assembly
Definitions
- This application relates to the technical field of consumer electronic products, and in particular to a control method of an electronic device and an electronic device.
- the laser projector is generally set on the front shell of the mobile phone, and the laser projector is only used in the front use state where the shooting distance is relatively short.
- the laser projector is only used for the front Set the depth image acquisition in the use state, resulting in fewer scenes where mobile phone users can use the laser projector.
- the embodiments of the present application provide a method for controlling an electronic device and an electronic device.
- the electronic device includes a casing, a first display screen, a second display screen, and a laser projector
- the casing includes a first side and a second side opposite to each other.
- a display screen and the second display screen are respectively arranged on the first surface and the second surface, and the laser projector is arranged on the first surface;
- the control method includes: determining the status of the electronic device Use state; when the electronic device is in the first use state, the laser projector projects the laser in the first mode; and, when the electronic device is in the second use state, the laser projector is in the second mode The laser light is projected, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the electronic device of the present application includes a casing, a first display screen, a second display screen, a laser projector, and a processor.
- the casing includes a first side and a second side opposite to each other.
- the second display screen is respectively arranged on the first surface and the second surface, the laser projector is arranged on the first surface;
- the processor is used to determine the use state of the electronic device;
- the laser The projector is used for projecting laser light in the first mode when the electronic device is in the first use state and used for projecting laser light in the second mode when the electronic device is in the second use state.
- the energy of the laser light is greater than the energy of the laser light projected in the first mode.
- FIG 1 and 2 are schematic diagrams of the structure of an electronic device according to some embodiments of the present application.
- FIG. 3 is a schematic flowchart of a control method of an electronic device according to some embodiments of the present application.
- FIG. 4 is a schematic diagram of the structure of the laser light source of the laser projector of the electronic device in some embodiments of the present application.
- FIG. 5 is a schematic diagram of a three-dimensional structure of an electronic device according to some embodiments of the present application.
- 6 and 7 are schematic flowcharts of control methods of electronic devices in some embodiments of the present application.
- FIG. 8 is a schematic diagram of a three-dimensional structure of an electronic device according to some embodiments of the present application.
- Fig. 9 is a cross-sectional view of the electronic device shown in Fig. 8 taken along line IX-IX.
- 10 to 12 are cross-sectional views of the electronic device according to some embodiments of the present application along the same cross-sectional line as the IX-IX line shown in FIG. 8.
- FIG. 13 and 14 are schematic diagrams of the structure of an electronic device according to some embodiments of the present application.
- 15 and 16 are cross-sectional views of the electronic device according to some embodiments of the present application along the same cross-sectional line as the IX-IX line shown in FIG. 8.
- FIG. 17 is a schematic diagram of a system architecture of an electronic device according to some embodiments of the present application.
- FIG. 18 is a schematic flowchart of a control method of an electronic device according to some embodiments of the present application.
- FIG. 19 is a schematic diagram of the principle of a control method of an electronic device according to some embodiments of the present application.
- FIG. 23 are schematic flowcharts of control methods of electronic devices according to some embodiments of the present application.
- FIG. 24 is a schematic diagram of the principle of a control method of an electronic device according to some embodiments of the present application.
- FIG. 25 is a schematic flowchart of a control method of an electronic device according to some embodiments of the present application.
- the control method of the electronic device 100 of the present application is applicable to the electronic device 100.
- the electronic device 100 includes a casing 11, a first display screen 12, a second display screen 13 and a laser projector 14.
- the casing 11 includes a first surface 111 and a second surface 112 opposite to each other.
- the first display screen 12 and the second display screen 13 are respectively arranged on the first side 111 and the second side 112.
- the laser projector 14 is arranged on the first surface 111, and the control method includes:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the electronic device 100 further includes a gravity sensor 16 to determine the use state of the electronic device (031), including:
- the use state of the electronic device 100 is determined by the gravity sensor 16.
- determining the use state of the electronic device 100 through the gravity sensor 16 includes:
- determining the use status of the electronic device 100 (031) includes:
- the use state of the electronic device is determined by the first touch signal detected by the first display screen 12 and/or the second touch signal detected by the second display screen 13.
- determining the use state of the electronic device 100 through the first touch signal detected by the first display screen 12 and/or the second touch signal detected by the second display screen 13 includes:
- the electronic device 100 further includes a state selection button 17 to determine the use state of the electronic device 100 (031), including:
- the use state of the electronic device 100 is determined by the state selection button.
- the power of laser projector 14 in the first mode is less than the power of laser in the second mode.
- the laser projector 14 includes multiple point light sources 141, and the multiple point light sources 141 are independently controlled;
- the number of point light sources 141 turned on by the laser projector 14 in the first mode is less than the number of point light sources 141 turned on in the second mode.
- the electronic device 100 further includes an image collector 15, and the image collector 15 is arranged on the first surface;
- the laser projector 14 projects laser light to the scene at the first operating frequency
- Control methods also include:
- the image collector 15 acquires and collects images at a second working frequency, and the second working frequency is greater than the first working frequency;
- the collected images distinguish the first image collected when the laser projector 14 is not projecting laser light and the second image collected when the laser projector 14 is projecting laser light;
- the electronic device 100 of the embodiment of the present application includes a casing 11, a first display screen 12, a second display screen 13, a laser projector 14 and a processor 20.
- the casing 11 includes a first surface 111 and a second surface 112 opposite to each other, and the first display 12 and the second display 13 are respectively disposed on the first surface 111 and the second surface 112.
- the laser projector 14 is arranged on the first surface 11.
- the processor 20 is used to determine the use state of the electronic device 100.
- the laser projector 14 is used to project laser light in the first mode when the electronic device 100 is in the first use state, and used to project laser light in the second mode when the electronic device 100 is in the second use state.
- the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the electronic device 100 further includes a gravity sensor 16.
- the processor 20 is used to determine the use state of the electronic device 100 through the gravity sensor 16.
- the processor 20 is also used to:
- the processor 20 is configured to determine the electronic device through the first touch signal detected by the first display screen 12 and/or the second touch signal detected by the second display screen 13 100 use status.
- the processor 20 is used to:
- the first display screen 12 detects the first touch signal, it is determined that the electronic device 100 is in the first use state.
- the second display screen 13 detects the second touch signal, it is determined that the electronic device 100 is in the second use state.
- the electronic device 100 further includes a state selection button 17.
- the processor 20 is configured to determine the use state of the electronic device 100 through the state selection button 17.
- the power of the laser projector 14 to project laser light in the first mode is less than the power of laser light projected in the second mode;
- the laser projector 14 includes a plurality of point light sources 141 that are independently controlled; the number of point light sources 141 turned on in the first mode of the laser projector 14 is less than the number of point light sources 141 turned on in the second mode.
- a plurality of point light sources 141 form a plurality of light emitting arrays 142.
- the multiple light emitting arrays 142 are arranged in a ring shape.
- the electronic device 100 further includes an image collector 15.
- the image collector 15 is arranged on the first surface 111.
- the laser projector 14 projects laser light
- the laser projector 14 is used to project laser light to the scene at a first operating frequency
- the image collector 15 is used to acquire and collect images at a second operating frequency, which is greater than the first operating frequency
- the processor 20 is also used for:
- the collected images distinguish the first image collected when the laser projector 14 is not projecting laser light and the second image collected when the laser projector 14 is projecting laser light;
- the first display screen 12 is an opening screen with an opening at one end
- the second display screen 13 is a full screen
- the laser projector 14 is installed in the opening
- the first display screen 12 and the second display screen 13 are both full screens, and the laser projector 14 is arranged below the first display screen 12.
- the orientation of the laser projector 14 in the first use state and the second use state are opposite.
- the first touch signal and the second touch signal are both capacitive signals.
- the control method of the electronic device 100 of the present application wherein the electronic device 100 includes a casing 11, a first display 12, a second display 13 and a laser projector 14, and the casing 11 includes a corresponding The first surface 111 and the second surface 112 of the back, the first display screen 12 and the second display screen 13 are respectively arranged on the first surface 111 and the second surface 112, and the laser projector 14 is arranged on the first surface 111.
- the control method includes :
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the electronic device 100 of the present application can be used to implement the aforementioned control method.
- the electronic device 100 further includes a processor 20, step 031 can be implemented by the processor 20, and step 032 and step 033 can both be implemented by the laser projector 14.
- the processor 20 can be used to determine the use state of the electronic device 100
- the laser projector 14 can be used to project laser light in the first mode when the electronic device 100 is in the first use state, and used to project the laser in the first mode when the electronic device 100 is in the first use state.
- the laser is projected in the second mode when in use.
- the electronic device 100 may be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (smart watch, smart bracelet, smart helmet, smart glasses, etc.), a virtual reality device, and the like.
- the electronic device 100 is a mobile phone as an example, but the form of the electronic device 100 is not limited to a mobile phone.
- the electronic device 100 further includes an image collector 15 arranged on the first surface 111, and the laser projector 14 is used in conjunction with the image collector 15 to obtain depth information of the object to be measured for use in three-dimensional modeling, generating three-dimensional images, Ranging, etc.
- the laser projector 14 and the image collector 15 can be installed on the bracket, the bracket, the laser projector 14 and the image collector 15 are installed in the casing 11 together.
- the use state of the electronic device 100 includes a first use state and a second use state.
- the first use state may be the front use state.
- the first display screen 12 and the laser projector 14 face the user of the electronic device 100.
- the content displayed on the first display screen 12 can use the laser projector 14 to project laser light toward the user, and the user can use the laser projector 14 (and the image collector 15) to perform face recognition, iris recognition, and the like.
- the second use state may be a rear use state.
- the first display screen 12 and the laser projector 14 are away from the user of the electronic device 100, and the second display screen 13 faces the electronic device 100.
- the user at this time, the user can watch the content displayed on the second display screen 13 and can use the laser projector 14 to project laser light toward the side far from the user.
- the user can use the laser projector 14 (and image capture The device 15) acquires a depth image of the object to be measured on the side of the electronic device 100 opposite to the user.
- the laser projector 14 projecting laser mode includes a first mode and a second mode, wherein the first mode corresponds to the laser projector 14 in the front use state, and the second mode corresponds to the laser projector 14 in the rear.
- the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the power of the laser projector 14 in the first mode may be less than the power of the laser in the second mode, so that the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- the maximum distance (projection distance) that the laser beam projected by the laser projector 14 can reach in the second mode is greater than the maximum distance (projection distance) that the laser beam projected in the first mode can reach.
- the rear distance range that the laser projector 14 and the image collector 15 can detect in the rear use state is greater than the front distance range that the laser projector 14 and the image collector 15 can detect in the front use state, for example :
- the laser projector 14 with the image collector 15 can detect the front distance range within 25cm, and the laser projector 14 with the image collector 15 can detect the rear distance range greater than 25cm (the accuracy of the distance range within 25cm) Very poor);
- the front distance range and the rear distance range slightly overlap.
- the front distance range that the laser projector 14 can detect with the image collector 15 is within 25cm, and the laser projector 14 cooperates with the image collector 15 can detect the rear distance range is greater than 20cm.
- the first display screen 12 and the second display screen 13 are respectively arranged on opposite sides of the casing 11, and the laser projector 14 and the first display screen 12 are arranged on On the same side of the casing 11, and the maximum distance that the laser projector 14 can project in the second use state is greater than the maximum distance that can be projected in the first use state, so that the laser projector 14 can be in the first use state and
- the use in the second use state increases the scene in which the user can use the electronic device 100; at the same time, there is no need to provide two laser projectors 14 on the electronic device 100 to use in the first use state and the second use state, saving The cost of the electronic device 100 is reduced.
- the laser projector 14 includes a laser light source 140
- the laser light source 140 includes a plurality of point light sources 141
- the multiple point light sources 141 can be independently controlled, specifically, each point light source 141 is Can be turned on and off independently.
- the number of point light sources 141 turned on by the laser projector 14 in the first mode is less than the number of point light sources 141 turned on in the second mode.
- each point light source 141 can project the same laser power, so that the second The energy of the laser light projected in the mode is greater than the energy of the laser light projected in the first mode, and the maximum distance that the laser projected by the laser projector 14 in the second mode can reach is greater than the maximum distance that the laser projected in the first mode can reach.
- multiple point light sources 141 form multiple light-emitting arrays 142, and multiple light-emitting arrays 142 are independently controlled. Specifically, the multiple point light sources 141 in each light emitting array 142 can be turned on and off at the same time. At this time, the power of the multiple point light sources 141 in each light emitting array 142 may be the same. In other embodiments, the multiple point light sources 141 in each light-emitting array 142 can also be independently controlled.
- the multiple light emitting arrays 142 are arranged in a ring shape.
- the laser light emitted by the point light sources 141 in the circularly arranged light emitting array 142 can cover a wider field of view, so that more depth information of the object to be measured can be obtained.
- the ring shape can be a square ring or a circular ring.
- the light-emitting array 142 is turned on in a manner that the light-emitting array 142 farther from the center of the laser light source 140 is turned on first.
- the total number of light-emitting arrays 142 is 6, and the 6 light-emitting arrays 142 include 5 ring-shaped sub-arrays 143 and 1 square sub-array 144.
- 5 ring-shaped sub-arrays The arrays 143 are arranged in sequence, and the numbers of the five annular sub-arrays 143 arranged in sequence are A, B, C, D, and E.
- the laser projector 14 turns on the point light sources 141 in the ring sub-array 143 numbered A and B in the first mode; the laser projector 14 can turn on the point light sources numbered A, B, and C in the second mode.
- the point light source 141 in the ring sub-array 143, or the point light source 141 in the ring sub-array 143 numbered A, B, C, and D is turned on, or the ring sub-array 143 numbered A, B, C, D, and E is turned on
- the point light sources 141 in the internal, or the point light sources 141 in the ring sub-array 143 numbered A, B, C, D, and E and the point light sources 141 in the square sub-array 144 are turned on.
- the number of point light sources 141 that are turned on in the first mode of the laser projector 14 is less than the number of point light sources 141 that are turned on in the second mode, so that the electronic device 100 can project when it is in the first use state.
- the energy of the laser light is less than the energy of the laser light projected in the second use state.
- the diffraction ability of the diffractive optical element 146 (shown in FIG. 9) of the laser projector 14 is limited, that is, part of the laser light emitted by the laser light source 140 is not diffracted but is directly emitted, and the directly emitted laser light does not pass through diffractive optics.
- the diffraction attenuation effect of element 146 is relatively large, which is very likely to cause harm to the user's eyes. Therefore, the laser projector 14 projects laser light in the first mode when the electronic device 100 is in the first use state, that is, the laser projector 14 first turns on the annular sub-array 143 away from the center of the laser light source 140 when the projection distance is small.
- the laser projector 14 is in the second use of the electronic device 100 In the state, the laser is projected in the second mode, that is, the laser projector 14 simultaneously turns on the ring sub-array 143 away from the center of the laser light source 140 and the ring sub-array 143 near the center of the laser light source 140 when the projection distance is large, so that The maximum distance that the laser beam projected by the laser projector 14 can reach is increased.
- the electronic device 100 further includes a gravity sensor 16.
- the determination of the use state of the electronic device 100 can be achieved by the gravity sensor 16.
- the control method includes:
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- step 0611, step 0612, and step 0613 can be sub-steps of step 031 described above
- step 062 is basically the same as step 032 described above
- step 063 is the same as the previous step.
- the step 033 is basically the same.
- the processor 20 is electrically connected to the gravity sensor 16.
- the processor 20 can also be used to determine the use state of the electronic device 100 through the gravity sensor 16, and the processor 20 can also be used to implement step 0611, step 0612, and step 0613.
- the processor 20 may also be used to obtain the z-axis value of the gravity sensor 16, and when the z-axis value is greater than a first preset value, determine that the electronic device 100 is in the first use state and is in the z-axis When the value of is less than the second preset value, it is determined that the electronic device 100 is in the second use state.
- the gravity sensor 16 includes a three-axis acceleration sensor 16.
- FIG. 5 is a schematic diagram of the principle of determining the use state of the electronic device 100 by the three-axis acceleration sensor 16.
- the x-axis, y-axis, and z-axis are all relative to the position of the casing 11.
- the direction of the y-axis is the upward direction of the casing 11 and the direction of the x-axis is the rightward direction of the casing 11
- the direction of the z-axis is perpendicular to the outward direction of the first surface 111.
- the value of the z-axis (acceleration value) corresponds to the orientation of the first display screen 12.
- the x-axis value defaults to 0, and the y-axis value defaults to 0.
- the z-axis value defaults to 9.81m/s 2 ; when the electronic device 100 is placed on the desktop with the first display screen 12 facing down, the x-axis value defaults to 0, the y-axis value defaults to 0, and the z-axis value
- the value is -9.81m/s 2 ; when the electronic device 100 is tilted to the left, the x-axis value is positive; when the electronic device 100 is tilted to the right, the x-axis value is negative; when the electronic device 100 is tilted When tilting upward, the value of the y-axis is negative; when tilting the electronic device 100 downward, the value of the y-axis is positive.
- the z-axis value of the gravity sensor 16 acquired by the processor 20 is greater than the first preset value, for example, the z-axis value of the gravity sensor 16 acquired by the processor 20 is 6 m/s 2 , which is greater than the first preset value.
- the processor 20 determines that the electronic device 100 is in the first position of use; If the processor 20 obtains a gravity sensor value of the z-axis 16 than the second preset value, e.g., z-axis values to the processor 20 acquires the gravity sensor 16 -6m / s 2, less than the second predetermined value / 2 -3m s, it is determined The second display screen 13 faces the user of the electronic device 100. At this time, the processor 20 determines that the electronic device 100 is in the second use state.
- the electronic device 100 and the control method of the embodiment of the present application determine the use state of the electronic device 100 through the gravity sensor 16, so that the laser projector 14 can be used to project laser light in a corresponding mode without the user manually selecting the use state of the electronic device 100.
- the user experience of the electronic device 100 is improved.
- the first display screen 12 and the second display screen 13 are both touch-sensitive display screens.
- the determination of the use state of the electronic device 100 can be achieved by the first touch signal detected by the first display screen 12 and/or the second touch signal detected by the second display screen 13.
- the control method includes:
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode.
- step 0711 and step 0712 can be sub-steps of step 031 described above
- step 072 is basically the same as step 032 described above
- step 073 is the same as the above described step.
- Step 033 is basically the same.
- the processor 20 may also be configured to determine the use state of the electronic device 100 through the first touch signal detected by the first display screen 12 and/or the second touch signal detected by the second display screen 13. 20 can also be used to implement step 0711 and step 0712.
- the processor 20 may also be used to determine that the electronic device 100 is in the first use state and the second touch signal is detected on the second display screen 13 when the first display screen 12 detects the first touch signal. When it is determined that the electronic device 100 is in the second use state.
- Both the first touch signal and the second touch signal may be capacitive signals.
- the processor 20 is electrically connected to the first display screen 12 and the second display screen 13.
- the processor 20 can also obtain the first touch signal detected by the first display screen 12 and the second touch signal detected by the second display screen 13 signal.
- the capacitance signal of the first display screen 12 changes and is detected by the first display screen 12 (that is, the first display screen 12 detects the first touch). Control signal).
- the first display screen 12 faces the user of the electronic device 100 and is used by the user, and the processor 20 determines that the electronic device 100 is in the first use state.
- the capacitance signal of the second display screen 13 changes and is detected by the second display screen 13 (that is, the second display screen 13 detects the second touch Control signal).
- the second display screen 13 faces the user of the electronic device 100 and is used by the user, and the processor 20 determines that the electronic device 100 is in the second use state.
- the electronic device 100 and the control method of the embodiment of the present application determine the use status of the electronic device 100 through the first display screen 12 or the second display screen 13, so that no additional hardware is required on the electronic device 100, which saves the electronic device 100. Cost of production.
- the electronic device 100 further includes a state selection button 17 electrically connected to the processor 20, and the determination of the use state of the electronic device 100 (step 010) can be achieved by the state selection button 17
- the processor 20 may also be used to determine the use state of the electronic device 100 through the state selection button 17.
- the state selection button 17 may be a physical button and includes a first state button 171 and a second state button 172. When the processor 20 detects that the first state button 171 is triggered, the processor 20 determines that the electronic device 100 is in the first use state; when the processor 20 detects that the second state button 172 is triggered, the processor 20 determines that the electronic device 100 In the second use state.
- the state selection button 17 may also be a virtual button, and the state selection button 17 may be displayed by the first display screen 12 and the second display screen 13.
- the state selection button 17 may be the first display screen 12 or the second display screen.
- the state switching button displayed on the second display screen 13 is used to switch between the first use state and the second use state.
- the electronic device 100 and the control method of the embodiment of the present application determine the use state of the electronic device 100 through the state selection button 17, so that the user can accurately select the required use state according to needs.
- the electronic device 100 further includes a visible light camera 40.
- the visible light camera 40 includes a main camera 41 and a sub camera 42. Both the main camera 41 and the sub camera 42 are installed on the first side of the casing 11. 111 upper inner.
- the main camera 41 can be a wide-angle camera.
- the wide-angle camera has low sensitivity to motion. The relatively low shutter speed can ensure the sharpness of image shooting.
- the wide-angle camera has a large field of view, which can cover a wide range of scenes, and can emphasize the foreground and highlight Comparison of near and far.
- the secondary camera 42 may be a telephoto camera, and the telephoto lens of the telephoto camera can identify further objects.
- the main camera 41 is a color camera
- the secondary camera 42 is a black and white camera.
- the number of cameras here can be multiple, three or four or more.
- the laser projector 14, the sub camera 42, the main camera 41, and the image collector 15 are sequentially arranged at intervals and located on the same straight line.
- the first display screen 12 is installed on the first side 111 of the casing 11.
- the first display screen 12 can cover 85% or more of the area of the first side 111, for example, 85%, 86%. %, 87%, 88%, 89%, 90%, 91%, 92%, 93%, 95% or even 100%.
- the second display screen 13 is installed on the second surface 112 of the casing 11.
- the second display screen 13 can cover 85% or more of the area of the second surface 112, such as 85%, 86%, 87%, 88%, 89%, 90%, 91%, 92%, 93%, 95% or even 100%.
- Both the first display screen 12 and the second display screen 13 can be used to display images, and the images can be text, images, videos, icons and other information.
- the first display screen 12 includes a display area 122, and the display area 122 can be used to display images.
- the shape of the display area 122 can be a circle, an oval, a racetrack, a rounded rectangle, a rectangle, etc.
- the display area 122 is formed with a front 123 and a back 124 opposite to each other.
- the display area 122 can be used to display images.
- the light emitted by the first display screen 12 is emitted along the direction from the back 124 to the front 123.
- the electrons When the device 100 is in the first use state (the user is on the side of the first surface 111), the light can be received by the user after passing through the front 123, that is, the user can observe the image displayed on the first display screen 12 from the front 123.
- the front surface 123 is the display surface
- the back surface 124 is the surface opposite to the display surface. Both the front surface 123 and the back surface 124 may be flat or curved.
- the first display screen 12 may further include a non-display area, and the non-display area may be formed on the periphery of the display area 122.
- the non-display area may not be used for display, and the non-display area may be used for combining with the casing 11 or for wiring.
- the non-display area and the casing 11 may be combined with glue without affecting the display function of the display area 122.
- the first display screen 12 may also be a touch display screen integrated with a touch function. After the user obtains the image information displayed on the first display screen 12, the user can perform a touch control on the first display screen 12 to achieve predetermined interactive operations. .
- the display area 122 includes a plurality of pixels, and the plurality of pixels are arranged in a predetermined manner, and there are microscopic gaps between adjacent pixels.
- the display area 122 includes a first sub-display area 1221 and a second sub-display area 1222.
- the pixel density of the first sub-display area 1221 is smaller than the pixel density of the second sub-display area 1222.
- the pixel density of the first sub-display area 1221 is less than the pixel density of the second sub-display area 1222, that is, the microscopic gap of the first sub-display area 1221 is larger than the microscopic gap of the second sub-display area 1222.
- the blocking effect of light is relatively small, and the transmittance of light passing through the first sub-display area 1221 is relatively high.
- the pixel density of the first sub-display area 1221 may also be greater than the pixel density of the second sub-display area 1222, or the pixel density of the first sub-display area 1221 may also be equal to the pixel density of the second sub-display area 1222 .
- the shapes of the first sub-display area 1221 and the second sub-display area 1222 can be set according to specific requirements, and are not limited here.
- the first sub-display area 1221 can be set in a racetrack shape or a water drop shape.
- the second sub-display area 1222 and the first sub-display area 1221 may be complementary and jointly form the display area 122 in the shape of a rectangle or a rounded rectangle.
- the first sub-display area 1221 may be located near the edge of the display area 122, and the second sub-display area 1222 may be located in the middle of the display area 122.
- the first sub-display area 1221 may be used to display status icons of the electronic device 100.
- the first sub-display area 1221 may be used to display the battery power, network connection status, system time, etc. of the electronic device 100.
- the first display screen 12 and the second display screen 13 are both full screens.
- the laser projector 14 is arranged on the side where the back 124 of the first display screen 12 is located.
- the laser projector 14 is arranged under the display area 122 (for example, it can be arranged under the first sub-display area 1221). Used to emit laser light passing through the display area 122.
- the laser projector 14 may include a laser light source 140, a collimating element 145, and a diffractive optical element 146.
- the light (for example, infrared laser) emitted by the laser light source 140 is first collimated by the collimating element 145, and then diffracted by the diffractive optical element 146.
- Diffraction occurs when light passes through the microscopic gap, that is, the microscopic gap of the display area 122 and the diffractive structure on the diffractive optical element 146 both have a diffractive effect on the light emitted by the laser light source 140.
- the laser light passing through the display area 122 and entering the outside may include a pattern diffracted by the diffractive optical element 146 (the pattern includes a plurality of spots diffracted by the diffractive optical element 146) and the microscopic gap of the first display screen 12.
- the pattern formed by diffraction (the pattern includes a plurality of spots diffracted by the diffractive optical element 146 and then diffracted by the first display screen 12), so that the speckle pattern after passing through the display area 122 has a higher irrelevance, which is beneficial for subsequent Process the obtained speckle pattern.
- the transmittance of the display area 122 can reach 60% or more, so that the laser light emitted by the laser projector 14 passes through the display area 122 with less loss.
- the image collector 15 can be an infrared camera.
- the laser is emitted to the object to be measured. After being modulated by the object to be measured, it can be obtained by the image collector 15.
- the image collector 15 obtains the speckle image after receiving the modulated laser. After processing, the depth data of the object to be measured is obtained.
- the image collector 15 can also be arranged on the side where the back 124 of the first display screen 12 is located, that is, under the first display screen 12 (for example, it can be arranged under the first sub-display area 1221). At this time, the light incident surface of the image collector 15 can be aligned with the display area 122, and the laser light modulated by the object to be measured passes through the display area 122 and then received by the image collector 15. Specifically, the laser light modulated by the object to be measured It may be diffracted by the microscopic gap of the first display screen 12 and then received by the image collector 15.
- the electronic device 100 further includes a floodlight 50, the floodlight 50 can be used to emit supplementary light, the supplementary light can be used to supplement the environment when the ambient light is weak Light intensity.
- the supplementary light may be infrared light. After the supplementary light is emitted to the object to be measured and reflected by the object to be measured, it can be acquired by the image collector 15 to obtain a two-dimensional image of the object to be measured, and the two-dimensional image information can be used for identification.
- the floodlight 50 can also be arranged on the side where the back 124 of the first display screen 12 is located, that is, under the first display screen 12 (for example, it can be arranged under the first sub-display area 1221), specifically, the floodlight 50 can be installed on the same bracket as the laser projector 14 and the image collector 15. At this time, the supplementary light emitted by the floodlight 50 passes through the microscopic gap of the display area 122 and enters the external environment, and the reflected supplementary light may pass through the microscopic gap again to be received by the image collector 15.
- the laser projector 14 and the image collector 15 are arranged on the side where the back 124 of the first display screen 12 is located, the first display screen 12 does not need to be aligned with the laser projector 14 and the image collector 15 Opening, the screen occupancy of the electronic device 100 is relatively high.
- the laser projector 14 and the image collector 15 are arranged below the first display screen 12. Compared with the flight time that can collect depth information, the depth obtained by the laser projector 14 and the image collector 15 is The resolution of the image is higher.
- the use of a higher resolution depth image in face recognition can improve the accuracy of face recognition, and the use of a higher resolution depth image in 3D scene modeling can improve the modeled 3D scene The degree of match before the actual scene.
- the laser beam projected by the laser projector 14 will be affected by the microscopic gaps in the first display screen 12 when passing through the first sub-display area 1221, that is, projected into the scene
- the spots in the speckle image include both the spots formed by the first diffraction of the laser light by the diffractive optical element 146 of the laser projector 14 and the spots formed by the first diffracting of the laser light by the diffractive optical element 146 and then the second diffraction by the first display screen 12.
- the image collector 15 is arranged below the first display screen 12, and the image collector 15 receives the laser light that is diffracted by the first display screen 12 when the object to be measured passes through the first display screen 12, then the image collector 15
- the spots in the collected speckle image also include spots where the laser light is only diffracted once by the diffractive optical element 146 and reflected by the object to be measured, and the laser light is diffracted once by the diffractive optical element 146 and then diffracted twice by the first display screen 12 and is The spot formed by the reflection of the measured object and the spot formed by the first diffracted laser by the diffractive optical element 146 and the second diffracted by the first display screen 12, and the spot formed by the first display screen 12 once again after being reflected by the test object.
- the following two calculation methods are included:
- the reference spot in the reference image also includes the reference spot formed by the laser being diffracted only once by the diffractive optical element 146 and reflected by the calibration object, and the laser is diffracted once by the diffractive optical element 146 and then diffracted twice by the first display screen 12
- the reference spots in the reference image only include the reference spots formed by the laser being diffracted only once by the diffractive optical element 146 and reflected by the calibration object.
- the brightness of the spots formed after different diffraction orders is different, therefore, various kinds of spots can be distinguished by brightness, so that unnecessary spots can be filtered out.
- the first display screen 12 is an opening screen with an opening 121 at one end, the laser projector 14 is inserted into the opening 121 for installation, and the image collector 15 is installed on the second display screen 12.
- the laser projector 14 is arranged under the first display screen 12, and the laser beam projected by the laser projector 14 passes through the first display screen 12 without being affected by the microscopic gap of the first display screen 12 (that is, the first display screen 12).
- 12 is formed with the opening 121 shown in FIG. 5), and the speckle image collected by the image collector 15 located below the first display screen 12 also includes the laser light which is only diffracted by the diffractive optical element 146 once and reflected by the object to be measured.
- the reference spot in the reference image also includes the reference spot formed by the laser being diffracted only once by the diffractive optical element 146 and reflected by the calibration object, and the laser is diffracted once by the diffractive optical element 146 and reflected by the calibration object and then reflected by the first display screen. 12 Reference spots formed by second-order diffraction.
- the reference spots in the reference image only include the reference spots formed by the laser beam being diffracted only once by the optical element 213 and reflected by the calibration object.
- the brightness of the spots formed after different diffraction orders is different, therefore, various kinds of spots can be distinguished by brightness, so that unnecessary spots can be filtered out.
- the first display screen 12 is formed with an opening 121, and the opening 121 does not have a display function.
- the opening 121 penetrates the front surface 123 and the back surface 124. While the laser projector 14 is arranged on the side where the back 124 of the first display screen 12 is located, the laser projector 14 is aligned with the opening 121, and the laser projector 14 is used to emit laser light passing through the opening 121.
- the opening 121 may be opened on the first sub-display area 1221.
- the light-emitting surface of the laser projector 14 can be aligned with the opening 121, and the laser light emitted by the laser projector 14 passes through the opening 121 and enters the outside.
- the laser light entering the outside does not need to pass through the microscopic gap of the display area 122, and will not be diffracted again by the microscopic gap, so that after the speckle image is acquired by the image collector 15, the subsequent calculation of the depth image based on the speckle image can be reduced.
- the difficulty of processing is not need to pass through the microscopic gap of the display area 122, and will not be diffracted again by the microscopic gap, so that after the speckle image is acquired by the image collector 15, the subsequent calculation of the depth image based on the speckle image can be reduced. The difficulty of processing.
- the opening 121 includes a notch formed on the edge of the first display screen 12, in other words, the opening 121 intersects the edge of the first display screen 12.
- the notch may be formed on any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the first display screen 12.
- the shape of the gap can be any shape such as a triangle, a semicircle, a rectangle, a racetrack, etc., and there is no limitation here.
- the opening 121 includes a through hole spaced from the edge of the first display screen 12, in other words, the opening 121 is opened in a range enclosed by the edge of the first display screen 12.
- the through hole may be close to any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the first display screen 12.
- the shape of the through hole can be any shape such as a triangle, a circle, a rectangle, a racetrack, etc., which is not limited here.
- the electronic device 100 further includes a cover plate 10, and the cover plate 10 is disposed on the side where the front surface 123 of the first display screen 12 is located.
- the first display screen 12 is provided with an opening 121, and the laser projector 14 and the image collector 15 are aligned with the opening 121, the area of the cover plate 10 corresponding to the laser projector 14 and the image collector 15 is provided with an infrared transparent ⁇ 61 ⁇ Over 61.
- the number of openings 121 may be one or two.
- the cover plate 10 can be made of a material with good light transmission properties such as glass or sapphire.
- the infrared permeable layer 61 can be an infrared permeable ink or an infrared permeable film.
- the infrared permeable layer 61 has high transmittance to infrared light (for example, light with a wavelength of 940 nanometers), for example, the transmittance can reach 85% Or above, and the transmittance of light other than infrared light is low or the light other than infrared light cannot be transmitted at all. Therefore, it is difficult for the user to see the laser projector 14 or the image collector 15 aligned with the opening 121 through the cover plate 10, and the appearance of the electronic device 100 is more beautiful.
- the electronic device 100 further includes a cover plate 10, the cover plate 10 is disposed on the side where the front 123 of the first display screen 12 is located, and the cover plate 10 corresponds to the laser projector 14
- An infrared antireflection film 62 is formed in the area.
- the infrared antireflection film 62 can increase the transmittance of infrared light.
- the infrared antireflection film 62 can increase the transmittance of the infrared laser through the cover 10 to reduce the infrared laser passing through the cover.
- the loss of the board 10 in turn reduces the power consumption of the electronic device 100.
- the infrared antireflection film 62 can be plated on the upper surface, or the lower surface of the cover plate 10, or both.
- an infrared anti-reflection film 62 may also be formed on the area of the cover plate 10 corresponding to the image collector 15 to reduce the loss of external infrared light passing through the cover plate 10 before reaching the image collector 15.
- An infrared anti-reflection film 62 may also be formed on the area of the cover plate 10 corresponding to the floodlight 50 to reduce the loss of the supplementary light emitted by the floodlight 50 when passing through the cover plate 10.
- a visible light anti-reflection film 63 may be provided on the cover 10 corresponding to the visible light camera 40 (including the main camera 41 and the secondary camera 42) to reduce the loss of external visible light passing through the cover 10 before reaching the visible light camera 40.
- the laser projector 14 includes a laser light source 140 and a first driver 147, and the first driver 147 can be used to drive the laser light source 140 to project laser light to the object to be measured.
- Both the laser projector 14 and the image collector 15 are connected to the processor 20.
- the processor 20 may provide an enable signal for the laser projector 14, specifically, the processor 20 may provide an enable signal for the first driver 147.
- the image collector 15 is connected to the processor 20 through an I2C bus.
- the laser projector 14 can emit laser light, such as an infrared laser, which is reflected after reaching an object in the scene. The reflected laser light can be received by the image collector 15, and the processor 20 can be based on the laser light and image emitted by the laser projector 14.
- the laser light received by the collector 15 calculates the depth information of the object.
- the laser projector 14 and the image collector 15 can obtain the depth information through the Time of Flight (TOF) ranging method.
- TOF Time of Flight
- the laser projector 14 and the image collector 15 can use the structure The principle of optical ranging to obtain depth information.
- the description of this application takes the laser projector 14 and the image collector 15 to obtain depth information through the structured light ranging principle as an example.
- the laser projector 14 is an infrared laser transmitter
- the image collector 15 is an infrared camera.
- the image collector 15 can control the projection timing of the laser projector 14 through a strobe signal (strobe signal), where the strobe signal is based on the image collector 15
- strobe signal can be regarded as an electrical signal with alternating high and low levels, and the laser projector 14 projects laser light according to the laser projection timing indicated by the strobe.
- the processor 20 may send an image acquisition instruction through the I2C bus to activate the laser projector 14 and the image collector 15 and make the laser projector 14 and the image collector 15 work. After the image collector 15 receives the image acquisition instruction, it controls the switching device 30 through the strobe signal.
- the switching device 30 sends the first pulse signal (pwn1) to the first driver 147, and the first driver 147
- the first pulse signal drives the laser light source 140 to project laser light into the scene.
- the switching device 30 stops sending the first pulse signal to the first driver 147, and the laser light source 140 does not project laser light; or, it can be When the strobe signal is low, the switching device 30 sends a first pulse signal to the first driver 147.
- the first driver 147 drives the laser light source 140 to project laser light into the scene according to the first pulse signal.
- the switch stops sending the first pulse signal to the first driver 147, and the laser light source 140 does not project laser light.
- the strobe signal may not be used when the image collector 15 and the laser projector 14 cooperate.
- the processor 20 sends an image acquisition command to the image collector 15 and simultaneously sends a laser projection command to the first driver 147.
- the image collector 15 After the image collector 15 receives the image acquisition instruction, it starts to acquire the acquired image.
- the first driver 147 receives the laser projection instruction, it drives the laser light source 140 to project laser light.
- the laser projector 14 projects the laser, the laser forms a laser pattern with spots and projects it on the object to be measured in the scene.
- the image collector 15 collects the laser pattern reflected by the object to be measured to obtain a speckle image, and sends the speckle image to the processor 20 through the Mobile Industry Processor Interface (MIPI).
- MIPI Mobile Industry Processor Interface
- the processor 20 may calculate the depth image based on the speckle image and the reference image pre-stored in the processor 20
- the visible light camera 40 is also connected to the processor 20 through the I2C bus, that is, both the main camera 41 and the secondary camera 42 can be connected to the processor 20 through the I2C bus.
- the visible light camera 40 can be used to collect visible light images; that is to say, the main camera 41 and the secondary camera 42 can be used to collect visible light images respectively, or the main camera 41 and the secondary camera 42 cooperate and are used to collect visible light images; in other words, Either or both of the main camera 41 and the sub camera 42 may be used to collect visible light images.
- the visible light camera 40 (the main camera 41 and/or the secondary camera 42) sends a frame of visible light image to the processor 20, the processor 20 receives a data stream.
- the visible light camera 40 can be used alone, that is, when the user only wants to obtain visible light images, the processor 20 sends an image acquisition instruction to the visible light camera 40 (either or both of the main camera 41 and the sub camera 42) through the I2C bus to enable The visible light camera 40 makes it work.
- the visible light camera 40 collects the visible light image of the scene after receiving the image acquisition instruction, and sends the visible light image to the processor 20 through the mobile industry processor interface.
- the visible light camera 40 (any one of the main camera 41 and the secondary camera 42, or the main camera 41 and the secondary camera 42) can also be used in conjunction with the laser projector 14 and the image collector 15, that is, the user wants to use the visible light image with
- the depth image acquires a three-dimensional image
- the image collector 15 and the visible light camera 40 realize hardware synchronization through a sync signal.
- the processor 20 sends an image acquisition instruction to the image acquisition device 15 through the I2C bus.
- the image collector 15 After the image collector 15 receives the image acquisition instruction, it can control the switching device 30 through the strobe signal to send the first pulse signal (pwn1) to the first driver 147, so that the first driver 147 drives the laser light source 140 to emit laser light according to the first pulse signal. ; At the same time, the image collector 15 and the visible light camera 40 are synchronized by a sync signal, which controls the visible light camera 40 to collect visible light images.
- the electronic device 100 may further include a floodlight 50 arranged on the first surface 111.
- the floodlight 50 can emit uniform surface light into the scene.
- the floodlight 50 includes a floodlight source 51 and a second driver 52.
- the second driver 52 can be used to drive the floodlight source 51 to emit uniform surface light.
- the light emitted by the floodlight 50 may be infrared light or other invisible light, such as ultraviolet light. This application takes the infrared light emitted by the floodlight 50 as an example for description, but the form of the light emitted by the floodlight 50 is not limited to this.
- the floodlight 50 is connected to the processor 20, and the processor 20 can provide an enable signal for driving the floodlight 50. Specifically, the processor 20 can provide an enable signal for the second driver 52.
- the floodlight 50 can work with the image collector 15 to collect infrared images.
- the image collector 15 can use a strobe signal (strobe signal, the strobe signal and the image collector 15 control the strobe signal of the laser projector 14 into two Independent strobe signal) controls the emission timing of the infrared light emitted by the floodlight 50.
- the strobe signal is generated according to the timing of the image acquisition by the image collector 15 and the strobe signal can be regarded as an electrical signal with alternating high and low levels. 50 emit infrared light according to the infrared light emission timing indicated by the strobe signal.
- the processor 20 may send an image acquisition instruction to the image acquisition device 15 through the I2C bus. After the image acquisition device 15 receives the image acquisition instruction, it controls the switching device 30 through the strobe signal. If the strobe signal is high, the switching device 30 sends a second pulse signal (pwn2) to the second driver 52. The second driver 52 controls the flood light source 51 to emit infrared light according to the second pulse signal.
- the switching device 30 stops sending the second pulse Signal to the second driver 52, the flood light source 51 does not emit infrared light; or, when the strobe signal is low, the switching device 30 sends the second pulse signal to the second driver 52, and the second driver 52
- the pulse signal controls the flood light source 51 to emit infrared light.
- the switching device 30 stops sending the second pulse signal to the second driver 52, and the flood light source 51 does not emit infrared light.
- the image collector 15 When the floodlight 50 emits infrared light, the image collector 15 receives the infrared light reflected by objects in the scene to form an infrared image, and sends the infrared image to the processor 20 through the mobile industrial processor interface. Each time the image collector 15 sends a frame of infrared image to the processor 20, the processor 20 receives a data stream. This infrared image is usually used for iris recognition, face recognition, etc.
- the electronic device 100 further includes an image collector 15, the image collector 15 is arranged on the first surface 111, when the laser projector 14 projects laser (laser projector 14 When the laser is projected in the first mode, or when the laser projector 14 projects the laser in the second mode), the laser projector 14 projects the laser to the scene at the first operating frequency.
- the control method further includes:
- the image collector 15 acquires a captured image at a second operating frequency, and the second operating frequency is greater than the first operating frequency;
- 0185 In the collected images, distinguish the first image collected when the laser projector 14 is not projecting laser light and the second image collected when the laser projector 14 is projecting laser light;
- 0186 Calculate the depth image based on the first image, the second image, and the reference image.
- control methods include:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires a captured image at a second operating frequency, and the second operating frequency is greater than the first operating frequency;
- 0185 In the collected images, distinguish the first image collected when the laser projector 14 is not projecting laser light and the second image collected when the laser projector 14 is projecting laser light;
- 0186 Calculate the depth image based on the first image, the second image, and the reference image.
- step 0181 is basically the same as step 031 described above
- step 0182 is basically the same as step 032 described above
- step 0183 is basically the same as step 033 described above.
- the image collector 15 can be used to implement step 0184
- the processor 20 can also be used to implement step 0185 and step 0186.
- the image collector 15 is used to obtain the captured image at the second operating frequency
- the processor 20 is also used to distinguish the first image captured when the laser projector 14 is not projecting laser light from the first image captured by the laser projector 14 14
- the second image collected when the laser is projected, and the depth image is calculated based on the first image, the second image, and the reference image.
- the processor 20 simultaneously sends an image acquisition instruction for acquiring a depth image to the image acquisition device 15 and the first driver 147 via the I2C bus.
- the first driver 147 drives the laser light source 140 to emit infrared laser light to the scene at the first operating frequency; after receiving the image acquisition instruction, the image acquisition device 15 acquires at the second operating frequency and is reflected back by objects in the scene Infrared laser to obtain the captured image. For example, as shown in FIG.
- the solid line represents the timing of the laser projector 14 emitting laser
- the dashed line represents the timing of the image capture 15 to acquire the captured image and the number of frames of the captured image
- the dash-dotted line represents the timing obtained from the first image and the second image.
- the number of frames of the third image, from top to bottom in FIG. 19, is a solid line, a dashed line, and a dot-dash line in order, wherein the second operating frequency is twice the first operating frequency.
- the image collector 15 first receives infrared light in the environment (hereinafter referred to as ambient infrared light) when the laser projector 14 is not projecting laser light to obtain the Nth frame of acquisition image (this time is the first An image, which can also be called a background image), and sends the Nth frame of the captured image to the processor 20 through the mobile industrial processor interface; then, the image collector 15 can receive the ambient infrared light and the ambient infrared light when the laser projector 14 projects laser light.
- ambient infrared light in the environment
- the image collector 15 can receive the ambient infrared light and the ambient infrared light when the laser projector 14 projects laser light.
- the infrared laser emitted by the laser projector 14 is used to acquire the N+1th frame of acquisition image (this time is the second image, which can also be called the interference speckle image), and the N+1th frame is acquired through the mobile industry processor interface
- the image is sent to the processor 20; then, the image collector 15 receives ambient infrared light when the laser projector 14 is not projecting laser light to obtain the N+2 frame of the captured image (the first image at this time), and processes it through the mobile industry
- the device interface sends the N+2th frame captured image to the processor 20, and so on, the image collector 15 alternately obtains the first image and the second image.
- the processor 20 sends a collection instruction for acquiring a depth image to the image collector 15 via the I2C bus.
- the image collector 15 After the image collector 15 receives the image acquisition instruction, it sends a first pulse signal to the first driver 147 through the strobe signal control switch, and the first driver 147 drives the laser light source 140 to project laser light at the first operating frequency according to the first pulse signal.
- the laser projector 14 projects laser light at the first working frequency, and the image collector 15 collects the infrared laser light reflected back by the objects in the scene at the second working frequency to obtain collected images. As shown in FIG.
- the solid line represents the timing of laser emission from the laser projector 14
- the dashed line represents the timing of the image acquisition by the image collector 15 and the number of frames of the acquired image
- the dash-dotted line represents the data obtained from the first image and the second image.
- the number of frames of the third image, from top to bottom in FIG. 19, is a solid line, a dashed line, and a dot-dash line in order, wherein the second operating frequency is twice the first operating frequency.
- the image collector 15 first receives ambient infrared light when the laser projector 14 is not projecting laser light to obtain the Nth frame of the collected image (this time is the first image, which can also be called the background image) ), and send the Nth frame captured image to the processor 20 through the mobile industry processor interface; then, the image collector 15 can receive the ambient infrared light and the infrared laser emitted by the laser projector 14 when the laser projector 14 projects the laser In order to obtain the N+1th frame of the collected image (the second image at this time, which can also be called the interference speckle image), and send the N+1th frame of the collected image to the processor 20 through the mobile industry processor interface; then, The image collector 15 then receives ambient infrared light when the laser projector 14 is not projecting laser light to obtain the N+2 frame of the captured image (the first image at this time), and captures the N+2 frame through the mobile industrial processor interface The image is sent to the processor 20, and so on, the image collector
- the image collector 15 may simultaneously execute the acquisition of the acquired image while sending the acquired image to the processor 20.
- the image collector 15 may also obtain the second image first, then obtain the first image, and alternately perform the acquisition of the collected images according to this sequence.
- the above-mentioned multiple relationship between the second operating frequency and the first operating frequency is only an example. In other embodiments, the multiple relationship between the second operating frequency and the first operating frequency may also be three times or four times. , Five times, six times and so on.
- the processor 20 After the processor 20 receives a frame of the captured image, it distinguishes the received captured image and determines whether the captured image is the first image or the second image. After the processor 20 receives at least one frame of the first image and at least one frame of the second image, it can calculate the depth image according to the first image, the second image, and the reference image. Specifically, since the first image is collected when the laser projector 14 is not projecting laser light, the light that forms the first image only includes ambient infrared light, and the second image is collected when the laser projector 14 projects laser light, forming the first image. The light of the two images includes both the ambient infrared light and the infrared laser emitted by the laser projector 14. Therefore, the processor 20 can remove the part of the collected image formed by the ambient infrared light in the second image according to the first image, thereby obtaining only Collected image formed by infrared laser (that is, speckle image formed by infrared laser).
- the ambient light includes infrared light with the same wavelength as the infrared laser emitted by the laser projector 14 (for example, including ambient infrared light at 940 nm), and when the image collector 15 acquires the captured image, this part of the infrared light will also be captured by the image collector. 15 received.
- the proportion of ambient infrared light in the light received by the image collector 15 will increase, resulting in inconspicuous laser speckles in the collected image, thereby affecting the calculation of the depth image.
- the control method of the present application controls the laser projector 14 and the image collector 15 to work at different operating frequencies.
- the image collector 15 can collect the first image formed by only ambient infrared light and at the same time by the ambient infrared light and the laser projector 14
- the second image formed by the emitted infrared laser and based on the first image, the part of the image formed by the ambient infrared light in the second image is removed, so that the laser speckles can be distinguished, and only the laser projector 14 can be used.
- the acquired image formed by the infrared laser is used to calculate the depth image, and the laser speckle matching is not affected, which can avoid partial or complete loss of depth information, thereby improving the accuracy of the depth image.
- control method includes:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires a captured image at a second operating frequency, and the second operating frequency is greater than the first operating frequency;
- control method includes:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires the collected images at the second operating frequency, and the second operating frequency is greater than the first operating frequency
- 021511 Determine the working state of the laser projector 14 at the collection time according to the collection time of each frame of collected images
- 0216 Calculate the depth image according to the first image, the second image, and the reference image.
- step 0201 is basically the same as step 031 described above
- step 0202 is basically the same as step 032 described above
- step 0203 is basically the same as step 033 described above.
- step 0204 is basically the same as step 0184 described above
- step 02051 and step 02052 can be substeps of step 0185 described above
- step 0206 is basically the same as step 0186 described above
- step 0211 is the same as step 031 described above
- Step 0212 is basically the same as step 032 described above.
- Step 0213 is basically the same as step 033 described above.
- Step 0214 is basically the same as step 0184 described above.
- Step 021511 and step 021512 can be the same as described above.
- step 02251, step 02152 is basically the same as step 02052 described above, and step 0216 is basically the same as step 0186 described above.
- step 02251, step 02052, step 02511, step 02512, and step 02152 can all be implemented by the processor 20.
- the processor 20 can also be used to add an image type to each frame of the captured image, and to distinguish the first image from the second image according to the image type.
- the processor 20 is used to add an image type to each frame of the captured image, specifically to determine the working status of the laser projector 14 at the time of collection according to the collection time of each frame of the collected image, and to collect each frame according to the working status Add image type to image.
- the processor 20 adds an image type (stream_type) to the captured image every time it receives a frame of the captured image from the image collector 15, so that the first image and the second image can be distinguished according to the image type in subsequent processing.
- the processor 20 will monitor the working status of the laser projector 14 in real time via the I2C bus. Each time the processor 20 receives a frame of collected images from the image collector 15, it will first acquire the collection time of the collected image, and then determine according to the collection time of the collected image that the working state of the laser projector 14 is projection The laser is still not projected, and the image type is added to the captured image based on the judgment result.
- the collection time of the collected image may be the start time, the end time, any time between the start time and the end time when the image collector 15 acquires each frame of the collected image, and so on. In this way, it is possible to realize the correspondence between each frame of collected image and the working state (projected laser or not projected laser) of the laser projector 14 during the acquisition of the frame of collected image, and accurately distinguish the type of the collected image.
- the structure of the image type stream_type is shown in Table 1:
- stream When stream is 0 in Table 1, it means that the data stream at this time is an image formed by infrared light and/or infrared laser.
- light When light is 00, it means that the data stream at this time is acquired without any equipment projecting infrared light and/or infrared laser (only ambient infrared light), then the processor 20 can add an image type of 000 to the collected image , To identify this captured image as the first image.
- light is 01 it means that the data stream at this time is obtained when the laser projector 14 projects infrared laser light (both ambient infrared light and infrared laser light).
- the processor 20 may add an image type of 001 to the captured image to identify this captured image as the second image.
- the processor 20 can then distinguish the image types of the collected images according to stream_type.
- control method includes:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires the collected images at the second operating frequency, and the second operating frequency is greater than the first operating frequency
- 02262 Calculate the depth image based on the third image and the reference image.
- step 0221 is basically the same as step 031 described above
- step 0222 is basically the same as step 032 described above
- step 0223 is basically the same as step 033 described above.
- step 0224 is basically the same as step 0184 described above
- step 0225 is basically the same as step 0185 described above
- step 02261 and step 02262 can be sub-steps of step 0186 described above;
- step 02261 and step 02262 can be processed ⁇ 20 Implementation.
- the processor 20 may also be used to calculate a third image based on the first image and the second image, and calculate a depth image based on the third image and the reference image, where the acquisition time of the first image is the same as that of the second image. The difference in acquisition time is less than the predetermined difference.
- the processor 20 may first distinguish the first image from the second image, and then select any frame of the second image and the first image of the specific frame corresponding to the arbitrary frame of the second image according to the acquisition time , Wherein the difference between the acquisition time of the first image of the specific frame and the acquisition time of the second image of the arbitrary frame is less than a predetermined difference. Subsequently, the processor 20 calculates a third image according to the first image of the specific frame and the second image of the arbitrary frame. The third image is the collected image formed by only the infrared laser emitted by the laser projector 14, or This is called the actual speckle image.
- the processor 20 may calculate a depth image based on the third image and the reference image, where the number of frames of the second image, the number of frames of the third image, and the number of frames of the depth image are all equal. It can be understood that since the difference between the acquisition time of the first image and the acquisition time of the second image is small, the intensity of the ambient infrared light in the first image is closer to the intensity of the ambient infrared light in the second image. The accuracy of the third image calculated from the image and the second image is higher, which helps to further reduce the influence of ambient infrared light on the acquisition of the depth image.
- the processor 20 may also add an image type to the third image and the depth image, so as to distinguish each data stream obtained after processing the acquired image. As shown in table 2:
- the stream in Table 2 When the stream in Table 2 is 0, it means that the data stream at this time is an image formed by infrared light and/or infrared laser; when the stream is 1, it means that the data stream at this time is a depth image.
- light When light is 11, it means background subtraction processing, which means removing the part formed by ambient infrared light in the captured image. Then the processor 20 can add the image type of 011 to the data stream after the background subtraction processing to identify this data
- the stream is the third image.
- Light is XX
- X indicates that the value is not limited, and the processor 20 may add an image type of 1XX to the data stream obtained after depth calculation to identify this data stream as a depth image.
- the acquisition time of the first image may be before the acquisition time of the second image or after the acquisition time of the second image. No restrictions.
- the first image and the second image may be images of adjacent frames or non-adjacent frames.
- the first image and the second image whose difference is less than the predetermined difference are images of adjacent frames; between the second operating frequency and the first operating frequency The multiple of is greater than twice.
- the first image and the second image whose difference is less than the predetermined difference can be images of adjacent frames or non-adjacent frames Image (at this time, there is still a frame of first image between the first image and the second image).
- the number of frames of the first image participating in the depth image calculation may also be multiple frames.
- the processor 20 may first perform fusion processing on the two frames of the first image, for example, add the pixel values of the corresponding pixels of the two frames of the first image and then take the average value to obtain the fusion processed first image, and then use the fusion processing The subsequent first image and the adjacent frame of the second image calculate the third image.
- the processor 20 may calculate multiple frames of the third image, such as the (N+1)-Nth frame of the third image and the (N+3)-(N+2)th frame in FIG. 19
- the third image, the (N+5)-(N+4)th frame of the third image, etc., and multiple frames of depth images are calculated corresponding to multiple frames of the third image.
- the processor 20 may also calculate only one frame of the third image, and calculate a frame of depth image corresponding to one frame of the third image. The number of frames of the third image can be determined according to the security level of the application scenario.
- the number of frames of the third image should be more. In this case, multiple frames of depth images are required to match the user's depth template
- the payment action is executed only after all successes to improve the security of payment; while the security level of the application scenario is low, for example, for the application scenario of portrait beautification based on depth information, the number of frames of the third image can be less, for example, One frame, at this time, using one frame of depth image is sufficient for portrait beautification. In this way, the calculation amount and power consumption of the processor 20 can be reduced, and the image processing speed can be increased.
- control method further includes:
- 0239 Determine the frame-synchronized visible light image and the second image according to the acquisition time of the visible light image, the acquisition time of the acquired image, and the image type of the acquired image.
- control methods include:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires the collected images at the second working frequency, and the second working frequency is greater than the first working frequency;
- 0236 Calculate the depth image according to the first image, the second image, and the reference image
- 0239 Determine the frame-synchronized visible light image and the second image according to the acquisition time of the visible light image, the acquisition time of the acquired image, and the image type of the acquired image.
- step 0231 is basically the same as step 031 described above
- step 0232 is basically the same as step 032 described above
- step 0233 is basically the same as step 033 described above.
- step 0234 is basically the same as step 0184 described above
- step 0235 is basically the same as step 0185 described above
- step 0236 is basically the same as step 0186 described above.
- Step 0237 can be controlled by the visible light camera 40 (main camera 41 and sub camera). Any one of 42, or the main camera 41 and the sub camera 42) are implemented together.
- Step 0238 and step 0239 may be implemented by the processor 20.
- the visible light camera 40 can be used to collect visible light images at a third operating frequency, which is greater than or less than the second operating frequency.
- the processor 20 can be used to add a collection time for each frame of visible light image and each frame of collected image, and determine the frame-synchronized visible light image and the second image according to the collection time of the visible light image, the collection time of the collected image, and the image type of the collected image. .
- the image collector 15 In some application scenarios, for example, in the application scenario of three-dimensional modeling of objects in the scene, it is necessary to use the image collector 15 to obtain the depth information of the objects in the scene, and use the visible light camera 40 to obtain the color information of the objects in the scene. Three-dimensional modeling. At this time, the processor 20 needs to turn on the image collector 15 to obtain the depth image and simultaneously turn on the visible light camera 40 to obtain the visible light image.
- the processor 20 can send an image acquisition instruction to the image collector 15 via the I2C bus, After the image collector 15 receives the image acquisition instruction, the image collector 15 and the visible light camera 40 are synchronized by a sync signal.
- the sync signal controls the visible light camera 40 to start collecting visible light images to realize the hardware of the image collector 15 and the visible light camera 40 Synchronize. At this time, the number of frames of the collected image is consistent with the number of frames of the visible light image, and each frame of the collected image corresponds to each frame of the visible light image one-to-one.
- the processor 20 needs to synchronize the image collector 15 and the visible light camera 40 through software synchronization. Specifically, the processor 20 sends an image acquisition instruction to the image acquisition device 15 through an I2C bus connected to the image acquisition device 15, and at the same time sends an image acquisition instruction to the visible light camera 40 through an I2C bus connected to the visible light camera 40.
- the processor 20 Whenever the processor 20 receives a frame of a captured image, it adds an image type to each frame of the captured image, and also adds a capture time to each frame of the captured image. Moreover, each time the processor 20 receives a frame of visible light image, it adds a collection time to each frame of visible light image. Among them, the collection time of the captured image can be the start time, end time, any time between the start time and the end time of each frame of the captured image collected by the image collector 15; the collection time of the visible light image can be visible light The camera 40 collects the start time, end time, any time between the start time and the end time of each frame of visible light image, and so on.
- the processor 20 may first base on the acquisition time of the visible light image, the acquisition time of the acquired image, and the acquisition The type of image first determines the frame-synchronized visible light image and the second image.
- frame synchronization means that the determined difference between the acquisition time of the second image and the acquisition time of the visible light image is less than the preset time difference.
- the acquisition time of the visible light image can be located before or before the acquisition time of the second image. After the acquisition time of the second image.
- the processor 20 selects the first image based on the determined second image to further calculate the depth image based on the second image, the first image, and the reference image. Finally, the processor 20 performs subsequent processing based on the depth image and the determined visible light image.
- the processor 20 may also add acquisition time to each frame of depth image, and then determine the frame-synchronized visible light image and the depth image according to the acquisition time of the visible light image and the acquisition time of the depth image, and finally the frame synchronization The visible light image and depth image are processed later.
- the acquisition time of each frame of depth image is the acquisition time of the second image corresponding to the frame of depth image.
- the captured image also includes an infrared image
- the infrared image is an image obtained by the image collector 15 collecting infrared light emitted by the floodlight 50.
- the processor 20 adds an image type to each frame of the captured image, it also adds an image type to the infrared image.
- the image types of infrared images are shown in Table 3:
- the stream in Table 3 When the stream in Table 3 is 0, it means that the data stream at this time is an image formed by infrared light and/or infrared laser. When light is 10, it means that the data stream at this time is obtained when the floodlight 50 projects infrared light and the laser projector 14 does not project laser light. Then, when the processor 20 adds the image type of 010 to the captured image, it identifies that this frame of captured image is an infrared image.
- the image collector 15 needs to be used in conjunction with the floodlight 50 and the laser projector 14.
- the image collector 15 can obtain the first image, the second image, and the infrared image in time sharing.
- the solid line represents the timing of laser emission from the laser projector 14
- the double-dot chain line represents the timing of infrared light emitted by the floodlight 50
- the dashed line represents the timing of the image acquisition by the image collector 15 and the number of frames of the image.
- the dash-dotted line represents the frame number of the third image obtained from the first image and the second image.
- the processor 20 can monitor the working status of the floodlight 50 in real time via the I2C bus. Each time the processor 20 receives a frame of collected images from the image collector 15, it will first acquire the collection time of the collected image, and then determine according to the collection time of the collected image that the working state of the floodlight 50 is emission during the collection time of the collected image The infrared light is still not emitting infrared light, and the image type is added to the captured image based on the judgment result.
- the processor 20 may subsequently determine the infrared image and the second image whose acquisition time difference is less than the set difference based on the acquisition time of the infrared image and the acquisition time of the second image. Further, the processor 20 may determine the infrared image and the second image. Depth image, and use the infrared image and the depth image for identity verification.
- control method further includes:
- step 0253 enter the step of determining the use state of the electronic device 100 (step 0253).
- control methods include:
- the laser projector 14 projects laser light in the first mode
- the laser projector 14 projects laser light in the second mode, and the energy of the laser light projected in the second mode is greater than the energy of the laser light projected in the first mode;
- the image collector 15 acquires the collected images at the second operating frequency, and the second operating frequency is greater than the first operating frequency
- step 0253 is basically the same as step 031 described above
- step 0254 is basically the same as step 032 described above
- step 0255 is basically the same as step 033 described above
- step 0256 is basically the same as step 0184 described above
- step 0257 is basically the same as step 0185 described above
- step 0258 is basically the same as step 0186 described above
- both steps 0251 and step 0252 can be implemented by the processor 20.
- the processor 20 can be used to obtain the brightness and type of the scene, and determine whether the brightness is greater than the brightness threshold and the type is an outdoor scene.
- the laser projector 14 can be used to project laser light to the scene at the first operating frequency when the brightness is greater than the brightness threshold and the type is an outdoor scene.
- the brightness of the scene can be obtained by analyzing the collected image obtained by the image collector 15 or the visible light image obtained by the visible light camera 40 (any one of the main camera 41 and the secondary camera 42, or the main camera 41 and the secondary camera 42 together); Alternatively, the brightness of the scene can also be directly detected by the light sensor, and the processor 20 reads the detected signal from the light sensor to obtain the brightness of the scene.
- the type of the scene can be obtained by analyzing the collected image obtained by the image collector 15 or the visible light image obtained by the visible light camera 40, for example, analyzing the object in the collected image or the visible light image obtained by the visible light camera 40 to determine whether the type of the scene is an outdoor scene or an indoor Scene; the type of scene can also be determined directly according to the geographic location.
- the processor 20 can obtain the positioning result of the global satellite positioning system for the scene, and then further judge the type of scene according to the positioning result, for example, the positioning result is a certain office Building, it means that the scene is an indoor scene; the positioning scene is a certain park, it means that the scene is an outdoor scene; when the positioning scene is a certain street, it means that the scene is an outdoor scene, and so on.
- the positioning result is a certain office Building, it means that the scene is an indoor scene; the positioning scene is a certain park, it means that the scene is an outdoor scene; when the positioning scene is a certain street, it means that the scene is an outdoor scene, and so on.
- the ambient infrared light in the collected image will take up a larger proportion, which will have a greater impact on spot recognition. At this time, the interference of ambient infrared light needs to be removed. However, when the brightness of the scene is low, the proportion of ambient infrared light in the captured image will be less, and the impact on spot recognition will be small and can be ignored.
- the image collector 15 and the laser projector 14 can be the same The working frequency works, and the processor 20 directly calculates the depth image according to the acquired image (ie, the second image) obtained by the image collector 15 and the reference image.
- the processor 20 directly calculates the depth image based on the acquired image (ie, the second image) acquired by the image collector 15 and the reference image. In this way, the operating frequency of the image collector 15 can be reduced, and the power consumption of the image collector 15 can be reduced.
- control method can also determine whether to perform step 0253 based only on the brightness of the scene.
- the processor 20 only obtains the brightness of the scene, determines whether the brightness of the scene is greater than the brightness threshold, and the laser projector 14 projects laser light onto the scene at the first operating frequency when the brightness is greater than the brightness threshold.
- the processor 20 may also add status information (status) to each data stream.
- status information (status)
- the processor 20 includes a first storage area, a second storage area, and a logical subtraction circuit, and the logical subtraction circuit is connected to both the first storage area and the second storage area.
- the first storage area is used to store the first image
- the second storage area is used to store the second image
- the logical subtraction circuit is used to process the first image and the second image to obtain the third image.
- the logical subtraction circuit reads the first image from the first storage area, reads the second image from the second storage area, and performs subtraction on the first image and the second image after acquiring the first image and the second image
- the third image is obtained by processing.
- the logic subtraction circuit is also connected to the depth calculation module in the processor 20 (for example, it may be an integrated circuit ASIC dedicated to calculating depth, etc.).
- the logic subtraction circuit sends the third image to the depth calculation module, and the depth calculation module is The third image and the reference image calculate the depth image.
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Networks & Wireless Communication (AREA)
- Environmental & Geological Engineering (AREA)
- Optics & Photonics (AREA)
- Transforming Electric Information Into Light Information (AREA)
- Controls And Circuits For Display Device (AREA)
- User Interface Of Digital Computer (AREA)
- Projection Apparatus (AREA)
Abstract
一种电子装置(100)及电子装置(100)的控制方法。控制方法包括:(031)确定电子装置(100)的使用状态;(032)在电子装置(100)处于第一使用状态时,激光投射器(14)以第一模式投射激光;及,(033)在电子装置(100)处于第二使用状态时,激光投射器(14)以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量。
Description
优先权信息
本申请请求2019年5月31日向中国国家知识产权局提交的、专利申请号为201910471662.X的专利申请的优先权和权益,并且通过参照将其全文并入此处。
本申请涉及消费性电子产品技术领域,特别涉及一种电子装置的控制方法及电子装置。
现有的具有激光投射器的手机中,激光投射器一般设置在手机的前壳上,并且激光投射器只用于拍摄距离较近的前置使用状态中,例如,激光投射器只用于前置深度图像获取的使用状态中,从而导致手机的使用者能够使用激光投射器的场景较少。
发明内容
本申请实施方式提供了一种电子装置的控制方法及电子装置。
本申请的电子装置的控制方法,所述电子装置包括机壳、第一显示屏、第二显示屏及激光投射器,所述机壳包括相背的第一面和第二面,所述第一显示屏和所述第二显示屏分别设置在所述第一面和所述第二面,所述激光投射器设置在所述第一面;所述控制方法包括:确定所述电子装置的使用状态;在所述电子装置处于第一使用状态时,所述激光投射器以第一模式投射激光;及,在所述电子装置处于第二使用状态时,所述激光投射器以第二模式投射激光,所述第二模式投射的所述激光的能量大于所述第一模式投射的所述激光的能量。
本申请的电子装置包括机壳、第一显示屏、第二显示屏、激光投射器及处理器,所述机壳包括相背的第一面和第二面,所述第一显示屏和所述第二显示屏分别设置在所述第一面和所述第二面,所述激光投射器设置在所述第一面;所述处理器用于确定所述电子装置的使用状态;所述激光投射器用于在所述电子装置处于第一使用状态时以第一模式投射激光、以及用于在所述电子装置处于第二使用状态时以第二模式投射激光,所述第二模式投射的所述激光的能量大于所述第一模式投射的所述激光的能量。
本申请实施方式的附加方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本申请的实践了解到。
本申请的上述和/或附加的方面和优点可以从结合下面附图对实施方式的描述中将变得明显和容易理解,其中:
图1和图2是本申请某些实施方式的电子装置的结构示意图。
图3是本申请某些实施方式的电子装置的控制方法的流程示意图。
图4是本申请某些实施方式的电子装置的激光投射器的激光光源的结构示意图。
图5是本申请某些实施方式的电子装置的立体结构示意图。
图6和图7是本申请某些实施方式的电子装置的控制方法的流程示意图。
图8是本申请某些实施方式的电子装置的立体结构示意图。
图9是图8所示的电子装置沿IX-IX线的剖视图。
图10至图12是本申请某些实施方式的电子装置沿与图8所示的IX-IX线相同剖面线下的剖视图。
图13和图14是本申请某些实施方式的电子装置的结构示意图。
图15和图16是本申请某些实施方式的电子装置沿与图8所示的IX-IX线相同剖面线下的剖视图。
图17是本申请某些实施方式的电子装置的***架构示意图。
图18是本申请某些实施方式的电子装置的控制方法的流程示意图。
图19是本申请某些实施方式的电子装置的控制方法的原理示意图。
图20至图23是本申请某些实施方式的电子装置的控制方法的流程示意图。
图24是本申请某些实施方式的电子装置的控制方法的原理示意图。
图25是本申请某些实施方式的电子装置的控制方法的流程示意图。
下面详细描述本申请的实施方式,所述实施方式的示例在附图中示出,其中,相同或类似的标号自始至终表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的,仅用于解释本申请的实施方式,而不能理解为对本申请的实施方式的限制。
请参阅图1至图3,本申请的电子装置100的控制方法适用于电子装置100。其中,电子装置100包括机壳11、第一显示屏12、第二显示屏13及激光投射器14。机壳11包括相背的第一面111和第二面112。第一显示屏12和第二显示屏13分别设置在第一面111和第二面112。激光投射器14设置在第一面111,控制方法包括:
031,确定电子装置100的使用状态;
032,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;及
033,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量。
请参阅图5,在某些实施方式中,电子装置100还包括重力传感器16,确定电子装置的使用状态(031),包括:
通过重力传感器16确定电子装置100的使用状态。
请参阅图5,在某些实施方式中,通过重力传感器16确定电子装置100的使用状态,包括:
0611,获取重力传感器16的z轴的数值;
0612,在z轴的数值大于第一预设值时,确定电子装置100处于第一使用状态;及
0613,在z轴的数值小于第二预设值时,确定电子装置100处于第二使用状态。
在某些实施方式中,确定电子装置100的使用状态(031),包括:
通过第一显示屏12检测到的第一触控信号和/或第二显示屏13检测到的第二触控信号确定电子装置的使用状态。
在某些实施方式中,通过第一显示屏12检测到的第一触控信号和/或第二显示屏13检测到的第二触控信号确定电子装置100的使用状态,包括:
0711,在第一显示屏12检测到第一触控信号时,确定电子装置100处于第一使用状态;
0712,在第二显示屏13检测到第二触控信号时,确定电子装置100处于第二使用状态。
在某些实施方式中,电子装置100还包括状态选择按键17,确定电子装置100的使用状态(031),包括:
通过状态选择按键确定电子装置100的使用状态。
在某些实施方式中,激光投射器14在第一模式下投射激光的功率小于在第二模式下投射激光的功率。
激光投射器14包括多个点光源141,多个点光源141独立控制;
激光投射器14在第一模式下开启的点光源141的数量少于在第二模式下开启的点光源141的数量。
在某些实施方式中,电子装置100还包括图像采集器15,图像采集器15设置在第一面上;
激光投射器14在投射激光时,激光投射器14以第一工作频率向场景投射激光;
控制方法还包括:
图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;和
根据第一图像、第二图像及参考图像计算深度图像。
请参阅图1,本申请实施方式的电子装置100包括机壳11、第一显示屏12、第二显示屏13、激光投射器14及处理器20。机壳11包括相背的第一面111和第二面112,第一显示屏12和第二显示屏13分别设置在第一面111和第二面112。激光投射器14设置在第一面11。处理器20用于确定电子装置100的使用状态。激光投射器14用于在电子装置100处于第一使用状态时以第一模式投射激光、以及用于在电子装置100处于第二使用状态时以第二模式投射激光。第二模式投射的激光的能量大于第一模式投射的激光的能量。
请参阅图5,在某些实施方式中,电子装置100还包括重力传感器16。处理器20用于通过重力传感器16确定电子装置100的使用状态。
请参阅图5,在某些实施方式中,处理器20还用于:
获取重力传感器16的z轴的数值;
在z轴的数值大于第一预设值时,确定电子装置100处于第一使用状态;及
在z轴的数值小于第二预设值时,确定电子装置100处于第二使用状态。
请参阅图1,在某些实施方式中,处理器20用于通过第一显示屏12检测到的第一触控信号和/或第二显示屏13检测到的第二触控信号确定电子装置100的使用状态。
请参阅图1,在某些实施方式中,处理器20用于:
在第一显示屏12检测到第一触控信号时,确定电子装置100处于第一使用状态;及
在第二显示屏13检测到第二触控信号时,确定电子装置100处于第二使用状态。
请参阅图8,在某些实施方式中,电子装置100还包括状态选择按键17。处理器20用于通过状态选择按键17确定电子装置100的使用状态。
请参阅图4,在某些实施方式中,激光投射器14在第一模式下投射激光的功率小于在第二模式下投射激光的功率;和/或
激光投射器14包括多个点光源141,多个点光源141独立控制;激光投射器14在第一模式下开启的点光源141的数量少于在第二模式下开启的点光源141的数量。
请参阅图4,在某些实施方式中,多个点光源141形成多个发光阵列142。多个发光阵列142呈环形排布。
请参阅图1,在某些实施方式中,电子装置100还包括图像采集器15。图像采集器15设置在第一面111。激光投射器14在投射激光时,激光投射器14用于以第一工作频率向场景投射激光;图像采集器15用于以第二工作频率获取采集图像,第二工作频率大于第一工作频率;处理器20还用于:
在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;和
根据第一图像、第二图像及参考图像计算深度图像。
请参阅图1,在某些实施方式中,第一显示屏12为一端开设有开口的开口屏,第二显示屏13为全面屏,激光投射器14安装在开口内;或
第一显示屏12和第二显示屏13均为全面屏,激光投射器14设置在第一显示屏12的下方。
请参阅图1,在某些实施方式中,第一使用状态和第二使用状态中的激光投射器14的朝向相反。
在某些实施方式中,第一触控信号和第二触控信号均为电容信号。
请参阅图1至图3,本申请的电子装置100的控制方法,其中,电子装置100包括机壳11、第一显示屏12、第二显示屏13及激光投射器14,机壳11包括相背的第一面111和第二面112,第一显示屏12和第二显示屏13分别设置在第一面111和第二面112,激光投射器14设置在第一面111,控制方法包括:
031,确定电子装置100的使用状态;
032,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;及
033,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量。
本申请的电子装置100能够用于实现上述控制方法,具体地,电子装置100还包括处理器20,步骤031可以由处理器20实现,步骤032和步骤033均可以由激光投射器14实现。也即是说,处理器20可用于确定电子装置100的使用状态,激光投射器14可用于在电子装置100处于第一使用状态时以第一模式投射激光、以及用于在电子装置100处于第二使用状态时以第二模式投射激光。
其中,电子装置100可以是手机、平板电脑、笔记本电脑、智能穿戴设备(智能手表、智能手环、智能头盔、智能眼镜等)、虚拟现实设备等。本申请以电子装置100是手机为例进行说明,但电子装置100的形式并不限于手机。
电子装置100还包括设置在第一面111上的图像采集器15,激光投射器14配合图像采集器15使用以用于获取待测物的深度信息,以用于三维建模、生成三维图像、测距等。激光投射器14和图像采集器15可以安装在支架上后,再将支架、激光投射器14和图像采集器15一同安装在机壳11内。
本申请实施方式中,电子装置100的使用状态包括第一使用状态和第二使用状态。相对于电子装置100的使用者,第一使用状态和第二使用状态中的激光投射器14的朝向相反。其中,第一使用状态可以为前置使用状态,在电子装置100处于前置使用状态时,第一显示屏12和激光投射器14朝向电子装置100的使用者,此时,使用者能够观看到第一显示屏12显示的内容、并且能够使用激光投射器 14朝使用者一侧投射激光,使用者能够使用激光投射器14(和图像采集器15)进行人脸识别、虹膜识别等。第二使用状态可以为后置使用状态,在电子装置100处于后置使用状态时,第一显示屏12和激光投射器14背离电子装置100的使用者,第二显示屏13朝向电子装置100的使用者,此时,使用者能够观看第二显示屏13显示的内容、并且能够使用激光投射器14朝远离使用者的一侧投射激光,例如,使用者能够使用激光投射器14(和图像采集器15)获取电子装置100的与使用者相背一侧的待测物的深度图像。
本申请实施方式中,激光投射器14投射激光的模式包括第一模式和第二模式,其中,第一模式对应激光投射器14处于前置使用状态,第二模式对应激光投射器14处于后置使用状态,第二模式投射的激光的能量大于第一模式投射的激光的能量。具体地,激光投射器14在第一模式下投射激光的功率可以小于在第二模式下投射激光的功率,以使第二模式投射的激光的能量大于第一模式投射的激光的能量,此时,激光投射器14在第二模式下投射的激光能够到达的最大距离(投射距离)大于在第一模式下投射的激光能够到达的最大距离(投射距离)。同时,后置使用状态下激光投射器14配合图像采集器15能够检测到的后置距离范围要大于前置使用状态下激光投射器14配合图像采集器15能够检测到的前置距离范围,例如:激光投射器14配合图像采集器15能够检测到的前置距离范围为25cm以内,而激光投射器14配合图像采集器15能够检测到的后置距离范围为大于25cm(25cm内的距离范围精度很差);或者,前置距离范围与后置距离范围有稍许交集,例如激光投射器14配合图像采集器15能够检测到的前置距离范围为25cm以内,而激光投射器14配合图像采集器15能够检测到的后置距离范围为大于20cm。
本申请的电子装置100及电子装置100的控制方法中,第一显示屏12和第二显示屏13分别设置在机壳11的相背两侧,激光投射器14和第一显示屏12设置在机壳11的同一侧,并且激光投射器14在第二使用状态时能够投射的最大距离大于在第一使用状态时能够投射的最大距离,从而使激光投射器14能够同时在第一使用状态和第二使用状态下使用,增加了使用者能够使用电子装置100的场景;同时,电子装置100上不需要设置两个激光投射器14以分别在第一使用状态和第二使用状态下使用,节省了电子装置100的成本。
请参阅图4,在某些实施方式中,激光投射器14包括激光光源140,激光光源140包括多个点光源141,多个点光源141能够被独立控制,具体地,每个点光源141均能够独立地开启和关闭。激光投射器14在第一模式下开启的点光源141的数量少于在第二模式下开启的点光源141的数量,此时,每个点光源141投射激光的功率可以相同,从而使得第二模式投射的激光的能量大于第一模式投射的激光的能量,激光投射器14在第二模式下投射的激光能够到达的最大距离也就大于在第一模式下投射的激光能够到达的最大距离。
请参阅图4,在某些实施方式中,多个点光源141形成多个发光阵列142,多个发光阵列142独立控制。具体地,每个发光阵列142内的多个点光源141能够同时开启和关闭,此时,每个发光阵列142内的多个点光源141的功率可以相同。在其他实施方式中,每个发光阵列142内的多个点光源141也可以独立控制。
本实施方式中,多个发光阵列142呈环形排布。环形排布的发光阵列142内的点光源141发出的激光可以覆盖更广的视场,如此,可以获得更多待测物的深度信息。其中,环形可以为方环形或圆环形。
请参阅图4,在某些实施方式中,随着投射距离的增大,发光阵列142的开启方式为:距离激光光源140的中心越远的发光阵列142越先开启。例如,请结合图4,发光阵列142的总数为6个,6个发光阵列142包括5个环形子阵列143和1个方形子阵列144,沿靠近激光光源140的中心的方向,5个环形子阵列143依次排布,依次排布的5个环形子阵列143的编号为A、B、C、D、E。本实施方式中,激光投射器14在第一模式下开启编号为A和B的环形子阵列143内的点光源141;激光投射器14在第二模式下可以开启编号为A、B和C的环形子阵列143内的点光源141、或者开启编号为A、B、C和D的环形子阵列143内的点光源141、或者开启编号为A、B、C、D和E的环形子阵列143内的点光源141、或者开启编号为A、B、C、D和E的环形子阵列143内的点光源141以及方形子阵列144内的点光源141。
本实施方式中,激光投射器14在第一模式下开启的点光源141的数量少于在第二模式下开启的点光源141的数量,从而使电子装置100在处于第一使用状态时投射的激光的能量小于处于第二使用状态时投射的激光的能量。
可以理解,激光投射器14的衍射光学元件146(图9所示)的衍射能力是有限的,即激光光源140发射的部分激光不会被衍射而是直接出射,直接出射的激光不经过衍射光学元件146的衍射衰减作用。而直接出射的激光的能量较大,极有可能对使用者的眼睛产生危害。因此,激光投射器14在电子装置 100处于第一使用状态时以第一模式投射激光,也就是说,激光投射器14在投射距离较小时先开启远离激光光源140的中心的环形子阵列143,可以避免激光光源140投射的激光不经衍射光学元件146的衍射衰减作用而直接投射到使用者的眼睛,从而提高了激光投射器14的安全性;激光投射器14在电子装置100处于第二使用状态时以第二模式投射激光,也就是说,激光投射器14在投射距离较大时同时开启远离激光光源140的中心的环形子阵列143和靠近激光光源140的中心的环形子阵列143,从而增大了激光投射器14投射的激光能够到达的最大距离。
请参阅图5及图6,在某些实施方式中,电子装置100还包括重力传感器16,所述确定电子装置100的使用状态可以通过重力传感器16实现,具体地,控制方法包括:
0611,获取重力传感器16的z轴的数值;
0612,在z轴的数值大于第一预设值时,确定电子装置100处于所述第一使用状态;
0613,在z轴的数值小于第二预设值时,确定电子装置100处于所述第二使用状态;
062,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;及
063,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量。
上述的控制方法也可以由电子装置100实现,其中,步骤0611、步骤0612和步骤0613可以为前文所述的步骤031的子步骤,步骤062与前文所述的步骤032基本相同,步骤063与前文所述的步骤033基本相同。具体地,处理器20电性连接重力传感器16,处理器20还可以用于通过重力传感器16确定电子装置100的使用状态,处理器20还可以用于实现步骤0611、步骤0612和步骤0613。也就是说,处理器20还可以用于获取重力传感器16的z轴的数值、在z轴的数值大于第一预设值时,确定电子装置100处于所述第一使用状态、及在z轴的数值小于第二预设值时,确定电子装置100处于所述第二使用状态。
本实施方式中,重力传感器16包括三轴加速度传感器16。请参照图5,图5为三轴加速度传感器16确定电子装置100的使用状态的原理示意图。如图5所示,x轴、y轴、z轴均是相对于机壳11的位置而言的,通常y轴的方向为机壳11向上的方向,x轴的方向为机壳11向右的方向,z轴的方向为垂直于第一面111向外的方向。z轴的数值(加速度值)与第一显示屏12的朝向对应。
以用户使用电子装置100(手机)的视角来观察,当将电子装置100平放在桌面上并使第一显示屏12朝上时,x轴的数值默认为0,y轴的数值默认0,z轴的数值默认9.81m/s
2;当将电子装置100朝下放在桌面上并使第一显示屏12朝下时,x轴的数值默认为0,y轴的数值默认0,z轴的数值为-9.81m/s
2;当将电子装置100向左倾斜时,x轴的数值为正值;当将电子装置100向右倾斜时,x轴的数值为负值;当将电子装置100向上倾斜时,y轴的数值为负值;当将电子装置100向下倾斜时,y轴的数值为正值。
本实施方式中,若处理器20获取到的重力传感器16的z轴的数值大于第一预设值,例如,处理器20获取到的重力传感器16的z轴的数值6m/s
2,大于第一预设值3m/s
2时,则确定第一显示屏12朝向电子装置100的使用者,此时,处理器20确定电子装置100处于第一使用状态;若处理器20获取到的重力传感器16的z轴的数值小于第二预设值,例如,处理器20获取到的重力传感器16的z轴的数值-6m/s
2,小于第二预设值-3m/s
2时,则确定第二显示屏13朝向电子装置100的使用者,此时,处理器20确定电子装置100处于第二使用状态。
本申请实施方式的电子装置100及控制方法通过重力传感器16确定电子装置100的使用状态,从而不需要使用者手动选择电子装置100的使用状态就能够使用激光投射器14以对应的模式投射激光,提升了电子装置100的使用体验。
请参阅图7,在某些实施方式中,第一显示屏12和第二显示屏13均为触控显示屏。所述确定电子装置100的使用状态可以通过第一显示屏12检测到的第一触控信号和/或第二显示屏13检测到的第二触控信号实现,具体地,控制方法包括:
0711,在第一显示屏12检测到第一触控信号时,确定电子装置100处于第一使用状态;
0712,在第二显示屏13检测到第二触控信号时,确定电子装置100处于第二使用状态;
072,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;及
073,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量。
上述的控制方法也可以由电子装置100实现,其中,步骤0711和步骤0712可以为前文所述的步骤031的子步骤,步骤072与前文所述的步骤032基本相同,步骤073与前文所述的步骤033基本相同。具体地,处理器20还可以用于通过第一显示屏12检测到的第一触控信号和/或第二显示屏13检测到的第二触控信号确定电子装置100的使用状态,处理器20还可以用于实现步骤0711和步骤0712。也就是说,处理器20还可以用于在第一显示屏12检测到第一触控信号时,确定电子装置100处于第 一使用状态、以及在第二显示屏13检测到第二触控信号时,确定电子装置100处于第二使用状态。
第一触控信号和第二触控信号均可以为电容信号。处理器20电性连接第一显示屏12和第二显示屏13,处理器20还可以获取第一显示屏12检测到的第一触控信号和第二显示屏13检测到的第二触控信号。当电子装置100的使用者触控操作第一显示屏12时,第一显示屏12的电容信号发生变化并被第一显示屏12检测到(也即是第一显示屏12检测到第一触控信号),此时,第一显示屏12朝向电子装置100的使用者并被使用者使用,处理器20确定电子装置100处于第一使用状态。当电子装置100的使用者触控操作第二显示屏13时,第二显示屏13的电容信号发生变化并被第二显示屏13检测到(也即是第二显示屏13检测到第二触控信号),此时,第二显示屏13朝向电子装置100的使用者并被使用者使用,处理器20确定电子装置100处于第二使用状态。
本申请实施方式的电子装置100及控制方法通过第一显示屏12或第二显示屏13确定电子装置100的使用状态,从而不需要在电子装置100上设置额外的硬件,节省了电子装置100的生产成本。
请参阅图8,在某些实施方式中,电子装置100还包括与处理器20电性连接的状态选择按键17,所述确定电子装置100的使用状态(步骤010)可以通过状态选择按键17实现,具体地,处理器20还可以用于通过状态选择按键17确定电子装置100的使用状态。状态选择按键17可以为实体按键并包括第一状态按键171和第二状态按键172。当处理器20检测到第一状态按键171被触发时,处理器20确定电子装置100处于第一使用状态;当处理器20检测到第二状态按键172被触发时,处理器20确定电子装置100处于第二使用状态。在其他实施方式中,状态选择按键17还可以为虚拟按键,状态选择按键17可以被第一显示屏12和第二显示屏13显示,例如,状态选择按键17可以为第一显示屏12或第二显示屏13显示的状态切换按键,该状态切换按键用于在第一使用状态和第二使用状态之间切换。
本申请实施方式的电子装置100及控制方法通过状态选择按键17确定电子装置100的使用状态,从而使用者能够根据需要准确地选择所需要的使用状态。
请参阅图1,在某些实施方式中,电子装置100还包括可见光摄像头40,可见光摄像头40包括主摄像头41和副摄像头42,主摄像头41和副摄像头42均安装在机壳11的第一面111上内。主摄像头41可以为广角摄像头,广角摄像头对运动敏感度低,相对较低的快门速度都能保证图像拍摄的清晰度,且广角摄像头视角范围大,可以涵盖大范围景物,并且能强调前景和突出远近对比。副摄像头42可以为长焦摄像头,长焦摄像头的长焦镜头,可以识别更远的物体。或者主摄像头41为彩色摄像头,副摄像头42为黑白摄像头。此处的摄像头可以为多个,三个或四个或者更多。本实施方式中,激光投射器14、副摄像头42、主摄像头41和图像采集器15依次间隔设置并位于同一直线上。
请参阅图1及图2,第一显示屏12安装在机壳11的第一面111上,第一显示屏12可以覆盖第一面111的面积的85%及以上,例如达到85%、86%、87%、88%、89%、90%、91%、92%、93%、95%甚至是100%。第二显示屏13安装在机壳11的第二面112上,第二显示屏13可以覆盖第二面112的面积的85%及以上,例如达到85%、86%、87%、88%、89%、90%、91%、92%、93%、95%甚至是100%。第一显示屏12和第二显示屏13均可以用于显示影像,影像可以是文字、图像、视频、图标等信息。
第一显示屏12包括显示区122,显示区122可用于显示影像。适配不同类型的电子装置100及不同使用者的需求,显示区122的形状可以呈圆形、椭圆形、跑道形、圆角矩形、矩形等形状。
请结合图9,显示区122形成有相背的正面123及背面124,显示区122可以用于显示影像,第一显示屏12发出的光线沿背面124指向正面123的方向向外发射,当电子装置100处于第一使用状态(使用者位于第一面111一侧)时,在光线穿过正面123后可由使用者接收,即,使用者可从正面123观察第一显示屏12显示的影像。可以理解,正面123为显示面,背面124为与显示面相背的面。正面123及背面124均可以是平面或曲面。
在某些例子中,第一显示屏12还可包括非显示区,非显示区可以形成在显示区122的周缘。非显示区可以不用于显示,非显示区可用于与机壳11结合或用于走线,例如可以将非显示区与机壳11通过粘胶结合,而不会影响显示区122的显示功能。第一显示屏12还可以是集成有触控功能的触控显示屏,使用者获取第一显示屏12显示的影像信息后,可以在第一显示屏12上进行触控以实现预定的交互操作。
请参阅图8,在某些实施方式中,显示区122包括多个像素,多个像素按照预定的方式排列,相邻的像素之间存在微观间隙。显示区122包括第一子显示区1221及第二子显示区1222。第一子显示区1221的像素密度小于第二子显示区1222的像素密度。
第一子显示区1221的像素密度小于第二子显示区1222的像素密度,也就是第一子显示区1221的微观间隙要大于第二子显示区1222的微观间隙,第一子显示区1221对光线的阻隔作用较小,穿过第 一子显示区1221的光线的透过率较高。在其他实施方式中,第一子显示区1221的像素密度也可以大于第二子显示区1222的像素密度,或者第一子显示区1221的像素密度也可以等于第二子显示区1222的像素密度。
在某些实施方式中,第一子显示区1221与第二子显示区1222的形状可以依据具体的需求进行设置,在此不作限制,例如第一子显示区1221可以设置成跑道形、水滴状等,第二子显示区1222与第一子显示区1221可以互补并共同形成矩形或圆角矩形等形状的显示区122。第一子显示区1221可以位于显示区122的靠近边缘的位置,第二子显示区1222可以位于显示区122的中间位置。第一子显示区1221可以用于显示电子装置100的状态图标,例如,第一子显示区1221可用于显示电子装置100的电池电量、网络连接状态、***时间等。
请参阅图9,第一显示屏12和第二显示屏13均为全面屏。激光投射器14设置在第一显示屏12的背面124所在的一侧,或者说,激光投射器14设置在显示区122下方(例如可以设置在第一子显示区1221下方),激光投射器14用于发射穿过显示区122的激光。具体地,激光投射器14可以包括激光光源140、准直元件145及衍射光学元件146,激光光源140发出的光(例如红外激光)先经过准直元件145准直,再经衍射光学元件146衍射后从激光投射器14中发出,然后再穿过显示区122以投射到外界。光线穿过微观间隙时会发生衍射,即显示区122的微观间隙与衍射光学元件146上的衍射结构对激光光源140发出的光均具有衍射作用。
穿过显示区122并进入外界的激光中,可能同时包含由衍射光学元件146衍射形成的图案(图案中包括多个由衍射光学元件146衍射的斑点),以及由第一显示屏12的微观间隙衍射形成的图案(图案中包括多个由衍射光学元件146衍射又被第一显示屏12衍射的斑点),以使穿过显示区122后的散斑图案具有较高的不相关性,利于后续对获得的散斑图案进行处理。在一个例子中,显示区122的透过率可以达到60%或以上,以使激光投射器14发出的激光穿过显示区122时损耗较小。
图像采集器15可以是红外摄像头,激光发射到待测物,由待测物调制后,可以由图像采集器15获取,图像采集器15接收被调制的激光后得到散斑图像,散斑图像被处理后得到待测物的深度数据。图像采集器15也可以设置在第一显示屏12的背面124所在的一侧,即设置在第一显示屏12下(例如可以设置在第一子显示区1221下方)。此时,图像采集器15的入光面可以对准显示区122,被待测物调制后的激光穿过显示区122后再由图像采集器15接收,具体地,被待测物调的激光可以由第一显示屏12的微观间隙衍射后,再由图像采集器15接收。
请参阅图10,在某些实施方式中,电子装置100还包括泛光灯50,泛光灯50可以用于向外发射补充光线,补充光线可以用于在环境光线较弱时补充环境中的光线强度。在一个例子中,补充光线可以是红外光。补充光线发射到待测物上被待测物反射后,可以由图像采集器15获取以得到待测物的二维图像,二维图像信息可用于身份识别。泛光灯50也可以设置在第一显示屏12的背面124所在的一侧,即设置在第一显示屏12下(例如可以设置在第一子显示区1221下方),具体地,泛光灯50可以与激光投射器14及图像采集器15设置在同一个支架上。此时,泛光灯50发出的补充光线穿过显示区122的微观间隙后进入外界环境,被反射后的补充光线可以再次穿过微观间隙以被图像采集器15接收。
综上,由于激光投射器14和图像采集器15设置在第一显示屏12的背面124所在的一侧,第一显示屏12上不需要开设与激光投射器14和图像采集器15对准的开口,电子装置100的屏占比较高。另外,本申请实施方式将激光投射器14和图像采集器15设置在第一显示屏12下方,相较于可以采集深度信息的飞行时间而言,激光投射器14和图像采集器15获取的深度图像的分辨率较高,在人脸识别中使用分辨率较高的深度图像可以提高人脸识别的准确率,在三维场景建模中使用分辨率较高的深度图像可以提高建模的三维场景与实际场景之前的匹配度。
若激光投射器14设置在第一显示屏12下方时,激光投射器14投射的激光穿过第一子显示区1221时会受到第一显示屏12中的微观间隙的影响,即投射到场景中的散斑图像中的斑点同时包括激光仅被激光投射器14的衍射光学元件146一次衍射形成的斑点、以及激光经过衍射光学元件146一次衍射再由第一显示屏12二次衍射形成的斑点。图像采集器15设置在第一显示屏12下方,且图像采集器15接收的是被待测物反射后穿过第一显示屏12时被第一显示屏12衍射的激光,则图像采集器15采集的散斑图像中的斑点同时包括激光仅被衍射光学元件146一次衍射并被待测物反射形成的斑点、激光经过衍射光学元件146一次衍射再由第一显示屏12二次衍射并被待测物反射形成的斑点、以及激光经过衍射光学元件146一次衍射再由第一显示屏12二次衍射并被待测物反射后又一次由第一显示屏12三次衍射形成的斑点。在计算深度图像时,包括以下两种计算方式:
(1)直接根据散斑图像中的所有斑点做深度计算。此时,参考图像中的参考斑点同时包括激光仅被衍射光学元件146一次衍射并被标定物体反射形成的参考斑点、激光经过衍射光学元件146一次衍射再由第一显示屏12二次衍射并被标定物体反射形成的参考斑点、以及激光经过衍射光学元件146一 次衍射再由第一显示屏12二次衍射并被标定物体反射后又一次由第一显示屏12三次衍射形成的参考斑点。
(2)仅留下激光仅被衍射光学元件146一次衍射并被待测物反射形成的斑点并滤掉其余斑点,以根据剩余的斑点做深度计算。此时,参考图像中的参考斑点仅包括激光仅被衍射光学元件146一次衍射并被标定物体反射形成的参考斑点。其中,经过不同衍射次数衍射后形成的斑点的亮度不同,因此,各类斑点可以通过亮度来区分,从而可以滤掉无需用到的斑点。
请参阅图11,在某些实施方式中,第一显示屏12为一端开设有开口121的开口屏,激光投射器14伸入开口121内安装,图像采集器15设置在第二显示屏12的下方。此时,激光投射器14设置在第一显示屏12下方,且激光投射器14投射的激光穿过第一显示屏12时未受到第一显示屏12的微观间隙的影响(即第一显示屏12形成有图5所示的开口121),设置在第一显示屏12下方的图像采集器15采集的散斑图像中的斑点同时包括激光仅被衍射光学元件146一次衍射并被待测物反射形成的斑点、以及激光经过衍射光学元件146一次衍射并被待测物反射后由第一显示屏12二次衍射形成的斑点。在计算深度图像时,包括以下两种计算方式:
(1)直接根据散斑图像中的所有斑点做深度计算。此时,参考图像中的参考斑点同时包括激光仅被衍射光学元件146一次衍射并被标定物体反射形成的参考斑点、以及激光经过衍射光学元件146一次衍射并被标定物体反射后由第一显示屏12二次衍射形成的参考斑点。
(2)仅留下激光仅被射光学元件213一次衍射并被待测物反射形成的斑点并滤掉其余斑点,以根据剩余的斑点做深度计算。此时,参考图像中的参考斑点仅包括激光仅被射光学元件213一次衍射并被标定物体反射形成的参考斑点。其中,经过不同衍射次数衍射后形成的斑点的亮度不同,因此,各类斑点可以通过亮度来区分,从而可以滤掉无需用到的斑点。
请参阅图12,在某些实施方式中,第一显示屏12形成有开口121,开口121不具有显示功能。开口121贯穿正面123及背面124。激光投射器14设置在第一显示屏12的背面124所在的一侧的同时,激光投射器14与开口121对准,激光投射器14用于发射穿过开口121的激光。在一个实施例中,开口121可以开设在第一子显示区1221上。
此时,激光投射器14的出光面可以对准开口121,激光投射器14发射的激光穿过开口121后进入外界。本实施方式中,进入外界的激光不需要穿过显示区122的微观间隙,不会由微观间隙再次衍射,从而在图像采集器15获取散斑图像后,可以降低后续基于散斑图像计算深度图像的处理难度。
具体地,在如图13所示的例子中,开口121包括形成在第一显示屏12的边缘上的缺口,或者说,开口121与第一显示屏12的边缘相交。缺口具体可以形成在第一显示屏12的上边缘、下边缘、左边缘、右边缘等任意一个或多个边缘上。缺口的形状可以是三角形、半圆形、矩形、跑道形等任意形状,在此不作限制。
在如图14所示的例子中,开口121包括与第一显示屏12的边缘间隔的通孔,或者说,开口121开设在第一显示屏12的边缘围成的范围内。通孔具体可以靠近第一显示屏12的上边缘、下边缘、左边缘、右边缘等任意一个或多个边缘上。通孔的形状可以是三角形、圆形、矩形、跑道形等任意形状,在此不作限制。
请参阅图15,在某些实施方式中,电子装置100还包括盖板10,盖板10设置在第一显示屏12的正面123所在的一侧。当第一显示屏12开设有开口121,并且激光投射器14和图像采集器15均与开口121对准时,盖板10的与激光投射器14和图像采集器15对应的区域上设置有红外透过层61。此时,开口121的数量可以为一个或两个。
盖板10可以由玻璃或者蓝宝石等透光性能较好的材料制成。红外透过层61可以是红外透过油墨或红外透过膜,红外透过层61对红外光(例如波长为940纳米的光)具有较高的透过率,例如透过率可以达到85%或以上,而对红外光以外的光线的透过率较低或者使得红外光以外的光线完全不能透过。因此,使用者难以通过盖板10看到与开口121对准的激光投射器14或者图像采集器15,电子装置100的外观较美观。
请参阅图16,在某些实施方式中,电子装置100还包括盖板10,盖板10设置在第一显示屏12的正面123所在的一侧,盖板10的与激光投射器14对应的区域形成有红外增透膜62。
红外增透膜62可以增加红外光的透过率,当激光投射器14投射红外激光时,红外增透膜62可以增加红外激光穿过盖板10的透过率,以减少红外激光穿过盖板10时的损耗,进而降低电子装置100的功耗。具体地,红外增透膜62可以镀在盖板10的上表面、或下表面、或同时镀在上表面及下表面。
当然,盖板10上与图像采集器15对应的区域也可以形成有红外增透膜62,以减少外界的红外光到达图像采集器15前穿过盖板10的损耗。盖板10上与泛光灯50对应的区域也可以形成有红外增透膜62,以减少泛光灯50发出的补充光线在穿过盖板10时的损耗。盖板10的与可见光摄像头40(包 括主摄像头41和副摄像头42)对应的位置可以设置有可见光增透膜63,以减少外界的可见光到达可见光摄像头40前穿过盖板10的损耗。
请参阅图17,在某些实施方式中,激光投射器14包括激光光源140和第一驱动器147,第一驱动器147可用于驱动激光光源140向待测物投射激光。激光投射器14和图像采集器15均与处理器20连接。处理器20可以为激光投射器14提供使能信号,具体地,处理器20可以为第一驱动器147提供使能信号。图像采集器15通过I2C总线与处理器20连接。激光投射器14可以向外发射激光,例如红外激光,激光到达场景中的物体上后被反射,被反射的激光可由图像采集器15接收,处理器20可以依据激光投射器14发射的激光及图像采集器15接收的激光计算物体的深度信息。在一个例子中,激光投射器14和图像采集器15可通过飞行时间(Time of flight,TOF)测距法获取深度信息,在另一个例子中,激光投射器14和图像采集器15可通过结构光测距原理获取深度信息。本申请说明书以激光投射器14和图像采集器15通过结构光测距原理获取深度信息为例进行说明,此时,激光投射器14为红外激光发射器,图像采集器15为红外摄像头。
图像采集器15与激光投射器14配合使用时,在一个例子中,图像采集器15可以通过选通信号(strobe信号)控制激光投射器14的投射时序,其中,strobe信号是根据图像采集器15获取采集图像的时序来生成的,strobe信号可视为高低电平交替的电信号,激光投射器14根据strobe指示的激光投射时序来投射激光。具体地,处理器20可以通过I2C总线发送图像采集指令以启用激光投射器14和图像采集器15,并使激光投射器14和图像采集器15工作。图像采集器15接收到图像采集指令后,通过strobe信号控制开关器件30,若strobe信号为高电平,则开关器件30向第一驱动器147发送第一脉冲信号(pwn1),第一驱动器147根据第一脉冲信号驱动激光光源140向场景中投射激光,若strobe信号为低电平,则开关器件30停止发送第一脉冲信号至第一驱动器147,激光光源140不投射激光;或者,也可以是在strobe信号为低电平时,开关器件30向第一驱动器147发送第一脉冲信号,第一驱动器147根据第一脉冲信号驱动激光光源140向场景中投射激光,在strobe信号为高电平时,开关器件30停止发送第一脉冲信号至第一驱动器147,激光光源140不投射激光。在另一个例子中,图像采集器15与激光投射器14配合时可以无需用到strobe信号,此时,处理器20发送图像采集指令至图像采集器15并同时发送激光投射指令至第一驱动器147,图像采集器15接收到图像采集指令后开始获取采集图像,第一驱动器147接收到激光投射指令时驱动激光光源140投射激光。激光投射器14投射激光时,激光形成带有斑点的激光图案投射在场景中的待测物上。图像采集器15采集被待测物反射的激光图案得到散斑图像,并通过移动产业处理器接口(Mobile Industry Processor Interface,MIPI)将散斑图像发送给处理器20。图像采集器15每发送一帧散斑图像给处理器20,处理器20就接收到一个数据流。处理器20可以根据散斑图像和预存在处理器20中的参考图像做深度图像的计算。
在某些实施方式中,可见光摄像头40也通过I2C总线与处理器20连接,也就是说,主摄像头41和副摄像头42均可以通过I2C总线与处理器20连接。可见光摄像头40可用于采集可见光图像;也就是说,主摄像头41和副摄像头42均可以分别用于采集可见光图像,或者主摄像头41和副摄像头42相配合并一起用于采集可见光图像;换句话说,主摄像头41和副摄像头42中的任意一个或两个可以用于采集可见光图像。可见光摄像头40(主摄像头41和/或副摄像头42)每发送一帧可见光图像给处理器20,处理器20就接收到一个数据流。可见光摄像头40可以单独使用,即使用者仅仅想要获取可见光图像时,处理器20通过I2C总线向可见光摄像头40(主摄像头41和副摄像头42中的任意一个或两个)发送图像采集指令以启用可见光摄像头40使其工作。可见光摄像头40接收到图像采集指令后采集场景的可见光图像,并通过移动产业处理器接口向处理器20发送可见光图像。可见光摄像头40(主摄像头41和副摄像头42中的任意一个,或者主摄像头41和副摄像头42一起)也可以与激光投射器14以及图像采集器15配合使用,即使用者想要通过可见光图像与深度图像获取三维图像时,若图像采集器15与可见光摄像头40工作频率相同,则图像采集器15与可见光摄像头40通过sync信号实现硬件同步。具体地,处理器20通过I2C总线向图像采集器15发送图像采集指令。图像采集器15接收到图像采集指令后,可以通过strobe信号控制开关器件30向第一驱动器147发送第一脉冲信号(pwn1),以使第一驱动器147根据第一脉冲信号驱动激光光源140发射激光;同时,图像采集器15与可见光摄像头40之间通过sync信号同步,该sync信号控制可见光摄像头40采集可见光图像。
电子装置100还可包括设置在第一面111上的泛光灯50。泛光灯50可以向场景中发射均匀的面光,泛光灯50包括泛光光源51及第二驱动器52,第二驱动器52可用于驱动泛光光源51发射均匀的面光。泛光灯50发出的光可以是红外光或其他不可见光,例如紫外光等。本申请以泛光灯50发射红外光为例进行说明,但泛光灯50发射的光的形式并不限于此。泛光灯50与处理器20连接,处理器20可以为驱动泛光灯50提供使能信号,具体地,处理器20可以为第二驱动器52提供使能信号。泛光灯50可以与图像采集器15配合工作以采集红外图像。图像采集器15与泛光灯50配合使用时,在一个例子 中,图像采集器15可以通过选通信号(strobe信号,该strobe信号与图像采集器15控制激光投射器14的strobe信号为两个独立的strobe信号)控制泛光灯50发射红外光的发射时序,strobe信号是根据图像采集器15获取采集图像的时序来生成的,strobe信号可视为高低电平交替的电信号,泛光灯50根据strobe信号指示的红外光发射时序来发射红外光。具体地,处理器20可以通过I2C总线向图像采集器15发送图像采集指令,图像采集器15接收到图像采集指令后,通过strobe信号控制开关器件30,若strobe信号为高电平,则开关器件30向第二驱动器52发送第二脉冲信号(pwn2),第二驱动器52根据第二脉冲信号控制泛光光源51发射红外光,若strobe信号为低电平,则开关器件30停止发送第二脉冲信号至第二驱动器52,泛光光源51不发射红外光;或者,也可以是在strobe信号为低电平时,开关器件30向第二驱动器52发送第二脉冲信号,第二驱动器52根据第二脉冲信号控制泛光光源51发射红外光,在strobe信号为高电平时,开关器件30停止发送第二脉冲信号至第二驱动器52,泛光光源51不发射红外光。泛光灯50发射红外光时,图像采集器15接收被场景中的物体反射的红外光以形成红外图像,并通过移动产业处理器接口将红外图像发送给处理器20。图像采集器15每发送一帧红外图像给处理器20,处理器20就接收到一个数据流。该红外图像通常用于虹膜识别、人脸识别等。
请参阅图1及图18,在某些实施方式中,电子装置100还包括图像采集器15,图像采集器15设置在第一面111上,在激光投射器14投射激光时(激光投射器14以第一模式投射激光时,或者激光投射器14以第二模式投射激光时),激光投射器14以第一工作频率向场景投射激光,控制方法还包括:
0184,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
0185,在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;和
0186,根据第一图像、第二图像及参考图像计算深度图像。
也就是说,控制方法包括:
0181,确定电子装置100的使用状态;
0182,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0183,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量;
0184,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
0185,在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;和
0186,根据第一图像、第二图像及参考图像计算深度图像。
上述的控制方法也可以由电子装置100实现,其中,步骤0181与前文所述的步骤031基本相同,步骤0182与前文所述的步骤032基本相同,步骤0183与前文所述的步骤033基本相同。图像采集器15可用于实现步骤0184,处理器20还可以用于实现步骤0185和步骤0186。也就是说,图像采集器15用于以第二工作频率获取采集图像,处理器20还用于在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像、及根据第一图像、第二图像及参考图像计算深度图像。
具体地,图像采集器15与激光投射器14工作频率不同(即第二工作频率大于第一工作频率)时,若需要获取深度图像,比如在解锁、支付、解密、三维建模等使用场景下。在一个例子中,处理器20通过I2C总线向图像采集器15和第一驱动器147同时发送获取深度图像的图像采集指令。第一驱动器147接收到图像采集指令后,驱动激光光源140以第一工作频率向场景发射红外激光;图像采集器15接收到图像采集指令后,以第二工作频率采集被场景中的物体反射回的红外激光以获取采集图像。例如图19所示,实线表示激光投射器14发射激光的时序,虚线表示图像采集器15获取采集图像的时序及采集图像的帧数,点划线表示根据第一图像和第二图像得到的第三图像的帧数,图19中由上至下,依次为实线、虚线及点划线,其中,第二工作频率为第一工作频率的两倍。请参阅图19中实线与虚线部分,图像采集器15在激光投射器14未投射激光时先接收环境中的红外光(下称环境红外光)以获取第N帧采集图像(此时为第一图像,也可称作背景图像),并通过移动产业处理器接口将第N帧采集图像发送给处理器20;随后,图像采集器15可以在激光投射器14投射激光时接收环境红外光以及由激光投射器14发射的红外激光以获取第N+1帧采集图像(此时为第二图像,也可称作干扰散斑图像),并通过移动产业处理器接口将第N+1帧采集图像发送给处理器20;随后,图像采集器15再在激光投射器14未投射激光时接收环境红外光以获取第N+2帧采集图像(此时为第一图像),并通过移动产业处理器接口将第N+2帧采集图像发送给处理器20,依此类推,图像采集器15交替地获取第一图像和第二图像。
在另一个例子中,处理器20通过I2C总线向图像采集器15发送获取深度图像的采集指令。图像 采集器15接收到图像采集指令后,通过strobe信号控制开关器向第一驱动器147发送第一脉冲信号,第一驱动器147根据第一脉冲信号驱动激光光源140以第一工作频率投射激光(即激光投射器14以第一工作频率投射激光),同时图像采集器15以第二工作频率采集被场景中的物体反射回的红外激光以获取采集图像。如图19所示,实线表示激光投射器14发射激光的时序,虚线表示图像采集器15获取采集图像的时序及采集图像的帧数,点划线表示根据第一图像和第二图像得到的第三图像的帧数,图19中由上至下,依次为实线、虚线及点划线,其中,第二工作频率为第一工作频率的两倍。请参阅图19中实线与虚线部分,图像采集器15在激光投射器14未投射激光时先接收环境红外光以获取第N帧采集图像(此时为第一图像,也可称作背景图像),并通过移动产业处理器接口将第N帧采集图像发送给处理器20;随后,图像采集器15可以在激光投射器14投射激光时接收环境红外光以及由激光投射器14发射的红外激光以获取第N+1帧采集图像(此时为第二图像,也可称作干扰散斑图像),并通过移动产业处理器接口将第N+1帧采集图像发送给处理器20;随后,图像采集器15再在激光投射器14未投射激光时接收环境红外光以获取第N+2帧采集图像(此时为第一图像),并通过移动产业处理器接口将第N+2帧采集图像发送给处理器20,依此类推,图像采集器15交替地获取第一图像和第二图像。
需要说明的是,图像采集器15可以在发送采集图像给处理器20的过程中同时执行采集图像的获取。并且,图像采集器15也可以先获取第二图像,再获取第一图像,并根据这个顺序交替执行采集图像的获取。另外,上述的第二工作频率与第一工作频率之间的倍数关系仅为示例,在其他实施例中,第二工作频率与第一工作频率之间的倍数关系还可以是三倍、四倍、五倍、六倍等等。
处理器20每接收到一帧采集图像后,都会对接收到的采集图像进行区分,判断采集图像是第一图像还是第二图像。处理器20接收到至少一帧第一图像和至少一帧第二图像后,即可根据第一图像、第二图像以及参考图像计算深度图像。具体地,由于第一图像是在激光投射器14未投射激光时采集的,形成第一图像的光线仅包括环境红外光,而第二图像是在激光投射器14投射激光时采集的,形成第二图像的光线同时包括环境红外光和激光投射器14发射的红外激光,因此,处理器20可以根据第一图像来去除第二图像中的由环境红外光形成的采集图像的部分,从而得到仅由红外激光形成的采集图像(即由红外激光形成的散斑图像)。
可以理解,环境光包括与激光投射器14发射的红外激光波长相同的红外光(例如,包含940nm的环境红外光),图像采集器15获取采集图像时,这部分红外光也会被图像采集器15接收。在场景的亮度较高时,图像采集器15接收的光线中环境红外光的占比会增大,导致采集图像中的激光散斑点不明显,从而影响深度图像的计算。
本申请的控制方法控制激光投射器14与图像采集器15以不同的工作频率工作,图像采集器15可以采集到仅由环境红外光形成的第一图像以及同时由环境红外光和激光投射器14发射的红外激光形成的第二图像,并基于第一图像去除掉第二图像中由环境红外光形成的图像部分,由此能够区分出激光散斑点,并能采用仅由激光投射器14发射的红外激光形成的采集图像来计算深度图像,激光散斑匹配不受影响,可以避免深度信息出现部分或全部缺失,从而提升深度图像的精确度。
请参阅图1和图20,在某些实施方式中,控制方法包括:
0201,确定电子装置100的使用状态;
0202,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0203,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量;
0204,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
02051,为每一帧采集图像添加图像类型;和
02052,根据图像类型区分第一图像与第二图像;和
0206,根据第一图像、第二图像及参考图像计算深度图像。
请参阅图1和图21,在某些实施方式中,控制方法包括:
0211,确定电子装置100的使用状态;
0212,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0213,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量;
0214,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
021511,根据每一帧采集图像的采集时间确定在采集时间下激光投射器14的工作状态;
021512,根据工作状态为每一帧采集图像添加图像类型;
02152,根据图像类型区分第一图像与第二图像;和
0216,根据第一图像、第二图像及参考图像计算深度图像。
上述的控制方法也可以由电子装置100实现,其中,步骤0201与前文所述的步骤031基本相同,步骤0202与前文所述的步骤032基本相同,步骤0203与前文所述的步骤033基本相同,步骤0204与前文所述的步骤0184基本相同,步骤02051和步骤02052可以为前文所述的步骤0185的子步骤,步骤0206与前文所述的步骤0186基本相同;步骤0211与前文所述的步骤031基本相同,步骤0212与前文所述的步骤032基本相同,步骤0213与前文所述的步骤033基本相同,步骤0214与前文所述的步骤0184基本相同,步骤021511和步骤021512可以为前文所述的步骤02051的子步骤,步骤02152与前文所述的步骤02052基本相同,步骤0216与前文所述的步骤0186基本相同。步骤02051、步骤02052、步骤021511、步骤021512和步骤02152均可以由处理器20实现。也即是说,处理器20还可用于为每一帧采集图像添加图像类型、以及根据图像类型区分第一图像与第二图像。处理器20用于为每一帧采集图像添加图像类型时,具体用于根据每一帧采集图像的采集时间确定在采集时间下激光投射器14的工作状态、以及根据工作状态为每一帧采集图像添加图像类型。
具体地,处理器20每从图像采集器15接收到一帧采集图像,都会为采集图像添加图像类型(stream_type),以便于后续处理中可以根据图像类型区分出第一图像和第二图像。具体地,在图像采集器15获取采集图像的期间,处理器20会通过I2C总线实时监测激光投射器14的工作状态。处理器20每从图像采集器15接收到一帧采集图像,会先获取采集图像的采集时间,再根据采集图像的采集时间来判断在采集图像的采集时间下激光投射器14的工作状态是投射激光还是未投射激光,并基于判断结果为采集图像添加图像类型。其中,采集图像的采集时间可以是图像采集器15获取每一帧采集图像的开始时间、结束时间、介于开始时间至结束时间之间的任意一个时间等等。如此,可以实现每一帧采集图像与激光投射器14在该帧采集图像获取期间的工作状态(投射激光或未投射激光)的对应,准确区分出采集图像的类型。在一个例子中,图像类型stream_type的结构如表1所示:
表1
表1中stream为0时,表示此时的数据流为由红外光和/或红外激光形成的图像。light为00时,表示此时的数据流是在没有任何设备投射红外光和/或红外激光(仅有环境红外光)的情形下获取的,那么处理器20可以对采集图像添加000的图像类型,以标识这一采集图像为第一图像。light为01时,表示此时的数据流是在激光投射器14投射红外激光(既有环境红外光,又有红外激光)的情形下获取的。处理器20可以对采集图像添加001的图像类型,以标识这一采集图像为第二图像。处理器20后续即可根据stream_type来区分采集图像的图像类型。
请参阅图1及图22,在某些实施方式中,控制方法包括:
0221,确定电子装置100的使用状态;
0222,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0223,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量;
0224,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
0225,在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;
02261,根据第一图像和第二图像计算第三图像,第一图像的采集时间与第二图像的采集时间的差值小于预定差值;和
02262,根据第三图像和参考图像计算深度图像。
上述的控制方法也可以由电子装置100实现,其中,步骤0221与前文所述的步骤031基本相同,步骤0222与前文所述的步骤032基本相同,步骤0223与前文所述的步骤033基本相同,步骤0224与前文所述的步骤0184基本相同,步骤0225与前文所述的步骤0185基本相同,步骤02261和步骤02262可以为前文所述的步骤0186的子步骤;步骤02261及步骤02262均可以由处理器20实现。也即是说,处理器20还可以用于根据第一图像和第二图像计算第三图像、以及根据第三图像和参考图像计算深度图像,其中,第一图像的采集时间与第二图像的采集时间的差值小于预定差值。
在计算深度图像的过程中,处理器20可以先区分出第一图像与第二图像,再根据采集时间选出任意帧第二图像和与该任意帧第二图像对应的特定帧的第一图像,其中该特定帧的第一图像的采集时间与该任意帧的第二图像的采集时间的差值小于预定差值。随后,处理器20再根据该特定帧的第一图像 和该任意帧的第二图像来计算第三图像,第三图像即为仅由激光投射器14发射的红外激光形成的采集图像,也可以称作实际散斑图像。具体地,第一图像中的多个像素点与第二图像中的多个像素点是一一对应的,假设第一图像为P1,第二图像为P2,第三图像为P3,处理器20可以将第二图像中的像素点P2
i,j的像素值减去第一图像中的像素点P1
i,j的像素值以得到第三图像中像素点P3
i,j的像素值,即P3
i,j=P2
i,j-P1
i,j,i∈N+,j∈N+。随后,处理器20可以根据第三图像与参考图像计算出深度图像,其中,第二图像的帧数、第三图像的帧数及深度图像的帧数均相等。可以理解,由于第一图像的采集时间和第二图像的采集时间的差值较小,那么第一图像中环境红外光的强度与第二图像中环境红外光的强度更为接近,基于第一图像和第二图像计算出的第三图像的精度更高,有利于进一步减小环境红外光对深度图像获取的影响。
在某些实施方式中,处理器20也可以为第三图像和深度图像添加图像类型,以便于对处理采集图像后得到的各个数据流进行区分。如表2所示:
表2
表2中的stream为0时,表示此时的数据流为由红外光和/或红外激光形成的图像,stream为1时,表示此时的数据流为深度图像。light为11时,表示减背景处理,减背景处理即去除采集图像中由环境红外光形成的部分,那么处理器20可以对减背景处理后的数据流添加011的图像类型,以标识这一数据流为第三图像。Light为XX时,X表示不限定取值,处理器20可对进行深度计算后得到的数据流添加1XX的图像类型,以标识这一数据流为深度图像。
在某些实施方式中,参与深度图像计算的第一图像和第二图像中,第一图像的采集时间可以位于第二图像的采集时间之前,也可以位于第二图像的采集时间之后,在此不作限制。
在某些实施方式中,第一图像的采集时间与第二图像的采集时间的差值小于预定差值时,第一图像和第二图像可以是相邻帧的图像,也可以是非相邻帧的图像。例如,在第二工作频率是第一工作频率的两倍时,差值小于预定差值的第一图像和第二图像是相邻帧的图像;在第二工作频率与第一工作频率之间的倍数大于两倍,例如第二工作频率是第一工作频率的三倍时,差值小于预定差值的第一图像和第二图像可以是相邻帧的图像,也可以是非相邻帧的图像(此时第一图像与第二图像之间还间隔有一帧第一图像)。
在某些实施方式中,参与深度图像计算的第一图像的帧数还可以为多帧。比如,在第二工作频率是第一工作频率的三倍时,可以选取两帧相邻的第一图像以及与这两帧第一图像相邻的一帧第二图像来计算第三图像。此时,处理器20可以先对两帧第一图像做融合处理,例如,将两帧第一图像对应像素点的像素值相加再取均值得到融合处理后的第一图像,再利用融合处理后的第一图像和该相邻的一帧第二图像计算第三图像。
在某些实施方式中,处理器20可以计算出多帧第三图像,如图19中的第(N+1)-N帧第三图像、第(N+3)-(N+2)帧第三图像、第(N+5)-(N+4)帧第三图像等等,并对应多帧第三图像计算出多帧深度图像。当然,在其他实施方式中,处理器20也可以仅计算出一帧第三图像,并对应一帧第三图像计算出一帧深度图像。第三图像的帧数可以根据应用场景的安全级别来确定。具体地,当应用场景的安全级别较高时,例如对于支付等安全级别较高的应用场景,第三图像的帧数应该较多,此时需要多帧深度图像与使用者的深度模板的匹配均成功才执行支付动作,以提升支付的安全性;而对于应用场景的安全级别较低,例如对于基于深度信息进行人像美颜的应用场景,第三图像的帧数可以较少,例如,为一帧,此时利用一帧深度图像即足够进行人像美颜,如此,可以减少处理器20的计算量及功耗,并可以提升图像处理的速度。
请参阅图1及图23,在某些实施方式中,控制方法还包括:
0237,以第三工作频率采集可见光图像,第三工作频率大于或小于第二工作频率;
0238,为每一帧可见光图像和每一帧采集图像添加采集时间;和
0239,根据可见光图像的采集时间、采集图像的采集时间及采集图像的图像类型确定帧同步的可见光图像和第二图像。
也就是说,控制方法包括:
0231,确定电子装置100的使用状态;
0232,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0233,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射 的激光的能量大于第一模式投射的激光的能量;
0234,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
0235,在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;
0236,根据第一图像、第二图像及参考图像计算深度图像;
0237,以第三工作频率采集可见光图像,第三工作频率大于或小于第二工作频率;
0238,为每一帧可见光图像和每一帧采集图像添加采集时间;和
0239,根据可见光图像的采集时间、采集图像的采集时间及采集图像的图像类型确定帧同步的可见光图像和第二图像。
上述的控制方法也可以由电子装置100实现,其中,步骤0231与前文所述的步骤031基本相同,步骤0232与前文所述的步骤032基本相同,步骤0233与前文所述的步骤033基本相同,步骤0234与前文所述的步骤0184基本相同,步骤0235与前文所述的步骤0185基本相同,步骤0236与前文所述的步骤0186基本相同;步骤0237可以由可见光摄像头40(主摄像头41和副摄像头42中的任意一个,或者主摄像头41和副摄像头42一起)实现。步骤0238和步骤0239可以由处理器20实现。也即是说,可见光摄像头40可用于以第三工作频率采集可见光图像,第三工作频率大于或小于第二工作频率。处理器20可用于为每一帧可见光图像和每一帧采集图像添加采集时间、以及根据可见光图像的采集时间、采集图像的采集时间及采集图像的图像类型确定帧同步的可见光图像和第二图像。
在一些应用场景,例如,对场景中的物体进行三维建模的应用场景下,需要借助图像采集器15获取场景中物体的深度信息,并且借助可见光摄像头40获取场景中物体的色彩信息,才能实现三维建模。此时,处理器20需要开启图像采集器15获取深度图像并同时开启可见光摄像头40获取可见光图像。
若图像采集器15与可见光摄像头40具有相同的工作频率,即图像采集器15与可见光摄像头40均以第二工作频率工作,那么处理器20可以通过I2C总线发送图像采集指令至图像采集器15,图像采集器15接收到图像采集指令后,图像采集器15与可见光摄像头40之间通过sync信号同步,该sync信号控制可见光摄像头40开启采集可见光图像,以实现图像采集器15与可见光摄像头40的硬件同步。此时,采集图像的帧数与可见光图像的帧数一致,每一帧采集图像与每一帧可见光图像一一对应。
但在图像采集器15与可见光摄像头40的工作频率不同,即图像采集器15以第二工作频率工作,可见光摄像头40以不等于第二工作频率的第三工作频率工作时,图像采集器15与可见光摄像头40无法实现硬件同步。此时,处理器20需要通过软件同步的方式来实现图像采集器15与可见光摄像头40的同步。具体地,处理器20通过与图像采集器15连接的I2C总线发送图像采集指令至图像采集器15,同时通过与可见光摄像头40连接的I2C总线发送图像采集指令至可见光摄像头40。处理器20每接收到一帧采集图像时,会为每一帧采集图像添加图像类型,还会为每一帧采集图像添加采集时间。并且,处理器20每接收到一帧可见光图像时,会为每一帧可见光图像添加采集时间。其中,采集图像的采集时间可以是图像采集器15采集每一帧采集图像的开始时间、结束时间、介于开始时间至结束时间之间的任意一个时间等等;可见光图像的采集时间可以是可见光摄像头40采集每一帧可见光图像的开始时间、结束时间、介于开始时间至结束时间之间的任意一个时间等等。那么,在后续基于深度图像和可见光图像做进一步处理(如三维建模、借助深度信息做人像美颜等处理)时,处理器20可以先根据可见光图像的采集时间、采集图像的采集时间及采集图像的类型先确定帧同步的可见光图像和第二图像。其中,帧同步指的是确定出的第二图像的采集时间与可见光图像的采集时间的差值小于预设的时间差值,可见光图像的采集时间可以位于第二图像的采集时间之前也可以位于第二图像的采集时间之后。随后,处理器20再根据确定的第二图像选出第一图像以进一步根据第二图像、第一图像及参考图像计算深度图像。最后,处理器20基于深度图像和确定出的可见光图像进行后续处理。
在某些实施方式中,处理器20也可以为每一帧深度图像添加采集时间,再根据可见光图像的采集时间和深度图像的采集时间确定帧同步的可见光图像以及深度图像,最后对帧同步的可见光图像以及深度图像做后续处理。其中,每一帧深度图像的采集时间为与该帧深度图像对应的第二图像的采集时间。
请参阅图24,在某些实施方式中,采集图像还包括红外图像,红外图像为图像采集器15采集泛光灯50发射的红外光所得到的图像。处理器20为每一帧采集图像添加图像类型时,还会为红外图像添加图像类型。在一个例子中,红外图像的图像类型如表3所示:
表3
表3中的stream为0时,表示此时的数据流为由红外光和/或红外激光形成的图像。light为10时,表示此时的数据流是在泛光灯50投射红外光且激光投射器14未投射激光的情形下获取的。那么,处理器20为采集图像添加010的图像类型时,即标识这一帧采集图像为红外图像。
在某些应用场景中,比如同时基于深度图像与深度模板的匹配以及红外图像与红外模板的匹配实现身份验证的应用场景,图像采集器15需要与泛光灯50及激光投射器14配合使用,图像采集器15可以分时获取第一图像、第二图像及红外图像。如图24所示,实线表示激光投射器14发射激光的时序,双点划线表示泛光灯50发射红外光的时序,虚线表示图像采集器15获取采集图像的时序及采集图像的帧数,点划线表示根据第一图像和第二图像得到的第三图像的帧数,图24中由上至下,依次为实线、双点划线、虚线及点划线,其中,第二工作频率为第一工作频率的三倍,第二工作频率为第四工作频率的三倍。处理器20可以通过I2C总线实时监测泛光灯50的工作状态。处理器20每从图像采集器15接收到一帧采集图像,会先获取采集图像的采集时间,再根据采集图像的采集时间来判断在采集图像的采集时间下泛光灯50的工作状态是发射红外光还是未发射红外光,并基于判断结果为采集图像添加图像类型。处理器20后续可以基于红外图像的采集时间和第二图像的采集时间确定出采集时间的差值小于设定差值的红外图像和第二图像,进一步地,处理器20可以确定出红外图像和深度图像,并利用该红外图像和该深度图像进行身份验证。
请参阅图1及图25,在某些实施方式中,控制方法还包括:
0251,获取场景的亮度及类型;
0252,判断亮度是否大于亮度阈值且类型为户外场景;
若是,则进入确定电子装置100的使用状态的步骤(步骤0253)。
也就是说,控制方法包括:
0251,获取场景的亮度及类型;
0252,判断亮度是否大于亮度阈值且类型为户外场景;
0253,若是,确定电子装置100的使用状态;
0254,在电子装置100处于第一使用状态时,激光投射器14以第一模式投射激光;
0255,在电子装置100处于第二使用状态时,激光投射器14以第二模式投射激光,第二模式投射的激光的能量大于第一模式投射的激光的能量;
0256,图像采集器15以第二工作频率获取采集图像,第二工作频率大于第一工作频率;
0257,在采集图像中区分出在激光投射器14未投射激光时采集的第一图像及在激光投射器14投射激光时采集的第二图像;和
0258,根据第一图像、第二图像及参考图像计算深度图像。
上述的控制方法也可以由电子装置100实现,其中,步骤0253与前文所述的步骤031基本相同,步骤0254与前文所述的步骤032基本相同,步骤0255与前文所述的步骤033基本相同,步骤0256与前文所述的步骤0184基本相同,步骤0257与前文所述的步骤0185基本相同,步骤0258与前文所述的步骤0186基本相同;步骤0251和步骤0252均可以由处理器20实现。也即是说,处理器20可用于获取场景的亮度及类型、以及判断亮度是否大于亮度阈值且类型为户外场景。激光投射器14可用于在亮度大于亮度阈值且类型为户外场景时以第一工作频率向场景投射激光。
具体地,场景的亮度可以通过分析图像采集器15获取的采集图像或可见光摄像头40(主摄像头41和副摄像头42中的任意一个,或者主摄像头41和副摄像头42一起)获取的可见光图像得到;或者,场景的亮度也可以由光线感应器来直接检测,处理器20从光线感应器读取检测得到的信号以获取场景的亮度。场景的类型可以通过分析图像采集器15获取的采集图像或可见光摄像头40获取的可见光图像得到,例如分析采集图像或可见光摄像头40获取的可见光图像中的物体来判断场景的类型为户外场景还是户内场景;场景的类型也可以直接根据地理位置来确定,具体地,处理器20可以获取全球卫星定位***对场景的定位结果,再根据定位结果进一步判断场景的类型,例如,定位结果为某某办公楼,则说明场景为户内场景;定位场景为某某公园,则说明场景为户外场景;定位场景为某某街道,则说明场景为户外场景等等。
可以理解,在场景的亮度较高(例如亮度大于亮度阈值)时,采集图像中环境红外光的占比会较多,对斑点的识别会较大影响,此时需要去除环境红外光的干扰。但是在场景的亮度较低时,采集图像中环境红外光的占比会较少,对斑点的识别产生的影响较小,可以忽略不计,此时图像采集器15和激光投射器14可以采用相同工作频率工作,处理器20直接根据图像采集器15获取的采集图像(即第二图像)与参考图像计算深度图像。另外,场景的亮度较高时可能是户内的灯光光线较强引起的,由于灯光光线不包括红外光,不会对斑点的识别产生较大影响,此时图像采集器15和激光投射器14采用相同工作频率工作,处理器20直接根据图像采集器15获取的采集图像(即第二图像)与参考图像 计算深度图像。如此,可以减小图像采集器15的工作频率,减少图像采集器15的功耗。
当然,在某些实施方式中,控制方法也可以仅仅基于场景的亮度来判断是否执行步骤0253。具体地,处理器20仅仅获取场景的亮度,判断场景的亮度是否大于亮度阈值,激光投射器14在亮度大于亮度阈值时以第一工作频率向场景投射激光。
在某些实施方式中,处理器20还可以为每一个数据流添加状态信息(status)。在一个例子中,如表4所示:
表4
状态信息status为0时,表示该数据流未经过减背景处理,状态信息status为1时,表示该数据流经过减背景处理。表4中的0000即表示第一图像;0010即表示第二图像;0100即表示泛光灯50开启时图像采集器15获取的红外图像;0111即表示第三图像;1XX1即表示经过减背景处理的深度图像;1XX0即表示未经过减背景处理的深度图像。如此,为每个数据流添加状态信息以便于处理器20分辨各个数据流是否经过减背景处理。
在某些实施方式中,处理器20包括第一存储区、第二存储区以及逻辑减电路,逻辑减电路与第一存储区及第二存储区均连接。其中,第一存储区用于存储第一图像,第二存储区用于存储第二图像,逻辑减电路用于处理第一图像和第二图像得到第三图像。具体地,逻辑减电路从第一存储区读取第一图像,从第二存储区读取第二图像,在获取到第一图像和第二图像后,对第一图像和第二图像执行减法处理得到第三图像。逻辑减电路还与处理器20中的深度计算模块(例如,可以是专门用于计算深度的集成电路ASIC等)连接,逻辑减电路将第三图像发送到深度计算模块中,由深度计算模块根据第三图像和参考图像计算深度图像。
在本说明书的描述中,参考术语“一个实施方式”、“一些实施方式”、“示意性实施方式”、“一个例子”“示例”、“具体示例”或“一些示例”等的描述意指结合所述实施方式或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施方式或示例中。在本说明书中,对上述术语的示意性表述不一定指的是相同的实施方式或示例。而且,描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施方式或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本申请的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本申请的实施例所属技术领域的技术人员所理解。
尽管上面已经示出和描述了本申请的实施方式,可以理解的是,上述实施方式是示例性的,不能理解为对本申请的限制,本领域的普通技术人员在本申请的范围内可以对上述实施方式进行变化、修改、替换和变型。
Claims (20)
- 一种电子装置的控制方法,其特征在于,所述电子装置包括机壳、第一显示屏、第二显示屏及激光投射器,所述机壳包括相背的第一面和第二面,所述第一显示屏和所述第二显示屏分别设置在所述第一面和所述第二面,所述激光投射器设置在所述第一面;所述控制方法包括:确定所述电子装置的使用状态;在所述电子装置处于第一使用状态时,所述激光投射器以第一模式投射激光;及在所述电子装置处于第二使用状态时,所述激光投射器以第二模式投射激光,所述第二模式投射的所述激光的能量大于所述第一模式投射的所述激光的能量。
- 根据权利要求1所述的控制方法,其特征在于,所述电子装置还包括重力传感器,所述确定所述电子装置的使用状态,包括:通过所述重力传感器确定所述电子装置的使用状态。
- 根据权利要求2所述的控制方法,其特征在于,所述通过所述重力传感器确定所述电子装置的使用状态,包括:获取所述重力传感器的z轴的数值;在所述z轴的数值大于第一预设值时,确定所述电子装置处于所述第一使用状态;及在所述z轴的数值小于第二预设值时,确定所述电子装置处于所述第二使用状态。
- 根据权利要求1所述的控制方法,其特征在于,所述确定所述电子装置的使用状态,包括:通过所述第一显示屏检测到的第一触控信号和/或所述第二显示屏检测到的第二触控信号确定所述电子装置的使用状态。
- 根据权利要求4所述的控制方法,其特征在于,所述通过所述第一显示屏检测到的第一触控信号和/或所述第二显示屏检测到的第二触控信号确定所述电子装置的使用状态,包括:在所述第一显示屏检测到所述第一触控信号时,确定所述电子装置处于所述第一使用状态;及在所述第二显示屏检测到所述第二触控信号时,确定所述电子装置处于所述第二使用状态。
- 根据权利要求1所述的控制方法,其特征在于,所述电子装置还包括状态选择按键,所述确定所述电子装置的使用状态,包括:通过所述状态选择按键确定所述电子装置的使用状态。
- 根据权利要求1至6任意一项所述的控制方法,其特征在于,所述激光投射器在所述第一模式下投射所述激光的功率小于在所述第二模式下投射所述激光的功率;和/或所述激光投射器包括多个点光源,多个所述点光源独立控制;所述激光投射器在所述第一模式下开启的所述点光源的数量少于在所述第二模式下开启的所述点光源的数量。
- 根据权利要求1至6任意一项所述的控制方法,其特征在于,所述电子装置还包括图像采集器,所述图像采集器设置在所述第一面上;所述激光投射器在投射激光时,所述激光投射器以第一工作频率向场景投射所述激光;所述控制方法还包括:所述图像采集器以第二工作频率获取采集图像,所述第二工作频率大于所述第一工作频率;在所述采集图像中区分出在所述激光投射器未投射激光时采集的第一图像及在所述激光投射器投射激光时采集的第二图像;和根据所述第一图像、所述第二图像及参考图像计算深度图像。
- 一种电子装置,其特征在于,所述电子装置包括机壳、第一显示屏、第二显示屏、激光投射器及处理器,所述机壳包括相背的第一面和第二面,所述第一显示屏和所述第二显示屏分别设置在所述第一面和所述第二面,所述激光投射器设置在所述第一面;所述处理器用于确定所述电子装置的使用状态;所述激光投射器用于在所述电子装置处于第一使用状态时以第一模式投射激光、以及用于在所述电子装置处于第二使用状态时以第二模式投射激光,所述第二模式投射的所述激光的能量大于所述第一模式投射的所述激光的能量。
- 根据权利要求9所述的电子装置,其特征在于,所述电子装置还包括重力传感器,所述 处理器用于通过所述重力传感器确定所述电子装置的使用状态。
- 根据权利要求10所述的电子装置,其特征在于,所述处理器还用于:获取所述重力传感器的z轴的数值;在所述z轴的数值大于第一预设值时,确定所述电子装置处于所述第一使用状态;及在所述z轴的数值小于第二预设值时,确定所述电子装置处于所述第二使用状态。
- 根据权利要求9所述的电子装置,其特征在于,所述处理器用于通过所述第一显示屏检测到的第一触控信号和/或所述第二显示屏检测到的第二触控信号确定所述电子装置的使用状态。
- 根据权利要求12所述的电子装置,其特征在于,所述处理器用于:在所述第一显示屏检测到所述第一触控信号时,确定所述电子装置处于所述第一使用状态;及在所述第二显示屏检测到所述第二触控信号时,确定所述电子装置处于所述第二使用状态。
- 根据权利要求9所述的电子装置,其特征在于,所述电子装置还包括状态选择按键,所述处理器用于通过所述状态选择按键确定所述电子装置的使用状态。
- 根据权利要求9至14任意一项所述的电子装置,其特征在于,所述激光投射器在所述第一模式下投射所述激光的功率小于在所述第二模式下投射所述激光的功率;和/或所述激光投射器包括多个点光源,多个所述点光源独立控制;所述激光投射器在所述第一模式下开启的所述点光源的数量少于在所述第二模式下开启的所述点光源的数量。
- 根据权利要求15所述的电子装置,其特征在于,多个所述点光源形成多个发光阵列,多个所述发光阵列呈环形排布。
- 根据权利要求9至14任意一项所述的电子装置,其特征在于,所述电子装置还包括图像采集器,所述图像采集器设置在所述第一面;所述激光投射器在投射激光时,所述激光投射器用于以第一工作频率向场景投射所述激光;所述图像采集器用于以第二工作频率获取采集图像,所述第二工作频率大于所述第一工作频率;所述处理器还用于:在所述采集图像中区分出在所述激光投射器未投射激光时采集的第一图像及在所述激光投射器投射激光时采集的第二图像;和根据所述第一图像、所述第二图像及参考图像计算深度图像。
- 根据权利要求9至14任意一项所述的电子装置,其特征在于,所述第一显示屏为一端开设有开口的开口屏,所述第二显示屏为全面屏,所述激光投射器安装在所述开口内;或所述第一显示屏和所述第二显示屏均为全面屏,所述激光投射器设置在所述第一显示屏的下方。
- 根据权利要求9至14任意一项所述的电子装置,其特征在于,所述第一使用状态和所述第二使用状态中的所述激光投射器的朝向相反。
- 根据权利要求12或13所述的电子装置,其特征在于,所述第一触控信号和所述第二触控信号均为电容信号。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20814207.5A EP3975528B1 (en) | 2019-05-31 | 2020-04-21 | Control method of electronic device and electronic device |
US17/537,352 US20220086266A1 (en) | 2019-05-31 | 2021-11-29 | Control Method of Electronic Device and Electronic Device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910471662.X | 2019-05-31 | ||
CN201910471662.XA CN112019674B (zh) | 2019-05-31 | 2019-05-31 | 电子装置的控制方法及电子装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/537,352 Continuation US20220086266A1 (en) | 2019-05-31 | 2021-11-29 | Control Method of Electronic Device and Electronic Device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020238491A1 true WO2020238491A1 (zh) | 2020-12-03 |
Family
ID=73506267
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/085973 WO2020238491A1 (zh) | 2019-05-31 | 2020-04-21 | 电子装置的控制方法及电子装置 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20220086266A1 (zh) |
EP (1) | EP3975528B1 (zh) |
CN (2) | CN112019674B (zh) |
WO (1) | WO2020238491A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
USD858514S1 (en) * | 2017-08-28 | 2019-09-03 | Apple Inc. | Electronic device |
CN110213413B (zh) * | 2019-05-31 | 2021-05-14 | Oppo广东移动通信有限公司 | 电子装置的控制方法及电子装置 |
CN115242978B (zh) * | 2022-07-21 | 2023-05-26 | 杭州萤石软件有限公司 | 图像采集设备及方法 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107743156A (zh) * | 2017-09-29 | 2018-02-27 | 努比亚技术有限公司 | 移动终端、移动终端的控制方法及计算机可读存储介质 |
CN107896274A (zh) * | 2017-10-27 | 2018-04-10 | 广东欧珀移动通信有限公司 | 红外发射器控制方法、终端及计算机可读存储介质 |
CN108594451A (zh) * | 2018-03-12 | 2018-09-28 | 广东欧珀移动通信有限公司 | 控制方法、控制装置、深度相机和电子装置 |
CN208445604U (zh) * | 2018-07-13 | 2019-01-29 | 北京小米移动软件有限公司 | 电子设备 |
CN109710150A (zh) * | 2018-12-25 | 2019-05-03 | 维沃移动通信有限公司 | 按键控制方法及终端 |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101512599B (zh) * | 2006-09-21 | 2012-07-18 | 汤姆森特许公司 | 三维模型获取的方法和*** |
US8284549B2 (en) * | 2010-05-24 | 2012-10-09 | Sunrex Technology Corp. | Inverted U frame with pivotal micro-projector and camera for releasably securing to laptop or keyboard |
US20140128132A1 (en) * | 2012-10-12 | 2014-05-08 | James L. Cox, III | Case with interchangeable back plates |
US8879782B2 (en) * | 2013-01-17 | 2014-11-04 | Disney Enterprises, Inc. | Projector light bulb |
DE102015202846B4 (de) * | 2014-02-19 | 2020-06-25 | Magna Electronics, Inc. | Fahrzeugsichtsystem mit Anzeige |
KR20170060353A (ko) * | 2015-11-24 | 2017-06-01 | 삼성전자주식회사 | 전자 장치, 거리 측정 센서 및 그 제어 방법 |
WO2019201010A1 (zh) * | 2018-04-16 | 2019-10-24 | Oppo广东移动通信有限公司 | 激光投射器、相机模组和电子装置 |
CN109066288A (zh) * | 2018-05-30 | 2018-12-21 | Oppo广东移动通信有限公司 | 激光投射器的控制***、终端和激光投射器的控制方法 |
CN108833693A (zh) * | 2018-06-06 | 2018-11-16 | Oppo广东移动通信有限公司 | 控制方法、控制装置、电子装置、存储介质和计算机设备 |
JP7396277B2 (ja) * | 2018-07-24 | 2023-12-12 | ソニーグループ株式会社 | 情報処理装置、情報処理方法及びプログラム |
CN109192153A (zh) * | 2018-08-31 | 2019-01-11 | 维沃移动通信有限公司 | 一种终端及终端控制方法 |
CN109324633B (zh) * | 2018-09-27 | 2021-07-02 | Oppo广东移动通信有限公司 | 控制方法与装置、飞行时间设备、终端及计算机可读存储介质 |
CN109327576B (zh) * | 2018-11-16 | 2021-02-02 | Oppo广东移动通信有限公司 | 电子装置及其控制方法和控制装置 |
CN109587295B (zh) * | 2018-11-16 | 2021-04-30 | Oppo广东移动通信有限公司 | 终端、终端的控制方法及终端的控制装置 |
CN112019660B (zh) * | 2019-05-31 | 2021-07-30 | Oppo广东移动通信有限公司 | 电子装置的控制方法及电子装置 |
US11561085B2 (en) * | 2019-06-05 | 2023-01-24 | Qualcomm Incorporated | Resolving multipath interference using a mixed active depth system |
-
2019
- 2019-05-31 CN CN201910471662.XA patent/CN112019674B/zh active Active
- 2019-05-31 CN CN202111107307.8A patent/CN113810530A/zh active Pending
-
2020
- 2020-04-21 EP EP20814207.5A patent/EP3975528B1/en active Active
- 2020-04-21 WO PCT/CN2020/085973 patent/WO2020238491A1/zh unknown
-
2021
- 2021-11-29 US US17/537,352 patent/US20220086266A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107743156A (zh) * | 2017-09-29 | 2018-02-27 | 努比亚技术有限公司 | 移动终端、移动终端的控制方法及计算机可读存储介质 |
CN107896274A (zh) * | 2017-10-27 | 2018-04-10 | 广东欧珀移动通信有限公司 | 红外发射器控制方法、终端及计算机可读存储介质 |
CN108594451A (zh) * | 2018-03-12 | 2018-09-28 | 广东欧珀移动通信有限公司 | 控制方法、控制装置、深度相机和电子装置 |
CN208445604U (zh) * | 2018-07-13 | 2019-01-29 | 北京小米移动软件有限公司 | 电子设备 |
CN109710150A (zh) * | 2018-12-25 | 2019-05-03 | 维沃移动通信有限公司 | 按键控制方法及终端 |
Also Published As
Publication number | Publication date |
---|---|
EP3975528A4 (en) | 2022-06-29 |
EP3975528B1 (en) | 2023-11-15 |
EP3975528A1 (en) | 2022-03-30 |
CN113810530A (zh) | 2021-12-17 |
CN112019674A (zh) | 2020-12-01 |
US20220086266A1 (en) | 2022-03-17 |
EP3975528C0 (en) | 2023-11-15 |
CN112019674B (zh) | 2021-10-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020238506A1 (zh) | 电子装置的控制方法及电子装置 | |
WO2020238491A1 (zh) | 电子装置的控制方法及电子装置 | |
CN107209008B (zh) | 结构化光图案生成 | |
US8576276B2 (en) | Head-mounted display device which provides surround video | |
WO2020098344A1 (zh) | 电子装置 | |
US20110234481A1 (en) | Enhancing presentations using depth sensing cameras | |
JP2015526927A (ja) | カメラ・パラメータのコンテキスト駆動型調整 | |
JPH11224160A (ja) | 座標入力システム及び表示装置 | |
WO2020088043A1 (zh) | 图像获取方法、图像获取装置、结构光组件及电子装置 | |
WO2020088045A1 (zh) | 图像获取方法、图像获取装置、结构光组件及电子装置 | |
US9268520B1 (en) | Altering content projection | |
US11143879B2 (en) | Semi-dense depth estimation from a dynamic vision sensor (DVS) stereo pair and a pulsed speckle pattern projector | |
TW202147255A (zh) | 用於活動深度感測技術的多種光學路徑成像技術和共享發射器 | |
WO2020238481A1 (zh) | 图像获取方法、图像获取装置、电子设备和可读存储介质 | |
US11947045B2 (en) | Controlling method for electronic device, and electronic device | |
US20160091966A1 (en) | Stereoscopic tracking status indicating method and display apparatus | |
Siriborvornratanakul et al. | ipProjector: Designs and Techniques for Geometry‐Based Interactive Applications Using a Portable Projector |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20814207 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2020814207 Country of ref document: EP Effective date: 20211221 |