WO2022242348A1 - dTOF深度图像的采集方法、装置、电子设备及介质 - Google Patents
dTOF深度图像的采集方法、装置、电子设备及介质 Download PDFInfo
- Publication number
- WO2022242348A1 WO2022242348A1 PCT/CN2022/085379 CN2022085379W WO2022242348A1 WO 2022242348 A1 WO2022242348 A1 WO 2022242348A1 CN 2022085379 W CN2022085379 W CN 2022085379W WO 2022242348 A1 WO2022242348 A1 WO 2022242348A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- exposure
- exposure time
- preset
- sub
- measurement period
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 141
- 238000005259 measurement Methods 0.000 claims abstract description 420
- 230000000875 corresponding effect Effects 0.000 claims description 271
- 230000003287 optical effect Effects 0.000 claims description 56
- 230000005540 biological transmission Effects 0.000 claims description 34
- 230000002596 correlated effect Effects 0.000 claims description 22
- 238000003860 storage Methods 0.000 claims description 14
- 230000008859 change Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 5
- 230000001105 regulatory effect Effects 0.000 claims 3
- 230000008569 process Effects 0.000 description 27
- 230000000694 effects Effects 0.000 description 14
- 230000006870 function Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 238000009825 accumulation Methods 0.000 description 10
- 238000009826 distribution Methods 0.000 description 10
- 238000002310 reflectometry Methods 0.000 description 10
- 230000009467 reduction Effects 0.000 description 8
- 230000003247 decreasing effect Effects 0.000 description 7
- 238000010791 quenching Methods 0.000 description 6
- 230000000171 quenching effect Effects 0.000 description 6
- 230000001934 delay Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 206010034960 Photophobia Diseases 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 208000013469 light sensitivity Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
Definitions
- the embodiments of the present application relate to the technical field of computer vision, and in particular to a method, device, electronic equipment and medium for collecting a dTOF depth image.
- the process of using dTOF to obtain depth information is as follows: the transmitting end emits modulated pulsed light with concentrated light energy, and at the same time opens the receiving end to receive photons within the pulse period. Enough photons, and then draw a distribution histogram of the number of photons received by each pixel (pixel) at different times in a pulse cycle, based on the distribution histogram, the depth information of the target to be measured can be recovered.
- This exposure method can simultaneously receive light signals reflected from long-distance targets and short-distance targets. However, when the distance is too short, the received reflected light is too strong, and overexposure may occur, resulting in measurement errors or pile up effects.
- the measurement error of short-distance targets is too large; if it is necessary to ensure accurate short-distance measurement, the number of laser pulses emitted by dTOF must be reduced or the exposure time must be shortened, but at the same time it will also result in the number of photons received from distant targets or low reflectivity targets Insufficient, thereby reducing the signal-to-noise ratio and measurement accuracy of long-distance targets or low reflectivity targets.
- the current exposure method cannot take into account the measurement accuracy of far and short distances in the same field of view, resulting in a small dynamic distance measurement range of the depth image acquisition device.
- Embodiments of the present application provide a dTOF depth image acquisition method, device, electronic equipment, and medium. Described technical scheme is as follows:
- the embodiment of the present application provides a dTOF depth image acquisition method, the method comprising:
- the laser signal is emitted through the transmitting end, wherein the measurement period of the depth information corresponding to the single-frame depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, and i is a positive value less than or equal to n integer;
- the measurement period includes sub-measurement periods corresponding to different exposure times
- the optical signal includes a reflected light signal of the laser signal
- the depth image frame is generated based on the received light signal.
- the embodiment of the present application provides a dTOF depth image acquisition device, the device includes:
- the transmitting module is used to transmit a laser signal through the transmitting terminal in the i-th sub-measurement period, wherein the measurement period of the depth information corresponding to the single-frame depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, and i is A positive integer less than or equal to n;
- the receiving module is configured to receive an optical signal through the receiving end according to the exposure time corresponding to the i-th sub-measurement period, the measurement period includes sub-measurement periods corresponding to different exposure times, and the optical signal includes the laser signal The reflected light signal;
- a generating module configured to generate the depth image frame based on the received light signal when each of the sub-measurement periods in the measurement period ends.
- an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, at least one program code is stored in the memory, and the program code is loaded and executed by the processor to realize The acquisition method of the dTOF depth image as described in the above aspects.
- an embodiment of the present application provides a computer-readable storage medium, where at least one piece of program code is stored in the computer-readable storage medium, and the program code is loaded and executed by a processor to implement the above-mentioned Acquisition method of dTOF depth image.
- an embodiment of the present application provides a computer program product, where the computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
- the processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for acquiring a dTOF depth image provided in various optional implementation manners of the above aspect.
- FIG. 1 shows a measurement sequence diagram of depth information corresponding to a single frame depth image frame in the related art
- FIG. 2 is a measurement sequence diagram corresponding to a single depth image frame shown in an exemplary embodiment of the present application
- Fig. 3 shows a flow chart of a method for acquiring a dTOF depth image provided by an exemplary embodiment of the present application
- FIG. 4 shows a measurement timing diagram corresponding to a single depth image frame shown in another exemplary embodiment of the present application
- FIG. 5 shows a flowchart of a method for acquiring a dTOF depth image provided by another exemplary embodiment of the present application
- Fig. 6 shows a schematic diagram of exposure delay time provided by an embodiment of the present application
- FIG. 7 shows a flowchart of a method for acquiring a dTOF depth image provided by another exemplary embodiment of the present application.
- Fig. 8 shows a flow chart of adjusting the exposure mode shown in an exemplary embodiment of the present application
- Fig. 9 shows a structural block diagram of a dTOF depth image acquisition device provided by an embodiment of the present application.
- Fig. 10 shows a structural block diagram of an electronic device provided by an exemplary embodiment of the present application.
- FIG. 1 shows a timing diagram of measuring depth information corresponding to a single depth image frame in the related art.
- TX the transmitter
- N the period from one laser pulse to the second laser pulse
- Td the pulse period
- N the exposure time of the receiving end (RX)
- T the maximum exposure time
- the reflected light of the short-distance target when collecting the next frame of depth image, adjust the number of pulses emitted by the transmitter to 600.
- the number of laser pulses corresponding to the acquisition of each frame of depth image although the received reflected light of close-range targets can be reduced, it will also reduce the received reflected light of long-distance targets, resulting in the received long-distance photons Insufficient number, thereby reducing the long-distance signal-to-noise ratio and measurement accuracy.
- the exposure method in the related art cannot take into account the measurement accuracy of far and near targets at different distances in the same field of view, and the distance dynamic range is small.
- the embodiment of the present application provides an exposure method for obtaining a depth image, as shown in FIG. 2 , which shows a single-frame depth image shown in an exemplary embodiment of the application.
- the measurement timing diagram corresponding to the frame.
- a measurement cycle is divided into three sub-measurement cycles (the measurement cycle is the measurement time required to obtain a single frame depth image frame), wherein different exposure times are set for different sub-measurement cycles, schematically, respectively Set m1 segment to correspond to exposure time T (T is the maximum exposure time), m2 segment to correspond to exposure time T/ 2 , m3 segment to correspond to exposure time T/4.
- the exposure method shown in Figure 2 by reducing the number of pulse cycles with exposure time T (from the original N to m 1 segment), the light intensity received at close range can be effectively reduced; and the m 2 and m 3 segments
- the exposure only receives the photons reflected back T/2 or T/4 after the pulse period, which can also effectively reduce the light intensity received at close range, and because the light intensity reflected at a long distance will be received in each sub-measurement period, In this way, while reducing the light intensity received at a short distance, the light intensity reflected at a long distance will not be reduced, so that the signal-to-noise ratio of long-distance measurement will not be reduced. This makes it possible to simultaneously take into account the measurement accuracy of long and short distance targets, and further improves the distance dynamic range.
- FIG. 3 shows a flow chart of a dTOF depth image acquisition method provided by an exemplary embodiment of the present application.
- the embodiment of the present application takes the application of the method to electronic equipment as an example.
- the method includes:
- Step 301 in the i-th sub-measurement period, transmit a laser signal through the transmitting end, wherein the measurement period of the depth information corresponding to the single-frame depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, and i is less than or equal to A positive integer of n.
- the measurement period indicates the time required to measure the depth information corresponding to a single depth image frame.
- each measurement period contains a fixed number of laser pulses, for example, a measurement period contains N laser pulses.
- the measurement period may be the time when 1000 laser signals (laser pulses) are emitted according to a certain emission period, and the emission period is the time difference between the emission times of two adjacent laser pulses.
- the number of laser pulses included in the measurement cycle is related to multiple factors such as emitted light power, sensor performance, reflectivity of the target to be measured, and ambient light noise intensity. situation to choose.
- the transmitted optical power is low, in order to ensure that the receiving end can receive sufficient reflected optical signals, it may be necessary to transmit more laser pulses during the measurement period; on the contrary, if the transmitted power is high, within the measurement period It is necessary to emit fewer laser pulses; if the reflectivity of the target to be measured is relatively low, in order to ensure that sufficient reflected light is received, it is also necessary to emit more laser pulses within the measurement period; on the contrary, if the reflectivity of the target to be measured is high , correspondingly it is necessary to reduce the number of laser pulses emitted in the measurement period.
- the transmitter needs to emit modulated pulsed light with concentrated light energy (for example, a laser signal), and at the same time, the receiver opens the exposure to receive photons during the emission period.
- the photon number distribution histogram corresponding to the pixel point is drawn, and the receiving time with the highest frequency in the photon number distribution histogram is determined as The light time-of-flight corresponding to the pixel point, and then based on the light time-of-flight, the depth distance information corresponding to the pixel point is restored.
- the same exposure time is used for each emission cycle in the measurement cycle.
- the exposure time in the measurement cycle is also adjusted synchronously, which makes it impossible to take into account the close-range measurement during the adjustment process. Accuracy and long-distance measurement accuracy. For example, if the exposure time is shortened, the number of photons received at a long distance will be insufficient, thereby affecting the long-distance measurement accuracy; if the exposure time is increased, the reflected light received at a short distance will be too strong. Overexposure or accumulation effects occur, which affect the accuracy of close-range measurements.
- the measurement period is firstly divided into n sub-measurement periods, so that different exposure times are used in different sub-measurement periods, so that the exposure time of the far and near targets is within one frame. That is, there is a certain gap, so that by further adjusting the exposure time corresponding to different sub-measurement periods, or the sub-measurement periods corresponding to different exposure times, the distance difference between the detection of far and near targets in the field of view can be adjusted at the same time.
- the minimum value of n in this embodiment is 2, that is, the measurement cycle is divided into two sub-measurement cycles.
- the measurement cycle contains 1000 emission cycles, the measurement The period is divided into two sub-measurement periods m 1 and m 2 , wherein the period m 1 may include 400 transmission periods, and the period m 2 may include 600 transmission periods.
- the value of n can also be an integer greater than 1, such as 3, 4, 5, etc.
- the measurement period can be divided into any n sub-measurement periods not less than 2 based on actual needs.
- the number of emission periods (pulse periods) included in different sub-measurement periods may be the same or different, which is not limited in this embodiment.
- N indicates the total number of pulse periods contained in the measurement period
- n ⁇ 2 indicates that the measurement period is divided into at least 2 sub-measurement periods; where the number of pulse periods is also the number of emission periods .
- the i-th sub-measurement period refers to any one of the n sub-measurement periods. Schematically, if n is 5, it means that the measurement period is divided into 5 sub-measurement periods, and the value of i can be 1 , 2, 3, 4, 5, when i is 1, it is the first sub-measurement period in the measurement period.
- Step 302 according to the exposure time corresponding to the i-th sub-measurement period, receive the optical signal through the receiving end, the measurement period includes sub-measurement periods corresponding to different exposure times, and the optical signal includes the reflected light signal of the laser signal.
- the exposure time indicates the time when the receiving end turns on the exposure to receive the light signal in each transmission period (or pulse period). Since the SPAD at the receiving end has a quenching time, that is, there will be a quenching time after the SPAD is successfully excited by a single photon at any time after the exposure of a pulse period is turned on, and the SPAD cannot continue to be excited by the photon during this period. Schematically, as shown in FIG. 2 , the short period between RX exposure off and TX emitting the second laser pulse is the quenching time. Therefore, it can be understood that the exposure time is shorter than the emission period.
- the corresponding relationship between each sub-measurement cycle and exposure time can be artificially preset.
- the user of the image acquisition device manually configures the corresponding relationship between the sub-measurement cycle and the exposure time in the exposure parameter configuration interface, and the image acquisition After receiving the setting operation of the exposure parameters, the device sets the corresponding relationship between the sub-measurement period and the exposure time indicated by the setting operation to the receiving end; optionally, the corresponding relationship between each sub-measurement period and the exposure time can also be determined by The image acquisition device obtains it after dynamic adjustment according to the exposure parameters of the previous frame, and sets the corresponding relationship between each sub-measurement period indicated by the adjusted exposure parameters and the exposure time to the receiving end.
- the measurement period includes sub-measurement periods corresponding to different exposure times, so that objects to be measured at different distances have different total exposure durations in the same measurement period.
- the analysis of the exposure process of far and near targets shows that the short-distance targets are generally received in the first half of the exposure time, and the closer the distance, the shorter the photon reception time. forward; and long-distance targets are generally received in the second half of the exposure time, and the farther the distance is, the later the photon reception time is.
- the exposure time is T/k (k ⁇ 2), and k can be any integer not less than 2, for example, 3, 4, 5 and so on.
- k can be fixed as a multiple of 2, because in the bottom layer of the hardware, the nth power of 2 can be easily realized through the shift operation, and the hardware overhead is relatively small.
- the exposure time is the same in the same sub-measurement period, but different in different sub-measurement periods.
- the exposure times corresponding to different sub-measurement periods do not need to be in an increasing or decreasing order, and may also be in other orders.
- the exposure time corresponding to m 1 , m 2 , m 3 can be T, T/2, T/4; the exposure time corresponding to m 1 , m 2 , m 3 can also be T/2, T, T /4; the exposure time corresponding to m 1 , m 2 , and m 3 can also be T, T/4, or T/2.
- FIG. 4 shows a measurement sequence diagram corresponding to a single depth image frame shown in another exemplary embodiment of the present application.
- the exposure time corresponding to the first sub-measurement cycle (m 1 segment) is T/2
- the exposure time corresponding to the second sub-measurement cycle (m 2 segment) is T
- the third sub-measurement cycle (m 3 segment) corresponds to The exposure time is T/4.
- the receiving end opens exposure to receive the light signal based on the exposure time corresponding to the i-th sub-measurement period.
- the optical signal received by the receiving end may include a reflected optical signal after the laser signal is reflected by the target to be measured, and may also include ambient light.
- the corresponding exposure time is T/2, and when the transmitting end emits the laser signal, the receiving end delays by T/2 After the time, open the exposure and continue to expose for T/2 time, that is, receive the reflected light signal within the last T/2 time; in the m 3 section, the corresponding exposure time is T/4, when the laser signal is emitted by the transmitter , the receiving end delays the time of 3T/4, opens the exposure to receive the reflected light, and continues to expose for T/4 time, in the m 2 section, the corresponding exposure time is T, when the transmitting end emits the laser signal, the receiving end immediately opens The exposure receives reflected light for T time.
- the exposure method shown in Figure 4 by reducing the number of pulse cycles with exposure time T (from the original N to m 2 segments), the light intensity received at close range can be effectively reduced, and m 1 and m 3 segments are exposed Only receive the photons reflected back after T/2 and T/4 after the pulse period, which can also effectively reduce the light intensity received at close range. While reducing the received light intensity at a short distance, it will not reduce the light intensity reflected at a long distance, so that the signal-to-noise ratio of long-distance measurement will not be reduced, so that the measurement accuracy of long- and short-distance targets can be taken into account at the same time.
- Step 303 when each sub-measurement period in the measurement period ends, generate a depth image frame based on the received light signal.
- the measurement period corresponding to the single depth image frame ends, that is, after each sub-measurement period is measured, the total number of times the SPAD at the receiving end is triggered under the excitation of N laser signals is counted and the trigger moment, draw the distribution histogram of the number of photons received by different pixels at different times within an exposure time T, and determine the corresponding light flight time of the pixel based on the histogram peak value of the distribution histogram, so that based on the light flight time
- the target distance corresponding to the pixel point is determined, and then the depth image information corresponding to the object to be measured can be obtained to generate a corresponding depth image frame.
- the pixel value of each pixel in the depth image frame is the distance corresponding to the pixel.
- the exposure time corresponding to different sub-measurement periods is preset to the receiving end, so that the receiving end determines when to receive the light signal based on the exposure time;
- the receiving end determines the timing of receiving the optical signal according to the exposure time corresponding to the sub-measurement cycle until the end of each sub-measurement cycle in the measurement cycle After that, a depth image frame is generated based on the received light signal.
- long-distance objects have a longer total exposure time than short-distance objects, which ensures the measurement of long-distance targets while preventing overexposure and accumulation effects.
- Accuracy so that the measurement accuracy of farther and closer objects in the same field of view can be taken into account, and the distance difference between distant and short-distance targets in the simultaneous detection of the field of view can be further increased, thereby improving the distance dynamic range of the depth image acquisition device.
- the optical signal is received through the receiving end, including:
- the light signal is received by the receiving end.
- the optical signal is received through the receiving end, including:
- the optical signal is received by the receiving end at the time of transmission, and the maximum exposure time is determined by the transmission period of the laser signal;
- the optical signal is received by the receiving end.
- the measurement period includes N emission periods, where N is an integer greater than 1, and the emission period is the emission time difference between adjacent laser signals;
- different sub-measurement periods include the same number of transmission cycles, or different sub-measurement periods include different numbers of transmission cycles.
- the exposure time corresponding to at least one sub-measurement period is the maximum exposure time, and the maximum exposure time is determined by the emission period of the laser signal.
- the method also includes:
- the j-1th exposure result data used to generate the j-1th frame depth image frame, the j-1th exposure result data includes the total number of photons received by each pixel point in the measurement period, where j is greater than 1 integer;
- the j-1th exposure parameter In the case that the j-1th exposure result data meets the preset conditions, adjust the j-1th exposure parameter according to the preset adjustment method to generate the jth exposure parameter, and the j-1th exposure parameter is to collect the j-1th depth
- the exposure parameters used in the image frame, the j-1th exposure parameters include: k sub-measurement periods into which the measurement period is divided and the exposure time corresponding to each sub-measurement period, the preset adjustment method is used to change the division method of the measurement period, k is an integer greater than 1;
- An exposure parameter update instruction is sent to the receiving end based on the jth exposure parameter, and the receiving end is used to receive the light signal based on the jth exposure parameter.
- the j-1th exposure parameter is adjusted according to the preset adjustment method, including:
- the j-1th exposure parameter is adjusted according to the first preset adjustment method, and the first preset adjustment method is used to reduce the number corresponding to the first exposure time.
- the number of emission cycles included in the measurement cycle, and increasing the second exposure time corresponding to the number of emission cycles included in the sub-measurement cycle, the second exposure time is less than the first exposure time;
- the j-1th exposure parameter is adjusted according to the second preset adjustment method, and the second preset adjustment method is used to increase the number corresponding to the first exposure time The number of emission cycles included in the measurement cycle, and the second exposure time is reduced corresponding to the number of emission cycles included in the sub-measurement cycle, the second exposure time being shorter than the first exposure time.
- the first preset condition includes: the total number of photons received by a single pixel in the measurement cycle is higher than the first preset photon number threshold, and the number of pixels with the total photon number higher than the first preset photon number threshold At least one of the proportion of pixels greater than the first preset number threshold and the total photon number higher than the first preset photon number threshold is higher than the first preset ratio threshold;
- the second preset condition includes: the total number of photons received by a single pixel in the measurement period is lower than the second preset photon number threshold, and the number of pixels with the total photon number lower than the second preset photon number threshold is greater than the second preset Set the number threshold, and the proportion of pixels whose total photon number is lower than the second preset photon number threshold is higher than at least one of the second preset ratio thresholds, wherein the second preset photon number threshold is less than or equal to the first Preset photon count threshold.
- adjust the j-1th exposure parameter according to the first preset adjustment method including:
- the first increase amount is positively correlated with the first amount difference
- the first decrease amount is positively correlated with the first amount difference
- the j-1th exposure parameter is adjusted.
- the j-1th exposure parameter is adjusted according to the second preset adjustment method, including:
- the second increase amount is positively correlated with the second amount difference
- the second decrease amount is positively correlated with the second amount difference
- the j-1th exposure parameter is adjusted.
- the light flight time is relatively short, and the reflected photons will be received in the first half of the exposure time of the receiving end, and the closer the distance, the earlier the time of receiving the photons, and the stronger the reflected light. It is easy to cause overexposure or accumulation effect.
- photons can be received in the second half of the exposure time, or the last T/4 period, so as to effectively reduce the light intensity received at close range, and at the same time avoid reducing the reception of long-distance Reflected light intensity.
- FIG. 5 shows a flow chart of a dTOF depth image acquisition method provided by another exemplary embodiment of the present application.
- the embodiment of the present application takes the application of the method to electronic equipment as an example.
- the method includes:
- Step 501 in the i-th sub-measurement period, transmit a laser signal through the transmitting end, wherein the measurement period of the depth information corresponding to the single-frame depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, and i is less than or equal to A positive integer of n.
- the measurement period includes N emission periods, N is an integer greater than 1, and the emission period is the emission time difference between adjacent laser signals. Schematically, as shown in FIG. 4 , the emission period is T d .
- different sub-measurement periods contain the same number of transmission periods.
- the measurement period contains 1200 transmission periods, which are divided into 3 sub-measurement periods, and each sub-measurement period may contain 400 transmission periods; or, Different sub-measurement periods may also include different numbers of emission cycles.
- the first sub-measurement period may include 200 emission cycles
- the second sub-measurement period may include 600 emission cycles
- the third sub-measurement period may include 400 launch cycles.
- Step 502 based on the exposure time corresponding to the i-th sub-measurement period, determine the exposure delay time corresponding to the i-th sub-measurement period.
- the receiving end delays for a period of time before continuing to receive the optical signal, wherein the exposure delay time of the receiving end can be determined by the exposure time, that is, the exposure time corresponding to the i-th sub-measurement cycle is used to determine the i-th sub-measurement cycle. Exposure delay time.
- the exposure delay time is determined by the maximum exposure time and the exposure time corresponding to the i-th sub-measurement period, that is, the sum of the exposure delay time and the exposure time is the maximum exposure time.
- the maximum exposure time when the maximum exposure time is applied, after the transmitting end transmits the laser signal, it immediately receives the reflected light through the receiving end (that is, there is no exposure delay time, or the exposure delay time is 0), and the maximum exposure delay time and the quenching time The sum is equal to the emission period (the emission period is the emission time difference between two adjacent laser signals).
- the maximum exposure time is T.
- the maximum exposure time is T. If the i-th sub-measurement period The exposure time corresponding to the cycle is T/2, then the exposure delay time corresponding to the i-th sub-measurement cycle is T/2, that is, after the transmitting end emits the laser pulse, it is delayed by T/2 time and received by the receiving end for the T/2 time period Optical signal; if the exposure time corresponding to the i-th sub-measurement cycle is T/4, then the exposure delay time corresponding to the i-th sub-measurement cycle is 3T/4, that is, after the laser signal is emitted at the transmitter, it is delayed by 3T/4 time to pass through the receiver The end receives the optical signal in the T/4 time period.
- Step 503 when the time difference between the time of transmitting the laser signal and the current time of receiving the laser signal reaches the exposure delay time, receive the light signal through the receiving end.
- the operating clock of the transmitting end is synchronized with the operating clock of the receiving end.
- the transmitting end emits a laser signal
- the receiving end starts with the emission time of the laser signal and enters the receiving cycle (the receiving cycle corresponds to the emitting cycle), that is, the receiving cycle
- the initial moment of is the transmission time, and when determining whether the exposure delay time is reached, it is only necessary to determine whether the time difference between the current receiving time and the transmission time of the receiving end reaches the exposure delay time.
- the receiving end is turned on to receive the optical signal after the time difference between the emitting moment of the laser signal and the current receiving moment reaches the exposure delay time.
- the exposure delay time is 3T/4, correspondingly, after the transmitting end emits the laser signal, when the emission time reaches 3T/4, the reflected light signal of T/4 time period is received by the receiving end.
- the emission duration is the time difference between the emission moment of the laser signal and the current reception moment.
- T a maximum exposure time in at least one sub-measurement period. That is, the exposure time corresponding to at least one sub-measurement period is the maximum exposure time, and the maximum exposure time is determined by the emission period of the laser signal.
- the maximum exposure time is shorter than the emission period of the laser signal. It can be understood that the sum of the maximum exposure time and the quenching time is the emission period.
- the receiving end should be turned on to receive the reflected light signal while emitting the laser signal; if the exposure time is less than the maximum exposure time, in order to ensure It can receive the reflected light of distant objects, and the receiving end should be turned on to receive the reflected light signal after the exposure delay time is reached.
- the process of receiving the reflected optical signal through the receiving end may include the following steps:
- the optical signal is received by the receiving end at the time of transmission, and the maximum exposure time is determined by the transmission period of the laser signal.
- the exposure delay time is determined by the maximum exposure time and the exposure time, that is, the exposure delay time is 0, that is, the laser light is emitted at the transmitting end At the same time as receiving the signal, turn on the receiving end immediately, and receive the reflected light signal through the receiving end.
- the optical signal is received by the receiving end.
- the transmitting end when the exposure time corresponding to the sub-measurement period is less than the maximum exposure time, in order to reduce the light intensity received at a short distance while ensuring the light intensity received at a long distance, after the transmitting end emits a laser signal, It is judged in real time whether the time difference between the transmitting moment and the current receiving moment reaches the exposure delay time, and after the exposure delay time is reached, the receiving end is turned on, and the optical signal is received through the receiving end within the exposure time.
- the first sub-measurement period (m 1 ) corresponds to the exposure time T/2
- the second sub-measurement period (m 2 ) corresponds to the maximum exposure time T
- the third sub-measurement period (m 3 ) corresponds to the exposure time T/4.
- the light flight time is short, and the reflected photons will be received in the first half period of the RX maximum exposure time T, and the closer the distance, the earlier the time of the received photons, and the reflected light When it is strong, it is easy to cause overexposure or accumulation effect.
- the number of emission cycles included in the sub-measurement period with exposure time T can be reduced, thereby effectively reducing the light intensity received at close range. ; while the m 1 segment exposure will receive the photons reflected back by T/2, and the m 3 segment exposure will receive the photons reflected back by T/4, so while reducing the short-range reflected light, it will not reduce the long-distance reflected light Strong, making it possible to take into account the measurement accuracy of long and short distances in the same field of view at the same time.
- Step 504 when each sub-measurement period in the measurement period ends, generate a depth image frame based on the received light signal.
- step 504 For the implementation manner of step 504, reference may be made to step 303, and details are not described in this embodiment here.
- the exposure delay time is determined, so that the receiving end determines when to receive the optical signal based on the exposure delay time; in addition, by setting the exposure time corresponding to the sub-measurement period as the maximum exposure Time, so that the receiving end can start receiving the optical signal immediately after the transmitting end emits the laser signal, so that it can receive the reflected light of the close-range object in the field of view; by setting the exposure time corresponding to the sub-measurement period less than the maximum exposure time, and making the receiving end
- the terminal receives the optical signal at the later stage of the maximum exposure time, which can effectively reduce the reflected light received at a short distance while not reducing the reflected light received at a long distance, so that the measurement accuracy of the long and short distance can be taken into account, and the dynamic measurement can be improved. scope.
- electronic device A requires a collection range of 50cm to 1m
- electronic device B requires a collection range of 50cm to 10m.
- the periods are divided, and the exposure time corresponding to different sub-measurement periods is set.
- the following embodiments mainly describe the process of how to adjust the exposure parameters.
- FIG. 7 shows a flow chart of a dTOF depth image acquisition method provided by another exemplary embodiment of the present application.
- the embodiment of the present application takes the application of this method to electronic equipment as an example.
- the method includes:
- Step 701 acquire the j-1th exposure result data used to generate the j-1th frame depth image frame, the j-1th exposure result data includes the total number of photons received by each pixel point in the measurement period, where j is An integer greater than 1.
- the object to be measured at a short distance may have a high reflectivity or a low reflectivity, if the near-distance objects are all in a scene with a low reflectivity, reduce the maximum exposure time T corresponding to the sub-measurement cycle.
- the number of emission cycles may reduce the intensity of the light signal reflected by the low-reflectivity target, thus affecting the distance measurement accuracy of the low-reflectivity target.
- the electronic device acquires the j-1th exposure result data of the generated previous frame (j-1th frame) depth image frame, and the exposure result data is the Measurement data collected during one frame of depth image.
- the exposure result data is the total number of photons received by each pixel point within a measurement period obtained through statistics.
- Step 702 if the j-1th exposure result data satisfies the preset condition, adjust the j-1th exposure parameter according to a preset adjustment method to generate the jth exposure parameter.
- the j-1th exposure parameter is the exposure parameter used when collecting the j-1th depth image frame
- the j-1th exposure parameter includes: k sub-measurement periods into which the measurement period is divided and the exposure time corresponding to each sub-measurement period , k is an integer greater than 1.
- the preset adjustment manner may be to change the division manner of the measurement period.
- the number of divided sub-measurement periods in the measurement period can be increased; or the exposure time corresponding to different sub-measurement periods can be changed; or the number of emission periods included in the sub-measurement periods corresponding to different exposure times can be changed.
- the received light intensity is relatively strong, which may be due to the fact that the long exposure time corresponds to a large number of emission cycles in the sub-measurement cycle, which makes the short-distance reception
- the received light intensity is weaker, which may be due to the fact that the longer exposure time corresponds to a smaller number of emission cycles included in the sub-measurement cycle, so that the close-up
- the light intensity received at a distance is relatively weak, and it is necessary to increase the light intensity received at a short distance;
- Set the condition (corresponding to the scene with strong light intensity) and the second preset condition (corresponding to the scene with weak light intensity), and the preset adjustment mode also includes the first preset adjustment mode (for when the first preset condition is met) use) and the second preset adjustment method (for use when the second preset condition is met), when the j-1th exposure result data satisfies the first preset condition, adopt the corresponding first preset adjustment method, when When the j-1th exposure result
- the process of adjusting exposure parameters may include the following steps:
- the second exposure time is shorter than the first exposure time.
- the first preset condition includes: the total number of photons received by a single pixel in the measurement cycle is higher than the first preset photon number threshold, and the number of pixels with the total photon number higher than the first preset photon number threshold is greater than the first preset photon number threshold. At least one of a preset number threshold and a proportion of pixels whose total photon count is higher than the first preset photon count threshold is higher than the first preset ratio threshold. It should be noted that the total number of photons in the embodiments of the present application refers to the total number of photons received by a single pixel point in one measurement cycle.
- the first preset photon number threshold may be a fixed value, or a dynamically adjusted value.
- the first preset photon threshold can be set according to the optical power of the transmitting end, the photosensitive capability of the receiving end sensor, and the application environment (the nearest and farthest measurement ranges, the intensity of environmental noise, the required distance dynamic range, etc.).
- the first preset photon number threshold can be set based on the photon number required for measuring depth information. For example, the first preset photon number threshold may be 1000.
- the exposure for generating the previous depth image frame is acquired
- the result data that is, the total number of photons received by each pixel in the measurement period, and the total number of photons received by each pixel is compared with the first preset photon number threshold, if there is a single pixel corresponding to
- the exposure parameters of the previous frame may be adjusted according to the first preset adjustment manner.
- the number of pixels is on the order of hundreds to tens of thousands to hundreds of thousands. Since the hardware cannot realize the individual control of the exposure time for each pixel, the exposure time setting is unified for all pixels, so it cannot When the pixel receives too many total photons, it adjusts the exposure mode.
- the first preset condition is set so that the total photon number is higher than the first preset photon number.
- the number of pixels with the threshold value is greater than the first preset number threshold, or the proportion of pixels whose total photon number is higher than the first preset photon number threshold is higher than the first preset ratio threshold.
- the first preset condition is that the number of pixels whose total photon number is higher than the first preset photon number threshold is greater than the first preset number threshold
- comparing the total number of pixels received by each pixel The relationship between the number of photons and the first preset photon number threshold, and the number of pixels with a larger total photon number (the total photon number is greater than the first preset photon number threshold), and then compare the number of pixels with the first preset photon number threshold.
- the first preset number threshold may be set based on an actual application scenario, for example, the first preset number threshold may be 500.
- the first preset condition is that the proportion of pixels whose total photon number is higher than the first preset photon number threshold is higher than the first preset ratio threshold
- comparing each pixel received The relationship between the total number of photons received and the first preset photon number threshold, and count the number of pixels with more total photon numbers (the total photon number is greater than the first preset photon number threshold), and then determine the proportion of the pixel number The ratio of the number of all pixels, and then compare the relationship between the ratio and the first preset ratio threshold. If the ratio is higher than the first preset ratio threshold, it means that the total number of photons received is too much, and the first preset ratio threshold needs to be followed.
- the adjustment mode is set to adjust the exposure parameter of the j-1th frame; otherwise, it is not necessary to adjust the exposure parameter of the j-1th frame according to the first preset adjustment mode.
- the first preset ratio threshold may also be set based on an actual application scenario, for example, the first preset ratio threshold may be 1/4.
- the total photon number exceeding the first preset photon number threshold is generally a short distance, that is, when it is determined that the j-1th exposure result data satisfies
- the first preset condition it means that the reflected light intensity received at close range is more, therefore, in order to reduce the number of photons received at close range, in a possible implementation manner, it can be achieved by reducing the longer exposure time (first Exposure time) corresponds to the number of emission cycles included in the sub-measurement cycle, so as to reduce the light intensity received at close range; and when the total number of emission cycles in the measurement cycle is constant, it is also necessary to increase the shorter exposure time (The second exposure time) corresponds to the number of emission periods included in the sub-measurement period, so as to ensure that the light intensity of long-distance reflection will not be reduced, thereby reducing the accuracy of long-distance measurement.
- the j-1th exposure parameter indicates: the first sub-measurement period (m 1 ) corresponds to the exposure time T/2, and m 1 contains 400 emission cycle, the second sub-measurement period (m 2 ) corresponds to the exposure time T, m 2 contains 500 emission cycles, the third sub-measurement cycle (m 3 ) corresponds to the exposure time T/4, m 3 contains 300 emission cycles,
- the j-1th exposure result data satisfies the first preset condition, it means that the reflected light intensity received at a short distance is relatively large, and the exposure time T can be appropriately reduced to correspond to the number of emission cycles included in the second sub-measurement cycle.
- the adjusted jth exposure parameter is: m 1 contains 600 emission periods, and m 2 contains 300 emission periods , m 3 contains 300 emission cycles.
- the short-distance reflected light intensity can also be reduced by increasing the sub-measurement period into which the measurement period is divided.
- the j-1th exposure Parameter indication the first sub-measurement cycle (m 1 ) corresponds to the exposure time T/2, m 1 contains 400 emission cycles, the second sub-measurement cycle (m 2 ) corresponds to the maximum exposure time T, and m 2 contains 800 emission cycles
- the jth exposure parameter after adjustment indicates that the measurement cycle is divided are m 1 , m 2 and m 3 , wherein m 1 (corresponding to exposure time T/2) includes 400 emission periods, m 2 (corresponding to exposure time T) includes 500 emission periods, and m 3 (corresponding to exposure time is T/4) consists of 300 transmission cycles
- the number of photons included in the sub-measurement period can be determined based on the relationship between the total photon number and the first preset photon number threshold.
- the rate of increase in the number of transmit cycles, or the decrease in the number of transmit cycles contained in a sub-measurement period can be determined based on the relationship between the total photon number and the first preset photon number threshold.
- the process of adjusting the j-1th exposure parameter according to the first preset adjustment method may further include the following steps:
- the exposure result data satisfies the first preset condition, it can be based on the total The first quantity difference between the photon number and the first preset photon number threshold determines the received light intensity at close range, so as to determine the adjustment range of each exposure time corresponding to the number of emission cycles based on the first quantity difference.
- the difference of the first number is larger, it means that the light intensity received at close range is stronger, and the number of emission cycles corresponding to a longer exposure time needs to be reduced, and correspondingly, the number of emission cycles corresponding to a shorter exposure time needs to be increased.
- the first quantity difference is small, it means that the light intensity received at close range is relatively weak.
- the electronic device may be preset with a corresponding relationship between the first quantity difference, the exposure time, the first increasing amount, and the first decreasing amount, and the corresponding relationship may be obtained by a developer after actual testing; so that in In the process of actually adjusting the exposure parameters, after the first quantitative difference is determined, the first increase and the first decrease corresponding to each exposure time can be determined according to the corresponding relationship.
- the exposure parameters are: m 1 (corresponding to exposure time T/2) includes 400 emission cycles, m 2 (corresponding to exposure time T) includes 800 emission cycles,
- the exposure parameters are used to collect the depth image frames corresponding to scene 1 and scene 2 respectively, and the collected exposure result data corresponding to scene 1 and scene 2 meet the first preset condition (that is, the number of photons received at close range is too much), among which, the scene
- the adjusted exposure parameters of scene 1 can be: m 1 (corresponding to exposure time T/2) includes 800 emission cycles, m 2 (corresponding to exposure time T) includes 400 emission cycles cycle, the shorter exposure time corresponds to an increase of 400 emission cycles, and the longer exposure time corresponds to a decrease of 400 emission cycles;
- the adjusted exposure parameters of scene two can be: m 1 (corresponding to exposure time T/ 2) Including 600 emission cycles, m 2 (corresponding to the exposure time T) includes 600 emission cycles, the increase in the number of emission cycles corresponding to a shorter exposure time is 200, and the decrease in the number of emission cycles corresponding to a longer exposure time is 200.
- the method of determining the first quantity difference it may first be determined that the total number of photons (the total number of photons refers to the number of photons received by a single pixel point in the measurement period) exceeds the first preset photon number threshold.
- the number difference between the first preset photon number threshold is determined as the first number difference; or the average value of the total photon number is not required, and the sum of the total photon number corresponding to a specific pixel point is directly compared with the first preset photon number
- the quantitative difference between the thresholds is determined as a first quantitative difference.
- the target specific pixel point can also be selected from the specific pixel point based on the preset area. is a specific pixel located in the preset area), and then based on the total photon number corresponding to the target specific pixel point and the first preset photon number threshold, the first quantity difference is determined.
- the manner of determining the increase range or decrease range of the number of emission cycles in the sub-measurement period may also be based on the pixels whose total photon number is higher than the first preset photon number threshold
- the relationship between the number of dots and the first preset number threshold determines the increase in the number of emission cycles or the reduction in the number of emission cycles in the sub-measurement period; that is, when the exposure result data satisfies the first preset condition, it can be based on the number of pixels ( The difference between the number of pixels whose total photon number is higher than the first preset photon number threshold) and the first preset number threshold determines the light intensity received at close range, and then determines the emission period corresponding to each exposure time
- the adjustment range of the number if the number difference between the number of pixels and the first preset number threshold is larger, the longer exposure time corresponds to a greater reduction in the number of emission cycles, and the shorter exposure time corresponds to an increase in the number of emission cycles The larger the amount; on
- the increase in the number of emission cycles or the emission cycle in the sub-measurement cycle can also be determined That is, when the exposure result data meets the first preset condition, it can be based on the proportion of pixels (the proportion of pixels whose total photon number is higher than the first preset photon number threshold) and the first preset Set the ratio difference between the ratio thresholds to determine the light intensity received at close range, and then determine the adjustment range of each exposure time corresponding to the number of emission cycles; if the ratio difference is larger, the longer exposure time corresponds to the number of emission cycles The greater the reduction, the greater the increase in the number of emission cycles corresponding to the shorter exposure time; on the contrary, if the proportional difference is smaller, the reduction in the number of emission cycles corresponding to the longer exposure time is smaller, and the shorter exposure time corresponds to the emission cycle. The less the number of cycles increases.
- the j-1th exposure parameter is adjusted, so as to generate the jth exposure parameter corresponding to the jth depth image frame.
- the j-1th exposure parameter is: m 1 (corresponding to exposure time T/2) includes 400 emission cycles, m 2 (corresponding to exposure time T) includes 800 emission cycles, the determined first increase The amount is 300, and the first reduction amount is 300.
- the jth exposure parameter can be: m 1 (corresponding to exposure time T/2) includes 700 emission periods, m 2 (corresponding to exposure time T) includes 500 emission cycle.
- the second exposure time is shorter than the first exposure time.
- the second preset condition includes: the total number of photons received by a single pixel in the measurement period is lower than the second preset photon number threshold, and the number of pixels whose total photon number is lower than the second preset photon number threshold is greater than the first Two preset number thresholds, and the proportion of pixels whose total photon number is lower than the second preset photon number threshold is higher than at least one of the second preset ratio thresholds, wherein the second preset photon number threshold is less than or equal to The first preset photon number threshold.
- the second preset photon number threshold may be a fixed value, or a dynamically adjusted value.
- the second preset photon threshold can be set according to the optical power of the transmitting end, the photosensitive capability of the receiving end sensor, and the application environment (the nearest and farthest measurement ranges, the intensity of environmental noise, the required distance dynamic range, etc.).
- the second preset photon number threshold can be set based on the photon number required for measuring depth information.
- the second preset photon number threshold may be the same as the first preset photon number threshold, or may be smaller than the first preset photon number threshold. Schematically, if the first preset photon number threshold is 1000, then The second preset photon number threshold may be 800. This embodiment does not constitute a limitation to this.
- the exposure for generating the previous depth image frame is acquired
- the result data that is, the total number of photons corresponding to each pixel, and compare the relationship between the total number of photons received by each pixel and the second preset photon number threshold, if the total photon number corresponding to a single pixel is lower than
- the second preset photon number threshold means that the light intensity received at close range is weak, and the depth information corresponding to the point cannot be accurately measured based on the photon number, and the exposure of the previous frame can be adjusted according to the second preset adjustment method parameters to adjust.
- the number of pixels is on the order of hundreds to tens of thousands to hundreds of thousands. Since the hardware cannot realize the individual control of the exposure time for each pixel, the exposure time setting is unified for all pixels, so it cannot When the pixel receives too many total photons, it adjusts the exposure mode.
- the second preset condition is set so that the total photon number is lower than the second preset photon number.
- the number of pixels with the threshold value is greater than the second preset number threshold, or the proportion of pixels whose total number of photons is lower than the second preset photon number threshold is higher than the second preset ratio threshold.
- the second preset condition is that the number of pixels whose total photon number is lower than the second preset photon number threshold is greater than the second preset number threshold
- the second preset number threshold can be set based on an actual application scenario, and the second preset number threshold can be the same as the first preset number threshold, or can be different from the first preset number threshold.
- the second preset quantity threshold may be 800.
- the second preset condition is that the proportion of pixels whose total photon number is lower than the second preset photon number threshold is higher than the second preset ratio threshold
- comparing each pixel received The relationship between the total number of photons received and the second preset photon number threshold, and count the number of pixels with less total photon number (the total photon number is less than the second preset photon number threshold), and then determine the proportion of the pixel number
- the ratio of the number of all pixels and then compare the relationship between the ratio and the second preset ratio threshold, if the ratio is higher than the second preset ratio threshold, it is necessary to expose the j-1th frame according to the second preset adjustment method parameters; otherwise, it is not necessary to adjust the exposure parameters of the j-1th frame according to the second preset adjustment method.
- the second preset ratio threshold can also be set based on the actual application scenario.
- the second preset ratio threshold can be 1/4.
- the second preset ratio threshold can be equal to the first preset ratio
- the thresholds are the same or may be different, and the embodiment of the present application does not limit the relationship between the first preset ratio threshold and the second preset ratio threshold.
- the longer exposure time (first exposure time) can be increased for the number of emission periods included in the sub-measurement period, and the shorter exposure time (second exposure time) can be decreased for the number of emission periods included in the sub-measurement period number.
- the j-1th exposure parameter indicates: the first sub-measurement period (m 1 ) corresponds to the exposure time T/2, and m 1 contains 600 emission cycle
- the second sub-measurement period (m 2 ) corresponds to the exposure time T
- m 2 contains 400 emission cycles
- the third sub-measurement cycle (m 3 ) corresponds to the exposure time T/4
- m 3 contains 200 emission cycles
- the exposure time T can be appropriately increased to correspond to the number of emission cycles included in the second sub-measurement time
- the exposure time can be appropriately reduced to T/2 or T/4 corresponding to the sub-
- the number of emission cycles included in the measurement cycle for example, the adjusted j-th exposure parameter is: m 1 includes 400 emission cycles, m 2 includes 600 emission cycles, and m 3 includes 200 emission cycles.
- the short-distance reflected light intensity can also be increased by reducing the sub-measurement period into which the measurement period is divided.
- the total photon number and the second preset photon number threshold can be used to determine the number of photons included in the sub-measurement period The magnitude by which the number of emission cycles is increased, or the number of emission cycles contained in a sub-measurement period is decreased.
- the process of adjusting the j-1th exposure parameter according to the second preset adjustment method may further include the following steps:
- the light intensity received at close range is small.
- Amount to adjust the number of transmit cycles If the difference of the second number is large, it means that the received light intensity at close range is weak, and the number of emission cycles corresponding to a longer exposure time needs to be increased, and the number of emission cycles corresponding to a shorter exposure time should be reduced; on the contrary, If the difference of the second number is small, in order to avoid overexposure or accumulation effect caused by increasing the number of emission cycles corresponding to a longer exposure time, it is necessary to increase the number of emission cycles corresponding to a longer exposure time and reduce the number of emission cycles corresponding to a shorter exposure time.
- the time corresponds to a smaller number of emission cycles. That is, a longer exposure time corresponds to an increase in the number of emission cycles and a positive correlation with the second quantity difference, and a shorter exposure time corresponds to a decrease in the number of emission cycles and a positive correlation with the second quantity difference.
- the electronic device may also be preset with a corresponding relationship between the second quantity difference, the exposure time, the second increase and the second decrease, and the corresponding relationship may also be obtained by the developer after actual testing. ; so that in the actual adjustment process of the exposure parameters, after the second quantity difference is determined, the second increase amount and the second decrease amount corresponding to each exposure time can be determined according to the corresponding relationship.
- the exposure parameters are: m 1 (corresponding to exposure time T/2) includes 900 emission cycles, m 2 (corresponding to exposure time T) includes 300 emission cycles,
- the exposure parameters are used to collect the depth image frames corresponding to scene 1 and scene 2 respectively, and the collected exposure result data corresponding to scene 1 and scene 2 meet the second preset condition (that is, the number of photons received at close range is small), where , the exposure result data indication corresponding to scene one: the quantity difference between the total photon number and the second preset photon number threshold is A 1 ; the exposure result data indication corresponding to scene two: the total photon number and the second preset photon number threshold The quantitative difference between the thresholds is A 2 , and A 1 is greater than A 2 .
- the adjusted exposure parameters of scene 1 can be: m 1 (corresponding to exposure time T/2) includes 500 emission cycles, m 2 (corresponding to exposure time T) includes 700 emission cycles cycle, the longer exposure time corresponds to an increase of 400 emission cycles, and the shorter exposure time corresponds to a decrease of 400 emission cycles;
- the adjusted exposure parameter of scene 2 can be: m 1 (corresponding to exposure time T/2) Including 700 emission cycles, m 2 (corresponding to exposure time T) includes 500 emission cycles, a longer exposure time corresponds to an increase of 200 emission cycles, and a shorter exposure time corresponds to a decrease of 200 emission cycles.
- the method of determining the increase or decrease of the number of emission cycles in the sub-measurement period may also be based on the pixels whose total photon number is lower than the second preset photon number threshold
- the relationship between the number of points and the second preset number threshold determines the increase in the number of emission cycles or the decrease in the number of emission cycles in the sub-measurement period; that is, when the exposure result data meets the second preset condition, it can be based on the number of pixels ( The difference between the number of pixels whose total photon number is lower than the second preset photon number threshold) and the second preset number threshold determines the light intensity received at close range, and then determines the emission period corresponding to each exposure time The adjustment range of the number; if the number difference between the number of pixels and the second preset number threshold is larger, it is necessary to increase the number of emission cycles corresponding to the longer exposure time, and correspondingly reduce the number of emission cycles corresponding to the shorter exposure time Conversely, if the difference between the number of pixels
- the increase in the number of emission cycles or the emission cycle in the sub-measurement period can be determined That is, when the exposure result data satisfies the second preset condition, it can be based on the proportion of pixels (the proportion of pixels whose total photon number is lower than the second preset photon number threshold) and the second preset Set the ratio difference between the ratio thresholds to determine the light intensity received at close range, and then determine the adjustment range of each exposure time corresponding to the number of emission cycles; if the ratio difference is larger, it is necessary to increase the emission cycle corresponding to the longer exposure time The larger the number, the more the number of emission cycles corresponding to the shorter exposure time will be reduced; on the contrary, if the proportional difference is smaller, the number of emission cycles corresponding to the longer exposure time will be increased, and the number of emission cycles corresponding to the shorter exposure time will be reduced. The number of emission cycles is low.
- the j-1th exposure parameter is adjusted, so as to generate the jth exposure parameter corresponding to the jth depth image frame.
- the j-1th exposure parameter is: m 1 (corresponding to exposure time T/2) includes 800 emission periods, m 2 (corresponding to exposure time T) includes 400 emission periods, the determined second increase The amount is 300, and the second reduction amount is 300.
- the jth exposure parameter can be: m 1 (corresponding to exposure time T/2) includes 500 emission periods, m 2 (corresponding to exposure time T) includes 700 emission cycle.
- Step 703 Send an exposure parameter update instruction to the receiving end based on the jth exposure parameter, and the receiving end is used to receive the light signal based on the jth exposure parameter.
- an exposure parameter adjustment instruction needs to be sent to the receiving end so that the receiving end can correspond to
- the specific adjustment method can refer to the adjustment process of the j-1th frame exposure parameter in the above embodiment, and this embodiment will not repeat it here.
- Step 704 in the i-th sub-measurement period, transmit a laser signal through the transmitting end, wherein the measurement period of the depth information corresponding to the single-frame depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, and i is less than or equal to A positive integer of n.
- Step 705 According to the exposure time corresponding to the i-th sub-measurement period, the optical signal is received through the receiving end, the measurement period includes sub-measurement periods corresponding to different exposure times, and the optical signal includes the reflected light signal of the laser signal.
- Step 706 when each sub-measurement period in the measurement period ends, generate a depth image frame based on the received light signal.
- step 704 and step 706 reference may be made to the foregoing embodiments, and details are not described in this embodiment here.
- steps 701 to 703 are executed before step 704, and the j-th exposure parameter in the order of execution of the steps satisfies the sub-measurement periods corresponding to different exposure times in a single measurement cycle, and the j-1th exposure parameter may exist Two cases: (1) The j-1th exposure parameter satisfies that a single measurement period contains sub-measurement periods corresponding to different exposure times; (2) The j-1th exposure parameter indicates that the same exposure time is used in a single measurement period, that is, the first The j-1 exposure parameter does not satisfy sub-measurement periods corresponding to different exposure times within a single measurement period.
- steps 701 to 703 may also be performed after step 706.
- the j-1th exposure parameter satisfies the requirement that a single measurement period contains sub-measurement periods corresponding to different exposure times
- the jth exposure parameter may There are two situations: (1) the jth exposure parameter also satisfies the requirement that a single measurement period contains sub-measurement periods corresponding to different exposure times; (2) the jth exposure parameter indicates that the same exposure time is used in a single measurement period, that is, the jth exposure The parameters do not satisfy sub-measurement periods corresponding to different exposure times within a single measurement period.
- the embodiment of the present application will not increase or decrease the number of emission cycles of the laser signal emitted by the transmitting end, that is to say, in the process of adjusting the exposure parameters, the corresponding measurement
- the total number of emission periods included in the period is a fixed value; and the embodiment of the present application itself divides the measurement period into at least two sub-measurement periods, and each sub-measurement period corresponds to a different exposure time.
- the main Based on the exposure result data collected in the previous frame, adjust the number of emission periods included in different sub-measurement periods.
- the initial exposure parameters are: m 1 (corresponding to exposure time T/2) includes 400 emission periods, m 2 (corresponding to exposure time T) includes 800 emission cycles, and the total number of emission cycles of the laser signal emitted by the transmitter is 1200; the adjusted exposure parameters are: m 1 (corresponding to exposure time T/2) includes 700 emission cycles , m 2 (corresponding to the exposure time T) includes 500 emission cycles, it can be seen that only the number of emission cycles contained in the sub-measurement cycles corresponding to different exposure times is adjusted, and the total number of emission cycles emitted by the transmitter is still 1200.
- FIG. 8 shows a flow chart of adjusting the exposure mode shown in an exemplary embodiment of the present application.
- Step 801 initially setting the exposure time in the measurement period as T.
- Step 802 setting the number of pulse periods contained in each sub-measurement period to the receiving end.
- the receiving end is configured to determine the timing of receiving the optical signal based on the exposure time corresponding to each sub-measurement period.
- the exposure time during the initial operation is T, that is, the measurement period includes a single sub-measurement period, and the number of pulse periods corresponding to the sub-measurement period is N.
- Step 803 in the i-th sub-measurement period, transmit the laser signal through the transmitting end, and receive the light signal according to the exposure time corresponding to the i-th sub-measurement period.
- Step 804 in response to the end of each sub-measurement period in the measurement period, generate current scene data based on the received optical signal.
- the current scene data is a photon number distribution histogram corresponding to each pixel point under the current exposure parameters, that is, the exposure result data in the above embodiment. Not only can the depth image frame collected under the current exposure parameters be generated based on the current scene data, but also it can be judged based on the current scene data whether the exposure mode needs to be adjusted.
- Step 805 generate a depth image frame based on the current scene data.
- Step 806 based on the current scene data, determine whether the total number of photons received at close range is too much.
- step 807 Based on the collected current scene data, it is judged whether the number of photons received at close range is too many, and if too many, in order to avoid overexposure or accumulation effect, enter step 807 and adjust the exposure mode, otherwise no need to adjust the exposure mode.
- Step 807 Decrease the number of pulse cycles with exposure time T at the receiving end, increase the number of pulse cycles with exposure time T/2 later, and/or increase the number of pulse cycles with exposure time T/4 later.
- step 802 can be entered to reset the number of pulse periods contained in each sub-measurement period to the receiving end based on the adjusted exposure parameters, so that scene data can be re-acquired based on the adjusted exposure parameters.
- FIG. 9 shows a structural block diagram of a device for acquiring a dTOF depth image provided by an embodiment of the present application.
- the apparatus has the functions executed by the electronic device in the foregoing method embodiments, and the functions may be implemented by hardware, or may be implemented by hardware executing corresponding software.
- the device may include:
- the transmitting module 901 is configured to transmit a laser signal through the transmitting end in the i-th sub-measurement period, wherein the measurement period of the depth information corresponding to a single depth image frame is divided into n sub-measurement periods, n is an integer greater than 1, i is a positive integer less than or equal to n;
- the receiving module 902 is configured to receive an optical signal through the receiving end according to the exposure time corresponding to the ith sub-measurement period, the measurement period includes sub-measurement periods corresponding to different exposure times, and the optical signal includes the laser The reflected light signal of the signal;
- the generation module 903 is configured to generate the depth image frame based on the received light signal when each of the sub-measurement periods in the measurement period ends.
- the receiving module 902 is also configured to:
- the optical signal is received by the receiving end.
- the receiving module 902 is also configured to:
- the optical signal is received by the receiving end at the transmission moment, and the maximum exposure time is determined by the transmission period of the laser signal;
- the optical signal is received by the receiving end.
- the measurement period includes N emission periods, where N is an integer greater than 1, and the emission period is the emission time difference between adjacent laser signals;
- different sub-measurement periods include the same number of transmission cycles, or different sub-measurement periods include different numbers of transmission cycles.
- the exposure time corresponding to at least one sub-measurement period is a maximum exposure time, and the maximum exposure time is determined by the emission period of the laser signal.
- the device also includes:
- An acquisition module configured to acquire the j-1th exposure result data used to generate the j-1th frame depth image frame, the j-1th exposure result data including the total photons received by each pixel point in the measurement period Number, wherein j is an integer greater than 1;
- An adjustment module configured to adjust the j-1th exposure parameter according to a preset adjustment method to generate a j-th exposure parameter when the j-1th exposure result data satisfies a preset condition, and the j-1th exposure parameter
- the exposure parameter is the exposure parameter used when acquiring the j-1th depth image frame
- the j-1th exposure parameter includes: k sub-measurement periods into which the measurement period is divided and the corresponding sub-measurement periods Exposure time, the preset adjustment method is used to change the division method of the measurement period, k is an integer greater than 1;
- a sending module configured to send an exposure parameter update instruction to the receiving end based on the jth exposure parameter, and the receiving end is configured to receive the optical signal based on the jth exposure parameter.
- the adjustment module is also used for:
- the j-1th exposure parameter is adjusted according to a first preset adjustment method, and the first preset adjustment method is used to reduce
- the first exposure time corresponds to the number of emission cycles included in the sub-measurement period
- increasing the second exposure time corresponds to the number of emission cycles included in the sub-measurement period, the second exposure time being less than the first exposure time
- the j-1th exposure parameter is adjusted according to a second preset adjustment method, and the second preset adjustment method is used to increase
- the first exposure time corresponds to the number of emission periods included in the sub-measurement period
- the second exposure time is reduced corresponding to the number of emission periods included in the sub-measurement period, the second exposure time being shorter than the first exposure time.
- the first preset condition includes: the total number of photons received by a single pixel in the measurement period is higher than a first preset photon number threshold, the total photon number is higher than the first preset
- the number of pixels of the photon number threshold is set to be greater than the first preset number threshold, and the proportion of pixels with the total photon number higher than the first preset photon number threshold is higher than at least one of the first preset ratio thresholds A sort of;
- the second preset condition includes: the total number of photons received by a single pixel in the measurement period is lower than a second preset photon number threshold, the total photon number is lower than the second preset photon number threshold The number of pixels is greater than the second preset number threshold, and the proportion of pixels with the total photon number lower than the second preset photon number threshold is higher than at least one of the second preset ratio thresholds, wherein , the second preset photon number threshold is less than or equal to the first preset photon number threshold.
- the adjustment module is also used for:
- the first increase is positively correlated with the first quantity difference
- the first decrease is positively correlated with the first quantity difference
- the j-1th exposure parameter is adjusted based on the first increase amount and the first decrease amount.
- the adjustment module is also used for:
- the second increase is positively correlated with the second quantity difference
- the second decrease is positively correlated with the second quantity difference
- the j-1th exposure parameter is adjusted based on the second increase amount and the second decrease amount.
- long-distance objects have a longer total exposure time than short-distance objects, which ensures the measurement of long-distance targets while preventing overexposure and accumulation effects.
- Accuracy so that the measurement accuracy of farther and closer objects in the same field of view can be taken into account, and the distance difference between distant and short-distance targets in the simultaneous detection of the field of view can be further increased, thereby improving the distance dynamic range of the depth image acquisition device.
- the acquisition device of the dTOF depth image provided by the above-mentioned embodiment implements its functions, it only uses the division of the above-mentioned functional modules as an example. In practical applications, the above-mentioned functions can be assigned to different functions according to needs. Module completion means that the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.
- the acquisition device of the dTOF depth image provided by the above embodiment and the embodiment of the acquisition method of the dTOF depth image belong to the same concept, and its specific implementation process is detailed in the method embodiment, and will not be repeated here.
- the electronic device 1000 may be a device with a depth image collection function, which may be a smart phone, a tablet computer, a smart TV, a digital camera, etc. This embodiment does not limit the electronic device.
- the electronic device 1000 in this application may include one or more of the following components: a processor 1001 , a memory 1002 and an image acquisition component 1003 .
- the memory 1002 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory, ROM).
- RAM Random Access Memory
- ROM Read-Only Memory
- the memory 1002 includes a non-transitory computer-readable storage medium.
- the memory 1002 may be used to store instructions, programs, codes, sets of codes, or sets of instructions.
- the memory 1002 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the above method embodiments, etc.; the storage data area can also store data created by the electronic device 1000 during use (such as phonebook, audio and video data, chat record data, image data) and the like.
- the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the above method embodiments, etc.
- the storage data area can also store data created by the electronic device 1000 during use (such as phonebook, audio and video data, chat record data, image data) and the like.
- Processor 1001 may include one or more processing cores.
- the processor 1001 uses various interfaces and lines to connect various parts of the entire electronic device 1000, and executes or executes instructions, programs, code sets or instruction sets stored in the memory 1002, and calls data stored in the memory 1002 to execute Various functions of the electronic device 1000 and processing data.
- the processor 1001 may be implemented in at least one hardware form among DSP, Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA).
- the processor 1004 may integrate one or a combination of a CPU, a Graphics Processing Unit (GPU), a modem, and the like.
- the image acquisition component 1003 is used to realize the dTOF depth image acquisition function.
- the image acquisition component 1003 may include a transmitting end and a receiving end, wherein the transmitting end is used for emitting laser signals, and the receiving end is used for receiving optical signals.
- the electronic device 1000 may also include a touch display screen, which may be a capacitive touch display screen, and the capacitive touch screen screen is used to receive any suitable information such as a finger or a stylus from the user. Objects are touched on or near it, and the user interface of each application is displayed.
- the touch display screen is usually arranged on the front panel of the electronic device 1000 .
- Touch screens can be designed as full screens, curved screens or special-shaped screens.
- the touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in this embodiment of the present application.
- the structure of the electronic device 1000 shown in the above drawings does not constitute a limitation on the electronic device 1000, and the electronic device may include more or fewer components than shown in the figure, or Combining certain parts, or different arrangements of parts.
- the electronic device 1000 also includes components such as a sensor, a radio frequency circuit, an audio circuit, a wireless fidelity (Wireless Fidelity, WiFi) component, a power supply, and a Bluetooth component, which will not be repeated here.
- the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores at least one program code, and the program code is loaded and executed by a processor to realize the dTOF depth image as described in each of the above embodiments collection method.
- a computer program product comprising computer instructions stored in a computer readable storage medium.
- the processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for acquiring a dTOF depth image provided in various optional implementation manners of the above aspects.
- the "plurality” mentioned herein refers to two or more than two.
- “And/or” describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently.
- the character "/” generally indicates that the contextual objects are an "or” relationship.
- the numbering of the steps described herein only exemplarily shows a possible sequence of execution among the steps. In some other embodiments, the above-mentioned steps may not be executed according to the order of the numbers, such as two different numbers The steps are executed at the same time, or two steps with different numbers are executed in the reverse order as shown in the illustration, which is not limited in this embodiment of the present application.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
一种dTOF深度图像的采集方法、装置、电子设备及介质,属于计算机视觉技术领域。该方法包括:在第i个子测量周期内,通过发射端发射激光信号(301);根据第i个子测量周期对应的曝光时间,通过接收端接收光信号(302);在测量周期内各个子测量周期结束的情况下,基于接收到的光信号生成深度图像帧(303)。采用dTOF深度图像的采集方法,可以兼顾同一视场内远近物体的测量精度。
Description
本申请要求于2021年05月21日提交的申请号为202110557552.2、发明名称为“dTOF深度图像的采集方法、装置、电子设备及介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请实施例涉及计算机视觉技术领域,特别涉及一种dTOF深度图像的采集方法、装置、电子设备及介质。
随着三维(Three Dimensions,3D)技术的发展,立体显示、机器视觉和卫星遥感等应用场景中均需要获取场景的深度信息。目前,获取深度信息的原理可以分为双目测距、结构光和飞行时间(Time of Flight,TOF)三种,其中,TOF基于测距原理的不同又分为直接测量飞行时间(direct Time-of-Flight,dTOF)和间接测量飞行时间(indirect Time-of-Flight,iTOF)两种。
相关技术中,采用dTOF获取深度信息的过程为:发射端发射光能量集中的调制脉冲光,同时在该脉冲周期内打开接收端接收光子,经过多个脉冲周期的曝光后,接收端接收到数量足够多的光子,进而绘制出各个像素点(pixel)在一个脉冲周期不同时刻接收到光子数的分布直方图,基于分布直方图可恢复出待测目标的深度信息。该曝光方式可以同时接收到来自远距离目标和近距离目标反射的光信号,但是,距离过近时接收到的反射光过强,可能出现过曝而测量出错或发生堆积(pile up)效应,使近距离目标测量误差过大;若需要保证近距离测量准确,则必须减少dTOF发射的激光脉冲个数或者缩短曝光时间,但同时也会导致接收到远距离目标或低反射率目标的光子数不足,从而降低远距离目标或低反射率目标的信噪比与测量精度。
显然,采用目前的曝光方式无法兼顾同一视场内较远和较近不同距离的测量精度,从而导致深度图像采集设备的距离动态测量范围较小。
发明内容
本申请实施例提供了一种dTOF深度图像的采集方法、装置、电子设备及介质。所述技术方案如下:
一方面,本申请实施例提供了一种dTOF深度图像的采集方法,所述方法包括:
在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数;
根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,所述测量周期中包含对应不同曝光时间的子测量周期,所述光信号中包含所述激光信号的反射光信号;
在所述测量周期内各个所述子测量周期结束的情况下,基于接收到的所述光信号生成所述深度图像帧。
另一方面,本申请实施例提供了一种dTOF深度图像的采集装置,所述装置包括:
发射模块,用于在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数;
接收模块,用于根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,所述测量周期中包含对应不同曝光时间的子测量周期,所述光信号中包含所述激光信号的反射光信号;
生成模块,用于在所述测量周期内各个所述子测量周期结束的情况下,基于接收到的所述光信号生成所述深度图像帧。
另一方面,本申请实施例提供了一种电子设备,所述电子设备包括处理器和存储器,所述存储器中存储有至少一条程序代码,所述程序代码由所述处理器加载并执行以实现如上述方面所述的dTOF深度图像的采集方法。
另一方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质中存储有至少一条程序代码,所述程序代码由处理器加载并执行以实现如上述方面所述的dTOF深度图像的采集方法。
另一方面,本申请实施例提供了一种计算机程序产品,该计算机程序产品包括计算机指令,该计算机指令存储在计算机可读存储介质中。电子设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该电子设备执行上述方面的各种可选实现方式中提供的dTOF深度图像的采集方法。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1示出了相关技术中单帧深度图像帧对应深度信息的测量时序图;
图2本申请一个示例性实施例示出的单帧深度图像帧对应的测量时序图;
图3示出了本申请一个示例性实施例提供的dTOF深度图像的采集方法的流程图;
图4示出了本申请另一个示例性实施例示出的单帧深度图像帧对应的测量时序图;
图5示出了本申请另一个示例性实施例提供的dTOF深度图像的采集方法的流程图;
图6示出了本申请一个实施例提供的曝光延迟时长的示意图;
图7示出了本申请另一个示例性实施例提供的dTOF深度图像的采集方法的流程图;
图8示出了本申请一个示例性实施例示出的曝光方式的调整流程图;
图9示出了本申请一个实施例提供的dTOF深度图像的采集装置的结构框图;
图10示出了本申请一个示例性实施例提供的电子设备的结构方框图。
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
相关技术中,如图1所示,其示出了相关技术中单帧深度图像帧对应深度信息的测量时序图。假设采集一帧深度图像帧对应的深度信息需要发射端(TX)发射N个激光脉冲,脉冲周期为Td,从一个激光脉冲发出到第二个激光脉冲发出的这段时间为一个脉冲周期。且针对单个激光脉冲,接收端(RX)的曝光时间为T(T为最大曝光时间),也就是说,在发射端发射激光脉冲的同时,接收端打开曝光等待接收反射光信号;且N个脉冲周期所对应的曝光时间均为T。
显然,采用上述曝光方式,每个脉冲周期内均可以接收到来自远距离目标和近距离目标反射的光信号,既可以测量远距离也可以测量近距离,但是,当目标距离过近且反射光较强时,可能会出现过曝现象而导致测量距离出错,或者可能会出现堆积效应,从而导致近距离目标测量误差较大,为了保证近距离目标测量的准确性,需要通过减少发射端发射的脉冲个数,以减少接收到的近距离目标的反射光,示意性的,若初始采集深度图像时,发射端通过发射1000个激光脉冲获取一帧深度图像,存在过曝现象,为了减少接收到的近距离目标的反射光,采集下一帧深度图像时,将发射端发射的脉冲个数调整为600个。通过减少采集每帧深度图像对应的激光脉冲个数,虽然可以减少接收到的近距离目标的反射光,但是同时也会减少接收到的远距离目标的反射光,从而导致接收到的远距离光子数不足,进而降低了远距离信噪比和测量精度。
显然,采用相关技术中的曝光方式,无法兼顾同一视场内较远和较近不同距离目标的测量精度,距离动态范围较小。
基于相关技术中曝光方式所带来的问题,本申请实施例提供了一种获取深度图像的曝光方式,如图2所示,其示出了本申请一个示例性实施例示出的单帧深度图像帧对应的测量时序图。本实施例将一次测量周期分为三段子测量周期(测量周期即获取单帧深度图像帧所需要的测量时间),其中,分别为不同的子测量周期设置不同的曝光时间,示意性的,分别设定m
1段对应曝光时间T(T为最大曝光时间),m
2段对应曝光时间T/2,m
3段对应曝光时间T/4。在实际应用过程中,针对m
1段,当TX开始发射激光脉冲,RX立刻打开曝光等待接收反射光,完成第一段曝光过程;针对m
2段,RX延迟T/2的时间后,继续曝光T/2的时间;针对m
3段,RX延迟T3/4的时间后,继续曝光T/4的时间。最后,统计N个激光脉冲下,单光子雪崩二极管(Single Photon Avalanche Diode,SPAD)被触发的总次数,绘制出各个像素点(pixel)在T不同时刻接收到光子数的分布直方图,并根据分布直方图峰值得到的飞行时间,进而获得该像素点对应的目标距离。
采用如图2所示的曝光方式,通过减少曝光时间为T的脉冲周期数(由原来的N减少为m
1段),可以有效减少近距离接收到的光强;且m
2、m
3段曝光均仅接收脉冲周期后T/2或后T/4反射回来的光子,同样可以有效减少近距离接收到的光强,同时由于每个子测量周期内均会接收到远距离反射的光强,使得在减少近距离接收到的光强的同时,不会减少远距离反射的光强,从而不会降低远距离测量的信噪比。使得可以同时兼顾远近距离目标的测量精度,同时进一步提高了距离动态范围。
请参考图3,其示出了本申请一个示例性实施例提供的dTOF深度图像的采集方法的流程图,本申请实施例以该方法应用于电子设备为例进行说明,该方法包括:
步骤301,在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数。
其中,测量周期指示测量单帧深度图像帧对应深度信息所需要的时间,以激光脉冲划分来看,即每个测量周期中包含固定数量的激光脉冲,比如,测量周期中包含N个激光脉冲。示意性的,测量周期可以是按照一定发射周期发射1000个激光信号(激光脉冲)的时间,发射周期为相邻两次激光脉冲的发射时间的时间差。
可选的,测量周期中包含的激光脉冲的个数与发射光功率、传感器(sensor)性能、待测目标反射率和环境光噪声强度等多个因素相关,在实际应用过程中,需要根据实际情况进行选择。示意性的,若发射光功率较低,为了保证接收端可以接收到较充足的反射光信号,在测量周期内可能需要发射较多的激光脉冲,反之,若发射功率较高,在测量周期内需要发射较少的激光脉冲;若待测目标反射率比较低,为了保证接收到较充足的反射光,也需要在测量周期内发射较多的激光脉冲,反之,若待测目标反射率较高,相应需要减少测量周期内发射的激光脉冲的个数。
通过dTOF原理获取待测目标的深度信息的过程中,需要发射端发射光能量集中的调制脉冲光(比如,激光信号),同时接收端在发射周期内打开曝光接收光子,经过多个脉冲周期的曝光后,通过统计单个像素点在整个测量周期内不同时刻接收到的光子数,绘制出该像素点对应的光子数分布直方图,并将光子数分布直方图中出现频率最高的接收时刻确定为该像素点对应的光飞行时间,进而基于该光飞行时间恢复出该像素点对应的深度距离信息。且相关技术中,对于测量周期内各个发射周期均采用相同的曝光时间,在进行曝光时间的调整时,测量周期内的曝光时间也对应同步调整,会使得在调节过程中无法同时兼顾近距离测量精度和远距离测量精度,比如,若曝光时间缩短,会使得远距离接收到的光子数不足,从而影响到远距离测量精度;若增加曝光时间,会使得近距离接收到的反射光过强,出现过曝或堆积效应,从而影响近距离测量精度。
基于相关技术中的问题,本实施例中,首先对测量周期进行划分,划分出n个子测量周期,从而在不同子测量周期内分别采用不同曝光时间,使得远近距离目标的曝光时间在一帧内即具有一定差距,从而进一步通过调整不同子测量周期所对应的曝光时间,或不同曝光时间对应的子测量周期,调整同时探测到视场内远近距离目标的距离差。
示意性的,为了达到提高动态范围的效果,本实施例中n的取值最小为2,即将测量周期划分为两个子测量周期,示意性的,若测量周期中包含1000个发射周期,将测量周期划分为两段子测量周期m
1和m
2,其中,m
1段可以包括400个发射周期,m
2段可以包括600个发射周期。可选的,n的取值还可以是3、4、5等大于1的整数,在实际应用过程中,可以基于实际需求将测量周期划分为任意不小于2的n个子测量周期。
可选的,不同子测量周期内包含的发射周期(脉冲周期)的数量可以相同,也可以不相同,本实施例对此不构成限定。
示意性的,不同子测量周期和测量周期的关系可以表示为:
其中,m
1表示第一个子测量周期(i=1),
表示第一个子测量周期中所包含的脉冲周期数,m
2表示第二个子测量周期(i=2),
表示第二个子测量周期中所包含的脉冲周期数,m
n表示第n个子测量周期(i=n),
表示第n个子测量周期中所包含的脉冲周期数,N表示测量周期中包含的脉冲周期总数,n≥2表示至少将测量周期划分为2个子测量周期;其中,脉冲周期数也就是发射周期数。
需要说明的是,第i个子测量周期指n个子测量周期中的任意一个子测量周期,示意性的,若n为5,表示测量周期被划分为5个子测量周期,i的取值可以是1、2、3、4、5,i取1时,即测量周期中的第一个子测量周期。
步骤302,根据第i个子测量周期对应的曝光时间,通过接收端接收光信号,测量周期中包含对应不同曝光时间的子测量周期,光信号中包含激光信号的反射光信号。
其中,曝光时间指示接收端在每次发射周期内(或脉冲周期内)打开曝光接收光信号的时间。由于接收端的SPAD具有淬灭时间,即SPAD在一个脉冲周期曝光开启后的任一时刻成功被单个光子激发后会有一段淬灭时间,这段时间SPAD无法继续被光子激发。示意性的,如图2所示,从RX曝光关闭到TX发射第二次激光脉冲之间的小段时间,为淬灭时间。因此,可以理解曝光时间小于发射周期。
示意性的,在采集深度图像帧之前,需要预先将各个子测量周期和曝光时间的对应关系设置到接收端,以便接收端在采集深度图像帧过程中,可以根据子测量周期对应的曝光时间,通过接收端接收光信号。
可选的,各个子测量周期和曝光时间的对应关系可以是人为预先设置的,比如,图像采集设备的使用人员在曝光参数配置界面中,手动配置子测量周期和曝光时间的对应关系,图像采集设备在接收到对曝光参数的设置操作后,将该设置操作所指示的子测量周期和曝光时间的对应关系设置到接收端;可选的,各个子测量周期和曝光时间的对应关系也可以由图像采集设备根据前一帧曝光参数动态调整后得到,并将调整后曝光参数所指示的各个子测量周期和曝光时间的对应关系设置到接收端。
在一种可能的实施方式中,测量周期中包含对应不同曝光时间的子测量周期,使得位于不同距离处的待测目标在同一测量周期内即具备不同的总曝光时长。而针对调整曝光参数过程中无法兼顾远近目标的情况,在分析远近距离目标的曝光过程可知,近距离目标一般会在曝光时间的前半段被接收到,且距离越近,光子的接收时刻就越靠前;而远距离目标一般会在曝光时间的后半段被接收到,且距离越远,光子的接收时刻就越靠后,基于该曝光特性,在设置不同子测量周期对应的曝光时间时,对于曝光时间较长的(可以同时接收到远近距离的反射光),设置较短的子测量周期,即子测量周期中包含较少的发射周期数,对于曝光时间较短的(将曝光时间设置在发射周期后半段,可以减少近距离目标的反射光,同时可以保证接收到远距离目标的反射光),设置较长的子测量周期,即子测量周期中包含较多的发射周期数,因此,当测量周期中既包含对应长曝光时间的子测量周期,又包含对应短曝光时间的子测量周期,同时将短曝光时间设置在发射周期后半段时,可以在减少近距离接收光强的同时,保证了远距离接收到的光强,使得在调整曝光参数的过程中可以兼顾远近距离目标的曝光情况。
示意性的,如图2所示,以最大曝光时间T为例,测量周期被划分为3个子测量周期,不同子测量周期对应的曝光时间可以为:m
1=T、m
2=T/2、m
3=T/4等。
需要说明的是,曝光时间为T/k(k≥2),k可以取不小于2的任意整数,比如,3、4、5等。但是在实际应用过程中,可以固定k为2的倍数,由于硬件底层中,2的n次幂通过移位操作可以很方便地实现,硬件开销较小。
可选的,在同一个子测量周期内曝光时间是相同的,而不同子测量周期内,曝光时间是不同的。
可选的,在n段子测量周期内,不同子测量周期对应的曝光时间不需要按照递增或者是递减的顺序,也可以是其他顺序。示意性的,m
1、m
2、m
3对应的曝光时间可以为T、T/2、T/4;m
1、m
2、m
3对应的曝光时间也可以为T/2、T、T/4;m
1、m
2、m
3对应的曝光时间也可以为T、T/4、T/2。
如图4所示,其示出了本申请另一个示例性实施例示出的单帧深度图像帧对应的测量时序图。其中,第一个子测量周期(m
1段)对应的曝光时间为T/2,第二个子测量周期(m
2段)对应的曝光时间为T,第三个子测量周期(m
3段)对应的曝光时间为T/4。
需要说明的是,图4和图2之间的差异仅是交换了不同子测量周期的测量时序,并未修改同一曝光时间对应子测量周期中所包含的发射周期数,也就是说,测量周期中不同距离物体的总曝光时长并未发生改变,因此,在其他外部条件(比如,发射端光功率、接收端sensor感光能力、应用环境等)相同的情况下,采用图2和图4的测量过程,采集到的光信号强度应该相同,可以达到相同的距离测量效果。
在一种可能的实施方式中,在第i个子测量周期内,当发射端发射激光信号时,接收端基于第i个子测量周期对应的曝光时间打开曝光接收光信号。其中,接收端接收到的光信号中可以包括激光信号被待测目标反射后的反射光信号,也可以包括环境光。
如图4所示,在第一个子测量周期内(即图4中的第m
1段),其对应的曝光时间为T/2,当发射端发射激光信号后,接收端延迟T/2的时间后,打开曝光,继续曝光T/2的时间,即在后T/2时间内接收反射光信号;在m
3段,其对应的曝光时间为T/4,当发射端发射激光信号后,接收端延迟3T/4的时间后,打开曝光接收反射光,继续曝光T/4的时间,在m
2段,其对应的曝光时间为T,当发射端发射激光信号后,接收端立即打开曝光接收T时间的反射光。采用图4所示的曝光方式,通过减少曝光时间为T的脉冲周期数(由原来的N减少为m
2段),可以有效减少近距离接收到的光强,且m
1和m
3段曝光均仅接收脉冲周期后T/2和后T/4反射回来的光子,同样可以有效减少近距离接收到的光强,同时由于每个子测量周期内均会接收到远距离反射的光强,使得在减少近距离接收到的光强的同时,不会减少远距离反射的光强,从而不会降低远距离测量的信噪比,使得可以同时兼顾远近距离目标的测量精度。
步骤303,在测量周期内各个子测量周期结束的情况下,基于接收到的光信号生成深度图像帧。
在一种可能的实施方式中,当单帧深度图像帧对应的测量周期结束后,即每个子测量周期均测量结束后,通过统计在N个激光信号激励下,接收端的SPAD被触发的总次数和触发时刻,绘制出不同像素点在一个曝光时间T内不同时刻接收到光子数的分布直方图,基于分布直方图的直方图峰值确定出该像素点对应的光飞行时间,从而基于光飞行时间确定出该像素点对应的目标距离,进而可以获取到待测物体对应的深度图像信息,生成对应的深度图像帧。
示意性的,深度图像帧中每个像素点的像素值为该像素点对应的距离。
在一种可能的实施方式中,在采集深度图像帧之前,预先设置不同子测量周期对应的曝光时间至接收端,使得接收端基于该曝光时间确定何时接收光信号;在深度图像帧对应深度信息的测量周期中,对于每个子测量周期,当发射端发射激光信号后,接收端均根据该子测量周期对应的曝光时间,确定接收光信号的时机,直至测量周期中的各个子测量周期结束后,基于接收到的光信号生成深度图像帧。
综上所述,本申请实施例中,在深度图像的获取场景中,通过将单帧深度图像帧对应深度信息的测量周期划分为不同子测量周期,进而为不同子测量周期设置不同曝光时间,使得可以通过调整不同子测量周期的划分,或调整不同子测量周期对应的曝光时间,进而调整在同一测量周期内,远距离目标的总曝光时长和近距离目标的总曝光时长,使得近距离物体可以对应较短的总曝光时长,防止过曝和堆积效应,同时远距离物体相对于近距离物体对应有较长的总曝光时长,在防止过曝和堆积效应的同时保证了远距离目标的测量精度,从而可以兼顾同一视场内更远和更近物体的测量精度,进一步可以增加同时探测视场内远、近距离目标的距离差,从而提高了深度图像采集设备的距离动态范围。
可选的,根据第i个子测量周期对应的曝光时间,通过接收端接收光信号,包括:
基于第i个子测量周期对应的曝光时间,确定第i个子测量周期对应的曝光延迟时长;
在激光信号的发射时刻与当前接收时刻之间的时间差达到曝光延迟时长的情况下,通过接收端接收光信号。
可选的,根据第i个子测量周期对应的曝光时间,通过接收端接收光信号,包括:
在曝光时间为最大曝光时间的情况下,在发射时刻通过接收端接收光信号,最大曝光时间由激光信号的发射周期确定;
在曝光时间小于最大曝光时间,且发射时刻与当前接收时刻之间的时间差达到曝光延迟时长的情况下,通过接收端接收光信号。
可选的,测量周期中包含N个发射周期,N为大于1的整数,发射周期为相邻激光信号之间的发射时间差值;
其中,不同子测量周期中包含相同发射周期数,或,不同子测量周期中包含不同发射周期数。
可选的,存在至少一段子测量周期对应的曝光时间为最大曝光时间,最大曝光时间由激光信号的发射周期确定。
可选的,方法还包括:
获取用于生成第j-1帧深度图像帧的第j-1曝光结果数据,第j-1曝光结果数据包括各个像素点在测量周期中接收到的总光子数,其中,j为大于1的整数;
在第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,生成第j曝光参数,第j-1曝光参数为采集第j-1深度图像帧时所使用的曝光参数,第j-1曝光参数包括:测量周期被划分的k个子测量周期以及各个子测量周期对应的曝光时间,预设调节方式用于改变测量周期的划分方式,k为大于1的整数;
基于第j曝光参数向接收端发送曝光参数更新指令,接收端用于基于第j曝光参数接收光信号。
可选的,在第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,包括:
在第j-1曝光结果数据满足第一预设条件的情况下,按照第一预设调节方式对第j-1曝光参数进行调节,第一预设调节方式用于减少第一曝光时间对应子测量周期中所包含的发射周期数,并增加第二曝光时间对应子测量周期中所包含的发射周期数,第二曝光时间小于第一曝光时间;
和/或,
在第j-1曝光结果数据满足第二预设条件的情况下,按照第二预设调节方式对第j-1曝光参数进行调节,第二预设调节方式用于增加第一曝光时间对应子测量周期中所包含的发射周期数,并减少第二曝光时间对应子测量周期中所包含的发射周期数,第二曝光时间小于第一曝光时间。
可选的,第一预设条件包括:单个像素点在测量周期中接收到的总光子数高于第一预设光子数阈值、总光子数高于第一预设光子数阈值的像素点数量大于第一预设数量阈值,以及总光子数高于第一预设光子数阈值的像素点所占比例高于第一预设比例阈值中的至少一种;
第二预设条件包括:单个像素点在测量周期中接收到的总光子数低于第二预设光子数阈值、总光子数低于第二预设光子数阈值的像素点数量大于第二预设数量阈值,以及总光子数低于第二预设光子数阈值的像素点所占比例高于第二预设比例阈值中的至少一种,其中,第二预设光子数阈值小于等于第一预设光子数阈值。
可选的,按照第一预设调节方式对第j-1曝光参数进行调节,包括:
基于总光子数和第一预设光子数阈值之间的第一数量差值,确定第二曝光时间对应发射周期数的第一增长量,以及确定第一曝光时间对应发射周期数的第一减少量,第一增长量与第一数量差值呈正相关关系,且第一减少量与第一数量差值呈正相关关系;
基于第一增长量和第一减少量,对第j-1曝光参数进行调节。
可选的,按照第二预设调节方式对第j-1曝光参数进行调节,包括:
基于总光子数和第二预设光子数阈值之间的第二数量差值,确定第一曝光时间对应发射周期数的第二增长量,以及确定第二曝光时间对应发射周期数的第二减少量,第二增长量与第二数量差值呈正相关关系,第二减少量与第二数量差值呈正相关关系;
基于第二增长量和第二减少量,对第j-1曝光参数进行调节。
由于距离较近的目标,光飞行时间比较短,反射回来的光子会在接收端曝光时间的前半段被接收到,且距离越近,接收到光子的时刻会越靠前,反射光比较强,容易造成过曝或者堆积效应,本实施例中,可以通过在曝光时间的后半段,或后T/4段接收光子,从而有效减少近距离接收到的光强,同时可以避免减少接收远距离反射光强。
请参考图5,其示出了本申请另一个示例性实施例提供的dTOF深度图像的采集方法的流程图,本申请实施例以该方法应用于电子设备为例进行说明,该方法包括:
步骤501,在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数。
其中,测量周期中包含N个发射周期,N为大于1的整数,发射周期为相邻激光信号之间的发射时间差值。示意性的,如图4所示,发射周期为T
d。
可选的,不同子测量周期中包含相同发射周期数,示意性的,测量周期包含1200个发射周期,被划分为3个子测量周期,每个子测量周期中均可以包含400个发射周期;或,不同子测量周期中也可以包含不同发射周期数,示意性的,第一个子测量周期可以包括200个发射周期,第二个子测量周期中可以包含600个发射周期,第三个子测量周期可以包含400个发射周期。
步骤502,基于第i个子测量周期对应的曝光时间,确定第i个子测量周期对应的曝光延迟时长。
由于远距离的反射光子一般会在发射周期的后半段被接收到,因此,为了保证在缩短曝光时间的同时不会减少接收远距离反射光强,在一种可能的实施方式中,当发射端发射激光信号后,接收端延迟一段时间再继续接收光信号,其中,接收端曝光延迟时长可以由曝光时间确定,即通过第i个子测量周期对应的曝光时间,确定第i个子测量周期对应的曝光延迟时长。
示意性的,曝光延迟时长由最大曝光时间和第i子测量周期对应的曝光时间确定,即曝光延迟时长和曝光时间之和为最大曝光时间。其中,应用最大曝光时间时,发射端发射激光信号后,立即通过接收端接收反射光(即不存在曝光延迟时长,或曝光延迟时长为0的情况),且最大曝光延迟时长和猝灭时长之和等于发射周期(发射周期为相邻两个激光信号的发射时间差)。
由图6可知,最大曝光时间为T,在确定曝光延迟时长时,可以基于最大曝光时间和第i个子测量周期对应的曝光时间确定,示意性的,最大曝光时间为T,若第i个子测量周期对应的曝光时间为T/2,则第i个子测量周期对应的曝光延迟时长为T/2,即在发射端发射激光脉冲后,延迟T/2时间通过接收端接收T/2时间段的光信号;若第i个子测量周期对应的曝光时间为T/4,则第i个子测量周期对应的曝光延迟时长为3T/4,即在发射端发射激光信号后,延迟3T/4时间通过接收端接收T/4时间段的光信号。
步骤503,在激光信号的发射时刻与当前接收时刻之间的时间差达到曝光延迟时长的情况下,通过接收端接收光信号。
本实施例中,发射端的工作时钟和接收端的工作时钟同步,当发射端发射激光信号时,接收端以该激光信号的发射时刻开始,进入接收周期(接收周期与发射周期对应),即接收周期的初始时刻为发射时刻,对应在确定是否达到曝光延迟时长时,仅需要确定接收端的当前接收时刻与发射时刻之间的时间差是否达到曝光延迟时长即可。
在一种可能的实施方式中,当发射端发射激光信号后,激光信号的发射时刻与当前接收时刻之间的时间差达到该曝光延迟时长后,打开接收端接收光信号。
示意性的,若曝光延迟时长为3T/4,对应的,当发射端发射激光信号后,发射时长达到3T/4时,通过接收端接收T/4时间段的反射光信号。其中,发射时长即激光信号的发射时刻与当前接收时刻之间的时间差。
为了保证可以接收到同一视场内近距离物体反射的光子,在一种可能的实施方式中,可以通过在至少一个子测量周期内采用最大曝光时间(T)实现。即存在至少一段子测量周期对应的曝光时间为最大曝光时间,最大曝光时间由激光信号的发射周期确定。
可选的,由于接收端的SPAD存在淬灭时间,则最大曝光时间小于激光信号的发射周期,可以理解:最大曝光时间和淬灭时间之和为发射周期。
可选的,若曝光时间中包含最大曝光时间,为了保证可以接收到近距离物体的反射光,应该在发射激光信号的同时打开接收端接收反射光信号;若曝光时间小于最大曝光时间,为了保证可以接收到远距离物体的反射光,应该在达到曝光延迟时长后打开接收端接收反射光信号。在一个示例性的例子中,通过接收端接收反射光信号的过程可以包括以下步骤:
一、在曝光时间为最大曝光时间的情况下,在发射时刻通过接收端接收光信号,最大曝光时间由激光信号的发射周期确定。
在一种可能的实施方式中,当存在某个子测量周期对应最大曝光时间,由于曝光延迟时长由最大曝光时间和曝光时间确定,也就是说,该曝光延迟时长为0,即在发射端发射激光信号的同时,立即打开接收端,并通过接收端接收反射光信号。
二、在曝光时间小于最大曝光时间,且发射时刻与当前接收时刻之间的时间差达到曝光延迟时长的情况下,通过接收端接收光信号。
在一种可能的实施方式中,当子测量周期对应的曝光时间小于最大曝光时间,为了减少近距离接收到的光强,同时保证远距离接收到的光强,当发射端发射激光信号后,实时判断发射时刻与当前接收时刻之间的时间差是否达到曝光延迟时长,并在达到曝光延迟时长后,打开接收端,并通过接收端在曝光时间内接收光信号。
示意性的,如图4所示,第一个子测量周期(m
1)对应曝光时间T/2,第二个子测量周期(m
2)对应最大曝光时间T,第三个子测量周期(m
3)对应曝光时间T/4。对于距离较近的目标,光飞行时间短,反射回来的光子会在RX最大曝光时间T的前半周期被接收到,且距离越近,接收到的光子的时刻就会越靠前,而反射光较强的时候,容易造成过曝或者堆积效应,为了降低近距离反射光强,可以通过减少曝光时间为T的子测量周期中所包含的发射周期数,从而有效减少近距离接收到的光强;而m
1段曝光会接收后T/2反射回来的光子,以及m
3段曝光会接收后T/4反射回来的光子,因此在减少近距离反射光的同时不会减少远距离反射的光强,使得可以同时兼顾同一视场内远近距离的测量精度。
步骤504,在测量周期内各个子测量周期结束的情况下,基于接收到的光信号生成深度图像帧。
步骤504的实施方式可以参考步骤303,本实施例在此不做赘述。
本实施例中,基于各个子测量周期对应的曝光时间,确定出曝光延迟时长,使得接收端基于该曝光延迟时长确定何时接收光信号;此外,通过设置子测量周期对应的曝光时间为最大曝光时间,使得接收端可以在发射端发射激光信号后立即开始接收光信号,从而可以接收到视场内近距离物体的反射光;通过设置子测量周期对应的曝光时间小于最大曝光时间,并使得接收端在最大曝光时间的后期接收光信号,可以有效减少近距离接收到的反射光的同时,不会减少远距离接收到的反射光,从而使得可以兼顾远近距离的测量精度,进而可以提高动态测量范围。
由于不同电子设备的采集范围要求不同,比如,电子设备A对采集范围的要求是50cm至1m,电子设备B对采集范围的要求是50cm至10m,在实际应用过程中,需要基于具体需求对测量周期进行划分,以及设置不同子测量周期对应的曝光时间,下文实施例中主要描述如何对曝光参数进行调节的过程。
如图7所示,其示出了本申请另一个示例性实施例提供的dTOF深度图像的采集方法的流程图,本申请实施例以该方法应用于电子设备为例进行说明,该方法包括:
步骤701,获取用于生成第j-1帧深度图像帧的第j-1曝光结果数据,第j-1曝光结果数据包括各个像素点在测量周期中接收到的总光子数,其中,j为大于1的整数。
由于近距离的待测物体可能是反射率较高的,也可能是反射率较低的,如果对于近距离物体均为反射率较低的场景下,减少最大曝光时间T对应子测量周期中包含的发射周期数,可能会降低低反射率目标反射回来的光信号强度,从而影响低反射率目标的距离测量精度,因此,在一种可能的实施方式中,在对测量周期进行划分之前,可以通过预先采集当前场景的一帧深度图像,再基于接收端接收光子数的情况来确定如何划分测量周期,即基于前一帧的曝光参数和曝光结果数据,指导后一帧的曝光参数调节方式。
在一种可能的实施方式中,电子设备在调试过程中,获取生成前一帧(第j-1帧)深度图像帧的第j-1曝光结果数据,该曝光结果数据为接收端在生成前一帧深度图像帧过程中采集到的测量数据,示意性的,本实施例中,该曝光结果数据为统计得到的各个像素点在测量周期内接收到的总光子数。
步骤702,在第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,生成第j曝光参数。
其中,第j-1曝光参数为采集第j-1深度图像帧时所使用的曝光参数,第j-1曝光参数包括:测量周期被划分的k个子测量周期以及各个子测量周期对应的曝光时间,k为大于1的整数。
可选的,预设调节方式可以是改变测量周期的划分方式。示意性的,可以增加测量周期中子测量周期的划分个数;或改变不同子测量周期对应的曝光时间;或改变不同曝光时间对应子测量周期中所包含的发射周期数。
在曝光方式调节过程中存在两种可能的情景,一种是接收到的光强较强,可能是由于较长曝光时间对应子测量周期中所包含的发射周期数较多,使得近距离接收到的光强较多,需要减少近距离接收到的光强;一种是接收到的光强较弱,可能是由于较长曝光时间对应子测量周期中所包含的发射周期数较少,使得近距离接收到的光强较弱,需要增加近距离接收到的光强;本实施例中,针对这两种情景,分别设置有对应的曝光参数调节方式,也即预设条件至少包括第一预设条件(对应光强较强的情景)和第二预设条件(对应光强较弱的情景),预设调节方式也包括第一预设调节方式(用于在满足第一预设条件时使用)和第二预设调节方式(用于在满足第二预设条件时使用),当第j-1曝光结果数据满足第一预设条件时,采用对应的第一预设调节方式,当第j-1曝光结果数据满足第二预设条件时,采用对应的第二预设调节方式。
当像素点接收到的总光子数过多时,可能会导致过曝现象,从而导致测量的距离有误;也可能会产生堆积效应,导致测量距离值偏小,因此,在一种可能的实施方式中,可以通过判断接收到的总光子数,确定是否需要进行曝光参数调节,以及若需要进行曝光参数调节,具体应该如何调节测量周期的划分方式。
由于远距离目标所对应像素点接收到的总光子数一般不会过多,出现接收到的总光子数过多的像素点一般是近距离物体的反射光,因此,当确定出接收到的总光子数过多时,可以适当减少较长曝光时间对应子测量周期中所包含的发射周期数,同时增加较短曝光时间对应子测量周期中所包含的发射周期数。在一个示例性的例子中,对曝光参数的调节过程可以包括以下步骤:
一、在第j-1曝光结果数据满足第一预设条件的情况下,按照第一预设调节方式对第j-1曝光参数进行调节,第一预设调节方式用于减少第一曝光时间对应子测量周期中所包含的发射周期数,并增加第二曝光时间对应子测量周期中所包含的发射周期数,第二曝光时间小于第一曝光时间。
其中,第一预设条件包括:单个像素点在测量周期中接收到的总光子数高于第一预设光子数阈值、总光子数高于第一预设光子数阈值的像素点数量大于第一预设数量阈值,以及总光子数高于第一预设光子数阈值的像素点所占比例高于第一预设比例阈值中的至少一种。需要说明的是,本申请实施例中的总光子数均是指单个像素点在一个测量周期中接收到的总光子数。
可选的,第一预设光子数阈值可以是固定值,也可以是动态调整的值。该第一预设光子数阈值可以依据发射端光功率、接收端sensor感光能力、应用环境(最近、最远测量范围,环境噪声强弱,所需要的距离动态范围等)等设定。示意性的,该第一预设光子数阈值可以基于测量深度信息所需要的光子数来设置。比如,第一预设光子数阈值可以是1000个。
当第一预设条件为单个像素点在测量周期中接收到的总光子数高于第一预设光子数阈值时,在一种可能的实施方式中,获取生成前一帧深度图像帧的曝光结果数据,即每个像素点分别在测量周期中接收到的总光子数,并将每个像素点接收到的总光子数与第一预设光子数阈值进行比较,若存在单个像素点对应的总光子数超过第一预设光子数阈值,可以按照第一预设调节方式对前一帧的曝光参数进行调节。
对于面阵dTOF,pixel数量在几百到几万、十几万数量级,由于硬件上无法实现对每个pixel单独控制曝光时间,曝光时间设定是针对所有pixel统一的,因此不能因为单独几个pixel接收到总光子数过多就调节曝光方式,为了提高曝光方式调节时机的准确性,在一种可能的实施方式中,设置第一预设条件为总光子数高于第一预设光子数阈值的像素点数量大于第一预设数量阈值,或总光子数高于第一预设光子数阈值的像素点所占比例高于第一预设比例阈值。
当第一预设条件为总光子数高于第一预设光子数阈值的像素点数量大于第一预设数量阈值时,在一种可能的实施方式中,比较每个像素点接收到的总光子数与第一预设光子数阈值之间的关系,并统计总光子数较多(总光子数大于第一预设光子数阈值)的像素点数量,进而比较该像素点数量与第一预设数量阈值之间的关系,若该像素点数量高于第一预设数量阈值,表示接收到的总光子数过多,需要按照第一预设调节方式对第j-1帧曝光参数进行调节;否则,可以无需按照第一预设调节方式对第j-1帧曝光参数进行调节。
示意性的,第一预设数量阈值可以基于实际应用场景进行设置,比如,第一预设数量阈值可以是500个。
当第一预设条件为总光子数高于第一预设光子数阈值的像素点所占比例高于第一预设比例阈值时,在一种可能的实施方式中,比较每个像素点接收到的总光子数与第一预设光子数阈值之间的关系,并统计总光子数较多(总光子数大于第一预设光子数阈值)的像素点数量,再确定该像素点数量占全部像素点数量的比值,进而比较该比值与第一预设比例阈值之间的关系,若该比值高于第一预设比例阈值,表示接收到的总光子数过多,需要按照第一预设调节方式对第j-1帧曝光参数进行调节;否则,可以无需按照第一预设调节方式对第j-1帧曝光参数进行调节。
示意性的,第一预设比例阈值也可以基于实际应用场景进行设置,比如,第一预设比例阈值可以是1/4。
对于同一视场内,由于远距离接收到光子数一般不会过多,因此,总光子数超过第一预设光子数阈值的一般是近距离,也即当确定第j-1曝光结果数据满足第一预设条件时,表示近距离接收到的反射光强较多,因此,为了减少近距离接收到的光子数,在一种可能的实施方式中,可以通过减少较长曝光时间(第一曝光时间)对应子测量周期中所包含的发射周期数,以减少近距离接收到的光强;而在测量周期内的总发射周期数一定的情况下,同时还需要增加较短曝光时间(第二曝光时间)对应子测量周期中所包含的发射周期数,以保证不会减少远距离反射的光强,从而不会降低远距离测量的精度。
示意性的,以测量周期被划分为三段子测量周期为例,若第j-1曝光参数指示:第一个子测量周期(m
1)对应曝光时间T/2,m
1中包含400个发射周期,第二个子测量周期(m
2)对应曝光时间T,m
2中包含500个发射周期,第三个子测量周期(m
3)对应曝光时间T/4,m
3中包含300个发射周期,当确定出第j-1曝光结果数据满足第一预设条件时,表示近距离接收到的反射光强较多,可以适当减少曝光时间T对应第二子测量周期中包含的发射周期数,适当增加曝光周期为T/2或T/4对应子测量周期中包含的发射周期数,比如,调整后的第j曝光参数为:m
1中包含600个发射周期,m
2中包含300个发射周期,m
3中包含300个发射周期。
可选的,当第j-1曝光参数满足第一预设条件时,也可以通过增加测量周期被划分的子测量周期来降低近距离的反射光强,示意性的,若第j-1曝光参数指示:第一个子测量周期(m
1)对应曝光时间T/2,m
1中包含400个发射周期,第二个子测量周期(m
2)对应最大曝光时间T,m
2中包含800个发射周期,可以在减少曝光时间T对应子测量周期中所包含的发射周期数的同时,增加曝光时间为T/4的子测量周期,比如,调整后的第j曝光参数指示:测量周期被划分为m
1、m
2和m
3,其中,m
1(对应曝光时间T/2)包括400个发射周期,m
2(对应曝光时间为T)包括500个发射周期,m
3(对应曝光时间为T/4)包括300个发射周期。
可选的,为了可以尽快获得较为合适的测量周期划分方式,在一种可能的实施方式中,可以基于总光子数与第一预设光子数阈值之间的关系,确定子测量周期中包含的发射周期数增加幅度,或子测量周期中包含的发射周期数减少幅度。
在一个示例性的例子中,按照第一预设调节方式对第j-1曝光参数进行调节的过程还可以包括以下步骤:
1.基于总光子数和第一预设光子数阈值之间的第一数量差值,确定第二曝光时间对应发射周期数的第一增长量,以及确定第一曝光时间对应发射周期数的第一减少量,第一增长量与第一数量差值呈正相关关系,且第一减少量与第一数量差值呈正相关关系。
当确定出单个像素点接收到的总光子数越多,表示近距离接收到的光强越大,在调整曝光方式时,可能需要减少较长曝光时间对应的发射周期数越多(以尽快降低近距离接收到的光强),同时需要增加较短曝光时间对应的发射周期数越多,因此,在一种可能的实施方式中,当曝光结果数据满足第一预设条件时,可以基于总光子数与第一预设光子数阈值之间的第一数量差值,确定近距离接收到的光强情况,以便基于该第一数量差值确定各个曝光时间对应发射周期数的调整幅度。若第一数量差值较大,表示近距离接收到的光强越强,需要减少较长曝光时间对应的发射周期数越多,对应的需要增加较短曝光时间对应的发射周期数越多,以尽快减少近距离接收到的光强;反之,若第一数量差值较小,表示近距离接收到的光强相对较弱,为了避免减少较长曝光时间对应的发射周期数过多而对近距离测量精度造成影响,可能需要减少较长曝光时间对应的发射周期数较少,同时需要增加较短曝光时间对应的发射周期数较少,也即较长曝光时间对应发射周期数的减少量与第一数量差值呈正相关关系,较短曝光时间对应发射周期数的增长量也与第一数量差值呈正相关关系。
可选的,电子设备中可以预先设置有第一数量差值、曝光时间、第一增长量和第一减少量之间的对应关系,该对应关系可以由开发人员经过实际测试后得到;使得在实际进行曝光参数的调整过程中,当确定出第一数量差值后,可以根据该对应关系,确定出各个曝光时间所对应的第一增长量和第一减少量。
示意性的,若第一预设光子数阈值为1000,曝光参数为:m
1(对应曝光时间T/2)包括400个发射周期,m
2(对应曝光时间为T)包括800个发射周期,应用该曝光参数分别采集场景一和场景二对应的深度图像帧,采集到的场景一和场景二对应的曝光结果数据满足第一预设条件(即近距离接收光子数过多),其中,场景一对应的曝光结果数据指示:总光子数与第一预设光子数阈值之间的数量差值为A
1;场景二对应的曝光结果数据指示:总光子数与第一预设光子数阈值之间的数量差值为A
2,且A
1大于A
2。由于两个场景对应的曝光结果数据不同,对应的在两个场景下对曝光参数的调整幅度也不相同,基于接收光强、数量差值和调整幅度之间的关系(光强越强,数量差值越大,调整幅度越大),场景一调整后的曝光参数可以为:m
1(对应曝光时间T/2)包括800个发射周期,m
2(对应曝光时间为T)包括400个发射周期,较短曝光时间对应发射周期数的增加量为400个,较长曝光时间对应发射周期数的减少量为400个;场景二调整后的曝光参数可以为:m
1(对应曝光时间T/2)包括600个发射周期,m
2(对应曝光时间为T)包括 600个发射周期,较短曝光时间对应发射周期数的增加量为200个,较长曝光时间对应发射周期数的减少量为200个。
可选的,针对确定第一数量差值的方式,可以首先确定出总光子数(总光子数是指单个像素点在测量周期中接收到的光子数)超过第一预设光子数阈值的特定像素点,并求得特定像素点对应的总光子数之和,再基于总光子数之和与特定像素点的数量,求得特定像素点对应的总光子平均值,进而将总光子数平均值和第一预设光子数阈值之间的数量差值确定为第一数量差值;或无需求总光子数平均值,直接将特定像素点对应的总光子数之和与第一预设光子数阈值之间的数量差值确定为第一数量差值。
可选的,在曝光方式调节过程中,可能存在某些特定应用场景,仅需要关注sensor中特定区域像素点的光强接收情况,比如,仅关注中心区域的像素点,对应的,在确定曝光参数的调节幅度过程中,当确定出总光子数超过第一预设光子数阈值的特定像素点后,还可以基于预设区域,从特定像素点中筛选出目标特定像素点(目标特定像素点是位于预设区域的特定像素点),进而基于目标特定像素点对应的总光子数和第一预设光子数阈值,确定第一数量差值。
可选的,针对确定子测量周期中发射周期数增加幅度或发射周期数减少幅度的方式,在一种可能的实施方式中,也可以基于总光子数高于第一预设光子数阈值的像素点数量与第一预设数量阈值的关系,确定子测量周期中发射周期数增加幅度或发射周期数减少幅度;也即,当曝光结果数据满足第一预设条件时,可以基于像素点数量(总光子数高于第一预设光子数阈值的像素点的数量)与第一预设数量阈值之间的数量差值,确定近距离接收到的光强情况,进而确定各个曝光时间对应发射周期数的调整幅度;若像素点数量与第一预设数量阈值之间的数量差值越大,则较长曝光时间对应发射周期数的减少量越大,较短曝光时间对应发射周期数的增加量越大;反之,若像素点数量与第一预设数量阈值之间的数量差值越小,则较长曝光时间对应发射周期数的减少量越少,且较短曝光时间对应发射周期数的增加量越少。
可选的,还可以基于总光子数高于第一预设光子数阈值的像素点所占比例与第一预设比例阈值之间的关系,确定子测量周期中发射周期数增加幅度或发射周期数减少幅度;也即当曝光结果数据满足第一预设条件时,可以基于像素点所占比例(总光子数高于第一预设光子数阈值的像素点所占的比例)与第一预设比例阈值之间的比例差值,确定近距离接收到的光强情况,进而确定各个曝光时间对应发射周期数的调整幅度;若比例差值越大,则较长曝光时间对应发射周期数的减少量越大,较短曝光时间对应发射周期数的增加量越大;反之,若比例差值越小,则较长曝光时间对应发射周期数的减少量越少,且较短曝光时间对应发射周期数的增加量越少。
2.基于第一增长量和第一减少量,对第j-1曝光参数进行调节。
在一种可能的实施方式中,基于确定出的各个曝光时间所对应发射周期数的增加幅度(第一增长量),以及各个曝光时间所对应发射周期数的减少幅度(第一减少量),对第j-1曝光参数进行调整,从而生成用于采集第j帧深度图像帧对应的第j曝光参数。
示意性的,若第j-1曝光参数为:m
1(对应曝光时间T/2)包括400个发射周期,m
2(对应曝光时间为T)包括800个发射周期,确定出的第一增长量为300个,第一减少量为300个,对应的,第j曝光参数可以为:m
1(对应曝光时间T/2)包括700个发射周期,m
2(对应曝光时间为T)包括500个发射周期。
二、在第j-1曝光结果数据满足第二预设条件的情况下,按照第二预设调节方式对第j-1曝光参数进行调节,第二预设调节方式用于增加第一曝光时间对应子测量周期中所包含的发射周期数,并减少第二曝光时间对应子测量周期中所包含的发射周期数,第二曝光时间小于第一曝光时间。
其中,第二预设条件包括:单个像素点在测量周期中接收到的总光子数低于第二预设光子数阈值、总光子数低于第二预设光子数阈值的像素点数量大于第二预设数量阈值,以及总光子数低于第二预设光子数阈值的像素点所占比例高于第二预设比例阈值中的至少一种,其中,第二预设光子数阈值小于等于第一预设光子数阈值。
可选的,第二预设光子数阈值可以是固定值,也可以是动态调整的值。该第二预设光子数阈值可以依据发射端光功率、接收端sensor感光能力、应用环境(最近、最远测量范围,环境噪声强弱,所需要的距离动态范围等)等设定。示意性的,该第二预设光子数阈值可以基于测量深度信息所需要的光子数来设置。
可选的,第二预设光子数阈值可以与第一预设光子数阈值相同,也可以小于第一预设光子数阈值,示意性的,若第一预设光子数阈值为1000个,则第二预设光子数阈值可以为800个。本实施例对此不构成限定。
当第二预设条件为单个像素点在测量周期中接收到的总光子数低于第二预设光子数阈值时,在一种可能的实施方式中,获取生成前一帧深度图像帧的曝光结果数据,即每个像素点分别对应的总光子数,并比较每个像素点接收到的总光子数与第二预设光子数阈值的关系,若存在单个像素点对应的总光子数低于第二预设光子数阈值,表示近距离接收到的光强较弱,无法基于该光子数较准确的测量出该点对应的深度信息,可以按照第二预设调节方式对前一帧的曝光参数进行调节。
对于面阵dTOF,pixel数量在几百到几万、十几万数量级,由于硬件上无法实现对每个pixel单独控制曝光时间,曝光时间设定是针对所有pixel统一的,因此不能因为单独几个pixel接收到总光子数过多就调节曝光方式,为了提高曝光方式调节时机的准确性,在一种可能的实施方式中,设置第二预设条件为总光子数低于第二预设光子数阈值的像素点数量大于第二预设数量阈值,或总光子数低于第二预设光子数阈值的像素点所占比例高于第二预设比例阈值。
当第二预设条件为总光子数低于第二预设光子数阈值的像素点数量大于第二预设数量阈值时,在一种可能的实施方式中,比较每个像素点接收到的总光子数与第二预设光子数阈值之间的关系,并统计总光子数较少(总光子数低于第二预设光子数阈值)的像素点数量,进而比较该像素点数量与第二预设数量阈值之间的关系,若该像素点数量高于第二预设数量阈值,表示近距离接收到的光强较弱,需要按照第二预设调节方式对第j-1帧曝光参数进行调节;否则,可以无需按照第二预设调节方式对第j-1帧曝光参数进行调节。
示意性的,第二预设数量阈值可以基于实际应用场景进行设置,第二预设数量阈值可以与第一预设数量阈值相同,也可以与第一预设数量阈值不同,本实施例对此不构成限定,比如,第二预设数量阈值可以是800个。
当第二预设条件为总光子数低于第二预设光子数阈值的像素点所占比例高于第二预设比例阈值时,在一种可能的实施方式中,比较每个像素点接收到的总光子数与第二预设光子数阈值之间的关系,并统计总光子数较少(总光子数小于第二预设光子数阈值)的像素点数量,再确定该像素点数量占全部像素点数量的比值,进而比较该比值与第二预设比例阈值之间的关系,若该比值高于第二预设比例阈值,需要按照第二预设调节方式对第j-1帧曝光参数进行调节;否则,可以无需按照第二预设调节方式对第j-1帧曝光参数进行调节。
示意性的,第二预设比例阈值也可以基于实际应用场景进行设置,比如,第二预设比例阈值可以是1/4,可选的,第二预设比例阈值可以与第一预设比例阈值相同,也可以不相同,本申请实施例对第一预设比例阈值和第二预设比例阈值之间的关系不构成限定。
在不调整总发射周期数(或发射端功率、接收端感光能力等外部条件)的基础上,划分测量周期和调整曝光时间一般仅会减少近距离目标的总曝光时长,但是并未减少远距离目标的总曝光时长,因此,若存在某些像素点接收到的光强较弱时,一般是近距离目标对应的像素点,因此,为了增加近距离接收到的光子数,在一种可能的实施方式中,可以增加较长曝光时间(第一曝光时间)对应子测量周期中所包含的发射周期数,并减少较短曝光时间(第二曝光时间)对应子测量周期中所包含的发射周期数。
示意性的,以测量周期被划分为三段子测量周期为例,若第j-1曝光参数指示:第一个子测量周期(m
1)对应曝光时间T/2,m
1中包含600个发射周期,第二个子测量周期(m
2)对应曝光时间T,m
2中包含400个发射周期,第三个子测量周期(m
3)对应曝光时间T/4,m
3中包含200个发射周期,当第j-1曝光结果数据满足第二预设条件时,可以适当增加曝光时间T对应第二子测量时间中包含的发射周期数,并适当减少曝光时间为T/2或T/4对应子测量周期中包含的发射周期数,比如,调整后的第j曝光参数为:m
1中包含400个发射周期,m
2中包含600个发射周期,m
3中包含200个发射周期。
可选的,当第j-1曝光参数满足第二预设条件时,也可以通过减少测量周期被划分的子测量周期来增加近距离的反射光强,示意性的,若第j-1曝光参数指示:m
1(对应曝光时间T/2)包括400个发射周期,m
2(对应曝光时间为T)包括500个发射周期,m
3(对应曝光时间为T/4)包括300个发射周期,可以在增加曝光时间T对应的发射周期数的同时,删除曝光时间为T/4的子测量周期,比如,第j曝光参数指示:测量周期被划分为m
1和m
2,其中,m
1(对应曝光时间T/2)包括500个发射周期,m
2(对应曝光时间为T)包括700个发射周期。
可选的,为了可以尽快获得较为合适的测量周期划分方式,在一种可能的实施方式中,可以基于总光子数与第二预设光子数阈值之间的关系,确定子测量周期中所包含发射周期数的增加幅度,或子测量周期中所包含发射周期数的减少幅度。
在一个示例性的例子中,按照第二预设调节方式对第j-1曝光参数进行调节的过程还可以包括以下步骤:
1.基于总光子数和第二预设光子数阈值之间的第二数量差值,确定第一曝光时间对应发射周期数的第二增长量,以及确定第二曝光时间对应发射周期数的第二减少量,第二增长量与第二数量差值呈正相关关系,第二减少量与第二数量差值呈正相关关系。
当确定出像素点接收到的总光子数较少,表示近距离接收到的光强较小,在调整曝光方式时,可能需要增加较长曝光时间对应的发射周期数越多(以提高近距离目标的反射光强),同时减少较短曝光时间对应的发射周期数越多,以尽快增加近距离接收到的光强,因此,在一种可能的实施方式中,当曝光结果数据满足第二预设条件时,可以基于总光子数与第二预设光子数阈值之间的第二数量差值,确定近距离接收 到的光强情况,以便基于该第二数量差值确定各个曝光时间对应发射周期数的调整幅度。若第二数量差值较大,表示近距离接收到的光强较弱,需要增加较长曝光时间对应的发射周期数越多,对应减少较短曝光时间对应的发射周期数越多;反之,若第二数量差值较小,为了避免增加较长曝光时间对应的发射周期数过多而导致过曝或堆积效应,需要增加较长曝光时间对应的发射周期数较少,对应减少较短曝光时间对应的发射周期数较少。也即较长曝光时间对应发射周期数的增长量与第二数量差值呈正相关关系,较短曝光时间对应发射周期数的减少量与第二数量差值呈正相关关系。
可选的,电子设备中也可以预设设置有第二数量差值、曝光时间、第二增长量和第二减少量之间的对应关系,该对应关系也可以由开发人员经过实际测试后得到;使得在实际进行曝光参数的调整过程中,当确定出第二数量差值后,可以根据该对应关系,确定出各个曝光时间所对应的第二增长量和第二减少量。
示意性的,若第二预设光子数阈值为800,曝光参数为:m
1(对应曝光时间T/2)包括900个发射周期,m
2(对应曝光时间为T)包括300个发射周期,应用该曝光参数分别采集场景一和场景二对应的深度图像帧,采集到的场景一和场景二对应的曝光结果数据满足第二预设条件(即近距离接收到的光子数较少),其中,场景一对应的曝光结果数据指示:总光子数与第二预设光子数阈值之间的数量差值为A
1;场景二对应的曝光结果数据指示:总光子数与第二预设光子数阈值之间的数量差值为A
2,且A
1大于A
2。由于两个场景对应的曝光结果数据不同,对应的在两个场景下对曝光参数的调整幅度也不相同,基于接收光强、数量差值和调整幅度之间的关系(光强越弱,数量差值越大,调整幅度越大),场景一调整后的曝光参数可以为:m
1(对应曝光时间T/2)包括500个发射周期,m
2(对应曝光时间为T)包括700个发射周期,较长曝光时间对应发射周期数的增加量为400,较短曝光时间对应发射周期数的减少量为400;场景二调整后的曝光参数可以为:m
1(对应曝光时间T/2)包括700个发射周期,m
2(对应曝光时间为T)包括500个发射周期,较长曝光时间对应发射周期数的增加量为200,较短曝光时间对应发射周期数的减少量为200。
其中,确定第二数量差值的实施方式可以参考确定第一数量差值的实施方式,本实施例在此不做赘述。
可选的,针对确定子测量周期中发射周期数增加幅度或发射周期数减少幅度的方式,在一种可能的实施方式中,也可以基于总光子数低于第二预设光子数阈值的像素点数量与第二预设数量阈值的关系,确定子测量周期中发射周期数增加幅度或发射周期数减少幅度;也即,当曝光结果数据满足第二预设条件时,可以基于像素点数量(总光子数低于第二预设光子数阈值的像素点的数量)与第二预设数量阈值之间的数量差值,确定近距离接收到的光强情况,进而确定各个曝光时间对应发射周期数的调整幅度;若像素点数量与第二预设数量阈值之间的数量差值越大,需要增加较长曝光时间对应的发射周期数越多,对应减少较短曝光时间对应的发射周期数越多;反之,若像素点与第二预设数量阈值之间的数量差值越小,则需要增加较长曝光时间对应的发射周期数较少,对应减少较短曝光时间对应的发射周期数较少。
可选的,还可以基于总光子数低于第二预设光子数阈值的像素点所占比例与第二预设比例阈值之间的关系,确定子测量周期中发射周期数增加幅度或发射周期数减少幅度;也即当曝光结果数据满足第二预设条件时,可以基于像素点所占比例(总光子数低于第二预设光子数阈值的像素点所占的比例)与第二预设比例阈值之间的比例差值,确定近距离接收到的光强情况,进而确定各个曝光时间对应发射周期数的调整幅度;若比例差值越大,需要增加较长曝光时间对应的发射周期数越多,对应减少较短曝光时间对应的发射周期数越多;反之,若比例差值越小,则需要增加较长曝光时间对应的发射周期数较少,对应减少较短曝光时间对应的发射周期数较少。
2.基于第二增长量和第二减少量,对第j-1曝光参数进行调节。
在一种可能的实施方式中,基于确定出的各个曝光时间所对应发射周期数的增加幅度(第二增长量),以及各个曝光时间所对应发射周期数的减少幅度(第二减少量),对第j-1曝光参数进行调整,从而生成用于采集第j帧深度图像帧对应的第j曝光参数。
示意性的,若第j-1曝光参数为:m
1(对应曝光时间T/2)包括800个发射周期,m
2(对应曝光时间为T)包括400个发射周期,确定出的第二增长量为300个,第二减少量为300个,对应的,第j曝光参数可以为:m
1(对应曝光时间T/2)包括500个发射周期,m
2(对应曝光时间为T)包括700个发射周期。
步骤703,基于第j曝光参数向接收端发送曝光参数更新指令,接收端用于基于第j曝光参数接收光信号。
在一种可能的实施方式中,当基于前一帧曝光参数和曝光结果数据,确定出当前帧对应的曝光参数后,需要向接收端发送曝光参数调节指令,以便接收端基于不同子测量周期对应的曝光参数(曝光时间)接收光信号,用于生成第j帧深度图像帧。
可选的,当生成第j帧深度图像帧后,也可以基于第j帧深度图像帧对应的第j曝光结果数据和第j曝光参数,确定是否需要调节第j+1帧对应的曝光参数,以及如何调节,具体的调节方式可以参考上文实施例中对第j-1帧曝光参数的调节过程,本实施例在此不做赘述。
步骤704,在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数。
步骤705,根据第i个子测量周期对应的曝光时间,通过接收端接收光信号,测量周期中包含对应不同曝光时间的子测量周期,光信号中包含激光信号的反射光信号。
步骤706,在测量周期内各个子测量周期结束的情况下,基于接收到的光信号生成深度图像帧。
步骤704和步骤706的实施方式可以参考上文实施例,本实施例在此不做赘述。
上文实施例中,步骤701至步骤703在步骤704之前执行,该步骤执行顺序下第j曝光参数满足单个测量周期内包含对应不同曝光时间的子测量周期,而第j-1曝光参数可能存在两种情况:(1)第j-1曝光参数满足单个测量周期内包含对应不同曝光时间的子测量周期;(2)第j-1曝光参数指示单个测量周期内均采用相同曝光时间,即第j-1曝光参数不满足单个测量周期内包含对应不同曝光时间的子测量周期。可选的,步骤701至步骤703也可以在步骤706之后执行,该步骤执行顺序下,第j-1曝光参数满足单个测量周期内包含对应不同曝光时间的子测量周期,而第j曝光参数可能存在两种情况:(1)第j曝光参数也满足单个测量周期内包含对应不同曝光时间的子测量周期;(2)第j曝光参数指示单个测量周期内均采用相同曝光时间,即第j曝光参数不满足单个测量周期内包含对应不同曝光时间的子测量周期。
本实施例中,通过分析采集前一帧深度图像帧时,每个像素点接收总光子数的情况,以便确定采用前一帧曝光参数采集深度图像时,是否会出现近距离光强较强,或近距离光强较弱的情况,来确定是否需要对前一帧曝光参数进行调节,以及如何调节,从而逐帧将曝光方式调整至该应用场景下对应的较佳曝光方式。
需要说明的是,本申请实施例在调整曝光参数过程中,并不会增加或减少发射端发射激光信号的发射周期数,也就是说,在曝光参数调整过程中,单帧深度图像帧对应测量周期中所包含的发射周期总数为固定值;而本申请实施例本身是对测量周期进行划分,划分出至少两段子测量周期,每个子测量周期对应不同曝光时间,在调整曝光参数过程中,主要基于上一帧采集到的曝光结果数据调整不同子测量周期中所包含的发射周期数,示意性的,初始曝光参数为:m
1(对应曝光时间T/2)包括400个发射周期,m
2(对应曝光时间为T)包括800个发射周期,且发射端所发射激光信号的发射周期总数为1200个;调整后的曝光参数为:m
1(对应曝光时间T/2)包括700个发射周期,m
2(对应曝光时间为T)包括500个发射周期,可见,仅对不同曝光时间对应子测量周期中所包含的发射周期数进行调整,而发射端发射的发射周期总数仍然为1200个。
请参考图8,其示出了本申请一个示例性实施例示出的曝光方式的调整流程图。
步骤801,初始设置测量周期内的曝光时间均为T。
也就是说,测量周期中仅包含单个子测量周期,m
1=N,测量周期采用相同曝光时间,均为T,且T为最大曝光时间。
步骤802,设置各个子测量周期中包含的脉冲周期数至接收端。
其中,接收端用于基于各个子测量周期对应的曝光时间确定接收光信号的时机。
示意性的,初次运行时曝光时间为T,即测量周期中包含单个子测量周期,且该子测量周期对应的脉冲周期数为N。
示意性的,若后续修改曝光方式后,测量周期被划分为三段,分别为m
1=T、m
2=T/2、m
3=T/4,需要分别设置不同子测量周期中所包含的脉冲周期数,使得接收端可以在特定子测量周期处采用特定的曝光时间。
步骤803,在第i个子测量周期内,通过发射端发射激光信号,且接收端根据第i子测量周期对应的曝光时间接收光信号。
步骤804,响应于测量周期内各个子测量周期结束,基于接收到的光信号生成当前场景数据。
其中,当前场景数据为当前曝光参数下各个像素点对应的光子数分布直方图,也即上文实施例中的曝光结果数据。不仅可以基于该当前场景数据生成当前曝光参数下采集到的深度图像帧,同时可以基于当前场景数据判断是否需要调节曝光方式。
步骤805,基于当前场景数据,生成深度图像帧。
基于各个像素点对应的光子数分布直方图,确定出各个像素点对应的飞行时间(由光子数分布直方图对应的直方图峰值确定),从而基于该飞行时间确定出各个像素点对应的深度距离,进而生成深度图像帧。
步骤806,基于当前场景数据,确定近距离接收到总光子数是否过多。
基于采集到的当前场景数据,判断近距离接收到的光子数是否过多,若过多,为了避免过曝或堆积效应,进入步骤807,调节曝光方式,否则无需调节曝光方式。
步骤807,减少接收端曝光时间为T的脉冲周期数,增加曝光时间为后T/2的脉冲周期数,和/或增加曝光时间为后T/4的脉冲周期数。
当确定出近距离接收到的光子数较多,表示需要减少近距离接收到的光子数,可以通过减少曝光时间为T的脉冲周期数实现,由于脉冲周期总数一定,还需要对应增加曝光时间为T/2和/或T/4的脉冲周期数。
当调节完曝光方式后,可以进入步骤802,基于调节后的曝光参数重新设置各个子测量周期中包含的脉冲周期数到接收端,使得基于调节后的曝光参数重新采集场景数据。
下述为本申请装置实施例,可以用于执行本申请方法实施例。对于本申请装置实施例中未披露的细节,请参照本申请方法实施例。
请参考图9,其示出了本申请一个实施例提供的dTOF深度图像的采集装置的结构框图。该装置具有实现上述方法实施例中由电子设备执行的功能,所述功能可以由硬件实现,也可以由硬件执行相应的软件实现。如图9所示,该装置可以包括:
发射模块901,用于在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数;
接收模块902,用于根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,所述测量周期中包含对应不同曝光时间的子测量周期,所述光信号中包含所述激光信号的反射光信号;
生成模块903,用于在所述测量周期内各个所述子测量周期结束的情况下,基于接收到的所述光信号生成所述深度图像帧。
可选的,所述接收模块902,还用于:
基于所述第i个子测量周期对应的所述曝光时间,确定所述第i个子测量周期对应的曝光延迟时长;
在所述激光信号的发射时刻与当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
可选的,所述接收模块902,还用于:
在所述曝光时间为最大曝光时间的情况下,在所述发射时刻通过所述接收端接收所述光信号,所述最大曝光时间由所述激光信号的发射周期确定;
在所述曝光时间小于所述最大曝光时间,且所述发射时刻与所述当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
可选的,所述测量周期中包含N个发射周期,N为大于1的整数,所述发射周期为相邻激光信号之间的发射时间差值;
其中,不同子测量周期中包含相同发射周期数,或,不同子测量周期中包含不同发射周期数。
可选的,存在至少一段所述子测量周期对应的所述曝光时间为最大曝光时间,所述最大曝光时间由所述激光信号的发射周期确定。
可选的,所述装置还包括:
获取模块,用于获取用于生成第j-1帧深度图像帧的第j-1曝光结果数据,所述第j-1曝光结果数据包括各个像素点在所述测量周期中接收到的总光子数,其中,j为大于1的整数;
调节模块,用于在所述第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,生成第j曝光参数,所述第j-1曝光参数为采集所述第j-1深度图像帧时所使用的曝光参数,所述第j-1曝光参数包括:所述测量周期被划分的k个子测量周期以及各个子测量周期对应的所述曝光时间,所述预设调节方式用于改变所述测量周期的划分方式,k为大于1的整数;
发送模块,用于基于所述第j曝光参数向所述接收端发送曝光参数更新指令,所述接收端用于基于所述第j曝光参数接收所述光信号。
可选的,所述调节模块,还用于:
在所述第j-1曝光结果数据满足第一预设条件的情况下,按照第一预设调节方式对所述第j-1曝光参数进行调节,所述第一预设调节方式用于减少第一曝光时间对应子测量周期中所包含的发射周期数,并增加第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间;
和/或,
在所述第j-1曝光结果数据满足第二预设条件的情况下,按照第二预设调节方式对所述第j-1曝光参数进行调节,所述第二预设调节方式用于增加第一曝光时间对应子测量周期中所包含的发射周期数,并减少第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间。
可选的,所述第一预设条件包括:单个像素点在所述测量周期中接收到的总光子数高于第一预设光子数阈值、所述总光子数高于所述第一预设光子数阈值的像素点数量大于第一预设数量阈值,以及所述总光子数高于所述第一预设光子数阈值的像素点所占比例高于第一预设比例阈值中的至少一种;
所述第二预设条件包括:单个像素点在所述测量周期中接收到的总光子数低于第二预设光子数阈值、所述总光子数低于所述第二预设光子数阈值的像素点数量大于第二预设数量阈值,以及所述总光子数低于所述第二预设光子数阈值的像素点所占比例高于第二预设比例阈值中的至少一种,其中,所述第二预设光子数阈值小于等于所述第一预设光子数阈值。
可选的,所述调节模块,还用于:
基于所述总光子数和所述第一预设光子数阈值之间的第一数量差值,确定所述第二曝光时间对应发射周期数的第一增长量,以及确定所述第一曝光时间对应发射周期数的第一减少量,所述第一增长量与所述第一数量差值呈正相关关系,且所述第一减少量与所述第一数量差值呈正相关关系;
基于所述第一增长量和所述第一减少量,对所述第j-1曝光参数进行调节。
可选的,所述调节模块,还用于:
基于所述总光子数和所述第二预设光子数阈值之间的第二数量差值,确定所述第一曝光时间对应发射周期数的第二增长量,以及确定所述第二曝光时间对应发射周期数的第二减少量,所述第二增长量与所述第二数量差值呈正相关关系,所述第二减少量与所述第二数量差值呈正相关关系;
基于所述第二增长量和所述第二减少量,对所述第j-1曝光参数进行调节。
综上所述,本申请实施例中,在深度图像的获取场景中,通过将单帧深度图像帧对应深度信息的测量周期划分为不同子测量周期,进而为不同子测量周期设置不同曝光时间,使得可以通过调整不同子测量周期的划分,或调整不同子测量周期对应的曝光时间,进而调整在同一测量周期内,远距离目标的总曝光时长和近距离目标的总曝光时长,使得近距离物体可以对应较短的总曝光时长,防止过曝和堆积效应,同时远距离物体相对于近距离物体对应有较长的总曝光时长,在防止过曝和堆积效应的同时保证了远距离目标的测量精度,从而可以兼顾同一视场内更远和更近物体的测量精度,进一步可以增加同时探测视场内远、近距离目标的距离差,从而提高了深度图像采集设备的距离动态范围。
需要说明的是:上述实施例提供的dTOF深度图像的采集装置在实现其功能时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将设备的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的dTOF深度图像的采集装置与dTOF深度图像的采集方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。
请参考图10,其示出了本申请一个示例性实施例提供的电子设备的结构方框图。该电子设备1000可以是具有深度图像采集功能的设备,其可以是智能手机、平板电脑、智能电视、数码相机等,本实施例对电子设备不构成限定。本申请中的电子设备1000可以包括一个或多个如下部件:处理器1001、存储器1002和图像采集组件1003。
存储器1002可以包括随机存储器(Random Access Memory,RAM),也可以包括只读存储器(Read-Only Memory,ROM)。可选地,该存储器1002包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器1002可用于存储指令、程序、代码、代码集或指令集。存储器1002可包括存储程序区和存储数据区,其中,存储程序区可存储用于实现操作***的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现上述各个方法实施例的指令等;存储数据区还可以存储电子设备1000在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据、图像数据)等。
处理器1001可以包括一个或者多个处理核心。处理器1001利用各种接口和线路连接整个电子设备1000内的各个部分,通过运行或执行存储在存储器1002内的指令、程序、代码集或指令集,以及调用存储在存储器1002内的数据,执行电子设备1000的各种功能和处理数据。可选地,处理器1001可以采用DSP、现场可编程门阵列(Field-Programmable Gate Array,FPGA)、可编程逻辑阵列(Programmable Logic Array,PLA)中的至少一种硬件形式来实现。处理器1004可集成CPU、图像处理器(Graphics Processing Unit,GPU)和调制解调器等中的一种或几种的组合。
图像采集组件1003用于实现dTOF深度图像采集功能。该图像采集组件1003可以包括发射端、接收端,其中,发射端用于发射激光信号,接收端用于接收光信号。
可选的,电子设备1000中还可以包括触控显示屏,该触控显示屏可以为电容式触控显示屏,该电容式触控显示屏用于接收用户使用手指、触摸笔等任何适合的物体在其上或附近的触摸操作,以及显示各个应用程序的用户界面。触控显示屏通常设置在电子设备1000的前面板。触控显示屏可被设计成为全面屏、曲面屏或异型屏。触控显示屏还可被设计成为全面屏与曲面屏的结合,异型屏与曲面屏的结合,本申请实施例对此不加以限定。
除此之外,本领域技术人员可以理解,上述附图所示出的电子设备1000的结构并不构成对电子设备1000的限定,电子设备可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。比如,电子设备1000中还包括传感器、射频电路、音频电路、无线保真(Wireless Fidelity,WiFi)组件、电源、蓝牙组件等部件,在此不再赘述。
本申请实施例还提供了一种计算机可读存储介质,该计算机可读存储介质存储有至少一条程序代码,所述程序代码由处理器加载并执行以实现如上各个实施例所述的dTOF深度图像的采集方法。
根据本申请的一个方面,提供了一种计算机程序产品,该计算机程序产品包括计算机指令,该计算机指令存储在计算机可读存储介质中。电子设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该电子设备执行上述方面的各种可选实现方式中提供的dTOF深度图像的采集方法。
应当理解的是,在本文中提及的“多个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。另外,本文中描述的步骤编号,仅示例性示出了步骤间的一种可能的执行先后顺序,在一些其它实施例中,上述步骤也可以不按照编号顺序来执行,如两个不同编号的步骤同时执行,或者两个不同编号的步骤按照与图示相反的顺序执行,本申请实施例对此不作限定。
以上所述仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。
Claims (23)
- 一种dTOF深度图像的采集方法,所述方法包括:在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数;根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,所述测量周期中包含对应不同曝光时间的子测量周期,所述光信号中包含所述激光信号的反射光信号;在所述测量周期内各个所述子测量周期结束的情况下,基于接收到的所述光信号生成所述深度图像帧。
- 根据权利要求1所述的方法,其中,所述根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,包括:基于所述第i个子测量周期对应的所述曝光时间,确定所述第i个子测量周期对应的曝光延迟时长;在所述激光信号的发射时刻与当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
- 根据权利要求2所述的方法,其中,所述根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,包括:在所述曝光时间为最大曝光时间的情况下,在所述发射时刻通过所述接收端接收所述光信号,所述最大曝光时间由所述激光信号的发射周期确定;在所述曝光时间小于所述最大曝光时间,且所述发射时刻与所述当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
- 根据权利要求1至3任一所述的方法,其中,所述测量周期中包含N个发射周期,N为大于1的整数,所述发射周期为相邻激光信号之间的发射时间差值;其中,不同子测量周期中包含相同发射周期数,或,不同子测量周期中包含不同发射周期数。
- 根据权利要求1至3任一所述的方法,其中,存在至少一段所述子测量周期对应的所述曝光时间为最大曝光时间,所述最大曝光时间由所述激光信号的发射周期确定。
- 根据权利要求1至3任一所述的方法,其中,所述方法还包括:获取用于生成第j-1帧深度图像帧的第j-1曝光结果数据,所述第j-1曝光结果数据包括各个像素点在所述测量周期中接收到的总光子数,其中,j为大于1的整数;在所述第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,生成第j曝光参数,所述第j-1曝光参数为采集所述第j-1深度图像帧时所使用的曝光参数,所述第j-1曝光参数包括:所述测量周期被划分的k个子测量周期以及各个子测量周期对应的所述曝光时间,所述预设调节方式用于改变所述测量周期的划分方式,k为大于1的整数;基于所述第j曝光参数向所述接收端发送曝光参数更新指令,所述接收端用于基于所述第j曝光参数接收所述光信号。
- 根据权利要求6所述的方法,其中,所述在所述第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,包括:在所述第j-1曝光结果数据满足第一预设条件的情况下,按照第一预设调节方式对所述第j-1曝光参数进行调节,所述第一预设调节方式用于减少第一曝光时间对应子测量周期中所包含的发射周期数,并增加第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间;和/或,在所述第j-1曝光结果数据满足第二预设条件的情况下,按照第二预设调节方式对所述第j-1曝光参数进行调节,所述第二预设调节方式用于增加第一曝光时间对应子测量周期中所包含的发射周期数,并减少第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间。
- 根据权利要求7所述的方法,其中,所述第一预设条件包括:单个像素点在所述测量周期中接收到的总光子数高于第一预设光子数阈值、 所述总光子数高于所述第一预设光子数阈值的像素点数量大于第一预设数量阈值,以及所述总光子数高于所述第一预设光子数阈值的像素点所占比例高于第一预设比例阈值中的至少一种;所述第二预设条件包括:单个像素点在所述测量周期中接收到的总光子数低于第二预设光子数阈值、所述总光子数低于所述第二预设光子数阈值的像素点数量大于第二预设数量阈值,以及所述总光子数低于所述第二预设光子数阈值的像素点所占比例高于第二预设比例阈值中的至少一种,其中,所述第二预设光子数阈值小于等于所述第一预设光子数阈值。
- 根据权利要求8所述的方法,其中,所述按照第一预设调节方式对所述第j-1曝光参数进行调节,包括:基于所述总光子数和所述第一预设光子数阈值之间的第一数量差值,确定所述第二曝光时间对应发射周期数的第一增长量,以及确定所述第一曝光时间对应发射周期数的第一减少量,所述第一增长量与所述第一数量差值呈正相关关系,且所述第一减少量与所述第一数量差值呈正相关关系;基于所述第一增长量和所述第一减少量,对所述第j-1曝光参数进行调节。
- 根据权利要求8所述的方法,其中,所述按照第二预设调节方式对所述第j-1曝光参数进行调节,包括:基于所述总光子数和所述第二预设光子数阈值之间的第二数量差值,确定所述第一曝光时间对应发射周期数的第二增长量,以及确定所述第二曝光时间对应发射周期数的第二减少量,所述第二增长量与所述第二数量差值呈正相关关系,所述第二减少量与所述第二数量差值呈正相关关系;基于所述第二增长量和所述第二减少量,对所述第j-1曝光参数进行调节。
- 一种dTOF深度图像的采集装置,所述装置包括:发射模块,用于在第i个子测量周期内,通过发射端发射激光信号,其中,单帧深度图像帧对应深度信息的测量周期被划分为n个子测量周期,n为大于1的整数,i为小于等于n的正整数;接收模块,用于根据所述第i个子测量周期对应的曝光时间,通过接收端接收光信号,所述测量周期中包含对应不同曝光时间的子测量周期,所述光信号中包含所述激光信号的反射光信号;生成模块,用于在所述测量周期内各个所述子测量周期结束的情况下,基于接收到的所述光信号生成所述深度图像帧。
- 根据权利要求11所述的装置,其中,所述接收模块,还用于:基于所述第i个子测量周期对应的所述曝光时间,确定所述第i个子测量周期对应的曝光延迟时长;在所述激光信号的发射时刻与当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
- 根据权利要求12所述的装置,其中,所述接收模块,还用于:在所述曝光时间为最大曝光时间的情况下,在所述发射时刻通过所述接收端接收所述光信号,所述最大曝光时间由所述激光信号的发射周期确定;在所述曝光时间小于所述最大曝光时间,且所述发射时刻与所述当前接收时刻之间的时间差达到所述曝光延迟时长的情况下,通过所述接收端接收所述光信号。
- 根据权利要求11至13任一所述的装置,其中,所述测量周期中包含N个发射周期,N为大于1的整数,所述发射周期为相邻激光信号之间的发射时间差值;其中,不同子测量周期中包含相同发射周期数,或,不同子测量周期中包含不同发射周期数。
- 根据权利要求11至13任一所述的装置,其中,存在至少一段所述子测量周期对应的所述曝光时间为最大曝光时间,所述最大曝光时间由所述激光信号的发射周期确定。
- 根据权利要求11至13任一所述的装置,其中,所述装置还包括:获取模块,用于获取用于生成第j-1帧深度图像帧的第j-1曝光结果数据,所述第j-1曝光结果数据包括各个像素点在所述测量周期中接收到的总光子数,其中,j为大于1的整数;调节模块,用于在所述第j-1曝光结果数据满足预设条件的情况下,按照预设调节方式对第j-1曝光参数进行调节,生成第j曝光参数,所述第j-1曝光参数为采集所述第j-1深度图像帧时所使用的曝光参数,所述第j-1曝光参数包括:所述测量周期被划分的k个子测量周期以及各个子测量周期对应的所述曝光时 间,所述预设调节方式用于改变所述测量周期的划分方式,k为大于1的整数;发送模块,用于基于所述第j曝光参数向所述接收端发送曝光参数更新指令,所述接收端用于基于所述第j曝光参数接收所述光信号。
- 根据权利要求16所述的装置,其中,所述调节模块,还用于:在所述第j-1曝光结果数据满足第一预设条件的情况下,按照第一预设调节方式对所述第j-1曝光参数进行调节,所述第一预设调节方式用于减少第一曝光时间对应子测量周期中所包含的发射周期数,并增加第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间;和/或,在所述第j-1曝光结果数据满足第二预设条件的情况下,按照第二预设调节方式对所述第j-1曝光参数进行调节,所述第二预设调节方式用于增加第一曝光时间对应子测量周期中所包含的发射周期数,并减少第二曝光时间对应子测量周期中所包含的发射周期数,所述第二曝光时间小于所述第一曝光时间。
- 根据权利要求17所述的装置,其中,所述第一预设条件包括:单个像素点在所述测量周期中接收到的总光子数高于第一预设光子数阈值、所述总光子数高于所述第一预设光子数阈值的像素点数量大于第一预设数量阈值,以及所述总光子数高于所述第一预设光子数阈值的像素点所占比例高于第一预设比例阈值中的至少一种;所述第二预设条件包括:单个像素点在所述测量周期中接收到的总光子数低于第二预设光子数阈值、所述总光子数低于所述第二预设光子数阈值的像素点数量大于第二预设数量阈值,以及所述总光子数低于所述第二预设光子数阈值的像素点所占比例高于第二预设比例阈值中的至少一种,其中,所述第二预设光子数阈值小于等于所述第一预设光子数阈值。
- 根据权利要求18所述的装置,其中,所述调节模块,还用于:基于所述总光子数和所述第一预设光子数阈值之间的第一数量差值,确定所述第二曝光时间对应发射周期数的第一增长量,以及确定所述第一曝光时间对应发射周期数的第一减少量,所述第一增长量与所述第一数量差值呈正相关关系,且所述第一减少量与所述第一数量差值呈正相关关系;基于所述第一增长量和所述第一减少量,对所述第j-1曝光参数进行调节。
- 根据权利要求18所述的装置,其中,所述调节模块,还用于:基于所述总光子数和所述第二预设光子数阈值之间的第二数量差值,确定所述第一曝光时间对应发射周期数的第二增长量,以及确定所述第二曝光时间对应发射周期数的第二减少量,所述第二增长量与所述第二数量差值呈正相关关系,所述第二减少量与所述第二数量差值呈正相关关系;基于所述第二增长量和所述第二减少量,对所述第j-1曝光参数进行调节。
- 一种电子设备,所述电子设备包括处理器和存储器,所述存储器中存储有至少一条程序代码,所述程序代码由所述处理器加载并执行以实现如权利要求1至10任一所述的dTOF深度图像的采集方法。
- 一种计算机可读存储介质,所述计算机可读存储介质中存储有至少一条程序代码,所述程序代码由处理器加载并执行以实现如权利要求1至10任一所述的dTOF深度图像的采集方法。
- 一种计算机程序产品,所述计算机程序产品包括计算机指令,所述计算机指令存储在计算机可读存储介质中,处理器从所述计算机可读存储介质读取并执行所述计算机指令,以实现如权利要求1至10任一所述的dTOF深度图像的采集方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110557552.2A CN113296114B (zh) | 2021-05-21 | 2021-05-21 | dTOF深度图像的采集方法、装置、电子设备及介质 |
CN202110557552.2 | 2021-05-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022242348A1 true WO2022242348A1 (zh) | 2022-11-24 |
Family
ID=77323553
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/085379 WO2022242348A1 (zh) | 2021-05-21 | 2022-04-06 | dTOF深度图像的采集方法、装置、电子设备及介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113296114B (zh) |
WO (1) | WO2022242348A1 (zh) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113296114B (zh) * | 2021-05-21 | 2024-05-14 | Oppo广东移动通信有限公司 | dTOF深度图像的采集方法、装置、电子设备及介质 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160178734A1 (en) * | 2013-06-27 | 2016-06-23 | Panasonic Intellectual Property Management Co., Ltd. | Distance measuring device and solid-state image sensor |
CN105899966A (zh) * | 2014-01-14 | 2016-08-24 | 松下知识产权经营株式会社 | 距离图像生成装置以及距离图像生成方法 |
CN107710015A (zh) * | 2015-07-03 | 2018-02-16 | 松下知识产权经营株式会社 | 距离测量装置以及距离图像合成方法 |
CN109073754A (zh) * | 2016-04-19 | 2018-12-21 | 日立乐金光科技株式会社 | 距离图像生成装置和距离图像生成方法 |
CN110023785A (zh) * | 2016-11-29 | 2019-07-16 | 松下知识产权经营株式会社 | 距离测定装置 |
CN113296114A (zh) * | 2021-05-21 | 2021-08-24 | Oppo广东移动通信有限公司 | dTOF深度图像的采集方法、装置、电子设备及介质 |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104881881B (zh) * | 2014-02-27 | 2018-04-10 | 株式会社理光 | 运动对象表示方法及其装置 |
CN106683130B (zh) * | 2015-11-11 | 2020-04-10 | 杭州海康威视数字技术股份有限公司 | 一种深度图像获得方法及装置 |
WO2019023868A1 (zh) * | 2017-07-31 | 2019-02-07 | 深圳市大疆创新科技有限公司 | 一种图像拍摄的控制方法、控制装置及控制*** |
-
2021
- 2021-05-21 CN CN202110557552.2A patent/CN113296114B/zh active Active
-
2022
- 2022-04-06 WO PCT/CN2022/085379 patent/WO2022242348A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160178734A1 (en) * | 2013-06-27 | 2016-06-23 | Panasonic Intellectual Property Management Co., Ltd. | Distance measuring device and solid-state image sensor |
CN105899966A (zh) * | 2014-01-14 | 2016-08-24 | 松下知识产权经营株式会社 | 距离图像生成装置以及距离图像生成方法 |
CN107710015A (zh) * | 2015-07-03 | 2018-02-16 | 松下知识产权经营株式会社 | 距离测量装置以及距离图像合成方法 |
CN109073754A (zh) * | 2016-04-19 | 2018-12-21 | 日立乐金光科技株式会社 | 距离图像生成装置和距离图像生成方法 |
CN110023785A (zh) * | 2016-11-29 | 2019-07-16 | 松下知识产权经营株式会社 | 距离测定装置 |
CN113296114A (zh) * | 2021-05-21 | 2021-08-24 | Oppo广东移动通信有限公司 | dTOF深度图像的采集方法、装置、电子设备及介质 |
Also Published As
Publication number | Publication date |
---|---|
CN113296114A (zh) | 2021-08-24 |
CN113296114B (zh) | 2024-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105190426B (zh) | 飞行时间传感器装仓 | |
US11187804B2 (en) | Time of flight range finder for a structured light system | |
CN108427109B (zh) | 确定与对象的距离的设备和对应方法 | |
TWI723413B (zh) | 測量一成像感測器與一物體間之一距離的系統及方法 | |
CN109804271B (zh) | 功率控制方法、测距模块及电子装置 | |
WO2022242348A1 (zh) | dTOF深度图像的采集方法、装置、电子设备及介质 | |
CN112596069A (zh) | 距离测量方法及***、计算机可读介质和电子设备 | |
US11561291B2 (en) | High pulse repetition frequency lidar | |
US10884101B2 (en) | Crosstalk detection and compensation | |
WO2022110947A1 (zh) | 电子装置的控制方法、电子装置及计算机可读存储介质 | |
CN116413730B (zh) | 测距方法、装置、存储介质及激光雷达 | |
CN112954230B (zh) | 深度测量方法、芯片和电子设备 | |
US20220206146A1 (en) | A method of measuring a distance from a target | |
CN112255635A (zh) | 一种距离测量方法、***及设备 | |
WO2022160622A1 (zh) | 一种距离测量方法、装置及*** | |
CN115792936A (zh) | 一种DToF测距方法、***及装置 | |
CN114814881A (zh) | 一种激光测距方法及激光测距芯片 | |
WO2022001228A1 (zh) | 一种激光雷达***及探测方法 | |
US20230019246A1 (en) | Time-of-flight imaging circuitry, time-of-flight imaging system, and time-of-flight imaging method | |
CN109946709B (zh) | 图像传感器、终端的景深测量方法及终端 | |
CN112987022A (zh) | 测距方法及装置、计算机可读介质和电子设备 | |
EP4407347A1 (en) | Detection method and apparatus | |
KR102604467B1 (ko) | 펄스 레이저 기반의 거리 측정 장치 및 방법과 이를 구비하는 차량 | |
CN116449337B (zh) | 像素电路、光电传感器、ToF装置及电子设备 | |
US20230221419A1 (en) | Lidar adaptive single-pass histogramming for low power lidar system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22803673 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22803673 Country of ref document: EP Kind code of ref document: A1 |