WO2019041121A1 - Exposure method and apparatus, computer system, and movable device - Google Patents

Exposure method and apparatus, computer system, and movable device Download PDF

Info

Publication number
WO2019041121A1
WO2019041121A1 PCT/CN2017/099419 CN2017099419W WO2019041121A1 WO 2019041121 A1 WO2019041121 A1 WO 2019041121A1 CN 2017099419 W CN2017099419 W CN 2017099419W WO 2019041121 A1 WO2019041121 A1 WO 2019041121A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
exposure time
determining
average brightness
weight
Prior art date
Application number
PCT/CN2017/099419
Other languages
French (fr)
Chinese (zh)
Inventor
李琛
李庆
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2017/099419 priority Critical patent/WO2019041121A1/en
Priority to CN201780004678.9A priority patent/CN108521863B/en
Publication of WO2019041121A1 publication Critical patent/WO2019041121A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time

Definitions

  • the present invention relates to the field of information technology and, more particularly, to a method, apparatus, computer system and mobile device for exposure.
  • Auto-exposure adjusts the exposure time and gain for different light illuminance and the difference in the intensity of the object in the scene, so that the brightness of the target object in the image is always within a certain range.
  • Embodiments of the present invention provide a method, an apparatus, a computer system, and a movable device for exposure, which can effectively adjust an exposure time.
  • a method for exposing includes: determining an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel; and according to an average brightness of the first image, the first The exposure time corresponding to the image and the desired average brightness of the second image determine the exposure time corresponding to the second image.
  • a method for exposing includes: determining exposure times of a plurality of vision sensors having dependencies, respectively; jointly optimizing exposure times of the plurality of vision sensors to enable the plurality of visions The difference between the optimized exposure times of the sensor is minimal.
  • an apparatus for exposing includes: an average brightness determination module for Determining, according to the brightness of the pixel of the first image and the brightness weight of the pixel, an average brightness of the first image; an exposure time determining module, configured to: according to an average brightness of the first image, an exposure time corresponding to the first image The expected average brightness of the two images determines the exposure time corresponding to the second image.
  • a fourth aspect provides an apparatus for exposing, comprising: a determining module for respectively determining exposure times of a plurality of vision sensors having a dependency; a joint optimization module for separately determined exposure of the plurality of vision sensors The time is jointly optimized to minimize the difference between the optimized exposure times of the plurality of vision sensors.
  • a computer system comprising: a memory for storing computer executable instructions; a processor for accessing the memory, and executing the computer executable instructions to perform the first or second The operation in the aspect of the method.
  • a mobile device comprising: the apparatus for exposing the third or fourth aspect; or the computer system of the above fifth aspect.
  • a computer storage medium having stored therein program code, the program code being operative to indicate a method of performing the first or second aspect described above.
  • the average brightness of the image is determined based on the brightness weight, and the exposure time is adjusted according to the average brightness, so that the obtained exposure time is associated with the importance of the pixel, so that the exposure time can be effectively adjusted.
  • FIG. 1 is a schematic structural diagram of a mobile device according to an embodiment of the present invention.
  • FIG. 2 is a schematic flow chart of a method of exposure according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of image area division according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of image overexposure according to an embodiment of the present invention.
  • Figure 5 is a schematic flow diagram of a method of exposure in accordance with another embodiment of the present invention.
  • Figure 6 is a schematic block diagram of an apparatus for exposure of one embodiment of the present invention.
  • Figure 7 is a schematic block diagram of an apparatus for exposure of another embodiment of the present invention.
  • Figure 8 is a schematic block diagram of an apparatus for exposure of still another embodiment of the present invention.
  • FIG. 9 is a schematic block diagram of a computer system in accordance with an embodiment of the present invention.
  • the size of the sequence numbers of the processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be construed as an embodiment of the present invention.
  • the implementation process constitutes any limitation.
  • the technical solution of the embodiments of the present invention can be applied to various image acquisition systems, and various devices that rely on an image or image acquisition system, such as a mobile device.
  • the removable device can also be referred to as a mobile device.
  • the mobile device may be a drone, an unmanned boat, an autonomous vehicle or a robot, etc., but the embodiment of the present invention is not limited thereto.
  • FIG. 1 is a schematic architectural diagram of a removable device 100 according to an embodiment of the present invention.
  • the mobile device 100 can include a power system 110, a control system 120, a sensing system 130, and a processing system 140.
  • Power system 110 is used to power the mobile device 100.
  • the power system of the drone may include an electronic governor (referred to as an electric current), a propeller, and a motor corresponding to the propeller.
  • the motor is connected between the electronic governor and the propeller, and the motor and the propeller are disposed on the corresponding arm; the electronic governor is used for receiving the driving signal generated by the control system, and providing driving current to the motor according to the driving signal to control the motor Rotating speed.
  • the motor is used to drive the propeller to rotate to power the drone's flight.
  • the sensing system 130 can be used to measure attitude information of the mobile device 100, that is, position information and state information of the mobile device 100 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity.
  • the sensing system 130 may include, for example, at least one of a gyroscope, an electronic compass, an Inertial Measurement Unit (IMU), a vision sensor, a Global Positioning System (GPS), a barometer, an airspeed meter, and the like.
  • IMU Inertial Measurement Unit
  • GPS Global Positioning System
  • barometer an airspeed meter
  • Sensing system 130 can also be used to acquire images, i.e., sensing system 130 includes sensors for acquiring images, such as cameras and the like.
  • Control system 120 is used to control the movement of mobile device 100.
  • Control system 120 can be pressed
  • the removable device 100 is controlled in accordance with a program instruction set in advance.
  • the control system 120 can control the movement of the mobile device 100 based on the attitude information of the mobile device 100 measured by the sensing system 130.
  • Control system 120 can also control mobile device 100 based on control signals from the remote control.
  • the control system 120 can be a flight control system (flying control) or a control circuit in a flight control.
  • Processing system 140 can process the images acquired by sensing system 130.
  • processing system 140 can be an Image Signal Processing (ISP) type of chip.
  • ISP Image Signal Processing
  • processing system 140 may determine an exposure time to acquire the next image based on images that sensor system 130 has acquired. For example, the processing system 140 can determine the exposure time using the technical solution of the embodiments of the present invention.
  • processing system 140 can be integrated with sensing system 130.
  • processing system 140 can be integrated into a corresponding visual sensor.
  • removable device 100 may also include other components not shown in FIG. 1, which are not limited by the embodiments of the present invention.
  • FIG. 2 shows a schematic flow diagram of a method 200 of exposure in accordance with one embodiment of the present invention.
  • the method 200 can be performed by the mobile device 100 shown in FIG. In particular, it can be performed by processing system 140 in FIG.
  • the luminance weight of the pixel is considered in determining the average luminance of the image. That is to say, for each pixel of the image, when calculating the average brightness of the image, the weight of the pixel is considered, that is, the more important the pixel is, the greater the weight.
  • the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
  • the average brightness of an image can be calculated according to the following formula (1)
  • the brightness weight of the pixel may include at least one of a region weight and a chrominance weight The item, wherein the area weight is a weight associated with the area of the pixel, the chromaticity weight being a weight associated with the chrominance of the pixel.
  • the luminance weight ⁇ can be expressed as:
  • ⁇ r is the regional weight and ⁇ c is the chromaticity weight.
  • the brightness weight in the embodiment of the present invention may include only the area weight, or only the chrominance weight, or both the area weight and the chrominance weight, or may also include other weights, which is not limited by the present invention.
  • the region weight if the region of the pixel is closer to the region of the target object, the region weight is larger.
  • the importance of different regions may be different for a particular scenario.
  • different area weights can be set according to the importance of the area. For example, if an area is closer to the area of the target object, the area weight is larger.
  • the images collected on the parallel road mainly include the traveling road, the target objects on both sides of the road, the sky and the distant target objects, among which the driving road surface (target object) is the most important. , followed by the distant target and the targets on both sides of the road, and again the rest of the image.
  • the driving road surface target object
  • the zoning strategy and weight assignment of the image may be determined according to the scene and the installation location of the camera. For example, in the forward-looking camera image, the area may be divided as shown in FIG. 3, and the corresponding weights of each area may be as shown in Table 1.
  • the chromaticity weight is larger.
  • the importance of different chrominances may also be different for a particular scenario.
  • the chromaticity can also be set to different chromaticity weights. For example, if a certain chromaticity is closer to the chromaticity of the target object, the chromaticity weight is larger.
  • the traveling road surface (target object) is mostly gray, and therefore, the chromaticity weight can be utilized to control the weights of the different color objects such that the closer the color is to the gray, the larger the weight.
  • the chromaticity weight ⁇ c can be determined according to the following formula (3):
  • the exposure time corresponding to the second image is determined according to the average brightness of the first image based on the brightness weight.
  • the first image is a captured image
  • the second image is an image to be captured.
  • the first image is the latest captured image. That is to say, the exposure time at which the next time is taken can be determined based on the average brightness based on the luminance weight of the image taken at the previous time.
  • an average brightness derivative may be determined according to an average brightness of the first image and an exposure time corresponding to the first image; and an average brightness of the first image and the average brightness derivative according to a desired average brightness of the second image Determining an exposure time adjustment amount; determining an exposure time corresponding to the second image according to the exposure time corresponding to the first image and the exposure time adjustment amount, optionally, according to an exposure time corresponding to the first image, The exposure time adjustment amount and the adjustment coefficient determine the exposure time corresponding to the second image.
  • the average brightness derivative It can be calculated by the following formula (4):
  • the average brightness function It is a monotonic auto-increment function with respect to the exposure time t. According to the 1st-order Taylor expansion formula, you can get:
  • ⁇ is the adjustment factor used to control the speed of the automatic exposure adjustment. For example, if the adjustment speed is too large, the adjustment speed can be slowed down by lowering the adjustment factor, and vice versa.
  • the concept of exposure time in the embodiment of the present invention can be understood as the total amount of exposure, and the control can be achieved by adjusting the shutter speed, the aperture size, or the gain of the camera, and Does not specifically refer to a certain component.
  • the average brightness of the image is determined based on the luminance weight, and the exposure time is adjusted according to the average brightness, so that the obtained exposure time is associated with the importance of the pixel, so that the exposure time can be effectively adjusted.
  • the desired average brightness of the second image may be preset. That is to say, the desired average brightness of the image to be captured can be set in advance, and accordingly the corresponding exposure time is obtained, thereby obtaining an image close to the desired average brightness. For example, in an 8-bit image, you can set
  • the desired average brightness of the second image can also be dynamically adjusted.
  • the desired average brightness of the second image may be adjusted according to a percentage of the overexposed pixels in the first image.
  • the difference in the reflective intensity of the road surface may reach several tens of times. If a fixed desired average brightness is applied, in the wooded areas on both sides of the road, the reflection of the trees on both sides of the road is reflected relative to the road surface. There may be a gap of more than ten times. At this time, the problem of over-exposure on the road surface is easy to occur, as shown in Figure 4. To do this, you can dynamically adjust the desired average brightness. To solve this problem. In the embodiment of the present invention, the desired average brightness can be adjusted by using the percentage of the image overexposed pixels.
  • ⁇ is the overexposed pixel in image I n (eg Percentage of For the desired overexposure ratio. For example, you can set It is 0.05.
  • the above technical solution of the embodiment of the present invention can be used to determine the exposure time of each visual sensor (camera).
  • Multiple image sensors with dependencies may be included in the image acquisition system.
  • binocular stereo vision requires the exposure time of the left and right eye cameras to be consistent, so as to facilitate matching of image feature points. If the exposure difference is large, it may cause the effect of binocular matching to be poor, which may affect the performance of the visual algorithm. If there is only one set of binocular vision system in the system, you can expose it.
  • the light time is set to be consistent (such as the average of the exposure times calculated for binocular independent exposure).
  • the separately determined exposure times of the plurality of vision sensors having dependencies may also be jointly optimized to minimize the difference between the optimized exposure times of the plurality of vision sensors.
  • the exposure time can be determined separately by the foregoing scheme, and then the exposure times of the plurality of vision sensors are jointly optimized.
  • determining an optimization matrix according to the configuration of the plurality of vision sensors determining optimization of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors After the exposure time.
  • b (t 1 , t 2 , ..., t n , 0, 0 ,...,0) T .
  • the above matrix can be calculated by the least squares method:
  • Matrix A relies on the configuration of multiple vision sensors, ie the specific configuration of the image acquisition system, so (A T A) -1 A T can be pre-computed without additional overhead at runtime.
  • the binocular weight coefficients are all 1 and the matrix A is:
  • the jointly optimized exposure time t' can be obtained by b and A.
  • the exposure time of multiple vision sensors can be optimized as a whole, thereby improving overall performance.
  • the technical solution for determining the exposure time and the joint optimization of the exposure time in the above embodiments of the present invention may be implemented jointly or separately.
  • the technical solution of the exposure time joint optimization may depend on the above-mentioned technical solution for determining the exposure time, or may not depend on the above-mentioned technical solution for determining the exposure time.
  • another embodiment of the present invention provides a method of exposure, which will be described below in conjunction with FIG. 5. It should be understood that some specific descriptions of the method shown in FIG. 5 may refer to the foregoing embodiments, and are not described herein again for brevity.
  • FIG. 5 shows a schematic flow diagram of a method 500 of exposure in accordance with another embodiment of the present invention. As shown in FIG. 5, the method 500 includes:
  • the exposure time may be determined by using the technical solution of the foregoing embodiment, and may be determined by other methods, which is not limited by the embodiment of the present invention.
  • the joint optimization technical solution of the foregoing embodiment may be adopted, and for brevity, no further details are provided herein. Through joint optimization, the exposure time of multiple vision sensors can be optimized as a whole, thereby improving overall performance.
  • an image with reasonable exposure can be obtained as an input of an upper computer vision algorithm such as a visual odometer, binocular stereo vision, image recognition and tracking, and the result can be ensured accurately. Sex and stability.
  • Figure 6 shows a schematic block diagram of an apparatus 600 for exposure of one embodiment of the present invention.
  • the apparatus 600 can include:
  • the average brightness determining module 610 is configured to determine an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel;
  • the exposure time determining module 620 is configured to determine an exposure time corresponding to the second image according to an average brightness of the first image, an exposure time corresponding to the first image, and a desired average brightness of the second image.
  • the brightness weight of the pixel includes at least one of a region weight and a chrominance weight, wherein the region weight is a weight associated with the region of the pixel, and the chrominance weight is The weight associated with the chrominance of the pixel.
  • the weight of the area is larger; and/or, if the chromaticity of the pixel is closer to the chromaticity of the target object, The chromaticity weight is greater.
  • the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
  • the desired average brightness of the second image is preset.
  • the apparatus 600 may further include:
  • the adjusting module 630 is configured to adjust a desired average brightness of the second image according to a percentage of the overexposed pixels in the first image.
  • the exposure time determining module 620 is configured to:
  • the exposure time determining module 620 is configured to:
  • the exposure time corresponding to the first image, the exposure time adjustment amount and the adjustment coefficient determine an exposure time corresponding to the second image.
  • the first image is a captured image
  • the image is The second image is an image to be taken.
  • the first image is the latest captured image.
  • the apparatus 600 may further include:
  • the joint optimization module 640 is configured to jointly optimize the respectively determined exposure times of the plurality of vision sensors having dependencies to minimize the difference between the optimized exposure times of the plurality of vision sensors.
  • the joint optimization module 640 is configured to:
  • An optimized exposure time of the plurality of vision sensors is determined based on the optimized matrix and the respective determined exposure times of the plurality of vision sensors.
  • FIG. 8 shows a schematic block diagram of an apparatus 800 for exposure of another embodiment of the present invention.
  • the apparatus 800 can include:
  • a determining module 810 configured to respectively determine an exposure time of the plurality of vision sensors having a dependency relationship
  • the joint optimization module 820 is configured to jointly optimize the respectively determined exposure times of the plurality of vision sensors to minimize the difference between the optimized exposure times of the plurality of vision sensors.
  • the joint optimization module 820 is configured to:
  • An optimized exposure time of the plurality of vision sensors is determined based on the optimized matrix and the respective determined exposure times of the plurality of vision sensors.
  • the apparatus for exposing the embodiment of the present invention may be a chip, which may be specifically implemented by a circuit, but the specific implementation form of the embodiment of the present invention is not limited.
  • Embodiments of the present invention also provide a processor that can include the apparatus for exposure of the various embodiments of the present invention described above.
  • FIG. 9 shows a schematic block diagram of a computer system 900 in accordance with an embodiment of the present invention.
  • the computer system 900 can include a processor 910 and a memory 920.
  • the computer system 900 may also include components that are generally included in other computer systems, such as input and output devices, communication interfaces, and the like, which are not limited by the embodiments of the present invention.
  • Memory 920 is for storing computer executable instructions.
  • the memory 920 may be various kinds of memories, for example, may include a high speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, which is implemented by the present invention. This example is not limited to this.
  • RAM high speed random access memory
  • non-volatile memory such as at least one disk memory
  • the processor 90 is operative to access the memory 920 and execute the computer executable instructions to perform the operations in the method of exposure of the various embodiments of the present invention described above.
  • the processor 910 may include a microprocessor, a Field-Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), etc., and is implemented by the present invention. This example is not limited to this.
  • Embodiments of the present invention also provide a removable device that can include an apparatus, processor, or computer system for exposure of various embodiments of the present invention described above.
  • the apparatus, computer system, and mobile device of the exposure of the embodiments of the present invention may correspond to an execution subject of the method of exposure of the embodiment of the present invention, and the above-described and other operations of the exposed device, the computer system, and each module in the movable device
  • the functions and/or functions are respectively omitted in order to implement the corresponding processes of the foregoing various methods, and are not described herein for brevity.
  • Embodiments of the present invention also provide a computer storage medium having stored therein program code, the program code being usable for indicating a method of performing the exposure of the embodiment of the present invention described above.
  • the term "and/or” is merely an association relationship describing an associated object, indicating that there may be three relationships.
  • a and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone.
  • the character "/" in this article generally indicates that the contextual object is an "or" relationship.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

Disclosed are an exposure method and apparatus, a computer system, and a movable device. The method comprises: determining, according to the luminance of a pixel of a first image and a luminance weight of the pixel, average luminance of the first image; determining, according to the average luminance of the first image, an exposure time corresponding to the first image, and desired average luminance of a second image, an exposure time corresponding to the second image. According to the technical solution in embodiments of the present invention, the exposure time can be effectively adjusted.

Description

曝光的方法、装置、计算机***和可移动设备Exposure method, device, computer system and mobile device
版权申明Copyright statement
本专利文件披露的内容包含受版权保护的材料。该版权为版权所有人所有。版权所有人不反对任何人复制专利与商标局的官方记录和档案中所存在的该专利文件或者该专利披露。The disclosure of this patent document contains material that is subject to copyright protection. This copyright is the property of the copyright holder. The copyright owner has no objection to the reproduction of the patent document or the patent disclosure in the official records and files of the Patent and Trademark Office.
技术领域Technical field
本发明涉及信息技术领域，并且更具体地，涉及一种曝光的方法、装置、计算机***和可移动设备。The present invention relates to the field of information technology and, more particularly, to a method, apparatus, computer system and mobile device for exposure.
背景技术Background technique
自动曝光是针对不同光线亮度和场景中物体反光强度的差异性对曝光时间和增益进行调节，以使目标物体在图像中亮度始终维持在一定范围内。Auto-exposure adjusts the exposure time and gain for different light illuminance and the difference in the intensity of the object in the scene, so that the brightness of the target object in the image is always within a certain range.
对于自动驾驶、辅助驾驶、户外作业机器人等***中多路相机组成的图像采集***，在光照强度动态变化的场景下实时调整相机曝光时间，可以得到曝光合理的图像，作为诸如视觉里程计、双目/多目立体视觉、图像识别与跟踪等上层计算机视觉算法的输入，能够保证其结果的准确性和稳定性。For image acquisition systems consisting of multi-channel cameras in systems such as autonomous driving, assisted driving, and outdoor working robots, real-time adjustment of camera exposure time in scenes with dynamically changing light intensity can result in images with reasonable exposure, such as visual odometers, dual The input of upper computer vision algorithms such as multi-dimensional stereo vision, image recognition and tracking can ensure the accuracy and stability of the results.
因此，如何有效地调整曝光时间，成为一个亟待解决的技术问题。Therefore, how to effectively adjust the exposure time becomes a technical problem to be solved urgently.
发明内容Summary of the invention
本发明实施例提供了一种曝光的方法、装置、计算机***和可移动设备，能够有效地调整曝光时间。Embodiments of the present invention provide a method, an apparatus, a computer system, and a movable device for exposure, which can effectively adjust an exposure time.
第一方面，提供了一种曝光的方法，包括：根据第一图像的像素的亮度和该像素的亮度权重，确定该第一图像的平均亮度；根据该第一图像的平均亮度，该第一图像对应的曝光时间和第二图像的期望平均亮度，确定该第二图像对应的曝光时间。In a first aspect, a method for exposing includes: determining an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel; and according to an average brightness of the first image, the first The exposure time corresponding to the image and the desired average brightness of the second image determine the exposure time corresponding to the second image.
第二方面，提供了一种曝光的方法，包括：分别确定具有依赖关系的多个视觉传感器的曝光时间；对该多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。In a second aspect, a method for exposing includes: determining exposure times of a plurality of vision sensors having dependencies, respectively; jointly optimizing exposure times of the plurality of vision sensors to enable the plurality of visions The difference between the optimized exposure times of the sensor is minimal.
第三方面，提供了一种曝光的装置，包括：平均亮度确定模块，用于 根据第一图像的像素的亮度和该像素的亮度权重，确定该第一图像的平均亮度；曝光时间确定模块，用于根据该第一图像的平均亮度，该第一图像对应的曝光时间和第二图像的期望平均亮度，确定该第二图像对应的曝光时间。In a third aspect, an apparatus for exposing includes: an average brightness determination module for Determining, according to the brightness of the pixel of the first image and the brightness weight of the pixel, an average brightness of the first image; an exposure time determining module, configured to: according to an average brightness of the first image, an exposure time corresponding to the first image The expected average brightness of the two images determines the exposure time corresponding to the second image.
第四方面，提供了一种曝光的装置，包括：确定模块，用于分别确定具有依赖关系的多个视觉传感器的曝光时间；联合优化模块，用于对该多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。A fourth aspect provides an apparatus for exposing, comprising: a determining module for respectively determining exposure times of a plurality of vision sensors having a dependency; a joint optimization module for separately determined exposure of the plurality of vision sensors The time is jointly optimized to minimize the difference between the optimized exposure times of the plurality of vision sensors.
第五方面，提供了一种计算机***，包括：存储器，用于存储计算机可执行指令；处理器，用于访问所述存储器，并执行所述计算机可执行指令，以进行上述第一或第二方面的方法中的操作。In a fifth aspect, a computer system is provided, comprising: a memory for storing computer executable instructions; a processor for accessing the memory, and executing the computer executable instructions to perform the first or second The operation in the aspect of the method.
第六方面，提供了一种移动设备，包括：上述第三或第四方面的曝光的装置；或者，上述第五方面的的计算机***。According to a sixth aspect, there is provided a mobile device comprising: the apparatus for exposing the third or fourth aspect; or the computer system of the above fifth aspect.
第七方面，提供了一种计算机存储介质，该计算机存储介质中存储有程序代码，该程序代码可以用于指示执行上述第一或第二方面的方法。In a seventh aspect, a computer storage medium is provided having stored therein program code, the program code being operative to indicate a method of performing the first or second aspect described above.
本发明实施例的技术方案，基于亮度权重确定图像的平均亮度，并根据该平均亮度调整曝光时间，这样所得到的曝光时间与像素的重要性关联，从而能够有效地调整曝光时间。According to the technical solution of the embodiment of the present invention, the average brightness of the image is determined based on the brightness weight, and the exposure time is adjusted according to the average brightness, so that the obtained exposure time is associated with the importance of the pixel, so that the exposure time can be effectively adjusted.
附图说明DRAWINGS
图1是本发明实施例的可移动设备的示意性架构图。FIG. 1 is a schematic structural diagram of a mobile device according to an embodiment of the present invention.
图2是本发明一个实施例的曝光的方法的示意性流程图。2 is a schematic flow chart of a method of exposure according to an embodiment of the present invention.
图3是本发明实施例的图像区域划分的示意图。FIG. 3 is a schematic diagram of image area division according to an embodiment of the present invention.
图4是本发明实施例的图像过曝光的示意图。4 is a schematic diagram of image overexposure according to an embodiment of the present invention.
图5是本发明另一个实施例的曝光的方法的示意性流程图。Figure 5 is a schematic flow diagram of a method of exposure in accordance with another embodiment of the present invention.
图6是本发明一个实施例的曝光的装置的示意性框图。Figure 6 is a schematic block diagram of an apparatus for exposure of one embodiment of the present invention.
图7是本发明另一个实施例的曝光的装置的示意性框图。Figure 7 is a schematic block diagram of an apparatus for exposure of another embodiment of the present invention.
图8是本发明又一个实施例的曝光的装置的示意性框图。Figure 8 is a schematic block diagram of an apparatus for exposure of still another embodiment of the present invention.
图9是本发明实施例的计算机***的示意性框图。9 is a schematic block diagram of a computer system in accordance with an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合附图，对本发明实施例中的技术方案进行描述。 The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings.
应理解，本文中的具体的例子只是为了帮助本领域技术人员更好地理解本发明实施例，而非限制本发明实施例的范围。It should be understood that the specific examples herein are merely intended to provide a better understanding of the embodiments of the invention.
还应理解，本发明实施例中的公式只是一种示例，而非限制本发明实施例的范围，各公式可以进行变形，这些变形也应属于本发明保护的范围。It should be understood that the formulas in the embodiments of the present invention are only examples, and are not intended to limit the scope of the embodiments of the present invention, and the formulas may be modified, and such modifications are also within the scope of the present invention.
还应理解，在本发明的各种实施例中，各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。It should also be understood that, in various embodiments of the present invention, the size of the sequence numbers of the processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be construed as an embodiment of the present invention. The implementation process constitutes any limitation.
还应理解，本说明书中描述的各种实施方式，既可以单独实施，也可以组合实施，本发明实施例对此并不限定。It should be understood that the various embodiments described in the specification may be implemented separately or in combination, and the embodiments of the present invention are not limited thereto.
本发明实施例的技术方案可以应用于各种图像采集***、以及依赖图像或图像采集***的各种设备，例如可移动设备。The technical solution of the embodiments of the present invention can be applied to various image acquisition systems, and various devices that rely on an image or image acquisition system, such as a mobile device.
该可移动设备也可以称为移动设备。该可移动设备可以是无人机、无人驾驶船、自动驾驶车辆或机器人等，但本发明实施例对此并不限定。The removable device can also be referred to as a mobile device. The mobile device may be a drone, an unmanned boat, an autonomous vehicle or a robot, etc., but the embodiment of the present invention is not limited thereto.
图1是本发明实施例的可移动设备100的示意性架构图。FIG. 1 is a schematic architectural diagram of a removable device 100 according to an embodiment of the present invention.
如图1所示，可移动设备100可以包括动力***110、控制***120、传感***130和处理***140。As shown in FIG. 1, the mobile device 100 can include a power system 110, a control system 120, a sensing system 130, and a processing system 140.
动力***110用于为该可移动设备100提供动力。Power system 110 is used to power the mobile device 100.
以无人机为例，无人机的动力***可以包括电子调速器(简称为电调)、螺旋桨以及与螺旋桨相对应的电机。电机连接在电子调速器与螺旋桨之间，电机和螺旋桨设置在对应的机臂上；电子调速器用于接收控制***产生的驱动信号，并根据驱动信号提供驱动电流给电机，以控制电机的转速。电机用于驱动螺旋桨旋转，从而为无人机的飞行提供动力。Taking the drone as an example, the power system of the drone may include an electronic governor (referred to as an electric current), a propeller, and a motor corresponding to the propeller. The motor is connected between the electronic governor and the propeller, and the motor and the propeller are disposed on the corresponding arm; the electronic governor is used for receiving the driving signal generated by the control system, and providing driving current to the motor according to the driving signal to control the motor Rotating speed. The motor is used to drive the propeller to rotate to power the drone's flight.
传感***130可以用于测量可移动设备100的姿态信息，即可移动设备100在空间的位置信息和状态信息，例如，三维位置、三维角度、三维速度、三维加速度和三维角速度等。传感***130例如可以包括陀螺仪、电子罗盘、惯性测量单元(Inertial Measurement Unit，IMU)、视觉传感器、全球定位***(Global Positioning System，GPS)、气压计、空速计等传感器中的至少一种。The sensing system 130 can be used to measure attitude information of the mobile device 100, that is, position information and state information of the mobile device 100 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity. The sensing system 130 may include, for example, at least one of a gyroscope, an electronic compass, an Inertial Measurement Unit (IMU), a vision sensor, a Global Positioning System (GPS), a barometer, an airspeed meter, and the like. Kind.
传感***130还可用于采集图像，即传感***130包括用于采集图像的传感器，例如相机等。Sensing system 130 can also be used to acquire images, i.e., sensing system 130 includes sensors for acquiring images, such as cameras and the like.
控制***120用于控制可移动设备100的移动。控制***120可以按 照预先设置的程序指令对可移动设备100进行控制。例如，控制***120可以根据传感***130测量的可移动设备100的姿态信息控制可移动设备100的移动。控制***120也可以根据来自遥控器的控制信号对可移动设备100进行控制。例如，对于无人机，控制***120可以为飞行控制***(飞控)，或者为飞控中的控制电路。Control system 120 is used to control the movement of mobile device 100. Control system 120 can be pressed The removable device 100 is controlled in accordance with a program instruction set in advance. For example, the control system 120 can control the movement of the mobile device 100 based on the attitude information of the mobile device 100 measured by the sensing system 130. Control system 120 can also control mobile device 100 based on control signals from the remote control. For example, for a drone, the control system 120 can be a flight control system (flying control) or a control circuit in a flight control.
处理***140可以处理传感***130采集的图像。例如，处理***140可以为图像信号处理(Image Signal Processing,ISP)类芯片。Processing system 140 can process the images acquired by sensing system 130. For example, processing system 140 can be an Image Signal Processing (ISP) type of chip.
在一些可能的设计中，处理***140可以根据传感***130已采集的图像确定采集下一图像的曝光时间。例如，处理***140可以采用本发明实施例的技术方案确定曝光时间。In some possible designs, processing system 140 may determine an exposure time to acquire the next image based on images that sensor system 130 has acquired. For example, the processing system 140 can determine the exposure time using the technical solution of the embodiments of the present invention.
在一些可能的设计中，处理***140可以与传感***130集成设置。例如，处理***140可以集成到相应的视觉传感器中。In some possible designs, processing system 140 can be integrated with sensing system 130. For example, processing system 140 can be integrated into a corresponding visual sensor.
应理解，上述对于可移动设备100的各组成部件的划分和命名仅仅是示例性的，并不应理解为对本发明实施例的限制。It should be understood that the above-mentioned division and naming of the components of the mobile device 100 are merely exemplary and should not be construed as limiting the embodiments of the present invention.
还应理解，可移动设备100还可以包括图1中未示出的其他部件，本发明实施例对此并不限定。It should also be understood that the removable device 100 may also include other components not shown in FIG. 1, which are not limited by the embodiments of the present invention.
图2示出了本发明一个实施例的曝光的方法200的示意性流程图。该方法200可以由图1所示的可移动设备100执行。具体地，可以由图1中的处理***140执行。2 shows a schematic flow diagram of a method 200 of exposure in accordance with one embodiment of the present invention. The method 200 can be performed by the mobile device 100 shown in FIG. In particular, it can be performed by processing system 140 in FIG.
210，根据第一图像的像素的亮度和该像素的亮度权重，确定该第一图像的平均亮度。210. Determine an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel.
在本发明实施例中，在确定图像的平均亮度时，考虑像素的亮度权重。也就是说，对于图像的每个像素，在计算图像的平均亮度时，要考虑该像素的权重，即越重要的像素所占的权重越大。In the embodiment of the present invention, the luminance weight of the pixel is considered in determining the average luminance of the image. That is to say, for each pixel of the image, when calculating the average brightness of the image, the weight of the pixel is considered, that is, the more important the pixel is, the greater the weight.
可选地，该第一图像的平均亮度为该第一图像的像素的亮度的加权平均值。Optionally, the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
例如，可以根据以下公式(1)计算图像的平均亮度
Figure PCTCN2017099419-appb-000001
For example, the average brightness of an image can be calculated according to the following formula (1)
Figure PCTCN2017099419-appb-000001
Figure PCTCN2017099419-appb-000002
Figure PCTCN2017099419-appb-000002
其中I＝{R,G,B}表示图像，
Figure PCTCN2017099419-appb-000003
为像素p的亮度，ωp为像素p的亮度权重，W为归一化系数，W＝∑ωp。Where I={R, G, B} represents the image,
Figure PCTCN2017099419-appb-000003
For the luminance of the pixel p, ω p is the luminance weight of the pixel p, and W is the normalization coefficient, W = ∑ ω p .
可选地，该像素的亮度权重可以包括区域权重和色度权重中的至少一 项，其中，该区域权重为与该像素的区域关联的权重，该色度权重为与该像素的色度关联的权重。Optionally, the brightness weight of the pixel may include at least one of a region weight and a chrominance weight The item, wherein the area weight is a weight associated with the area of the pixel, the chromaticity weight being a weight associated with the chrominance of the pixel.
可选地，作为一个示例，亮度权重ω可以表示为：Alternatively, as an example, the luminance weight ω can be expressed as:
ω＝ωrωc (2)ω=ω r ω c (2)
其中ωr为区域权重，ωc为色度权重。Where ω r is the regional weight and ω c is the chromaticity weight.
应理解，本发明实施例中的亮度权重可以只包括区域权重，或者只包括色度权重，或者同时包括区域权重和色度权重，或者还可以包括其他权重，本发明对此并不限定。It should be understood that the brightness weight in the embodiment of the present invention may include only the area weight, or only the chrominance weight, or both the area weight and the chrominance weight, or may also include other weights, which is not limited by the present invention.
可选地，对于该区域权重，若该像素的区域越接近目标对象的区域，则该区域权重越大。Optionally, for the region weight, if the region of the pixel is closer to the region of the target object, the region weight is larger.
具体而言，针对特定场景，不同区域的重要性可能不同。对于不同的区域，可以根据区域的重要性设置不同的区域权重。例如，若某一区域越接近目标对象的区域，则区域权重越大。In particular, the importance of different regions may be different for a particular scenario. For different areas, different area weights can be set according to the importance of the area. For example, if an area is closer to the area of the target object, the area weight is larger.
例如，以自动驾驶项目中的前视相机为例，在平行路面采集的图像中主要有行驶道路、道路两旁目标物体，天空和远处目标物体等组成，其中以行驶路面(目标对象)最为重要，其次是远处的目标和道路两侧的目标，再次则是图像中的剩余部分。根据这个先验知识，我们可以对图像进行区域划分，对于不同的区域，根据其重要性设置不同的权重，以保证最重要的区域获得更加合理的曝光设置。For example, taking the forward-looking camera in the automatic driving project as an example, the images collected on the parallel road mainly include the traveling road, the target objects on both sides of the road, the sky and the distant target objects, among which the driving road surface (target object) is the most important. , followed by the distant target and the targets on both sides of the road, and again the rest of the image. According to this prior knowledge, we can divide the image into regions. For different regions, different weights are set according to their importance to ensure that the most important regions get more reasonable exposure settings.
可选地，图像的区域划分策略和权重分配可以根据场景以及相机的安装位置决定。举例来说，在前视相机图像中可以如图3所示划分区域，各区域相应的权重可以如表1所示。Alternatively, the zoning strategy and weight assignment of the image may be determined according to the scene and the installation location of the camera. For example, in the forward-looking camera image, the area may be divided as shown in FIG. 3, and the corresponding weights of each area may be as shown in Table 1.
表1Table 1
0.10.1 0.20.2 0.20.2 0.10.1
0.20.2 1.01.0 1.01.0 0.20.2
0.50.5 1.51.5 1.51.5 0.50.5
在表1中，行驶路面所在区域的权重最大，而远离行驶路面的区域的权重较小。In Table 1, the area in which the road surface is traveled has the largest weight, and the area away from the road surface has a smaller weight.
可选地，对于该色度权重，若该像素的色度越接近目标对象的色度，则该色度权重越大。Optionally, for the chrominance weight, if the chromaticity of the pixel is closer to the chromaticity of the target object, the chromaticity weight is larger.
具体而言，针对特定场景，不同色度的重要性也可能不同。对于不同 的色度，也可以设置不同的色度权重。例如，若某一色度越接近目标对象的色度，则色度权重越大。In particular, the importance of different chrominances may also be different for a particular scenario. For different The chromaticity can also be set to different chromaticity weights. For example, if a certain chromaticity is closer to the chromaticity of the target object, the chromaticity weight is larger.
例如，在图2中，行驶路面(目标对象)大部分为灰色，因此，可以利用色度权重来控制不同颜色物体的权重，使得颜色越接近灰色，权重越大。可选地，可以根据以下公式(3)确定色度权重ωc：For example, in FIG. 2, the traveling road surface (target object) is mostly gray, and therefore, the chromaticity weight can be utilized to control the weights of the different color objects such that the closer the color is to the gray, the larger the weight. Alternatively, the chromaticity weight ω c can be determined according to the following formula (3):
Figure PCTCN2017099419-appb-000004
Figure PCTCN2017099419-appb-000004
其中列向量
Figure PCTCN2017099419-appb-000005
为灰色的色度。Column vector
Figure PCTCN2017099419-appb-000005
Gray shades.
220，根据该第一图像的平均亮度，该第一图像对应的曝光时间和第二图像的期望平均亮度，确定该第二图像对应的曝光时间。220. Determine an exposure time corresponding to the second image according to an average brightness of the first image, an exposure time corresponding to the first image, and a desired average brightness of the second image.
在本发明实施例中，根据上述基于亮度权重的第一图像的平均亮度确定第二图像对应的曝光时间。该第一图像为已拍摄的图像，该第二图像为待拍摄的图像。可选地，该第一图像为最新的已拍摄的图像。也就是说，可以根据上一时刻拍摄的图像的基于亮度权重的平均亮度确定下一时刻进行拍摄的曝光时间。In an embodiment of the invention, the exposure time corresponding to the second image is determined according to the average brightness of the first image based on the brightness weight. The first image is a captured image, and the second image is an image to be captured. Optionally, the first image is the latest captured image. That is to say, the exposure time at which the next time is taken can be determined based on the average brightness based on the luminance weight of the image taken at the previous time.
可选地，可以根据该第一图像的平均亮度和该第一图像对应的曝光时间，确定平均亮度导数；根据该第二图像的期望平均亮度，该第一图像的平均亮度和该平均亮度导数，确定曝光时间调整量；再根据该第一图像对应的曝光时间和该曝光时间调整量，确定该第二图像对应的曝光时间，可选地，可以根据该第一图像对应的曝光时间，该曝光时间调整量和调节系数，确定该第二图像对应的曝光时间。Optionally, an average brightness derivative may be determined according to an average brightness of the first image and an exposure time corresponding to the first image; and an average brightness of the first image and the average brightness derivative according to a desired average brightness of the second image Determining an exposure time adjustment amount; determining an exposure time corresponding to the second image according to the exposure time corresponding to the first image and the exposure time adjustment amount, optionally, according to an exposure time corresponding to the first image, The exposure time adjustment amount and the adjustment coefficient determine the exposure time corresponding to the second image.
例如，平均亮度导数
Figure PCTCN2017099419-appb-000006
可以通过以下公式(4)计算：For example, the average brightness derivative
Figure PCTCN2017099419-appb-000006
It can be calculated by the following formula (4):
Figure PCTCN2017099419-appb-000007
Figure PCTCN2017099419-appb-000007
其中，平均亮度函数
Figure PCTCN2017099419-appb-000008
是关于曝光时间t的单调自增函数。根据1阶的泰勒展开公式可以得到：Among them, the average brightness function
Figure PCTCN2017099419-appb-000008
It is a monotonic auto-increment function with respect to the exposure time t. According to the 1st-order Taylor expansion formula, you can get:
Figure PCTCN2017099419-appb-000009
Figure PCTCN2017099419-appb-000009
若期望平均亮度为
Figure PCTCN2017099419-appb-000010
假设
Figure PCTCN2017099419-appb-000011
则可以得到曝光时间调整量：If the average brightness is expected to be
Figure PCTCN2017099419-appb-000010
Hypothesis
Figure PCTCN2017099419-appb-000011
You can get the exposure time adjustment:
Figure PCTCN2017099419-appb-000012
Figure PCTCN2017099419-appb-000012
所以可以得到下一时刻的曝光时间tn+1为： So you can get the exposure time t n+1 at the next moment:
Figure PCTCN2017099419-appb-000013
Figure PCTCN2017099419-appb-000013
其中β为调节系数，用于控制自动曝光调节的速度。例如，若调节速度偏大，则可以通过降低调节系数来减缓调节速度，反之亦然。Where β is the adjustment factor used to control the speed of the automatic exposure adjustment. For example, if the adjustment speed is too large, the adjustment speed can be slowed down by lowering the adjustment factor, and vice versa.
应理解，本发明实施例中的曝光时间的概念可以理解为曝光总量，其控制可以通过调整相机的快门速度(shutter speed)、光圈大小(aperture size)、或者增益(gain)来达到，并不特指某一分量。It should be understood that the concept of exposure time in the embodiment of the present invention can be understood as the total amount of exposure, and the control can be achieved by adjusting the shutter speed, the aperture size, or the gain of the camera, and Does not specifically refer to a certain component.
在本发明实施例中，基于亮度权重确定图像的平均亮度，并根据该平均亮度调整曝光时间，这样所得到的曝光时间与像素的重要性关联，从而能够有效地调整曝光时间。In the embodiment of the present invention, the average brightness of the image is determined based on the luminance weight, and the exposure time is adjusted according to the average brightness, so that the obtained exposure time is associated with the importance of the pixel, so that the exposure time can be effectively adjusted.
可选地，该第二图像的期望平均亮度可以为预先设定的。也就是说，可以预先设定待拍摄图像的期望平均亮度，据此得到相应的曝光时间，进而拍摄得到接近该期望平均亮度的图像。比如在8位的图像中，可以设置
Figure PCTCN2017099419-appb-000014
Figure PCTCN2017099419-appb-000015
Optionally, the desired average brightness of the second image may be preset. That is to say, the desired average brightness of the image to be captured can be set in advance, and accordingly the corresponding exposure time is obtained, thereby obtaining an image close to the desired average brightness. For example, in an 8-bit image, you can set
Figure PCTCN2017099419-appb-000014
Figure PCTCN2017099419-appb-000015
可选地，还可以对该第二图像的期望平均亮度进行动态调整。可选地，可以根据该第一图像中过曝光像素的百分比，调整该第二图像的期望平均亮度。Optionally, the desired average brightness of the second image can also be dynamically adjusted. Optionally, the desired average brightness of the second image may be adjusted according to a percentage of the overexposed pixels in the first image.
例如，在不同天气情况或者不同场景下，道路路面的反光强度的差异有可能达到几十倍，假如应用固定的期望平均亮度，在道路两旁树木茂盛的地方，由于道路两旁树木反光相对于路面反光有可能出现十几倍的差距，这时候很容易出现路面过曝的问题，如图4所示。为此，可以通过动态调整期望平均亮度
Figure PCTCN2017099419-appb-000016
来解决这个问题。在本发明实施例中，可以通过利用图像过曝光像素所占百分比来调节期望平均亮度
Figure PCTCN2017099419-appb-000017
For example, in different weather conditions or different scenarios, the difference in the reflective intensity of the road surface may reach several tens of times. If a fixed desired average brightness is applied, in the wooded areas on both sides of the road, the reflection of the trees on both sides of the road is reflected relative to the road surface. There may be a gap of more than ten times. At this time, the problem of over-exposure on the road surface is easy to occur, as shown in Figure 4. To do this, you can dynamically adjust the desired average brightness.
Figure PCTCN2017099419-appb-000016
To solve this problem. In the embodiment of the present invention, the desired average brightness can be adjusted by using the percentage of the image overexposed pixels.
Figure PCTCN2017099419-appb-000017
Figure PCTCN2017099419-appb-000018
Figure PCTCN2017099419-appb-000018
其中σ为图像In中过度曝光像素(例如
Figure PCTCN2017099419-appb-000019
)所占的百分比，
Figure PCTCN2017099419-appb-000020
为期望的过曝光比例。例如，可以设置
Figure PCTCN2017099419-appb-000021
为0.05。Where σ is the overexposed pixel in image I n (eg
Figure PCTCN2017099419-appb-000019
Percentage of
Figure PCTCN2017099419-appb-000020
For the desired overexposure ratio. For example, you can set
Figure PCTCN2017099419-appb-000021
It is 0.05.
通过动态调整期望平均亮度，可以有效避免过曝光问题。By dynamically adjusting the desired average brightness, overexposure problems can be effectively avoided.
上述本发明实施例的技术方案可以用于确定每个视觉传感器(相机)的曝光时间。图像采集***中可能会包括多个具有依赖关系的视觉传感器。以双目立体视觉为例，其要求左右目相机的曝光时间一致，以便于进行图像特征点的匹配。如果曝光差异较大，有可能导致双目匹配的效果不佳，进而影响视觉算法的性能。如果***中只存在一套双目视觉***，则可以将其曝 光时间设置为一致(比如针对双目独立曝光所计算的曝光时间的平均值)。但是当***中存在多目立体视觉、或者某一相机被用于多套双目视觉***中时，就需要对***中所有相机的曝光时间进行联合优化。The above technical solution of the embodiment of the present invention can be used to determine the exposure time of each visual sensor (camera). Multiple image sensors with dependencies may be included in the image acquisition system. Taking binocular stereo vision as an example, it requires the exposure time of the left and right eye cameras to be consistent, so as to facilitate matching of image feature points. If the exposure difference is large, it may cause the effect of binocular matching to be poor, which may affect the performance of the visual algorithm. If there is only one set of binocular vision system in the system, you can expose it. The light time is set to be consistent (such as the average of the exposure times calculated for binocular independent exposure). However, when there is multi-eye stereo vision in the system, or when a camera is used in a multi-set binocular vision system, it is necessary to jointly optimize the exposure time of all cameras in the system.
相应地，在本发明实施例中，还可以对具有依赖关系的多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。Correspondingly, in the embodiment of the present invention, the separately determined exposure times of the plurality of vision sensors having dependencies may also be jointly optimized to minimize the difference between the optimized exposure times of the plurality of vision sensors.
对于每个视觉传感器，可以采用前述方案分别确定其曝光时间，然后再对多个视觉传感器的曝光时间进行联合优化。For each of the vision sensors, the exposure time can be determined separately by the foregoing scheme, and then the exposure times of the plurality of vision sensors are jointly optimized.
可选地，在进行进行联合优化时，可以根据该多个视觉传感器的配置，确定优化矩阵；根据该优化矩阵和该多个视觉传感器的分别确定的曝光时间，确定该多个视觉传感器的优化后的曝光时间。Optionally, when performing joint optimization, determining an optimization matrix according to the configuration of the plurality of vision sensors; determining optimization of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors After the exposure time.
以***中存在多套双目立体视觉***为例(多目立体视觉与此情况类似)，在具有任意路相机的***中，假设有n个相机，分别确定的曝光时间为{t1,t2,t3,…,tn}，其中存在m对双目立体视觉***
Figure PCTCN2017099419-appb-000022
每一对双目立体视觉***的权重系数分别为
Figure PCTCN2017099419-appb-000023
则可以得到以下方程组：Taking multiple sets of binocular stereo vision systems in the system as an example (multi-view stereo vision is similar to this case), in a system with an arbitrary camera, assuming n cameras, the exposure time determined is {t 1 ,t 2 , t 3 ,...,t n }, where there is an m-to-binocular stereo vision system
Figure PCTCN2017099419-appb-000022
The weight coefficients of each pair of binocular stereo vision systems are
Figure PCTCN2017099419-appb-000023
Then you can get the following equations:
Figure PCTCN2017099419-appb-000024
Figure PCTCN2017099419-appb-000024
上述线性方程组可以表达为以下矩阵的形式：The above linear equations can be expressed in the form of the following matrix:
At′＝b  (10)At'=b (10)
其中t′＝(t‘1,t′2,t′3,…,t′n)T表示联合优化后的相机曝光时间，b＝(t1,t2,…,tn,0,0,…,0)T。上述矩阵可以通过最小二乘法计算得到：Where t' = (t' 1 , t' 2 , t' 3 , ..., t' n ) T represents the camera exposure time after joint optimization, b = (t 1 , t 2 , ..., t n , 0, 0 ,...,0) T . The above matrix can be calculated by the least squares method:
t′＝(ATA)-1ATb  (11)t'=(A T A) -1 A T b (11)
矩阵A依赖于多个视觉传感器的配置，即图像采集***的具体配置，因此(ATA)-1AT可以进行预计算，并不会在运行时带来额外的开销。Matrix A relies on the configuration of multiple vision sensors, ie the specific configuration of the image acquisition system, so (A T A) -1 A T can be pre-computed without additional overhead at runtime.
以***中采用3个相机，其中1,2、1,3号相机组成双目***，双目权重系数均为1为例，矩阵A为： Three cameras are used in the system, and the cameras of 1, 2, 1, and 3 form a binocular system. The binocular weight coefficients are all 1 and the matrix A is:
Figure PCTCN2017099419-appb-000025
Figure PCTCN2017099419-appb-000025
这样，通过b和A可以得到联合优化后的曝光时间t′。Thus, the jointly optimized exposure time t' can be obtained by b and A.
通过联合优化，可以使多个视觉传感器的曝光时间整体最优，从而提升整体性能。Through joint optimization, the exposure time of multiple vision sensors can be optimized as a whole, thereby improving overall performance.
应理解，上述本发明实施例的确定曝光时间的技术方案和曝光时间联合优化的技术方案可以联合实施，也可以单独实施。换句话说，曝光时间联合优化的技术方案可以依赖于上述确定曝光时间的技术方案，也可以不依赖于上述确定曝光时间的技术方案。基于此，本发明实施例又提供了另一种曝光的方法，下面结合图5进行描述。应理解，图5所示方法中的一些具体的描述可以参考前述实施例，以下为了简洁，不再赘述。It should be understood that the technical solution for determining the exposure time and the joint optimization of the exposure time in the above embodiments of the present invention may be implemented jointly or separately. In other words, the technical solution of the exposure time joint optimization may depend on the above-mentioned technical solution for determining the exposure time, or may not depend on the above-mentioned technical solution for determining the exposure time. Based on this, another embodiment of the present invention provides a method of exposure, which will be described below in conjunction with FIG. 5. It should be understood that some specific descriptions of the method shown in FIG. 5 may refer to the foregoing embodiments, and are not described herein again for brevity.
图5示出了本发明另一个实施例的曝光的方法500的示意性流程图。如图5所示，该方法500包括：FIG. 5 shows a schematic flow diagram of a method 500 of exposure in accordance with another embodiment of the present invention. As shown in FIG. 5, the method 500 includes:
510，分别确定具有依赖关系的多个视觉传感器的曝光时间；510, respectively, determining an exposure time of the plurality of vision sensors having dependencies;
520，对该多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。520. Perform joint optimization on the determined exposure times of the plurality of vision sensors to minimize the difference between the optimized exposure times of the plurality of vision sensors.
对于每个视觉传感器，其曝光时间可以采用前述实施例的技术方案确定，也可以采用其他方式确定，本发明实施例对此并不限定。对分别确定的多个视觉传感器的曝光时间的联合优化可以采用前述实施例的联合优化的技术方案，为了简洁，在此不再赘述。通过联合优化，可以使多个视觉传感器的曝光时间整体最优，从而提升整体性能。For each of the visual sensors, the exposure time may be determined by using the technical solution of the foregoing embodiment, and may be determined by other methods, which is not limited by the embodiment of the present invention. For the joint optimization of the exposure time of the plurality of visual sensors respectively determined, the joint optimization technical solution of the foregoing embodiment may be adopted, and for brevity, no further details are provided herein. Through joint optimization, the exposure time of multiple vision sensors can be optimized as a whole, thereby improving overall performance.
根据采用本发明实施例的技术方案确定的曝光时间，可以得到曝光合理的图像，作为诸如视觉里程计、双目立体视觉、图像识别与跟踪等上层计算机视觉算法的输入，能够保证其结果的准确性和稳定性。According to the exposure time determined by the technical solution of the embodiment of the present invention, an image with reasonable exposure can be obtained as an input of an upper computer vision algorithm such as a visual odometer, binocular stereo vision, image recognition and tracking, and the result can be ensured accurately. Sex and stability.
上文详细描述了本发明实施例的曝光的方法，下面将描述本发明实施例的曝光的装置、计算机***和可移动设备。应理解，本发明实施例的曝光的装置、计算机***和可移动设备可以执行前述本发明实施例的各种方法，即以下各种产品的具体工作过程，可以参考前述方法实施例中的对应过程。The method of exposure of the embodiment of the present invention has been described in detail above, and an apparatus, a computer system, and a portable device of the exposure of the embodiment of the present invention will be described below. It should be understood that the apparatus, the computer system, and the mobile device of the embodiment of the present invention may perform the foregoing various methods of the embodiments of the present invention, that is, the specific working processes of the following various products, and may refer to the corresponding processes in the foregoing method embodiments. .
图6示出了本发明一个实施例的曝光的装置600的示意性框图。如图 6所示，该装置600可以包括：Figure 6 shows a schematic block diagram of an apparatus 600 for exposure of one embodiment of the present invention. As shown As shown in Figure 6, the apparatus 600 can include:
平均亮度确定模块610，用于根据第一图像的像素的亮度和该像素的亮度权重，确定该第一图像的平均亮度；The average brightness determining module 610 is configured to determine an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel;
曝光时间确定模块620，用于根据该第一图像的平均亮度，该第一图像对应的曝光时间和第二图像的期望平均亮度，确定该第二图像对应的曝光时间。The exposure time determining module 620 is configured to determine an exposure time corresponding to the second image according to an average brightness of the first image, an exposure time corresponding to the first image, and a desired average brightness of the second image.
可选地，在本发明一个实施例中，该像素的亮度权重包括区域权重和色度权重中的至少一项，其中，该区域权重为与该像素的区域关联的权重，该色度权重为与该像素的色度关联的权重。Optionally, in an embodiment of the present invention, the brightness weight of the pixel includes at least one of a region weight and a chrominance weight, wherein the region weight is a weight associated with the region of the pixel, and the chrominance weight is The weight associated with the chrominance of the pixel.
可选地，在本发明一个实施例中，若该像素的区域越接近目标对象的区域，则该区域权重越大；和/或，若该像素的色度越接近目标对象的色度，则该色度权重越大。Optionally, in an embodiment of the present invention, if the area of the pixel is closer to the area of the target object, the weight of the area is larger; and/or, if the chromaticity of the pixel is closer to the chromaticity of the target object, The chromaticity weight is greater.
可选地，在本发明一个实施例中，该第一图像的平均亮度为该第一图像的像素的亮度的加权平均值。Optionally, in an embodiment of the invention, the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
可选地，在本发明一个实施例中，该第二图像的期望平均亮度为预先设定的。Optionally, in an embodiment of the invention, the desired average brightness of the second image is preset.
可选地，如图7所示，在本发明一个实施例中，该装置600还可以包括：Optionally, as shown in FIG. 7, in an embodiment of the present invention, the apparatus 600 may further include:
调整模块630，用于根据该第一图像中过曝光像素的百分比，调整该第二图像的期望平均亮度。The adjusting module 630 is configured to adjust a desired average brightness of the second image according to a percentage of the overexposed pixels in the first image.
可选地，在本发明一个实施例中，该曝光时间确定模块620用于：Optionally, in an embodiment of the present invention, the exposure time determining module 620 is configured to:
根据该第一图像的平均亮度和该第一图像对应的曝光时间，确定平均亮度导数；Determining an average brightness derivative according to an average brightness of the first image and an exposure time corresponding to the first image;
根据该第二图像的期望平均亮度，该第一图像的平均亮度和该平均亮度导数，确定曝光时间调整量；Determining an exposure time adjustment amount according to a desired average brightness of the second image, an average brightness of the first image, and the average brightness derivative;
根据该第一图像对应的曝光时间和该曝光时间调整量，确定该第二图像对应的曝光时间。And determining an exposure time corresponding to the second image according to the exposure time corresponding to the first image and the exposure time adjustment amount.
可选地，在本发明一个实施例中，该曝光时间确定模块620用于：Optionally, in an embodiment of the present invention, the exposure time determining module 620 is configured to:
该根据该第一图像对应的曝光时间，该曝光时间调整量和调节系数，确定该第二图像对应的曝光时间。The exposure time corresponding to the first image, the exposure time adjustment amount and the adjustment coefficient determine an exposure time corresponding to the second image.
可选地，在本发明一个实施例中，该第一图像为已拍摄的图像，该 第二图像为待拍摄的图像。Optionally, in an embodiment of the present invention, the first image is a captured image, and the image is The second image is an image to be taken.
可选地，在本发明一个实施例中，该第一图像为最新的已拍摄的图像。Optionally, in an embodiment of the invention, the first image is the latest captured image.
可选地，如图7所示，在本发明一个实施例中，该装置600还可以包括：Optionally, as shown in FIG. 7, in an embodiment of the present invention, the apparatus 600 may further include:
联合优化模块640，用于对具有依赖关系的多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。The joint optimization module 640 is configured to jointly optimize the respectively determined exposure times of the plurality of vision sensors having dependencies to minimize the difference between the optimized exposure times of the plurality of vision sensors.
可选地，在本发明一个实施例中，该联合优化模块640用于：Optionally, in an embodiment of the present invention, the joint optimization module 640 is configured to:
根据该多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
根据该优化矩阵和该多个视觉传感器的分别确定的曝光时间，确定该多个视觉传感器的优化后的曝光时间。An optimized exposure time of the plurality of vision sensors is determined based on the optimized matrix and the respective determined exposure times of the plurality of vision sensors.
图8示出了本发明另一个实施例的曝光的装置800的示意性框图。如图8所示，该装置800可以包括：FIG. 8 shows a schematic block diagram of an apparatus 800 for exposure of another embodiment of the present invention. As shown in FIG. 8, the apparatus 800 can include:
确定模块810，用于分别确定具有依赖关系的多个视觉传感器的曝光时间；a determining module 810, configured to respectively determine an exposure time of the plurality of vision sensors having a dependency relationship;
联合优化模块820，用于对多个视觉传感器的分别确定的曝光时间进行联合优化，以使该多个视觉传感器的优化后的曝光时间间的差异最小。The joint optimization module 820 is configured to jointly optimize the respectively determined exposure times of the plurality of vision sensors to minimize the difference between the optimized exposure times of the plurality of vision sensors.
可选地，在本发明一个实施例中，该联合优化模块820用于：Optionally, in an embodiment of the present invention, the joint optimization module 820 is configured to:
根据该多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
根据该优化矩阵和该多个视觉传感器的分别确定的曝光时间，确定该多个视觉传感器的优化后的曝光时间。An optimized exposure time of the plurality of vision sensors is determined based on the optimized matrix and the respective determined exposure times of the plurality of vision sensors.
应理解，上述本发明实施例的曝光的装置可以是芯片，其具体可以由电路实现，但本发明实施例对具体的实现形式不做限定。It should be understood that the apparatus for exposing the embodiment of the present invention may be a chip, which may be specifically implemented by a circuit, but the specific implementation form of the embodiment of the present invention is not limited.
本发明实施例还提供了一种处理器，该处理器可以包括上述本发明各种实施例的曝光的装置。Embodiments of the present invention also provide a processor that can include the apparatus for exposure of the various embodiments of the present invention described above.
图9示出了本发明实施例的计算机***900的示意性框图。FIG. 9 shows a schematic block diagram of a computer system 900 in accordance with an embodiment of the present invention.
如图9所示，该计算机***900可以包括处理器910和存储器920。As shown in FIG. 9, the computer system 900 can include a processor 910 and a memory 920.
应理解，该计算机***900还可以包括其他计算机***中通常所包括的部件，例如，输入输出设备、通信接口等，本发明实施例对此并不限定。 It should be understood that the computer system 900 may also include components that are generally included in other computer systems, such as input and output devices, communication interfaces, and the like, which are not limited by the embodiments of the present invention.
存储器920用于存储计算机可执行指令。 Memory 920 is for storing computer executable instructions.
存储器920可以是各种种类的存储器，例如可以包括高速随机存取存储器(Random Access Memory，RAM)，还可以包括非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器，本发明实施例对此并不限定。The memory 920 may be various kinds of memories, for example, may include a high speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, which is implemented by the present invention. This example is not limited to this.
处理器90用于访问该存储器920，并执行该计算机可执行指令，以进行上述本发明各种实施例的曝光的方法中的操作。The processor 90 is operative to access the memory 920 and execute the computer executable instructions to perform the operations in the method of exposure of the various embodiments of the present invention described above.
处理器910可以包括微处理器，现场可编程门阵列(Field－Programmable Gate Array，FPGA)，中央处理器(Central Processing unit，CPU)，图形处理器(Graphics Processing Unit，GPU)等，本发明实施例对此并不限定。The processor 910 may include a microprocessor, a Field-Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), etc., and is implemented by the present invention. This example is not limited to this.
本发明实施例还提供了一种可移动设备，该移动设备可以包括上述本发明各种实施例的曝光的装置、处理器或者计算机***。Embodiments of the present invention also provide a removable device that can include an apparatus, processor, or computer system for exposure of various embodiments of the present invention described above.
本发明实施例的曝光的装置、计算机***和可移动设备可对应于本发明实施例的曝光的方法的执行主体，并且曝光的装置、计算机***和可移动设备中的各个模块的上述和其它操作和/或功能分别为了实现前述各个方法的相应流程，为了简洁，在此不再赘述。The apparatus, computer system, and mobile device of the exposure of the embodiments of the present invention may correspond to an execution subject of the method of exposure of the embodiment of the present invention, and the above-described and other operations of the exposed device, the computer system, and each module in the movable device The functions and/or functions are respectively omitted in order to implement the corresponding processes of the foregoing various methods, and are not described herein for brevity.
本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序代码，该程序代码可以用于指示执行上述本发明实施例的曝光的方法。Embodiments of the present invention also provide a computer storage medium having stored therein program code, the program code being usable for indicating a method of performing the exposure of the embodiment of the present invention described above.
应理解，在本发明实施例中，术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系。例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。It should be understood that in the embodiment of the present invention, the term "and/or" is merely an association relationship describing an associated object, indicating that there may be three relationships. For example, A and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。 Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本发明实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。 The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent person can be easily conceived within the technical scope of the present invention by any person skilled in the art. Modifications or substitutions are intended to be included within the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims (31)

  1. 一种曝光的方法，其特征在于，包括：A method of exposure, comprising:
    根据第一图像的像素的亮度和所述像素的亮度权重，确定所述第一图像的平均亮度；Determining an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel;
    根据所述第一图像的平均亮度，所述第一图像对应的曝光时间和第二图像的期望平均亮度，确定所述第二图像对应的曝光时间。And determining an exposure time corresponding to the second image according to an average brightness of the first image, an exposure time corresponding to the first image, and a desired average brightness of the second image.
  2. 根据权利要求1所述的方法，其特征在于，所述像素的亮度权重包括区域权重和色度权重中的至少一项，其中，所述区域权重为与所述像素的区域关联的权重，所述色度权重为与所述像素的色度关联的权重。The method according to claim 1, wherein the brightness weight of the pixel comprises at least one of a region weight and a chrominance weight, wherein the region weight is a weight associated with the region of the pixel, The chrominance weight is the weight associated with the chrominance of the pixel.
  3. 根据权利要求2所述的方法，其特征在于，若所述像素的区域越接近目标对象的区域，则所述区域权重越大；和/或，若所述像素的色度越接近目标对象的色度，则所述色度权重越大。The method according to claim 2, wherein if the area of the pixel is closer to the area of the target object, the area weight is larger; and/or, if the chromaticity of the pixel is closer to the target object For chrominance, the chromaticity weight is greater.
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述第一图像的平均亮度为所述第一图像的像素的亮度的加权平均值。The method according to any one of claims 1 to 3, wherein the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述第二图像的期望平均亮度为预先设定的。The method according to any one of claims 1 to 4, wherein the desired average brightness of the second image is preset.
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，在所述确定所述第二图像对应的曝光时间之前，所述方法还包括：The method according to any one of claims 1 to 5, wherein before the determining an exposure time corresponding to the second image, the method further comprises:
    根据所述第一图像中过曝光像素的百分比，调整所述第二图像的期望平均亮度。Adjusting a desired average brightness of the second image based on a percentage of overexposed pixels in the first image.
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述确定所述第二图像对应的曝光时间包括：The method according to any one of claims 1 to 6, wherein the determining an exposure time corresponding to the second image comprises:
    根据所述第一图像的平均亮度和所述第一图像对应的曝光时间，确定平均亮度导数；Determining an average brightness derivative according to an average brightness of the first image and an exposure time corresponding to the first image;
    根据所述第二图像的期望平均亮度，所述第一图像的平均亮度和所述平均亮度导数，确定曝光时间调整量；Determining an exposure time adjustment amount according to a desired average brightness of the second image, an average brightness of the first image, and the average brightness derivative;
    根据所述第一图像对应的曝光时间和所述曝光时间调整量，确定所述第二图像对应的曝光时间。And determining an exposure time corresponding to the second image according to an exposure time corresponding to the first image and the exposure time adjustment amount.
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述第一图像对应的曝光时间和所述曝光时间调整量，确定所述第二图像对应的曝光时间，包括： The method according to claim 7, wherein the determining the exposure time corresponding to the second image according to the exposure time corresponding to the first image and the exposure time adjustment amount comprises:
    所述根据所述第一图像对应的曝光时间，所述曝光时间调整量和调节系数，确定所述第二图像对应的曝光时间。And determining, according to the exposure time corresponding to the first image, the exposure time adjustment amount and the adjustment coefficient, an exposure time corresponding to the second image.
  9. 根据权利要求1至8中任一项所述的方法，其特征在于，所述第一图像为已拍摄的图像，所述第二图像为待拍摄的图像。The method according to any one of claims 1 to 8, wherein the first image is a captured image and the second image is an image to be captured.
  10. 根据权利要求9所述的方法，其特征在于，所述第一图像为最新的已拍摄的图像。The method of claim 9 wherein said first image is the most recent captured image.
  11. 根据权利要求1至10中任一项所述的方法，其特征在于，所述方法还包括：The method according to any one of claims 1 to 10, further comprising:
    对具有依赖关系的多个视觉传感器的分别确定的曝光时间进行联合优化，以使所述多个视觉传感器的优化后的曝光时间间的差异最小。The separately determined exposure times of the plurality of vision sensors having dependencies are jointly optimized to minimize differences between the optimized exposure times of the plurality of vision sensors.
  12. 根据权利要求11所述的方法，其特征在于，所述对具有依赖关系的多个视觉传感器的分别确定的曝光时间进行联合优化，包括：The method of claim 11 wherein said jointly optimizing said separately determined exposure times of a plurality of vision sensors having dependencies comprises:
    根据所述多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
    根据所述优化矩阵和所述多个视觉传感器的分别确定的曝光时间，确定所述多个视觉传感器的优化后的曝光时间。And determining an optimized exposure time of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors.
  13. 一种曝光的方法，其特征在于，包括：A method of exposure, comprising:
    分别确定具有依赖关系的多个视觉传感器的曝光时间；Determining exposure times of a plurality of vision sensors having dependencies, respectively;
    对所述多个视觉传感器的分别确定的曝光时间进行联合优化，以使所述多个视觉传感器的优化后的曝光时间间的差异最小。The separately determined exposure times of the plurality of vision sensors are jointly optimized to minimize differences between the optimized exposure times of the plurality of vision sensors.
  14. 根据权利要求13所述的方法，其特征在于，所述对所述多个视觉传感器的分别确定的曝光时间进行联合优化，包括：The method according to claim 13, wherein said jointly optimizing said separately determined exposure times of said plurality of vision sensors comprises:
    根据所述多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
    根据所述优化矩阵和所述多个视觉传感器的分别确定的曝光时间，确定所述多个视觉传感器的优化后的曝光时间。And determining an optimized exposure time of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors.
  15. 一种曝光的装置，其特征在于，包括：An apparatus for exposing, comprising:
    平均亮度确定模块，用于根据第一图像的像素的亮度和所述像素的亮度权重，确定所述第一图像的平均亮度；An average brightness determining module, configured to determine an average brightness of the first image according to a brightness of a pixel of the first image and a brightness weight of the pixel;
    曝光时间确定模块，用于根据所述第一图像的平均亮度，所述第一图像对应的曝光时间和第二图像的期望平均亮度，确定所述第二图像对应的曝光时间。And an exposure time determining module, configured to determine an exposure time corresponding to the second image according to an average brightness of the first image, an exposure time corresponding to the first image, and a desired average brightness of the second image.
  16. 根据权利要求15所述的装置，其特征在于，所述像素的亮度权重 包括区域权重和色度权重中的至少一项，其中，所述区域权重为与所述像素的区域关联的权重，所述色度权重为与所述像素的色度关联的权重。The device according to claim 15, wherein the luminance weight of said pixel At least one of a region weight and a chroma weight, wherein the region weight is a weight associated with a region of the pixel, the chroma weight being a weight associated with a chroma of the pixel.
  17. 根据权利要求16所述的装置，其特征在于，若所述像素的区域越接近目标对象的区域，则所述区域权重越大；和/或，若所述像素的色度越接近目标对象的色度，则所述色度权重越大。The apparatus according to claim 16, wherein if the area of the pixel is closer to the area of the target object, the area weight is larger; and/or, if the chromaticity of the pixel is closer to the target object For chrominance, the chromaticity weight is greater.
  18. 根据权利要求15至17中任一项所述的装置，其特征在于，所述第一图像的平均亮度为所述第一图像的像素的亮度的加权平均值。Apparatus according to any one of claims 15 to 17, wherein the average brightness of the first image is a weighted average of the brightness of the pixels of the first image.
  19. 根据权利要求15至18中任一项所述的装置，其特征在于，所述第二图像的期望平均亮度为预先设定的。The apparatus according to any one of claims 15 to 18, wherein the desired average brightness of the second image is preset.
  20. 根据权利要求15至19中任一项所述的装置，其特征在于，所述装置还包括：The device according to any one of claims 15 to 19, wherein the device further comprises:
    调整模块，用于根据所述第一图像中过曝光像素的百分比，调整所述第二图像的期望平均亮度。And an adjustment module, configured to adjust a desired average brightness of the second image according to a percentage of the overexposed pixels in the first image.
  21. 根据权利要求15至20中任一项所述的装置，其特征在于，所述曝光时间确定模块用于：The apparatus according to any one of claims 15 to 20, wherein the exposure time determination module is configured to:
    根据所述第一图像的平均亮度和所述第一图像对应的曝光时间，确定平均亮度导数；Determining an average brightness derivative according to an average brightness of the first image and an exposure time corresponding to the first image;
    根据所述第二图像的期望平均亮度，所述第一图像的平均亮度和所述平均亮度导数，确定曝光时间调整量；Determining an exposure time adjustment amount according to a desired average brightness of the second image, an average brightness of the first image, and the average brightness derivative;
    根据所述第一图像对应的曝光时间和所述曝光时间调整量，确定所述第二图像对应的曝光时间。And determining an exposure time corresponding to the second image according to an exposure time corresponding to the first image and the exposure time adjustment amount.
  22. 根据权利要求21所述的装置，其特征在于，所述曝光时间确定模块用于：The apparatus according to claim 21, wherein said exposure time determining module is configured to:
    所述根据所述第一图像对应的曝光时间，所述曝光时间调整量和调节系数，确定所述第二图像对应的曝光时间。And determining, according to the exposure time corresponding to the first image, the exposure time adjustment amount and the adjustment coefficient, an exposure time corresponding to the second image.
  23. 根据权利要求15至22中任一项所述的装置，其特征在于，所述第一图像为已拍摄的图像，所述第二图像为待拍摄的图像。The apparatus according to any one of claims 15 to 22, wherein the first image is a captured image and the second image is an image to be captured.
  24. 根据权利要求23所述的装置，其特征在于，所述第一图像为最新的已拍摄的图像。The apparatus of claim 23 wherein said first image is the most recent captured image.
  25. 根据权利要求15至24中任一项所述的装置，其特征在于，所述装置还包括： The device according to any one of claims 15 to 24, wherein the device further comprises:
    联合优化模块，用于对具有依赖关系的多个视觉传感器的分别确定的曝光时间进行联合优化，以使所述多个视觉传感器的优化后的曝光时间间的差异最小。The joint optimization module is configured to jointly optimize the separately determined exposure times of the plurality of vision sensors having dependencies to minimize the difference between the optimized exposure times of the plurality of vision sensors.
  26. 根据权利要求25所述的装置，其特征在于，所述联合优化模块用于：The apparatus according to claim 25, wherein said joint optimization module is configured to:
    根据所述多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
    根据所述优化矩阵和所述多个视觉传感器的分别确定的曝光时间，确定所述多个视觉传感器的优化后的曝光时间。And determining an optimized exposure time of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors.
  27. 一种曝光的装置，其特征在于，包括：An apparatus for exposing, comprising:
    确定模块，用于分别确定具有依赖关系的多个视觉传感器的曝光时间；Determining a module for respectively determining exposure times of a plurality of vision sensors having dependencies;
    联合优化模块，用于对所述多个视觉传感器的分别确定的曝光时间进行联合优化，以使所述多个视觉传感器的优化后的曝光时间间的差异最小。And a joint optimization module, configured to jointly optimize the separately determined exposure times of the plurality of vision sensors to minimize a difference between the optimized exposure times of the plurality of vision sensors.
  28. 根据权利要求27所述的装置，其特征在于，所述联合优化模块用于：The apparatus according to claim 27, wherein said joint optimization module is configured to:
    根据所述多个视觉传感器的配置，确定优化矩阵；Determining an optimization matrix according to the configuration of the plurality of vision sensors;
    根据所述优化矩阵和所述多个视觉传感器的分别确定的曝光时间，确定所述多个视觉传感器的优化后的曝光时间。And determining an optimized exposure time of the plurality of vision sensors according to the optimized matrix and the respectively determined exposure times of the plurality of vision sensors.
  29. 一种计算机***，其特征在于，包括：A computer system, comprising:
    存储器，用于存储计算机可执行指令；a memory for storing computer executable instructions;
    处理器，用于访问所述存储器，并执行所述计算机可执行指令，以进行根据权利要求1至14中任一项所述的方法中的操作。A processor for accessing the memory and executing the computer executable instructions to perform the operations of the method of any one of claims 1-14.
  30. 一种可移动设备，其特征在于，包括：A mobile device, comprising:
    根据权利要求15至28中任一项所述的曝光的装置。Apparatus for exposure according to any one of claims 15 to 28.
  31. 一种可移动设备，其特征在于，包括：A mobile device, comprising:
    根据权利要求29所述的计算机***。 A computer system according to claim 29.
PCT/CN2017/099419 2017-08-29 2017-08-29 Exposure method and apparatus, computer system, and movable device WO2019041121A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/099419 WO2019041121A1 (en) 2017-08-29 2017-08-29 Exposure method and apparatus, computer system, and movable device
CN201780004678.9A CN108521863B (en) 2017-08-29 2017-08-29 Exposure method, device, computer system and movable equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/099419 WO2019041121A1 (en) 2017-08-29 2017-08-29 Exposure method and apparatus, computer system, and movable device

Publications (1)

Publication Number Publication Date
WO2019041121A1 true WO2019041121A1 (en) 2019-03-07

Family

ID=63434072

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/099419 WO2019041121A1 (en) 2017-08-29 2017-08-29 Exposure method and apparatus, computer system, and movable device

Country Status (2)

Country Link
CN (1) CN108521863B (en)
WO (1) WO2019041121A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115334250A (en) * 2022-08-09 2022-11-11 阿波罗智能技术(北京)有限公司 Image processing method and device and electronic equipment
CN116074633A (en) * 2023-03-06 2023-05-05 宜科(天津)电子有限公司 Automatic multiple exposure method

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113473032B (en) * 2020-03-31 2024-06-18 比亚迪半导体股份有限公司 Automatic exposure adjusting method, device, equipment and storage medium
CN115022552B (en) * 2021-02-18 2024-04-16 北京石头创新科技有限公司 Image pick-up exposure method of self-walking equipment and self-walking equipment
CN113382143B (en) * 2021-06-09 2022-07-29 中国矿业大学 Automatic exposure adjusting method for binocular camera of fire-fighting robot
CN114374801A (en) * 2021-12-28 2022-04-19 苏州凌云视界智能设备有限责任公司 Method, device and equipment for determining exposure time and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090091645A1 (en) * 2007-10-03 2009-04-09 Nokia Corporation Multi-exposure pattern for enhancing dynamic range of images
US20100246989A1 (en) * 2009-03-30 2010-09-30 Amit Agrawal Multi-Image Deblurring
CN103826066A (en) * 2014-02-26 2014-05-28 芯原微电子(上海)有限公司 Automatic exposure adjusting method and system
CN104320593A (en) * 2014-11-19 2015-01-28 湖南国科微电子有限公司 Automatic exposure control method for digital photographic device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101783888A (en) * 2010-03-23 2010-07-21 中国科学院西安光学精密机械研究所 Automatic exposure method based on class histogram
US8233789B2 (en) * 2010-04-07 2012-07-31 Apple Inc. Dynamic exposure metering based on face detection
CN102316253A (en) * 2010-07-05 2012-01-11 东莞德英电子通讯设备有限公司 Stereo digital camera and synchronous shooting method
CN104917975B (en) * 2015-06-01 2018-01-05 北京空间机电研究所 A kind of adaptive automatic explosion method based on target signature
CN104917976B (en) * 2015-06-05 2018-08-21 北京大恒图像视觉有限公司 A kind of camera automatic exposure and automatic gain regulating method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090091645A1 (en) * 2007-10-03 2009-04-09 Nokia Corporation Multi-exposure pattern for enhancing dynamic range of images
US20100246989A1 (en) * 2009-03-30 2010-09-30 Amit Agrawal Multi-Image Deblurring
CN103826066A (en) * 2014-02-26 2014-05-28 芯原微电子(上海)有限公司 Automatic exposure adjusting method and system
CN104320593A (en) * 2014-11-19 2015-01-28 湖南国科微电子有限公司 Automatic exposure control method for digital photographic device

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115334250A (en) * 2022-08-09 2022-11-11 阿波罗智能技术(北京)有限公司 Image processing method and device and electronic equipment
CN115334250B (en) * 2022-08-09 2024-03-08 阿波罗智能技术(北京)有限公司 Image processing method and device and electronic equipment
CN116074633A (en) * 2023-03-06 2023-05-05 宜科(天津)电子有限公司 Automatic multiple exposure method
CN116074633B (en) * 2023-03-06 2023-08-01 天津宜科自动化股份有限公司 Automatic multiple exposure method

Also Published As

Publication number Publication date
CN108521863A (en) 2018-09-11
CN108521863B (en) 2020-10-16

Similar Documents

Publication Publication Date Title
WO2019041121A1 (en) Exposure method and apparatus, computer system, and movable device
WO2020014909A1 (en) Photographing method and device and unmanned aerial vehicle
US20170336203A1 (en) Methods and systems for remote sensing with drones and mounted sensor devices
WO2018210078A1 (en) Distance measurement method for unmanned aerial vehicle, and unmanned aerial vehicle
WO2020113423A1 (en) Target scene three-dimensional reconstruction method and system, and unmanned aerial vehicle
WO2018185939A1 (en) Imaging control device, imaging device, imaging system, mobile body, imaging control method and program
WO2022077296A1 (en) Three-dimensional reconstruction method, gimbal load, removable platform and computer-readable storage medium
WO2019061064A1 (en) Image processing method and device
US10679376B2 (en) Determining a pose of a handheld object
US20230384803A1 (en) Autonomous orbiting method and device and uav
WO2021081774A1 (en) Parameter optimization method and apparatus, control device, and aircraft
WO2021104308A1 (en) Panoramic depth measurement method, four-eye fisheye camera, and binocular fisheye camera
KR20200037502A (en) Method and apparatus of outputting pose information
JP6384000B1 (en) Control device, imaging device, imaging system, moving object, control method, and program
KR20150064761A (en) Electro device comprising transparent display and method for controlling thereof
WO2020019175A1 (en) Image processing method and apparatus, and photographing device and unmanned aerial vehicle
US20200202567A1 (en) Calibrating a machine vision camera
US20240071018A1 (en) Smooth object correction for augmented reality devices
US10884415B2 (en) Unmanned aerial vehicle light flash synchronization
CN109891188A (en) Mobile platform, camera paths generation method, program and recording medium
CN112955712A (en) Target tracking method, device and storage medium
US20210092306A1 (en) Movable body, image generation method, program, and recording medium
JP6515423B2 (en) CONTROL DEVICE, MOBILE OBJECT, CONTROL METHOD, AND PROGRAM
CN112330726B (en) Image processing method and device
WO2022016340A1 (en) Method and system for determining exposure parameters of main camera device, mobile platform, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17923759

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17923759

Country of ref document: EP

Kind code of ref document: A1