WO2018232631A1 - 一种图像处理方法、装置、***及计算机存储介质 - Google Patents

一种图像处理方法、装置、***及计算机存储介质 Download PDF

Info

Publication number
WO2018232631A1
WO2018232631A1 PCT/CN2017/089380 CN2017089380W WO2018232631A1 WO 2018232631 A1 WO2018232631 A1 WO 2018232631A1 CN 2017089380 W CN2017089380 W CN 2017089380W WO 2018232631 A1 WO2018232631 A1 WO 2018232631A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
depth information
current
target
current image
Prior art date
Application number
PCT/CN2017/089380
Other languages
English (en)
French (fr)
Inventor
阳光
Original Assignee
深圳配天智能技术研究院有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳配天智能技术研究院有限公司 filed Critical 深圳配天智能技术研究院有限公司
Priority to CN201780036116.2A priority Critical patent/CN109429560B/zh
Priority to PCT/CN2017/089380 priority patent/WO2018232631A1/zh
Publication of WO2018232631A1 publication Critical patent/WO2018232631A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, system, and computer storage medium.
  • 3D vision is a necessary technology in vehicle-mounted automatic or assisted driving systems.
  • 3D vision is implemented based on the fusion of laser radar and vision.
  • laser radar provides low-resolution depth information.
  • Vision provides high-resolution texture information and then fuses the two to achieve 3D vision.
  • the resolution of the laser radar is low, which causes visual errors, especially in some complicated situations. For example, the lack of texture, the depth information changes dramatically (such as grid railings, etc.).
  • the object of the present invention is to provide an image processing method, device, system and computer storage medium for solving problems caused by realizing low-resolution depth information of laser radar in 3D vision in the prior art, and improving resolution of depth information. Rate, which in turn increases the resolution of the surrounding environment obtained by the moving target.
  • the technical solution adopted by the present invention to solve the above technical problem is to provide an image processing method, comprising: acquiring a current image acquired by using the target as a current time at a current time when the target moves; and calculating depth information of the current image. Adjusting the depth information of the current image by using the depth information of the previously acquired image.
  • Another technical solution adopted by the present invention to solve the above technical problem is to provide an image processing system including an image capturing device and an image processing device; the image capturing device is configured to use the target as a viewpoint when the target moves Collecting a current image of the current time; the image processing device is configured to calculate depth information of the current image when acquiring the current image, and use depth information of the previously acquired image to depth of the current image Information is adjusted.
  • Another technical solution adopted by the present invention to solve the above technical problem is to provide a vehicle, wherein the vehicle includes an image capturing device and an image processing device; and the image capturing device is configured to Observing an environment image in real time; the image processing device is configured to acquire a current image acquired by the image capturing device at a current time, calculate depth information of the current image, and utilize depth information of the previously acquired image, Adjusting the depth information of the current image.
  • an image processing apparatus including a memory and a processor for storing computer instructions configured to be executed by the processor;
  • the processor executes the computer instruction, configured to: acquire a current image acquired by the current time from the target when the target moves; calculate depth information of the current image; and use depth information of the previously acquired image And adjusting the depth information of the current image.
  • Another technical solution adopted by the present invention to solve the above technical problems is to provide a computer storage medium in which computer instructions executable by a processor are stored, the computer instructions being used to execute the image processing method described above.
  • the invention has the beneficial effects that the depth information of the current image is adjusted by using the depth information of the image at the previous moment, and the depth resolution of the current image is improved, so that the resolution of the moving target to obtain the surrounding environment becomes higher.
  • FIG. 1 is a flow chart of an embodiment of an image processing method of the present invention
  • FIG. 2 is a flowchart of step S106 in the above embodiment of the present invention.
  • 3-4 are schematic diagrams of application scenarios of the above embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an embodiment of an image processing system of the present invention.
  • FIG. 6 is a block diagram showing an embodiment of an image processing apparatus of the present invention.
  • FIG. 1 it is a flowchart of an embodiment of an image processing method of the present invention, which can be applied to a moving target to acquire an image of a surrounding environment thereof, for example, realizing 3D vision in a vehicle assisted driving, that is, moving vehicle acquisition.
  • An image of the driving environment comprising the steps of:
  • Step S102 When the target moves, acquire the current image acquired by the current time from the perspective of the target.
  • step S102 the current image is acquired from the perspective of the target, indicating that the collection device is set on the target, and when the target moves, the acquisition device moves accordingly, and the acquired image is also different with the movement, for example, the target is a vehicle.
  • the collecting device may be a laser radar.
  • Step S104 Calculate depth information of the current image.
  • Step S106 Adjust the depth information of the current image by using the depth information of the previously acquired image.
  • the previously acquired image refers to the image acquired before the current time, and may include but not limited to one or more sheets.
  • the previously acquired images are not completely identical, and thus the depth thereof.
  • the information is not completely the same, and the depth information of the current image is adjusted by the depth information of the images that are not identical, and the resolution of the depth information of the current image is improved, thereby improving the resolution of the surrounding environment that can be seen by the moving target.
  • the depth information of the current image is adjusted by using the depth information of the image at the previous time, and the depth resolution of the current image is improved, so that the resolution of the surrounding environment of the moving target is increased.
  • step S106 includes:
  • Step S1061 Find a superimposed area between the current image and the previous time image acquired at the previous moment.
  • finding a superimposed region between the current image and a previous moment image acquired at a previous moment includes: first, finding a reference object in the current image and the previous moment image, wherein the reference object is at the target The ground is stationary while moving relative to the target during the move. Then, the reference object is compared with the position of the current image and the previous time image to obtain the pose difference between the current moment and the previous moment; finally, the pose between the current image and the previous moment image is obtained according to the pose difference. Overlay area.
  • the resolution of the adjusted depth information is high, so that the moving target has a high resolution in obtaining its surrounding environment.
  • searching for the reference object in the current image and the previous time image comprises: first, acquiring a static object having the set feature, and setting the feature to the depth information and the texture information respectively satisfying the preset condition
  • the feature for example, the depth information and the texture information of a certain feature are respectively greater than the corresponding preset values, for example, street lights, buildings, plants, etc.
  • the preset condition includes a preset parameter, wherein the preset parameter may be a static object in the current image and The degree of display in the image at the previous moment, for example, in the moving direction, a certain static object whose degree of display is above 80% in the images at the two moments can be used as a reference object.
  • the preset parameter may also be a static object and The distance of the target, for example, when the static object is at a preset range in the image of the two moments, the static object can also Selecting as a reference.
  • finding the superimposed area between the current image and the previous time image acquired at the previous moment includes: if the reference object is not found in the current image and the previous time image, for example, not found here.
  • the static object with the set feature is displayed in the image at both time points with the degree of more than 80%; according to the movement parameter of the target movement, the position difference between the current time and the previous time is estimated; the current position and the position deviation are obtained according to the current position and position deviation.
  • the movement parameters in which the target moves include the moving speed and the like.
  • Step S1062 Superimpose the depth information of the previous time image into the depth information of the superimposed area of the current image to obtain the adjusted depth information of the current image.
  • the depth information of the image at the previous moment is obtained by adjusting the depth information of the image before the previous moment and the depth information of the image of the previous moment.
  • the depth information of the current image is adjusted by using the depth information of the previously acquired image, and further includes:
  • the captured image may also have errors or errors due to visual errors, resulting in the vehicle obtained by comparing the images.
  • the difference between the pose difference and the actual pose difference of the vehicle may be deviated; and the cumulative deviation of the pose of the target in the image at the previous moment is calculated, and the cumulative deviation refers to the sum of the deviations of all the moments before and immediately before the previous moment; Whether the cumulative deviation of the pose of the target in the time image exceeds the preset value; if not exceeded, the superimposed area between the current image and the previous time image is found, and if it is exceeded, the previous moment collected at the previous moment is directly The image is discarded. At this time, the depth information of the current image is not adjusted, and the depth information of the current image is used as a basis for adjusting the image acquired when the subsequent target is moved.
  • the vehicle travels to the first time. 1 and the second time Time 2, the first time Time 1 corresponds to the first position Position 1, and the first image A1 acquired at this time; the second time Time 2 corresponds to the second position Position 3, and the second image A2 acquired at this time, and at the same time, obj 1-3 are all static objects.
  • First time Time 1 and second time Time The interval of 2 can be determined by the frequency of the image acquisition device.
  • the second time Time 2 is the current time of the vehicle
  • the second position Position 3 is the current position of the vehicle
  • the second image A2 collected at the second time Time 2 is the current image
  • the first image A1 collected at the first time Time 1 is the previous time image.
  • the static object obj 2 is used as a reference object, and further by the reference object obj. 2 Aligning the positions in the second image A2 and the first image A1, the second position Position 3 of the vehicle in the second image A2 is obtained, and the vehicle is at the second time Time 2 and the first time Time 1 o'clock position difference, thus depending on the position difference and the second position Position 3, a superimposed region between the second image and the first image may be obtained. At this time, the depth information in the superimposed region in the first image may be superimposed on the depth information of the superimposed region in the second image as the second image. The depth information is such that the resolution of the depth information of the superimposed region becomes higher.
  • the depth information of the second image A2 collected is used as a reference for depth adjustment of the image acquired at a later time, and the second time is used.
  • the depth information of the second acquired image A2 is depth-adjusted for the image acquired at a later time.
  • the static object obj 3 can be traced.
  • the vehicle car1 can estimate the vehicle car1 at the second time according to the moving speed of the vehicle car1. 2 with the first moment Time At 1 o'clock, the pose is poor, so that the superimposed region between the second image A2 and the first image A1 can be obtained according to the pose difference.
  • the depth information in the superimposed region in the first image A1 can be superimposed to the second image.
  • the depth information of the superimposed region in the image A2 is taken as the depth information in the second image A2, so that the resolution of the depth information of the superimposed region becomes high.
  • the second time Time The depth information of the second image A2 collected is used as a reference for depth adjustment of the image acquired at a later time, and the second time is used.
  • the depth information of the second acquired image A2 is depth-adjusted for the image acquired at a later time.
  • the system 500 includes an image capture device 510 and an image processing device 520, wherein the image capture device is coupled to the image processing device 520.
  • the image capturing device 510 is configured to collect a current image of the current moment from a target perspective when the target moves;
  • the image processing device 520 is configured to calculate depth information of the current image when the current image is acquired, and adjust depth information of the current image by using depth information of the previously acquired image.
  • the present invention also provides a vehicle, such as the vehicle car1 of Figures 3 and 4, which includes the image capture device 510 and image processing device 520 of the above-described embodiments.
  • the image processing device 520 is as follows.
  • the image processing device 520 will be described in detail below, and will not be described here.
  • the image processing apparatus 600 can be mounted on a moving target, including a memory 610, a processor 620, and a bus 630.
  • the memory 610 is used to store computer instructions configured to be executed by the processor 620 and data that needs to be saved or cached during operation of the processor 620, such as depth information of previously acquired images.
  • the processor 620 is configured to execute by calling a computer instruction stored in the memory 610:
  • the current image acquired at the current time is acquired from the perspective of the target;
  • the depth information of the current image is adjusted using the depth information of the previously acquired image.
  • the processor 620 adjusts the depth information of the current image by using the depth information of the previously acquired image, including: searching for a superimposed area between the current image and the previous time image acquired at a previous moment. And superimposing the depth information of the superimposed area of the previous time image into the depth information of the superimposed area of the current image to obtain the adjusted depth information of the current image.
  • searching for a superimposed region between the current image and the previous moment image acquired at a previous moment includes: first, finding a reference object in the current image and the previous moment image, wherein the reference object The ground is stationary while moving relative to the target during the movement of the target.
  • finding a superimposed area between the current image and a previous time image acquired at a previous moment includes: if the reference object is not found in the current image and the previous time image; The parameter estimates the difference between the current position and the previous moment of the target; and the superimposed area between the current image and the previous time image is obtained according to the current position and the pose difference.
  • the processor 620 searches for the reference object in the current image and the previous time image, and includes: acquiring a static object having the set feature, wherein the setting feature is that the depth information and the texture information respectively satisfy the preset condition.
  • the depth information and the texture information of a certain feature are respectively greater than the corresponding preset values; determining the moving direction when the target moves; and finding a static object satisfying the preset condition in the moving direction as a reference object.
  • the processor 620 is further configured to record the deviation of the pose of the target in the image acquired at each moment, and calculate the cumulative deviation of the pose of the target in the image at the previous moment; and determine the image at the previous moment. Whether the cumulative deviation of the pose of the middle target exceeds a preset value; if not, finds a superimposed area between the current image and the previous time image. If it is exceeded, the image of the previous moment collected at the previous moment is directly discarded.
  • the processor 620 may also be referred to as a CPU (Central Processing). Unit, central processing unit).
  • Memory 610 can include read only memory and random access memory and provides instructions and data to processor 620.
  • a portion of memory 610 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the foregoing components of the mobile terminal are coupled together by a bus 630.
  • the bus 630 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus 630 in the figure.
  • Processor 620 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 620 or an instruction in a form of software.
  • the processor 620 described above may be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or discrete hardware. Component.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present invention may be implemented or carried out.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present invention may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
  • the storage medium is located in the memory 610, and the processor 620 reads the information in the memory 610 and completes the steps of the above method in combination with its hardware.
  • the present invention also provides a computer readable storage medium, which is specifically usable as a memory 610 as shown in FIG. 6, which stores computer instructions executable on the processor 620, in particular,
  • computer instructions can be executed to implement the image processing method in the above embodiment.
  • the computer readable storage medium storing computer instructions executable on the processor 630 corresponds to the steps in the above method embodiments.
  • the depth information of the current image is adjusted by using the depth information of the image at the previous time, and the depth resolution of the current image is improved, so that the resolution of the surrounding environment of the moving target is increased.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开一种图像处理方法，包括：在目标移动时，获取当前时刻以所述目标为视角所采集的当前图像；计算所述当前图像的深度信息；利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。本发明还公开对应的装置以及***。本发明有效提高深度信息的分辨率，进而提高移动目标所获取到的其周围环境的分辨率。

Description

一种图像处理方法、装置、***及计算机存储介质
【技术领域】
本发明涉及图像处理技术领域，尤其涉及一种图像处理方法、装置、***及计算机存储介质。
【背景技术】
3D视觉是目前车载自动或者辅助驾驶***中必要的一项技术，而目前，实现3D视觉，采用的方法是基于激光雷达和视觉的融合，在此方法中，激光雷达提供低分辨率的深度信息，视觉提供高分辨率的纹理信息，然后将两者进行融合，进而实现3D视觉，但是，在此方法中，激光雷达的分辨率较低，会让视觉发生错误，尤其在一些复杂的情况下，例如，缺少纹理，深度信息变化差异剧烈（如网格栏杆等）等。
【发明内容】
本发明的目的在于，针对现有技术中的实现3D视觉时激光雷达的低分辨率的深度信息而引起的问题，提供一种图像处理方法、装置、***及计算机存储介质，提高深度信息的分辨率，进而提高移动目标所获取到的其周围环境的分辨率。
本发明解决上述技术问题所采用的技术方案是提供了一种图像处理方法，包括：在目标移动时，获取当前时刻以所述目标为视角所采集的当前图像；计算所述当前图像的深度信息；利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
本发明为解决上述技术问题所采用的另一技术方案是提供了一种图像处理***，包括图像采集装置和图像处理装置；所述图像采集装置用于在目标移动时，以所述目标为视点采集当前时刻的当前图像；所述图像处理装置用于在获取到所述当前图像时，计算所述当前图像的深度信息，并利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
本发明为解决上述技术问题所采用的另一技术方案是提供了一种车辆，其中，所述车辆包括图像采集装置和图像处理装置；所述图像采集装置用于在车辆移动时，以所述车辆的视角实时采集环境图像；所述图像处理装置用于获取所述图像采集装置在当前时刻所采集的当前图像，计算所述当前图像的深度信息，并利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
本发明为解决上述技术问题所采用的另一技术方案是提供了一种图像处理装置，包括存储器和处理器；所述存储器用于存储被配置为被所述处理器执行的计算机指令；所述处理器执行所述计算机指令，用于：在目标移动时，获取当前时刻以所述目标为视角所采集的当前图像；计算所述当前图像的深度信息；以及利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
本发明为解决上述技术问题所采用的另一技术方案是提供了一种计算机存储介质，其中，存储有处理器可运行的计算机指令，所述计算机指令用于执行上述的图像处理方法。
本发明的有益效果有：利用之前时刻的图像的深度信息对当前图像的深度信息进行调整，提高当前图像的深度分辨率，进而使得移动目标获取其周围环境的分辨率变高。
【附图说明】
下面将结合附图及实施方式对本发明作进一步说明，附图中：
图1是本发明的图像处理方法一实施例的流程图；
图2是本发明的上述实施例中的步骤S106的流程图；
图3-4是本发明的上述实施例的应用场景的示意图；
图5是本发明的图像处理***一实施例的结构示意图；
图6是本发明的图像处理装置一实施例的结构示意图。
【具体实施方式】
为使本领域的技术人员更好地理解本发明的技术方案，下面结合附图和具体实施方式对本发明的技术方案做进一步详细描述。
如图1所示，是本发明的图像处理方法一实施例的流程图，该方法可应用于移动的目标获取其周围环境的图像，例如，车辆辅助驾驶中实现3D视觉，即移动的车辆获取其行驶环境的图像，该方法包括以下步骤：
步骤S102：在目标移动时，获取当前时刻以目标为视角所采集的当前图像。
在步骤S102中，以目标为视角来采集当前图像，表示采集装置设置在目标上，目标移动时，采集装置随之移动，进而获取的图像也随着移动而不相同，例如，在目标为车辆时，为了采集车辆行驶环境，采集装置可为激光雷达。
步骤S104：计算当前图像的深度信息。
步骤S106：利用之前所采集的图像的深度信息，对当前图像的深度信息进行调整。
在步骤S106中，之前所采集的图像是指在当前时刻之前所采集的图像，可以包括但不限于一张或者多张，随着目标的移动，之前所采集的图像不完全相同，进而其深度信息也不完全相同，通过这些不完全相同的图像的深度信息对当前图像的深度信息进行调整，提高当前图像的深度信息的分辨率，进而提高移动目标所能看到的周围环境的分辨率。
通过上述实施例的实施，利用之前时刻的图像的深度信息对当前图像的深度信息进行调整，提高当前图像的深度分辨率，进而使得移动目标获取其周围环境的分辨率变高。
具体地，在一个实施例中，如图2所示，步骤S106包括：
步骤S1061：查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域。
在一个实施例中，查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域包括：首先，在当前图像与前一时刻图像中查找出参照物，其中，参照物在目标移动过程中相对于目标移动时的地面静止。随后，将参照物在当前图像和前一时刻图像的位置进行比对，得到目标在当前时刻与前一时刻的位姿差；最后，根据位姿差得到当前图像与前一时刻图像之间的叠加区域。这样，在一些复杂的情况下，因为深度叠加的效果，调整出来的深度信息的分辨率高，进而使得移动目标在获取其周围环境的分辨率高。进一步地，在一个实施例中，在当前图像与前一时刻图像中查找出参照物，包括：首先，获取具有设定特征的静态物，设定特征为深度信息和纹理信息分别满足预设条件的特征，例如，某个特征的深度信息和纹理信息分别大于相应的预设值，例如，目标移动时，沿途的路灯、建筑物、植物等；随后，确定目标移动时的移动方向；最后，在移动方向上查找出满足预设条件的静态物，以作为参照物，具体地，在一个实施例中，预设条件包括一预设参数，其中，预设参数可以是静态物在当前图像和前一时刻图像中所显示程度，例如，移动方向上，在此两时刻的图像中显示程度均在80%以上的某个静态物可以作为参照物，当然，预设参数也可以是静态物与目标的距离，例如，某静态物在此两时刻图像中与目标的距离均处于一个预设范围值时，该静态物也可以被选取来作为参照物。
在另一个实施例中，查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域包括：若在当前图像与前一时刻图像中查找不出参照物，例如没有找到在此两时刻图像中显示程度都在80%以上的具有设定特征的静态物；根据目标移动的移动参数，估计得到目标在当前时刻与前一时刻的位姿差；根据当前位置与位置偏差得到当前图像与前一时刻图像之间的叠加区域。其中目标移动的移动参数包括移动速度等。
步骤S1062：将前一时刻图像的深度信息叠加至当前图像的叠加区域的深度信息中，以得到当前图像的调整后的深度信息。前一时刻图像的深度信息是由前一时刻之前的图像的深度信息与前一时刻图像的深度信息经过调整而得。
在一个实施例中，利用之前所采集图像的深度信息，对当前图像的深度信息进行调整，还包括：
记录每一时刻所采集到的图像中的目标的位姿的偏差，因为在车辆行驶过程中，可能因为视觉的错误，导致采集到的图像也发生错误或误差，从而导致通过对比图像得到的车辆位姿差与车辆的实际位姿差发生会产生偏差；并计算出前一时刻图像中目标的位姿的累积偏差，累积偏差是指前一刻以及前一刻之前的所有时刻的偏差总和；判断前一时刻图像中目标的位姿的累积偏差是否超过预设值；若没超过，查找出当前图像与前一时刻图像之间的叠加区域，若超过，则直接将前一时刻所采集的前一时刻图像丢弃，此时，不对当前图像的深度信息进行调整，将当前图像的深度信息作为后续目标移动时所采集的图像进行调整的基础。
下面将根据上述实施例的说明对本发明的其中一个应用场景进行说明。
假设目标为车辆car1，为获取高分辨率的车辆行驶环境，如图3和图4所示，分别是车辆行驶至在第一时刻Time 1和第二时刻Time 2时，第一时刻Time 1对应于第一位置Position 1，以及此时所采集的第一图像A1；第二时刻Time 2对应于第二位置Position 3，以及此时所采集的第二图像A2，同时，obj 1-3均是表示静态物。第一时刻Time 1与第二时刻Time 2的间隔可由图像采集装置的频率来决定。假设第二时刻Time 2为车辆的当前时刻，对应地，第二位置Position 3是车辆行驶的当前位置，进而在第二时刻Time 2所采集的第二图像A2为当前图像，在第一时刻Time 1时所采集的第一图像A1为前一时刻图像。
假设在处理第二图像A2时，以静态物obj 2为参照物，进而通过参照物obj 2在第二图像A2和第一图像A1中的位置，进行比对，可以得到车辆在第二图像A2中的第二位置Position 3，以及车辆在第二时刻Time 2和第一时刻Time 1时的位置差，从而根据位置差及第二位置Position 3，可以得到第二图像与第一图像之间的叠加区域，此时，将第一图像中在叠加区域的深度信息可以叠加到第二图像中的叠加区域的深度信息，作为第二图像中的深度信息，从而叠加区域的深度信息的分辨率变高。依次重复，在后续的车辆移动的过程中，第二时刻Time 2所采集的第二图像A2的深度信息作为后面时刻所采集图像进行深度调整的基准，即将利用第二时刻Time 2所采集的第二图像A2的深度信息对后面时刻所采集的图像进行深度调整。
假设在处理第二图像A2时，无法追踪到为静态物obj 1和obj 2，而在处理第一图像A1时，可以追踪到静态物obj 3，此时，根据车辆car1的移动速度来估计得到车辆car1可以估计得到车辆car1在第二时刻Time 2与第一时刻Time 1时的位姿差，从而根据位姿差，可以得到第二图像A2与第一图像A1之间的叠加区域，此时，将第一图像A1中在叠加区域的深度信息可以叠加到第二图像A2中的叠加区域的深度信息，作为第二图像A2中的深度信息，从而叠加区域的深度信息的分辨率变高。依次重复，在后续的车辆移动的过程中，第二时刻Time 2所采集的第二图像A2的深度信息作为后面时刻所采集图像进行深度调整的基准，即将利用第二时刻Time 2所采集的第二图像A2的深度信息对后面时刻所采集的图像进行深度调整。
如图5所示，是本发明的图像处理***一实施例的结构示意图，该***500包括一图像采集装置510和一图像处理装置520，其中，图像采集装置与610图像处理装置520连接。
图像采集装置510用于在目标移动时，以目标为视角采集当前时刻的当前图像；
图像处理装置520用于在获取到当前图像时，计算当前图像的深度信息，并利用之前所采集的图像的深度信息，对当前图像的深度信息进行调整。
本发明还提供一种车辆，例如图3和图4中的车辆car1，该车辆包括上述实施例中的图像采集装置510和图像处理装置520。
具体地，图像处理装置520如下述实施例，下面将对该图像处理装置520作详细说明，在此不再说明。
如图6所示，是本发明的图像处理装置一实施例的结构示意图，该图像处理装置600可安装在移动的目标上，包括存储器610、处理器620和总线630。
存储器610用于存储被配置为被处理器620执行的计算机指令以及在处理器620工作过程中所需保存或缓存的数据，例如之前所采集的图像的深度信息。
在本实施例中，处理器620通过调用存储器610存储的计算机指令，用于执行：
在目标移动时，以目标为视角获取当前时刻所采集的当前图像；
计算当前图像的深度信息；
利用之前所采集的图像的深度信息，对当前图像的深度信息进行调整。
进一步地，具体地，处理器620利用之前所采集的图像的深度信息，对当前图像的深度信息进行调整，包括：查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域；将前一时刻图像的叠加区域的深度信息叠加至当前图像的叠加区域的深度信息中，以得到当前图像的调整后的深度信息。进一步地，在一个实施例中，查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域包括：首先，在当前图像与前一时刻图像中查找出参照物，其中参照物在目标移动过程中相对于目标移动时的地面静止。随后，将参照物在当前图像和前一时刻图像的位置进行比对，得到在当前时刻与前一时刻的位姿差；最后，根据位姿差得到当前图像与前一时刻图像之间的叠加区域。在另一个实施例中，查找出当前图像与前一时刻所采集的前一时刻图像之间的叠加区域包括：若在当前图像与前一时刻图像中查找不出参照物；根据目标移动的移动参数，估计得到目标在当前时刻与前一时刻的位姿差；根据当前位置与位姿差得到当前图像与前一时刻图像之间的叠加区域。
进一步地，处理器620在当前图像与前一时刻图像中查找出参照物，包括：获取具有设定特征的静态物，其中，设定特征为深度信息和纹理信息分别满足预设条件的特征，例如，某个特征的深度信息和纹理信息分别大于相应的预设值；确定目标移动时的移动方向；在移动方向上查找出满足预设条件的静态物，以作为参照物。详细内容请参考上述方法实施例的说明。
进一步地，处理器620还用于记录在每一时刻所采集到的图像中目标的位姿的偏差，并计算出在前一时刻图像中目标的位姿的累积偏差；判断在前一时刻图像中目标的位姿的累积偏差是否超过预设值；若没超过，查找出所述当前图像与所述前一时刻图像之间的叠加区域。若超过，则直接将前一时刻所采集的前一时刻图像丢弃。
上述处理器620还可以称为CPU（Central Processing Unit，中央处理单元）。存储器610可以包括只读存储器和随机存取存储器，并向处理器620提供指令和数据。存储器610的一部分还可以包括非易失性随机存取存储器（NVRAM）。具体的应用中，移动终端的上述各个组件通过总线630耦合在一起，其中总线630除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线630。
上述本发明实施例揭示的方法可以应用于处理器620中，或者由处理器620实现。处理器620可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器620中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器620可以是通用处理器、数字信号处理器（DSP）、专用集成电路（ASIC）、现成可编程门阵列（FPGA）或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本发明实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本发明实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器610，处理器620读取存储器610中的信息，结合其硬件完成上述方法的步骤。
为此，本发明还提供了计算机可读存储介质，该计算机可读存储介质具体可用作如图6所示的存储器610，其存储有可在处理器620上运行的计算机指令，具体地，在本实施例中，计算机指令能够被执行以实现上述实施例中的图像处理方法。
需要注意的是，计算机可读存储介质存储有可在处理器630上运行的计算机指令对应于上述方法实施例中的步骤。
在本实施例中，利用之前时刻的图像的深度信息对当前图像的深度信息进行调整，提高当前图像的深度分辨率，进而使得移动目标获取其周围环境的分辨率变高。
以上仅为本发明的实施方式，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围。

Claims (15)

  1. 一种图像处理方法，其中，包括：
    在目标移动时，获取当前时刻以所述目标为视角所采集的当前图像；
    计算所述当前图像的深度信息；以及
    利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
  2. 根据权利要求1中所述的方法，其中，所述利用之前所采集图像的深度信息，对所述当前图像的深度信息进行调整，包括：
    查找出所述当前图像与前一时刻图像之间的叠加区域；以及
    将所述前一时刻图像的叠加区域的深度信息叠加至所述当前图像的叠加区域的深度信息中，以得到所述当前图像的调整后的深度信息。
  3. 根据权利要求2中所述的方法，其中，
    所述查找出所述当前图像与前一时刻图像之间的叠加区域，包括：
    在所述当前图像与所述前一时刻图像中查找出参照物，其中所述参照物在所述目标移动过程中相对于所述目标移动时的地面静止；
    将所述参照物在当前图像和前一时刻图像的位置进行比对，得到所述目标在当前时刻与前一时刻的位姿差；以及
    根据所述位姿差得到所述当前图像与所述前一时刻图像之间的叠加区域。
  4. 根据权利要求3中所述的方法，其中，
    所述在所述当前图像与所述前一时刻图像中查找出参照物，包括：
    获取具有设定特征的静态物；
    确定所述目标移动时的移动方向；以及
    在所述移动方向上查找出满足预设条件的静态物，以作为所述参照物。
  5. 根据权利要求2中所述的方法，其中，
    所述查找出所述当前图像与前一时刻图像之间的叠加区域，包括：
    若在所述当前图像与所述前一时刻图像中查找不出参照物，根据所述目标移动的移动参数，估计得到所述目标在当前时刻与前一时刻的位姿差；
    根据所述位姿差得到所述当前图像与所述前一时刻图像之间的叠加区域。
  6. 根据权利要求2中所述的方法，其中，
    所述利用之前所采集图像的深度信息，对所述当前图像的深度信息进行调整，还包括：
    记录在每一时刻所采集到的图像中所述目标的位姿的偏差，并计算出在所述前一时刻图像中所述目标的位姿的累积偏差；
    判断在所述前一时刻图像中所述目标的位姿的累积偏差是否超过预设值；
    若没超过，查找出所述当前图像与所述前一时刻图像之间的叠加区域。
  7. 一种图像处理***，其中，包括图像采集装置和图像处理装置；
    所述图像采集装置用于在目标移动时，以所述目标为视点采集当前时刻的当前图像；
    所述图像处理装置用于在获取到所述当前图像时，计算所述当前图像的深度信息，并利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
  8. 一种车辆，其中，所述车辆包括图像采集装置和图像处理装置；
    所述图像采集装置用于在车辆移动时，以所述车辆的视角实时采集环境图像；
    所述图像处理装置用于获取所述图像采集装置在当前时刻所采集的当前图像，计算所述当前图像的深度信息，并利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
  9. 一种图像处理装置，其中，包括存储器和处理器；
    所述存储器用于存储被配置为被所述处理器执行的计算机指令；
    所述处理器执行所述计算机指令，用于：
    在目标移动时，获取当前时刻以所述目标为视角所采集的当前图像；
    计算所述当前图像的深度信息；以及
    利用之前所采集的图像的深度信息，对所述当前图像的深度信息进行调整。
  10. 根据权利要求9所述的图像处理装置，其中，
    所述处理器执行利用之前所采集图像的深度信息，对所述当前图像的深度信息进行调整，包括：
    查找出所述当前图像与前一时刻所采集的前一时刻图像之间的叠加区域；
    将所述前一时刻图像的叠加区域的深度信息叠加至所述当前图像的叠加区域的深度信息中，以得到所述当前图像的调整后的深度信息。
  11. 根据权利要求10所述的图像处理装置，其中，
    所述处理器执行查找出所述当前图像与前一时刻所采集的前一时刻图像之间的叠加区域，包括：
    在所述当前图像与所述前一时刻图像中查找出参照物，其中所述参照物在所述目标移动过程中相对于所述目标移动时的地面静止；
    将所述参照物在当前图像和前一时刻图像的位置进行比对，得到所述目标在当前时刻与前一时刻的位姿差；
    根据所述位姿差得到所述当前图像与所述前一时刻图像之间的叠加区域。
  12. 根据权利要求11所述的图像处理装置，其中，
    所述处理器执行在所述当前图像与所述前一时刻图像中查找出参照物，包括：
    获取具有设定特征的静态物；
    确定所述目标移动时的移动方向；
    在所述移动方向上查找出满足预设条件的静态物，以作为所述参照物。
  13. 根据权利要求10所述的图像处理装置，其中，
    所述处理器执行所述查找出所述当前图像与前一时刻所采集的前一时刻图像之间的叠加区域，包括：
    若在所述当前图像与所述前一时刻图像中查找不出参照物，根据所述目标移动的移动参数，估计得到所述目标在当前时刻与前一时刻的位姿差；
    根据所述位姿差得到所述当前图像与所述前一时刻图像之间的叠加区域。
  14. 根据权利要求10所述的图像处理装置，其中，
    所述处理器执行利用之前所采集图像的深度信息，对所述当前图像的深度信息进行调整，还包括：
    记录在每一时刻所采集到的图像中所述目标的位姿的偏差，并计算出在所述前一时刻图像中所述目标的位姿的累积偏差；
    判断在所述前一时刻图像中所述目标的位姿的累积偏差是否超过预设值；
    若没超过，查找出所述当前图像与所述前一时刻图像之间的叠加区域。
  15. 一种计算机存储介质，其中，存储有处理器可运行的计算机指令，所述计算机指令用于执行权利要求1至6任一项所述的图像处理方法。
PCT/CN2017/089380 2017-06-21 2017-06-21 一种图像处理方法、装置、***及计算机存储介质 WO2018232631A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201780036116.2A CN109429560B (zh) 2017-06-21 2017-06-21 一种图像处理方法、装置、***及计算机存储介质
PCT/CN2017/089380 WO2018232631A1 (zh) 2017-06-21 2017-06-21 一种图像处理方法、装置、***及计算机存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/089380 WO2018232631A1 (zh) 2017-06-21 2017-06-21 一种图像处理方法、装置、***及计算机存储介质

Publications (1)

Publication Number Publication Date
WO2018232631A1 true WO2018232631A1 (zh) 2018-12-27

Family

ID=64737471

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/089380 WO2018232631A1 (zh) 2017-06-21 2017-06-21 一种图像处理方法、装置、***及计算机存储介质

Country Status (2)

Country Link
CN (1) CN109429560B (zh)
WO (1) WO2018232631A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101577795A (zh) * 2009-06-17 2009-11-11 深圳华为通信技术有限公司 一种实现全景图像的实时预览的方法和装置
CN102037325A (zh) * 2008-07-31 2011-04-27 电子地图有限公司 用于以3d显示导航数据的计算机布置及方法
US9137511B1 (en) * 2011-12-15 2015-09-15 Rawles Llc 3D modeling with depth camera and surface normals
CN105488459A (zh) * 2015-11-23 2016-04-13 上海汽车集团股份有限公司 车载3d道路实时重构方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101051386B (zh) * 2007-05-23 2010-12-08 北京航空航天大学 多幅深度图像的精确配准方法
US9699434B2 (en) * 2009-10-07 2017-07-04 Samsung Electronics Co., Ltd. Apparatus and method for adjusting depth
TWI455062B (zh) * 2011-04-26 2014-10-01 Univ Nat Cheng Kung 三維視訊內容產生方法
JP6097150B2 (ja) * 2013-05-24 2017-03-15 ソニーセミコンダクタソリューションズ株式会社 画像処理装置、画像処理方法およびプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102037325A (zh) * 2008-07-31 2011-04-27 电子地图有限公司 用于以3d显示导航数据的计算机布置及方法
CN101577795A (zh) * 2009-06-17 2009-11-11 深圳华为通信技术有限公司 一种实现全景图像的实时预览的方法和装置
US9137511B1 (en) * 2011-12-15 2015-09-15 Rawles Llc 3D modeling with depth camera and surface normals
CN105488459A (zh) * 2015-11-23 2016-04-13 上海汽车集团股份有限公司 车载3d道路实时重构方法及装置

Also Published As

Publication number Publication date
CN109429560B (zh) 2020-11-27
CN109429560A (zh) 2019-03-05

Similar Documents

Publication Publication Date Title
JP4172554B2 (ja) ステレオカメラの調整装置
CN110567469B (zh) 视觉定位方法、装置、电子设备及***
US10260862B2 (en) Pose estimation using sensors
US20080285799A1 (en) Apparatus and method for detecting obstacle through stereovision
CN112529966B (zh) 一种车载环视***的在线标定方法及其车载环视***
WO2022267481A1 (zh) 多摄像设备的外参标定设备和方法、存储介质、电子设备
JP2014074632A (ja) 車載ステレオカメラの校正装置及び校正方法
JP2014101075A (ja) 画像処理装置、画像処理方法及びプログラム
CN101383899A (zh) 一种空基平台悬停视频稳像方法
JP2001082955A (ja) ステレオ画像の位置ずれ調整装置
CN110163963B (zh) 一种基于slam的建图装置和建图方法
WO2020063058A1 (zh) 一种多自由度可动视觉***的标定方法
CN113012197A (zh) 一种适用于动态交通场景的双目视觉里程计定位方法
CN112017246A (zh) 一种基于逆透视变换的图像获取方法及装置
CN113111715A (zh) 一种无人机目标跟踪与信息采集***及方法
CN116704048B (zh) 一种双光配准方法
WO2024131200A1 (zh) 基于单目视觉的车辆3d定位方法、装置及汽车
CN116749198A (zh) 一种基于双目立体视觉引导机械臂抓取方法
JP2003304561A (ja) ステレオ画像処理装置
CN113781576A (zh) 多自由度位姿实时调整的双目视觉检测***、方法、装置
WO2018232631A1 (zh) 一种图像处理方法、装置、***及计算机存储介质
CN105530419B (zh) 图像采集***、图像采集处理***和图像采集处理方法
CN116704046A (zh) 一种跨镜图像匹配方法及装置
WO2023184869A1 (zh) 室内停车场的语义地图构建及定位方法和装置
JP3399655B2 (ja) 両眼立体視装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17914984

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17914984

Country of ref document: EP

Kind code of ref document: A1