WO2010061859A1 - Stereo image processing device, stereo image processing method, and recording medium - Google Patents

Stereo image processing device, stereo image processing method, and recording medium Download PDF

Info

Publication number
WO2010061859A1
WO2010061859A1 PCT/JP2009/069886 JP2009069886W WO2010061859A1 WO 2010061859 A1 WO2010061859 A1 WO 2010061859A1 JP 2009069886 W JP2009069886 W JP 2009069886W WO 2010061859 A1 WO2010061859 A1 WO 2010061859A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
stereo
pair
row
rows
Prior art date
Application number
PCT/JP2009/069886
Other languages
French (fr)
Japanese (ja)
Inventor
神谷　俊之
弘之 柳生
小泉　博一
Original Assignee
Ｎｅｃシステムテクノロジー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ｎｅｃシステムテクノロジー株式会社 filed Critical Ｎｅｃシステムテクノロジー株式会社
Publication of WO2010061859A1 publication Critical patent/WO2010061859A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing

Definitions

  • the present invention relates to a stereo image processing apparatus, a stereo image processing method, and a recording medium.
  • stereo matching processing refers to the principle of triangulation using the parallax obtained by finding corresponding points in each image capturing the same point for two images taken from different viewpoints, so-called stereo images. To obtain the depth and shape to the target.
  • Patent Document 1 discloses a method using an area correlation method that is generally widely used. This area correlation method sets a correlation window in the left image as a template, moves the search window in the right image to calculate the cross-correlation coefficient with the template, This is a method of obtaining corresponding points by searching.
  • the movement range of the search window is limited to the direction of the epipolar line in the image, thereby shifting the position of the corresponding point in the right image for each point in the left image.
  • the epipolar line is a straight line that can be drawn as an existing range of a point corresponding to the point in one image in a stereo image.
  • the epipolar line is described in, for example, “Image Analysis Handbook” (supervised by Mikio Takagi and Yoshihisa Shimoda, published by the University of Tokyo Press, January 1991, pages 597-599).
  • the epipolar line direction is different from the scanning line direction of the stereo image.
  • the epipolar line direction can be matched with the scanning line direction of the stereo image, and rearrangement can be performed. This coordinate transformation method is described in the above “Image Analysis Handbook”.
  • the moving range of the search window for the corresponding points can be limited to the scanning line (epipolar line), so the y coordinate between the corresponding points in the left and right images is one.
  • the parallax can be expressed only by the difference in x-coordinates between corresponding points in the left and right images.
  • the present invention has been made in view of the above circumstances, and an object thereof is to provide a stereo image processing apparatus or the like that can correct a vertical shift of a stereo pair image.
  • a stereo image processing apparatus provides: A stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other; A row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image; A shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit; An image correction unit that corrects the stereo image so that the deviation calculated by the deviation determination unit is eliminated; It is characterized by providing.
  • a stereo image processing method includes: A stereo pair image data acquisition step of acquiring image data of a stereo pair image obtained by photographing a predetermined region from two different positions; A row specifying step for specifying a pair of rows having a high correlation between one image and the other image of the stereo pair image for an image region having a predetermined width parallel to the scanning line direction of the stereo pair image; A shift obtaining step for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified in the row specifying step; An image correction step of correcting the stereo image so that the shift calculated in the shift determination step is eliminated; It is characterized by providing.
  • a computer-readable recording medium on which a program according to the third aspect of the present invention is recorded Computer
  • a stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other;
  • a row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
  • a shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit;
  • An image correction unit that corrects the stereo image so that the shift calculated by the shift determination unit is eliminated; It is characterized by recording a program for making it function as.
  • the vertical shift of the stereo pair image can be corrected.
  • the stereo image processing apparatus 1 is a device that corrects a shift in the position of the left and right images with respect to an image (stereo pair image) composed of a left image and a right image obtained by photographing the same region from different positions on the left and right.
  • This stereo pair image can be obtained by, for example, the epipolar line direction according to the method described in “Image Analysis Handbook” (supervised by Mikio Takagi / Yoshihisa Shimoda, published by the University of Tokyo, January 1991, pages 597-599). Are rearranged (parallelized) so that the scanning directions of the images coincide with each other. That is, the y coordinates of corresponding points in the right image and the left image are matched to some extent.
  • FIG. 1 is a block diagram showing an internal configuration of the stereo image processing apparatus 1.
  • the stereo image processing apparatus 1 includes a control unit 11, a RAM 12, a ROM 13, and a storage unit 14.
  • the control unit 11 includes a CPU (Central Processing Unit) and the like, and executes predetermined processing based on a program stored in the ROM 13 or the storage unit 14 to control the entire stereo image processing apparatus 1. For example, the control unit 11 determines the scan line (row) having the highest correlation between the right image and the left image of the stereo pair image, and shifts to the vertical position (Y coordinate) of the scan line determined to have the high correlation (vertical). When there is a parallax, vertical parallax correction processing is performed to correct this. Details of the vertical parallax correction process will be described later.
  • CPU Central Processing Unit
  • a RAM (Random Access Memory) 12 is a volatile memory that functions as a work memory for storing a program read by the control unit 11 to execute a predetermined process and data necessary for the control unit 11 to execute the program. It is memory.
  • a ROM (Read Only Memory) 13 is a non-volatile memory that stores in advance a program for the control unit 11 to execute a predetermined process, fixed data, and the like.
  • the control unit 11 reads out a program or the like from the ROM 13 as necessary, develops it in the RAM 12, and executes predetermined processing based on the program or the like.
  • the storage unit 14 is composed of a storage device such as a hard disk drive, and stores various information.
  • the storage unit 14 stores a stereo pair image that is a target of vertical parallax correction processing.
  • FIG. 2 is a diagram illustrating an example of parameters stored in the storage unit 14.
  • the storage unit 14 stores various parameters used in vertical parallax correction processing, which will be described later, as illustrated in FIG.
  • the counter i is a counter used in the vertical parallax correction process.
  • y1, y2, and y3 the value of the Y coordinate of the row for comparing the correlation between the right image and the left image in the vertical parallax correction processing is set.
  • stereo image processing apparatus 1 can be configured using a computer or the like that is generally popular.
  • FIG. 3 is a flowchart illustrating an example of the operation of the vertical parallax correction process.
  • a description will be given with reference to FIG.
  • FIG. 4 is a diagram showing an example of a stereo pair image.
  • the control unit 11 selects one unprocessed stereo pair image, that is, one pair of a right image and a left image, as shown in FIG. 4 and stored in the storage unit 14 (step S101).
  • control unit 11 initializes the value of the counter i to 1 (step S102).
  • control unit 11 selects a set (row) of pixels whose Y coordinate value is yi from the left image (step S103). Note that this row may be a set of pixels having a height of several pixels in the Y direction.
  • control unit 11 selects a total of three rows separated by ⁇ 1 pixel from y1 and y1 from the right image (step S103). That is, in this process, three rows with Y coordinates yi-1, yi, yi + 1 from the bottom are selected from the right image (step S104).
  • FIG. 5 is a diagram for explaining processing for specifying a pair of highly correlated rows from the left image and the right image of the stereo pair image. Subsequently, as shown in FIG. 5, the control unit 11 obtains a correlation between the yi row of the left image selected in step S102 and the yi ⁇ 1, yi, yi + 1 rows of the right image selected in step S103. The right image row having the highest correlation with the yi row of the left image is specified (step S105).
  • step S105 may be performed as follows. First, using the method described in JP-A-4-299474, the control unit 11 obtains each pixel of the right image corresponding to each pixel constituting the yi row of the left image. Subsequently, the control unit 11 may identify one of the yi ⁇ 1, yi, and yi + 1 rows that most closely matches the position of each pixel of the obtained right image as the row of the right image having the highest correlation. Note that the processing in step S105 is not limited to the above example, and other matching methods may be used as long as the row with the highest correlation can be identified.
  • the control unit 11 causes the storage unit 14 to store the amount of deviation between the positions of the left and right image rows identified as having a high correlation in step S105 (step S106). For example, if the line yi of the right image is identified as the line having the highest correlation with the line yi of the left image in step S105, it is stored that there is no line shift (amount of deviation: 0). If the yi + 1 row of the right image is identified as the row having the highest correlation with the yi row of the left image in step S105, the right image is 1 upward (Y axis positive direction) with respect to the left image. It is stored that there is a pixel shift.
  • step S107 When it is determined that the value of the counter i is 3 (step S107; Yes), the control unit 11 calculates the average value of the deviation amounts stored in step S106 (that is, the deviation amounts at y1, y2, and y3). Obtained (step S109). When this process ends, the control unit 11 deletes (resets) the information indicating the amount of deviation stored in step S106 because it will not be used thereafter.
  • the control unit 11 corrects the stereo pair image (the right image and the left image) based on the average value of the shift amounts obtained in step S109 (step S110). For example, if the average value of the shift amounts obtained in step S109 indicates that the right image is shifted by 2 pixels in the upward direction, the entire right image is moved downward by 2 pixels (right image) The overall y-coordinate value is -2. Conversely, the entire left image may be moved upward by two pixels (the y coordinate value of the entire left image is incremented by +2).
  • control unit 11 determines whether or not there is an unprocessed (unselected) stereo pair image (step S111).
  • step S111 If it is determined that there is an unprocessed (unselected) stereo image (step S111; Yes), the control unit 11 moves the process to step S101, and the right image and the left image are shifted from the unprocessed stereo image. The process of correcting (Steps S101 to S110) is repeated. When it is determined that there is no unprocessed (unselected) stereo image (step S111; No), the control unit 11 ends the vertical parallax correction process.
  • the correlation is examined in the vicinity of the position (yi) corresponding to both images, and the line shift is obtained. Then, the vertical position of the stereo pair image is corrected so that the obtained line shift is eliminated. Therefore, it is possible to correct the vertical shift of the stereo pair image.
  • the present invention can be variously applied and modified.
  • the most correlated row is identified from the three rows yi-1, yi, and yi + 1 in the right image with respect to the row yi in the left image.
  • the present invention is not limited to this, and a highly correlated line can be identified from an arbitrary line.
  • FIG. 6 is a diagram for explaining processing for specifying a pair of rows having a high correlation from the left image and the right image of the stereo pair image.
  • the most correlated row may be identified from the five rows yi-2, yi-1, yi, yi + 1, yi + 2 in the right image.
  • the row of the right image for specifying the highly correlated row with respect to the row of yi of the left image may be any plurality of rows near yi and yi.
  • the amount of deviation between the left and right images is obtained and recorded in three rows y1, y2, and y3 (step S106), and the average value is obtained (step S109).
  • the number of lines to be recorded for obtaining the amount of deviation may be three or less or more.
  • the stereo pair image is described as being composed of a left image and a right image taken from the right and left.
  • the present invention is not limited to this, and the present invention is applied to a stereo pair image obtained by photographing the same region from different positions. Is applicable.
  • the central part for performing the processing of the stereo image processing apparatus 1 including the control unit 11, the RAM 12, the ROM 13, the storage unit 14, and the like can be realized using a normal computer system without using a dedicated system.
  • a computer program for executing the above operation is stored and distributed on a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.), and the computer program is installed in the computer.
  • the stereo image processing apparatus 1 that executes the above-described processing may be configured.
  • the stereo image processing apparatus 1 may be configured by storing the computer program in a storage device included in a server device on a communication network such as the Internet and downloading it by a normal computer system.
  • the functions of the stereo image processing apparatus 1 are realized by sharing of an OS (operating system) and an application program, or by cooperation between the OS and the application program, only the application program portion is stored in a recording medium or a storage device. It may be stored.
  • the computer program may be posted on a bulletin board (BBS, Bulletin Board System) on a communication network, and the computer program may be distributed via the network.
  • BSS bulletin Board System
  • the computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

With respect to image regions with a predetermined width parallel to the direction of the scan line (X-direction) of a stereo pair image which is created by capturing images of a predetermined region from two different positions and in which the direction of the scan line is in alignment with that of the epipolar line, a stereo image processing device specifies a pair of lines having a high level of correlation between the first image and the second image constituting the stereo pair image, determines the deviance (longitudinal parallax) in the longitudinal direction (Y-direction) between the first image and the second image on the basis of the respective positions of the specified pair of lines, and corrects the stereo image so as to eliminate the determined deviance.

Description

ステレオ画像処理装置、ステレオ画像処理方法、および、記録媒体Stereo image processing apparatus, stereo image processing method, and recording medium
　本発明は、ステレオ画像処理装置、ステレオ画像処理方法、および、記録媒体に関する。 The present invention relates to a stereo image processing apparatus, a stereo image processing method, and a recording medium.
　人工衛星や航空機等から得られる画像を基に、地形を示す3次元データ（ＤＥＭ（Ｄｉｇｉｔａｌ Ｅｌｅｖａｔｉｏｎ Ｍａｐ）データ）をステレオマッチングによって生成する方法が広く行われている。 2. Description of the Related Art There is a widespread method for generating three-dimensional data (DEM (Digital Elevation Map) data) indicating terrain by stereo matching based on an image obtained from an artificial satellite or an aircraft.
　ここで、ステレオマッチング処理とは、異なる視点から撮影した2枚の画像、いわゆるステレオ画像について、同一の点を撮像している各画像中の対応点を求め、その視差を用いて三角測量の原理によって対象までの奥行きや形状を求めることである。 Here, stereo matching processing refers to the principle of triangulation using the parallax obtained by finding corresponding points in each image capturing the same point for two images taken from different viewpoints, so-called stereo images. To obtain the depth and shape to the target.
　このステレオマッチング処理については既に様々な手法が提案されている。例えば、特許文献１には、一般的に広く用いられている面積相関法を用いる手法が開示されている。この面積相関法は左画像中に相関窓を設定してこれをテンプレートとし、右画像中の探索窓を動かしてテンプレートとの相互相関係数を算出し、これを一致度とみなして高いものを探索することによって対応点を得る方法である。 Various methods have already been proposed for this stereo matching process. For example, Patent Document 1 discloses a method using an area correlation method that is generally widely used. This area correlation method sets a correlation window in the left image as a template, moves the search window in the right image to calculate the cross-correlation coefficient with the template, This is a method of obtaining corresponding points by searching.
　上記の方法においては処理量を軽減するために、探索窓の移動範囲を画像中のエピポーラ線方向に限定することにより、左画像中の各点について、対応する右画像中の点の位置のずれの量、すなわち視差を得ることができる。ここで、エピポーラ線とはステレオ画像において片方の画像中のある点について、他方の画像中で当該点に対応する点の存在範囲として引くことができる直線である。エピポーラ線については、例えば、「画像解析ハンドブック」（高木幹夫・下田陽久監修、東京大学出版会刊、1991年1月、頁597－599）に記載されている。 In the above method, in order to reduce the amount of processing, the movement range of the search window is limited to the direction of the epipolar line in the image, thereby shifting the position of the corresponding point in the right image for each point in the left image. Can be obtained, that is, parallax. Here, the epipolar line is a straight line that can be drawn as an existing range of a point corresponding to the point in one image in a stereo image. The epipolar line is described in, for example, “Image Analysis Handbook” (supervised by Mikio Takagi and Yoshihisa Shimoda, published by the University of Tokyo Press, January 1991, pages 597-599).
　通常、エピポーラ線方向はステレオ画像の走査線方向とは異なる。しかし、座標変換を行うことで、エピポーラ線方向をステレオ画像の走査線方向に一致させ、再配列を行うことができる。この座標変換の方法については上記の「画像解析ハンドブック」に記載されている。 Usually, the epipolar line direction is different from the scanning line direction of the stereo image. However, by performing coordinate conversion, the epipolar line direction can be matched with the scanning line direction of the stereo image, and rearrangement can be performed. This coordinate transformation method is described in the above “Image Analysis Handbook”.
　上記のような再配列を行ったステレオ画像においては、対応点の探索窓の移動範囲を走査線（エピポーラ線）上に限定することができるため、左右画像中の対応点同士のｙ座標は一致し、視差は左右画像中の対応点同士のｘ座標の差のみで表すことが可能となる。 In the stereo image that has been rearranged as described above, the moving range of the search window for the corresponding points can be limited to the scanning line (epipolar line), so the y coordinate between the corresponding points in the left and right images is one. In addition, the parallax can be expressed only by the difference in x-coordinates between corresponding points in the left and right images.
特開平３－１６７６７８号公報Japanese Patent Laid-Open No. 3-167678
　しかしながら、ステレオ画像撮影時のカメラのぶれなどに起因して、座標変換を行ってエピポーラ線方向を画像の走査線方向に一致させる処理をしても、ステレオ画像の左右画中の対応点同士の縦方向にずれが生じてしまう虞がある。 However, due to camera shake at the time of shooting a stereo image, even if the process of coordinate conversion is performed to match the epipolar line direction with the scan line direction of the image, the corresponding points in the left and right images of the stereo image There is a possibility that a shift occurs in the vertical direction.
　本発明は上記実情に鑑みてなされたものであり、ステレオペア画像の縦方向のずれを補正することができる、ステレオ画像処理装置等を提供することを目的とする。 The present invention has been made in view of the above circumstances, and an object thereof is to provide a stereo image processing apparatus or the like that can correct a vertical shift of a stereo pair image.
　本発明の第１の観点に係るステレオ画像処理装置は、
　２つの異なる位置から所定の領域を撮影した、走査線方向とエピポーラ線方向が略一致するステレオペア画像の画像データを取得するステレオペア画像データ取得部と、
　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定部と、
　前記行特定部で特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求め部と、
　前記ズレ求め部で算出されたズレがなくなるように前記ステレオ画像を補正する画像補正部と、
　を備えることを特徴とする。 A stereo image processing apparatus according to a first aspect of the present invention provides:
A stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other;
A row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
A shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit;
An image correction unit that corrects the stereo image so that the deviation calculated by the deviation determination unit is eliminated;
It is characterized by providing.
　本発明の第２の観点に係るステレオ画像処理方法は、
　２つの異なる位置から所定の領域を撮影したステレオペア画像の画像データを取得するステレオペア画像データ取得ステップと、
　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定ステップと、
　前記行特定ステップで特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求めステップと、
　前記ズレ求めステップで算出されたズレがなくなるように前記ステレオ画像を補正する画像補正ステップと、
　を備えることを特徴とする。 A stereo image processing method according to a second aspect of the present invention includes:
A stereo pair image data acquisition step of acquiring image data of a stereo pair image obtained by photographing a predetermined region from two different positions;
A row specifying step for specifying a pair of rows having a high correlation between one image and the other image of the stereo pair image for an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
A shift obtaining step for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified in the row specifying step;
An image correction step of correcting the stereo image so that the shift calculated in the shift determination step is eliminated;
It is characterized by providing.
　本発明の第３の観点に係るプログラムを記録したコンピュータ読み取り可能な記録媒体は、
　コンピュータを、
　２つの異なる位置から所定の領域を撮影した、走査線方向とエピポーラ線方向が略一致するステレオペア画像の画像データを取得するステレオペア画像データ取得部と、
　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定部と、
　前記行特定部で特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求め部と、
　前記ズレ求め部で算出されたズレがなくなるように前記ステレオ画像を補正する画像補正部、
　として機能させるためのプログラムを記録することを特徴とする。 A computer-readable recording medium on which a program according to the third aspect of the present invention is recorded,
Computer
A stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other;
A row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
A shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit;
An image correction unit that corrects the stereo image so that the shift calculated by the shift determination unit is eliminated;
It is characterized by recording a program for making it function as.
　本発明によれば、ステレオペア画像の縦方向のずれを補正することができる。 According to the present invention, the vertical shift of the stereo pair image can be corrected.
ステレオ画像処理装置の内部構成を示すブロック図である。It is a block diagram which shows the internal structure of a stereo image processing apparatus. 記憶部に格納されているパラメータの例を示した図である。It is the figure which showed the example of the parameter stored in the memory | storage part. 縦視差補正処理の動作の一例を示すフローチャートである。It is a flowchart which shows an example of operation | movement of a vertical parallax correction process. ステレオペア画像の例を示した図である。It is the figure which showed the example of the stereo pair image. ステレオペア画像の左画像と右画像から相関の高い行のペアを特定する処理を説明するための図である。It is a figure for demonstrating the process which specifies the pair of a line with high correlation from the left image and right image of a stereo pair image. ステレオペア画像の左画像と右画像から相関の高い行のペアを特定する処理を説明するための図である。It is a figure for demonstrating the process which specifies the pair of a line with high correlation from the left image and right image of a stereo pair image.
　以下、本発明の実施の形態に係るステレオ画像処理装置１について説明する。ステレオ画像処理装置１は、左右の異なる位置から同じ領域を撮影した、左画像と右画像からなる画像（ステレオペア画像）に対して、左右画像の位置のズレを補正する装置である。
　なお、このステレオペア画像は、例えば、「画像解析ハンドブック」（高木幹夫・下田陽久監修、東京大学出版会刊、1991年1月、頁597－599）に記載されている手法により、エピポーラ線方向と画像の走査方向が一致するように再配列（平行化）がなされている画像である。即ち、右画像と左画像内の対応点同士のｙ座標はある程度一致している。 The stereo image processing apparatus 1 according to the embodiment of the present invention will be described below. The stereo image processing device 1 is a device that corrects a shift in the position of the left and right images with respect to an image (stereo pair image) composed of a left image and a right image obtained by photographing the same region from different positions on the left and right.
This stereo pair image can be obtained by, for example, the epipolar line direction according to the method described in “Image Analysis Handbook” (supervised by Mikio Takagi / Yoshihisa Shimoda, published by the University of Tokyo, January 1991, pages 597-599). Are rearranged (parallelized) so that the scanning directions of the images coincide with each other. That is, the y coordinates of corresponding points in the right image and the left image are matched to some extent.
　図１は、ステレオ画像処理装置１の内部構成を示すブロック図である。ステレオ画像処理装置１は、図１に示すように、制御部１１、ＲＡＭ１２、ＲＯＭ１３、及び、記憶部１４を含む。 FIG. 1 is a block diagram showing an internal configuration of the stereo image processing apparatus 1. As shown in FIG. 1, the stereo image processing apparatus 1 includes a control unit 11, a RAM 12, a ROM 13, and a storage unit 14.
　制御部１１は、ＣＰＵ（Central Processing Unit）等から構成され、ＲＯＭ１３や記憶部１４に格納されたプログラムに基づいて所定の処理を実行し、ステレオ画像処理装置１全体の制御を行う。
　例えば、制御部１１は、ステレオペア画像の右画像と左画像とで最も相関が高い走査ライン（行）を判別し、相関が高いと判別した走査ラインの縦位置（Ｙ座標）にずれ（縦視差）がある場合にこれを補正する縦視差補正処理を行う。なお、縦視差補正処理の詳細については後述する。 The control unit 11 includes a CPU (Central Processing Unit) and the like, and executes predetermined processing based on a program stored in the ROM 13 or the storage unit 14 to control the entire stereo image processing apparatus 1.
For example, the control unit 11 determines the scan line (row) having the highest correlation between the right image and the left image of the stereo pair image, and shifts to the vertical position (Y coordinate) of the scan line determined to have the high correlation (vertical). When there is a parallax, vertical parallax correction processing is performed to correct this. Details of the vertical parallax correction process will be described later.
　ＲＡＭ（Random Access Memory）１２は、制御部１１が所定の処理を実行するために読み出したプログラムや、制御部１１が当該プログラムを実行するために必要なデータを格納するワークメモリとして機能する揮発性メモリである。 A RAM (Random Access Memory) 12 is a volatile memory that functions as a work memory for storing a program read by the control unit 11 to execute a predetermined process and data necessary for the control unit 11 to execute the program. It is memory.
　ＲＯＭ（Read Only Memory）１３は、制御部１１が所定の処理を実行するためのプログラムや固定データ等を予め格納する不揮発性メモリである。制御部１１は、ＲＯＭ１３から必要に応じてプログラム等を読み出して、ＲＡＭ１２に展開し、当該プログラム等に基づいて所定の処理を実行する。 A ROM (Read Only Memory) 13 is a non-volatile memory that stores in advance a program for the control unit 11 to execute a predetermined process, fixed data, and the like. The control unit 11 reads out a program or the like from the ROM 13 as necessary, develops it in the RAM 12, and executes predetermined processing based on the program or the like.
　記憶部１４は、ハードディスクドライブ等の記憶装置から構成され、種々の情報を記憶する。例えば、記憶部１４は、縦視差補正処理の対象となるステレオペア画像を記憶する。 The storage unit 14 is composed of a storage device such as a hard disk drive, and stores various information. For example, the storage unit 14 stores a stereo pair image that is a target of vertical parallax correction processing.
　図２は、記憶部１４に格納されているパラメータの例を示した図である。また、記憶部１４は、図２に示すような、後述する縦視差補正処理で使用する各種のパラメータを記憶する。
　カウンタｉは、縦視差補正処理で利用されるカウンタである。
　ｙ１、ｙ２、ｙ３には、縦視差補正処理で、右画像と左画像とで相関を比較する行のＹ座標の値が設定されている。 FIG. 2 is a diagram illustrating an example of parameters stored in the storage unit 14. In addition, the storage unit 14 stores various parameters used in vertical parallax correction processing, which will be described later, as illustrated in FIG.
The counter i is a counter used in the vertical parallax correction process.
In y1, y2, and y3, the value of the Y coordinate of the row for comparing the correlation between the right image and the left image in the vertical parallax correction processing is set.
　なお、ステレオ画像処理装置１は、一般的に普及しているコンピュータ等を用いて構成することができる。 Note that the stereo image processing apparatus 1 can be configured using a computer or the like that is generally popular.
　次に、縦視差補正処理の動作について説明する。図３は、縦視差補正処理の動作の一例を示すフローチャートである。以下、本図を参照して説明する。 Next, the operation of the vertical parallax correction process will be described. FIG. 3 is a flowchart illustrating an example of the operation of the vertical parallax correction process. Hereinafter, a description will be given with reference to FIG.
　図４は、ステレオペア画像の例を示した図である。まず、制御部１１は、記憶部１４に記憶される、図４に示すような、未処理のステレオペア画像、即ち、右画像と左画像のペアを１つ選択する（ステップＳ１０１）。 FIG. 4 is a diagram showing an example of a stereo pair image. First, the control unit 11 selects one unprocessed stereo pair image, that is, one pair of a right image and a left image, as shown in FIG. 4 and stored in the storage unit 14 (step S101).
　続いて、制御部１１は、カウンタｉの値を１に初期化する（ステップＳ１０２）。 Subsequently, the control unit 11 initializes the value of the counter i to 1 (step S102).
　続いて、制御部１１は、左画像からＹ座標の値がｙｉである画素の集合（行）を選択する（ステップＳ１０３）。 なお、この行は、Ｙ方向に数画素分の高さを有する画素の集合であってもよい。 Subsequently, the control unit 11 selects a set (row) of pixels whose Y coordinate value is yi from the left image (step S103). Note that this row may be a set of pixels having a height of several pixels in the Y direction.
　続いて、制御部１１は、右画像から、ｙ１およびｙ１から±１画素ずつ離れた合計３本の行を選択する（ステップＳ１０３）。即ち、この処理で、右画像から、Ｙ座標が下からｙｉ－１、ｙｉ、ｙｉ＋１である３本の行が選択される（ステップＳ１０４）。 Subsequently, the control unit 11 selects a total of three rows separated by ± 1 pixel from y1 and y1 from the right image (step S103). That is, in this process, three rows with Y coordinates yi-1, yi, yi + 1 from the bottom are selected from the right image (step S104).
　図５は、ステレオペア画像の左画像と右画像から相関の高い行のペアを特定する処理を説明するための図である。続いて、制御部１１は、図５に示すように、ステップＳ１０２で選択した左画像のｙｉの行と、ステップＳ１０３で選択した右画像のｙｉ－１、ｙｉ、ｙｉ＋１の各行との相関を求め、左画像のｙｉの行と最も相関の高い右画像の行を特定する（ステップＳ１０５）。 FIG. 5 is a diagram for explaining processing for specifying a pair of highly correlated rows from the left image and the right image of the stereo pair image. Subsequently, as shown in FIG. 5, the control unit 11 obtains a correlation between the yi row of the left image selected in step S102 and the yi−1, yi, yi + 1 rows of the right image selected in step S103. The right image row having the highest correlation with the yi row of the left image is specified (step S105).
　このステップＳ１０５の処理は、具体的には、以下のようにすればよい。
　まず、特開平４－２９９４７４号公報に記載されている手法等を用いて、制御部１１は、左画像のｙｉの行を構成する各画素に対応する右画像の各画素を求める。続いて、制御部１１は、求めた右画像の各画素の位置と最も一致するｙｉ－１、ｙｉ、ｙｉ＋１の行の何れかを、最も相関の高い右画像の行として特定すればよい。
　なお、ステップＳ１０５の処理は上述の例に限定されず、最も相関の高い行を特定することができれば、他のマッチングの手法を利用してもよい。 Specifically, the processing in step S105 may be performed as follows.
First, using the method described in JP-A-4-299474, the control unit 11 obtains each pixel of the right image corresponding to each pixel constituting the yi row of the left image. Subsequently, the control unit 11 may identify one of the yi−1, yi, and yi + 1 rows that most closely matches the position of each pixel of the obtained right image as the row of the right image having the highest correlation.
Note that the processing in step S105 is not limited to the above example, and other matching methods may be used as long as the row with the highest correlation can be identified.
　続いて、制御部１１は、ステップＳ１０５で相関が高いと特定した左画像と右画像の行の位置のズレの量を記憶部１４に記憶させる（ステップＳ１０６）。
　例えば、ステップＳ１０５で右画像のｙｉの行が左画像のｙｉの行と最も相関が高い行として特定された場合は、行のずれは無いこと（ズレの量：０）が記憶される。また、ステップＳ１０５で右画像のｙｉ＋１の行が左画像のｙｉの行と最も相関が高い行として特定された場合は、右画像が左画像に対して上方向（Ｙ軸の正方向）に１画素ずれていることが記憶される。 Subsequently, the control unit 11 causes the storage unit 14 to store the amount of deviation between the positions of the left and right image rows identified as having a high correlation in step S105 (step S106).
For example, if the line yi of the right image is identified as the line having the highest correlation with the line yi of the left image in step S105, it is stored that there is no line shift (amount of deviation: 0). If the yi + 1 row of the right image is identified as the row having the highest correlation with the yi row of the left image in step S105, the right image is 1 upward (Y axis positive direction) with respect to the left image. It is stored that there is a pixel shift.
　続いて制御部１１は、カウンタｉの値が３であるか否かを判別する（ステップＳ１０７）。カウンタｉの値が３でないと判別した場合（ステップＳ１０７；Ｎｏ）、制御部１１は、カウンタｉを１だけインクリメントし（ステップＳ１０８）、カウンタｉ＝３となるまで、ステップＳ１０３～Ｓ１０７の処理を繰り返す。これにより、別のｙｉの位置で右画像と左画像の縦方向のずれが算出されて記憶される。 Subsequently, the control unit 11 determines whether or not the value of the counter i is 3 (step S107). When it is determined that the value of the counter i is not 3 (step S107; No), the control unit 11 increments the counter i by 1 (step S108), and performs the processing of steps S103 to S107 until the counter i = 3. repeat. Thus, the vertical shift between the right image and the left image is calculated and stored at another yi position.
　カウンタｉの値が３であると判別した場合（ステップＳ１０７；Ｙｅｓ）、制御部１１は、ステップＳ１０６で記憶したずれの量（即ち、ｙ１、ｙ２、ｙ３における各ズレの量）の平均値を求める（ステップＳ１０９）。なお、制御部１１は、この処理が終了すると、ステップＳ１０６で記憶したズレの量を示した情報は以後利用しないので消去（リセット）する。 When it is determined that the value of the counter i is 3 (step S107; Yes), the control unit 11 calculates the average value of the deviation amounts stored in step S106 (that is, the deviation amounts at y1, y2, and y3). Obtained (step S109). When this process ends, the control unit 11 deletes (resets) the information indicating the amount of deviation stored in step S106 because it will not be used thereafter.
　続いて、制御部１１は、ステップＳ１０９で求めたズレの量の平均値に基づいて、ステレオペア画像（右画像と左画像）を補正する（ステップＳ１１０）。例えば、ステップＳ１０９で求めたズレの量の平均値が、右画像が上方向に２画素ずれていることを示す場合には、右画像全体を、下方向に２画素分だけ移動させる（右画像全体のｙ座標値を－２する）。なお、逆に、左画像全体を、上方向に２画素分だけ移動させてもよい（左画像全体のｙ座標値を＋２する）。 Subsequently, the control unit 11 corrects the stereo pair image (the right image and the left image) based on the average value of the shift amounts obtained in step S109 (step S110). For example, if the average value of the shift amounts obtained in step S109 indicates that the right image is shifted by 2 pixels in the upward direction, the entire right image is moved downward by 2 pixels (right image) The overall y-coordinate value is -2. Conversely, the entire left image may be moved upward by two pixels (the y coordinate value of the entire left image is incremented by +2).
　続いて制御部１１は、未処理（未選択）のステレオペア画像が有るか否かを判別する（ステップＳ１１１）。 Subsequently, the control unit 11 determines whether or not there is an unprocessed (unselected) stereo pair image (step S111).
　未処理（未選択）のステレオ画像が有ると判別した場合（ステップＳ１１１；Ｙｅｓ）、制御部１１は、ステップＳ１０１に処理を移し、未処理のステレオ画像に対して、右画像と左画像のズレを補正する処理（ステップＳ１０１～１１０）を繰り返す。
　未処理（未選択）のステレオ画像がないと判別した場合（ステップＳ１１１；Ｎｏ）、制御部１１は、縦視差補正処理を終了する。 If it is determined that there is an unprocessed (unselected) stereo image (step S111; Yes), the control unit 11 moves the process to step S101, and the right image and the left image are shifted from the unprocessed stereo image. The process of correcting (Steps S101 to S110) is repeated.
When it is determined that there is no unprocessed (unselected) stereo image (step S111; No), the control unit 11 ends the vertical parallax correction process.
　このように、本実施形態では、平行化されたステレオペア画像において、両画像で対応する位置（ｙｉ）の近傍で相関を調べ、行のズレを求める。そして、求めた行のズレがなくなるように、ステレオペア画像の縦位置を補正する。従って、ステレオペア画像の縦方向のずれを補正することができる。 As described above, in the present embodiment, in the paralleled stereo pair images, the correlation is examined in the vicinity of the position (yi) corresponding to both images, and the line shift is obtained. Then, the vertical position of the stereo pair image is corrected so that the obtained line shift is eliminated. Therefore, it is possible to correct the vertical shift of the stereo pair image.
　なお、本発明は種々の応用、及び、変形が可能である。
　例えば、本実施形態では、ステップＳ１０６において、左画像のｙｉの行に対して右画像のｙｉ－１、ｙｉ、ｙｉ＋１の３本の行から最も相関の高い行を特定した。しかし、これに限らず、任意の行から相関の高い行を特定することもできる。図６は、ステレオペア画像の左画像と右画像から相関の高い行のペアを特定する処理を説明するための図である。例えば、図６に示すように、右画像のｙｉ－２、ｙｉ－１、ｙｉ、ｙｉ＋１、ｙｉ＋２の５本の行から最も相関の高い行を特定してもよい。要するに、左画像のｙｉの行に対して、相関の高い行を特定するための右画像の行は、ｙｉおよびｙｉ近傍の任意の複数行であればよい。 The present invention can be variously applied and modified.
For example, in the present embodiment, in step S106, the most correlated row is identified from the three rows yi-1, yi, and yi + 1 in the right image with respect to the row yi in the left image. However, the present invention is not limited to this, and a highly correlated line can be identified from an arbitrary line. FIG. 6 is a diagram for explaining processing for specifying a pair of rows having a high correlation from the left image and the right image of the stereo pair image. For example, as shown in FIG. 6, the most correlated row may be identified from the five rows yi-2, yi-1, yi, yi + 1, yi + 2 in the right image. In short, the row of the right image for specifying the highly correlated row with respect to the row of yi of the left image may be any plurality of rows near yi and yi.
　また、本実施形態では、ｙ１，ｙ２、ｙ３の３つの行で左右画像のズレの量を求めて記録し（ステップＳ１０６）、その平均値を求めた（ステップＳ１０９）。しかし、ズレの量を求めて記録する行は３本以下でも以上でもよく任意である。 In the present embodiment, the amount of deviation between the left and right images is obtained and recorded in three rows y1, y2, and y3 (step S106), and the average value is obtained (step S109). However, the number of lines to be recorded for obtaining the amount of deviation may be three or less or more.
　また、本実施形態では、ステレオペア画像が右と左から撮影した左画像と右画像とから構成されるとして説明したがこれに限らず、異なる位置から同じ領域を撮影したステレオペア画像に本発明は適用可能である。 In the present embodiment, the stereo pair image is described as being composed of a left image and a right image taken from the right and left. However, the present invention is not limited to this, and the present invention is applied to a stereo pair image obtained by photographing the same region from different positions. Is applicable.
　前記のハードウエェア構成やフローチャートは一例であり、任意に変更および修正が可能である。 The above-described hardware configuration and flowchart are examples, and can be arbitrarily changed and modified.
　制御部１１、ＲＡＭ１２、ＲＯＭ１３、および、記憶部１４などから構成されるステレオ画像処理装置１の処理を行う中心となる部分は、専用のシステムによらず、通常のコンピュータシステムを用いて実現可能である。たとえば、前記の動作を実行するためのコンピュータプログラムを、コンピュータが読み取り可能な記録媒体（フレキシブルディスク、ＣＤ－ＲＯＭ、ＤＶＤ－ＲＯＭ等）に格納して配布し、当該コンピュータプログラムをコンピュータにインストールすることにより、前記の処理を実行するステレオ画像処理装置１を構成してもよい。また、インターネット等の通信ネットワーク上のサーバ装置が有する記憶装置に当該コンピュータプログラムを格納しておき、通常のコンピュータシステムがダウンロード等することでステレオ画像処理装置１を構成してもよい。 The central part for performing the processing of the stereo image processing apparatus 1 including the control unit 11, the RAM 12, the ROM 13, the storage unit 14, and the like can be realized using a normal computer system without using a dedicated system. is there. For example, a computer program for executing the above operation is stored and distributed on a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.), and the computer program is installed in the computer. Thus, the stereo image processing apparatus 1 that executes the above-described processing may be configured. In addition, the stereo image processing apparatus 1 may be configured by storing the computer program in a storage device included in a server device on a communication network such as the Internet and downloading it by a normal computer system.
　また、ステレオ画像処理装置１の機能を、ＯＳ（オペレーティングシステム）とアプリケーションプログラムの分担、またはＯＳとアプリケーションプログラムとの協働により実現する場合などには、アプリケーションプログラム部分のみを記録媒体や記憶装置に格納してもよい。 Further, when the functions of the stereo image processing apparatus 1 are realized by sharing of an OS (operating system) and an application program, or by cooperation between the OS and the application program, only the application program portion is stored in a recording medium or a storage device. It may be stored.
　また、搬送波にコンピュータプログラムを重畳し、通信ネットワークを介して配信することも可能である。たとえば、通信ネットワーク上の掲示板(BBS, Bulletin Board System)に前記コンピュータプログラムを掲示し、ネットワークを介して前記コンピュータプログラムを配信してもよい。そして、このコンピュータプログラムを起動し、ＯＳの制御下で、他のアプリケーションプログラムと同様に実行することにより、前記の処理を実行できるように構成してもよい。 Also, it is possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board (BBS, Bulletin Board System) on a communication network, and the computer program may be distributed via the network. The computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.
　なお、本願については、日本国特許出願２００８－２９９７８１号を基礎とする優先権を主張し、本明細書中に日本国特許出願２００８－２９９７８１号の明細書、特許請求の範囲、図面全体を参照として取り込むものとする。 For this application, the priority based on Japanese Patent Application No. 2008-299781 is claimed, and in this specification, refer to the description of Japanese Patent Application No. 2008-299781, the scope of claims, and the entire drawing. Shall be taken in as
　１　ステレオ画像処理装置
１１　制御部
１２　ＲＡＭ
１３　ＲＯＭ
１４　記憶部
 
　
　 DESCRIPTION OF SYMBOLS 1 Stereo image processing apparatus 11 Control part 12 RAM
13 ROM
14 Storage unit

Claims (7)

  1. 　２つの異なる位置から所定の領域を撮影した、走査線方向とエピポーラ線方向が略一致するステレオペア画像の画像データを取得するステレオペア画像データ取得部と、
    　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定部と、
    　前記行特定部で特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求め部と、
    　前記ズレ求め部で算出されたズレがなくなるように前記ステレオ画像を補正する画像補正部と、
    　を備えることを特徴とするステレオ画像処理装置。     A stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other;
    A row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
    A shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit;
    An image correction unit that corrects the stereo image so that the deviation calculated by the deviation determination unit is eliminated;
    A stereo image processing apparatus comprising:
  2. 　前記行特定部は、前記ステレオペア画像の一方の画像の一の行の周辺位置に対応する、前記ステレオペア画像の他方の画像の複数の行のなかから、前記一の行と最も相関が高い行を該一の行と相関が高いペアとして特定する、
    　ことを特徴とする請求項１に記載のステレオ画像処理装置。     The row specifying unit has the highest correlation with the one row among a plurality of rows of the other image of the stereo pair image corresponding to a peripheral position of one row of the one image of the stereo pair image. Identify a row as a highly correlated pair with the one row,
    The stereo image processing apparatus according to claim 1.
  3. 　前記行特定部は、前記ステレオペア画像の走査線方向に平行な所定幅の複数の画像領域について、各画像領域毎に、相関が高い行のペアを特定し、
    　前記ズレ求め部は、前記行特定部で特定された各行のペア毎に、行のペアの位置に基づいてズレを求め、求めた複数のズレの平均値を前記ステレオ画像の一方の画像と他方の画像との間のズレとして求める、
    　ことを特徴とする請求項２に記載のステレオ画像処理装置。     The row specifying unit specifies a pair of rows having high correlation for each image region for a plurality of image regions having a predetermined width parallel to the scanning line direction of the stereo pair image,
    The deviation obtaining unit obtains a deviation based on the position of the pair of rows for each pair of rows identified by the row identifying unit, and calculates an average value of the obtained plural deviations from one image of the stereo image and the other As a gap between the images of
    The stereo image processing apparatus according to claim 2.
  4. 　２つの異なる位置から所定の領域を撮影したステレオペア画像の画像データを取得するステレオペア画像データ取得ステップと、
    　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定ステップと、
    　前記行特定ステップで特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求めステップと、
    　前記ズレ求めステップで算出されたズレがなくなるように前記ステレオ画像を補正する画像補正ステップと、
    　を備えることを特徴とするステレオ画像処理方法。     A stereo pair image data acquisition step of acquiring image data of a stereo pair image obtained by photographing a predetermined region from two different positions;
    A row specifying step for specifying a pair of rows having a high correlation between one image and the other image of the stereo pair image for an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
    A shift obtaining step for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified in the row specifying step;
    An image correction step of correcting the stereo image so that the shift calculated in the shift determination step is eliminated;
    A stereo image processing method comprising:
  5. 　前記行特定ステップは、前記ステレオペア画像の一方の画像の一の行の周辺位置に対応する、前記ステレオペア画像の他方の画像の複数の行のなかから、前記一の行と最も相関が高い行を該一の行と相関が高いペアとして特定する、
    　ことを特徴とする請求項４に記載のステレオ画像処理方法。     The row specifying step has the highest correlation with the one row among a plurality of rows of the other image of the stereo pair image corresponding to a peripheral position of one row of the one image of the stereo pair image. Identify a row as a highly correlated pair with the one row,
    The stereo image processing method according to claim 4.
  6. 　前記行特定ステップは、前記ステレオペア画像の走査線方向に平行な所定幅の複数の画像領域について、各画像領域毎に、相関が高い行のペアを特定し、
    　前記ズレ求めステップは、前記行特定ステップで特定された各行のペア毎に、行のペアの位置に基づいてズレを求め、求めた複数のズレの平均値を前記ステレオ画像の一方の画像と他方の画像との間のズレとして求める、
    　ことを特徴とする請求項５に記載のステレオ画像処理方法。     In the row specifying step, for a plurality of image regions having a predetermined width parallel to the scanning line direction of the stereo pair image, a pair of rows having a high correlation is specified for each image region,
    The deviation obtaining step obtains a deviation based on the position of a pair of rows for each pair of rows identified in the row identifying step, and calculates an average value of the obtained plural deviations from one image of the stereo image and the other As a gap between the images of
    The stereo image processing method according to claim 5.
  7. 　コンピュータを、
    　２つの異なる位置から所定の領域を撮影した、走査線方向とエピポーラ線方向が略一致するステレオペア画像の画像データを取得するステレオペア画像データ取得部と、
    　前記ステレオペア画像の走査線方向に平行な所定幅の画像領域について、該ステレオペア画像の一方の画像と他方の画像とで相関が高い行のペアを特定する行特定部と、
    　前記行特定部で特定された行のペアの各位置に基づいて、前記ステレオ画像の一方の画像と他方の画像との間のズレを求めるズレ求め部と、
    　前記ズレ求め部で算出されたズレがなくなるように前記ステレオ画像を補正する画像補正部、
    　として機能させることを特徴とするプログラムを記録したコンピュータ読み取り可能な記録媒体。
     
    　
    　     Computer
    A stereo pair image data acquisition unit that acquires image data of a stereo pair image in which a predetermined area is captured from two different positions and the scanning line direction and the epipolar line direction substantially coincide with each other;
    A row specifying unit for specifying a pair of rows having a high correlation between one image of the stereo pair image and the other image with respect to an image region having a predetermined width parallel to the scanning line direction of the stereo pair image;
    A shift obtaining unit for obtaining a shift between one image of the stereo image and the other image based on each position of the pair of rows specified by the row specifying unit;
    An image correction unit that corrects the stereo image so that the shift calculated by the shift determination unit is eliminated;
    A computer-readable recording medium having recorded thereon a program that is caused to function as:


PCT/JP2009/069886 2008-11-25 2009-11-25 Stereo image processing device, stereo image processing method, and recording medium WO2010061859A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2008-299781 2008-11-25
JP2008299781A JP2010128575A (en) 2008-11-25 2008-11-25 Stereo image processing device, stereo image processing method, and program

Publications (1)

Publication Number Publication Date
WO2010061859A1 true WO2010061859A1 (en) 2010-06-03

Family

ID=42225732

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2009/069886 WO2010061859A1 (en) 2008-11-25 2009-11-25 Stereo image processing device, stereo image processing method, and recording medium

Country Status (2)

Country Link
JP (1) JP2010128575A (en)
WO (1) WO2010061859A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013160602A (en) * 2012-02-03 2013-08-19 Mitsubishi Electric Corp Photographic surveying apparatus

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101142279B1 (en) 2010-10-20 2012-05-07 주식회사 아이닉스 An apparatus for aligning images in stereo vision system and the method thereof
US9380285B2 (en) 2010-12-20 2016-06-28 Samsung Display Co., Ltd. Stereo image processing method, stereo image processing device and display device
JP2013113600A (en) * 2011-11-25 2013-06-10 Sharp Corp Stereo three-dimensional measuring apparatus
JP6077425B2 (en) * 2013-09-25 2017-02-08 Kddi株式会社 Video management apparatus and program

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11203476A (en) * 1998-01-19 1999-07-30 Oki Electric Ind Co Ltd Method for aligning epipolar line and stereoscopic image device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07151525A (en) * 1993-11-30 1995-06-16 Canon Inc Image processing method and device
JPWO2004098204A1 (en) * 2003-05-02 2006-07-13 富田　誠次郎 3D image photographing method and apparatus

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11203476A (en) * 1998-01-19 1999-07-30 Oki Electric Ind Co Ltd Method for aligning epipolar line and stereoscopic image device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013160602A (en) * 2012-02-03 2013-08-19 Mitsubishi Electric Corp Photographic surveying apparatus

Also Published As

Publication number Publication date
JP2010128575A (en) 2010-06-10

Similar Documents

Publication Publication Date Title
CN109461174B (en) Video target area tracking method and video plane advertisement implanting method and system
KR102141290B1 (en) Image processing apparatus, image processing method, image processing program and storage medium
EP2811457A1 (en) Image processing method and apparatus
WO2010061859A1 (en) Stereo image processing device, stereo image processing method, and recording medium
US9633280B2 (en) Image processing apparatus, method, and storage medium for determining pixel similarities
KR20130003135A (en) Apparatus and method for capturing light field geometry using multi-view camera
WO2017207647A1 (en) Apparatus and method for performing 3d estimation based on locally determined 3d information hypotheses
US20100118125A1 (en) Method and apparatus for generating three-dimensional (3d) image data
JP5318168B2 (en) Stereoscopic image processing apparatus, stereoscopic image processing method, and program
JP2017021759A (en) Image processor, image processing method and program
WO2020017334A1 (en) Vehicle-mounted environment recognition device
JP2010152521A (en) Apparatus and method for performing stereographic processing to image
KR20110089299A (en) Stereo matching process system, stereo matching process method, and recording medium
JP2011237296A (en) Three dimensional shape measuring method, three dimensional shape measuring device, and program
US9654764B2 (en) Stereoscopic image processing device, stereoscopic image processing method, and program
JP6212021B2 (en) Distance detector
JP6395429B2 (en) Image processing apparatus, control method thereof, and storage medium
JP6473769B2 (en) Correction amount calculation device, correction device, and correction amount calculation method
US9674503B2 (en) Stereo matching apparatus using image property
US20170178351A1 (en) Method for determining missing values in a depth map, corresponding device, computer program product and non-transitory computer-readable carrier medium
US20100128973A1 (en) Stereo image processing apparatus, stereo image processing method and computer-readable recording medium
US10212412B2 (en) Method of increasing photographing speed of photographing device
JP5820716B2 (en) Image processing apparatus, image processing method, computer program, recording medium, and stereoscopic image display apparatus
JP5769248B2 (en) Stereo matching processing device, stereo matching processing method, and program
KR102122523B1 (en) Device for correcting depth map of three dimensional image and method for correcting the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09829107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09829107

Country of ref document: EP

Kind code of ref document: A1