WO2009124663A1 - Semi-global correspondence search in stereo images - Google Patents

Semi-global correspondence search in stereo images Download PDF

Info

Publication number
WO2009124663A1
WO2009124663A1 PCT/EP2009/002228 EP2009002228W WO2009124663A1 WO 2009124663 A1 WO2009124663 A1 WO 2009124663A1 EP 2009002228 W EP2009002228 W EP 2009002228W WO 2009124663 A1 WO2009124663 A1 WO 2009124663A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
costs
pixel
accumulated
dimensional paths
Prior art date
Application number
PCT/EP2009/002228
Other languages
German (de)
French (fr)
Inventor
Stefan Hahn
Original Assignee
Daimler Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Daimler Ag filed Critical Daimler Ag
Publication of WO2009124663A1 publication Critical patent/WO2009124663A1/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G06T7/596—Depth or shape recovery from multiple images from stereo images from three or more stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows

Definitions

  • the invention relates to a method for determining correspondences of pixels in at least two stereoscopically recorded images according to the preamble of claim 1.
  • the invention is therefore based on the object to provide an improved method for determining correspondences of pixels in stereoscopically recorded images, in which a memory requirement is reduced.
  • a calculation of costs of dissimilarity is performed for each pixel of one of the images based on intensities of the pixel and a pixel of the other image regarded as potentially corresponding. Along a number of one-dimensional paths that flow into the pixel, the costs are accumulated. From the pixels of the other image considered as potentially corresponding, the one to create a disparity map is selected in which a global energy is at least minimizing the cost of dissimilarity.
  • the image for determining the correspondences is subdivided into image sections of size (n + 1) x (m + 1) in an inventive manner.
  • not all accumulated costs are stored, as is conventional in the art, but only the costs accumulated in each of the one-dimensional paths for every nth row and every mth column of the picture.
  • the storage takes place for example in an external memory, while the calculation for example, in an FPGA.
  • one of the image sections is loaded in each case.
  • the stored accumulated costs which relate to an edge of the image section are loaded.
  • the costs accumulated in each of the one-dimensional paths for an interior of the image section are recalculated.
  • the computation outlay increases by about a factor of 2, but the external memory bandwidth can be reduced by at least the factor m / 2 to factor m, so that less memory is required.
  • the external memory bandwidth can be reduced by at least the factor m / 2 to factor m, so that less memory is required.
  • both the cost of the memory itself and the power consumption of the memory are reduced.
  • the prerequisite is that an internal memory of a microprocessor or an FPGA, in which the calculation is performed, is sufficiently large for the calculation of all one-dimensional paths for the selected size of the image section. If necessary, adjust the size of the image section.
  • Fig. 1 is a schematic representation of an image with a pixel and eight one-dimensional paths that open in the pixel, and
  • Fig. 2 is a diagram of a disparity of a pixel in dependence on a direction of the one-dimensional path.
  • the figure shows a picture 1 and a pixel p.
  • the other image 1 'and the pixel p' regarded as potentially corresponding are not shown.
  • the method is used to determine correspondences of pixels (pixels p, p ') in at least two stereoscopically recorded images 1, 1'.
  • the cost of disparity is the cost of matching p to d.
  • the one for creating a disparity map D (p) is selected in which a global energy is at least comprising the cost C (p, d) of dissimilarity.
  • the global energy can too Strafterme Pl, P2 are included, which take into account the changes of adjacent disparities to each other, as is clear from Figure 2 and as shown in the following formula:
  • a small penalty term pl in the determination of global energy is taken into account when the disparity d between adjacent pixels p, p 'varies slightly and a large penalty term p2, when an abrupt change of the disparity d between adjacent pixels p, p 1 is present. This is also called a smoothness constraint.
  • the image 1 is subdivided into image sections of size (n + l) x (m + l) in order to determine the correspondences.
  • the costs S (p, d) accumulated in each of the one-dimensional paths L for every nth row and every mth column of the picture 1 are stored in an accumulated cost matrix, the complexity of the algorithm relating to the Time 0 (WHD) is (WHD - width, height, disparity rank, width, height, disparity range).
  • the storage takes place, for example, in an external memory, while the calculation takes place, for example, in an FPGA or a microcontroller. Subsequently, one of the image sections is loaded in each case. In this case, the stored accumulated costs S (p, d) which relate to an edge of the image section are loaded. The costs S (p, d) accumulated in each of the one-dimensional paths L for an interior of the image section are recalculated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to a method for determining correspondence of image points (p, p') in at least two stereoscopically produced images (1, 1'), wherein a calculation is performed for each pixel (p) of one of the images (1) of the costs (C(p,d)) of a dissimilarity in the intensities of the pixel (p) and of a potentially corresponding pixel (p') of the other image (1'), wherein the costs (C(p,d)) are accumulated along a number of one-dimensional paths (L) opening on the pixel (p), where that pixel is selected for producing a disparity map (D(p)) from the pixels (p') of the other image (1') considered to be potentially corresponding that has a minimal global energy comprising at least the costs (C(p,d)) of the dissimilarity, wherein the image (1) is divided into image segment of size (n+1)x(n+1) for determining the correspondences, wherein the costs (S(p,d)) accumulated in each of the one-dimensional paths (L) are stored for each nth line and each mth column, wherein one each of the image segments is loaded, wherein the accumulated costs (S(p,d)) stored for an edge of the image segment is loaded, wherein the costs (S(p,d)) accumulated in each of the one-dimensional paths (L) for an internal of the image segment are newly calculated.

Description

SEMI-GLOBALE KORRESPONDENZSUCHE IN STEREOBILDERN SEMI GLOBAL CORRESPONDENSE SEARCH IN STEREO IMPORTERS
Die Erfindung betrifft ein Verfahren zur Bestimmung von Korrespondenzen von Bildpunkten in mindestens zwei stereoskopisch aufgenommenen Bildern gemäß dem Oberbegriff des Anspruchs 1.The invention relates to a method for determining correspondences of pixels in at least two stereoscopically recorded images according to the preamble of claim 1.
Die Bestimmung von Korrespondenzen von Pixeln, auch Bildpunkte genannt, in stereoskopisch aufgenommenen Bildern ist ein Standardproblem der Bildverarbeitung. Zur Lösung dieses Problems sind bereits zahlreiche Algorithmen bekannt geworden. Zur Lösung werden häufig einschränkende Annahmen getroffen, z. B. Epipolar Constraint, Ordering Constraint, Smoothness Assumption, Uniqueness Constraint. Ein häufig auftretendes Problem ist die unterschiedliche Helligkeit, auch Intensität genannt, korrespondierender Pixel in den stereoskopisch aufgenommenen Bildern. EineThe determination of correspondences of pixels, also called pixels, in stereoscopically recorded images is a standard problem of image processing. Numerous algorithms have already become known for solving this problem. To solve often restrictive assumptions are made, for. Epipolar Constraint, Ordering Constraint, Smoothness Assumption, Uniqueness Constraint. A common problem is the different brightness, also called intensity, of corresponding pixels in the stereoscopically recorded images. A
Korrespondenzbildung wird in diesem Fall dadurch erschwert, dass für globale Stereoverfahren meist ein pixelbasiertes Ähnlichkeitskriterium verwendet wird, das sensitiv auf unterschiedliche Helligkeiten reagiert. In (H. Hirschmüller, Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR105) - Volume 2, pp. 807-814) wird Transinformation (mutual Information) als Ähnlichkeitskriterium, das globale Helligkeitsschwankungen kompensiert, beschrieben. Zur Tiefenberechnung von Eingangsbildern in Embedded-Echtzeit- Systemen muss der Algorithmus in programmierbarer Hardware oder anderer Spezialhardware implementiert und diese Hardware mit externem Speicher versehen werden. Um die Leistungsfähigkeit des Verfahrens hoch und die elektrische Leistungsaufnahme gering zu halten, muss die Speicherbandbreite minimiert werden.Correspondence formation in this case is made more difficult by the fact that global stereo methods usually use a pixel-based similarity criterion that reacts sensitively to different brightnesses. In (H. Hirschmuller, Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1 05) - Volume 2, pp. 807-814), transinformation (mutual Information) as a similarity criterion that compensates for global brightness variations. For depth calculation of input images in embedded real-time systems, the algorithm must be in programmable hardware or other specialized hardware, and external memory is added to this hardware. To keep the performance of the process high and the electrical power consumption low, the memory bandwidth must be minimized.
Der Erfindung liegt daher die Aufgabe zugrunde, ein verbessertes Verfahren zur Bestimmung von Korrespondenzen von Bildpunkten in stereoskopisch aufgenommenen Bildern anzugeben, bei dem ein Speicherbedarf reduziert ist.The invention is therefore based on the object to provide an improved method for determining correspondences of pixels in stereoscopically recorded images, in which a memory requirement is reduced.
Die Aufgabe wird erfindungsgemäß gelöst durch ein Verfahren mit den Merkmalen des Anspruch 1.The object is achieved by a method having the features of claim 1.
Vorteilhafte Weiterbildungen der Erfindung sind Gegenstand der Unteransprüche.Advantageous developments of the invention are the subject of the dependent claims.
Bei dem Verfahren zur Bestimmung von Korrespondenzen von Bildpunkten in mindestens zwei stereoskopisch aufgenommenen Bildern wird für jedes Pixel eines der Bilder eine Berechnung von Kosten einer Unähnlichkeit anhand von Intensitäten des Pixels und eines als potentiell korrespondierend betrachteten Pixels des anderen Bildes durchgeführt. Entlang einer Anzahl von eindimensionalen Pfaden, die im Pixel münden, erfolgt eine Akkumulierung der Kosten. Aus den als potentiell korrespondierend betrachteten Pixeln des anderen Bildes wird dasjenige zur Erstellung einer Disparitätskarte ausgewählt, bei dem eine globale Energie minimal ist, die zumindest die Kosten der Unähnlichkeit umfasst. Dabei wird in erfinderischer Weise das Bild zur Bestimmung der Korrespondenzen in Bildausschnitte der Größe (n+l)x(m+l) unterteilt. Zur Weiterberechnung werden nicht alle akkumulierten Kosten gespeichert, wie im Stand der Technik gebräuchlich, sondern nur die in jedem der eindimensionalen Pfade akkumulierten Kosten für jede n-te Zeile und jede m-te Spalte des Bildes. Die Speicherung erfolgt zum Beispiel in einem externen Speicher, während die Berechnung beispielsweise in einem FPGA stattfindet. Anschließend wird jeweils einer der Bildausschnitte geladen. Dabei werden die gespeicherten akkumulierten Kosten, die einen Rand des Bildausschnitts betreffen, geladen. Die in jedem der eindimensionalen Pfade akkumulierten Kosten für ein Inneres des Bildausschnitts werden neu berechnet.In the method for determining correspondences of pixels in at least two stereoscopically recorded images, a calculation of costs of dissimilarity is performed for each pixel of one of the images based on intensities of the pixel and a pixel of the other image regarded as potentially corresponding. Along a number of one-dimensional paths that flow into the pixel, the costs are accumulated. From the pixels of the other image considered as potentially corresponding, the one to create a disparity map is selected in which a global energy is at least minimizing the cost of dissimilarity. In the process, the image for determining the correspondences is subdivided into image sections of size (n + 1) x (m + 1) in an inventive manner. For further calculation, not all accumulated costs are stored, as is conventional in the art, but only the costs accumulated in each of the one-dimensional paths for every nth row and every mth column of the picture. The storage takes place for example in an external memory, while the calculation for example, in an FPGA. Subsequently, one of the image sections is loaded in each case. In this case, the stored accumulated costs which relate to an edge of the image section are loaded. The costs accumulated in each of the one-dimensional paths for an interior of the image section are recalculated.
Im Vergleich mit dem aus dem Stand der Technik bekannten Speichern aller akkumulierten Kosten erhöht sich zwar der Berechnungsaufwand um etwa einen Faktor 2, jedoch kann die externe Speicherbandbreite mindestens um den Faktor m/2 bis Faktor m reduziert werden, so dass weniger Speicher benötigt wird. Bei weniger installiertem Speicher sinken sowohl die Kosten für den Speicher selbst als auch für die Leistungsaufnahme des Speichers. Voraussetzung ist, dass ein interner Speicher eines Mikroprozessors oder eines FPGA, in dem die Berechnung durchgeführt wird, hinreichend groß für die Berechnung aller eindimensionalen Pfade für die gewählte Größe des Bildausschnitts ist. Gegebenenfalls ist die Größe des Bildausschnitts anzupassen.Compared with the storage of all accumulated costs known from the prior art, the computation outlay increases by about a factor of 2, but the external memory bandwidth can be reduced by at least the factor m / 2 to factor m, so that less memory is required. With less memory installed, both the cost of the memory itself and the power consumption of the memory are reduced. The prerequisite is that an internal memory of a microprocessor or an FPGA, in which the calculation is performed, is sufficiently large for the calculation of all one-dimensional paths for the selected size of the image section. If necessary, adjust the size of the image section.
Im Folgenden werden Ausführungsbeispiele der Erfindung anhand von Zeichnungen näher erläutert.Embodiments of the invention are explained in more detail below with reference to drawings.
Dabei zeigt:Showing:
Fig. 1 eine schematische Darstellung eines Bildes mit einem Pixel und acht eindimensionalen Pfaden, die in dem Pixel münden, undFig. 1 is a schematic representation of an image with a pixel and eight one-dimensional paths that open in the pixel, and
Fig. 2 ein Diagramm einer Disparität eines Pixels in Abhängigkeit von einer Richtung des eindimensionalen Pfades.Fig. 2 is a diagram of a disparity of a pixel in dependence on a direction of the one-dimensional path.
Die Figur zeigt ein Bild 1 und einen Pixel p. Das andere Bild 1' und der als potentiell korrespondierend betrachtete Pixel p' sind nicht gezeigt. Zur Veranschaulichung lässt sich das Verfahren mit dem Übereinanderlegen der Bilder 1 und 1' und dem Verschieben der Bilder 1, 1' relativ zueinander zur Ermittlung einer Übereinstimmung in einem Pixel p, p1 vergleichen.The figure shows a picture 1 and a pixel p. The other image 1 'and the pixel p' regarded as potentially corresponding are not shown. By way of illustration can be Compare the method with the superimposition of the images 1 and 1 'and the shifting of the images 1, 1' relative to each other to determine a match in a pixel p, p 1 .
Das Verfahren wird zur Bestimmung von Korrespondenzen von Bildpunkten (Pixeln p, p') in mindestens zwei stereoskopisch aufgenommenen Bildern 1, 1' angewandt. Dabei wird für jedes Pixel p eines der Bilder 1 eine Berechnung der Kosten der Unähnlichkeit C(p, d) (= Kosten der Disparität) zu einem als potentiell korrespondierend betrachteten Pixel p' des anderen Bildes I1 durchgeführt. Dies geschieht beispielsweise anhand von Intensitäten des Pixels p und des als potentiell korrespondierend betrachteten Pixels p' des anderen Bildes I1. Mit anderen Worten: Kosten der Disparität sind Kosten, die beim Matchen von p auf d entstehen.The method is used to determine correspondences of pixels (pixels p, p ') in at least two stereoscopically recorded images 1, 1'. In this case, for each pixel p of one of the images 1, a calculation of the costs of the dissimilarity C (p, d) (= cost of disparity) to a pixel p 'of the other image I 1 regarded as potentially corresponding is carried out. This occurs, for example, on the basis of intensities of the pixel p and of the pixel p 'of the other image I 1 considered as potentially corresponding. In other words, the cost of disparity is the cost of matching p to d.
Entlang einer Anzahl von eindimensionalen Pfaden L, die aus verschiedenen Richtungen r im Pixel p münden, erfolgt eine Akkumulierung der Kosten C(p,d) . Dabei werden die niedrigsten Kosten C(p,d) entlang des eindimensionalen Pfades L fortgepflanzt, ähnlich wie bei dynamischer Programmierung (ohne Zurückverfolgung) .Along a number of one-dimensional paths L, which result from different directions r in the pixel p, an accumulation of the costs C (p, d) takes place. Here, the lowest cost C (p, d) is propagated along the one-dimensional path L, similar to dynamic programming (without traceability).
In Figur 1 werden acht eindimensionale Pfade L zugrunde gelegt. Es ist aber auch eine andere Anzahl denkbar, beispielsweise sechzehn. Die akkumulierten Kosten S (p, d) werden gemäß folgender Formel ermittelt:In FIG. 1, eight one-dimensional paths L are used. But it is also conceivable another number, for example, sixteen. The accumulated costs S (p, d) are determined according to the following formula:
S{p,d)=∑Lr{p,d) [1] rS {p, d) = ΣL r {p, d) [1] r
Aus den als potentiell korrespondierend betrachteten Pixel p1 des anderen Bildes 1' wird dasjenige zur Erstellung einer Disparitätskarte D(p) ausgewählt, bei dem eine globale Energie minimal ist, die zumindest die Kosten C(p,d) der Unähnlichkeit umfasst. Die globale Energie kann auch Strafterme Pl, P2 enthalten, die die Änderungen einander benachbarter Disparitäten zueinander berücksichtigen, wie aus Figur 2 deutlich wird und wie in folgender Formel gezeigt ist :From the pixels p 1 of the other image 1 'regarded as potentially corresponding, the one for creating a disparity map D (p) is selected in which a global energy is at least comprising the cost C (p, d) of dissimilarity. The global energy can too Strafterme Pl, P2 are included, which take into account the changes of adjacent disparities to each other, as is clear from Figure 2 and as shown in the following formula:
Figure imgf000007_0001
Figure imgf000007_0001
Beispielsweise wird bei der Bestimmung der globalen Energie ein kleiner Strafterm pl berücksichtigt, wenn sich die Disparität d zwischen benachbarten Pixeln p, p' geringfügig ändert und ein großer Strafterm p2, wenn eine sprungartige Änderung der Disparität d zwischen benachbarten Pixeln p, p1 vorliegt. Man spricht hierbei auch von einer Glattheitsbeschränkung (smoothness constraint) .For example, a small penalty term pl in the determination of global energy is taken into account when the disparity d between adjacent pixels p, p 'varies slightly and a large penalty term p2, when an abrupt change of the disparity d between adjacent pixels p, p 1 is present. This is also called a smoothness constraint.
Das Bild 1 wird zur Bestimmung der Korrespondenzen in Bildausschnitte der Größe (n+l)x(m+l) unterteilt. Zur Weiterberechnung werden nur die in jedem der eindimensionalen Pfade L akkumulierten Kosten S (p, d) für jede n-te Zeile und jede m-te Spalte des Bildes 1 in einer Akkumulierte-Kosten- Matrix gespeichert, wobei die Komplexität des Algorithmus bezüglich der Zeit 0 (WHD) ist (WHD - width, height, disparity ränge; Breite, Höhe, Disparitätsbereich) .The image 1 is subdivided into image sections of size (n + l) x (m + l) in order to determine the correspondences. For further calculation, only the costs S (p, d) accumulated in each of the one-dimensional paths L for every nth row and every mth column of the picture 1 are stored in an accumulated cost matrix, the complexity of the algorithm relating to the Time 0 (WHD) is (WHD - width, height, disparity rank, width, height, disparity range).
Die Speicherung erfolgt zum Beispiel in einem externen Speicher, während die Berechnung beispielsweise in einem FPGA oder einem MikroController stattfindet. Anschließend wird jeweils einer der Bildausschnitte geladen. Dabei werden die gespeicherten akkumulierten Kosten S (p, d) , die einen Rand des Bildausschnitts betreffen, geladen. Die in jedem der eindimensionalen Pfade L akkumulierten Kosten S (p, d) für ein Inneres des Bildausschnitts werden neu berechnet. The storage takes place, for example, in an external memory, while the calculation takes place, for example, in an FPGA or a microcontroller. Subsequently, one of the image sections is loaded in each case. In this case, the stored accumulated costs S (p, d) which relate to an edge of the image section are loaded. The costs S (p, d) accumulated in each of the one-dimensional paths L for an interior of the image section are recalculated.
Bezugs zeichenlisteReference sign list
1, 1' Bild1, 1 'image
C(p,d) Kosten der Unähnlichkeit d Disparität bei ausgerichteten Bildern, sonst LinienparameterC (p, d) Cost of dissimilarity d Disparity in aligned images, otherwise line parameters
D(p) Disparitätsbild, -karteD (p) disparity image, map
L eindimensionaler PfadL one-dimensional path
P, p' PixelP, p 'pixels
Pl Strafterm 1Pl Strafterm 1
P2 Strafterm 2 r RichtungP2 Strafterm 2 r direction
S(p,d) akkumulierte Kosten für einen Pixel S (p, d) accumulated cost for a pixel

Claims

Patentansprüche claims
1. Verfahren zur Bestimmung von Korrespondenzen von1. Method for determining correspondences of
Bildpunkten (p, p') in mindestens zwei stereoskopisch aufgenommenen Bildern (1, 1'), bei dem für jedes Pixel (p) eines der Bilder (1) eine Berechnung von Kosten (C(p,d)) einer Unähnlichkeit anhand von Intensitäten des Pixels (p) und eines als potentiell korrespondierend betrachteten Pixels (p?) des anderen Bildes (I1) durchgeführt wird, wobei entlang einer Anzahl von eindimensionalen Pfaden (L) , die im Pixel (p) münden, eine Akkumulierung der Kosten (C(p,d)) erfolgt, wobei aus den als potentiell korrespondierend betrachteten Pixeln (p?) des anderen Bildes (I1) dasjenige zur Erstellung einer Disparitätskarte (D (p) ) ausgewählt wird, bei dem eine globale Energie minimal ist, die zumindest die Kosten (C(p,d)) der Unähnlichkeit umfasst, dadurch gekennzeichnet, dass das Bild (1) zur Bestimmung der Korrespondenzen in Bildausschnitte der Größe (n+l)x(n+l) unterteilt wird, wobei die in jedem der eindimensionalen Pfade (L) akkumulierten Kosten (S(p,d)) für jede n-te Zeile und jede m-te Spalte gespeichert werden, wobei jeweils einer der Bildausschnitte geladen wird, wobei die für einen Rand des Bildausschnitts gespeicherten akkumulierten Kosten (S(p,d)) geladen werden, wobei die in jedem der eindimensionalen Pfade (L) akkumulierten Kosten (S(p,d)) für ein Inneres des Bildausschnitts neu berechnet werden. Pixels (p, p ') in at least two stereoscopically recorded images (1, 1'), wherein for each pixel (p) of one of the images (1) a calculation of costs (C (p, d)) of a dissimilarity based on Intensities of the pixel (p) and a pixel (p ? ) Of the other image (I 1 ) regarded as potentially corresponding, wherein along a number of one-dimensional paths (L) that terminate in the pixel (p), an accumulation of the costs (C (p, d)), wherein from the pixels (p ? ) Of the other image (I 1 ) regarded as potentially corresponding, the one for creating a disparity map (D (p)) is selected in which a global energy is minimal comprising at least the cost (C (p, d)) of the dissimilarity, characterized in that the image (1) for determining the correspondences is subdivided into image sections of size (n + l) x (n + l), the in each of the one-dimensional paths (L) accumulated costs (S (p, d)) for every nth row u and each mth column, each loading one of the image sections, loading the accumulated costs (S (p, d)) stored for one edge of the image section, the costs accumulated in each of the one-dimensional paths (L) (S (p, d)) are recalculated for an interior of the image section.
2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass bei der Bestimmung der globalen Energie ein kleiner Strafterm (Pl) berücksichtigt wird, wenn sich die Disparität (d) zwischen benachbarten Pixeln (p, p1) geringfügig ändert und ein großer Strafterm (P2) berücksichtigt wird, wenn eine sprungartige Änderung der Disparität (d) zwischen benachbarten Pixeln (p, p1) vorliegt.2. Method according to claim 1, characterized in that in the determination of the global energy a small penalty (Pl) is taken into account when the disparity (d) between adjacent pixels (p, p 1 ) changes slightly and a large penalty term (P2 ) is considered if there is a sudden change in disparity (d) between adjacent pixels (p, p 1 ).
3. Verfahren nach einem der Ansprüche 1 oder 2, dadurch gekennzeichnet, dass die Kosten (C(p,d)) entlang von acht eindimensionalen Pfaden (L) akkumuliert werden. 3. The method according to any one of claims 1 or 2, characterized in that the costs (C (p, d)) along eight one-dimensional paths (L) are accumulated.
PCT/EP2009/002228 2008-04-08 2009-03-26 Semi-global correspondence search in stereo images WO2009124663A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102008017834.9 2008-04-08
DE102008017834A DE102008017834A1 (en) 2008-04-08 2008-04-08 Pixels correspondences determining method for stereoscopically recorded images, involves dividing image into image sections of specific size, and calculating accumulated costs for interior of image sections in each unidimensional path

Publications (1)

Publication Number Publication Date
WO2009124663A1 true WO2009124663A1 (en) 2009-10-15

Family

ID=39777746

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2009/002228 WO2009124663A1 (en) 2008-04-08 2009-03-26 Semi-global correspondence search in stereo images

Country Status (2)

Country Link
DE (1) DE102008017834A1 (en)
WO (1) WO2009124663A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008046505B4 (en) 2008-09-09 2023-06-29 Mercedes-Benz Group AG Process for image processing of stereo images
DE102009012441B4 (en) 2009-03-12 2010-12-09 Deutsches Zentrum für Luft- und Raumfahrt e.V. Method for reducing the memory requirement when determining disparity values for at least two stereoscopically recorded images
DE102009052868A1 (en) 2009-11-13 2010-06-17 Daimler Ag Method for operating e.g. overrun assistance system of lorry, involves representing image areas on overall image based on image area distance of image area limit to vehicle in two different colors
DE102013014314A1 (en) 2013-08-29 2014-03-06 Daimler Ag Method for determining disparity of image pixel in stereoscopically determined image recording, involves performing displacement of pixels around predetermined value prior to determination of disparities on one of image recordings

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HEIKO HIRSCHMULLER ET AL: "Evaluation of Cost Functions for Stereo Matching", COMPUTER VISION AND PATTERN RECOGNITION, 2007. CVPR '07. IEEE CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 1 June 2007 (2007-06-01), pages 1 - 8, XP031114505, ISBN: 978-1-4244-1179-5 *
HEIKO HIRSCHMULLER: "Stereo Processing by Semiglobal Matching and Mutual Information", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE SERVICE CENTER, LOS ALAMITOS, CA, US, vol. 30, no. 2, 1 February 2008 (2008-02-01), pages 328 - 341, XP011195575, ISSN: 0162-8828 *
HIRSCHMULLER H ED - GANG PAN ET AL: "Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information", COMPUTER VISION AND PATTERN RECOGNITION, 2005 IEEE COMPUTER SOCIETY CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, vol. 2, 20 June 2005 (2005-06-20), pages 807 - 814, XP010817537, ISBN: 978-0-7695-2372-9 *

Also Published As

Publication number Publication date
DE102008017834A1 (en) 2008-10-30

Similar Documents

Publication Publication Date Title
DE112013004103B4 (en) Method and apparatus for generating a disparity card
DE112007002221B4 (en) Graphical layout with maximum page coverage and minimal content removal
DE69027633T2 (en) Automatic vehicle guidance along guidance lines
DE102010012262A1 (en) Image processing device
DE102006057552A1 (en) System and method for measuring the distance of a preceding vehicle
DE112014000155B4 (en) Method for determining depth maps from stereo images with improved depth resolution in an area
DE102007019057A1 (en) Apparatus and method for determining an edge histogram, apparatus and method for storing an image in an image database, apparatus and method for finding two similar images and computer program
DE102012221259A1 (en) Optimization of setup families
DE102007040070A1 (en) Image density conversion method, image enhancement processing device and program therefor
WO2009124663A1 (en) Semi-global correspondence search in stereo images
EP2602591B1 (en) Method for simplified storage of data representing shapes
DE102008046505B4 (en) Process for image processing of stereo images
DE19645983C2 (en) Device and method for image processing
EP3867796A1 (en) Method and device for determining an area map
DE102008015535B4 (en) Process for image processing of stereo images
DE10017551A1 (en) Cyclical interactive image analysis involves transforming image at all/selected image points, normalizing results, dividing transformation space into classes using user-influenced separators
DE3837068B4 (en) Image processing method for obtaining a silhouette
AT511286A2 (en) Method for processing calibration data
DE102020114964A1 (en) Modeling a surface of an object
DE102005041633B4 (en) Method and device for determining the position and similarity of object points in images
DE102018126212A1 (en) Procedure for aligning a camera according to its extrinsic parameters
DE102018102691A1 (en) Parallaxenberechnungsvorrichtung
EP3465608A1 (en) Method and device for determining a transfer between two display images, and vehicle
WO2010031868A1 (en) Method for detecting a block raster
AT504372B1 (en) METHOD AND DEVICE FOR DETERMINING INTERPOLATION VALUES IN DATA FIELDS

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09729841

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09729841

Country of ref document: EP

Kind code of ref document: A1