JP7042582B2 - Image pickup device and distance calculation method for image pickup device - Google Patents

Image pickup device and distance calculation method for image pickup device Download PDF

Info

Publication number
JP7042582B2
JP7042582B2 JP2017188177A JP2017188177A JP7042582B2 JP 7042582 B2 JP7042582 B2 JP 7042582B2 JP 2017188177 A JP2017188177 A JP 2017188177A JP 2017188177 A JP2017188177 A JP 2017188177A JP 7042582 B2 JP7042582 B2 JP 7042582B2
Authority
JP
Japan
Prior art keywords
subject
distance
light
image pickup
light sources
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2017188177A
Other languages
Japanese (ja)
Other versions
JP2019066185A (en
Inventor
知弘 大林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Toshiba Infrastructure Systems and Solutions Corp
Original Assignee
Toshiba Corp
Toshiba Infrastructure Systems and Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp, Toshiba Infrastructure Systems and Solutions Corp filed Critical Toshiba Corp
Priority to JP2017188177A priority Critical patent/JP7042582B2/en
Publication of JP2019066185A publication Critical patent/JP2019066185A/en
Application granted granted Critical
Publication of JP7042582B2 publication Critical patent/JP7042582B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Measurement Of Optical Distance (AREA)
  • Optical Radar Systems And Details Thereof (AREA)

Description

本発明の実施形態は、被写体までの距離画像を取得する撮像装置及び撮像装置の距離算
出方法に関する。 An embodiment of the present invention relates to an image pickup device for acquiring a distance image to a subject and a method for calculating the distance of the image pickup device.

一般に、被写体までの距離を測定できるＴＯＦ（Time Of Flight）カメラなどの３次元
カメラが知られている。ＴＯＦカメラは、照射光が被写体に当たって撮像部に戻ってくる
までの時間により、距離を測定する技術である。しかし、被写体が細い円柱など平面が少
ない形状の場合、照射光が被写体により拡散し、反射光の受光量が減少してしまうため、
十分な精度で距離の測定ができないという問題がある。また、被写体が局所的に鏡面反射
するような場合は、鏡面反射した箇所の反射光が撮像部に戻ってこなかったり、光源から
の照射光のほとんどが撮像部に戻り、撮像部が飽和して距離を算出できない、という問題
がある。このような例はビニールのような凹凸のある被写体や、アルミ箔で被写体を覆っ
たようなケースで発生する。 Generally, a three-dimensional camera such as a TOF (Time Of Flight) camera capable of measuring the distance to a subject is known. The TOF camera is a technique for measuring the distance based on the time required for the irradiation light to hit the subject and return to the image pickup unit. However, if the subject has a shape with few flat surfaces such as a thin cylinder, the irradiation light is diffused by the subject and the amount of reflected light received is reduced.
There is a problem that the distance cannot be measured with sufficient accuracy. In addition, when the subject is locally specularly reflected, the reflected light at the specularly reflected portion does not return to the image pickup section, or most of the irradiation light from the light source returns to the image pickup section, and the image pickup section is saturated. There is a problem that the distance cannot be calculated. Such an example occurs in a subject having irregularities such as vinyl or a case in which the subject is covered with aluminum foil.

特開２０１０－１９０６７５号公報Japanese Unexamined Patent Publication No. 2010-190675

本発明が解決しようとする課題は、被写体までの距離を高精度に測定できる撮像装置を
提供することである。 An object to be solved by the present invention is to provide an image pickup apparatus capable of measuring a distance to a subject with high accuracy.

実施形態によれば、撮像装置は、３つ以上の光源と、撮像部と、制御部とを備える。３つ以上の光源は、光を被写体に照射する。撮像部は、前記光源が照射した光の前記被写体からの反射光を受光する。制御部は、前記３つ以上の光源のうち、少なくとも１つの光源を除く光源が光を照射する点灯光源の組合せにより、光を照射してから前記撮像部が前記被写体からの反射光を受光するまでの時間に基づいて、それぞれの前記点灯光源の組合せにおいて画素ごとに前記被写体までの参照距離を算出し、全ての前記参照距離に基づいて、画素ごとに前記被写体までの距離を算出するとともに、前記被写体までの距離が算出できた任意の画素に対応する前記参照距離のうち、最小値と最大値との差が所定の範囲以内の場合、前記被写体までの距離が算出できた前記任意の画素に対応する前記参照距離の平均値を前記任意の画素に対応する前記被写体までの距離として算出し、前記最小値と前記最大値との差が前記所定の範囲を超える場合、前記被写体までの参照距離を算出できた画素を最も多く有する前記点灯光源の組合せにより得られた前記参照距離を前記任意の画素に対応する前記被写体までの距離として算出する。 According to the embodiment, the image pickup apparatus includes three or more light sources, an image pickup unit, and a control unit . Three or more light sources illuminate the subject with light. The image pickup unit receives the reflected light from the subject of the light emitted by the light source. The control unit irradiates light with a combination of lighting light sources irradiating light from a light source other than at least one of the three or more light sources, and then the image pickup unit receives the reflected light from the subject. Based on the time to, the reference distance to the subject is calculated for each pixel in each combination of the lighting light sources, and the distance to the subject is calculated for each pixel based on all the reference distances . Of the reference distances corresponding to arbitrary pixels for which the distance to the subject can be calculated, when the difference between the minimum value and the maximum value is within a predetermined range, the arbitrary pixels for which the distance to the subject can be calculated. The average value of the reference distances corresponding to the above is calculated as the distance to the subject corresponding to the arbitrary pixel, and when the difference between the minimum value and the maximum value exceeds the predetermined range, the reference to the subject is made. The reference distance obtained by the combination of the lighting light sources having the largest number of pixels for which the distance can be calculated is calculated as the distance to the subject corresponding to the arbitrary pixel.

実施形態に係る撮像装置の外観の一例を示す正面図および及び光源の点灯の組合せを示す図。The front view which shows an example of the appearance of the image pickup apparatus which concerns on embodiment, and the figure which shows the combination of lighting of a light source. 実施形態に係る撮像装置が備える制御部の機能構成の一例を示すブロック図。The block diagram which shows an example of the functional structure of the control part provided in the image pickup apparatus which concerns on embodiment. 実施形態に係る撮像装置を用いた撮像状況の一例を示す斜視図。The perspective view which shows an example of the image pickup situation using the image pickup apparatus which concerns on embodiment. 実施形態に係る撮像装置が取得する参照距離画像の画素の一例を示す図。The figure which shows an example of the pixel of the reference distance image acquired by the image pickup apparatus which concerns on embodiment. 実施形態に係る撮像装置が備える制御部が、被写体までの距離を算出する流れの一例を示すフローチャートA flowchart showing an example of a flow in which a control unit included in an image pickup apparatus according to an embodiment calculates a distance to a subject.

以下、実施形態に係る撮像装置１を、図面を用いて説明する。
図１（Ａ）は、実施形態に係る撮像装置１としてのＴＯＦカメラの外観の一例を示す正面
図である。撮像装置１は、被写体に光を照射する複数の光源２ａ乃至２ｆ、撮像部３を備
える。なお、光源については、図１に示した６個の光源２ａ乃至２ｆを個々に区別する必
要がないときは、光源２と略記する。 Hereinafter, the image pickup apparatus 1 according to the embodiment will be described with reference to the drawings.
FIG. 1A is a front view showing an example of the appearance of the TOF camera as the image pickup apparatus 1 according to the embodiment. The image pickup apparatus 1 includes a plurality of light sources 2a to 2f for irradiating a subject with light, and an image pickup unit 3. The light source is abbreviated as light source 2 when it is not necessary to individually distinguish the six light sources 2a to 2f shown in FIG.

撮像装置１は、光源２からの照射光が被写体に当たって、撮像部３で受光するまでの時
間に基づいて、被写体までの距離を画素ごとに測定することが可能に構成されている。ま
た、撮像装置１は、測定した距離の情報を画素ごとに表わした距離画像を生成することが
可能である。 The image pickup apparatus 1 is configured to be capable of measuring the distance to the subject pixel by pixel based on the time until the irradiation light from the light source 2 hits the subject and is received by the image pickup unit 3. Further, the image pickup apparatus 1 can generate a distance image in which the measured distance information is represented for each pixel.

光源２は、撮像部３の周囲に３つ以上設置され、図１（Ａ）では６つの光源２から光を
被写体に向けて照射する。光源２が照射する光は、赤外線でも良いし、可視光でも良く、
撮像部３が受光することが可能な波長帯の光であれば良い。また、光源２は、被写体に照
射する光量、もしくは照射強度を高めるために、複数のＬＥＤ光源を有しても良い。 Three or more light sources 2 are installed around the image pickup unit 3, and in FIG. 1A, light is emitted from the six light sources 2 toward the subject. The light emitted by the light source 2 may be infrared light or visible light.
Any light in a wavelength band that can be received by the image pickup unit 3 may be used. Further, the light source 2 may have a plurality of LED light sources in order to increase the amount of light irradiating the subject or the irradiation intensity.

光源２は６つの光源２のうち、全ての光源２及び少なくとも１つの光源２を除く光源２
が光を照射する点灯光源の組合せにより光源２を点灯させる。図１（Ｂ）は光源２の点灯
の組合せを示す図であり、光源２ａ乃至２ｆのうち少なくとも１つを除く光源２を順次点
灯させる組合せを示している。 The light source 2 is a light source 2 other than all the light sources 2 and at least one light source 2 out of the six light sources 2.
Light source 2 is turned on by a combination of lighting light sources that irradiate light. FIG. 1B is a diagram showing a combination of lighting of the light source 2, and shows a combination of sequentially lighting the light source 2 excluding at least one of the light sources 2a to 2f.

なお、光源２の設置位置、個数は図１に限定されるものではない。例えば、図１におい
て光源２は、撮像部３を中心に対称に設置されているが、非対称に設置されても良い。ま
た、光量を増加したいときはそれぞれの光源２ａ乃至２ｆに複数のＬＥＤを配置してもよ
い。 The installation position and number of the light sources 2 are not limited to FIG. For example, in FIG. 1, the light source 2 is installed symmetrically with respect to the image pickup unit 3, but may be installed asymmetrically. Further, when it is desired to increase the amount of light, a plurality of LEDs may be arranged in each of the light sources 2a to 2f.

撮像部３は、光源２が照射した光の被写体からの反射光を受光し、２次元画像を撮像す
る。例えば、撮像部３は、被写体からの反射光を図示しない、開口部、レンズ、受光部を
介して受光する。 The image pickup unit 3 receives the reflected light from the subject of the light emitted by the light source 2 and captures a two-dimensional image. For example, the image pickup unit 3 receives light from the subject through an opening, a lens, and a light receiving unit (not shown).

開口部は、レンズに入射する光量を調節するための絞りである。また、レンズは、被写
体からの光を受光部に集光する。なお、レンズは、１枚に限らず、例えば、複数枚を組み
合わせることで、フォーカスレンズや、ズームレンズを構成しても良い。受光部は、イメ
ージセンサなどの受光素子である。 The aperture is a diaphragm for adjusting the amount of light incident on the lens. In addition, the lens collects the light from the subject on the light receiving portion. The lens is not limited to one, and a focus lens or a zoom lens may be formed by combining a plurality of lenses, for example. The light receiving unit is a light receiving element such as an image sensor.

次に、図２を用いて、撮像装置１が備える制御部について説明する。図２は実施形態に
係る撮像装置１が備える制御部４の機能構成の一例を示すブロック図である。 Next, the control unit included in the image pickup apparatus 1 will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the functional configuration of the control unit 4 included in the image pickup apparatus 1 according to the embodiment.

制御部４は、図示しないＣＰＵや記憶部を備え、撮像装置１が有する各構成部の動作を
統括的に制御する。また、制御部４は、参照距離算出部５、２次元画像生成部６、発光制
御部７を備える。 The control unit 4 includes a CPU and a storage unit (not shown), and comprehensively controls the operation of each component of the image pickup apparatus 1. Further, the control unit 4 includes a reference distance calculation unit 5, a two-dimensional image generation unit 6, and a light emission control unit 7.

参照距離算出部５は、撮像部３が受光した結果に基づいて、被写体までの距離である参
照距離を算出する処理を行う。つまり、参照距離算出部５は、光源２が光を照射してから
撮像部３が被写体からの反射光を受光するまでの時間に基づいて、撮像部３が撮像する２
次元画像の画素ごとに被写体までの参照距離を算出する。 The reference distance calculation unit 5 performs a process of calculating a reference distance, which is the distance to the subject, based on the result received by the image pickup unit 3. That is, in the reference distance calculation unit 5, the image pickup unit 3 takes an image based on the time from when the light source 2 irradiates the light until the image pickup unit 3 receives the reflected light from the subject.
The reference distance to the subject is calculated for each pixel of the dimensional image.

２次元画像生成部６は、撮像部３が受光した光に基づいて、２次元画像を生成する。な
お、参照距離算出部５が算出する被写体までの参照距離、参照距離画像は、２次元画像生
成部６が生成する２次元画像の画素に基づいて算出される。 The two-dimensional image generation unit 6 generates a two-dimensional image based on the light received by the image pickup unit 3. The reference distance to the subject and the reference distance image calculated by the reference distance calculation unit 5 are calculated based on the pixels of the two-dimensional image generated by the two-dimensional image generation unit 6.

２次元画像は、光源２が照射した光の被写体からの反射光に基づいて２次元画像処理部
６によって生成されるため、グレースケール画像や、赤外画像など、光源に依存した種類
の画像である。 Since the two-dimensional image is generated by the two-dimensional image processing unit 6 based on the reflected light from the subject of the light emitted by the light source 2, it is a type of image depending on the light source such as a gray scale image or an infrared image. be.

発光制御部７は、６つの光源２に対し、図１（Ｂ）に示すようにそれぞれ独立に点灯と
消灯を制御する。被写体を撮像する際には、６つの光源２のうち、全ての光源又は少なく
とも１つの光源２を除く光源２が光を照射する点灯光源の組合せに従って、光源２を点灯
させる。 The light emission control unit 7 independently controls lighting and extinguishing of the six light sources 2 as shown in FIG. 1 (B). When the subject is imaged, the light source 2 is turned on according to the combination of the lighting light sources in which all the light sources or the light sources 2 other than at least one light source 2 irradiate the light among the six light sources 2.

また、制御部４は、被写体までの参照距離を測定する際には、全ての点灯光源の組合せ
に対して撮像を行うように各部を制御する。つまり、制御部４は、６つの光源２のうち、
少なくとも１つの光源２を除く光源２が光を照射する点灯光源の組合せにより、光を照射
してから撮像部３が被写体からの反射光を受光するまでの時間に基づいて、それぞれの点
灯光源の組合せにおいて画素ごとに被写体までの参照距離を算出し、参照距離画像を生成
する参照距離算出手段を備える。この場合、被写体によって鏡面反射した光が全て撮像部
に到達した画素については、撮像部の光量が飽和して参照距離が得られなかったり、逆に
鏡面反射した光が撮像部以外の方向に反射して撮像部の光量が不足して参照距離が得られ
ない画素も存在することとなる。 Further, when measuring the reference distance to the subject, the control unit 4 controls each unit so as to perform imaging for all combinations of lighting light sources. That is, the control unit 4 is out of the six light sources 2.
Depending on the combination of lighting light sources that the light source 2 excluding at least one light source 2 irradiates with light, each lighting light source is based on the time from the irradiation of light to the reception of the reflected light from the subject by the image pickup unit 3. In combination, the reference distance calculation means for calculating the reference distance to the subject for each pixel and generating the reference distance image is provided. In this case, for the pixels in which all the light reflected by the subject reaches the image pickup unit, the amount of light in the image pickup section is saturated and the reference distance cannot be obtained, or conversely, the light reflected by the mirror surface is reflected in a direction other than the image pickup section. Therefore, there are some pixels in which the amount of light in the image pickup unit is insufficient and the reference distance cannot be obtained.

さらに、制御部４は、全ての参照距離から、被写体までの距離を算出する。つまり、制
御部４は、参照距離算出部５により生成した全ての参照距離画像に基づいて、画素ごとに
被写体までの距離を算出し、距離画像を生成する距離算出手段を備える。 Further, the control unit 4 calculates the distance to the subject from all the reference distances. That is, the control unit 4 includes a distance calculation means that calculates the distance to the subject for each pixel based on all the reference distance images generated by the reference distance calculation unit 5, and generates a distance image.

次に、図３乃至図５を用いて参照距離算出部５が画素ごとに参照距離を算出する方法を
説明する。まず、図３は、実施形態に係る撮像装置１を用いた撮像状況の一例を示す図で
ある。図３には、撮像装置１の正面に直方体の物体１０と、撮像装置１と物体１０との間
に柵１１がある状況を示す。なお、物体１０は、撮像装置１から見ると物体１０の一面の
みが見える角度で位置しているとする。 Next, a method in which the reference distance calculation unit 5 calculates the reference distance for each pixel will be described with reference to FIGS. 3 to 5. First, FIG. 3 is a diagram showing an example of an image pickup situation using the image pickup apparatus 1 according to the embodiment. FIG. 3 shows a situation in which a rectangular parallelepiped object 10 is located in front of the image pickup device 1 and a fence 11 is provided between the image pickup device 1 and the object 10. It is assumed that the object 10 is located at an angle at which only one surface of the object 10 can be seen when viewed from the image pickup apparatus 1.

図４は、実施形態に係る撮像装置１が取得する参照距離画像の画素の一例を示す図であ
る。また、図４に示す画素は、図３の撮像状況において、撮像した画素の一例である。な
お、図４の画素は、光源２ｂ、２ｅを点灯させた点灯光源の組合せにおいて撮像されたも
のとする。また、説明を簡易にするために、図４に示す画素の行と列にはそれぞれアルフ
ァベットと数字を振っている。例えば、一番左上に位置する画素はＡ行１列と呼称する。 FIG. 4 is a diagram showing an example of pixels of a reference distance image acquired by the image pickup apparatus 1 according to the embodiment. Further, the pixel shown in FIG. 4 is an example of the imaged pixel in the imaging situation of FIG. It is assumed that the pixels in FIG. 4 are captured by a combination of lighting light sources in which the light sources 2b and 2e are turned on. Further, in order to simplify the explanation, alphabets and numbers are assigned to the rows and columns of the pixels shown in FIG. 4, respectively. For example, the pixel located at the upper left is referred to as A row and 1 column.

また、各画素に記載されている数値は、被写体までの距離を意味しており、数字が大き
いほど被写体までの距離が遠いこととしている。ただし、数値０の画素は距離を算出する
ことができなかったことを意味している。数値０の画素が意味する、距離を算出すること
ができないとは、例えば被写体までの距離が遠すぎる場合や、受光する光量が少なすぎる
場合や、受光する光量が多すぎるため撮像部３が飽和してしまうなどである。 Further, the numerical value described in each pixel means the distance to the subject, and the larger the number, the farther the distance to the subject is. However, the pixel with the numerical value 0 means that the distance could not be calculated. Pixels with a numerical value of 0 mean that the distance cannot be calculated, for example, when the distance to the subject is too long, the amount of light received is too small, or the amount of light received is too large, so that the image pickup unit 3 is saturated. And so on.

図４に示す画素には、５列と８列を除く、Ｂ行４列とＢ行９列とＦ行４列とＦ行９列の
画素を頂点とする四角形の範囲に数値６が示されている。これは、物体１０の平面までの
距離が示されたもので、５列と８列は柵１１によって物体１０が遮られていることを示し
ている。ＴＯＦカメラは、一般的に、平面までの距離を測定するには有利であるが、エッ
ジである柵１１までの距離については照射光が拡散してしまうため、測定するには不利で
あることを例示した極端な例である。 In the pixels shown in FIG. 4, the numerical value 6 is shown in the range of the quadrangle having the pixels of B row 4 columns, B row 9 columns, F row 4 columns, and F row 9 columns as the vertices, excluding 5 columns and 8 columns. ing. This shows the distance to the plane of the object 10, and the 5th and 8th rows show that the object 10 is blocked by the fence 11. A TOF camera is generally advantageous for measuring the distance to a plane, but it is disadvantageous for measuring the distance to the fence 11 which is an edge because the irradiation light is diffused. It is an extreme example illustrated.

ここで、図４では、光源２ｂ、２ｅを点灯させた点灯光源の組合せにより撮像したが、
光源２ｂ、２ｅに加え、例えば、光源２ａ、２ｄを点灯させた場合、位置、角度が違う光
源２からの光の照射が与えられるため、柵１１までの距離を算出できる場合がある。従っ
て、ある点灯光源の組合せでは、ある画素の被写体までの距離を算出できなかったとして
も、他の点灯光源の組合せを用いることで、その画素の被写体までの距離を算出でき、算
出結果を補完することが可能である。 Here, in FIG. 4, the image was taken by the combination of the lighting light sources in which the light sources 2b and 2e were turned on.
For example, when the light sources 2a and 2d are turned on in addition to the light sources 2b and 2e, the distance to the fence 11 may be calculated because the light sources 2 having different positions and angles are irradiated. Therefore, even if the distance of a certain pixel to the subject cannot be calculated with a certain combination of lighting light sources, the distance of the pixel to the subject can be calculated by using the combination of other lighting light sources, and the calculation result is complemented. It is possible to do.

図１では、６個の光源２を設置しているため、６つの光源２のうち、少なくとも１つの
光源２を除く光源２が光を照射する点灯光源の組合せは図１（Ｂ）に示すように６３通り
存在する。この場合、６３通り全てに対して距離を算出すると、組合せが多すぎるため、
かえって不便となることも考えられる。従って、参照距離を算出する点灯光源の組合せは
、事前に制限するよう設定しても良い。例えば、６つの光源２のうち、少なくとも１つの
光源２を除く光源２が点灯するのであれば、撮像部３との距離が短い光源２ｂ、２ｅは必
ず点灯させるとしても良い。また、光源２の数は６つに限らず３つ以上あればよい。 In FIG. 1, since six light sources 2 are installed, the combination of lighting light sources in which the light sources 2 other than at least one light source 2 among the six light sources 2 irradiate light is as shown in FIG. 1 (B). There are 63 ways in. In this case, if the distance is calculated for all 63 ways, there are too many combinations.
On the contrary, it may be inconvenient. Therefore, the combination of lighting light sources for calculating the reference distance may be set to be limited in advance. For example, if the light source 2 other than at least one of the six light sources 2 is turned on, the light sources 2b and 2e having a short distance from the image pickup unit 3 may be turned on without fail. Further, the number of the light sources 2 is not limited to six, and may be three or more.

次に、図５を用いて、実施形態に係る撮像装置１が備える制御部４が、被写体までの距
離を算出する方法を説明する。図５は、実施形態に係る撮像装置１が備える制御部４が、
被写体までの距離を算出する流れの一例を示すフローチャートである。 Next, with reference to FIG. 5, a method in which the control unit 4 included in the image pickup apparatus 1 according to the embodiment calculates the distance to the subject will be described. FIG. 5 shows that the control unit 4 included in the image pickup apparatus 1 according to the embodiment is
It is a flowchart which shows an example of the flow of calculating the distance to a subject.

まず、制御部４は、設定された全ての点灯光源の組合せにおいて参照距離を算出する（
ステップＳ００）。次に、制御部４は、任意の画素を一つ選択し（ステップＳ０１）、全
ての参照距離の選択した画素に対応する画素の参照距離のうち、数値が０でない有効な参
照距離が存在するか否かを判定する（ステップＳ０３）。制御部４は、有効な参照距離が
存在する場合（ステップＳ０３、Ｙｅｓ）、被写体までの距離が算出できた選択した画素
に対応する参照距離のうち、距離が算出できなかった数値０を除く最小値と最大値との差
が所定の範囲以内か否かを判定する（ステップＳ０５）。ここで、所定の範囲とは、許容
できる参照距離のばらつきを定義するための閾値である。例えば、所定の範囲を、最小値
と最大値との差が最小値の２０％と、定義しても良い。 First, the control unit 4 calculates the reference distance for all the combinations of the set lighting light sources (
Step S00). Next, the control unit 4 selects one arbitrary pixel (step S01), and among the reference distances of the pixels corresponding to the selected pixels of all the reference distances, there is an effective reference distance whose numerical value is not 0. Whether or not it is determined (step S03). When a valid reference distance exists (step S03, Yes), the control unit 4 is the minimum of the reference distances corresponding to the selected pixels for which the distance to the subject can be calculated, excluding the numerical value 0 for which the distance could not be calculated. It is determined whether or not the difference between the value and the maximum value is within a predetermined range (step S05). Here, the predetermined range is a threshold value for defining the variation of the allowable reference distance. For example, a predetermined range may be defined as the difference between the minimum value and the maximum value is 20% of the minimum value.

制御部４は、最小値と最大値との差が所定の範囲以内の場合（ステップＳ０５、Ｙｅｓ
）、有効な参照距離の平均値を選択した画素に対応する被写体までの距離として算出する
（ステップＳ０７）。 The control unit 4 is in the case where the difference between the minimum value and the maximum value is within a predetermined range (step S05, Yes).
), The average value of the valid reference distances is calculated as the distance to the subject corresponding to the selected pixel (step S07).

次に、制御部４は、全ての画素に対して処理を行ったか否かを判定し（ステップＳ０９
）、全ての画素に対して処理を行っていない場合（ステップＳ０９、Ｎｏ）、処理をステ
ップＳ０１に戻し、全ての画素に対して処理を行った場合（ステップＳ０９、Ｙｅｓ）、
処理を終了する。 Next, the control unit 4 determines whether or not processing has been performed on all the pixels (step S09).
), When the processing is not performed for all the pixels (step S09, No), the processing is returned to step S01, and the processing is performed for all the pixels (step S09, Yes).
End the process.

また、制御部４は、ステップＳ０５において、最小値と最大値との差が所定の範囲を超
える場合（ステップＳ０５、Ｎｏ）、参照距離を算出できた画素を最も多く有する点灯光
源の組合せにより得られた参照距離画像の選択した画素に対応する画素を、選択した画素
に対応する被写体までの距離として算出する（ステップＳ１１）。そして、制御部は４、
処理をステップＳ０９に進める。これは、点灯光源の組合せが、鏡面反射の影響が最も少
なく、被写体の参照距離が適切に得られた参照距離画像であることを意味している。 Further, when the difference between the minimum value and the maximum value exceeds a predetermined range in step S05 (step S05, No), the control unit 4 obtains the combination of lighting light sources having the largest number of pixels for which the reference distance can be calculated. The pixel corresponding to the selected pixel of the obtained reference distance image is calculated as the distance to the subject corresponding to the selected pixel (step S11). And the control unit is 4,
The process proceeds to step S09. This means that the combination of lighting light sources is a reference distance image in which the influence of specular reflection is the least and the reference distance of the subject is appropriately obtained.

また、制御部４は、ステップＳ０３において、有効な参照距離が存在しないと判定した
場合（ステップＳ０３、Ｎｏ）、数値０を選択した画素に対応する被写体までの距離とし
て算出する（ステップＳ１３）。そして、制御部４は、処理をステップＳ０９に進める。 Further, when the control unit 4 determines in step S03 that a valid reference distance does not exist (step S03, No), the control unit 4 calculates the numerical value 0 as the distance to the subject corresponding to the selected pixel (step S13). Then, the control unit 4 advances the process to step S09.

以上の流れで動作を実行することにより、実施形態に係る撮像装置１は、光源２から照
射される光が被写体によって拡散されてしまうことが少ない、かつ、受光量が多すぎて撮
像部３が飽和しない最適な点灯光源の組合せを選択することができる。その結果、被写体
までの距離をより高精度に測定することが可能である。
なお、上記の実施形態では６つの光源２を例に説明したが、光源２は３つ以上であれば
よく、その数は制限されるものではない。 By executing the operation according to the above flow, in the image pickup apparatus 1 according to the embodiment, the light emitted from the light source 2 is less likely to be diffused by the subject, and the amount of light received is too large, so that the image pickup unit 3 It is possible to select the optimum combination of lighting sources that does not saturate. As a result, it is possible to measure the distance to the subject with higher accuracy.
In the above embodiment, six light sources 2 have been described as an example, but the number of the light sources 2 may be three or more, and the number thereof is not limited.

また、上記した実施形態の撮像装置１における処理はいくつかのソフトウェアによって
実行することが可能である。このため、上記処理の手順を実行するいくつかのプログラム
を格納したコンピュータ読み取り可能な記憶媒体を通じてこれらプログラムを撮像装置１
へインストールして実行するだけで、上記処理を容易に実現することができる。例えば、
撮像装置１は、上記プログラムをネットワーク経由でダウンロードし、ダウンロードした
プログラムを記憶し、プログラムのインストールを完了することができる。或いは、撮像
装置１は、上記プログラムを情報記憶媒体から読み取り、読み取ったプログラムを記憶し
、プログラムのインストールを完了することができる。 Further, the processing in the image pickup apparatus 1 of the above-described embodiment can be executed by some software. Therefore, these programs are imaged through a computer-readable storage medium that stores some programs that execute the above processing procedure.
The above process can be easily realized simply by installing and executing it in. for example,
The image pickup apparatus 1 can download the above program via the network, store the downloaded program, and complete the installation of the program. Alternatively, the image pickup apparatus 1 can read the program from the information storage medium, store the read program, and complete the installation of the program.

本発明のいくつかの実施形態を説明したが、これらの実施形態は、例として提示したも
のであり、発明の範囲を限定することは意図していない。これら新規な実施形態は、その
他の様々な形態で実施されることが可能であり、発明の要旨を逸脱しない範囲で、種々の
省略、置き換え、変更を行うことができる。これら実施形態やその変形は、発明の範囲や
要旨に含まれるとともに、特許請求の範囲に記載された発明とその均等の範囲に含まれる
。 Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

１ 撮像装置
２、２ａ、２ｂ、２ｄ、２ｅ、２ｆ 光源
３ 撮像部
４ 制御部
５ 参照距離算出部
６ 次元画像生成部
７ 発光制御部
１０ 物体
１１ 柵 1 Image pickup device 2, 2a, 2b, 2d, 2e, 2f Light source 3 Image pickup unit 4 Control unit 5 Reference distance calculation unit 6 Dimensional image generation unit 7 Light emission control unit 10 Object 11 Fence

Claims (1)

光を被写体に照射する３つ以上の光源と、
前記光源が照射した光の前記被写体からの反射光を受光する撮像部と、
前記３つ以上の光源のうち、少なくとも１つの光源を除く光源が光を照射する点灯光源の組合せにより、光を照射してから前記撮像部が前記被写体からの反射光を受光するまでの時間に基づいて、それぞれの前記点灯光源の組合せにおいて画素ごとに前記被写体までの参照距離を算出し、全ての前記参照距離に基づいて、画素ごとに前記被写体までの距離を算出するとともに、前記被写体までの距離が算出できた任意の画素に対応する前記参照距離のうち、最小値と最大値との差が所定の範囲以内の場合、前記被写体までの距離が算出できた前記任意の画素に対応する前記参照距離の平均値を前記任意の画素に対応する前記被写体までの距離として算出し、前記最小値と前記最大値との差が前記所定の範囲を超える場合、前記被写体までの参照距離を算出できた画素を最も多く有する前記点灯光源の組合せにより得られた前記参照距離を前記任意の画素に対応する前記被写体までの距離として算出する制御部と、
を備える撮像装置。 Three or more light sources that illuminate the subject with light,
An image pickup unit that receives the reflected light from the subject of the light emitted by the light source, and an image pickup unit.
Depending on the combination of lighting light sources that the light sources other than at least one of the three or more light sources irradiate with light, the time from the irradiation of light to the reception of the reflected light from the subject by the imaging unit Based on this, the reference distance to the subject is calculated for each pixel in each combination of the lighting light sources, and the distance to the subject is calculated for each pixel based on all the reference distances, and the distance to the subject is calculated. When the difference between the minimum value and the maximum value is within a predetermined range among the reference distances corresponding to the arbitrary pixels for which the distance can be calculated, the said corresponding to the arbitrary pixels for which the distance to the subject can be calculated. The average value of the reference distance is calculated as the distance to the subject corresponding to the arbitrary pixel, and when the difference between the minimum value and the maximum value exceeds the predetermined range, the reference distance to the subject can be calculated. A control unit that calculates the reference distance obtained by the combination of the lighting light sources having the largest number of pixels as the distance to the subject corresponding to the arbitrary pixel.
An image pickup device equipped with .
JP2017188177A 2017-09-28 2017-09-28 Image pickup device and distance calculation method for image pickup device Active JP7042582B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2017188177A JP7042582B2 (en) 2017-09-28 2017-09-28 Image pickup device and distance calculation method for image pickup device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2017188177A JP7042582B2 (en) 2017-09-28 2017-09-28 Image pickup device and distance calculation method for image pickup device

Publications (2)

Publication Number Publication Date
JP2019066185A JP2019066185A (en) 2019-04-25
JP7042582B2 true JP7042582B2 (en) 2022-03-28

Family

ID=66339427

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2017188177A Active JP7042582B2 (en) 2017-09-28 2017-09-28 Image pickup device and distance calculation method for image pickup device

Country Status (1)

Country Link
JP (1) JP7042582B2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021076408A (en) * 2019-11-06 2021-05-20 パイオニア株式会社 Sensor device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004028874A (en) 2002-06-27 2004-01-29 Matsushita Electric Ind Co Ltd Range finder device, and device and method for detecting object
JP2007271373A (en) 2006-03-30 2007-10-18 Toyota Central Res & Dev Lab Inc Object detecting device
JP2009192499A (en) 2008-02-18 2009-08-27 Stanley Electric Co Ltd Apparatus for generating distance image
US20120038903A1 (en) 2010-08-16 2012-02-16 Ball Aerospace & Technologies Corp. Electronically steered flash lidar
JP2014077668A (en) 2012-10-09 2014-05-01 Optex Co Ltd Dimension measurement device and dimension measurement method
US20160200161A1 (en) 2015-01-13 2016-07-14 Xenomatix Nv Surround sensing system with telecentric optics
CN106662640A (en) 2014-12-22 2017-05-10 谷歌公司 Smart illumination time of flight system and method
US20170273161A1 (en) 2016-03-16 2017-09-21 Tadashi Nakamura Object detection apparatus and moveable apparatus
WO2017213052A1 (en) 2016-06-08 2017-12-14 パナソニックＩｐマネジメント株式会社 Ranging system and ranging method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11108660A (en) * 1997-10-03 1999-04-23 Kansei Corp Equipment for detecting distance

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004028874A (en) 2002-06-27 2004-01-29 Matsushita Electric Ind Co Ltd Range finder device, and device and method for detecting object
JP2007271373A (en) 2006-03-30 2007-10-18 Toyota Central Res & Dev Lab Inc Object detecting device
JP2009192499A (en) 2008-02-18 2009-08-27 Stanley Electric Co Ltd Apparatus for generating distance image
US20120038903A1 (en) 2010-08-16 2012-02-16 Ball Aerospace & Technologies Corp. Electronically steered flash lidar
JP2014077668A (en) 2012-10-09 2014-05-01 Optex Co Ltd Dimension measurement device and dimension measurement method
CN106662640A (en) 2014-12-22 2017-05-10 谷歌公司 Smart illumination time of flight system and method
US20160200161A1 (en) 2015-01-13 2016-07-14 Xenomatix Nv Surround sensing system with telecentric optics
US20170273161A1 (en) 2016-03-16 2017-09-21 Tadashi Nakamura Object detection apparatus and moveable apparatus
WO2017213052A1 (en) 2016-06-08 2017-12-14 パナソニックＩｐマネジメント株式会社 Ranging system and ranging method

Also Published As

Publication number Publication date
JP2019066185A (en) 2019-04-25

Similar Documents

Publication Publication Date Title
US10041787B2 (en) Object detection device
JP7191921B2 (en) TOF camera system and method for measuring distance with same
KR102369792B1 (en) Photographing apparatus and photographing method
JP7292315B2 (en) Distance measurement using high density projection pattern
US9599463B2 (en) Object detection device
US20160191867A1 (en) Structured light projector
CN107037443A (en) Method and apparatus for the range measurement based on triangulation
US20190335073A1 (en) Imaging device
JP6748984B2 (en) Distance measuring device and moving body
US11019249B2 (en) Mapping three-dimensional depth map data onto two-dimensional images
JP7042582B2 (en) Image pickup device and distance calculation method for image pickup device
EP3596425B1 (en) Optoelectronic devices for collecting three-dimensional data
US10753726B2 (en) System and method for 3D profile determination using model-based peak selection
CN115427830A (en) Illumination device for a distance measuring camera system, corresponding illumination method and TOF camera system
US20170069110A1 (en) Shape measuring method
CN110836647B (en) Three-dimensional scanning system
US10091404B2 (en) Illumination apparatus, imaging system, and illumination method
JP2011133360A (en) Distance measuring device, distance measurement method, and program
JP2016177750A5 (en)
JP6888429B2 (en) Pattern irradiation device, imaging system and handling system
JP7474759B2 (en) Vehicle lighting fixtures
JP2016224590A (en) Self position calculation device and self position calculation method
US20180158205A1 (en) Apparatus and method for generating three-dimensional information
US20200191917A1 (en) Image processing device, distance detection device, image processing method, and non-transitory storage medium
US20230289985A1 (en) Three-dimensional sensing device and specular reflection object detection method

Legal Events

Date Code Title Description
RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20180831

RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20190125

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20200602

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20210304

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20210326

RD02 Notification of acceptance of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7422

Effective date: 20210514

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20210519

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20210518

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20211206

RD01 Notification of change of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7421

Effective date: 20211221

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220112

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220214

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220315

R150 Certificate of patent or registration of utility model

Ref document number: 7042582

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150