JP7151571B2 - DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD - Google Patents

DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD Download PDF

Info

Publication number
JP7151571B2
JP7151571B2 JP2019049598A JP2019049598A JP7151571B2 JP 7151571 B2 JP7151571 B2 JP 7151571B2 JP 2019049598 A JP2019049598 A JP 2019049598A JP 2019049598 A JP2019049598 A JP 2019049598A JP 7151571 B2 JP7151571 B2 JP 7151571B2
Authority
JP
Japan
Prior art keywords
driver
visible light
state
vehicle
state determination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019049598A
Other languages
Japanese (ja)
Other versions
JP2020152133A (en
Inventor
淳志 鹿又
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to JP2019049598A priority Critical patent/JP7151571B2/en
Publication of JP2020152133A publication Critical patent/JP2020152133A/en
Application granted granted Critical
Publication of JP7151571B2 publication Critical patent/JP7151571B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Camera Bodies And Camera Details Or Accessories (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Arrangements Of Lighting Devices For Vehicle Interiors, Mounting And Supporting Thereof, Circuits Therefore (AREA)
  • Studio Devices (AREA)
  • Traffic Control Systems (AREA)

Description

この発明は撮像装置の撮像した画像から運転者の状態を判定する運転者状態判定装置及び運転者状態判定方法に関するものである。 The present invention relates to a driver's state determination device and a driver's state determination method for determining the state of a driver from an image captured by an imaging device.

運転者の顔色の情報を取得し、取得した顔色の情報から運転者が飲酒状態であるかを判定する飲酒運転防止装置が提案されている。 (下記特許文献1)。 A drunk driving prevention device has been proposed that acquires information on the complexion of a driver and determines whether the driver is in a drunken state from the acquired information on the complexion. (Patent Document 1 below).

特開２０１０－５２６３５号公報 JP 2010-52635 A

従来例のように、運転者の顔色の情報を取得するためには可視光カメラを用いる必要がある。しかし、走行中の車両では、外光による車両内部の輝度の変化は非常に大きいため、可視光カメラでは、撮像した画像に頻繁に白飛び、黒飛びが発生してしまう。従って、居眠り状態や脇見状態など車両が走行中に取得したい運転者の状態を精度よく検出し続けることは困難である。 As in the conventional example, it is necessary to use a visible light camera in order to obtain information on the complexion of the driver. However, when a vehicle is running, the change in brightness inside the vehicle due to outside light is very large, so that the visible light camera frequently causes overexposure and overexposure in images captured by the visible light camera. Therefore, it is difficult to continue to accurately detect the driver's state, such as the state of dozing off or looking aside, while the vehicle is running.

一方、近赤外線カメラなど、非可視光の光を検出する非可視光カメラを用いる場合、外光の影響は受けにくいため、車両走行中に運転者の状態を精度よく検出できるものの、運転者の顔色の情報は取得することができない。 On the other hand, when using an invisible light camera that detects invisible light, such as a near-infrared camera, it is not easily affected by outside light, so it is possible to accurately detect the driver's condition while the vehicle is running. Information on complexion cannot be obtained.

運転者の顔色の情報を取得しつつ、走行中の車両で運転者の状態を精度よく判定するためには、可視光カメラと非可視光カメラの両方を用い、可視光カメラと非可視光カメラの両方で撮像することが考えられる。しかし、可視光カメラと非可視光カメラの撮像した両方の画像を常に画像処理する場合、ＣＰＵの処理負荷は非常に高いものになる。 In order to accurately determine the condition of the driver in a moving vehicle while acquiring information on the driver's complexion, both a visible light camera and an invisible light camera are used. It is conceivable to image with both. However, when constantly processing images captured by both the visible light camera and the invisible light camera, the processing load on the CPU becomes very high.

この発明は上記した問題点を解決するためになされたものであり、可視光カメラと非可視光カメラのＯＮ／ＯＦＦを適切に切り替えることで、ＣＰＵの処理負荷を抑えつつ、運転者の顔色の情報および車両走行中の運転者の状態を精度よく検出することができる運転者状態判定装置を提供することを目的とする。 SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems. It is an object of the present invention to provide a driver's state determination device capable of accurately detecting information and the state of a driver while a vehicle is running.

この発明に係わる運転者状態判定装置は、運転者の顔を撮像する可視光カメラの撮像した可視光画像を取得する可視光画像取得部と、可視光画像に基づいて運転者の状態を判定する第１の状態判定部と、運転者の顔を撮像する近赤外線カメラの撮像した近赤外線画像を取得する近赤外線画像取得部と、近赤外線画像に基づいて運転者の状態を判定する第２の状態判定部と、車両が走行状態か停車状態かを判定する車両走行状態判定部と、車両が停車状態の場合は、可視光カメラの電源をＯＮ、近赤外線カメラの電源をＯＦＦの状態にし、車両が走行状態の場合は、可視光カメラをＯＦＦ、近赤外線カメラの電源をＯＮの状態にする電源制御部と、を備える。 A driver condition determination device according to the present invention includes a visible light image acquisition unit that acquires a visible light image captured by a visible light camera that captures an image of a driver's face, and a driver condition determination unit that determines the driver's condition based on the visible light image. A first state determination unit, a near-infrared image acquisition unit that acquires a near-infrared image captured by a near-infrared camera that captures the driver's face, and a second state determination unit that determines the driver's state based on the near-infrared image. A state determination unit, a vehicle running state determination unit for determining whether the vehicle is in a running state or a stopped state, and when the vehicle is in a stopped state, turn on the power of the visible light camera and turn off the power of the near infrared camera, a power control unit that turns off the visible light camera and turns on the power of the near-infrared camera when the vehicle is in a running state.

この発明の運転者状態判定装置は、可視光カメラと近赤外線カメラのＯＮ／ＯＦＦを適切に切り替えることで、ＣＰＵの処理負荷を抑えつつ、運転者の顔色の情報および車両走行中の運転者の状態を精度よく検出することができる。 By appropriately switching ON/OFF of the visible light camera and the near-infrared camera, the driver condition determination device of the present invention suppresses the processing load of the CPU while suppressing the information on the driver's complexion and the image of the driver while the vehicle is running. The state can be detected with high accuracy.

この発明の実施の形態1に係わる運転者状態判定装置１００の構成を示すブロック図である。1 is a block diagram showing the configuration of a driver's condition determination device 100 according to Embodiment 1 of the present invention; FIG. 図２Ａ及び図２Ｂは、この発明の実施の形態に係わる運転者状態判定装置１００のハードウェア構成例を示す図である。2A and 2B are diagrams showing a hardware configuration example of the driver's condition determination device 100 according to the embodiment of the invention. この発明の実施の形態１に係わる運転者状態判定装置１００の動作を表すフローチャートである。4 is a flow chart showing the operation of the driver's condition determination device 100 according to Embodiment 1 of the present invention; この発明の実施の形態１に係わる運転者状態判定装置１００のルームランプ点灯処理の動作を表すフローチャートである。4 is a flow chart showing the operation of room lamp lighting processing of driver's condition determination device 100 according to Embodiment 1 of the present invention;

以下、この発明を実施するための形態について、図を参照して説明する。なお、各図中の同一又は相当する部分には同一の符号を付しており、その重複説明は適宜に簡略化ないし省略する。 BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The same or corresponding parts in each figure are denoted by the same reference numerals, and redundant description thereof will be appropriately simplified or omitted.

実施の形態1
図１は本実施の形態1に係わる運転者状態判定装置１００の構成を示すブロック図である。
運転者状態判定装置１００は、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５および電源制御部６を備える。
図１に示すように、可視光画像取得部１は可視光カメラ１０、近赤外画像取得部３は近赤外線カメラ２０に接続されており、近赤外線カメラ２０には近赤外線ＬＥＤ２１が接続されている。また、電源制御部６は、可視光カメラ１０、近赤外線カメラ２０およびルームランプ３０に接続されている。 Embodiment 1
FIG. 1 is a block diagram showing the configuration of a driver's condition determination device 100 according to the first embodiment.
The driver state determination device 100 includes a visible light image acquisition unit 1, a first state determination unit 2, a near-infrared image acquisition unit 3, a second state determination unit 4, a vehicle running state determination unit 5, and a power supply control unit 6. Prepare.
As shown in FIG. 1, the visible light image acquisition unit 1 is connected to a visible light camera 10, the near-infrared image acquisition unit 3 is connected to a near-infrared camera 20, and the near-infrared camera 20 is connected to a near-infrared LED 21. there is Also, the power control unit 6 is connected to the visible light camera 10 , the near-infrared camera 20 and the room lamp 30 .

可視光カメラ１０は、可視光を撮像可能なＣＣＤ（Ｃｈａｒｇｅ－Ｃｏｕｐｌｅｄ Ｄｅｖｉｃｅ）カメラによって構成され、可視光を撮像する。
近赤外線カメラ２０は、近赤外線を撮像可能なＣＣＤカメラによって構成され、近赤外線ＬＥＤ（Ｌｉｇｈｔ Ｅｍｉｓｓｉｏｎ Ｄｉｏｄｅ）２１から発光された近赤外線を撮像する。近赤外線カメラ２０および近赤外線ＬＥＤ２１は一つのモジュールとなっていてもよいし、分離されていてもよい。また、近赤外線ＬＥＤ２１は、近赤外線カメラ２０の撮像タイミングに同期して発光するものとする。
可視光カメラ１０および近赤外線カメラ２０は、ハンドルコラムやセンターコンソール、ダッシュボード、ルームミラー等に設けられており、少なくとも運転者の顔が撮像範囲に含まれる角度で設置されている。その際、助手席の乗員や後部座席の乗員も同時に撮像できるように、設置位置および画角等が調整されてもよい。 The visible light camera 10 is composed of a CCD (Charge-Coupled Device) camera capable of capturing visible light, and captures visible light.
The near-infrared camera 20 is composed of a CCD camera capable of capturing near-infrared rays, and captures near-infrared rays emitted from a near-infrared LED (Light Emission Diode) 21 . The near-infrared camera 20 and the near-infrared LED 21 may be one module, or may be separated. Also, the near-infrared LED 21 is assumed to emit light in synchronization with the imaging timing of the near-infrared camera 20 .
The visible light camera 10 and the near-infrared camera 20 are provided on a steering column, a center console, a dashboard, a room mirror, etc., and are installed at an angle that includes at least the driver's face in the imaging range. At that time, the installation position, the angle of view, and the like may be adjusted so that the passenger in the front passenger seat and the passenger in the rear seat can be imaged at the same time.

可視光画像取得部１は、可視光カメラ１０が撮像した可視光画像を可視光カメラ１０から取得し、第１の状態判定部２に可視光画像を出力する。
また、可視光画像取得部１は、取得した可視光画像の平均輝度を算出し、当該平均輝度が閾値以下の場合、電源制御部６に、可視光画像の平均輝度が低い旨の通知を行う。
第１の状態判定部２は、可視光画像取得部１から可視光画像が出力された場合、当該可視光画像から運転者の顔色の情報を取得し、当該運転者の顔色の情報に基づいて運転者の飲酒状態を判定する。なお、第１の状態判定部２は、運転者が飲酒状態と判定した場合、不図示の車両制御部に運転者が飲酒状態である旨を出力し、車両が走行できないようにロックをかけさせる。このようにすることで、運転者が飲酒状態の場合に、運転者が車両を運転することが不可能となるので。運転者が飲酒状態のまま車両を運転してしまうことを防ぐことが可能となる。 The visible light image acquisition unit 1 acquires a visible light image captured by the visible light camera 10 from the visible light camera 10 and outputs the visible light image to the first state determination unit 2 .
In addition, the visible light image acquisition unit 1 calculates the average brightness of the acquired visible light image, and if the average brightness is equal to or less than the threshold value, notifies the power supply control unit 6 that the average brightness of the visible light image is low. .
When the visible light image is output from the visible light image acquisition unit 1, the first state determination unit 2 acquires information on the complexion of the driver from the visible light image, and based on the information on the complexion of the driver Determine the drunkenness of the driver. When the first state determination unit 2 determines that the driver is in a drunken state, the first state determination unit 2 outputs to a vehicle control unit (not shown) that the driver is in an intoxicated state, and locks the vehicle so that it cannot travel. . This makes it impossible for the driver to drive the vehicle when the driver is intoxicated. It is possible to prevent the driver from driving the vehicle while intoxicated.

具体的には、第１の状態判定部２は、不図示の記憶部に記憶されている運転者の通常の顔色の情報と可視光画像から取得した運転者の顔色の情報を比較し、可視光画像から取得した運転者の顔色が通常の運転者の顔色よりも閾値以上赤みがかっていると判定された場合、運転者が飲酒状態であると判定する。
第１の状態判定部２が判定する運転者の状態は、運転者の顔色を取得できることで、より高精度に判定できる状態であり、飲酒状態判定以外にも、例えば、化粧状態判定、個人判定（個人認証）等を行ってもよい。 Specifically, the first state determination unit 2 compares information about the driver's normal complexion stored in a storage unit (not shown) with information about the driver's complexion obtained from the visible light image, If it is determined that the driver's complexion obtained from the optical image is more reddish than the normal driver's complexion by a threshold value or more, it is determined that the driver is in a drunken state.
The state of the driver determined by the first state determination unit 2 is a state that can be determined with higher accuracy by being able to acquire the complexion of the driver. (Personal authentication), etc. may be performed.

近赤外線画像取得部３は、近赤外線カメラ２０が撮像した近赤外線画像を近赤外線カメラ２０から取得し、当該近赤外線画像を第２の状態判定部４に出力する。
第２の状態判定部４は、近赤外線画像取得部３から近赤外線画像を受信した場合、当該近赤外線画像から運転者の目の開眼度、顔向き等、色の情報が不要な情報を取得し、取得した目の開眼度、顔向き等に基づいて運転者の覚醒度の低下状態（居眠り状態、運転不能状態等）や脇見状態を判定する。なお、第２の状態判定部４は、運転者が覚醒度の低下状態または脇見状態にあると判定した場合、不図示の警報装置に警報を出力させるように制御する。このようにすることで、運転者が覚醒度の低下状態または脇見状態にある場合に、運転者を正常な状態に戻るように促すことが可能となる。 The near-infrared image acquisition unit 3 acquires the near-infrared image captured by the near-infrared camera 20 from the near-infrared camera 20 and outputs the near-infrared image to the second state determination unit 4 .
When a near-infrared image is received from the near-infrared image acquisition unit 3, the second state determination unit 4 acquires information that does not require color information, such as the driver's eye opening degree and face direction, from the near-infrared image. Then, based on the obtained degree of eye opening, facial orientation, etc., the driver's lowered state of arousal (drowsiness, inability to drive, etc.) or inattentive state is determined. The second state determination unit 4 controls an alarm device (not shown) to output an alarm when it determines that the driver is in a state of reduced arousal or inattentiveness. By doing so, it is possible to encourage the driver to return to a normal state when the driver is in a state of reduced arousal or a state of looking aside.

車両走行状態判定部５は、不図示のＥＣＵ（Ｅｌｅｃｔｒｏｎｉｃ Ｃｏｎｔｒｏｌ Ｕｎｉｔ）等から、シフトポジションの情報を取得し、車両が停車状態か走行状態かを判定する。車両走行状態判定部５は、車両走行状態の判定結果を電源制御部６に出力する。
なお、車両走行状態判定部５は、シフトポジションがパーキングの時には車両が停車状態、シフトポジションがドライブ、リバースまたはニュートラルの時には車両が走行状態と判定する。 The vehicle running state determination unit 5 acquires shift position information from an ECU (Electronic Control Unit) (not shown) or the like, and determines whether the vehicle is in a stopped state or a running state. The vehicle running state determination unit 5 outputs the vehicle running state determination result to the power supply control unit 6 .
The vehicle running state determination unit 5 determines that the vehicle is in a stopped state when the shift position is in parking, and determines that the vehicle is in a running state when the shift position is in drive, reverse or neutral.

電源制御部６は、車両走行状態判定部５が停車状態であると判定した場合、可視光カメラ１０の電源をＯＮ、近赤外線カメラ２０の電源をＯＦＦの状態にする。一方、車両走行状態判定部５が走行状態であると判定した場合、可視光カメラ１０の電源をＯＦＦ、近赤外線カメラ２０の電源をＯＮの状態にする。 When the vehicle running state determination unit 5 determines that the vehicle is stopped, the power control unit 6 turns on the power of the visible light camera 10 and turns off the power of the near-infrared camera 20 . On the other hand, when the vehicle running state determining unit 5 determines that the vehicle is running, the power of the visible light camera 10 is turned off, and the power of the near-infrared camera 20 is turned on.

つまり、車両が停車時には、可視光カメラ１０で運転者を撮像し、撮像した可視光画像から運転者の顔色の情報を取得する。そして、取得した顔色情報から飲酒状態を判定する。
一方、車両が走行中には、近赤外線カメラ２０で運転者を撮像し、撮像した近赤外線画像から運転者の目の開眼度や顔向き等、色の情報が不要な情報を取得する。そして、取得した運転者の目の開眼度や顔向き等に基づいて運転者の覚醒度の低下状態や脇見状態を判定する。
このように、車両が停車中か走行中かに応じて、撮像するカメラを切り替えることによって、ＣＰＵの処理負荷を抑えつつ、運転者の顔色の情報および車両走行中の運転者の状態を精度よく検出することができる。 That is, when the vehicle is stopped, the visible light camera 10 captures an image of the driver, and the facial color information of the driver is acquired from the captured visible light image. Then, the drinking level is determined from the obtained complexion information.
On the other hand, while the vehicle is running, the near-infrared camera 20 captures an image of the driver, and acquires information that does not require color information, such as the degree of eye opening and face direction of the driver, from the captured near-infrared image. Then, it determines whether the driver's arousal level is lowered or whether the driver is looking aside based on the acquired degree of eye opening and face direction of the driver.
In this way, by switching the camera for imaging depending on whether the vehicle is stopped or running, information on the driver's complexion and the state of the driver while the vehicle is traveling can be obtained with high accuracy while suppressing the processing load on the CPU. can be detected.

また、電源制御部６は、可視光画像取得部１から可視光画像の平均輝度が低い旨の通知が行われた場合、車両のルームランプ３０を点灯させる。このようにすることで、夜間での撮像時など、車内が暗く、運転者の顔色の情報をうまく取得できない場合に、ルームランプで車内を照らすことで、車内を明るくすることが可能となり、精度よく運転者の顔色の情報を取得することが可能となる。
また、予め車内に備えられているルームランプ３０を可視光カメラ１０の光源として用いることで、別途可視光カメラ用の光源を備えることが不要となり、コストを削減することが可能となる。
なお、電源制御部６は、車両が走行状態となった場合に、ルームランプ３０が点灯しているか否かを判定し、ルームランプ３０が点灯していると判定した場合は、ルームランプ３０を消灯させる。このようにすることで、近赤外線カメラ２０での撮像時に、ルームランプ３０の光がノイズとして乗ってしまうのを抑制することができる。 Further, when the visible light image acquiring unit 1 notifies that the average brightness of the visible light image is low, the power control unit 6 turns on the room lamp 30 of the vehicle. This makes it possible to brighten the interior of the vehicle by illuminating the interior with the interior lamps when the interior of the vehicle is dark and it is not possible to obtain information on the driver's complexion, such as when taking an image at night. It is often possible to obtain information on the complexion of the driver.
Further, by using the room lamp 30, which is provided in advance in the vehicle, as the light source of the visible light camera 10, it becomes unnecessary to provide a separate light source for the visible light camera, and the cost can be reduced.
The power control unit 6 determines whether or not the room lamp 30 is lit when the vehicle is in a running state. turn off the light. By doing so, it is possible to prevent the light from the room lamp 30 from appearing as noise when an image is captured by the near-infrared camera 20 .

次に、運転者状態判定装置１００のハードウェア構成例を説明する。
図２Ａ及び図２Ｂは、運転者状態判定装置１００のハードウェア構成例を示す図である。
運転者状態判定装置１００における可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６の各機能は、処理回路によって実現される。即ち、運転者状態判定装置１００は上記各機能を実現するための処理回路を備える。当該処理回路は、図２Ａに示すように専用のハードウェアである処理回路２００ａであってもよいし、図２Ｂに示すようにメモリ２００ｃに格納されているプログラムを実行するプロセッサ２００ｂであってもよい。 Next, a hardware configuration example of the driver's condition determination device 100 will be described.
2A and 2B are diagrams showing a hardware configuration example of the driver's condition determination device 100. FIG.
Visible light image acquisition unit 1, first state determination unit 2, near-infrared image acquisition unit 3, second state determination unit 4, vehicle running state determination unit 5, and power supply control unit 6 in driver state determination device 100 Each function is implemented by a processing circuit. That is, the driver's condition determination device 100 includes a processing circuit for realizing each function described above. The processing circuit may be a processing circuit 200a that is dedicated hardware as shown in FIG. 2A, or a processor 200b that executes a program stored in a memory 200c as shown in FIG. 2B. good.

図２Ａに示すように、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６が専用のハードウェアである場合、処理回路２００ａは、例えば、単一回路、複合回路、プログラム化したプロセッサ、並列プログラム化したプロセッサ、ＡＳＩＣ（Ａｐｐｌｉｃａｔｉｏｎ Ｓｐｅｃｉｆｉｃ Ｉｎｔｅｇｒａｔｅｄ Ｃｉｒｃｕｉｔ）、ＦＰＧＡ（Ｆｉｅｌｄ－ｐｒｏｇｒａｍｍａｂｌｅ Ｇａｔｅ Ａｒｒａｙ）、またはこれらを組み合わせたものが該当する。可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６の各部の機能をそれぞれ処理回路で実現してもよいし、各部の機能をまとめて１つの処理回路で実現してもよい。 As shown in FIG. 2A, a visible light image acquisition unit 1, a first state determination unit 2, a near-infrared image acquisition unit 3, a second state determination unit 4, a vehicle running state determination unit 5, and a power supply control unit 6 If dedicated hardware, the processing circuit 200a may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or a combination of these. The functions of the visible light image acquisition unit 1, the first state determination unit 2, the near-infrared image acquisition unit 3, the second state determination unit 4, the vehicle running state determination unit 5, and the power supply control unit 6 are respectively processed by a processing circuit. , or the functions of each unit may be collectively realized by one processing circuit.

図２Ｂに示すように、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６がプロセッサ２００ｂである場合、各部の機能は、ソフトウェア、ファームウェア、またはソフトウェアとファームウェアとの組み合わせにより実現される。ソフトウェアまたはファームウェアはプログラムとして記述され、メモリ２００ｃに格納される。プロセッサ２００ｂは、メモリ２００ｃに記憶されたプログラムを読み出して実行することにより、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６の各機能を実現する。即ち、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６は、プロセッサ２００ｂにより実行されるときに、後述する図３および図４に示す各ステップが結果的に実行されることになる。また、これらのプログラムは、可視光画像取得部１、第１の状態判定部２、近赤外画像取得部３、第２の状態判定部４、車両走行状態判定部５及び電源制御部６の手順または方法をコンピュータに実行させるものであるともいえる。 As shown in FIG. 2B, the visible light image acquisition unit 1, the first state determination unit 2, the near-infrared image acquisition unit 3, the second state determination unit 4, the vehicle running state determination unit 5, and the power supply control unit 6 In the case of the processor 200b, the function of each unit is implemented by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 200c. The processor 200b reads and executes the programs stored in the memory 200c to obtain a visible light image acquisition unit 1, a first state determination unit 2, a near-infrared image acquisition unit 3, a second state determination unit 4, Each function of the vehicle running state determination unit 5 and the power supply control unit 6 is realized. That is, the visible light image acquisition unit 1, the first state determination unit 2, the near-infrared image acquisition unit 3, the second state determination unit 4, the vehicle running state determination unit 5, and the power supply control unit 6 are executed by the processor 200b. 3 and 4, which will be described later, are executed as a result. Further, these programs include the visible light image acquisition unit 1, the first state determination unit 2, the near-infrared image acquisition unit 3, the second state determination unit 4, the vehicle running state determination unit 5, and the power control unit 6. It can also be said that it causes a computer to execute a procedure or a method.

ここで、プロセッサ２００ｂとは、例えば、ＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）、処理装置、演算装置、プロセッサ、マイクロプロセッサ、マイクロコンピュータ、またはＤＳＰ（Ｄｉｇｉｔａｌ Ｓｉｇｎａｌ Ｐｒｏｃｅｓｓｏｒ）などのことである。メモリ２００ｃは、例えば、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）、フラッシュメモリ、ＥＰＲＯＭ（Ｅｒａｓａｂｌｅ Ｐｒｏｇｒａｍｍａｂｌｅ ＲＯＭ）、ＥＥＰＲＯＭ（Ｅｌｅｃｔｒｉｃａｌｌｙ ＥＰＲＯＭ）等の不揮発性または揮発性の半導体メモリであってもよいし、ハードディスク、フレキシブルディスク等の磁気ディスクであってもよいし、ミニディスク、ＣＤ（Ｃｏｍｐａｃｔ Ｄｉｓｃ）、ＤＶＤ（Ｄｉｇｉｔａｌ Ｖｅｒｓａｔｉｌｅ Ｄｉｓｃ）等の光ディスクであってもよい。 Here, the processor 200b is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a processor, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor). The memory 200c may be non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM). However, it may be a magnetic disk such as a hard disk or a flexible disk, or an optical disk such as a mini disk, a CD (Compact Disc) or a DVD (Digital Versatile Disc).

次に、運転者状態判定装置１００の動作について図３を用いて説明する。なお、図３のフローチヤートでは、可視光画像からは飲酒状態の判定を行い、近赤外線画像からは居眠り判定を行う動作を例にとって説明している。
図３は実施の形態１に係る運転者状態判定装置１００の動作を表すフローチャートである。
運転者状態判定装置１００の電源がＯＮになった場合、車両走行状態５は、車両が停車状態か否かを判定する（ステップＳ００１）。車両が停車状態の場合（ステップＳ００２：ＹＥＳ）、電源制御部５は、可視光カメラ１０の電源をＯＮ、近赤外線カメラ２０の電源をＯＦＦの状態にする（ステップＳ００２）。そして、可視光画像取得部１は、可視光カメラ１０の撮像した可視光画像を取得し（ステップＳ００３）、取得した画像に基づいて後述するルームランプ点灯処理（ステップＳ００４）を実施する。次に、第１の状態判定部２は、可視光画像から運転者の顔色を検出し、顔色に基づいて運転者が飲酒状態を判定する（Ｓ００５）。そして、運転者の飲酒状態の判定が終了した場合、ステップ００１の判定に戻る。 Next, the operation of the driver's condition determination device 100 will be described with reference to FIG. Note that in the flowchart of FIG. 3, the operation of judging the drunkenness from the visible light image and judging dozing off from the near-infrared image is explained as an example.
FIG. 3 is a flow chart showing the operation of the driver's condition determination device 100 according to the first embodiment.
When the power of the driver state determination device 100 is turned on, the vehicle running state 5 determines whether or not the vehicle is in a stopped state (step S001). When the vehicle is stopped (step S002: YES), the power control unit 5 turns on the power of the visible light camera 10 and turns off the power of the near-infrared camera 20 (step S002). Then, the visible light image acquisition unit 1 acquires the visible light image captured by the visible light camera 10 (step S003), and performs room lamp lighting processing (step S004), which will be described later, based on the acquired image. Next, the first state determination unit 2 detects the facial color of the driver from the visible light image, and determines the drunkenness of the driver based on the facial color (S005). When the determination of the drunkenness of the driver is completed, the process returns to step 001 for determination.

一方、車両が走行状態の場合（ステップＳ００１：ＮＯ）、電源制御部５は、ルームランプ３０が点灯しているか否かを判定し（ステップＳ００６）、ルームランプ３０が点灯している場合（ステップＳ００６：ＹＥＳ）、ルームランプ３０を消灯し（ステップＳ００７）、可視光カメラ１０の電源をＯＦＦ、近赤外線カメラ２０の電源をＯＮの状態にする（ステップＳ００８）。一方、ルームランプ３０が消灯している場合（ステップＳ００６：ＮＯ）、ステップＳ００７をスキップしステップ００８の処理を実行する、
次に、近赤外線画像取得部３は、近赤外線カメラ２０の撮像した近赤外線画像を取得し（Ｓ００９）、第２の状態判定部４は、近赤外線画像から運転者の居眠り状態を判定する（ステップＳ０１０）。そして、居眠り判定が終了した場合、ステップＳ００１の判定に戻る。 On the other hand, when the vehicle is running (step S001: NO), the power control unit 5 determines whether the room lamp 30 is lit (step S006), and if the room lamp 30 is lit (step S006: YES), the room lamp 30 is turned off (step S007), the power of the visible light camera 10 is turned off, and the power of the near-infrared camera 20 is turned on (step S008). On the other hand, if the room lamp 30 is off (step S006: NO), skip step S007 and execute the process of step 008.
Next, the near-infrared image acquisition unit 3 acquires the near-infrared image captured by the near-infrared camera 20 (S009), and the second state determination unit 4 determines the dozing state of the driver from the near-infrared image ( step S010). Then, when the doze determination is finished, the process returns to the determination of step S001.

次に、ルームランプ点灯処理の動作について、図４を用いて説明する。
図４は、実施の形態１に係わる運転者状態判定装置１００のルームランプ点灯処理の動作を表すフローチャートである。
可視光画像取得部１がステップＳ００２で可視光画像を取得した場合、可視光画像取得部１は、可視光画像の平均輝度を算出する（ステップＳ１０１）。可視光画像取得部１は、ステップＳ１０１で算出した平均輝度が閾値以下か否かを判定し（ステップＳ１０２）、平均輝度が閾値以下の場合、電源制御部６は車両に備えられているルームランプの電源をＯＮにし（Ｓ１０３）、再度可視光画像を取得する（ステップＳ１０４）。そして処理を終了する。一方、平均輝度が閾値より高い場合（ステップＳ１０２：ＮＯ）、ステップＳ１０３およびステップＳ１０４をスキップして、処理を終了する。 Next, the operation of room lamp lighting processing will be described with reference to FIG.
FIG. 4 is a flowchart showing the operation of room lamp lighting processing of the driver's condition determination device 100 according to the first embodiment.
When the visible light image acquisition unit 1 acquires the visible light image in step S002, the visible light image acquisition unit 1 calculates the average brightness of the visible light image (step S101). The visible light image acquisition unit 1 determines whether or not the average luminance calculated in step S101 is equal to or less than the threshold (step S102). is turned on (S103), and a visible light image is acquired again (step S104). Then the process ends. On the other hand, if the average brightness is higher than the threshold (step S102: NO), the process is terminated by skipping steps S103 and S104.

以上のように、実施の形態１によれば、運転者の顔を撮像する可視光カメラ１０の撮像した可視光画像を取得する可視光画像取得部１と、可視光画像に基づいて運転者の状態を判定する第１の状態判定部２と、運転者の顔を撮像する近赤外線カメラ２０の撮像した近赤外線画像を取得する近赤外線画像取得部３と、近赤外線画像に基づいて運転者の状態を判定する第２の状態判定部４と、車両が走行状態か停車状態かを判定する車両走行状態判定部５と、車両が停車状態の場合は、可視光カメラ１０の電源をＯＮ、近赤外線カメラ２０の電源をＯＦＦの状態にし、車両が走行状態の場合は、可視光カメラ１０をＯＦＦ、近赤外線カメラ２０の電源をＯＮの状態にする電源制御部６と、を備える構成としたので、ＣＰＵの処理負荷を抑えつつ、運転者の顔色の情報および車両走行中の運転者の状態を精度よく検出することができる。 As described above, according to Embodiment 1, the visible light image acquisition unit 1 that acquires the visible light image captured by the visible light camera 10 that captures the face of the driver, and the visible light image of the driver based on the visible light image A first state determination unit 2 that determines the state, a near-infrared image acquisition unit 3 that acquires a near-infrared image captured by a near-infrared camera 20 that captures the face of the driver, and a near-infrared image of the driver based on the near-infrared image. A second state determination unit 4 that determines the state, a vehicle running state determination unit 5 that determines whether the vehicle is in a running state or a stopped state, and when the vehicle is in a stopped state, the power of the visible light camera 10 is turned on, The power control unit 6 turns off the power of the infrared camera 20, turns off the visible light camera 10, and turns on the power of the near-infrared camera 20 when the vehicle is running. It is possible to accurately detect information on the complexion of the driver and the condition of the driver while the vehicle is running, while suppressing the processing load on the CPU.

１００ 運転者状態判定装置、１ 可視光画像取得部、２ 第１の状態判定部 、３ 近赤外線画像取得部、４ 第２の状態判定部、５ 車両走行状態判定部、６ 電源制御部、１０ 可視光カメラ、２０ 近赤外線カメラ、２１ 近赤外線ＬＥＤ、３０ ルームランプ、２００ａ 処理回路、２００ｂ プロセッサ、２００ｃ メモリ 100 driver state determination device 1 visible light image acquisition unit 2 first state determination unit 3 near-infrared image acquisition unit 4 second state determination unit 5 vehicle running state determination unit 6 power control unit 10 Visible light camera, 20 near-infrared camera, 21 near-infrared LED, 30 room lamp, 200a processing circuit, 200b processor, 200c memory

Claims (6)

運転者の顔を撮像する可視光カメラの撮像した可視光画像を取得する可視光画像取得部と、
前記可視光画像に基づいて前記運転者の状態を判定する第１の状態判定部と、
前記運転者の顔を撮像する近赤外線カメラの撮像した近赤外線画像を取得する近赤外線画像取得部と、
前記近赤外線画像に基づいて前記運転者の状態を判定する第２の状態判定部と、
車両が走行状態か停車状態かを判定する車両走行状態判定部と、
前記車両が停車状態の場合は、前記可視光カメラの電源をＯＮ、前記近赤外線カメラの電源をＯＦＦの状態にし、前記車両が走行状態の場合は、前記可視光カメラの電源をＯＦＦ、前記近赤外線カメラの電源をＯＮの状態にする電源制御部と、
を備える運転者状態判定装置。 a visible light image acquisition unit that acquires a visible light image captured by a visible light camera that captures an image of the driver's face;
a first state determination unit that determines the state of the driver based on the visible light image;
a near-infrared image acquisition unit that acquires a near-infrared image captured by a near-infrared camera that captures the driver's face;
a second state determination unit that determines the state of the driver based on the near-infrared image;
a vehicle running state determination unit that determines whether the vehicle is running or stopped;
When the vehicle is in a stopped state, the power of the visible light camera is turned on, and the power of the near-infrared camera is turned off. a power control unit for turning on the power of the infrared camera;
A driver state determination device comprising: 前記可視光画像取得部は前記可視光画像の平均輝度を取得し、
前記電源制御部は、前記平均輝度が予め定められた輝度以下の場合、前記車両のルームランプを点灯させる請求項１記載の運転者状態判定装置。 The visible light image acquisition unit acquires the average luminance of the visible light image,
2. The driver's state determination device according to claim 1, wherein said power control unit turns on a room lamp of said vehicle when said average luminance is equal to or lower than a predetermined luminance. 前記電源制御部は、前記車両が走行状態となった場合、前記ルームランプを消灯させる請求項２記載の運転者状態判定装置。 3. The driver's state determination device according to claim 2, wherein the power control unit turns off the room lamp when the vehicle is in a running state. 前記第１の状態判定部は、前記運転者の飲酒状態を判定し、前記第２の状態判定部は、前記運転者の居眠り状態、脇見状態および運転不能状態の内少なくとも１つを判定する請求項１記載の運転者状態判定装置。 The first state determination unit determines the drunkenness state of the driver, and the second state determination unit determines at least one of the driver's state of dozing off, looking aside, and unable to drive. Item 1. The driver's condition determination device according to item 1. 前記車両走行状態判定部は、シフトポジションがパーキングの時に停車状態、シフトポジションがパーキング以外の時に走行状態と判定する請求項１記載の運転者状態判定装置。 2. The driver's state determination device according to claim 1, wherein the vehicle running state determining unit determines that the vehicle is in a stopped state when the shift position is in parking, and determines that the vehicle is in a running state when the shift position is in a position other than parking. 運転者の顔を撮像する可視光カメラの撮像した可視光画像を取得するステップと、
前記可視光画像に基づいて前記運転者の状態を判定するステップと、
前記運転者の顔を撮像する近赤外線カメラの撮像した近赤外線画像を取得するステップと、
前記近赤外線画像に基づいて前記運転者の状態を判定するステップと、
車両が走行状態か停車状態かを判定するステップと、
前記車両が停車状態の場合は、前記可視光カメラの電源をＯＮ、前記近赤外線カメラの電源をＯＦＦの状態にし、前記車両が走行状態の場合は、前記可視光カメラをＯＦＦ、前記近赤外線カメラの電源をＯＮの状態にするステップと、
を備える運転者状態判定方法。 obtaining a visible light image captured by a visible light camera that captures the driver's face;
determining the state of the driver based on the visible light image;
obtaining a near-infrared image captured by a near-infrared camera that captures the driver's face;
determining the state of the driver based on the near-infrared image;
determining whether the vehicle is running or stopped;
When the vehicle is in a stopped state, the power of the visible light camera is turned on and the power of the near infrared camera is turned off, and when the vehicle is in a running state, the visible light camera is turned off and the near infrared camera is turned off. a step of turning on the power of
A driver state determination method comprising:
JP2019049598A 2019-03-18 2019-03-18 DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD Active JP7151571B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2019049598A JP7151571B2 (en) 2019-03-18 2019-03-18 DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2019049598A JP7151571B2 (en) 2019-03-18 2019-03-18 DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD

Publications (2)

Publication Number Publication Date
JP2020152133A JP2020152133A (en) 2020-09-24
JP7151571B2 true JP7151571B2 (en) 2022-10-12

Family

ID=72557339

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2019049598A Active JP7151571B2 (en) 2019-03-18 2019-03-18 DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD

Country Status (1)

Country Link
JP (1) JP7151571B2 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022102494A (en) * 2020-12-25 2022-07-07 スズキ株式会社 Occupant monitoring device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006099395A (en) 2004-09-29 2006-04-13 Aisin Seiki Co Ltd Driver monitoring system
JP2006240500A (en) 2005-03-03 2006-09-14 Furukawa Electric Co Ltd:The Seat storage control device, and seat storage control system
JP2007295946A (en) 2006-04-27 2007-11-15 Konica Minolta Holdings Inc Detection system for alcohol influence and method
JP2010240094A (en) 2009-04-03 2010-10-28 Toyota Motor Corp Apparatus for estimating state under influence of alcohol
JP2018180594A (en) 2017-04-03 2018-11-15 株式会社デンソー Running support device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006099395A (en) 2004-09-29 2006-04-13 Aisin Seiki Co Ltd Driver monitoring system
JP2006240500A (en) 2005-03-03 2006-09-14 Furukawa Electric Co Ltd:The Seat storage control device, and seat storage control system
JP2007295946A (en) 2006-04-27 2007-11-15 Konica Minolta Holdings Inc Detection system for alcohol influence and method
JP2010240094A (en) 2009-04-03 2010-10-28 Toyota Motor Corp Apparatus for estimating state under influence of alcohol
JP2018180594A (en) 2017-04-03 2018-11-15 株式会社デンソー Running support device

Also Published As

Publication number Publication date
JP2020152133A (en) 2020-09-24

Similar Documents

Publication Publication Date Title
JP5327392B1 (en) Vehicle with vehicle side illumination / visual means
JP4564320B2 (en) Driver monitor system
JP5761074B2 (en) Imaging control apparatus and program
EP2471691B1 (en) Obstacle detection device, obstacle detection system provided therewith, and obstacle detection method
KR102128048B1 (en) Welcome ystem with around view monitoring of vehicle and method thereof
JP5233322B2 (en) Information processing apparatus and method, and program
US20190193633A1 (en) Viewing device for vehicle
JP7151571B2 (en) DRIVER'S STATE DETERMINATION DEVICE AND DRIVER'S STATE DETERMINATION METHOD
JP7331483B2 (en) Imaging control device
WO2019215879A1 (en) Occupant monitoring device and occupant monitoring method
US9623819B2 (en) Method for controlling a lighting brightness of a lit motor vehicle instrument as well as a motor vehicle with at least one dimmably lit motor vehicle instrument
JP2006021591A (en) Illumination control system
JP2009120148A (en) Vehicular lamp control system
JP2016107864A (en) Vehicle interior light control device and computer program
US11235723B2 (en) Vehicle system
JP5182567B2 (en) Crew monitoring device
JP7051667B2 (en) In-vehicle device
JP2019127080A (en) Lighting control device, lighting control method, and vehicle lamp fitting
WO2023203630A1 (en) Illumination control device and illumination control method
WO2023032029A1 (en) Blocking determination device, passenger monitoring device, and blocking determination method
JP2018143285A (en) Biological state determination apparatus, biological state determination system and biological state determination method
JP5668937B2 (en) Headlight control device
JP2009303192A (en) Face detection system
JP7158561B2 (en) DRIVER'S CONDITION MONITORING DEVICE AND DRIVER'S CONDITION MONITORING METHOD
WO2024084647A1 (en) Illumination control device and illumination control method

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20210618

RD01 Notification of change of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7421

Effective date: 20220427

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20220606

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20220614

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220727

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220830

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220912

R151 Written notification of patent or utility model registration

Ref document number: 7151571

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151