JP2020149499A - 乗員観察装置 - Google Patents
乗員観察装置 Download PDFInfo
- Publication number
- JP2020149499A JP2020149499A JP2019047661A JP2019047661A JP2020149499A JP 2020149499 A JP2020149499 A JP 2020149499A JP 2019047661 A JP2019047661 A JP 2019047661A JP 2019047661 A JP2019047661 A JP 2019047661A JP 2020149499 A JP2020149499 A JP 2020149499A
- Authority
- JP
- Japan
- Prior art keywords
- occupant
- eye
- unit
- detection unit
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K28/00—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
- B60K28/02—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
- B60K28/06—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver responsive to incapacity of driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
- B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
- B60R21/015—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting the presence or position of passengers, passenger seats or child seats, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
- B60R21/01512—Passenger detection systems
- B60R21/01552—Passenger detection systems detecting position of specific human body parts, e.g. face, eyes or hands
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/225—Direction of gaze
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Transportation (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Eye Examination Apparatus (AREA)
- Image Processing (AREA)
Abstract
Description
(1):この発明の一態様に係る乗員観察装置は、車両の乗員の頭部を撮像する撮像部と、前記撮像部により撮像された画像において、少なくとも前記乗員の眼の一部を検出する眼検出部と、前記撮像部により撮像された画像において、少なくとも前記乗員の口の一部を検出する口検出部と、前記眼検出部の検出結果と前記口検出部の結果とに基づいて前記乗員の状態を推定する状態推定部と、を備え、前記状態推定部は、前記眼検出部の検出結果または前記眼検出部の検出結果に基づいて行われる処理の結果に基づいて、前記眼検出部の検出結果と前記口検出部の結果とのそれぞれを前記乗員の状態の推定に反映させる割合を変更するものである。
(1)エッジEGからの眼の輪郭へのフィッティング率が基準よりも低い。
(2)エッジEGの分布のフィッティングした曲線に対するバラつきが基準よりも大きい。
(3)瞳孔が判別できない、或いは白目と黒目が判別できない。
(4)開眼度合いRoが所定度合い以下である状態が所定時間以上継続した。
10 撮像部
20 画像処理装置
24 眼検出部
26 口検出部
28 開眼率導出部
30 状態推定部
100 各種車載装置
Claims (4)
- 車両の乗員の頭部を撮像する撮像部と、
前記撮像部により撮像された画像において、少なくとも前記乗員の眼の一部を検出する眼検出部と、
前記撮像部により撮像された画像において、少なくとも前記乗員の口の一部を検出する口検出部と、
前記眼検出部の検出結果と前記口検出部の結果とに基づいて前記乗員の状態を推定する状態推定部と、を備え、
前記状態推定部は、前記眼検出部の検出結果または前記眼検出部の検出結果に基づいて行われる処理の結果に基づいて、前記眼検出部の検出結果と前記口検出部の検出結果とのそれぞれを前記乗員の状態の推定に反映させる割合を変更する、
乗員観察装置。 - 前記眼検出部は、前記乗員の眼の輪郭の一部を少なくとも検出し、
前記眼検出部により検出された輪郭における複数の特徴点の位置関係に基づいて、前記乗員の眼の開眼率を導出する開眼率導出部を更に備え、
前記状態推定部は、前記乗員の眼の開眼率と、前記口検出部の検出結果とに基づいて前記乗員の状態を推定し、
前記眼検出部の検出結果に基づいて行われる処理の結果は、前記開眼率導出部の処理によって得られる前記乗員の眼の開眼度合いである、
請求項1記載の乗員観察装置。 - 前記状態推定部は、前記乗員の眼の開眼度合いが所定度合い以下である状態が所定時間以上継続した場合、前記眼検出部の検出結果と前記口検出部の検出結果とを前記乗員の状態の推定に反映させる割合を変更する、
請求項2記載の乗員観察装置。 - 前記状態推定部は、前記眼検出部の検出結果と前記口検出部の検出結果とを前記乗員の状態の推定に反映させる割合を変更する場合、前記眼検出部の結果を前記乗員の状態の推定に反映させる割合を小さくする、
請求項1から3のうちいずれか1項記載の乗員観察装置。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019047661A JP7240910B2 (ja) | 2019-03-14 | 2019-03-14 | 乗員観察装置 |
CN202010164050.9A CN111688709A (zh) | 2019-03-14 | 2020-03-10 | 乘员观察装置 |
US16/815,034 US11161470B2 (en) | 2019-03-14 | 2020-03-11 | Occupant observation device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019047661A JP7240910B2 (ja) | 2019-03-14 | 2019-03-14 | 乗員観察装置 |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2020149499A true JP2020149499A (ja) | 2020-09-17 |
JP7240910B2 JP7240910B2 (ja) | 2023-03-16 |
Family
ID=72424870
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2019047661A Active JP7240910B2 (ja) | 2019-03-14 | 2019-03-14 | 乗員観察装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US11161470B2 (ja) |
JP (1) | JP7240910B2 (ja) |
CN (1) | CN111688709A (ja) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11568655B2 (en) * | 2020-03-26 | 2023-01-31 | Intel Corporation | Methods and devices for triggering vehicular actions based on passenger actions |
SE2250299A1 (en) * | 2022-03-04 | 2023-09-05 | Tobii Ab | Eye openness |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008225838A (ja) * | 2007-03-13 | 2008-09-25 | Aisin Seiki Co Ltd | 顔特徴点検出装置、顔特徴点検出方法及びプログラム |
JP2010204984A (ja) * | 2009-03-04 | 2010-09-16 | Nissan Motor Co Ltd | 運転支援装置 |
JP2017194772A (ja) * | 2016-04-19 | 2017-10-26 | トヨタ自動車株式会社 | 覚醒度判定装置 |
JP2018132974A (ja) * | 2017-02-16 | 2018-08-23 | アルパイン株式会社 | 状態検出装置、状態検出方法、及びプログラム |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3293171B2 (ja) | 1992-06-11 | 2002-06-17 | ジェイエスアール株式会社 | カラーフィルター用感放射線性組成物およびカラーフィルター |
US8879790B2 (en) * | 2010-02-22 | 2014-11-04 | Toyota Jidosha Kabushiki Kaisha | Image processing device and drowsiness assessment device |
CN104809445B (zh) * | 2015-05-07 | 2017-12-19 | 吉林大学 | 基于眼部和嘴部状态的疲劳驾驶检测方法 |
WO2017208529A1 (ja) * | 2016-06-02 | 2017-12-07 | オムロン株式会社 | 運転者状態推定装置、運転者状態推定システム、運転者状態推定方法、運転者状態推定プログラム、対象者状態推定装置、対象者状態推定方法、対象者状態推定プログラム、および記録媒体 |
US20180012090A1 (en) * | 2016-07-07 | 2018-01-11 | Jungo Connectivity Ltd. | Visual learning system and method for determining a driver's state |
CN106781282A (zh) * | 2016-12-29 | 2017-05-31 | 天津中科智能识别产业技术研究院有限公司 | 一种智能行车驾驶员疲劳预警*** |
US20190092337A1 (en) * | 2017-09-22 | 2019-03-28 | Aurora Flight Sciences Corporation | System for Monitoring an Operator |
-
2019
- 2019-03-14 JP JP2019047661A patent/JP7240910B2/ja active Active
-
2020
- 2020-03-10 CN CN202010164050.9A patent/CN111688709A/zh active Pending
- 2020-03-11 US US16/815,034 patent/US11161470B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008225838A (ja) * | 2007-03-13 | 2008-09-25 | Aisin Seiki Co Ltd | 顔特徴点検出装置、顔特徴点検出方法及びプログラム |
JP2010204984A (ja) * | 2009-03-04 | 2010-09-16 | Nissan Motor Co Ltd | 運転支援装置 |
JP2017194772A (ja) * | 2016-04-19 | 2017-10-26 | トヨタ自動車株式会社 | 覚醒度判定装置 |
JP2018132974A (ja) * | 2017-02-16 | 2018-08-23 | アルパイン株式会社 | 状態検出装置、状態検出方法、及びプログラム |
Also Published As
Publication number | Publication date |
---|---|
CN111688709A (zh) | 2020-09-22 |
US11161470B2 (en) | 2021-11-02 |
US20200290543A1 (en) | 2020-09-17 |
JP7240910B2 (ja) | 2023-03-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3488382B1 (en) | Method and system for monitoring the status of the driver of a vehicle | |
US20200334477A1 (en) | State estimation apparatus, state estimation method, and state estimation program | |
JP4826506B2 (ja) | 視線推定装置 | |
JP6338701B2 (ja) | 瞬間的な眠りを検出するデバイス、方法およびコンピュータプログラム | |
US10346698B2 (en) | Vehicle display system and method of controlling vehicle display system | |
JP5482737B2 (ja) | 視認負荷量推定装置、運転支援装置、および視認負荷量推定プログラム | |
JP2016539446A (ja) | 動作と、顔面と、目及び口形状の認知を通じた2段階に亘っての居眠り運転の防止装置 | |
JP2008113902A (ja) | 眼開度検出装置及び方法 | |
JP6973928B2 (ja) | 瞼開閉判定装置および眠気検知装置 | |
JPWO2013008305A1 (ja) | 瞼検出装置 | |
JP6855872B2 (ja) | 顔認識装置 | |
US11161470B2 (en) | Occupant observation device | |
JP2009219555A (ja) | 眠気検知装置、運転支援装置、眠気検知方法 | |
WO2020222785A1 (en) | Facial action unit detection | |
US11077814B2 (en) | Occupant eye(s) observation device | |
US10945651B2 (en) | Arousal level determination device | |
CN109657550B (zh) | 一种疲劳度检测方法及装置 | |
US20220388527A1 (en) | Method and System for Monitoring an Occupant of a Vehicle | |
KR20170028631A (ko) | 정면 얼굴 영상의 복원을 통한 운전자 부주의 검출 방법 및 장치 | |
KR101786579B1 (ko) | 정면 얼굴 판단 방법 및 장치 | |
JP2011086051A (ja) | 眼位置認識装置 | |
JP2021007717A (ja) | 乗員観察装置、乗員観察方法、及びプログラム | |
JP7286022B2 (ja) | 乗員状態検出装置および乗員状態検出方法 | |
JP2011095827A (ja) | 眼部検出装置 | |
CN117765515A (zh) | 一种疲劳与不良行为监测预警方法及*** |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20211126 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20220930 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20221004 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20221129 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20230214 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20230306 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 7240910 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |