TWI818395B - Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method - Google Patents
Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method Download PDFInfo
- Publication number
- TWI818395B TWI818395B TW110149219A TW110149219A TWI818395B TW I818395 B TWI818395 B TW I818395B TW 110149219 A TW110149219 A TW 110149219A TW 110149219 A TW110149219 A TW 110149219A TW I818395 B TWI818395 B TW I818395B
- Authority
- TW
- Taiwan
- Prior art keywords
- driver
- vehicle
- recommended
- monitoring method
- recommended action
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012544 monitoring process Methods 0.000 title claims abstract description 36
- 230000000694 effects Effects 0.000 title claims abstract description 24
- 230000009471 action Effects 0.000 claims abstract description 81
- 230000004044 response Effects 0.000 claims abstract description 14
- 238000011156 evaluation Methods 0.000 claims 1
- 230000006399 behavior Effects 0.000 description 13
- 230000001976 improved effect Effects 0.000 description 11
- 206010041349 Somnolence Diseases 0.000 description 7
- 230000000875 corresponding effect Effects 0.000 description 7
- 230000001965 increasing effect Effects 0.000 description 7
- 230000036642 wellbeing Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000284 resting effect Effects 0.000 description 3
- 230000004424 eye movement Effects 0.000 description 2
- 230000001939 inductive effect Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 241001282135 Poromitra oscitans Species 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 235000012054 meals Nutrition 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000010490 psychological well-being Effects 0.000 description 1
- 230000002040 relaxant effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/146—Display means
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Auxiliary Drives, Propulsion Controls, And Safety Devices (AREA)
Abstract
Description
本發明涉及用於監控一車輛駕駛員的活動的一駕駛員監控方法，包括步驟：評估該駕駛員的疲勞程度(fatigue level of the driver)及/或該駕駛員的等待時間(time spent waiting by the driver)，確立該車輛的駕駛脈絡(driving context)及/或車輛狀態(vehicle status)，輸出用於改變該駕駛員的活動的多個推薦行動中的一者(one of a plurality of recommended actions)基於該疲勞程度、該等待時間、該推薦行動的一成功率、該駕駛脈絡及/或該車輛狀態，評估該駕駛員對該推薦行動的回應以及評估與更新該推薦行動的成功率。本發明還涉及一種車輛，包括用於執行一對應方法的一控制器、一輸入裝置及一輸出裝置。The invention relates to a driver monitoring method for monitoring the activities of a vehicle driver, including the steps of: assessing the driver's fatigue level (fatigue level of the driver) and/or the driver's time spent waiting by the driver), establishes the driving context and/or vehicle status of the vehicle, and outputs one of a plurality of recommended actions for changing the driver's activities. ) Based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and/or the vehicle status, evaluate the driver's response to the recommended action and evaluate and update the success rate of the recommended action. The invention also relates to a vehicle, including a controller, an input device and an output device for executing a corresponding method.
用於監控駕駛員的當前方法及對應系統被設計為檢測一車輛駕駛員的活動，從而可以檢測到危險或非期望的活動，對其作出反應，使得駕駛員的安全感(safety)及幸福感(wellbeing)可被提高。Current methods and corresponding systems for monitoring drivers are designed to detect the activities of a vehicle driver so that dangerous or undesirable activities can be detected and reacted to, thereby improving the driver's safety and well-being. (wellbeing) can be improved.
例如，可以檢測一駕駛員的疲勞或困倦或該駕駛員被迫保持空閒並等待某些事件的一情況，以及對其作出反應。顯然，所述術語活動應在廣義上被理解並且可以意指為指示一疲倦的駕駛員的行為，諸如該車輛的不穩定控制行跡或駕駛員的眼睛的過久閉合。此外，該駕駛員的監控行為可以指示該車輛可能處於靜止並且該駕駛員可能沒有進行與該車輛行跡相關的任何重要控制活動的一等待情況。For example, it is possible to detect and react to a driver's fatigue or drowsiness or a situation in which the driver is forced to remain idle and wait for certain events. Obviously, the term activity is to be understood in a broad sense and may mean behavior indicative of a tired driver, such as erratic control behavior of the vehicle or excessive closing of the driver's eyes. Additionally, the driver's monitoring behavior may indicate a waiting situation where the vehicle may be stationary and the driver may not be performing any significant control activities related to the vehicle's behavior.
一旦一對應的活動被檢測，諸多預定義的反應可被產生，以減少昏昏欲睡的駕駛員相關的事故風險或增加一等待的駕駛員的幸福感。然而，已知的方法對於諸多預定義的反應是否真的適合各自的駕駛員並實現減少昏昏欲睡的駕駛員的駕駛量或增加駕駛員在怠速情況下的幸福感的諸多期望結果是不敏感的。這代表一個問題，因為已知的方法在減少昏昏欲睡的駕駛員的駕駛量及改善駕駛員的幸福感方面無效，或者更糟糕的是，該駕駛員實際上可能會發現該多個預定義的反應對她及她來說是一種滋擾作法而結果可能完全停止該方法的執行。Once a corresponding activity is detected, predefined responses can be generated to reduce the risk of accidents associated with drowsy drivers or to increase the well-being of a waiting driver. However, the known methods are uncertain as to whether the many predefined reactions are actually suitable for the respective driver and achieve the desired results of reducing the driving volume of drowsy drivers or increasing the driver's happiness in idling situations. Sensitive. This represents a problem because known methods are ineffective at reducing the amount of driving by a drowsy driver and improving the driver's well-being, or worse, the driver may actually find that the multiple predispositions The defined reaction is a nuisance to her and her and the result may be a complete cessation of the execution of the method.
本發明的目的是提供克服上述問題的一改良方法及一改良車輛。The object of the present invention is to provide an improved method and an improved vehicle to overcome the above problems.
此目的被實現，通過根據請求項1的用於監控一車輛駕駛員的活動的一駕駛員監控方法及根據請求項10的一車輛，包括用於執行一對應方法的一控制器、一輸入裝置及一輸出裝置。本發明的諸多較佳實施例經受諸多附屬項。This object is achieved by a driver monitoring method for monitoring the activity of a vehicle driver according to claim 1 and a vehicle according to claim 10, comprising a controller for executing a corresponding method, an input device and an output device. The preferred embodiments of the invention are subject to numerous appendages.
根據請求項1，提供一種駕駛員監控方法，用於監控一車輛駕駛員的活動，該駕駛員監控方法包括步驟：評估該駕駛員的一疲勞程度及/或該駕駛員的等待時間；確立該車輛的駕駛脈絡及/或車輛狀態；輸出用於改變該駕駛員的活動的多個推薦行動中的一者，基於該疲勞程度、該等待時間、該推薦行動的一成功率、該駕駛脈絡及/或該車輛狀態；評估該駕駛員對該推薦行動的回應；及評估與更新該推薦行動的該成功率。According to claim 1, a driver monitoring method is provided for monitoring the activities of a vehicle driver. The driver monitoring method includes the steps of: assessing a fatigue degree of the driver and/or the waiting time of the driver; establishing the The driving context and/or vehicle status of the vehicle; output one of a plurality of recommended actions for changing the driver's activity, based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context, and /or the vehicle status; evaluate the driver's response to the recommended action; and evaluate and update the success rate of the recommended action.
所述術語監控(monitoring)應從廣義上被理解，包括對該駕駛員的行為的監視(surveillance)以及對改變或增強該駕駛員的行為的建議(suggestion)。The term monitoring is to be understood in a broad sense, including surveillance of the driver's behavior as well as suggestions for changing or enhancing the driver's behavior.
該諸多推薦行動可能包括改變該車輛的設置，例如其內部照明或座椅位置。它們可能包括播放某些音樂或發起一些待被該駕駛員玩的遊戲。該諸多推薦行動可以包括對該駕駛員可以進行的活動的諸多建議，例如去散步、去小憩或進行某種的冥想(meditation)。該諸多推薦行動可以直接輸出或執行，或者可以提示該駕駛員在他實際輸出或執行之前確認一建議行動(a suggested action)。一般而言，一推薦行動(a recommended action)的輸出可以包括輸出光學及/或視覺資訊及/或改變該車輛的物理配置。Many of the recommended actions may include changing the vehicle's settings, such as its interior lighting or seating position. They may include playing some music or initiating some games to be played by the driver. The recommended actions may include suggestions for activities that the driver can do, such as going for a walk, taking a nap, or engaging in some kind of meditation. The recommended actions can be output or executed directly, or the driver can be prompted to confirm a suggested action before he actually outputs or executes it. Generally speaking, the output of a recommended action may include outputting optical and/or visual information and/or changing the physical configuration of the vehicle.
一給定的推薦行動的成功率可以是一駕駛員接受一推薦行動的次數與對一駕駛員建議的行動的總次數的比率的一函數。The success rate of a given recommended action may be a function of the ratio of the number of times a driver accepts a recommended action to the total number of times a driver is recommended an action.
根據本發明，可以識別一特定駕駛員，使得該方法可被適用於一給定車輛的每個獨特的駕駛員。該駕駛員的識別可以基於一些面部識別步驟及對應硬體諸如一駕駛員監控攝影機被進行。According to the present invention, a specific driver can be identified so that the method can be applied to each unique driver of a given vehicle. The identification of the driver may be performed based on some facial recognition steps and corresponding hardware such as a driver monitoring camera.
在一駕駛員使用多輛車輛的情況下，該多個車輛可被相互連接，使得該方法可以在由同一駕駛員駕駛的任何車輛上被執行。因此，該方法不限於單一車輛。In the case of multiple vehicles used by one driver, the multiple vehicles can be interconnected so that the method can be performed on any vehicle driven by the same driver. Therefore, the method is not limited to a single vehicle.
當前描述的方法可以在該駕駛員正在駕駛該車輛時被連續執行。替代地，該方法的執行可以由一些預定義的訊號來發起，諸如一潛在的駕駛錯誤的發生，諸如突然轉向及/或打破或穿越一些道路標記。The presently described method may be performed continuously while the driver is driving the vehicle. Alternatively, execution of the method may be initiated by some predefined signal, such as the occurrence of a potential driving error, such as sudden steering and/or breaking or crossing some road markings.
該方法可被並行地執行，即，同時地並且關於不同的建議行動，使得多於一個的推薦行動可以被輸出給該駕駛員。The method may be performed in parallel, ie simultaneously and with respect to different recommended actions, so that more than one recommended action may be output to the driver.
根據本發明，危險駕駛被降低，通過有效地對抗疲勞、提高對抗駕駛員困倦的方法的有效性，以及通過使諸多措施適應該獨特的駕駛員以提高用戶體驗。此外，該駕駛員的總體幸福感可以被增強，因為可以支持駕駛員提升潛在的疲乏等候期(wearing waiting periods)。According to the present invention, risky driving is reduced by effectively combating fatigue, increasing the effectiveness of methods to combat driver drowsiness, and improving the user experience by adapting measures to the unique driver. Furthermore, the driver's overall well-being can be enhanced as the driver can be supported in potential wearing waiting periods.
與當前的駕駛員監控系統相比，此解決方案整合諸多升級狀態並提供諸多抵消措施(譬如諸多模式)，首先，取決於該駕駛員的當前活動或狀態的類型及她的疲勞程度，其次，一提議解決方案或推薦行動的先前確立的成功率(previously established success rate of a proposed solution or recommended action)。基於這些，演算法可以學習及適應最有效的方法，以抵消個人的疲勞及/或改善她的整體幸福感。為了實現這一點，該方法可以包括一駕駛員識別步驟，其中該駕駛員的身份被確立。因此，該方法可以適用於各種駕駛員及他們的個人特徵及偏好。In contrast to current driver monitoring systems, this solution integrates numerous upgrade states and provides numerous offsets (e.g. numerous modes), firstly, depending on the type of current activity or state of the driver and her level of fatigue, and secondly, A previously established success rate of a proposed solution or recommended action. Based on these, the algorithm can learn and adapt the most effective ways to offset an individual's fatigue and/or improve her overall well-being. To achieve this, the method may include a driver identification step in which the identity of the driver is established. Therefore, the method can be adapted to a variety of drivers and their personal characteristics and preferences.
該疲勞程度可以是該駕駛員的實際疲勞相關聯的且可為一疲勞函數的一些分數，該疲勞函數指示該駕駛員的行為，諸如打哈欠、眼球運動及/或錯誤駕駛行為。此行為可以通過當前描述的方法被檢測並且可以用於評估或計算該疲勞程度。The fatigue level may be related to the driver's actual fatigue and may be some score of a fatigue function indicative of the driver's behavior, such as yawning, eye movements, and/or erroneous driving behavior. This behavior can be detected by the currently described methods and can be used to assess or calculate the degree of fatigue.
本發明提供一種基於脈絡(context-based)的推薦演算法，其中如果一駕駛員的疲勞情況或一駕駛員的等待情況或態樣被檢測，一車輛可以進行一些當前描述的補救措施(remedies)。The present invention provides a context-based recommendation algorithm in which a vehicle can perform some of the currently described remedies if a driver's fatigue condition or a driver's waiting condition or attitude is detected. .
在本發明的一較佳實施例中，通過一駕駛員監控攝影機及/或通過評估由該駕駛員進行的橫向及縱向車輛控制，以評估該駕駛員的疲勞程度及/或駕駛員的等待時間。In a preferred embodiment of the invention, the driver's fatigue level and/or the driver's waiting time are assessed via a driver monitoring camera and/or by evaluating the lateral and longitudinal vehicle controls performed by the driver. .
在本發明的另一較佳實施例中，確立該車輛的駕駛脈絡及/或車輛狀態包括：確立一當前工作日、一日間時段、一駕駛時間、一天氣、一道路類型、一駕駛歷史、多個先進駕駛員輔助系統響應、一電池電量及/或該車輛的一位置。In another preferred embodiment of the present invention, establishing the driving context and/or vehicle status of the vehicle includes: establishing a current working day, a day time period, a driving time, a weather, a road type, a driving history, Advanced driver assistance system responses, a battery level, and/or a location of the vehicle.
在本發明的另一較佳實施例中，該多個推薦行動包括建議一刷新模式及/或一休息模式。諸多另外的模式可以被附加地或替代地應用。一模式或一模式相關聯的諸多行動的建議可以包括改變一些車輛設定、輸出一些資訊、提示該駕駛員以確認及發起一些資訊的輸出及/或能夠改良駕駛安全及增加該駕駛員的幸福感的諸多其他方法步驟。該諸多模式可以被理解為各種可推薦行動的一集合(a collection)，其中每個模式可以與一特定駕駛員的疲勞程度及/或駕駛員活動相關聯。In another preferred embodiment of the present invention, the plurality of recommended actions include suggesting a refresh mode and/or a rest mode. Many additional modes may be applied additionally or alternatively. Suggestions for a mode or a plurality of actions associated with a mode may include changing some vehicle settings, outputting some information, prompting the driver to confirm and initiate the output of some information, and/or improving driving safety and increasing the driver's happiness. many other method steps. The patterns may be understood as a collection of recommendable actions, where each pattern may be associated with a specific driver's fatigue level and/or driver activity.
在本發明的另一較佳實施例中，該刷新模式在該駕駛員的疲勞程度低於該休息模式時被輸出。如果該疲勞程度超過一閾值或由於檢測譬如在一鄉村道路上的一長駕駛時間而使疲勞被視為是可能的，則提議一對應抵消措施(a corresponding counteracting measure)，即一推薦行動。In another preferred embodiment of the present invention, the refresh mode is output when the driver's fatigue level is lower than the rest mode. If the fatigue level exceeds a threshold or fatigue is considered possible due to detection of, for example, a long driving time on a country road, a corresponding counteracting measure, ie a recommended action, is proposed.
在本發明的另一較佳實施例中，該休息模式包括：建議一休息區(a resting area)，特別是具備一電池充電站的一休息區；建議設置用於輸出一警報的一定時器、基於該電池的充電狀態輸出一警報、播放音樂及/或提供一按摩以向該駕駛員發出訊號表明該休息模式的結束；建議執行噪音消除、抑制無關消息、設定環境光及/或溫度。In another preferred embodiment of the invention, the resting mode includes: proposing a resting area, in particular a resting area with a battery charging station; proposing a timer for outputting an alarm , output an alarm based on the charge status of the battery, play music and/or provide a massage to signal the end of the rest mode to the driver; it is recommended to perform noise cancellation, suppress irrelevant messages, set ambient light and/or temperature.
在本發明的另一較佳實施例中，該刷新模式包括：建議一娛樂活動。In another preferred embodiment of the present invention, the refresh mode includes: suggesting an entertainment activity.
該刷新模式可以作為一推薦行動被輸出，尤其是當該駕駛員監控方法已經評估該駕駛員已經耗費超過一預定量的時間等待及/或沒有顯著控制該車輛時。當該駕駛員陷入一交通擁塞或正在等待其他乘客上車時，可能會出現這種情況。替代地或附加地，當該駕駛員監控方法已經評估為一低疲勞程度時，該刷新模式可以被輸出。The refresh pattern may be output as a recommended action, particularly when the driver monitoring method has assessed that the driver has spent more than a predetermined amount of time waiting and/or not significantly controlling the vehicle. This may occur when the driver is stuck in a traffic jam or is waiting for other passengers to board the vehicle. Alternatively or additionally, the refresh mode may be output when the driver monitoring method has assessed a low fatigue level.
該刷新模式中的推薦行動的輸出可以包括輸出諸多冥想指令，使得該駕駛員可以利用該等待時間以增加心理幸福感。在這種情況下，該車輛可以被設定為提供一輕鬆的環境，例如通過播放合適的音樂或自然聲音、提供合適的環境照明及/或一按摩功能。類似地，進行輕瑜伽(譬如調息，Pranayama)的指導可以在該刷新模式被輸出。The output of recommended actions in the refresh mode may include outputting many meditation instructions, so that the driver can use the waiting time to increase psychological well-being. In this case, the vehicle can be configured to provide a relaxing environment, for example by playing suitable music or nature sounds, providing suitable ambient lighting and/or a massage function. Similarly, instructions for performing light yoga (such as Pranayama) can be output in this refresh mode.
原則上，任何類型的可能被該駕駛員視為有幫助的資訊都可以在該刷新模式被輸出。這可以包括玩賞電子書、在該車輛的計算機系統上致能(enable)一遊戲諸如棋藝或一紙牌遊戲或建議在一附近區域散步。該駕駛員可以接受一合適的推薦行動，並且該方法確保該被選擇的推薦行動的成功率被據以更新。In principle, any type of information that the driver may consider helpful can be output in this refresh mode. This may include browsing an e-book, enabling a game on the vehicle's computer system such as board games or a card game, or suggesting a walk in a nearby area. The driver can accept an appropriate recommended action, and the method ensures that the success rate of the selected recommended action is updated accordingly.
在本發明的另一較佳實施例中，評估該駕駛員對該推薦行動的回應包括：確立哪些推薦行動已被該駕駛員接受以及被接受到何種程度。In another preferred embodiment of the invention, assessing the driver's response to the recommended actions includes establishing which recommended actions have been accepted by the driver and to what extent.
因此，每次駕駛員接受該推薦行動時，一接受推薦行動的該成功率可以被增加。因此，該駕駛員對該推薦行動的回應被評估。在未來情況下，該方法可能會更頻繁地建議一些推薦行動，如果該多個推薦行動的成功率高於其他行動的成功率。已被一駕駛員頻繁選擇或接受並且改善一駕駛員的疲勞程度的一推薦行動可以被儲存及/或據以被評定(rated)並且可以被建議為一最喜愛者，例如在將來的情況下更頻繁地，尤其是在類似的脈絡中及/或當類似的疲勞程度被檢測時。Therefore, the success rate of accepting a recommended action can be increased each time the driver accepts the recommended action. Therefore, the driver's response to the recommended action is evaluated. In future situations, the method may suggest some recommended actions more frequently if the success rate of the multiple recommended actions is higher than the success rate of other actions. A recommended action that has been frequently selected or accepted by a driver and improves a driver's fatigue level may be stored and/or rated accordingly and may be suggested as a favorite, for example in future situations More frequently, especially in similar contexts and/or when similar levels of fatigue are detected.
該方法可以監控一推薦行動已經被該駕駛員接受或執行多久，以便確定該推薦行動已經被接受何種程度。The method can monitor how long a recommended action has been accepted or performed by the driver in order to determine to what extent the recommended action has been accepted.
在本發明的另一較佳實施例中，評估該駕駛員對該推薦行動的回應包括：在該推薦行動已被完成後，評估該駕駛員的疲勞程度。In another preferred embodiment of the present invention, evaluating the driver's response to the recommended action includes: evaluating the driver's fatigue level after the recommended action has been completed.
本發明還針對一種車輛，包括一控制器、一輸入裝置及一輸出裝置，用於執行根據請求項1至9任一項所述的方法。該車輛可以被設計成能夠執行任何當前概述的方法步驟，即，該車輛可以包括執行當前描述的方法步驟所必需的任何硬體。The present invention is also directed to a vehicle, including a controller, an input device and an output device for executing the method according to any one of claims 1 to 9. The vehicle may be designed to be capable of performing any of the presently outlined method steps, ie the vehicle may comprise any hardware necessary to perform the presently described method steps.
因此，本發明提供用於監控一駕駛員的一脈絡感知及智能方法的諸多益處，該多個益處包括被提高的駕駛安全性、被改良的對抗方法的有效性以及由於一高度個性化及可調適的監控方法而被改善的駕駛員滿意度。此外，本發明強化一智能車輛的感知性。The present invention therefore provides numerous benefits of a context-aware and intelligent approach to monitoring a driver, including increased driving safety, improved effectiveness of countermeasures and improved safety due to a highly personalized and configurable approach. Improved driver satisfaction through adapted monitoring methods. In addition, the present invention enhances the perception of an intelligent vehicle.
在第1圖所示的一個方法步驟中，車輛的脈絡可以被檢查。這可以對應於確立該車輛的駕駛脈絡及/或車輛狀態。在此步驟中，從一些可存取的地圖資料及/或從諸如捕捉該車輛周圍環境的一攝影機的諸多車輛感測器，可以檢索該車輛所在的道路類型。進一步的資料，諸如日間時段時間、交通密度、駕駛員行為及/或某些駕駛員行為的規律性，可以從諸多車輛感測器、從諸多車輛儲存裝置諸如硬碟驅動器及/或從連接到該車輛並且本發明方法可存取的其他來源獲得，諸如網際網路及/或合適的雲端儲存來源。In a method step shown in Figure 1, the context of the vehicle can be checked. This may correspond to establishing the driving context and/or vehicle status of the vehicle. In this step, the road type on which the vehicle is located may be retrieved from accessible map data and/or from vehicle sensors such as a camera capturing the vehicle's surroundings. Further data, such as time of day, traffic density, driver behavior and/or regularity of certain driver behaviors, may be obtained from vehicle sensors, from vehicle storage devices such as hard drives and/or from connections to The vehicle and methods of the invention may be obtained from other sources accessible, such as the Internet and/or suitable cloud storage sources.
在另一方法步驟中，該駕駛員的疲勞程度及/或駕駛員的等待時間可以根據從一駕駛員監控攝影機及/或從測量該車輛的駕駛特性的諸多感測器所檢索的資料來評估。In another method step, the driver's fatigue level and/or the driver's waiting time can be assessed based on data retrieved from a driver monitoring camera and/or from sensors measuring driving characteristics of the vehicle. .
如果同時發現該車輛的駕駛脈絡及該駕駛員的當前活動表示一些不希望的情況，諸如一昏昏欲睡的駕駛員以一不安全的方式駕駛該汽車，或者一駕駛員在該汽車內等待，而該汽車是靜止的，可以根據該方法輸出多個推薦行動中的一些推薦行動。該多個推薦行動旨在基於疲該勞程度、該等待時間、該推薦行動的一成功率、該駕駛脈絡及/或該車輛狀態來改變該駕駛員的活動。If both the vehicle's driving context and the driver's current activities are found to indicate some undesirable situation, such as a drowsy driver driving the vehicle in an unsafe manner, or a driver waiting in the vehicle , while the car is stationary, some of the multiple recommended actions can be output according to this method. The plurality of recommended actions are intended to change the driver's activities based on the fatigue level, the waiting time, a success rate of the recommended actions, the driving context and/or the vehicle status.
初始地，未嘗試的推薦行動的成功率可以被設置為某個預定值，該預定值起初對於所有推薦行動可能是相同的。一旦該駕駛員對該推薦行動的回應被評估，該推薦行動的成功率就可以被評估與被更新。因此，在執行該方法多次且針對各種不同推薦行動後，該多個推薦行動的成功率可以被自動更新，以反映它們對一特定駕駛員的實際有效性。Initially, the success rate of untried recommended actions may be set to some predetermined value, which may initially be the same for all recommended actions. Once the driver's response to the recommended action is evaluated, the success rate of the recommended action can be evaluated and updated. Therefore, after executing the method multiple times and for various recommended actions, the success rates of the multiple recommended actions can be automatically updated to reflect their actual effectiveness for a particular driver.
作為一示例，該方法可以檢測到一昏昏欲睡的駕駛員正在駕駛該汽車。這可能衍生於譬如該駕駛員比平均值更規律地且持續時間更久地閉眼及/或衍生於該駕駛員對該汽車進行錯誤控制。如果該方法確立該車輛正在行駛，則可以建議休息。基於直接的駕駛員行為諸如該駕駛員的眼球運動，可以檢測駕駛員的睡意。替代地，它可以從間接資料中被檢測，諸如當引起嗜睡的駕駛脈絡被預期時，例如從通過該方法被檢測及被儲存的常規駕駛行為或駕駛軌跡，或者當引起嗜睡的環境被實際檢測時，例如，如果車輛的當前路線是一條交通密度較低的鄉村道路。As an example, the method may detect that a drowsy driver is driving the car. This may result, for example, from the driver closing his eyes more regularly and for a longer period of time than average and/or from the driver exercising incorrect control of the vehicle. If the method establishes that the vehicle is moving, a break can be recommended. Driver drowsiness can be detected based on direct driver behavior such as the driver's eye movements. Alternatively, it may be detected from indirect data, such as when a drowsiness-inducing driving context is anticipated, for example from regular driving behavior or driving trajectories detected and stored by the method, or when a drowsiness-inducing environment is actually detected. For example, if the vehicle's current route is a rural road with low traffic density.
替代地，該方法可以檢測到該汽車是靜止的並且該駕駛員沒有對該汽車進行任何實質控制，這表示駕駛員可能正在等待一些事件，諸如該車輛的另一乘客抵達。在這種情況下，可以執行其他行動，諸如發出瑜伽指令。Alternatively, the method may detect that the car is stationary and the driver does not have any substantial control of the car, which indicates that the driver may be waiting for some event, such as the arrival of another passenger of the vehicle. In this case, other actions can be performed, such as issuing yoga instructions.
一旦該推薦行動已被該駕駛員完成或拒絕，可再次檢查該駕駛員的活動或狀態，其中該駕駛員對該推薦行動的回應被評估用於其有效性。如果發現該給定的推薦行動是有效的，譬如該駕駛員接受該推薦行動，並且她的睡意在進行該推薦行動後減少，則此推薦行動的成功率可以從一先前的值被增加到一新的更高的值，反之亦然。因此，一提議(proposal)的成功可以被有效地檢查。如果沒有發現車內有一昏昏欲睡或等待的駕駛員，則根據該方法可以沒有提議被輸出。Once the recommended action has been completed or rejected by the driver, the driver's activity or status can be checked again, with the driver's response to the recommended action evaluated for its effectiveness. If the given recommended action is found to be effective, for example if the driver accepts the recommended action and her drowsiness decreases after taking the recommended action, the success rate of the recommended action can be increased from a previous value to new higher value and vice versa. Therefore, the success of a proposal can be effectively checked. If a drowsy or waiting driver is not detected in the vehicle, no proposal can be output according to this method.
實際上，本發明包括的諸多演算法能夠檢索關於該駕駛員的狀態及該車輛的當前脈絡的資料。根據被檢索到的資料，該諸多演算法會推薦合適的手段或行動，用於改善該駕駛員的當前狀態，並評估該多個推薦行動的成功與否。In fact, the present invention includes algorithms capable of retrieving information about the driver's status and the current context of the vehicle. Based on the retrieved data, the algorithms will recommend appropriate means or actions to improve the driver's current status and evaluate the success of the multiple recommended actions.
如果在另一場合檢測到類似的情況，即類似的駕駛員狀態及車輛環境，則一先前成功的行動可以被再次推薦。諸多情況的相似性可以意指該車輛的相似行駛路線及/或一些常規現象或重複性事件，諸如重複相似的等待時間以使更多乘客在某些上車地點來到該車輛。該諸多演算法可以被設計為迭代地學習一駕駛員的個人偏好及給定建議行動的效率。If a similar situation is detected on another occasion, ie similar driver status and vehicle environment, a previously successful action can be recommended again. Similarity of situations may mean similar driving routes of the vehicle and/or some regular phenomena or recurring events, such as repeating similar waiting times to allow more passengers to come to the vehicle at certain pick-up locations. The algorithms can be designed to iteratively learn a driver's personal preferences and the efficiency of given recommended actions.
本發明解決關於由諸多駕駛員駕駛的諸多車輛的兩個重要議題。The present invention addresses two important issues regarding many vehicles driven by many drivers.
在一方面，研究顯示，諸多駕駛員耗費大量時間(在某些情況下平均每週約75分鐘)在方向盤後方等待，例如從學校接孩童時、帶孩童去俱樂部時、在諸如購物或一醫生的約診等活動期間等待其他人時。尤其是在流行病期間，最好在該車輛中等待，例如一醫生的約診、進行疾病檢測，諸如此類。該駕駛員可能會將該等待時間用於像是閱讀、參與某種形式的網際網路活動，諸如使用YouTube或Facebook、聽音樂、計劃用餐等。然而，很少有駕駛員利用這段時間來真正改善他們的幸福感。On the one hand, studies show that many drivers spend a significant amount of time (in some cases an average of about 75 minutes per week) waiting behind the wheel, such as when picking up children from school, taking them to a club, while shopping or visiting a doctor. When waiting for others during medical appointments and other activities. Especially during epidemics, it is better to wait in the vehicle for, for example, a doctor's appointment, a disease test, etc. The driver may use this waiting time for things like reading, engaging in some form of Internet activity such as using YouTube or Facebook, listening to music, planning meals, etc. However, few drivers use this time to actually improve their well-being.
另一方面，一相當數量的車輛事故是由昏昏欲睡的駕駛員造成的，他們繼續控制該車輛遠遠超出操作任何機械的一安全狀態。On the other hand, a significant number of vehicle accidents are caused by drowsy drivers who continue to control the vehicle well beyond a safe condition for operating any machinery.
本發明提供一種同時檢測一駕駛員的睡意及等待期的方法，使得諸多對應問題可以被改善並且該駕駛員的狀況可以被改善。The present invention provides a method for simultaneously detecting a driver's drowsiness and waiting period, so that many corresponding problems can be improved and the driver's condition can be improved.
在一實際示例中，該方法可以檢測一駕駛員的疲勞並且可以建議切換到該刷新模式，包括該刷新模式相關聯的多個可能的建議行動。該駕駛員可以接受該建議並且該車輛可以採取適當的行動。在一給定時間量之後，譬如在15分鐘後，該駕駛員的疲勞程度可以被再次評估，並且仍然一升高的疲勞程度仍可以被檢測。在這種情況下，可能會發現該被建議的行動沒有成功。結果，該方法可以輸出該休息模式相關聯的諸多建議行動。該駕駛員可能會被提示以確認該被建議的選擇。在該駕駛員接受該建議的情況下，該方法可以在一導航設備上指示下一個可能的休息區，以便該駕駛員可以選擇一合適的休息區。當建議一休息區時，該方法可以考慮一車輛電池的充電狀態。如果該充電狀態低於某個給定值(譬如，60%)，則該方法可以提議一區域以為該車輛充電。較佳地，能夠在一給定時間(譬如15分鐘)內到達的一休息區可以被建議。一旦已到達該區域，該方法可以建議該駕駛員設定一定時器或考慮以一喚醒的定時器來完成充電。在休息期間，除了設定該車輛為包括令人愉悅的環境光、合適的溫度等條件外，該方法還可以提供噪音消除(noise cancellation)及無關消息抑制(irrelevant message suppression)。一旦該計時器已經期滿終止，該方法通過譬如播放安靜的音樂及/或提供一按摩來慢慢喚醒該駕駛員。然後，該方法可以再次檢查該駕駛員的疲勞程度並且據以更新該推薦行動的成功率。隨著時間的推移，該方法的演算法從一駕駛員對諸多輸出的推薦的反應，即該駕駛員是否接受或拒絕該多個推薦、從當前的脈絡(即工作日、日間時段、天氣、駕駛時間等，以及從該被提議的模式的成功與否，即該重新評估的疲勞程度，來確立該駕駛員的偏好和特徵。因此，諸多未來的建議可以被改良，並且該駕駛員可以體驗到一高度個性化及有效率的車輛。In a practical example, the method may detect a driver's fatigue and may recommend switching to the refresh mode, including a plurality of possible recommended actions associated with the refresh mode. The driver can accept the advice and the vehicle can take appropriate action. After a given amount of time, such as after 15 minutes, the driver's fatigue level can be assessed again, and still an elevated level of fatigue can still be detected. In this case, it may be found that the recommended action was unsuccessful. As a result, the method can output a number of recommended actions associated with this rest mode. The driver may be prompted to confirm the suggested selection. If the driver accepts the suggestion, the method can indicate the next possible rest area on a navigation device so that the driver can select a suitable rest area. The method can take into account the state of charge of a vehicle battery when suggesting a rest area. If the state of charge is below a given value (eg, 60%), the method can propose an area to charge the vehicle. Preferably, a rest area that can be reached within a given time (eg 15 minutes) can be suggested. Once the area has been reached, the method may advise the driver to set a timer or consider completing the charge with a wake-up timer. During rest periods, in addition to setting the vehicle to conditions including pleasant ambient light, appropriate temperature, etc., this method can also provide noise cancellation and irrelevant message suppression. Once the timer has expired, the method slowly wakes up the driver by, for example, playing quiet music and/or providing a massage. The method can then recheck the driver's fatigue level and update the success rate of the recommended action accordingly. Over time, the algorithm of the method learns from a driver's reaction to multiple output recommendations, i.e. whether the driver accepts or rejects the multiple recommendations, from the current context (i.e. workday, daytime period, weather, Driving time, etc., and from the success or failure of the proposed mode, i.e. the level of fatigue that should be re-evaluated, to establish the driver's preferences and characteristics. Therefore, many future suggestions can be refined and the driver can experience to a highly personalized and efficient vehicle.
因此，本發明提供用於監控一駕駛員的一脈絡感知及智能方法的諸多益處，該多個益處包括被提高的駕駛安全性、被改良的對抗方法的有效性，以及由於一高度個性化及可調適的監控方法而被改善的駕駛員滿意度。此外，本發明強化一智能車輛的感知性。Accordingly, the present invention provides numerous benefits of a context-aware and intelligent approach to monitoring a driver, including increased driving safety, improved effectiveness of countermeasures, and improved safety due to a highly personalized and Improved driver satisfaction with adaptable monitoring methods. In addition, the present invention enhances the perception of an intelligent vehicle.
本發明不限於當前描述的實施例，而是能夠以不同的方式變化。在申請專利範圍、說明書及包括結構細節、空間佈置的圖式中被定義的所有特徵可能與本發明相關。The invention is not limited to the embodiments currently described, but can be varied in different ways. All features defined in the patent application, the description and the drawings including structural details and spatial arrangement may be relevant to the invention.
(無)(without)
本發明的其他特徵、細節及優點被揭露在請求項、以下圖式描述及圖式本身中。該圖式顯示： ［第1圖］：根據本發明的方法的一些步驟的一流程圖。 Other features, details and advantages of the invention are disclosed in the claims, the following description of the drawings and the drawings themselves. This diagram shows: [Figure 1]: A flow chart of some steps of the method according to the invention.
Claims (10)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| WOPCT/CN2020/140782 | 2020-12-29 | ||
| PCT/CN2020/140782 WO2022141041A1 (en) | 2020-12-29 | 2020-12-29 | Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW202241742A TW202241742A (en) | 2022-11-01 |
| TWI818395B true TWI818395B (en) | 2023-10-11 |
Family
ID=82259885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW110149219A TWI818395B (en) | 2020-12-29 | 2021-12-28 | Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP4271591A1 (en) |
| CN (1) | CN116829415A (en) |
| TW (1) | TWI818395B (en) |
| WO (1) | WO2022141041A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102023100571A1 (en) | 2023-01-12 | 2024-07-18 | Audi Aktiengesellschaft | Method for operating a motor vehicle with a motion assistance system, as well as an associated control system and motor vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10609148B1 (en) * | 2019-09-17 | 2020-03-31 | Ha Q Tran | Smart vehicle |
| CN111152790A (en) * | 2019-12-29 | 2020-05-15 | 的卢技术有限公司 | Multi-device interactive vehicle-mounted head-up display method and system based on use scene |
| CN111797309A (en) * | 2020-06-19 | 2020-10-20 | 一汽奔腾轿车有限公司 | Vehicle-mounted intelligent recommendation device and method based on travel mode |
| CN112078591A (en) * | 2019-06-14 | 2020-12-15 | 上海博泰悦臻电子设备制造有限公司 | Vehicle, vehicle machine equipment and driving assistance method thereof |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100268051A1 (en) * | 2009-04-16 | 2010-10-21 | Ford Global Technologies, Llc | System and method for wellness monitoring in a vehicle |
| CN110019740B (en) * | 2018-05-23 | 2021-10-01 | 京东方科技集团股份有限公司 | Interaction method of vehicle-mounted terminal, server and storage medium |
| CN108831460A (en) * | 2018-06-15 | 2018-11-16 | 浙江吉利控股集团有限公司 | A kind of interactive voice control system and method based on fatigue monitoring |
| CN110065455A (en) * | 2019-04-24 | 2019-07-30 | 深圳市麦谷科技有限公司 | Vehicle-mounted function intelligent starting method, apparatus, computer equipment and storage medium |
-
2020
- 2020-12-29 CN CN202080108236.0A patent/CN116829415A/en active Pending
- 2020-12-29 EP EP20967390.4A patent/EP4271591A1/en active Pending
- 2020-12-29 WO PCT/CN2020/140782 patent/WO2022141041A1/en active Application Filing
-
2021
- 2021-12-28 TW TW110149219A patent/TWI818395B/en active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112078591A (en) * | 2019-06-14 | 2020-12-15 | 上海博泰悦臻电子设备制造有限公司 | Vehicle, vehicle machine equipment and driving assistance method thereof |
| US10609148B1 (en) * | 2019-09-17 | 2020-03-31 | Ha Q Tran | Smart vehicle |
| CN111152790A (en) * | 2019-12-29 | 2020-05-15 | 的卢技术有限公司 | Multi-device interactive vehicle-mounted head-up display method and system based on use scene |
| CN111797309A (en) * | 2020-06-19 | 2020-10-20 | 一汽奔腾轿车有限公司 | Vehicle-mounted intelligent recommendation device and method based on travel mode |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022141041A1 (en) | 2022-07-07 |
| CN116829415A (en) | 2023-09-29 |
| EP4271591A1 (en) | 2023-11-08 |
| TW202241742A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7155122B2 (en) | Vehicle control device and vehicle control method | |
| JP7288911B2 (en) | Information processing device, mobile device, method, and program | |
| US11507086B2 (en) | Manual control re-engagement in an autonomous vehicle | |
| US9896030B2 (en) | System and method for vehicle collision mitigation with vulnerable road user context sensing | |
| US9266018B2 (en) | Customizable in-vehicle gaming system | |
| US10046618B2 (en) | System and method for vehicle control integrating environmental conditions | |
| JP7092116B2 (en) | Information processing equipment, information processing methods, and programs | |
| JP6972629B2 (en) | Information processing equipment, information processing methods, and programs | |
| KR20200135799A (en) | Information processing device, mobile device, and method, and program | |
| US10937334B2 (en) | Information providing system | |
| JP2022552860A (en) | Passenger Condition Adjustment System for Passenger Cars Based on Predictive and Anticipatory Control | |
| JPWO2019202881A1 (en) | Information processing equipment, mobile devices, information processing systems, and methods, and programs | |
| WO2019208450A1 (en) | Driving assistance device, driving assistance method, and program | |
| JP7357006B2 (en) | Information processing device, mobile device, method, and program | |
| KR20200113202A (en) | Information processing device, mobile device, and method, and program | |
| WO2013052644A1 (en) | Emotional control methods and apparatus | |
| CN108688675A (en) | Vehicle drive support system | |
| TWI818395B (en) | Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method | |
| WO2021185468A1 (en) | Technique for providing a user-adapted service to a user | |
| CN114572226A (en) | Vehicle control method and device based on driver state, vehicle and storage medium | |
| US20230211744A1 (en) | Technique for providing a user-adapted service to a user | |
| JP2020170471A (en) | Drive support device | |
| JP2020169956A (en) | Vehicle destination proposal system | |
| JP7043726B2 (en) | Appropriate state judgment method and proper state judgment device | |
| JP7238193B2 (en) | Vehicle control device and vehicle control method |