WO2021095153A1 - Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme - Google Patents

Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme Download PDF

Info

Publication number
WO2021095153A1
WO2021095153A1 PCT/JP2019/044496 JP2019044496W WO2021095153A1 WO 2021095153 A1 WO2021095153 A1 WO 2021095153A1 JP 2019044496 W JP2019044496 W JP 2019044496W WO 2021095153 A1 WO2021095153 A1 WO 2021095153A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
normal
unknown
image
abnormality
Prior art date
Application number
PCT/JP2019/044496
Other languages
English (en)
Japanese (ja)
Inventor
佐藤　公則
宏夫 川島
憲之 根本
将仁 谷口
Original Assignee
株式会社日立物流
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立物流 filed Critical 株式会社日立物流
Priority to PCT/JP2019/044496 priority Critical patent/WO2021095153A1/fr
Priority to JP2021555685A priority patent/JPWO2021095153A1/ja
Publication of WO2021095153A1 publication Critical patent/WO2021095153A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y10/00—Economic sectors
    • G16Y10/40—Transportation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/40—Information sensed or collected by the things relating to personal data, e.g. biometric data, records or preferences
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring

Definitions

  • the present invention is a driver abnormality response system and a driver abnormality response method capable of determining an abnormality level of a driver of a vehicle or the like, selecting a countermeasure according to the determined abnormality level, and executing the selected countermeasure. , And about the program.
  • Patent Document 1 A technique for determining that a person is in an abnormal posture has been proposed.
  • Patent Document 1 describes a deceleration stop type abnormality response system having a lane departure prevention function and an off-road deviation prevention function, it determines the abnormality level of the driver during driving and sets the abnormality level. There is a problem that it is not possible to implement the corresponding countermeasures.
  • the present inventor can determine the abnormality level according to the degree of the difference when the driver's driving method is different from the learning result if the driver's driving method in the normal state is learned. Since it is possible to do so, we focused on the fact that it is possible to implement countermeasures according to the determined abnormality level.
  • the present invention determines an abnormality level when the driver's driving method differs from the learning result by learning the normal driving method of a driver such as a vehicle, and takes countermeasures according to the determined abnormality level. It is an object of the present invention to provide a driver abnormality response system, a driver abnormality response method, and a program capable of selecting and executing the selected countermeasure.
  • the present invention provides the following solutions.
  • the invention according to the first feature is It is a driver abnormality response system that determines the abnormality level of the driver while driving and executes the selected countermeasure.
  • the first acquisition means for acquiring a normal image captured by a normal driver, and The first detection means for analyzing the acquired normal image and detecting the normal operation, The learning means for learning the detected normal operation and A second acquisition means for acquiring an unknown image captured by an unknown driver, whether it is normal or abnormal, and A second detection means that analyzes the acquired unknown image and detects an unknown operation, A determination means for determining an abnormality level according to the degree to which the detected unknown motion differs from the learned result.
  • a selection means for selecting a countermeasure according to the determined abnormality level, and An execution means for executing the selected countermeasure and Provide a driver abnormality response system equipped with.
  • the normal driver is captured and the normal image is acquired.
  • the first acquisition means for detecting the normal operation by analyzing the acquired normal image
  • the learning means for learning the detected normal operation, and whether it is normal.
  • a second acquisition means for acquiring an unknown image captured by an abnormal or unknown driver, a second detection means for analyzing the acquired unknown image to detect an unknown operation, and the detected unknown.
  • a determination means for determining an abnormality level according to the degree to which the operation differs from the learned result, a selection means for selecting a countermeasure according to the determined abnormality level, and the selected countermeasure. It is provided with an execution means to be executed.
  • the invention according to the first feature is in the category of driver abnormality response system, but the same action and effect can be obtained even with a driver abnormality response method and a program.
  • the invention according to the second feature is the driver abnormality response system which is the invention according to the first feature.
  • the biological information of the driver at the normal time is further acquired, and the biological information is acquired.
  • the first detection means further normal biometric information is detected.
  • the learning means further learning the biological information at the time of normal.
  • the biological information of the driver which is unknown whether it is normal or abnormal, is further acquired.
  • the second detection means it is further detected whether the driver's biological information is normal or abnormal, and the driver's biological information is detected.
  • the determination means provides a driver abnormality response system that determines an abnormality level in consideration of the degree of whether the driver's biological information is normal or abnormal.
  • the first acquisition means further acquires the biological information of the driver at the normal time, and the first detection means.
  • the normal biometric information is further detected
  • the normal biometric information is further learned by the learning means
  • the biometric information of the driver whose normal state or abnormal state is unknown is further learned by the second acquisition means.
  • the second detection means further detects whether the driver's biological information is normal or abnormal
  • the determination means determines that the abnormal level of the driver's biological information is normal. The judgment is made in consideration of the degree of whether it is abnormal or abnormal.
  • the invention according to the third feature is a driver abnormality response system which is an invention according to the first feature or the second feature.
  • the first acquisition means the normal time image captured by the normal time driver is acquired in chronological order, and the normal time image is acquired.
  • the first detection means the acquired time-series normal time image is analyzed to detect the normal time operation, and the normal time operation is detected.
  • the learning means the normal operation of the detected time series is learned, and
  • the determination means provides a driver abnormality response system that determines an abnormality level according to the degree to which the detected unknown operation differs from the learned time-series result.
  • the normal driver is imaged in chronological order by the first acquisition means.
  • the time image is acquired
  • the first detection means analyzes the acquired normal time image of the time series to detect the normal operation
  • the learning means detects the normal operation of the detected time series.
  • the determination means when determining the abnormality level, the determination is performed according to the degree to which the detected unknown operation differs from the result of the learned time series.
  • the invention according to the fourth feature is a driver abnormality response system which is an invention according to any one of the first to third features.
  • the location information during normal operation is further acquired, and the location information is acquired.
  • the learning means further learning including location information during normal driving is performed.
  • the location information during operation which is unknown whether it is normal or abnormal, is further acquired.
  • the determination means provides a driver abnormality response system that makes a determination by adding the location information to the unknown operation when determining an abnormality level.
  • the first acquisition means further provides location information during normal operation. Is acquired, and the learning means further learns including the location information during normal driving, and the second acquisition means further obtains location information during driving that is unknown whether it is normal or abnormal.
  • the determination is performed by adding the location information to the unknown operation.
  • the invention according to the fifth feature is a driver abnormality response system which is an invention according to any one of the first to fourth features.
  • the determination means provides a driver abnormality response system that determines an abnormality level according to the number of times or time that the detected unknown operation differs from the learned result.
  • the detected unknown operation is learned by the determination means.
  • the abnormal level is determined according to the number of times or time different from the result.
  • the invention according to the sixth feature is a driver abnormality response system which is an invention according to any one of the first to fifth features.
  • the first detection means the normal image is analyzed to identify the driver, and the normal operation of the driver is detected.
  • the learning means the detected normal operation is learned for each of the specified drivers.
  • the second detection means the acquired unknown image is analyzed to identify the driver, and the unknown operation is detected.
  • the determination means provides a driver abnormality response system that determines an abnormality level according to the degree to which the detected unknown operation of the identified driver differs from the learned result of the identified driver.
  • the first detection means analyzes a normal image to obtain a driver.
  • the normal operation of the driver is specified, the normal operation of the driver is detected, the detected normal operation is learned for each of the specified drivers in the learning means, and the acquired unknown image is analyzed by the second detection means.
  • the driver is identified, the unknown motion is detected, and in the determination means, the abnormal level is increased according to the degree to which the detected unknown motion of the identified driver is different from the learned result of the identified driver. To judge.
  • the invention according to the seventh feature is a driver abnormality response system which is an invention according to any one of the first to fifth features.
  • the normal driver images captured by the normal driver are acquired for a plurality of people, and the images are acquired for a plurality of people.
  • the determination means provides a driver abnormality response system that determines an abnormality level according to the degree to which the detected unknown operation differs from the average value or the median value of the learned results.
  • the normal driver in which the normal driver is imaged is captured by the first acquisition means.
  • Time images are acquired for a plurality of people, and the determination means determines the abnormality level according to the degree to which the detected unknown motion differs from the average value or the median value of the learned results.
  • the invention according to the eighth feature is To the driver abnormality response system that determines the abnormality level of the driver while driving and executes the selected countermeasure.
  • the invention according to the ninth feature is To the driver abnormality response system that determines the abnormality level of the driver while driving and executes the selected countermeasure. Steps to acquire a normal image captured by a normal driver, A step of analyzing the acquired normal image to detect normal operation, The step of learning the detected normal operation, Steps to acquire an unknown image taken by an unknown driver, whether it is normal or abnormal, The step of analyzing the acquired unknown image to detect an unknown operation, A step of determining an abnormality level according to the degree to which the detected unknown motion differs from the learned result. A step of selecting a countermeasure according to the determined abnormality level, Steps to implement the selected workaround, Provide a program to execute.
  • an abnormality level is determined when the driving method of the driver is different from the learning result, and a response is made according to the determined abnormality level. It is possible to provide a driver abnormality response system, a driver abnormality response method, and a program capable of selecting a countermeasure and executing the selected countermeasure.
  • FIG. 1 is a schematic view of a preferred embodiment of the present invention.
  • FIG. 2 is a diagram showing the relationship between the functional blocks of the computer 100 and the imaging device 200 and each function.
  • FIG. 3 is a flowchart of the driver abnormality handling process performed by the driver abnormality handling system.
  • FIG. 4 is a diagram showing the relationship between the functional blocks of the computer 100, the imaging device 200, and the biological information acquisition device 300 and each function when the biological information is also acquired.
  • FIG. 5 is a flowchart of a driver abnormality handling process when determining an abnormality level with the same driver.
  • FIG. 6 is an example of a normal image.
  • FIG. 7 is an example of an unknown image.
  • FIG. 8 is an example showing an abnormal operation corresponding to an abnormal level and a countermeasure.
  • FIG. 9 is an example showing the number of times and the time for determining the abnormality level.
  • FIG. 1 is a schematic view of a preferred embodiment of the present invention. The outline of the present invention will be described with reference to FIG.
  • the driver abnormality response system includes a computer 100, an image pickup device 200, and a communication network 400.
  • the number of image pickup devices 200 is not limited to one, and may be plural.
  • the computer 100 is not limited to an existing device, but may be a virtual device, a cloud service, or the like.
  • the computer 100 includes a control unit 110, a communication unit 120, a storage unit 130, an input unit 140, and an output unit 150.
  • the control unit 110 includes a first acquisition means 111, a first detection means 112, a learning means 113, a second acquisition means 114, a second detection means 115, a determination means 116, a selection means 117, and an execution means 118.
  • the control unit 110 cooperates with the communication unit 120, the storage unit 130, the input unit 140, and the output unit 150 to realize the functions of the above means, if necessary. Further, as shown in FIG.
  • the image pickup apparatus 200 includes an image pickup unit 20, a control unit 210, a communication unit 220, a storage unit 230, an input unit 240, and an output unit 250.
  • the communication network 400 is a network that enables communication between the computer 100 and the image pickup apparatus 200.
  • the computer 100 is a calculation device capable of data communication with the image pickup device 200.
  • the illustration is based on the premise of an in-vehicle device, it may be a mobile terminal such as a smartphone, a notebook computer, a wearable device, or the like, and may be a virtual device as well as an existing device. It may also be a service such as a cloud.
  • the image pickup device 200 is an image pickup device provided with an image pickup device such as an image pickup device and a lens capable of data communication with the computer 100, and can take an image of a driver in operation.
  • an image pickup device such as an image pickup device and a lens capable of data communication with the computer 100
  • the illustration is based on the assumption of an in-vehicle camera, it may be an image pickup device having necessary functions such as a digital camera, a digital video, a camera of a mobile terminal, a laptop computer, or a wearable device.
  • the captured image may be stored in the storage unit 230.
  • the image to be captured shall have the resolution required for analysis, and is basically a moving image, but may be a still image.
  • the first acquisition means 111 acquires a normal image captured by the normal driver (step S101).
  • the acquisition here may be performed directly from the image pickup device 200, may be performed from a database that collects normal images, an external server, or the like, or may be performed from a digital camera, a video camera, or the like other than the image pickup device 200. Alternatively, it may be performed from a storage medium such as a CD-ROM, a DVD, or a USB memory. If it is a moving image, audio data may be acquired at the same time.
  • FIG. 6 is an example of a normal image.
  • the normal image shall have a resolution and an imaging range in which the driver's normal movement of the face and hands, gestures, posture, line of sight, facial expression, etc. can be confirmed.
  • the normal image is basically a moving image, but may be a still image.
  • the first detection means 112 of the computer 100 analyzes the normal image acquired in step S101 and detects the normal operation (step S102).
  • normal operation is detected for each of the images.
  • the normal operation here is defined as the normal movement of the driver's face and hands, gestures, posture, line of sight, facial expression, and the like.
  • step S103 the learning means 113 learns the normal operation detected in step step S102 (step S103). Since the movements, gestures, postures, eyes, facial expressions, etc. of the face and hands detected in step S102 are normal, what range each of them falls within, how often and how many times are performed. Learn what kind of time it will last, and so on. If the normal image is insufficient for learning the normal operation, step S101 and step S102 are repeated. On the contrary, if the learning is sufficiently performed, steps S101 to S103 may be omitted.
  • the second acquisition means 114 acquires an unknown image captured by an unknown driver, whether it is normal or abnormal (step S104).
  • the timing for starting the acquisition of step S104 the timing when the driver starts driving can be considered.
  • the computer 100 may be notified when the imaging device 200 starts imaging.
  • FIG. 7 is an example of an unknown image.
  • the unknown image shall have a resolution and an imaging range in which the movement, gesture, posture, line of sight, facial expression, etc. of the driver's face and hands can be confirmed.
  • the unknown image is basically a moving image, but may be a still image.
  • the second detection means 115 of the computer 100 analyzes the acquired unknown image and detects an unknown operation (step S105).
  • the unknown motion here is assumed to be the movement of the face or hand, the gesture, the posture, the line of sight, the facial expression, etc. of the driver of the acquired unknown image.
  • the determination means 116 of the computer 100 determines the abnormality level according to the degree to which the unknown operation detected in step S105 is different from the result learned in step S103 (step S106).
  • the normal movements in which the detected unknown movements are learned and how much they differ from each index such as the driver's face and hand movements, gestures, postures, eyes, and facial expressions.
  • the number of levels of abnormality can be set according to the system. In FIG. 1, it is assumed that the abnormality level is determined to be level 2 inattentive driving.
  • FIG. 8 is an example showing an abnormal operation corresponding to an abnormal level and a countermeasure.
  • the abnormality level is set to 5 levels from level 1 to level 5.
  • the abnormal behavior of level 1 is "more times of rubbing the nose than usual (cold / allergy, etc.)". In this case, the determination can be made from "the movement of the driver's hand is more likely to come near the nose than in the normal state", "there are more gestures to rub the nose than in the normal state", and the like.
  • the abnormal operation of level 2 is "inattentive driving".
  • the abnormal behavior of level 3 is "screaming". In this case, "the driver's face is moving his mouth more violently than in normal times", “the driver is waving his hands more than in normal times”, “the facial expression is full of anger compared to normal times", etc. Judgment can be made.
  • the abnormal behavior of level 4 is "consciousness is jumping (sudden illness / lack of sleep, etc.)".
  • FIG. 9 is an example showing the number of times and the time for determining the abnormality level.
  • the abnormality level is set to 5 levels from level 1 to level 5 as in the example of FIG. Level 1 abnormal behavior
  • “more times of rubbing the nose than usual (cold / allergy, etc.)” "the movement of the driver's hand is more likely to come near the nose than in normal times”
  • “ For indicators such as “more gestures of rubbing the nose than in normal times” if the number of unknown movements is "5 times or more” and the time is "total 3 minutes or more”
  • the state of level 4 has continued for a long time, that is, the state is further 2 minutes or more, that is, “total 3 minutes” as the time from the state judged to be level 4.
  • the abnormality level is 5.
  • the selection means 117 of the computer 100 selects a countermeasure according to the abnormality level determined in step S106 (step S107).
  • a countermeasure for example, a notification system for notifying and a vehicle control system for controlling the driver's vehicle can be considered, and not one countermeasure but a plurality of countermeasures may be taken at the same time.
  • FIG. 1 it is assumed that "notify the driver” and “notify the administrator” are selected as the notification system and "speed down” is selected as the vehicle control system as countermeasures against "abnormal level 2: inattentive driving".
  • An example of countermeasures corresponding to the abnormality level will be described with reference to FIG. In the example of FIG.
  • level 1 As a countermeasure in the case of level 1 "more times of rubbing the nose than usual (cold / allergy, etc.)", "notify the driver (warning / follow-up)" is used as the notification system. Shown. For example, it is possible to call attention by notifying the driver by means of voice, sound, light, image or character display, vibration, or the like from the vehicle. After that, as follow-up observation, the level 1 may be determined again with a smaller number of times and time than the normal index shown in FIG. 9, or the abnormal level may be raised if the same state continues. ..
  • the "speed limit” of the vehicle control system is, for example, the possibility of driving by the driver or an accident by limiting the upper limit speed at which the vehicle can be directly controlled for a certain period of time after the level 2 judgment. I think it can be lowered.
  • notification system As a countermeasure in the case of level 4 "consciousness is jumping (sudden illness / lack of sleep, etc.)", "notification to surrounding people” is shown as a notification system, and “sudden stop” is shown as a vehicle control system. ..
  • the notification system "notification to surrounding people” is, for example, from the vehicle to the inside and outside of the vehicle by using means such as voice, sound, light, image and character display to the passenger and people around the vehicle. By giving the notification, the effect of having the driver of the vehicle work can be expected. Further, the "sudden stop" of the vehicle control system is to prevent the occurrence of an accident by promptly stopping at a safe place, not by suddenly braking.
  • the hazard lamp or the like may be turned on at the same time.
  • "report to an emergency agency” is shown as a notification system.
  • the "report to the emergency agency” for example, a public institution such as a fire department may be notified by calling 119, etc., to the effect that the vehicle position information and the driver are unconscious, or depending on the type of vehicle. A similar report may be made to the corresponding emergency contact.
  • FIG. 8 since it is an example of one abnormal operation for each abnormal level, countermeasures are shown for each level, but when there are multiple abnormal operations at the same level, countermeasures are set according to the abnormal operation. You can do it.
  • step S108 the execution means 118 of the computer 100 executes the countermeasure selected in step S107 (step S108).
  • FIG. 1 it is assumed that "notify the driver” and “notify the administrator” are executed as the notification system, and "speed down” is executed as the vehicle control system.
  • an abnormality level is determined when the driving method of the driver is different from the learning result, and the determined abnormality level is determined. It is possible to provide a driver abnormality response system, a driver abnormality response method, and a program capable of selecting a countermeasure according to the situation and executing the selected countermeasure.
  • FIG. 2 is a diagram showing the relationship between the functional blocks of the computer 100 and the imaging device 200 and each function.
  • the computer 100 includes a control unit 110, a communication unit 120, a storage unit 130, an input unit 140, and an output unit 150.
  • the control unit 110 includes a first acquisition means 111, a first detection means 112, a learning means 113, a second acquisition means 114, a second detection means 115, a determination means 116, a selection means 117, and an execution means 118.
  • the control unit 110 cooperates with the communication unit 120, the storage unit 130, the input unit 140, and the output unit 150 to realize the functions of the above means, if necessary.
  • the image pickup device 200 includes an image pickup unit 20, a control unit 210, a communication unit 220, a storage unit 230, an input unit 240, and an output unit 250.
  • the communication network 400 is a network that enables communication between the computer 100 and the image pickup apparatus 200.
  • the computer 100 is a calculation device capable of data communication with the image pickup device 200.
  • the illustration is based on the premise of an in-vehicle device, it may be a mobile terminal such as a smartphone, a notebook computer, a wearable device, or the like, and may be a virtual device as well as an existing device. It may also be a service such as a cloud.
  • the control unit 110 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.
  • the control unit 110 cooperates with the communication unit 120, the storage unit 130, the input unit 140, and the output unit 150 to realize the functions of the above means, if necessary.
  • the communication unit 120 is provided with a device for enabling communication with other devices.
  • the communication may be a wired connection or a wireless connection. Further, it is assumed that communication with an external server or database is performed via the communication unit 120 in order to acquire a normal image as needed.
  • the storage unit 130 includes a data storage unit using a hard disk or a semiconductor memory, and stores data necessary for processing such as a normal image, a normal operation, a learning result, an unknown image, an abnormality level, a judgment result, and a countermeasure. ..
  • the storage unit 130 may be an external server, a database, a cloud service, or the like.
  • the input unit 140 shall be provided with the functions necessary for managing or operating the driver abnormality response system. As an example for realizing input, it is possible to provide a keyboard, a mouse, a pen tablet, a touch panel, a microphone for voice recognition, and the like. It may be possible to input via a network.
  • the function of the present invention is not particularly limited depending on the input method.
  • the output unit 150 shall be provided with the functions necessary for managing or operating the driver abnormality response system.
  • a display a display such as a projection on a projector, and a form such as an audio output can be considered. It may be possible to output via a network.
  • the function of the present invention is not particularly limited depending on the output method.
  • the image pickup device 200 is an image pickup device provided with an image pickup device such as an image pickup device and a lens capable of data communication with the computer 100, and captures an image of a driver during operation. It is possible to image the driver while driving.
  • an image pickup device such as an image pickup device and a lens capable of data communication with the computer 100
  • the illustration is based on the assumption of an in-vehicle camera, it may be an image pickup device having necessary functions such as a digital camera, a digital video, a camera of a mobile terminal, a laptop computer, or a wearable device.
  • the captured image may be stored in the storage unit 230.
  • the image to be captured shall have the resolution required for analysis, and is basically a moving image, but may be a still image.
  • the image pickup device 200 includes an image pickup device such as a lens, an image pickup element, and the like as an image pickup unit 20, and captures an image as an image pickup image such as a moving image or a still image. Further, the image obtained by imaging is a precise image having a resolution necessary for image analysis.
  • the control unit 210 includes a CPU, RAM, ROM, and the like.
  • the communication unit 220 is provided with a device for enabling communication with other devices.
  • the communication may be a wired connection or a wireless connection.
  • the storage unit 230 is provided with a data storage unit using a hard disk or a semiconductor memory, and stores necessary data such as captured images. Information on the date and time when the image was taken and the location may also be stored.
  • the storage unit 230 may be an external server, a database, a cloud service, or the like.
  • the input unit 240 is provided with functions necessary for operating the image pickup apparatus 200. As an example for realizing input, it is possible to provide a hardware button, a touch panel, a microphone, and the like. It may be possible to input via a network.
  • the output unit 250 shall have functions necessary for operating the image pickup apparatus 200. As an example for realizing the output, a form such as a display on a display or an audio output can be considered. It may be possible to output via a network.
  • FIG. 3 is a flowchart of the driver abnormality handling process performed by the driver abnormality handling system. The processing executed by each of the above-mentioned means will be described with reference to this flowchart.
  • the first acquisition means 111 of the computer 100 acquires a normal image captured by the normal driver (step S301).
  • the acquisition here may be performed directly from the image pickup device 200, may be performed from a database that collects normal images, an external server, or the like, or may be performed from a digital camera, a video camera, or the like other than the image pickup device 200. Alternatively, it may be performed from a storage medium such as a CD-ROM, a DVD, or a USB memory. If it is a moving image, audio data may be acquired at the same time.
  • FIG. 6 is an example of a normal image.
  • the normal image shall have a resolution and an imaging range in which the driver's normal movement of the face and hands, gestures, posture, line of sight, facial expression, etc. can be confirmed.
  • the normal image is basically a moving image, but may be a still image.
  • the first detection means 112 of the computer 100 analyzes the normal image acquired in step S301 and detects the normal operation (step S302).
  • the normal operation here is defined as the normal movement of the driver's face and hands, gestures, posture, line of sight, facial expression, and the like.
  • step S303 the learning means 113 learns the normal operation detected in step step S302 (step S303). Since the movements, gestures, postures, eyes, facial expressions, etc. of the face and hands detected in step S302 are normal, what range each of them falls within, how often and how many times are performed. Learn what kind of time it will last, and so on. If the normal image is insufficient for learning the normal operation, step S301 and step S302 are repeated. On the contrary, if the learning is sufficiently performed, steps S301 to S303 may be omitted.
  • the imaging device 200 starts imaging at the timing when the driver starts driving (step S304).
  • the imaging device 200 notifies the computer 100 of the start of imaging at the same time as the start of imaging.
  • the second acquisition means 114 of the computer 100 Upon receiving the notification from the imaging device 200 of the start of imaging, the second acquisition means 114 of the computer 100 acquires an unknown image captured by an unknown driver, whether it is normal or abnormal, from the imaging device 200 ( Step S305).
  • FIG. 7 is an example of an unknown image.
  • the unknown image shall have a resolution and an imaging range in which the movement, gesture, posture, line of sight, facial expression, etc. of the driver's face and hands can be confirmed.
  • the unknown image is basically a moving image, but may be a still image.
  • the second detection means 115 of the computer 100 analyzes the acquired unknown image to detect the unknown operation (step S306).
  • the unknown motion here is assumed to be the movement of the face or hand, the gesture, the posture, the line of sight, the facial expression, etc. of the driver of the acquired unknown image.
  • the determination means 116 of the computer 100 determines the abnormality level according to the degree to which the unknown operation detected in step S306 is different from the result learned in step S303 (step S307).
  • the number of levels of abnormality can be set according to the system. In the flowchart of FIG. 3, it is assumed that the abnormality level is determined to be level 2 inattentive operation.
  • FIG. 8 is an example showing an abnormal operation corresponding to an abnormal level and a countermeasure.
  • the abnormality level is set to 5 levels from level 1 to level 5.
  • the abnormal behavior of level 1 is "more times of rubbing the nose than usual (cold / allergy, etc.)". In this case, the determination can be made from "the movement of the driver's hand is more likely to come near the nose than in the normal state", "there are more gestures to rub the nose than in the normal state", and the like.
  • the abnormal operation of level 2 is "inattentive driving".
  • the abnormal behavior of level 3 is "screaming". In this case, "the driver's face is moving his mouth more violently than in normal times", “the driver is waving his hands more than in normal times”, “the facial expression is full of anger compared to normal times", etc. Judgment can be made.
  • the abnormal behavior of level 4 is "consciousness is jumping (sudden illness / lack of sleep, etc.)".
  • FIG. 9 is an example showing the number of times and the time for determining the abnormality level.
  • the abnormality level is set to 5 levels from level 1 to level 5 as in the example of FIG. Level 1 abnormal behavior
  • “more times of rubbing the nose than usual (cold / allergy, etc.)” "the movement of the driver's hand is more likely to come near the nose than in normal times”
  • “ For indicators such as “more gestures of rubbing the nose than in normal times” if the number of unknown movements is "5 times or more” and the time is "total 3 minutes or more”
  • the state of level 4 has continued for a long time, that is, the state is further 2 minutes or more, that is, “total 3 minutes” as the time from the state judged to be level 4.
  • the abnormality level is 5.
  • the selection means 117 of the computer 100 selects a countermeasure according to the abnormality level determined in step S307 (step S308).
  • a countermeasure for example, a notification system for notifying and a vehicle control system for controlling the driver's vehicle can be considered, and not one countermeasure but a plurality of countermeasures may be taken at the same time. To do.
  • An example of countermeasures corresponding to the abnormality level will be described with reference to FIG. In the example of FIG.
  • level 1 As a countermeasure in the case of level 1 "more times of rubbing the nose than usual (cold / allergy, etc.)", "notify the driver (warning / follow-up)" is used as the notification system. Shown. For example, it is possible to call attention by notifying the driver by means of voice, sound, light, image or character display, vibration, or the like from the vehicle. After that, as follow-up observation, the level 1 may be determined again with a smaller number of times and time than the normal index shown in FIG. 9, or the abnormal level may be raised if the same state continues. ..
  • the "speed limit” of the vehicle control system is, for example, the possibility of driving by the driver or an accident by limiting the upper limit speed at which the vehicle can be directly controlled for a certain period of time after the level 2 judgment. I think it can be lowered.
  • notification system As a countermeasure in the case of level 4 "consciousness is jumping (sudden illness / lack of sleep, etc.)", "notification to surrounding people” is shown as a notification system, and “sudden stop” is shown as a vehicle control system. ..
  • the notification system "notification to surrounding people” is, for example, from the vehicle to the inside and outside of the vehicle by using means such as voice, sound, light, image and character display to the passenger and people around the vehicle. By giving the notification, the effect of having the driver of the vehicle work can be expected. Further, the "sudden stop" of the vehicle control system is to prevent the occurrence of an accident by promptly stopping at a safe place, not by suddenly braking.
  • the hazard lamp or the like may be turned on at the same time.
  • "report to an emergency agency” is shown as a notification system.
  • the "report to the emergency agency” for example, a public institution such as a fire department may be notified by calling 119, etc., to the effect that the vehicle position information and the driver are unconscious, or depending on the type of vehicle. A similar report may be made to the corresponding emergency contact.
  • FIG. 8 since it is an example of one abnormal operation for each abnormal level, countermeasures are shown for each level, but when there are multiple abnormal operations at the same level, countermeasures are set according to the abnormal operation. You can do it.
  • step S309 the execution means 118 of the computer 100 executes the countermeasure selected in step S308 (step S309).
  • step S309 it is assumed that "notify the driver” and “notify the administrator” are executed as the notification system, and "speed down” is executed as the vehicle control system.
  • control unit 110 of the computer 100 confirms whether or not to end the driver abnormality response process (step S310). If the driver abnormality handling process is not completed, the unknown image acquisition process in step S305 is performed, and if the driver abnormality handling process is terminated, the processing of the computer 100 is terminated. As the timing for ending the driver abnormality handling process, it is conceivable that a notification of the end of imaging is received from the imaging device 200.
  • step S307 determines whether the abnormality level is determined in step S307 to be level 2 has been described as an example, but here, when it does not correspond to any abnormality level from level 1 to level 5, that is, when it is normal, the step. It is assumed that step S309 is skipped from S308 and the process proceeds to step S310.
  • an unknown image in the case where the abnormality level is determined in step S307 and does not correspond to any abnormality level may be acquired as a normal image and used for learning the next normal operation.
  • an abnormality level is determined when the driving method of the driver is different from the learning result, and the determined abnormality level is determined. It is possible to provide a driver abnormality response system, a driver abnormality response method, and a program capable of selecting a countermeasure according to the situation and executing the selected countermeasure.
  • FIG. 4 is a diagram showing the relationship between the functional blocks of the computer 100, the imaging device 200, and the biological information acquisition device 300 and each function when the biological information is also acquired.
  • a biological information acquisition device 300 is provided. Although only the functional blocks of the biological information acquisition device 300 are shown in FIG. 4, it is assumed that the functional blocks of the computer 100 and the imaging device 200 are the same as those in FIG.
  • the communication network 400 enables communication between the computer 100 and the imaging device 200, and between the computer 100 and the biological information acquisition device 300, respectively.
  • the biological information acquisition device 300 is composed of a biological information acquisition unit 30, a control unit 310, a communication unit 320, a storage unit 330, an input unit 340, and an output unit 350.
  • the biological information acquisition device 300 is a device including a camera device or a sensor device capable of acquiring biological information such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve, which can perform data communication with the computer 100, and is operated. Acquire the biometric information of the driver inside.
  • the illustration is based on the premise of a wearable device, it may be a device capable of acquiring biological information such as a camera device, a medical terminal, or an in-vehicle device. Further, the biological information acquired in the storage unit 330 may be stored.
  • the biological information acquisition device 300 includes a device such as a camera device and a sensor device capable of acquiring biological information as the biological information acquisition unit 30.
  • the biological information here is information capable of determining the physical or mental state of the driver such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve.
  • the biological information acquisition unit 30 is a camera device, it is possible to analyze the image acquired by the control unit 310 to obtain biological information such as heartbeat and respiration.
  • the biological information acquisition unit is a sensor device, it is possible to obtain heartbeat, body temperature, electrocardiogram, blood pressure, respiration, etc. from each sensor. Further, by analyzing the heartbeat, body temperature, electrocardiogram, blood pressure, respiration, etc. by the control unit 310, it is possible to obtain biological information such as sympathetic nerves and parasympathetic nerves.
  • the control unit 310 includes a CPU, RAM, ROM, and the like.
  • the communication unit 320 is provided with a device for enabling communication with other devices.
  • the storage unit 330 is provided with a data storage unit using a hard disk or a semiconductor memory, and stores necessary data such as biometric information. Information on the date and time and place where the biological information was acquired may also be stored.
  • the storage unit 330 may be an external server, a database, a cloud service, or the like.
  • the input unit 340 shall have a function necessary for operating the biological information acquisition device 300.
  • a function necessary for operating the biological information acquisition device 300 As an example for realizing input, it is possible to provide a hardware button, a touch panel, a microphone, and the like.
  • the output unit 350 is provided with functions necessary for operating the biological information acquisition device 300.
  • a form such as a display on a display or an audio output can be considered.
  • the first acquisition means 111 of the computer 100 further acquires the biological information of the driver in the normal state together with the image in the normal state captured by the driver in the normal state (step S301).
  • the acquisition of the biometric information of the driver in the normal state here may be performed directly from the biometric information acquisition device 300, may be performed from a database collecting the biometric information at the time of living body, an external server, or the like, or the biometric information. It may be performed from a device other than the acquisition device 300, or may be performed from a storage medium such as a CD-ROM, a DVD, or a USB memory.
  • the acquired biological information shall be information that can determine the physical or mental state of the driver such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve.
  • the first detection means 112 of the computer 100 further detects the normal biometric information in addition to the detection of the normal operation (step S302).
  • the normal biometric information is detected for each of the acquired biometric information.
  • the normal biological information here is assumed to be the driver's normal heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, parasympathetic nerve, and the like.
  • the learning means 113 further learns the normal biometric information in addition to the learning of the normal movement (step S303). Since the biological information such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve detected in step S302 is normal, it is learned what range each of them numerically falls within. To do. If the normal biometric information is insufficient for learning the normal biometric information, step S301 and step S302 are repeated. On the contrary, if the learning is sufficiently performed, steps S301 to S303 may be omitted.
  • the biometric information acquisition device 300 starts the biometric information acquisition at a timing before the driver starts driving (step S304). Further, the biometric information acquisition device 300 notifies the computer 100 whether or not the biometric information is acquired at the timing when the driver starts driving.
  • the second acquisition means 114 of the computer 100 is in the normal state or the abnormal state together with the acquisition of the unknown image from the image pickup device 200.
  • the biometric information of the unknown driver is acquired from the biometric information acquisition device 300 (step S305).
  • the unknown driver biometric information here is data such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve when the driver is normal or abnormal or unknown.
  • the second detection means 115 of the computer 100 detects whether the driver's biological information is normal or abnormal, in addition to detecting an unknown operation (step S306). Whether the driver's biological information is normal or abnormal here is whether the data such as heart rate, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve are numerically within the normal range. Detected by.
  • the determination means 116 of the computer 100 determines the abnormality level in consideration of the degree of whether the driver's biological information is normal or abnormal when determining the abnormality level of the unknown operation (step S307). ). For example, if any one or more of the data such as heartbeat, body temperature, electrocardiogram, blood pressure, respiration, sympathetic nerve, and parasympathetic nerve are numerically out of the normal range, that is, abnormal, the abnormality is found.
  • the anomaly level is determined by taking into consideration whether is numerically large or small, how many anomalous data are available, and the like. If the abnormality is numerically large or there is a lot of abnormality data, the abnormality level is raised to make a judgment. How much biological information is added can be set according to the system.
  • driver abnormality response system driver abnormality that can determine the abnormality level more appropriately, select the countermeasure according to the determined abnormality level, and execute the selected countermeasure by using the biometric information. It becomes possible to provide a response method and a program.
  • driver error handling processing considering time series By considering the time series of the acquired images in the driver abnormality handling process, it is possible to perform appropriate driver abnormality handling processing even when the acquired image is not a moving image but a still image, for example.
  • the driver abnormality handling process in consideration of the time series will be described with reference to the flowchart of FIG. Here, it is assumed that the processing that considers the time series is mainly described.
  • the first acquisition means 111 of the computer 100 acquires a normal image captured by the normal driver in time series (step S301).
  • the first detection means 112 of the computer 100 analyzes the time-series normal-time image acquired in step S301 to detect the normal-time operation (step S302).
  • the learning means 113 learns the normal operation of the time series detected in step S302 (step S303).
  • the determination means 116 of the computer 100 determines the abnormal level of the unknown operation according to the degree to which the unknown operation detected in step S306 differs from the learned time-series result (step). S307). For example, when the unknown image 01, the unknown image, 02, the unknown image 03, the unknown image 04, and the unknown image 05 are acquired in chronological order, the abnormality level of each of the five unknown images is not determined to be level 1 or higher. Also, considering the order of the time series, there is a possibility that the abnormality level is level 1 or higher when compared with the normal operation of the time series.
  • the abnormality level is determined when the driving method of the driver is different from the learning result.
  • the countermeasure is selected according to the determined abnormality level, and the selected countermeasure is executed. It is possible to provide a driver abnormality response system, a driver abnormality response method, and a program that can be used.
  • driver error handling processing that takes into account location information
  • location information such as GPS information and map information together with the image and adding it.
  • the driver abnormality handling process in consideration of the location information will be described with reference to the flowchart of FIG. Here, it is assumed that the processing related to the location information is mainly described.
  • the first acquisition means 111 of the computer 100 acquires the location information during normal operation together with the normal image captured by the driver during normal operation (step S301).
  • the acquisition of the location information during normal driving is based on the assumption that GPS information and map information are added to the image information in advance, but it may be acquired by other means.
  • the location information to be acquired shall be information such as GPS information and map information that can determine the location at the time of driving.
  • the learning means 113 learns the normal operation including the location information during the normal operation (step S303). This is because, for example, a road with a good view and a road with a poor view have different movements of the driver such as face and hand movements, gestures, postures, eyes, and facial expressions even under normal conditions. In addition, it is considered that the driver's behavior differs between general roads and highways, and between streets and mountain roads.
  • the second acquisition means 114 further acquires the location information together with the acquisition of the unknown image from the imaging device 200 (step S305).
  • the determination means 116 makes a determination by adding the location information to the unknown operation when determining the abnormal level of the unknown operation (step S307).
  • a method for making a judgment that takes into account location conditions, depending on whether the location information of unknown operation is a road with a good view or a road with a poor view, a general road or a highway, a city or a mountain road, etc. , It is conceivable to make a judgment by comparing with the normal operation learned under the same conditions.
  • an abnormality level is determined when the driving method of the driver is different from the learning result.
  • a driver anomaly response system that can more appropriately determine the anomaly level by taking into account location information, select countermeasures according to the determined anomaly level, and execute the selected countermeasures. It is possible to provide a driver abnormality response method and a program.
  • FIG. 5 is a flowchart of a driver abnormality handling process when determining an abnormality level with the same driver, and can be realized with the same configuration as in FIG.
  • the processing executed by each of the above-mentioned means will be described with reference to this flowchart.
  • the difference from the flow of FIG. 3 will be mainly described.
  • the first acquisition means 111 of the computer 100 acquires a normal image captured by the normal driver (step S501).
  • the first detection means 112 analyzes the normal image acquired in step S501 to identify the driver, and detects the normal operation of the driver (step S502).
  • the image data of the person shown in the normal image not only the image data of the person shown in the normal image but also the image data of the interior of the vehicle may be used, which is useful for identifying the driver in addition to the image data. Data may be added in advance.
  • the learning means 113 learns the normal operation detected in step S502 for each specified driver (step S503). Regarding the face and hand movements, gestures, postures, eyes, facial expressions, etc. of each driver detected in step S502, what range does each fall within, how often and how many times are performed? Learn whether the time will continue.
  • the imaging device 200 starts imaging at the timing when the driver starts driving (step S504).
  • the imaging device 200 notifies the computer 100 of the start of imaging at the same time as the start of imaging.
  • the second acquisition means 114 of the computer 100 Upon receiving the notification from the imaging device 200 of the start of imaging, the second acquisition means 114 of the computer 100 acquires an unknown image captured by an unknown driver, whether it is normal or abnormal, from the imaging device 200 ( Step S505).
  • the second detection means 115 analyzes the acquired unknown image, identifies the driver, and detects the unknown operation (step S506).
  • the determination means 116 determines the abnormality level according to the degree to which the unknown operation detected in step S506 of the specified driver is different from the result learned in step S503 of the specified driver (step S507).
  • the selection means 117 selects a countermeasure according to the abnormality level determined in step S507 (step S508).
  • Execution means 118 executes the countermeasure selected in step S508 (step S509).
  • control unit 110 of the computer 100 confirms whether or not to end the driver abnormality handling process (step S510). If the driver abnormality handling process is not completed, the unknown image acquisition process in step S505 is performed, and if the driver abnormality handling process is ended, the processing of the computer 100 is terminated. As the timing for ending the driver abnormality handling process, it is conceivable that a notification of the end of imaging is received from the imaging device 200.
  • the present invention by identifying the driver and learning the normal driving method of the driver such as a vehicle for each driver, the abnormal level when the driving method of the driver is different from the learning result. It is possible to provide a driver abnormality response system, a driver abnormality response method, and a program capable of determining with high accuracy, selecting a countermeasure according to the determined abnormality level, and executing the selected countermeasure. It will be possible.
  • the high-precision driver abnormality handling process when determining the abnormality level with the same driver has been described, but conversely, it is also possible to compare a large number of normal drivers with a certain driver. is there. In that case, there is an advantage that the driver abnormality handling process can be performed even for a new driver.
  • the normal image is acquired in step S301 of FIG. 3
  • the normal images captured by the normal drivers are acquired for a plurality of people, and the normal image is acquired in step S307.
  • the above-mentioned means and functions are realized by a computer (including a CPU, an information processing device, various terminals, a virtual device, and a cloud service) reading and executing a predetermined program.
  • the program may be provided, for example, from a computer via a network, or may be provided in a form recorded on a readable recording medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Accounting & Taxation (AREA)
  • Business, Economics & Management (AREA)
  • Operations Research (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)

Abstract

La présente invention comprend : un premier moyen d'acquisition (111) qui acquiert une image de situation normale dans laquelle un conducteur est imagé dans une situation normale ; un premier moyen de détection (112) qui analyse l'image de situation normale acquise et détecte une action de situation normale ; un moyen d'apprentissage (113) qui apprend l'action de situation normale détectée ; un second moyen d'acquisition (114) qui acquiert une image inconnue dans laquelle un conducteur est imagé dans une situation dans laquelle on ne sait pas si la situation est normale ou anormale ; un second moyen de détection (115) qui analyse l'image inconnue acquise et détecte une action inconnue ; un moyen de détermination (116) qui détermine un niveau d'anomalie en fonction de l'étendue à laquelle l'action inconnue détectée diffère du résultat appris ; un moyen de sélection (117) qui sélectionne une contre-mesure en fonction du niveau d'anomalie déterminé ; et un moyen d'exécution (118) qui exécute la contre-mesure sélectionnée. Ainsi, il est possible de fournir un système de réponse d'anomalie de conducteur, un procédé de réponse d'anomalie de conducteur et un programme qui peuvent déterminer un niveau d'anomalie d'un conducteur pendant la conduite et réaliser une contre-mesure en fonction du niveau d'anomalie.
PCT/JP2019/044496 2019-11-13 2019-11-13 Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme WO2021095153A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2019/044496 WO2021095153A1 (fr) 2019-11-13 2019-11-13 Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme
JP2021555685A JPWO2021095153A1 (fr) 2019-11-13 2019-11-13

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/044496 WO2021095153A1 (fr) 2019-11-13 2019-11-13 Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme

Publications (1)

Publication Number Publication Date
WO2021095153A1 true WO2021095153A1 (fr) 2021-05-20

Family

ID=75911927

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/044496 WO2021095153A1 (fr) 2019-11-13 2019-11-13 Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme

Country Status (2)

Country Link
JP (1) JPWO2021095153A1 (fr)
WO (1) WO2021095153A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113353086A (zh) * 2021-07-16 2021-09-07 恒大恒驰新能源汽车研究院(上海)有限公司 一种车辆控制方法、装置及电子设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0554300A (ja) * 1991-08-22 1993-03-05 Omron Corp 覚醒度検知装置
JPH08332871A (ja) * 1995-06-06 1996-12-17 Mitsubishi Electric Corp 覚醒度検出装置
JPH11339200A (ja) * 1998-05-28 1999-12-10 Toyota Motor Corp 居眠り運転検出装置
JP2011096048A (ja) * 2009-10-30 2011-05-12 Konica Minolta Holdings Inc 運転解析システムおよび運転記録装置
JP2014511301A (ja) * 2011-02-18 2014-05-15 本田技研工業株式会社 運転者の挙動に応答するシステムおよび方法
JP2017100562A (ja) * 2015-12-02 2017-06-08 株式会社デンソーアイティーラボラトリ 運転制御装置、運転制御方法及びプログラム
JP2017174093A (ja) * 2016-03-23 2017-09-28 日野自動車株式会社 運転者状態判定装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4191313B2 (ja) * 1999-04-23 2008-12-03 富士通株式会社 事故抑止装置
JP2008197916A (ja) * 2007-02-13 2008-08-28 Toyota Motor Corp 車両用居眠り運転防止装置
US20200331458A1 (en) * 2017-12-28 2020-10-22 Honda Motor Co., Ltd. Vehicle control system, vehicle control method, and storage medium
JP7204739B2 (ja) * 2018-03-30 2023-01-16 ソニーセミコンダクタソリューションズ株式会社 情報処理装置、移動装置、および方法、並びにプログラム

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0554300A (ja) * 1991-08-22 1993-03-05 Omron Corp 覚醒度検知装置
JPH08332871A (ja) * 1995-06-06 1996-12-17 Mitsubishi Electric Corp 覚醒度検出装置
JPH11339200A (ja) * 1998-05-28 1999-12-10 Toyota Motor Corp 居眠り運転検出装置
JP2011096048A (ja) * 2009-10-30 2011-05-12 Konica Minolta Holdings Inc 運転解析システムおよび運転記録装置
JP2014511301A (ja) * 2011-02-18 2014-05-15 本田技研工業株式会社 運転者の挙動に応答するシステムおよび方法
JP2017100562A (ja) * 2015-12-02 2017-06-08 株式会社デンソーアイティーラボラトリ 運転制御装置、運転制御方法及びプログラム
JP2017174093A (ja) * 2016-03-23 2017-09-28 日野自動車株式会社 運転者状態判定装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113353086A (zh) * 2021-07-16 2021-09-07 恒大恒驰新能源汽车研究院(上海)有限公司 一种车辆控制方法、装置及电子设备

Also Published As

Publication number Publication date
JPWO2021095153A1 (fr) 2021-05-20

Similar Documents

Publication Publication Date Title
Biswal et al. IoT‐based smart alert system for drowsy driver detection
Ramzan et al. A survey on state-of-the-art drowsiness detection techniques
Li et al. Modeling of driver behavior in real world scenarios using multiple noninvasive sensors
El Khatib et al. Driver inattention detection in the context of next-generation autonomous vehicles design: A survey
US11609565B2 (en) Methods and systems to facilitate monitoring center for ride share and safe testing method based for selfdriving cars to reduce the false call by deuddaction systems based on deep learning machine
US20160166204A1 (en) Detecting visual impairment through normal use of a mobile device
WO2022142614A1 (fr) Procédé et appareil d'alerte précoce de conduite dangereuse, dispositif informatique et support de stockage
Abulkhair et al. Mobile platform detect and alerts system for driver fatigue
CN111445146A (zh) 一种订单监控的方法及装置
Guria et al. Iot-enabled driver drowsiness detection using machine learning
Alam et al. Vision-based driver’s attention monitoring system for smart vehicles
Alam et al. Active vision-based attention monitoring system for non-distracted driving
WO2021095153A1 (fr) Système de réponse d'anomalie de conducteur, procédé de réponse d'anomalie de conducteur et programme
Bajaj et al. A real-time driver drowsiness detection using OpenCV, DLib
Vasudevan et al. Driver drowsiness monitoring by learning vehicle telemetry data
Li et al. Real-time driver drowsiness estimation by multi-source information fusion with Dempster–Shafer theory
Presta et al. Driver monitoring systems to increase road safety
Agarkar et al. Driver Drowsiness Detection and Warning using Facial Features and Hand Gestures
CN110751810A (zh) 一种疲劳驾驶检测方法和装置
Swetha et al. Vehicle Accident Prevention System Using Artificial Intelligence
Peddarapu et al. Raspberry Pi-Based Driver Drowsiness Detection
Brown et al. Yawning Detection using Earphone Inertial Measurement Units
Pradhan et al. Driver Drowsiness Detection Model System Using EAR
JP2019016213A (ja) 車載機、処理装置およびプログラム
Tejashwini et al. Drowsy Driving Detection System–IoT Perspective

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19952552

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021555685

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19952552

Country of ref document: EP

Kind code of ref document: A1