WO2021124548A1 - In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method - Google Patents

In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method Download PDF

Info

Publication number
WO2021124548A1
WO2021124548A1 PCT/JP2019/050091 JP2019050091W WO2021124548A1 WO 2021124548 A1 WO2021124548 A1 WO 2021124548A1 JP 2019050091 W JP2019050091 W JP 2019050091W WO 2021124548 A1 WO2021124548 A1 WO 2021124548A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
area
determination unit
infant
data
Prior art date
Application number
PCT/JP2019/050091
Other languages
French (fr)
Japanese (ja)
Inventor
大輔 森本
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to PCT/JP2019/050091 priority Critical patent/WO2021124548A1/en
Priority to JP2021565284A priority patent/JP7154443B2/en
Priority to DE112019007988.3T priority patent/DE112019007988B4/en
Publication of WO2021124548A1 publication Critical patent/WO2021124548A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/593—Recognising seat occupancy

Definitions

  • This disclosure relates to an in-vehicle monitoring device, an in-vehicle monitoring system, and an in-vehicle monitoring method.
  • the conventional in-vehicle monitoring device When the vehicle is parked, the conventional in-vehicle monitoring device performs image processing on the image of the inside of the vehicle captured by the camera to extract the contour of the occupant, and detects the infant remaining in the vehicle based on the extracted contour. (See, for example, Patent Document 1).
  • the camera alone had blind spots such as hidden parts in the seat, so the in-vehicle monitoring device could not detect the infant in the blind spot.
  • This disclosure was made to solve the above problems, and aims to detect infants in the car.
  • the in-vehicle monitoring device is an in-vehicle monitoring device that detects an occupant when the vehicle stops, and is provided by a first data acquisition unit that acquires the first data acquired by the first sensor and a second sensor.
  • the occupant who is in the vehicle is an infant using the second data acquisition unit that acquires the acquired second data and the first data acquired by the first data acquisition unit when the vehicle stops.
  • the first determination unit that determines whether or not the vehicle is stopped and the second data acquired by the second data acquisition unit when the vehicle is stopped, it is determined whether or not the occupant in the vehicle is an infant.
  • the in-vehicle monitoring device uses a plurality of sensors, it is possible to detect an infant in the vehicle.
  • FIGS. 1A and 1B are block diagrams showing a configuration example of an in-vehicle monitoring system according to the first embodiment. It is a figure which shows the installation example of a camera and a millimeter wave sensor in a vehicle. It is a figure which shows the example of the image pickup range of a camera, and the detection range of a millimeter wave sensor.
  • 4A, 4B, and 4C are diagrams showing an example of the positional relationship between the rear seats and the first area, the second area, and the third area, and FIG. 4D shows the front seats, the fourth area, and the third area. It is a figure which shows the example of the positional relationship with 5 areas.
  • FIG. 7A and 7B are diagrams showing a hardware configuration example of the in-vehicle monitoring device according to the first embodiment. It is a figure which shows the positional relationship between a front seat and a 1st area, a 2nd area, and a 3rd area.
  • FIG. 1A is a block diagram showing a configuration example of the in-vehicle monitoring system 20 according to the first embodiment.
  • the in-vehicle monitoring system 20 includes a first sensor 1, a second sensor 2, and an in-vehicle monitoring device 10.
  • the first sensor 1, the second sensor 2, and the notification unit 21 are connected to the in-vehicle monitoring device 10.
  • the first sensor 1, the second sensor 2, and the in-vehicle monitoring device 10 are mounted on the vehicle 30 (see FIG. 2).
  • the notification unit 21 may be at least one of a speaker or a display mounted on the vehicle 30, a mobile terminal or the like possessed by an occupant in the vehicle 30, or the vehicle 30. It may be a mounted hazard lamp, horn (horn), or the like.
  • the first sensor 1 and the second sensor 2 are sensors capable of detecting an occupant in the vehicle, and are, for example, a camera (visible light camera or a camera having a specific electromagnetic wave filter), an electrocardiographic sensor (EKG or ECG), and a photoelectric type. Volumetric pulse wave recording (photopretismography) sensing, brain wave measuring instrument (EEG), lung activity meter, respiratory activity measuring device, TOF (Time-of-Flight) sensor, millimeter wave sensor, millimeter wave radar, pulse oximeter, thermo With graphers, thermal imagers, infrared imagers, facial muscle movement detectors, skin temperature conductance sensors, skin resistance sensors, sweating amount sensors, near-infrared spectrometers, computer tomography (CT), voice sensors, etc. is there.
  • the first sensor 1 and the second sensor 2 are different types of sensors.
  • the in-vehicle monitoring device 10 includes a first data acquisition unit 11, a second data acquisition unit 12, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15.
  • the first data acquisition unit 11 acquires the first data acquired by the first sensor 1 and outputs the acquired first data to the first determination unit 13.
  • the second data acquisition unit 12 acquires the second data acquired by the second sensor 2 and outputs the acquired second data to the second determination unit 14.
  • the first determination unit 13 detects the occupants in each seat of the vehicle 30 by using the first data acquired by the first data acquisition unit 11.
  • the second determination unit 14 detects the occupants in each seat of the vehicle 30 by using the second data acquired by the second data acquisition unit 12.
  • the occupant detection performed by the first determination unit 13 and the second determination unit 14 may be performed by analyzing the acquired data, or by comparing the acquired data with the data prepared in advance, the threshold value, or the like. Good. Further, the first determination unit 13 and the second determination unit 14 may perform occupant detection by combining the data acquired while the vehicle 30 is running and the data acquired after the vehicle 30 is stopped. The occupant detection includes determination of whether or not the occupant in the vehicle 30 is an infant.
  • the timing of occupant detection may be the moment when the vehicle 30 is stopped, or may be after a certain time has elapsed from the moment when the vehicle 30 is stopped.
  • the stop of the vehicle 30 may be a temporary stop (that is, a stop) or a continuous stop (that is, parking).
  • the in-vehicle monitoring device 10 may acquire information indicating whether or not the vehicle 30 has stopped, such as information on a shift lever or a parking brake, from the vehicle 30.
  • the estimation result output unit 15 receives the determination result from the first determination unit 13 and also receives the determination result from the second determination unit 14. The estimation result output unit 15 estimates the riding of the infant based on the received determination result, and outputs the estimation result. When the estimation result output unit 15 determines that the infant is on board by at least one of the first determination unit 13 and the second determination unit 14, it is estimated that the infant is on the vehicle 30.
  • the in-vehicle monitoring device 10 detects an infant in the vehicle by using a plurality of different types of sensors, the in-vehicle monitoring device 10 more reliably detects the infant in the vehicle as compared with the case where the infant in the vehicle is detected by using only one sensor. be able to.
  • the first sensor 1 is a camera and the second sensor is a Doppler sensor such as a millimeter wave sensor will be described, but the embodiments shown below are examples, and the present disclosure is based on these embodiments. It is not limited.
  • FIG. 1B is a block diagram showing a configuration example of the vehicle interior monitoring device 10 according to the first embodiment, in which a camera 1A is used as the first sensor 1 and a millimeter wave sensor 2A is used as the second sensor 2. is there.
  • the first data acquisition unit 11 is referred to as an image acquisition unit 11A for acquiring image data captured by the camera 1A
  • the second data acquisition unit 12 is a distance for acquiring the distance data measured by the millimeter wave sensor 2A. It is referred to as acquisition unit 12A.
  • FIG. 2 is a diagram showing an installation example of the camera 1A and the millimeter wave sensor 2A in the vehicle 30.
  • FIG. 3 is a diagram showing an example of an imaging range 1a of the camera 1A and a detection range 2a of the millimeter wave sensor 2A.
  • an adult 34 is seated in the front seat 31.
  • an auxiliary device for infants 33 (hereinafter referred to as CRS33; CRS: Child Restraint System) is attached to the rear seat 32, and the infant 35 is seated on the CRS 33.
  • CRS33 CRS: Child Restraint System
  • the CRS 33 in the illustrated example is a backward type, it may be a forward type.
  • the camera 1A is installed on an overhead console or the like in front of the vehicle and images the front seats 31 and the rear seats 32.
  • the imaging range 1a of the camera 1A there are blind spots such as a portion hidden by the front seat 31 and the inside of the trunk 36, and the vehicle interior monitoring device 10 cannot detect an occupant existing in the blind spot.
  • the in-vehicle monitoring device 10 cannot detect an occupant whose whole body is covered with a blanket or the like.
  • the millimeter wave sensor 2A is installed near the ceiling at the rear of the vehicle, transmits millimeter waves to the rear seats 32, and receives the reflected waves reflected by the moving object. Then, the millimeter wave sensor 2A measures the distance from the installation position of the millimeter wave sensor 2A to the moving object to which the millimeter wave is reflected by using the transmission / reception result of the millimeter wave. Further, the millimeter wave sensor 2A may transmit a millimeter wave to the trunk 36 in addition to the rear seat 32 to measure the distance of a moving object in the trunk 36.
  • a Doppler sensor such as the millimeter wave sensor 2A, the in-vehicle monitoring device 10 can detect the occupant from the movement of the chest or the heartbeat (movement of the heart) due to breathing.
  • the millimeter-wave sensor 2A is an image captured by the camera 1A, such as an occupant blocked by a front seat 31, CRS33 or a sunshade (not shown), an occupant whose whole body is covered with a blanket or the like, and an occupant trapped in the trunk 36. Can react to occupants who do not appear in the image.
  • the millimeter wave sensor 2A also reacts to moving dolls and water swaying in PET bottles, it is difficult to distinguish between dolls and occupants. Therefore, it is difficult for the in-vehicle monitoring device 10 to accurately detect the occupant only by the millimeter wave sensor 2A, and there is a possibility that the occupant may be erroneously detected even though the vehicle 30 has no occupant.
  • the in-vehicle monitoring system 20 combines the camera 1A and the millimeter wave sensor 2A to complement each other's weaknesses, and aims to improve the robustness of occupant detection and the detection accuracy.
  • the in-vehicle monitoring device 10 includes an image acquisition unit 11A, a distance acquisition unit 12A, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15.
  • the image acquisition unit 11A acquires image data obtained by the camera 1A capturing the inside of the vehicle 30 from the camera 1A.
  • the image acquisition unit 11A outputs the image data acquired from the camera 1A to the first determination unit 13.
  • the distance acquisition unit 12A acquires the distance data obtained by the millimeter wave sensor 2A measuring the distance to each object in the vehicle 30 from the millimeter wave sensor 2A.
  • the distance acquisition unit 12A outputs the distance data acquired from the millimeter wave sensor 2A to 14.
  • the first determination unit 13 detects the occupants in the front seats 31 and the rear seats 32 by using the image data acquired by the image acquisition unit 11A, and of each of the detected occupants. Determine your physique. The first determination unit 13 determines whether each occupant is an adult or an infant based on the physique determination result of each occupant. The first determination unit 13 outputs the determination result to the estimation result output unit 15. For example, in the first embodiment, an adult is a occupant having a physique that allows him / her to go out of the vehicle by himself / herself even if he / she is left behind in the vehicle 30, and an infant is an occupant who can go out of the vehicle by himself / herself when left in the vehicle 30.
  • the criteria for determining whether each occupant is an adult or a child is not limited to the above example, and can be arbitrarily set by the user. For example, even if the occupant has a physique that allows him / her to get out of the vehicle by himself / herself, if the physique of a child is an age at which he / she is obliged to wear a child seat or the like, it can be set to be judged as an infant.
  • the second determination unit 14 detects the occupants in the rear seat 32 by using the distance data acquired by the distance acquisition unit 12A, and whether each of the detected occupants is an adult or an infant. To judge. The second determination unit 14 outputs the determination result to the estimation result output unit 15. The second determination unit 14 may detect an occupant in the trunk 36 by using the distance data acquired by the distance acquisition unit 12A. The arrows for outputting information from the first determination unit 13 to the second determination unit 14 shown in FIGS. 1A and 1B will be described later.
  • FIG. 4A, 4B, and 4C are diagrams showing an example of the positional relationship between the rear seat 32 and the first area 41, the second area 42, and the third area 43.
  • FIG. 4A shows an example in which an adult 37 is seated in the rear seat 32.
  • FIG. 4B shows an example in which the infant 35 is seated on the forward-facing type CRS 33 mounted on the rear seat 32.
  • FIG. 4C shows an example in which the infant 35 is seated on the rear-facing type CRS 33 mounted on the rear seat 32.
  • the millimeter wave sensor 2A is installed above the rear seat 32 and transmits millimeter waves toward the bottom of the front seat 31.
  • the second determination unit 14 sets the detection range 2a of the millimeter wave sensor 2A to the first area 41 closer to the installation position of the millimeter wave sensor 2A, the second area 42 farther from the installation position than the first area 41, and the second. It is divided into a third area 43, which is farther from the installation position than the area 42. Information indicating the distances between the first area 41, the second area 42, and the third area 43 is held in advance by the second determination unit 14.
  • the first area 41 is an area including the headrest of the rear seat 32.
  • the second area 42 is an area including the backrest of the rear seat 32, and the seat surface of the rear seat 32 serves as a boundary surface between the second area 42 and the third area 43.
  • the third area 43 is an area including the feet of the rear seats 32.
  • the second determination unit 14 classifies the distance data measured by the millimeter wave sensor 2A into the first area 41, the second area 42, and the third area 43.
  • the second determination unit 14 when the distance data is classified into all of the first area 41, the second area 42, and the third area 43, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the first area 41.
  • the reflection is performed in all of the second area 42 and the third area 43, it is determined that the adult 37 is seated in the rear seat 32.
  • the second determination unit 14 when the distance data is not classified into the first area 41 and the third area 43 but is classified into the second area 42, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the second.
  • the reflection occurs in the area 42, it is determined that the infant 35 is seated on the CRS 33 mounted on the rear seat 32. Further, in the second determination unit 14, when the distance data is not classified into the first area 41 and the second area 42 but is classified into the third area 43, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the third. When the reflection occurs in the area 43, it is determined that there is an infant (not shown) at the foot of the rear seat 32.
  • FIG. 4D is a diagram showing an example of the positional relationship between the front seat 31 and the fourth area 44 and the fifth area 45.
  • FIG. 4D shows an example in which an adult 34 is seated on the driver's seat side of the front seat 31 and an infant 38 is seated on a forward-facing type CRS 33 mounted on the passenger seat side of the front seat 31.
  • a fourth area 44 (seat portion of the front seat 31) and a fifth area 45 (foot portion of the front seat 31) are set on the front seat 31 side of the third area 43.
  • the second determination unit 14 determines that the adult 34 is seated in the front seat 31. Further, the second determination unit 14 determines that the infant 38 is seated on the CRS 33 mounted on the front seat 31 when the distance data is classified into the fourth area 44. Further, the second determination unit 14 determines that there is an infant (not shown) at the foot of the front seat 31 when the distance data is classified into the fifth area 45. In the example of FIG. 4D, since not only the infant 38 but also the adult 34 is seated in the front seat 31, the distance data is classified into the third area 43, the fourth area 44, and the fifth area 45. , It is determined that the adult 34 is seated.
  • FIG. 5 is a reference example for assisting the understanding of the in-vehicle monitoring system 20 according to the first embodiment, and shows an example in which the millimeter wave sensor 2A is installed in front of the rear seat 32.
  • the millimeter wave sensor 2A is installed on the ceiling in front of the rear seat 32 and transmits millimeter waves toward the rear seat 32.
  • the detection range 2a of the millimeter wave sensor 2A is divided into a first area 41 and a second area 42 according to the distance from the installation position of the millimeter wave sensor 2A. .. Therefore, it is difficult to determine whether the occupant seated in the rear seat 32 is an infant 35 or an adult 37 only by the distance data of the millimeter wave sensor 2A.
  • the estimation result output unit 15 receives the determination result from the first determination unit 13 and also receives the determination result from the second determination unit 14. The estimation result output unit 15 estimates the riding of the infant based on the received determination result, and outputs the estimation result to the notification unit 21. When the estimation result output unit 15 determines that the infant is on board by at least one of the first determination unit 13 and the second determination unit 14, it is estimated that the infant is on board.
  • the notification unit 21 notifies based on the estimation result output by the estimation result output unit 15. For example, when the estimation result that the infant is on board is input from the estimation result output unit 15, the notification unit 21 notifies that the infant remains in the vehicle 30 by voice or display, and the infant is notified. If the estimation result that the vehicle is not on board is input from the estimation result output unit 15, no notification is given. As a result, the in-vehicle monitoring system 20 can notify the occupant that the infant is in the vehicle, so that the infant can be prevented from being left behind.
  • FIG. 6 is a flowchart showing an operation example of the in-vehicle monitoring device 10 according to the first embodiment.
  • the in-vehicle monitoring device 10 operates after step ST2, and when the vehicle 30 is running (step ST1 "NO"), until the vehicle 30 is stopped. The operation of step ST1 is repeated.
  • step ST2 the image acquisition unit 11A acquires image data from the camera 1A.
  • step ST3 the first determination unit 13 detects the occupants of the front seats 31 and the rear seats 32 using the image data, and determines whether or not the detected occupants are infants.
  • step ST4 the distance acquisition unit 12A acquires distance data from the millimeter wave sensor 2A.
  • step ST5 the second determination unit 14 classifies the distance data into the first area 41, the second area 42, and the third area 43, and the occupants of the front seat 31 and the rear seat 32 are infants based on the classification result. Determine if it exists.
  • the operations of steps ST2 and ST3 and the operations of steps ST4 and ST5 are performed in parallel.
  • step ST6 the estimation result output unit 15 estimates the riding of the infant based on the determination results received from the first determination unit 13 and the second determination unit 14, and outputs the estimation result.
  • step ST7 the notification unit 21 notifies based on the estimation result output by the estimation result output unit 15. Then, the in-vehicle monitoring device 10 ends the operation shown in the flowchart of FIG.
  • FIG. 7A and 7B are diagrams showing a hardware configuration example of the in-vehicle monitoring device 10 according to the first embodiment.
  • the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 in the in-vehicle monitoring device 10 are realized by the processing circuit. That is, the in-vehicle monitoring device 10 includes a processing circuit for realizing the above functions.
  • the processing circuit may be a processing circuit 100 as dedicated hardware, or a processor 101 that executes a program stored in the memory 102.
  • the processing circuit 100 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific Integrated Circuit). ), FPGA (Field Processor Gate Array), or a combination thereof.
  • the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 may be realized by a plurality of processing circuits 100, or the functions of each unit may be integrated. It may be realized by one processing circuit 100.
  • the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 are software. , Firmware, or a combination of software and firmware.
  • the software or firmware is described as a program and stored in the memory 102.
  • the processor 101 realizes the functions of each part by reading and executing the program stored in the memory 102. That is, the in-vehicle monitoring device 10 includes a memory 102 for storing a program in which the step shown in the flowchart of FIG. 6 is eventually executed when executed by the processor 101. Further, it can be said that this program causes the computer to execute the procedure or method of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15.
  • the processor 101 is a CPU (Central Processing Unit), a processing device, an arithmetic unit, a microprocessor, or the like.
  • the memory 102 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Program ROM), or a flash memory, and may be a non-volatile or volatile semiconductor memory such as a hard disk or a flexible disk. It may be an optical disc such as a CD (Compact Disc) or a DVD (Digital Versaille Disc).
  • the processing circuit in the in-vehicle monitoring device 10 can realize the above-mentioned functions by hardware, software, firmware, or a combination thereof.
  • the in-vehicle monitoring device 10 includes an image acquisition unit 11A, a distance acquisition unit 12A, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15.
  • the image acquisition unit 11A acquires image data obtained by the camera 1A capturing the inside of the vehicle.
  • the distance acquisition unit 12A acquires distance data obtained by measuring the distance to a moving object in the vehicle by the millimeter wave sensor 2A.
  • the first determination unit 13 determines whether or not the occupant seated in the vehicle 30 is an infant or not by using the image data acquired by the image acquisition unit 11A.
  • the second determination unit 14 determines whether or not the occupant seated in the vehicle 30 is an infant or not by using the distance data acquired by the distance acquisition unit 12A.
  • the estimation result output unit 15 instructs the notification unit 21 to output a warning when at least one of the first determination unit 13 and the second determination unit 14 determines that the infant is on board.
  • the millimeter wave sensor 2A of the first embodiment is provided above the rear seat 32 and transmits millimeter waves toward the feet of the rear seat 32.
  • the second determination unit 14 uses the distance data acquired by the distance acquisition unit 12A as the first area 41 closer to the installation position of the millimeter wave sensor 2A, the second area 42 farther from the installation position than the first area 41, and the second area 42. It is classified into the third area 43, which is farther from the installation position than the second area 42. Then, the second determination unit 14 determines that there is an adult in the rear seat 32 when the distance data is classified into all of the first area 41, the second area 42, and the third area 43.
  • the second determination unit 14 determines that there is an infant in the rear seat 32 when the distance data is not classified into the first area 41 but is classified into at least one of the second area 42 or the third area 43. With this configuration, the second determination unit 14 can distinguish between an adult and an infant by using the distance data measured by the millimeter wave sensor 2A.
  • the notification unit 21 of the first embodiment is configured to notify the ride of the infant when the estimation result that the infant is on board is input from the estimation result output unit 15, but the configuration of the notification unit 21 Is not limited to this.
  • the estimation result output unit 15 estimates that an adult and an infant are on board when the first determination unit 13 determines that an adult is on board and the second determination unit 14 determines that an infant is on board.
  • the notification unit 21 not only the estimation result that the infant is on board but also the estimation result that the adult is on board is input.
  • the notification unit 21 may be configured not to notify the ride of an infant when the estimation result that an adult is on board is input. This is because notification is required when only infants are left in the vehicle 30, and notification is not required when an adult is on board.
  • the in-vehicle monitoring device 10 when it is determined that one or more adults are in the vehicle 30 based on the image data of the camera 1A, the in-vehicle monitoring device 10 notifies even if there is an infant in the blind spot of the camera 1A. Since it can be done, unnecessary notifications can be reduced. Further, for example, when the second determination unit 13 detects the occupant in the trunk 36, the estimation result output unit 15 may output the estimation result that the occupant is in the trunk 36 to the notification unit 21. The notification unit 21 notifies when the estimation result that there is an occupant in the trunk 36 is input.
  • FIG. 8 is a diagram showing the positional relationship between the front seat 31 and the first area 41, the second area 42, and the third area 43. As shown in FIG. 8, in the third area 43, the feet of the rear seats 32 and the front seats 31 are at the same distance. Therefore, when there is an adult 34 in the front seat 31, there is a possibility that the second determination unit 14 erroneously determines that there is an infant at the foot of the rear seat 32 even though there is no infant at the foot of the rear seat 32. is there.
  • the determination result is transmitted to the second determination unit 14 in addition to the estimation result output unit 15. Output.
  • the second determination unit 14 invalidates the third area 43.
  • the second determination unit 14 determines that there is an adult in the rear seat 32 when the distance data is classified into both the first area 41 and the second area 42, and the distance data is classified into the first area 41.
  • it is classified into the second area 42 it is determined that there is an infant in the rear seat 32.
  • the in-vehicle monitoring device 10 can accurately detect the infant left in the vehicle 30.
  • the application of the in-vehicle monitoring system 20 according to the first embodiment is not limited to the application of detecting an infant in the vehicle 30 when the vehicle 30 is stopped.
  • the in-vehicle monitoring device 10 may have a function of determining whether or not the occupant is wearing a seatbelt.
  • the first determination unit 13 and the second determination unit 14 detect the presence or absence of occupants in each seat.
  • the above function determines whether or not each occupant detected by the first determination unit 13 and the second determination unit 14 is wearing a seatbelt based on the information from the seatbelt sensor of each seat.
  • the estimation result output unit 15 outputs the wearing result of each occupant's seatbelt, and the notification unit 21 gives a notification based on the wearing result output by the estimation result output unit 15.
  • the in-vehicle monitoring device 10 may have a function of switching whether or not the airbag is activated.
  • the first determination unit 13 and the second determination unit 14 detect the presence or absence of occupants in each seat and determine the physique of each detected occupant. The above-mentioned function switches whether or not the airbag of each seat is activated according to the physique of each occupant determined by the first determination unit 13 and the second determination unit 14.
  • the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 are mounted on the vehicle 30.
  • all or part of the functions may be in the server device on the network.
  • any component of the embodiment can be modified or any component of the embodiment can be omitted in the present disclosure.
  • the in-vehicle monitoring system determines whether an infant is on board, it is suitable for use in an in-vehicle monitoring system or the like that prevents the infant from being left behind.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Geophysics And Detection Of Objects (AREA)
  • Traffic Control Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

In the present invention, if a vehicle (30) stops, a first determination unit (13) determines whether an occupant riding in the vehicle (30) is an infant by using image data obtained by a camera (1A) which captures the inside of the vehicle. If the vehicle (30) stops, a second determination unit (14) determines whether the occupant in the vehicle (30) is an infant by using distance data obtained by measuring the distance to a moving object in the vehicle with a millimeter wave sensor (2A) An estimation results output unit (15) estimates that an infant is riding in the vehicle, on the basis of the determination results of the first determination unit (13) and the second determination unit (14).

Description

車内監視装置、車内監視システム、及び車内監視方法In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method
　本開示は、車内監視装置、車内監視システム、及び車内監視方法に関するものである。 This disclosure relates to an in-vehicle monitoring device, an in-vehicle monitoring system, and an in-vehicle monitoring method.
　従来の車内監視装置は、車両駐車時、カメラにより撮像された車内の画像に対して画像処理を行って乗員の輪郭を抽出し、抽出した輪郭に基づいて車内に残っている乳幼児を検知していた（例えば、特許文献１参照）。 When the vehicle is parked, the conventional in-vehicle monitoring device performs image processing on the image of the inside of the vehicle captured by the camera to extract the contour of the occupant, and detects the infant remaining in the vehicle based on the extracted contour. (See, for example, Patent Document 1).
特開２００８－２９６７７７号公報Japanese Unexamined Patent Publication No. 2008-296777
　カメラ単体では、座席で隠れた部分等の死角が存在するため、車内監視装置は、死角にいる乳幼児を検知できなかった。 The camera alone had blind spots such as hidden parts in the seat, so the in-vehicle monitoring device could not detect the infant in the blind spot.
　本開示は、上記のような課題を解決するためになされたもので、車内にいる乳幼児を検知することを目的とする。 This disclosure was made to solve the above problems, and aims to detect infants in the car.
　本開示に係る車内監視装置は、車両が停止した場合に乗員を検知する車内監視装置であって、第１センサによって取得された第１データを取得する第１データ取得部と、第２センサによって取得された第２データを取得する第２データ取得部と、車両が停止した場合に、第１データ取得部によって取得された第１データを用いて、車両に乗車している乗員が乳幼児であるか否かを判定する第１判定部と、車両が停止した場合に、第２データ取得部によって取得された第２データを用いて、車両に乗車している乗員が乳幼児であるか否かを判定する第２判定部と、第１判定部及び第２判定部の判定結果に基づき、乳幼児の乗車を推定し、推定結果を出力する推定結果出力部と、を備える。 The in-vehicle monitoring device according to the present disclosure is an in-vehicle monitoring device that detects an occupant when the vehicle stops, and is provided by a first data acquisition unit that acquires the first data acquired by the first sensor and a second sensor. The occupant who is in the vehicle is an infant using the second data acquisition unit that acquires the acquired second data and the first data acquired by the first data acquisition unit when the vehicle stops. Using the first determination unit that determines whether or not the vehicle is stopped and the second data acquired by the second data acquisition unit when the vehicle is stopped, it is determined whether or not the occupant in the vehicle is an infant. It includes a second determination unit for determination, and an estimation result output unit that estimates the ride of an infant based on the determination results of the first determination unit and the second determination unit and outputs the estimation result.
　本開示によれば、車内監視装置は、複数のセンサを用いるようにしたので、車内にいる乳幼児を検知することができる。 According to the present disclosure, since the in-vehicle monitoring device uses a plurality of sensors, it is possible to detect an infant in the vehicle.
図１Ａ及び図１Ｂは、実施の形態１に係る車内監視システムの構成例を示すブロック図である。1A and 1B are block diagrams showing a configuration example of an in-vehicle monitoring system according to the first embodiment. 車両におけるカメラとミリ波センサの設置例を示す図である。It is a figure which shows the installation example of a camera and a millimeter wave sensor in a vehicle. カメラの撮像範囲とミリ波センサの検知範囲の例を示す図である。It is a figure which shows the example of the image pickup range of a camera, and the detection range of a millimeter wave sensor. 図４Ａ、図４Ｂ、及び図４Ｃは、後席と第１エリア、第２エリア、及び第３エリアとの位置関係の例を示す図であり、図４Ｄは、前席と第４エリア及び第５エリアとの位置関係の例を示す図である。4A, 4B, and 4C are diagrams showing an example of the positional relationship between the rear seats and the first area, the second area, and the third area, and FIG. 4D shows the front seats, the fourth area, and the third area. It is a figure which shows the example of the positional relationship with 5 areas. 実施の形態１に係る車内監視システムの理解を助けるための参考例であり、ミリ波センサが後席の前方に設置された例を示す。This is a reference example for assisting the understanding of the in-vehicle monitoring system according to the first embodiment, and shows an example in which a millimeter wave sensor is installed in front of the rear seat. 実施の形態１に係る車内監視装置の動作例を示すフローチャートである。It is a flowchart which shows the operation example of the vehicle-in-vehicle monitoring device which concerns on Embodiment 1. FIG. 図７Ａ及び図７Ｂは、実施の形態１に係る車内監視装置のハードウェア構成例を示す図である。7A and 7B are diagrams showing a hardware configuration example of the in-vehicle monitoring device according to the first embodiment. 前席と第１エリア、第２エリア、及び第３エリアとの位置関係を示す図である。It is a figure which shows the positional relationship between a front seat and a 1st area, a 2nd area, and a 3rd area.
　以下、本開示をより詳細に説明するために、本開示を実施するための形態について、添付の図面に従って説明する。
実施の形態１．
　図１Ａは、実施の形態１に係る車内監視システム２０の構成例を示すブロック図である。車内監視システム２０は、第１センサ１、第２センサ２、及び車内監視装置１０を備える。車内監視装置１０には、第１センサ１、第２センサ２、及び通知部２１が接続される。第１センサ１、第２センサ２、及び車内監視装置１０は、車両３０（図２参照）に搭載される。通知部２１は、車両３０に搭載されたスピーカ又はディスプレイの少なくとも一方であってもよいし、車両３０に乗車している乗員が所持している携帯端末等であってもよいし、車両３０に搭載されたハザードランプ又はホーン（クラクション）等であってもよい。 Hereinafter, in order to explain the present disclosure in more detail, a mode for carrying out the present disclosure will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1A is a block diagram showing a configuration example of the in-vehicle monitoring system 20 according to the first embodiment. The in-vehicle monitoring system 20 includes a first sensor 1, a second sensor 2, and an in-vehicle monitoring device 10. The first sensor 1, the second sensor 2, and the notification unit 21 are connected to the in-vehicle monitoring device 10. The first sensor 1, the second sensor 2, and the in-vehicle monitoring device 10 are mounted on the vehicle 30 (see FIG. 2). The notification unit 21 may be at least one of a speaker or a display mounted on the vehicle 30, a mobile terminal or the like possessed by an occupant in the vehicle 30, or the vehicle 30. It may be a mounted hazard lamp, horn (horn), or the like.
　第１センサ１及び第２センサ２とは車内の乗員を検知可能なセンサであり、例えば、カメラ（可視光カメラ又は特定の電磁波フィルタを有するカメラ）、心電センサ（ＥＫＧ又はＥＣＧ）、光電式容積脈波記録法（フォトプレチスモグラフィ）センシング、脳波測定器具（ＥＥＧ）、肺活量計、呼吸活性測定器、ＴＯＦ（Ｔｉｍｅ－ｏｆ－Ｆｌｉｇｈｔ）センサ、ミリ波センサ、ミリ波レーダ、パルスオキシメータ、サーモグラファー、サーマルイメージャ、赤外イメージャ、顔の筋肉の動きの検知器、皮膚温コンダクタンスセンサ、皮膚抵抗センサ、発汗量センサ、近赤外分光器、コンピュータ断層撮影器（ＣＴ）、及び音声センサ等である。なお、第１センサ１と第２センサ２とは異なる種類のセンサである。 The first sensor 1 and the second sensor 2 are sensors capable of detecting an occupant in the vehicle, and are, for example, a camera (visible light camera or a camera having a specific electromagnetic wave filter), an electrocardiographic sensor (EKG or ECG), and a photoelectric type. Volumetric pulse wave recording (photopretismography) sensing, brain wave measuring instrument (EEG), lung activity meter, respiratory activity measuring device, TOF (Time-of-Flight) sensor, millimeter wave sensor, millimeter wave radar, pulse oximeter, thermo With graphers, thermal imagers, infrared imagers, facial muscle movement detectors, skin temperature conductance sensors, skin resistance sensors, sweating amount sensors, near-infrared spectrometers, computer tomography (CT), voice sensors, etc. is there. The first sensor 1 and the second sensor 2 are different types of sensors.
　車内監視装置１０は、第１データ取得部１１、第２データ取得部１２、第１判定部１３、第２判定部１４、及び推定結果出力部１５を備える。 The in-vehicle monitoring device 10 includes a first data acquisition unit 11, a second data acquisition unit 12, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15.
　第１データ取得部１１は、第１センサ１によって取得された第１データを取得し、取得した第１データを第１判定部１３へ出力する。 The first data acquisition unit 11 acquires the first data acquired by the first sensor 1 and outputs the acquired first data to the first determination unit 13.
　第２データ取得部１２は、第２センサ２によって取得された第２データを取得し、取得した第２データを第２判定部１４へ出力する。 The second data acquisition unit 12 acquires the second data acquired by the second sensor 2 and outputs the acquired second data to the second determination unit 14.
　第１判定部１３は、車両３０が停止した場合、第１データ取得部１１により取得された第１データを用いて、車両３０の各座席に乗車している乗員検知を行う。 When the vehicle 30 is stopped, the first determination unit 13 detects the occupants in each seat of the vehicle 30 by using the first data acquired by the first data acquisition unit 11.
　第２判定部１４は、車両３０が停止した場合、第２データ取得部１２により取得された第２データを用いて、車両３０の各座席に乗車している乗員検知を行う。 When the vehicle 30 is stopped, the second determination unit 14 detects the occupants in each seat of the vehicle 30 by using the second data acquired by the second data acquisition unit 12.
　第１判定部１３及び第２判定部１４によって行われる乗員検知は、取得したデータを解析して行ってもよいし、取得したデータを予め用意されたデータ又は閾値等との比較により行ってもよい。また、第１判定部１３及び第２判定部１４は、車両３０の走行中に取得したデータと、車両３０の停止後に取得したデータとを組み合わせて乗員検知を行ってもよい。なお、乗員検知とは車両３０に乗車している乗員が乳幼児であるか否かの判定を含む。 The occupant detection performed by the first determination unit 13 and the second determination unit 14 may be performed by analyzing the acquired data, or by comparing the acquired data with the data prepared in advance, the threshold value, or the like. Good. Further, the first determination unit 13 and the second determination unit 14 may perform occupant detection by combining the data acquired while the vehicle 30 is running and the data acquired after the vehicle 30 is stopped. The occupant detection includes determination of whether or not the occupant in the vehicle 30 is an infant.
　また、乗員検知を行うタイミングは、車両３０が停止した瞬間でもよいし、車両３０が停止した瞬間から一定時間が経過した後であってもよい。ここで、車両３０の停止とは、一時的な停止（つまり停車）であってもよいし、継続的な停止（つまり駐車）であってもよい。なお、車内監視装置１０は、シフトレバー又はパーキングブレーキの情報等、車両３０が停止したか否かを示す情報を、車両３０から取得すればよい。 Further, the timing of occupant detection may be the moment when the vehicle 30 is stopped, or may be after a certain time has elapsed from the moment when the vehicle 30 is stopped. Here, the stop of the vehicle 30 may be a temporary stop (that is, a stop) or a continuous stop (that is, parking). The in-vehicle monitoring device 10 may acquire information indicating whether or not the vehicle 30 has stopped, such as information on a shift lever or a parking brake, from the vehicle 30.
　推定結果出力部１５は、第１判定部１３から判定結果を受け取ると共に、第２判定部１４から判定結果を受け取る。推定結果出力部１５は、受け取った判定結果に基づき、乳幼児の乗車を推定し、推定結果を出力する。推定結果出力部１５は、第１判定部１３又は第２判定部１４の少なくとも一方により乳幼児の乗車が判定された場合、車両３０に乳幼児が乗車していると推定する。 The estimation result output unit 15 receives the determination result from the first determination unit 13 and also receives the determination result from the second determination unit 14. The estimation result output unit 15 estimates the riding of the infant based on the received determination result, and outputs the estimation result. When the estimation result output unit 15 determines that the infant is on board by at least one of the first determination unit 13 and the second determination unit 14, it is estimated that the infant is on the vehicle 30.
　車内監視装置１０は、異なる種類のセンサを複数用いて車内の乳幼児を検知するため、一つのセンサのみを用いて車内の乳幼児を検知する場合と比較して、より確実に車内の乳幼児を検知することができる。 Since the in-vehicle monitoring device 10 detects an infant in the vehicle by using a plurality of different types of sensors, the in-vehicle monitoring device 10 more reliably detects the infant in the vehicle as compared with the case where the infant in the vehicle is detected by using only one sensor. be able to.
　以下、第１センサ１がカメラであり、第２センサがミリ波センサ等のドップラーセンサである例について記載するが、以下に示す実施の形態は一例であり、これらの実施の形態によって本開示が限定されるものではない。 Hereinafter, an example in which the first sensor 1 is a camera and the second sensor is a Doppler sensor such as a millimeter wave sensor will be described, but the embodiments shown below are examples, and the present disclosure is based on these embodiments. It is not limited.
　図１Ｂは、実施の形態１に係る車内監視装置１０の構成例を示すブロック図であり、第１センサ１としてカメラ１Ａが使用され、第２センサ２としてミリ波センサ２Ａが使用される例である。この例において、第１データ取得部１１を、カメラ１Ａが撮像した画像データを取得する画像取得部１１Ａと称し、第２データ取得部１２を、ミリ波センサ２Ａが測定した距離データを取得する距離取得部１２Ａと称する。 FIG. 1B is a block diagram showing a configuration example of the vehicle interior monitoring device 10 according to the first embodiment, in which a camera 1A is used as the first sensor 1 and a millimeter wave sensor 2A is used as the second sensor 2. is there. In this example, the first data acquisition unit 11 is referred to as an image acquisition unit 11A for acquiring image data captured by the camera 1A, and the second data acquisition unit 12 is a distance for acquiring the distance data measured by the millimeter wave sensor 2A. It is referred to as acquisition unit 12A.
　図２は、車両３０におけるカメラ１Ａとミリ波センサ２Ａの設置例を示す図である。図３は、カメラ１Ａの撮像範囲１ａとミリ波センサ２Ａの検知範囲２ａの例を示す図である。図示例では、前席３１に大人３４が着座している。また、後席３２に幼児用補助装置３３（以下、ＣＲＳ３３と称する。ＣＲＳ：Ｃｈｉｌｄ　Ｒｅｓｔｒａｉｎｔ　Ｓｙｓｔｅｍ）が装着され、このＣＲＳ３３に乳幼児３５が着座している。なお、図示例のＣＲＳ３３は後ろ向きタイプであるが、前向きタイプでもよい。 FIG. 2 is a diagram showing an installation example of the camera 1A and the millimeter wave sensor 2A in the vehicle 30. FIG. 3 is a diagram showing an example of an imaging range 1a of the camera 1A and a detection range 2a of the millimeter wave sensor 2A. In the illustrated example, an adult 34 is seated in the front seat 31. Further, an auxiliary device for infants 33 (hereinafter referred to as CRS33; CRS: Child Restraint System) is attached to the rear seat 32, and the infant 35 is seated on the CRS 33. Although the CRS 33 in the illustrated example is a backward type, it may be a forward type.
　カメラ１Ａは、車内前方のオーバヘッドコンソール等に設置され、前席３１及び後席３２を撮像する。このカメラ１Ａの撮像範囲１ａには、前席３１で隠れた部分及びトランク３６の内部等の死角が存在し、車内監視装置１０は、死角に存在する乗員を検知できない。また、カメラ１Ａの設置数を増やしてカメラ１Ａが撮像した画像に写らない死角を無くしたとしても、車内監視装置１０は、毛布等で全身を覆われた乗員を検知できない。 The camera 1A is installed on an overhead console or the like in front of the vehicle and images the front seats 31 and the rear seats 32. In the imaging range 1a of the camera 1A, there are blind spots such as a portion hidden by the front seat 31 and the inside of the trunk 36, and the vehicle interior monitoring device 10 cannot detect an occupant existing in the blind spot. Further, even if the number of cameras 1A installed is increased to eliminate blind spots that are not reflected in the image captured by the camera 1A, the in-vehicle monitoring device 10 cannot detect an occupant whose whole body is covered with a blanket or the like.
　ミリ波センサ２Ａは、車内後方の天井付近に設置され、後席３２に対してミリ波を送信し、当該ミリ波が動く物体で反射した反射波を受信する。そして、ミリ波センサ２Ａは、ミリ波の送受信結果を用いて、ミリ波センサ２Ａの設置位置からミリ波が反射した動く物体までの距離を測定する。また、ミリ波センサ２Ａは、後席３２に加えてトランク３６に対してミリ波を送信し、トランク３６内の動く物体の距離を測定してもよい。
　なお、ミリ波センサ２Ａ等のドップラーセンサを用いることで、車内監視装置１０は、呼吸による胸部の動き又は心拍（心臓の動き）等から乗員を検知することができる。 The millimeter wave sensor 2A is installed near the ceiling at the rear of the vehicle, transmits millimeter waves to the rear seats 32, and receives the reflected waves reflected by the moving object. Then, the millimeter wave sensor 2A measures the distance from the installation position of the millimeter wave sensor 2A to the moving object to which the millimeter wave is reflected by using the transmission / reception result of the millimeter wave. Further, the millimeter wave sensor 2A may transmit a millimeter wave to the trunk 36 in addition to the rear seat 32 to measure the distance of a moving object in the trunk 36.
By using a Doppler sensor such as the millimeter wave sensor 2A, the in-vehicle monitoring device 10 can detect the occupant from the movement of the chest or the heartbeat (movement of the heart) due to breathing.
　ミリ波は、樹脂及び毛布等を透過する。そのため、ミリ波センサ２Ａは、前席３１、ＣＲＳ３３若しくは図示しないサンシェード等で遮られた乗員、毛布等で全身を覆われた乗員、及びトランク３６に閉じ込められた乗員等、カメラ１Ａが撮像した画像に写らない乗員に反応できる。ただし、ミリ波センサ２Ａは、動いている人形及びペットボトルの中で揺れている水等にも反応するため、人形等と乗員との区別が困難である。そのため、車内監視装置１０は、ミリ波センサ２Ａだけで乗員を正確に検知することは困難であり、車両３０に乗員がいないにもかかわらず乗員を誤検知する可能性がある。 Millimeter waves pass through resin, blankets, etc. Therefore, the millimeter-wave sensor 2A is an image captured by the camera 1A, such as an occupant blocked by a front seat 31, CRS33 or a sunshade (not shown), an occupant whose whole body is covered with a blanket or the like, and an occupant trapped in the trunk 36. Can react to occupants who do not appear in the image. However, since the millimeter wave sensor 2A also reacts to moving dolls and water swaying in PET bottles, it is difficult to distinguish between dolls and occupants. Therefore, it is difficult for the in-vehicle monitoring device 10 to accurately detect the occupant only by the millimeter wave sensor 2A, and there is a possibility that the occupant may be erroneously detected even though the vehicle 30 has no occupant.
　このようにカメラ１Ａ及びミリ波センサ２Ａには、乗員検知において長所と短所がある。そこで、実施の形態１に係る車内監視システム２０は、カメラ１Ａとミリ波センサ２Ａとを組み合わせて、互いの短所を補完し、乗員検知のロバスト性向上及び検知精度向上を図る。 As described above, the camera 1A and the millimeter wave sensor 2A have advantages and disadvantages in occupant detection. Therefore, the in-vehicle monitoring system 20 according to the first embodiment combines the camera 1A and the millimeter wave sensor 2A to complement each other's weaknesses, and aims to improve the robustness of occupant detection and the detection accuracy.
　車内監視装置１０は、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５を備える。 The in-vehicle monitoring device 10 includes an image acquisition unit 11A, a distance acquisition unit 12A, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15.
　画像取得部１１Ａは、カメラ１Ａが車両３０内を撮像した画像データを、カメラ１Ａから取得する。画像取得部１１Ａは、カメラ１Ａから取得した画像データを第１判定部１３へ出力する。 The image acquisition unit 11A acquires image data obtained by the camera 1A capturing the inside of the vehicle 30 from the camera 1A. The image acquisition unit 11A outputs the image data acquired from the camera 1A to the first determination unit 13.
　距離取得部１２Ａは、ミリ波センサ２Ａが車両３０内の各物体までの距離を測定した距離データを、ミリ波センサ２Ａから取得する。距離取得部１２Ａは、ミリ波センサ２Ａから取得した距離データを１４へ出力する。 The distance acquisition unit 12A acquires the distance data obtained by the millimeter wave sensor 2A measuring the distance to each object in the vehicle 30 from the millimeter wave sensor 2A. The distance acquisition unit 12A outputs the distance data acquired from the millimeter wave sensor 2A to 14.
　第１判定部１３は、車両３０が停止した場合、画像取得部１１Ａにより取得された画像データを用いて、前席３１及び後席３２に乗車している乗員を検知し、検知した各乗員の体格を判定する。第１判定部１３は、各乗員の体格判定結果に基づいて、各乗員が大人か乳幼児かを判定する。第１判定部１３は、判定結果を推定結果出力部１５へ出力する。
　例えば、実施の形態１において、大人とは、車両３０に置き去りにされても自力で車外へ出ることができる体格の乗員であり、乳幼児とは、車両３０に置き去りにされると自力で車外へ出ることが困難な体格の乗員である。なお、各乗員が大人か子供かを判定する判定基準は、上記の例に限られるものではなく、ユーザが任意に設定することができる。例えば、車外に自力で出ることができる体格の乗員であっても、チャイルドシート等の着用が義務付けられている年齢の子供の体格であれば、乳幼児と判定するように設定することもできる。 When the vehicle 30 is stopped, the first determination unit 13 detects the occupants in the front seats 31 and the rear seats 32 by using the image data acquired by the image acquisition unit 11A, and of each of the detected occupants. Determine your physique. The first determination unit 13 determines whether each occupant is an adult or an infant based on the physique determination result of each occupant. The first determination unit 13 outputs the determination result to the estimation result output unit 15.
For example, in the first embodiment, an adult is a occupant having a physique that allows him / her to go out of the vehicle by himself / herself even if he / she is left behind in the vehicle 30, and an infant is an occupant who can go out of the vehicle by himself / herself when left in the vehicle 30. It is a occupant with a physique that is difficult to get out. The criteria for determining whether each occupant is an adult or a child is not limited to the above example, and can be arbitrarily set by the user. For example, even if the occupant has a physique that allows him / her to get out of the vehicle by himself / herself, if the physique of a child is an age at which he / she is obliged to wear a child seat or the like, it can be set to be judged as an infant.
　第２判定部１４は、車両３０が停止した場合、距離取得部１２Ａにより取得された距離データを用いて、後席３２に乗車している乗員を検知し、検知した各乗員が大人か乳幼児かを判定する。第２判定部１４は、判定結果を推定結果出力部１５へ出力する。なお、第２判定部１４は、距離取得部１２Ａにより取得された距離データを用いて、トランク３６にいる乗員を検知してもよい。
　なお、図１Ａ及び図１Ｂに示された、第１判定部１３から第２判定部１４へ情報を出力する矢印については、後述する。 When the vehicle 30 is stopped, the second determination unit 14 detects the occupants in the rear seat 32 by using the distance data acquired by the distance acquisition unit 12A, and whether each of the detected occupants is an adult or an infant. To judge. The second determination unit 14 outputs the determination result to the estimation result output unit 15. The second determination unit 14 may detect an occupant in the trunk 36 by using the distance data acquired by the distance acquisition unit 12A.
The arrows for outputting information from the first determination unit 13 to the second determination unit 14 shown in FIGS. 1A and 1B will be described later.
　ここで、ミリ波センサ２Ａの距離データを用いた、大人と乳幼児の判定方法の具体例を説明する。図４Ａ、図４Ｂ、及び図４Ｃは、後席３２と第１エリア４１、第２エリア４２、及び第３エリア４３との位置関係の例を示す図である。図４Ａは、後席３２に大人３７が着座している例である。図４Ｂは、後席３２に装着された前向きタイプのＣＲＳ３３に乳幼児３５が着座している例である。図４Ｃは、後席３２に装着された後ろ向きタイプのＣＲＳ３３に乳幼児３５が着座している例である。ミリ波センサ２Ａは、後席３２の上方に設置されて前席３１の下方に向けてミリ波を送信する。第２判定部１４は、ミリ波センサ２Ａの検知範囲２ａを、ミリ波センサ２Ａの設置位置に近い第１エリア４１、第１エリア４１よりも設置位置から離れた第２エリア４２、及び第２エリア４２よりも設置位置から離れた第３エリア４３に分割する。第１エリア４１、第２エリア４２、及び第３エリア４３の距離を示す情報は、第２判定部１４が予め保持している。図示例では、第１エリア４１は、後席３２のヘッドレストを含むエリアである。第２エリア４２は、後席３２の背もたれを含むエリアであり、後席３２の座面が第２エリア４２と第３エリア４３との境界面になる。第３エリア４３は、後席３２の足元を含むエリアである。 Here, a specific example of a method for determining an adult and an infant using the distance data of the millimeter wave sensor 2A will be described. 4A, 4B, and 4C are diagrams showing an example of the positional relationship between the rear seat 32 and the first area 41, the second area 42, and the third area 43. FIG. 4A shows an example in which an adult 37 is seated in the rear seat 32. FIG. 4B shows an example in which the infant 35 is seated on the forward-facing type CRS 33 mounted on the rear seat 32. FIG. 4C shows an example in which the infant 35 is seated on the rear-facing type CRS 33 mounted on the rear seat 32. The millimeter wave sensor 2A is installed above the rear seat 32 and transmits millimeter waves toward the bottom of the front seat 31. The second determination unit 14 sets the detection range 2a of the millimeter wave sensor 2A to the first area 41 closer to the installation position of the millimeter wave sensor 2A, the second area 42 farther from the installation position than the first area 41, and the second. It is divided into a third area 43, which is farther from the installation position than the area 42. Information indicating the distances between the first area 41, the second area 42, and the third area 43 is held in advance by the second determination unit 14. In the illustrated example, the first area 41 is an area including the headrest of the rear seat 32. The second area 42 is an area including the backrest of the rear seat 32, and the seat surface of the rear seat 32 serves as a boundary surface between the second area 42 and the third area 43. The third area 43 is an area including the feet of the rear seats 32.
　第２判定部１４は、ミリ波センサ２Ａが測定した距離データを、第１エリア４１、第２エリア４２、及び第３エリア４３に分類する。第２判定部１４は、距離データが第１エリア４１、第２エリア４２、及び第３エリア４３の全てに分類された場合、つまり、ミリ波センサ２Ａが送信したミリ波が第１エリア４１、第２エリア４２、及び第３エリア４３の全てで反射した場合、後席３２に大人３７が着座していると判定する。また、第２判定部１４は、距離データが第１エリア４１及び第３エリア４３に分類されず、第２エリア４２に分類された場合、つまり、ミリ波センサ２Ａが送信したミリ波が第２エリア４２で反射した場合、後席３２に装着されたＣＲＳ３３に乳幼児３５が着座していると判定する。また、第２判定部１４は、距離データが第１エリア４１及び第２エリア４２に分類されず、第３エリア４３に分類された場合、つまり、ミリ波センサ２Ａが送信したミリ波が第３エリア４３で反射した場合、後席３２の足元に乳幼児（図示せず）がいると判定する。 The second determination unit 14 classifies the distance data measured by the millimeter wave sensor 2A into the first area 41, the second area 42, and the third area 43. In the second determination unit 14, when the distance data is classified into all of the first area 41, the second area 42, and the third area 43, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the first area 41. When the reflection is performed in all of the second area 42 and the third area 43, it is determined that the adult 37 is seated in the rear seat 32. Further, in the second determination unit 14, when the distance data is not classified into the first area 41 and the third area 43 but is classified into the second area 42, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the second. When the reflection occurs in the area 42, it is determined that the infant 35 is seated on the CRS 33 mounted on the rear seat 32. Further, in the second determination unit 14, when the distance data is not classified into the first area 41 and the second area 42 but is classified into the third area 43, that is, the millimeter wave transmitted by the millimeter wave sensor 2A is the third. When the reflection occurs in the area 43, it is determined that there is an infant (not shown) at the foot of the rear seat 32.
　判定エリアを前席３１まで拡張し、前席３１の大人と乳幼児の判定を行うこともできる。図４Ｄは、前席３１と第４エリア４４及び第５エリア４５との位置関係の例を示す図である。この図４Ｄは、前席３１の運転席側に大人３４が着座していると共に、前席３１の助手席側に装着された前向きタイプのＣＲＳ３３に乳幼児３８が着座している例である。例えば、図４Ｄでは、第３エリア４３よりもさらに前席３１側に第４エリア４４（前席３１のシート部分）と第５エリア４５（前席３１の足元部分）が設定されている。第２判定部１４は、距離データが第３エリア４３、第４エリア４４、及び第５エリア４５に分類された場合、前席３１に大人３４が着座していると判定する。また、第２判定部１４は、距離データが第４エリア４４に分類された場合、前席３１に装着されたＣＲＳ３３に乳幼児３８が着座していると判定する。また、第２判定部１４は、距離データが第５エリア４５に分類された場合、前席３１の足元に乳幼児（図示せず）がいると判定する。
　なお、図４Ｄの例では、前席３１に乳幼児３８だけでなく大人３４も着座しているため、距離データが第３エリア４３、第４エリア４４、及び第５エリア４５に分類されることとなり、大人３４が着座していると判定される。 It is also possible to extend the determination area to the front seat 31 and judge an adult and an infant in the front seat 31. FIG. 4D is a diagram showing an example of the positional relationship between the front seat 31 and the fourth area 44 and the fifth area 45. FIG. 4D shows an example in which an adult 34 is seated on the driver's seat side of the front seat 31 and an infant 38 is seated on a forward-facing type CRS 33 mounted on the passenger seat side of the front seat 31. For example, in FIG. 4D, a fourth area 44 (seat portion of the front seat 31) and a fifth area 45 (foot portion of the front seat 31) are set on the front seat 31 side of the third area 43. When the distance data is classified into the third area 43, the fourth area 44, and the fifth area 45, the second determination unit 14 determines that the adult 34 is seated in the front seat 31. Further, the second determination unit 14 determines that the infant 38 is seated on the CRS 33 mounted on the front seat 31 when the distance data is classified into the fourth area 44. Further, the second determination unit 14 determines that there is an infant (not shown) at the foot of the front seat 31 when the distance data is classified into the fifth area 45.
In the example of FIG. 4D, since not only the infant 38 but also the adult 34 is seated in the front seat 31, the distance data is classified into the third area 43, the fourth area 44, and the fifth area 45. , It is determined that the adult 34 is seated.
　図５は、実施の形態１に係る車内監視システム２０の理解を助けるための参考例であり、ミリ波センサ２Ａが後席３２の前方に設置された例を示す。参考例では、ミリ波センサ２Ａが後席３２前方の天井に設置されて後席３２に向けてミリ波を送信する。ミリ波センサ２Ａがこのように設置された場合、ミリ波センサ２Ａの検知範囲２ａは、ミリ波センサ２Ａの設置位置からの距離に応じて第１エリア４１と第２エリア４２とに分割される。そのため、後席３２に着座している乗員が乳幼児３５であるか大人３７であるか、ミリ波センサ２Ａの距離データだけでは判定が困難である。 FIG. 5 is a reference example for assisting the understanding of the in-vehicle monitoring system 20 according to the first embodiment, and shows an example in which the millimeter wave sensor 2A is installed in front of the rear seat 32. In the reference example, the millimeter wave sensor 2A is installed on the ceiling in front of the rear seat 32 and transmits millimeter waves toward the rear seat 32. When the millimeter wave sensor 2A is installed in this way, the detection range 2a of the millimeter wave sensor 2A is divided into a first area 41 and a second area 42 according to the distance from the installation position of the millimeter wave sensor 2A. .. Therefore, it is difficult to determine whether the occupant seated in the rear seat 32 is an infant 35 or an adult 37 only by the distance data of the millimeter wave sensor 2A.
　推定結果出力部１５は、第１判定部１３から判定結果を受け取ると共に、第２判定部１４から判定結果を受け取る。推定結果出力部１５は、受け取った判定結果に基づき、乳幼児の乗車を推定し、推定結果を通知部２１に出力する。推定結果出力部１５は、第１判定部１３又は第２判定部１４の少なくとも一方により乳幼児の乗車が判定された場合、乳幼児が乗車していると推定する。 The estimation result output unit 15 receives the determination result from the first determination unit 13 and also receives the determination result from the second determination unit 14. The estimation result output unit 15 estimates the riding of the infant based on the received determination result, and outputs the estimation result to the notification unit 21. When the estimation result output unit 15 determines that the infant is on board by at least one of the first determination unit 13 and the second determination unit 14, it is estimated that the infant is on board.
　通知部２１は、推定結果出力部１５によって出力された推定結果に基づき、通知を行う。例えば、通知部２１は、乳幼児が乗車しているとの推定結果が推定結果出力部１５から入力された場合、音声又は表示等によって、乳幼児が車両３０に残っていることを通知し、乳幼児が乗車していないとの推定結果が推定結果出力部１５から入力された場合は、通知を行わない。これにより、車内監視システム２０は、乗員に対して車内に乳幼児が乗車していることを通知することができるため、乳幼児の置き去りを防止できる。 The notification unit 21 notifies based on the estimation result output by the estimation result output unit 15. For example, when the estimation result that the infant is on board is input from the estimation result output unit 15, the notification unit 21 notifies that the infant remains in the vehicle 30 by voice or display, and the infant is notified. If the estimation result that the vehicle is not on board is input from the estimation result output unit 15, no notification is given. As a result, the in-vehicle monitoring system 20 can notify the occupant that the infant is in the vehicle, so that the infant can be prevented from being left behind.
　図６は、実施の形態１に係る車内監視装置１０の動作例を示すフローチャートである。車内監視装置１０は、車両３０が停止した場合（ステップＳＴ１“ＹＥＳ”）、ステップＳＴ２以降の動作を行い、車両３０が走行している場合（ステップＳＴ１“ＮＯ”）、車両３０が停止するまでステップＳＴ１の動作を繰り返す。 FIG. 6 is a flowchart showing an operation example of the in-vehicle monitoring device 10 according to the first embodiment. When the vehicle 30 is stopped (step ST1 "YES"), the in-vehicle monitoring device 10 operates after step ST2, and when the vehicle 30 is running (step ST1 "NO"), until the vehicle 30 is stopped. The operation of step ST1 is repeated.
　ステップＳＴ２において、画像取得部１１Ａは、カメラ１Ａから画像データを取得する。ステップＳＴ３において、第１判定部１３は、画像データを用いて、前席３１及び後席３２の乗員を検知すると共に検知した乗員が乳幼児であるか否かを判定する。 In step ST2, the image acquisition unit 11A acquires image data from the camera 1A. In step ST3, the first determination unit 13 detects the occupants of the front seats 31 and the rear seats 32 using the image data, and determines whether or not the detected occupants are infants.
　ステップＳＴ４において、距離取得部１２Ａは、ミリ波センサ２Ａから距離データを取得する。ステップＳＴ５において、第２判定部１４は、距離データを第１エリア４１、第２エリア４２、及び第３エリア４３に分類し、分類結果に基づいて前席３１及び後席３２の乗員が乳幼児であるか否かを判定する。
　ステップＳＴ２，ＳＴ３の動作と、ステップＳＴ４，ＳＴ５の動作は、並行して行われる。 In step ST4, the distance acquisition unit 12A acquires distance data from the millimeter wave sensor 2A. In step ST5, the second determination unit 14 classifies the distance data into the first area 41, the second area 42, and the third area 43, and the occupants of the front seat 31 and the rear seat 32 are infants based on the classification result. Determine if it exists.
The operations of steps ST2 and ST3 and the operations of steps ST4 and ST5 are performed in parallel.
　ステップＳＴ６において、推定結果出力部１５は、第１判定部１３及び第２判定部１４から受け取った判定結果に基づき、乳幼児の乗車を推定し、推定結果を出力する。 In step ST6, the estimation result output unit 15 estimates the riding of the infant based on the determination results received from the first determination unit 13 and the second determination unit 14, and outputs the estimation result.
　ステップＳＴ７において、通知部２１は、推定結果出力部１５が出力した推定結果に基づき通知を行う。そして、車内監視装置１０は、図６のフローチャートに示される動作を終了する。 In step ST7, the notification unit 21 notifies based on the estimation result output by the estimation result output unit 15. Then, the in-vehicle monitoring device 10 ends the operation shown in the flowchart of FIG.
　図７Ａ及び図７Ｂは、実施の形態１に係る車内監視装置１０のハードウェア構成例を示す図である。車内監視装置１０における画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の機能は、処理回路により実現される。即ち、車内監視装置１０は、上記機能を実現するための処理回路を備える。処理回路は、専用のハードウェアとしての処理回路１００であってもよいし、メモリ１０２に格納されるプログラムを実行するプロセッサ１０１であってもよい。 7A and 7B are diagrams showing a hardware configuration example of the in-vehicle monitoring device 10 according to the first embodiment. The functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 in the in-vehicle monitoring device 10 are realized by the processing circuit. That is, the in-vehicle monitoring device 10 includes a processing circuit for realizing the above functions. The processing circuit may be a processing circuit 100 as dedicated hardware, or a processor 101 that executes a program stored in the memory 102.
　図７Ａに示されるように、処理回路が専用のハードウェアである場合、処理回路１００は、例えば、単一回路、複合回路、プログラム化したプロセッサ、並列プログラム化したプロセッサ、ＡＳＩＣ（Ａｐｐｌｉｃａｔｉｏｎ　Ｓｐｅｃｉｆｉｃ　Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）、ＦＰＧＡ（Ｆｉｅｌｄ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ）、又はこれらを組み合わせたものが該当する。画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の機能を複数の処理回路１００で実現してもよいし、各部の機能をまとめて１つの処理回路１００で実現してもよい。 As shown in FIG. 7A, when the processing circuit is dedicated hardware, the processing circuit 100 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific Integrated Circuit). ), FPGA (Field Processor Gate Array), or a combination thereof. The functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 may be realized by a plurality of processing circuits 100, or the functions of each unit may be integrated. It may be realized by one processing circuit 100.
　図７Ｂに示されるように、処理回路がプロセッサ１０１である場合、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の機能は、ソフトウェア、ファームウェア、又はソフトウェアとファームウェアとの組み合わせにより実現される。ソフトウェア又はファームウェアはプログラムとして記述され、メモリ１０２に格納される。プロセッサ１０１は、メモリ１０２に格納されたプログラムを読みだして実行することにより、各部の機能を実現する。即ち、車内監視装置１０は、プロセッサ１０１により実行されるときに、図６のフローチャートで示されるステップが結果的に実行されることになるプログラムを格納するためのメモリ１０２を備える。また、このプログラムは、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の手順又は方法をコンピュータに実行させるものであるとも言える。 As shown in FIG. 7B, when the processing circuit is the processor 101, the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 are software. , Firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 102. The processor 101 realizes the functions of each part by reading and executing the program stored in the memory 102. That is, the in-vehicle monitoring device 10 includes a memory 102 for storing a program in which the step shown in the flowchart of FIG. 6 is eventually executed when executed by the processor 101. Further, it can be said that this program causes the computer to execute the procedure or method of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15.
　ここで、プロセッサ１０１とは、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）、処理装置、演算装置、又はマイクロプロセッサ等のことである。
　メモリ１０２は、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、ＥＰＲＯＭ（Ｅｒａｓａｂｌｅ　Ｐｒｏｇｒａｍｍａｂｌｅ　ＲＯＭ）、又はフラッシュメモリ等の不揮発性もしくは揮発性の半導体メモリであってもよいし、ハードディスク又はフレキシブルディスク等の磁気ディスクであってもよいし、ＣＤ（Ｃｏｍｐａｃｔ　Ｄｉｓｃ）又はＤＶＤ（Ｄｉｇｉｔａｌ　Ｖｅｒｓａｔｉｌｅ　Ｄｉｓｃ）等の光ディスクであってもよい。 Here, the processor 101 is a CPU (Central Processing Unit), a processing device, an arithmetic unit, a microprocessor, or the like.
The memory 102 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Program ROM), or a flash memory, and may be a non-volatile or volatile semiconductor memory such as a hard disk or a flexible disk. It may be an optical disc such as a CD (Compact Disc) or a DVD (Digital Versaille Disc).
　なお、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の機能について、一部を専用のハードウェアで実現し、一部をソフトウェア又はファームウェアで実現するようにしてもよい。このように、車内監視装置１０における処理回路は、ハードウェア、ソフトウェア、ファームウェア、又はこれらの組み合わせによって、上述の機能を実現することができる。 Some of the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 are realized by dedicated hardware, and some are software or software. It may be realized by firmware. As described above, the processing circuit in the in-vehicle monitoring device 10 can realize the above-mentioned functions by hardware, software, firmware, or a combination thereof.
　以上のように、実施の形態１に係る車内監視装置１０は、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５を備える。画像取得部１１Ａは、カメラ１Ａが車内を撮像した画像データを取得する。距離取得部１２Ａは、ミリ波センサ２Ａが車内の動く物体までの距離を測定した距離データを取得する。第１判定部１３は、車両３０が停止した場合、画像取得部１１Ａにより取得された画像データを用いて、車両３０に着座している乗員が乳幼児であるか否かを判定する。第２判定部１４は、車両３０が停止した場合、距離取得部１２Ａにより取得された距離データを用いて、車両３０に着座している乗員が乳幼児であるか否かを判定する。推定結果出力部１５は、第１判定部１３又は第２判定部１４の少なくとも一方により乳幼児の乗車が判定された場合、通知部２１に対して警告を出力するように指示する。この構成により、カメラ１Ａの短所とミリ波センサ２Ａの短所を補完することができるので、車内監視装置１０は、車内にいる乳幼児を検知することができる。 As described above, the in-vehicle monitoring device 10 according to the first embodiment includes an image acquisition unit 11A, a distance acquisition unit 12A, a first determination unit 13, a second determination unit 14, and an estimation result output unit 15. The image acquisition unit 11A acquires image data obtained by the camera 1A capturing the inside of the vehicle. The distance acquisition unit 12A acquires distance data obtained by measuring the distance to a moving object in the vehicle by the millimeter wave sensor 2A. When the vehicle 30 is stopped, the first determination unit 13 determines whether or not the occupant seated in the vehicle 30 is an infant or not by using the image data acquired by the image acquisition unit 11A. When the vehicle 30 is stopped, the second determination unit 14 determines whether or not the occupant seated in the vehicle 30 is an infant or not by using the distance data acquired by the distance acquisition unit 12A. The estimation result output unit 15 instructs the notification unit 21 to output a warning when at least one of the first determination unit 13 and the second determination unit 14 determines that the infant is on board. With this configuration, the disadvantages of the camera 1A and the disadvantages of the millimeter wave sensor 2A can be complemented, so that the in-vehicle monitoring device 10 can detect an infant in the vehicle.
　また、実施の形態１のミリ波センサ２Ａは、後席３２の上方に設けられてこの後席３２の足元に向けてミリ波を送信する。第２判定部１４は、距離取得部１２Ａにより取得された距離データを、ミリ波センサ２Ａの設置位置に近い第１エリア４１、第１エリア４１よりも設置位置から離れた第２エリア４２、及び第２エリア４２よりも設置位置から離れた第３エリア４３に分類する。そして、第２判定部１４は、距離データが第１エリア４１、第２エリア４２、及び第３エリア４３の全てに分類された場合に後席３２に大人がいると判定する。一方、第２判定部１４は、距離データが第１エリア４１に分類されず、第２エリア４２又は第３エリア４３の少なくとも一方に分類された場合に後席３２に乳幼児がいると判定する。この構成により、第２判定部１４は、ミリ波センサ２Ａが測定した距離データを用いて、大人と乳幼児とを区別できる。 Further, the millimeter wave sensor 2A of the first embodiment is provided above the rear seat 32 and transmits millimeter waves toward the feet of the rear seat 32. The second determination unit 14 uses the distance data acquired by the distance acquisition unit 12A as the first area 41 closer to the installation position of the millimeter wave sensor 2A, the second area 42 farther from the installation position than the first area 41, and the second area 42. It is classified into the third area 43, which is farther from the installation position than the second area 42. Then, the second determination unit 14 determines that there is an adult in the rear seat 32 when the distance data is classified into all of the first area 41, the second area 42, and the third area 43. On the other hand, the second determination unit 14 determines that there is an infant in the rear seat 32 when the distance data is not classified into the first area 41 but is classified into at least one of the second area 42 or the third area 43. With this configuration, the second determination unit 14 can distinguish between an adult and an infant by using the distance data measured by the millimeter wave sensor 2A.
　なお、実施の形態１の通知部２１は、乳幼児が乗車しているとの推定結果が推定結果出力部１５から入力された場合、乳幼児の乗車を通知する構成であるが、通知部２１の構成はこれに限定されない。
　例えば、推定結果出力部１５は、第１判定部１３により大人の乗車が判定され、第２判定部１４により乳幼児の乗車が判定された場合、大人及び乳幼児が乗車していると推定する。通知部２１には、乳幼児が乗車しているとの推定結果が入力されるだけでなく、大人が乗車しているとの推定結果も入力される。通知部２１は、大人が乗車しているとの推定結果が入力された場合は、乳幼児の乗車を通知しない構成であってもよい。これは、車両３０に乳幼児だけが残され場合に通知が必要であり、大人が同乗している場合は通知が不要なためである。この構成により、車内監視装置１０は、カメラ１Ａの画像データに基づいて車両３０に１人以上の大人が乗車していると判定された場合には、カメラ１Ａの死角に乳幼児がいても通知をしないことができるので、不要な通知を軽減できる。
　また、例えば、推定結果出力部１５は、第２判定部１３によりトランク３６にいる乗員が検知された場合、トランク３６に乗員がいるとの推定結果を通知部２１に出力してもよい。通知部２１は、トランク３６に乗員がいるとの推定結果が入力された場合、通知を行う。 The notification unit 21 of the first embodiment is configured to notify the ride of the infant when the estimation result that the infant is on board is input from the estimation result output unit 15, but the configuration of the notification unit 21 Is not limited to this.
For example, the estimation result output unit 15 estimates that an adult and an infant are on board when the first determination unit 13 determines that an adult is on board and the second determination unit 14 determines that an infant is on board. In the notification unit 21, not only the estimation result that the infant is on board but also the estimation result that the adult is on board is input. The notification unit 21 may be configured not to notify the ride of an infant when the estimation result that an adult is on board is input. This is because notification is required when only infants are left in the vehicle 30, and notification is not required when an adult is on board. With this configuration, when it is determined that one or more adults are in the vehicle 30 based on the image data of the camera 1A, the in-vehicle monitoring device 10 notifies even if there is an infant in the blind spot of the camera 1A. Since it can be done, unnecessary notifications can be reduced.
Further, for example, when the second determination unit 13 detects the occupant in the trunk 36, the estimation result output unit 15 may output the estimation result that the occupant is in the trunk 36 to the notification unit 21. The notification unit 21 notifies when the estimation result that there is an occupant in the trunk 36 is input.
　また、第２判定部１４は、第１判定部１３により前席３１に大人の乗車が判定された場合、第３エリア４３を無効にしてもよい。図８は、前席３１と第１エリア４１、第２エリア４２、及び第３エリア４３との位置関係を示す図である。図８に示されるように、第３エリア４３においては、後席３２の足元と前席３１とが同じ距離にある。そのため、前席３１に大人３４がいる場合、後席３２の足元に乳幼児がいないにもかかわらず、第２判定部１４は後席３２の足元に乳幼児がいると誤判定してしまう可能性がある。そこで、上述のように、第１判定部１３は、前席３１に大人３４が乗車していると判定した場合、推定結果出力部１５に加えて第２判定部１４に対してその判定結果を出力する。第２判定部１４は、第１判定部１３により前席３１に大人３４がいると判定された場合、第３エリア４３を無効にする。そして、第２判定部１４は、距離データが第１エリア４１及び第２エリア４２の両方に分類された場合に後席３２に大人がいると判定し、距離データが第１エリア４１に分類されず第２エリア４２に分類された場合に後席３２に乳幼児がいると判定する。この構成により、車内監視装置１０は、車両３０に残された乳幼児を精度よく検知できる。 Further, the second determination unit 14 may invalidate the third area 43 when the first determination unit 13 determines that an adult is on the front seat 31. FIG. 8 is a diagram showing the positional relationship between the front seat 31 and the first area 41, the second area 42, and the third area 43. As shown in FIG. 8, in the third area 43, the feet of the rear seats 32 and the front seats 31 are at the same distance. Therefore, when there is an adult 34 in the front seat 31, there is a possibility that the second determination unit 14 erroneously determines that there is an infant at the foot of the rear seat 32 even though there is no infant at the foot of the rear seat 32. is there. Therefore, as described above, when the first determination unit 13 determines that the adult 34 is in the front seat 31, the determination result is transmitted to the second determination unit 14 in addition to the estimation result output unit 15. Output. When the first determination unit 13 determines that the front seat 31 has an adult 34, the second determination unit 14 invalidates the third area 43. Then, the second determination unit 14 determines that there is an adult in the rear seat 32 when the distance data is classified into both the first area 41 and the second area 42, and the distance data is classified into the first area 41. When it is classified into the second area 42, it is determined that there is an infant in the rear seat 32. With this configuration, the in-vehicle monitoring device 10 can accurately detect the infant left in the vehicle 30.
　なお、実施の形態１に係る車内監視システム２０の用途は、車両３０が停止した場合に車両３０にいる乳幼児を検知する用途に限定されない。
　例えば、車内監視装置１０は、乗員のシートベルト着用有無を判定する機能を有してもよい。この場合、第１判定部１３及び第２判定部１４は各座席の乗員の有無を検知する。上記機能は、各座席のシートベルトセンサからの情報に基づいて、第１判定部１３及び第２判定部１４により検知された各乗員がシートベルトを着用しているか否かを判定する。推定結果出力部１５は各乗員のシートベルトの着用結果を出力し、通知部２１は推定結果出力部１５によって出力された着用結果に基づき、通知を行う。
　また、例えば、車内監視装置１０は、エアバッグの作動有無を切り替える機能を有してもよい。この場合、第１判定部１３及び第２判定部１４は各座席の乗員の有無を検知し、検知した各乗員の体格を判定する。上記機能は、第１判定部１３及び第２判定部１４により判定された各乗員の体格に応じて、各座席のエアバッグの作動有無を切り替える。 The application of the in-vehicle monitoring system 20 according to the first embodiment is not limited to the application of detecting an infant in the vehicle 30 when the vehicle 30 is stopped.
For example, the in-vehicle monitoring device 10 may have a function of determining whether or not the occupant is wearing a seatbelt. In this case, the first determination unit 13 and the second determination unit 14 detect the presence or absence of occupants in each seat. The above function determines whether or not each occupant detected by the first determination unit 13 and the second determination unit 14 is wearing a seatbelt based on the information from the seatbelt sensor of each seat. The estimation result output unit 15 outputs the wearing result of each occupant's seatbelt, and the notification unit 21 gives a notification based on the wearing result output by the estimation result output unit 15.
Further, for example, the in-vehicle monitoring device 10 may have a function of switching whether or not the airbag is activated. In this case, the first determination unit 13 and the second determination unit 14 detect the presence or absence of occupants in each seat and determine the physique of each detected occupant. The above-mentioned function switches whether or not the airbag of each seat is activated according to the physique of each occupant determined by the first determination unit 13 and the second determination unit 14.
　また、実施の形態１では、画像取得部１１Ａ、距離取得部１２Ａ、第１判定部１３、第２判定部１４、及び推定結果出力部１５の機能が、車両３０に搭載された車内監視装置１０に集約された構成であったが、全部又は一部の機能がネットワーク上のサーバ装置にあってもよい。 Further, in the first embodiment, the functions of the image acquisition unit 11A, the distance acquisition unit 12A, the first determination unit 13, the second determination unit 14, and the estimation result output unit 15 are mounted on the vehicle 30. Although the configuration was integrated into, all or part of the functions may be in the server device on the network.
　なお、本開示はその開示の範囲内において、実施の形態の任意の構成要素の変形、又は実施の形態の任意の構成要素の省略が可能である。 It should be noted that, within the scope of the disclosure, any component of the embodiment can be modified or any component of the embodiment can be omitted in the present disclosure.
　本開示に係る車内監視システムは、乳幼児の乗車を判定するようにしたので、乳幼児の置き去りを防止する車内監視システム等に用いるのに適している。 Since the in-vehicle monitoring system according to the present disclosure determines whether an infant is on board, it is suitable for use in an in-vehicle monitoring system or the like that prevents the infant from being left behind.
　１　第１センサ、１Ａ　カメラ、１ａ　撮像範囲、２　第２センサ、２Ａ　ミリ波センサ、２ａ　検知範囲、１０　車内監視装置、１１　第１データ取得部、１１Ａ　画像取得部、１２　第２データ取得部、１２Ａ　距離取得部、１３　第１判定部、１４　第２判定部、１５　推定結果出力部、２０　車内監視システム、２１　通知部、３０　車両、３１　前席、３２　後席、３３　ＣＲＳ、３４，３７　大人、３５，３８　乳幼児、３６　トランク、４１　第１エリア、４２　第２エリア、４３　第３エリア、４４　第４エリア、４５　第５エリア。 1 1st sensor, 1A camera, 1a imaging range, 2nd sensor, 2A millimeter wave sensor, 2a detection range, 10 in-vehicle monitoring device, 11 1st data acquisition unit, 11A image acquisition unit, 12 2nd data acquisition unit, 12A distance acquisition unit, 13 first judgment unit, 14 second judgment unit, 15 estimation result output unit, 20 in-vehicle monitoring system, 21 notification unit, 30 vehicles, 31 front seats, 32 rear seats, 33 CRS, 34, 37 adults , 35, 38 infants, 36 trunks, 41 1st area, 42 2nd area, 43 3rd area, 44 4th area, 45 5th area.

Claims (10)

  1. 　車両が停止した場合に乗員を検知する車内監視装置であって、
    　第１センサによって取得された第１データを取得する第１データ取得部と、
    　第２センサによって取得された第２データを取得する第２データ取得部と、
    　前記車両が停止した場合に、前記第１データ取得部によって取得された前記第１データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定する第１判定部と、
    　前記車両が停止した場合に、前記第２データ取得部によって取得された前記第２データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定する第２判定部と、
    　前記第１判定部及び前記第２判定部の判定結果に基づき、乳幼児の乗車を推定し、推定結果を出力する推定結果出力部と、
    　を備える車内監視装置。     An in-vehicle monitoring device that detects occupants when the vehicle stops.
    The first data acquisition unit that acquires the first data acquired by the first sensor, and
    A second data acquisition unit that acquires the second data acquired by the second sensor, and
    When the vehicle is stopped, the first determination unit that determines whether or not the occupant in the vehicle is an infant by using the first data acquired by the first data acquisition unit, and the first determination unit.
    When the vehicle is stopped, the second determination unit that determines whether or not the occupant in the vehicle is an infant by using the second data acquired by the second data acquisition unit, and the second determination unit.
    Based on the determination results of the first determination unit and the second determination unit, the estimation result output unit that estimates the ride of the infant and outputs the estimation result, and the estimation result output unit.
    In-vehicle monitoring device equipped with.
  2. 　前記推定結果出力部は、前記第１判定部又は前記第２判定部の少なくとも一方により乳幼児の乗車が判定された場合、乳幼児が乗車していると推定することを特徴とする請求項１記載の車内監視装置。 The first aspect of claim 1, wherein the estimation result output unit estimates that an infant is on board when the first determination unit or at least one of the second determination units determines that the infant is on board. In-vehicle monitoring device.
  3. 　前記第１判定部は、前記第１センサにより取得された前記車両停止後の前記第１データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定し、
    　前記第２判定部は、前記第２センサにより取得された前記車両停止後の前記第２データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定することを特徴とする請求項１記載の車内監視装置。     The first determination unit determines whether or not the occupant in the vehicle is an infant by using the first data after the vehicle is stopped acquired by the first sensor.
    The second determination unit is characterized in that it determines whether or not the occupant in the vehicle is an infant or not by using the second data after the vehicle is stopped acquired by the second sensor. The in-vehicle monitoring device according to claim 1.
  4. 　前記第１データは、カメラが前記車内を撮像した画像データであることを特徴とする請求項１から請求項３のうちのいずれか１項記載の車内監視装置。 The vehicle interior monitoring device according to any one of claims 1 to 3, wherein the first data is image data obtained by capturing the inside of the vehicle.
  5. 　前記第２データは、ドップラーセンサが前記車内の動く物体までの距離を測定した距離データであることを特徴とする請求項４記載の車内監視装置。 The vehicle interior monitoring device according to claim 4, wherein the second data is distance data obtained by measuring the distance to a moving object in the vehicle by the Doppler sensor.
  6. 　前記ドップラーセンサは、後席の上方に設けられて前記後席の足元に向けてミリ波を送信することを特徴とする請求項５記載の車内監視装置。 The in-vehicle monitoring device according to claim 5, wherein the Doppler sensor is provided above the rear seats and transmits millimeter waves toward the feet of the rear seats.
  7. 　前記第２判定部は、前記第２データ取得部により取得された距離データを、前記ドップラーセンサの設置位置に近い第１エリア、前記第１エリアよりも前記設置位置から離れた第２エリア、及び前記第２エリアよりも前記設置位置から離れた第３エリアに分類し、前記距離データが前記第１エリア、前記第２エリア、及び前記第３エリアの全てに分類された場合に前記後席に大人がいると判定し、前記距離データが前記第１エリアに分類されず前記第２エリア又は前記第３エリアの少なくとも一方に分類された場合に前記後席に乳幼児がいると判定することを特徴とする請求項６記載の車内監視装置。 The second determination unit uses the distance data acquired by the second data acquisition unit in a first area closer to the installation position of the Doppler sensor, a second area farther from the installation position than the first area, and a second area. When the distance data is classified into a third area farther from the installation position than the second area, and the distance data is classified into all of the first area, the second area, and the third area, the rear seat is used. It is characterized in that it is determined that there is an adult, and when the distance data is not classified into the first area but is classified into at least one of the second area or the third area, it is determined that there is an infant in the rear seat. The in-vehicle monitoring device according to claim 6.
  8. 　前記第２判定部は、前記第１判定部により前席に大人がいると判定された場合、前記第３エリアを無効にし、前記距離データが前記第１エリア及び前記第２エリアの両方に分類された場合に前記後席に大人がいると判定し、前記距離データが前記第１エリアに分類されず前記第２エリアに分類された場合に前記後席に乳幼児がいると判定することを特徴とする請求項７記載の車内監視装置。 When the first determination unit determines that there is an adult in the front seat, the second determination unit invalidates the third area and classifies the distance data into both the first area and the second area. If this is done, it is determined that there is an adult in the rear seat, and if the distance data is not classified in the first area but is classified in the second area, it is determined that there is an infant in the rear seat. The in-vehicle monitoring device according to claim 7.
  9. 　車内を撮像するカメラと、
    　前記車内の動く物体までの距離を測定するドップラーセンサと、
    　請求項５記載の車内監視装置とを備えることを特徴とする車内監視システム。     A camera that captures the inside of the car and
    A Doppler sensor that measures the distance to a moving object in the car,
    An in-vehicle monitoring system including the in-vehicle monitoring device according to claim 5.
  10. 　車両が停止した場合に乗員を検知する車内監視方法であって、
    　第１データ取得部が、第１センサによって取得された第１データを取得し、
    　第２データ取得部が、第２センサによって取得された第２データを取得し、
    　第１判定部が、前記車両が停止した場合に、前記第１データ取得部によって取得された前記第１データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定し、
    　第２判定部が、前記車両が停止した場合に、前記第２データ取得部によって取得された前記第２データを用いて、前記車両に乗車している乗員が乳幼児であるか否かを判定し、
    　推定結果出力部が、前記第１判定部及び前記第２判定部の判定結果に基づき、乳幼児の乗車を推定し、推定結果を出力する車内監視方法。     It is an in-vehicle monitoring method that detects occupants when the vehicle stops.
    The first data acquisition unit acquires the first data acquired by the first sensor, and
    The second data acquisition unit acquires the second data acquired by the second sensor, and
    When the vehicle is stopped, the first determination unit determines whether or not the occupant in the vehicle is an infant by using the first data acquired by the first data acquisition unit. ,
    When the vehicle is stopped, the second determination unit determines whether or not the occupant in the vehicle is an infant by using the second data acquired by the second data acquisition unit. ,
    An in-vehicle monitoring method in which the estimation result output unit estimates the riding of an infant based on the determination results of the first determination unit and the second determination unit, and outputs the estimation result.
PCT/JP2019/050091 2019-12-20 2019-12-20 In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method WO2021124548A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2019/050091 WO2021124548A1 (en) 2019-12-20 2019-12-20 In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method
JP2021565284A JP7154443B2 (en) 2019-12-20 2019-12-20 In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method
DE112019007988.3T DE112019007988B4 (en) 2019-12-20 2019-12-20 Vehicle interior monitoring device, vehicle interior monitoring system and vehicle interior monitoring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/050091 WO2021124548A1 (en) 2019-12-20 2019-12-20 In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method

Publications (1)

Publication Number Publication Date
WO2021124548A1 true WO2021124548A1 (en) 2021-06-24

Family

ID=76478376

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/050091 WO2021124548A1 (en) 2019-12-20 2019-12-20 In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method

Country Status (3)

Country Link
JP (1) JP7154443B2 (en)
DE (1) DE112019007988B4 (en)
WO (1) WO2021124548A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023127072A1 (en) * 2021-12-28 2023-07-06 三菱電機株式会社 Child presence detection device and child presence detection method
WO2023152936A1 (en) * 2022-02-14 2023-08-17 三菱電機株式会社 Occupant-left-unattended warning device, occupant-left-unattended warning system, and occupant-left-unattended warning method
WO2023195060A1 (en) * 2022-04-05 2023-10-12 三菱電機株式会社 Occupant detection device and occupant detection method
WO2024134815A1 (en) * 2022-12-22 2024-06-27 三菱電機株式会社 Information processing device, detection sensor, information processing system, and information processing method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016062414A (en) * 2014-09-19 2016-04-25 クラリオン株式会社 In-vehicle monitoring device and in-vehicle monitoring system
JP2019168379A (en) * 2018-03-26 2019-10-03 アイシン精機株式会社 Living body detection system
JP2019199153A (en) * 2018-05-16 2019-11-21 株式会社Ｊｖｃケンウッド Video output device, video output method and computer program

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10158415C2 (en) 2001-11-29 2003-10-02 Daimler Chrysler Ag Method for monitoring the interior of a vehicle, as well as a vehicle with at least one camera in the vehicle interior
JP2008296777A (en) 2007-05-31 2008-12-11 Toyota Motor Corp Vehicular opening/closing control device
US20180111506A1 (en) 2015-03-24 2018-04-26 Taleese Penna Vehicle safety system and method
JP2019123354A (en) 2018-01-16 2019-07-25 株式会社デンソー Occupant detection device
JP2019217830A (en) 2018-06-15 2019-12-26 アイシン精機株式会社 Occupant recognition device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016062414A (en) * 2014-09-19 2016-04-25 クラリオン株式会社 In-vehicle monitoring device and in-vehicle monitoring system
JP2019168379A (en) * 2018-03-26 2019-10-03 アイシン精機株式会社 Living body detection system
JP2019199153A (en) * 2018-05-16 2019-11-21 株式会社Ｊｖｃケンウッド Video output device, video output method and computer program

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023127072A1 (en) * 2021-12-28 2023-07-06 三菱電機株式会社 Child presence detection device and child presence detection method
WO2023152936A1 (en) * 2022-02-14 2023-08-17 三菱電機株式会社 Occupant-left-unattended warning device, occupant-left-unattended warning system, and occupant-left-unattended warning method
WO2023195060A1 (en) * 2022-04-05 2023-10-12 三菱電機株式会社 Occupant detection device and occupant detection method
JP7499997B2 (en) 2022-04-05 2024-06-14 三菱電機株式会社 Occupant detection device and occupant detection method
WO2024134815A1 (en) * 2022-12-22 2024-06-27 三菱電機株式会社 Information processing device, detection sensor, information processing system, and information processing method

Also Published As

Publication number Publication date
DE112019007988B4 (en) 2023-06-29
JPWO2021124548A1 (en) 2021-06-24
DE112019007988T5 (en) 2022-10-13
JP7154443B2 (en) 2022-10-17

Similar Documents

Publication Publication Date Title
WO2021124548A1 (en) In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method
US7607509B2 (en) Safety device for a vehicle
EP1819554B1 (en) Child seat detection system
JPWO2021124548A5 (en)
KR102338204B1 (en) Biosignal detecting device and biosignal detecting system including the same
US10894546B2 (en) Method and apparatus for in-vehicle impairment detection with driver verification
JP7399283B2 (en) Occupant detection device, child left behind warning system, passenger detection method, and child left behind warning method
US20230166737A1 (en) Cabin-inside detection device and cabin-inside detection method
JP2003522674A (en) Method and apparatus for sheet occupancy identification
KR20180024771A (en) Passenger monitoring method and system in vehicle
US20210209386A1 (en) Radar-based monitoring in a vehicle
CN117222360A (en) Multi-sensor system for cardiovascular and respiratory tracking
US20060126901A1 (en) Device for determining the age of a person by measuring pupil size
WO2021240769A1 (en) Passenger detection device and passenger detection method
US20230326324A1 (en) In-vehicle monitoring device, in-vehicle monitoring system, and in-vehicle monitoring method
US20200231111A1 (en) Vehicle footwell reflector
JP6402554B2 (en) Information output control device for vehicle
JP6597524B2 (en) Biological information recording device
JP2019137169A (en) Guidance device, guidance system, and program
US20180065505A1 (en) Method for classifying an occupant and providing the occupant classification for a safety device in a motor vehicle
WO2023195060A1 (en) Occupant detection device and occupant detection method
JP6558343B2 (en) Biosensor
JP7509157B2 (en) Driver monitoring device, driver monitoring computer program, and driver monitoring method
JP7309061B2 (en) Flood detection device and flood detection method
JP2022189451A (en) Living body detection device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19956630

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021565284

Country of ref document: JP

122 Ep: pct application non-entry in european phase

Ref document number: 19956630

Country of ref document: EP

Kind code of ref document: A1