WO2013111745A1 - Device and method for detecting remaining occupant in vehicle interior - Google Patents

Device and method for detecting remaining occupant in vehicle interior Download PDF

Info

Publication number
WO2013111745A1
WO2013111745A1 PCT/JP2013/051185 JP2013051185W WO2013111745A1 WO 2013111745 A1 WO2013111745 A1 WO 2013111745A1 JP 2013051185 W JP2013051185 W JP 2013051185W WO 2013111745 A1 WO2013111745 A1 WO 2013111745A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
occupant
reference image
state
image
Prior art date
Application number
PCT/JP2013/051185
Other languages
French (fr)
Japanese (ja)
Inventor
辻　正文
誠秀 中村
久家　伸友
智晴 中野
博文 橋口
Original Assignee
日産自動車株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日産自動車株式会社 filed Critical 日産自動車株式会社
Publication of WO2013111745A1 publication Critical patent/WO2013111745A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2/00—Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles
    • B60N2/02—Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles the seat or part thereof being movable, e.g. adjustable
    • B60N2/0224—Non-manual adjustments, e.g. with electrical operation
    • B60N2/0244—Non-manual adjustments, e.g. with electrical operation with logic circuits
    • B60N2/0276—Non-manual adjustments, e.g. with electrical operation with logic circuits reaction to emergency situations, e.g. crash
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2/00—Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles
    • B60N2/002—Seats provided with an occupancy detection means mounted therein or thereon
    • B60N2/0021—Seats provided with an occupancy detection means mounted therein or thereon characterised by the type of sensor or measurement
    • B60N2/0024—Seats provided with an occupancy detection means mounted therein or thereon characterised by the type of sensor or measurement for identifying, categorising or investigation of the occupant or object on the seat
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2210/00—Sensor types, e.g. for passenger detection systems or for controlling seats
    • B60N2210/10—Field detection presence sensors
    • B60N2210/16—Electromagnetic waves
    • B60N2210/22—Optical; Photoelectric; Lidar [Light Detection and Ranging]
    • B60N2210/24—Cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2220/00—Computerised treatment of data for controlling of seats
    • B60N2220/10—Computerised treatment of data for controlling of seats using a database
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2220/00—Computerised treatment of data for controlling of seats
    • B60N2220/20—Computerised treatment of data for controlling of seats using a deterministic algorithm

Definitions

  • the present invention relates to a vehicle interior residual occupant detection device and a vehicle interior residual occupant detection method in which a driver detects and notifies passengers remaining in the vehicle interior after getting off.
  • Patent Document 1 in order to prevent confinement in the vehicle interior of an infant or the like, when it is detected that a person is confined in the vehicle interior at the time of door lock operation, it is informed that confinement has occurred, A technique for forcibly prohibiting door locking is described. In this prior art, it is detected whether or not a person is confined in the vehicle interior based on an image captured by a camera provided in the vehicle interior.
  • the presence or absence of a passenger is determined using an image captured by a camera provided in the passenger compartment. For this reason, for example, depending on changes in the vehicle interior environment such as illuminance and temperature in the vehicle interior, an image suitable for the determination of the presence or absence of the occupant cannot be obtained, and the presence or absence of the occupant cannot be determined accurately, and accurate notification can be made. There was a risk of not being able to.
  • the present invention has been made in view of the above problems, and provides a vehicle interior residual occupant detection device and a vehicle interior residual occupant detection method that improve the accuracy with which a driver detects the presence or absence of an occupant remaining in the vehicle interior after locking.
  • the purpose is to provide.
  • a vehicle interior remaining occupant detection device in a vehicle, and includes a camera that captures an image of the vehicle interior and whether or not an occupant is present in the vehicle interior. Based on the occupant presence / absence determining means and the predetermined vehicle state signal, whether the vehicle state is a getting-off preparation state where the driver is preparing to get off, or is a locked state where the door of the vehicle is locked A first vehicle image is stored as a first reference image when the vehicle state determination unit and the vehicle state determination unit determine that the vehicle state is the getting-off preparation state.
  • the vehicle state determining means determines that the vehicle state is the locked state, and the occupant presence / absence determining means determines that an occupant is present in the vehicle interior.
  • the comparison image acquisition means for acquiring the image captured by the camera as a first comparison image, the first reference image and the first comparison image are compared, and the passenger is in the vehicle compartment in the locked state.
  • Remaining occupant determination means for determining whether or not the vehicle remains, and notification control means for informing when the remaining occupant determination means determines that an occupant remains.
  • the vehicle interior remaining passenger detection method includes a step of determining whether there is a passenger in the vehicle interior and a vehicle state based on a predetermined vehicle state signal, and the driver getting off the vehicle.
  • Determining whether the vehicle is ready to get off or being locked with the door of the vehicle locked, and when it is determined that the vehicle state is the ready to get off Imaging the vehicle interior and storing it as a reference image; and imaging the vehicle interior when it is determined that the vehicle state is the locked state and an occupant is present in the vehicle interior Obtaining a comparison image, comparing the reference image with the comparison image, determining whether or not an occupant remains in the vehicle interior in the locked state, and an occupant remaining in the vehicle interior Determined to be The case, and a step of notifying, the.
  • FIG. 1 is a diagram illustrating a configuration of a vehicle interior remaining occupant detection device according to Embodiment 1 of the present invention.
  • FIG. 2 is a diagram illustrating an example of the attachment position of the camera.
  • FIG. 3 is a flowchart showing a procedure of operation processing according to the first embodiment of the present invention.
  • FIG. 4 is a diagram illustrating an example of a reference image, a comparative image, and a determination result in the determination process for the presence or absence of a remaining occupant according to the first embodiment of the present invention.
  • FIG. 5 is a diagram showing a configuration of a vehicle interior remaining occupant detection device according to Embodiment 2 of the present invention.
  • FIG. 6 is a flowchart showing a procedure of operation processing according to the second embodiment of the present invention.
  • FIG. 7 is a diagram illustrating an example of a reference image, a comparative image, and a determination result in the determination process for the presence or absence of an occupant according to the second embodiment of the present invention.
  • FIG. 1 is a diagram showing a configuration of a vehicle interior remaining passenger detection device 1 according to Embodiment 1 of the present invention.
  • a vehicle interior remaining occupant detection device 1 of Embodiment 1 shown in FIG. 1 includes an ECU (Electronic Control Unit) 2.
  • the ECU 2 has a function of controlling the entire apparatus in an integrated manner, and controls the processing operation of the apparatus by operating according to a control program.
  • a microcomputer mainly composed of a CPU, a ROM, a RAM, and an I / O interface can be configured as a main component.
  • the ECU 2 performs various arithmetic processes, and detects and notifies passengers including pets (animals) remaining in the vehicle interior based on the results of the arithmetic processes.
  • the ECU 2 is connected to a door lock detection sensor 3 that outputs an output signal (locking signal) when the door lock of the vehicle is locked (locked).
  • the ECU 2 is connected to an unlock detection sensor 4 that outputs an output signal (unlock signal) when the door lock of the vehicle is unlocked (unlocked).
  • a shift position sensor 5 that outputs a signal (shift position signal) indicating the position of the shift lever is connected to the ECU 2.
  • the ECU 2 receives output signals from the sensors.
  • the ECU 2 also has a vehicle speed signal output from the vehicle speed sensor 6 that detects the vehicle speed, and a signal (seat belt signal) from the seat belt mounting sensor 7 that outputs an output signal when the seat belt is mounted. Is entered.
  • a signal output from at least one of the door lock detection sensor 3, the unlock detection sensor 4, the shift position sensor 5, the vehicle speed sensor 6, and the seat belt mounting sensor 7 is referred to as a vehicle state sensor. In some cases.
  • the ECU 2 is connected to an occupant position input switch 8 for inputting a position where the occupant is seated in the vehicle interior and a camera 9 installed in the vehicle interior. Also, a hazard lamp 10 and a buzzer 11 are connected to the ECU 2.
  • the occupant position input switch 8 allows the occupant to input a seating position of an occupant other than the driver by operating from the menu screen of the navigation device.
  • FIGS. 2 (a) and 2 (b) are a top view and FIG. 2 (b) is a side view
  • the camera 9 can image all seats in the vehicle interior. , Provided in the ceiling of the vehicle interior, and outputs the captured image to the ECU 2.
  • the hazard lamp 10 and the buzzer 11 function as notifying means for notifying the driver when an occupant remaining in the passenger compartment is detected.
  • the hazard lamp 10 visually alerts the user by, for example, a blinking operation, and the buzzer 11 provided in the passenger compartment alerts the user with a sound having a volume that can be recognized by a driver outside the vehicle, for example.
  • the ECU 2 includes a vehicle state determination unit 12, an occupant presence determination unit 13, a reference image storage unit 14, a reference image diagnosis unit 15, a remaining occupant determination unit 17, and a notification control unit 18.
  • the functions performed by these units are realized by specific means in which software and hardware resources included in the computer described above cooperate.
  • the vehicle state determination unit 12 functions as a vehicle state determination unit that determines the state of the vehicle based on the vehicle state signal.
  • the vehicle state determination unit 12 is in any of the unlocked state, the traveling state, the unloading preparation state, and the locked state as the vehicle state from the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals. Determine whether.
  • the vehicle state determination unit 12 determines that the vehicle is in the traveling state when the shift position is other than the parking position after the vehicle door is unlocked and the vehicle speed is not 0 km / h.
  • the vehicle state determination unit 12 determines that the driver is in a state of preparing to get off and getting off (a getting-off preparation state). The vehicle state determination unit 12 determines that the vehicle is locked when the vehicle door is locked.
  • the occupant presence / absence determination unit 13 functions as an occupant presence / absence determination unit that determines whether an occupant is present in the vehicle interior while the vehicle is traveling.
  • the occupant presence / absence determination unit 13 determines the presence / absence of an occupant other than the driver based on the seat belt wearing state based on the seat belt signal that is a vehicle state signal and the seating position of the occupant input from the occupant position input switch 8. . Since the occupant position input switch 8 is not necessarily input by the driver, the presence or absence of an occupant other than the driver may be determined only by the seat belt signal.
  • the reference image storage unit 14 stores the image captured by the camera 9 as a reference image 16 (first reference image) when the vehicle state determination unit 12 determines that the vehicle is ready to get off. Functions as image storage means.
  • the reference image diagnosis unit 15 is a first reference image diagnosis unit that diagnoses whether or not the stored image is suitable for comparison with the comparison image when the reference image 16 is stored in the reference image storage unit 14. Function. When the reference image diagnosis unit 15 determines that the stored image is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 serves as a first reference image diagnosis unit that performs re-diagnosis with a new image captured again by the camera 9. Function. *
  • the reference image diagnostic unit 15 diagnoses whether the average gray value is within a predetermined range so that the determination result is obtained as good.
  • the predetermined range is set, for example, to a value that allows accurate determination in subsequent determination processing.
  • the reference image diagnosis unit 15 outputs the reference image 16 diagnosed as being suitable as the reference image 16 to the remaining occupant determination unit 17.
  • the remaining occupant determination unit 17 compares the reference image 16 with the comparison image when the occupant presence / absence determination unit 13 determines that an occupant is present in the vehicle interior, and the occupant remains in the vehicle interior in the locked state. It functions as a remaining occupant determination means for determining whether or not there is.
  • the remaining occupant determination unit 17 functions as a comparison image acquisition unit that acquires an image captured by the camera 9 as a comparison image (first comparison image) when the vehicle state determination unit 12 determines that the vehicle is locked. .
  • the remaining occupant determination unit 17 determines whether or not an occupant other than the driver remains in the passenger compartment. Specifically, the remaining occupant determination unit 17 acquires, as a comparative image, an image captured by the camera 9 at a timing determined to be in the locked state, and performs image difference processing for each pixel between the comparative image and the reference image 16. Compare the shading difference between. As a result of the comparison, the remaining occupant determination unit 17 determines that the occupant remains in the passenger compartment if the number of pixels having a light and shade difference equal to or greater than a predetermined value is equal to or less than a predetermined value.
  • the remaining occupant determination unit 17 captures a new image captured by the camera 9 as a comparative image, Make a decision.
  • the notification control unit 18 functions as a notification unit that notifies when the remaining occupant determination unit 17 determines that an occupant remains.
  • the notification control unit 18 notifies the driver of the hazard lamp 10 or the buzzer when the hazard lamp 10 blinks or the buzzer 11 sounds. 11 is driven and controlled.
  • step S101 initialization processing for initializing timers, counters, and flags used for subsequent processing is performed.
  • step S102 the current vehicle state is determined based on the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals. In this determination, if the shift position is other than the parking position after the door is unlocked and the vehicle speed is not 0 km / h, it is determined that the vehicle is traveling. Subsequently, when the vehicle speed is 0 km / h and the shift position is the parking position, the driver determines that the vehicle is in the preparation state for getting off, and determines that the vehicle is locked when the vehicle door is locked.
  • step S103 a branching process is performed based on the vehicle state determined in step S102. If it is determined in step S102 that the vehicle state is the traveling state, the process proceeds to step S104, and if not, the process proceeds to step S108.
  • step S104 the presence or absence of a passenger other than the driver in the passenger compartment is determined.
  • the presence / absence of an occupant other than the driver is determined based on input signals from the seat belt mounting sensor 7 and the occupant position input switch 8 which are vehicle state signals.
  • step S105 a branching process is performed based on the presence or absence of an occupant other than the driver, which is the determination result in step S104. If it is determined in step S104 that there is an occupant other than the driver, the process proceeds to step S106, and if not, the process proceeds to step S107.
  • step S106 determines whether there is an occupant other than the driver in the process so far.
  • 1 is set in the occupant flag for the subsequent process, and the process returns to step S102 to determine the vehicle state.
  • step S107 determines whether there is no occupant other than the driver in the processing so far.
  • the occupant flag is set to 0 for the subsequent processing, and the process returns to step S102 to determine the vehicle state. .
  • step S108 a branch process is subsequently performed in step S108 based on the vehicle state determined in step S102. If it is determined in the previous step S102 that the vehicle state is the preparation for getting off, the process proceeds to step S109, and if it is determined that the vehicle is not in the preparation for getting off, the process proceeds to step S114.
  • step S109 branch processing is performed based on the occupant flag set in step S106 or step S107. If the occupant flag is 1, that is, if there is an occupant other than the driver, the process proceeds to step S110. If the occupant flag is 0, that is, if there is no occupant other than the driver, it is determined that it is not necessary to notify the presence of the remaining occupant. A series of processing ends.
  • step S110 an image captured by the camera 9 is taken as a reference image 16 into a processing area of a storage device provided in the microcomputer of the ECU 2.
  • a rear seat that has a high possibility of remaining will be described.
  • FIGS. 2 (a) and 2 (b) among the images of the rear seats taken by the camera 9 installed on the vehicle ceiling, the left or right side seat as shown in FIG.
  • step S111 it is diagnosed whether or not the image captured in the processing area as the reference image 16 in step S110 is suitable as the reference image 16.
  • a diagnostic method for example, when the average gray value of the entire image is too bright or too dark for a brightness range suitable for the preset reference image 16, it is diagnosed as inappropriate as the reference image 16. The other images are diagnosed as being suitable for the reference image 16.
  • step S112 branch processing is performed based on the diagnosis result in step S111. If it is determined that the image captured in the processing area in step S110 is suitable in step S111, the process proceeds to step S113. If it is diagnosed as inappropriate, the process returns to step S110, and the image is captured again. Make a diagnosis. In the first embodiment, the process is repeated until it is diagnosed as being suitable. However, if the diagnosis result is unsuitable even if it is repeated a predetermined number of times or more, the remaining determination cannot be made. The driver may be notified.
  • step S113 the image captured in the processing area in step S110 is stored as a reference image 16 for subsequent processing in a memory provided in the microcomputer of the ECU 2, and the process returns to step S102 to determine the vehicle state.
  • An example of the reference image 16 stored here is shown in FIG. Since it is determined that there is an occupant other than the driver in the processing so far, the reference image 16 is in a state where there is an occupant.
  • step S114 branch processing is performed based on the vehicle state determined in step S102.
  • the vehicle state is not the traveling state and in step S108 that the vehicle is not in the getting-off preparation state. If it is determined in step S102 that the vehicle state is locked, the driver determines that the vehicle is getting off and the vehicle is locked, and proceeds to step S115. If the vehicle state is not locked, the driver returns to step S102 to return to the vehicle state. Perform the determination.
  • step S115 branch processing is performed based on the occupant flag set in step S106 or step S107, as in step S109. If the occupant flag is 1, that is, if there is an occupant other than the driver, the process proceeds to step S116, and if the occupant flag is 0, that is, if there is no occupant other than the driver, it is determined that there is no need to notify the presence of the remaining occupant. To end the series of processing.
  • step S116 the image captured by the camera 9 is taken into the processing area as a comparison image, and the process proceeds to step S117.
  • An example of the comparative image acquired here is shown in FIGS. FIG. 4B is an example when there is no occupant, and FIG. 4C is an example when there is an occupant.
  • step S117 the comparison image captured in step S116 is compared with the reference image 16 stored in step S113.
  • a comparison method for example, the density difference for each corresponding pixel between the comparison image and the reference image 16 is calculated, and the number of pixels having a density difference equal to or greater than a predetermined value set in advance is counted.
  • step S118 branch processing is performed based on the processing result in step S117. If the count value that is the difference between the comparison image calculated in step S117 and the reference image 16 (that is, the area in the image) is equal to or smaller than a predetermined value TH_NG (for example, 3/4 of the entire image), The process proceeds to step S119. On the other hand, if it is larger than the predetermined value TH_NG, the difference between the two is too large, and for example, there is a possibility of an instantaneous change in illuminance as if it was instantaneously irradiated with a light of another vehicle. Therefore, the process returns to the previous step S116 to capture the comparison image again.
  • a predetermined value TH_NG for example, 3/4 of the entire image
  • the process is repeated until the difference between the comparison image and the reference image 16 becomes equal to or smaller than a predetermined value TH_NG. However, if it does not become the predetermined value TH_NG or less even after being repeated a predetermined number of times, the driver may be notified that the remaining occupant cannot be determined.
  • step S119 branch processing is performed based on the processing result in step S117. If the count value that is the difference between the comparison image calculated in step S117 and the reference image 16 (that is, the area in the image) is equal to or less than a predetermined value TH_R (for example, 1/8 of the entire image), the reference It is determined that there is no difference from the image 16. That is, it is estimated that there is a passenger, and the process proceeds to step S120. On the other hand, if it is larger than the predetermined value TH_R, there is a difference from the reference image 16, that is, it is estimated that there is no occupant, and it is determined that there is no need to notify the presence of the remaining occupant, and the series of processing ends. An example of the determination image is shown in FIGS. 4 (d) and 4 (e). FIG. 4D shows a case where no occupant exists, and FIG. 4E shows a case where an occupant exists.
  • a predetermined value TH_R for example, 1/8 of the entire image
  • step S120 if it is determined in step S119 that an occupant other than the driver remains in the passenger compartment, the driver is notified by blinking the hazard lamp 10 or the sound of the buzzer 11, and a series of The process ends.
  • a series of processing is terminated immediately after the presence of the remaining occupant is notified.
  • the notification operation may be continued until the remaining occupant is canceled or until the driver's intention, for example, the intention of the re-locking operation can be confirmed.
  • the time interval for acquiring the reference image 16 and the comparison image is set shorter than in the past. Yes.
  • crew in the vehicle interior after a driver gets off can be improved.
  • the occupant remaining in the passenger compartment can be detected with high accuracy, and when the occupant remains, it can be accurately notified. This increases the possibility of preventing a situation where an infant or the like is left behind and confined in the vehicle interior.
  • the state of the vehicle is determined such that the vehicle is traveling, the driver prepares to get off the vehicle, the vehicle is locked, and the vehicle is locked, and the driver gets off based on the combination of the determination results of these states.
  • the vehicle is determined to be locked.
  • a reference image diagnosis unit 15 for diagnosing whether or not the stored image is suitable for comparison with the comparison image, based on preset suitability requirements. It has. If the reference image diagnosis unit 15 diagnoses that the image stored in the reference image storage unit 14 is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 performs re-diagnosis with a new image taken again by the camera 9.
  • the reference image storage unit 14 stores, as the reference image 16, an image diagnosed as an image stored by the reference image diagnostic unit 15 suitable for comparison with a comparison image. Thereby, it is possible to eliminate erroneous determination of the presence or absence of a remaining occupant in the vehicle interior due to the use of the inappropriate reference image 16.
  • FIG. 5 is a diagram showing a configuration of the vehicle interior remaining passenger detection device 21 according to Embodiment 2 of the present invention.
  • constituent elements having the same or equivalent functions as those in FIG. 1 showing the configuration of the first embodiment are given the same reference numerals, and descriptions thereof are omitted.
  • the vehicle interior remaining passenger detection device 21 includes a new ECU 22 instead of the ECU 2 according to the first embodiment.
  • the ECU 22 receives the lock signal from the door lock detection sensor 3, the unlock signal from the unlock detection sensor 4, the shift position signal from the shift position sensor 5, and the vehicle speed signal from the vehicle speed sensor 6 as in the first embodiment.
  • a camera 9 similar to that in Embodiment 1 is connected.
  • the ECU 22 includes a new vehicle state determination unit 23 instead of the vehicle state determination unit 12 of the first embodiment, and includes a new passenger presence determination unit 24 instead of the occupant presence / absence determination unit 13 of the first embodiment.
  • the ECU 22 includes a new reference image storage unit 25 instead of the reference image storage unit 14 of the first embodiment, and includes a new remaining occupant determination unit 26 instead of the remaining occupant determination unit 17 of the first embodiment. .
  • the vehicle state determination unit 23 determines, from the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal, whether the vehicle state is an unlocked state, a travel start state, a getting-off preparation state, or a locked state.
  • the vehicle state determination unit 23 determines that the vehicle is unlocked according to the unlock signal, and if the counters and flags provided in the microcomputer constituting the ECU have been initialized, the occupant including the driver is It is determined that The vehicle state determination unit 23 determines that the vehicle is in the travel start state when the shift position is other than the parking position after the door lock is released and the vehicle speed is no longer 0 km / h.
  • the vehicle state determination unit 23 determines that the driver is in a state of preparing to get off and getting off (preparation for getting off).
  • the vehicle state determination unit 23 determines that the vehicle is locked when the vehicle door is locked.
  • the vehicle state determination unit 23 determines that the vehicle is locked, the driver gets off and the vehicle is locked. It is determined that
  • the occupant presence / absence determination unit 24 determines whether there is an occupant other than the driver in the passenger compartment when the vehicle state determination unit 23 determines that the vehicle has been unlocked and travel has started. In other words, the occupant presence / absence determination unit 24 compares the image captured by the camera 9 when it is determined to be unlocked with the image captured by the camera 9 when it is determined that traveling has started. Then, it is determined whether there are passengers other than the driver in the passenger compartment.
  • the passenger presence / absence determination unit 24 acquires, as a comparative image (second comparative image), an image captured by the camera 9 at a timing when it is determined that the vehicle has started running after the passenger has boarded the vehicle.
  • the occupant presence / absence determination unit 24 compares the acquired comparison image and the reference image 16 (second reference image) of the image captured by the camera 9 when the occupant including the driver is determined to be unlocked before boarding.
  • the difference between the images is compared by gradation difference processing for each pixel. As a result of the comparison, if there is a difference between the comparison image and the reference image 16 when there is no occupant, it is estimated that there is an occupant and it is determined that there is an occupant other than the driver.
  • the occupant presence / absence determination unit 24 calculates a difference in shade of corresponding pixels between the reference image 16 and a comparison image captured by the camera.
  • the occupant presence / absence determination unit 24 determines that the new image captured by the camera 9 is in an unlocked state when the number of pixels having a difference in density equal to or greater than a predetermined value is greater than a predetermined value. When this is done, it is compared with the image captured by the camera 9 and re-determined.
  • the reference image storage unit 25 determines an image captured by the camera 9 when it is determined that the vehicle state determination unit 23 is preparing to get off and the passenger presence determination unit 24 determines that there is an occupant other than the driver. It functions as a first reference image storage unit that stores the reference image 16 (first reference image), and when the vehicle state determination unit 23 determines that the vehicle is unlocked, the image captured by the camera 9 is used as the reference image 16. It functions as second reference image storage means for storing (second reference image).
  • the reference image diagnosis unit 15 is configured to store the reference image 16 in the reference image storage unit 25 when the vehicle state determination unit 23 determines that the vehicle is in the travel start state.
  • the reference image diagnosis unit 15 functions as a second reference image diagnostic means for diagnosing whether or not the stored image is suitable for comparison with the comparative image.
  • the reference image diagnosis unit 15 determines that the stored image is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 serves as a second reference image diagnosis unit that performs re-diagnosis with a new image captured again by the camera 9. Function.
  • FIG. 6 the same reference numerals are given to the processing operations equivalent to those in FIG. 3 showing the processing operations of the first embodiment, and the description thereof is omitted.
  • step S201 after performing the process of step S101 shown in FIG. 3, in step S201, based on the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals, Make a decision.
  • the unlocked state is determined based on the unlock signal
  • the travel start state is determined when the shift position is other than the parking position and the vehicle speed is no longer 0 km / h after unlocking.
  • the vehicle speed becomes 0 km / h and the shift position becomes the parking position, it is determined that the vehicle is in the preparation state for getting off, and the locked state is determined based on the locking signal.
  • step S202 branch processing is performed based on the vehicle state determined in step S201. If it is determined in step S201 that the vehicle state is an unlocked state after initialization, it is determined that an occupant including a driver is not in the vehicle, and the process proceeds to step S203. The process proceeds to S207.
  • step S203 the image captured by the camera 9 is taken into the processing area as the reference image 16, and the process proceeds to step S204.
  • step S204 a diagnosis is made as to whether or not the image captured in the processing area as the reference image 16 in step S203 is suitable as the reference image 16.
  • a diagnostic method for example, when the average gray value of the entire image is too bright or too dark for a brightness range suitable for the preset reference image 16, it is diagnosed as inappropriate as the reference image 16. The other images are diagnosed as being suitable for the reference image 16.
  • step S205 branch processing is performed based on the diagnosis result in step S204. If it is determined in step S204 that the image captured in the processing area is suitable in step S204, the process proceeds to step S206. If it is diagnosed as inappropriate, the process returns to step S203, and the image is captured again. Make a diagnosis. In the second embodiment, the process is repeated until it is diagnosed as being suitable. However, if the diagnosis result is unsuitable even if it is repeated a predetermined number of times or more, it is impossible to perform the remaining determination. The driver may be notified.
  • step S206 the image captured in the processing area in step S203 is stored in the memory provided in the microcomputer of the ECU 2 as a reference image 16 for subsequent processing, and the process returns to step S201 to determine the vehicle state.
  • An example of the reference image 16 stored here is shown in FIG. Since the state shown in FIG. 7A is when the vehicle is unlocked, the reference image 16 has no occupant.
  • step S207 branch processing is performed based on the vehicle state determined in step S201.
  • step S202 it is determined in step S202 that the vehicle state is not an unlocked state, and if it is determined in step S201 that the vehicle state is a travel start state, the process proceeds to step S208 and must be in a travel start state. If so, the process proceeds to step S108.
  • step S208 the image captured by the camera 9 is taken into the processing area as a comparative image, and the process proceeds to step S209.
  • An example of the comparative image acquired here is shown in FIGS. 7B and 7C.
  • FIG. 7B shows a case where there is no passenger
  • FIG. 7C shows a case where there is a passenger.
  • step S209 the comparison image captured in step S208 is compared with the reference image 16 stored in step S206.
  • a comparison method for example, the density difference for each corresponding pixel between the comparison image and the reference image 16 is calculated, and the number of pixels having a density difference equal to or greater than a predetermined value set in advance is counted.
  • step S210 branch processing is performed based on the processing result of step S209. If the count value that is the difference between the comparison image calculated in step S209 and the reference image 16 (that is, the area in the image) is equal to or smaller than a predetermined value TH_NG (for example, 3/4 of the entire image), The process proceeds to step S211. On the other hand, if it is larger than the predetermined value TH_NG, the difference between the two is too large, and for example, there is a possibility of an instantaneous change in illuminance as if it was instantaneously irradiated with a light of another vehicle. Therefore, the process returns to the previous step S208 to capture the comparison image again.
  • a predetermined value TH_NG for example, 3/4 of the entire image
  • the process is repeated until the difference between the comparison image and the reference image 16 becomes equal to or smaller than a predetermined value TH_NG. However, if it does not become the predetermined value TH_NG or less even after being repeated a predetermined number of times, the driver may be notified that the remaining occupant cannot be determined.
  • step S211 branch processing is performed based on the processing result of step S209. If the count value (that is, the area in the image) that is the difference between the comparison image calculated in step S209 and the reference image 16 is equal to or less than a predetermined value TH_R (for example, 1/8 of the entire image), the reference It is determined that there is no difference from the image 16. That is, it is estimated that no passenger is present, and the process proceeds to step S212. On the other hand, if it is larger than the predetermined value TH_R, it is estimated that there is a difference from the reference image 16, that is, there is an occupant, and the process proceeds to step S213. An example of the determination image is shown in FIGS. 7D and 7E. FIG.
  • FIG. 7D shows a case where there is no passenger
  • FIG. 7E shows a case where there is a passenger.
  • step S212 it is determined that there is no occupant other than the driver in the processing so far, so the occupant flag is set to 0 for the subsequent processing, and the process returns to step S201 to determine the vehicle state.
  • step S213 since it is determined in step S213 that there is an occupant other than the driver in the process so far, 1 is set in the occupant flag for the subsequent process, and the process returns to step S201 to determine the vehicle state.
  • step S201 the vehicle state is determined in step S201.
  • the same processing as the series of steps S108 to S120 shown in the flowchart of FIG. 3 is executed. To do.
  • this series of processing it is determined whether there is a remaining passenger in the vehicle interior as in the first embodiment, and if it is determined that there is a passenger remaining in the vehicle interior, a notification to that effect is given. .
  • the reference image storage unit 25 stores the image captured by the camera 9 as the reference image 16 when the vehicle state determination unit 23 determines that the occupant including the driver is before getting on the vehicle.
  • the occupant presence / absence determination unit 24 compares the reference image 16 stored in the reference image storage unit 25 with a comparative image that is an image captured by the camera 9 after the occupant gets into the vehicle, and occupants other than the driver are placed in the passenger compartment. It is determined whether or not exists. This makes it possible to determine the presence or absence of an occupant on the basis of the comparison result between the reference image 16 and the comparison image in the same manner as the determination of the presence or absence of a remaining occupant. The presence or absence can be determined. As a result, it is possible to share part of the processing for determining the presence / absence of the occupant and the presence / absence of the remaining occupant, and the presence / absence of the occupant can be determined without increasing the size of the apparatus.
  • the occupant presence / absence determination unit 24 calculates the density difference between the corresponding pixels of the reference image 16 and the comparison image.
  • the new image is re-determined as a comparison image.
  • an appropriate image can be used as a comparative image even for an instantaneous illuminance change such as when a light of another vehicle is instantaneously illuminated, and an erroneous determination of the presence or absence of an occupant can be eliminated.
  • the unlocked state in which the vehicle is unlocked As the vehicle state, the unlocked state in which the vehicle is unlocked, the state in which the vehicle has started traveling, the preparation state for getting off the vehicle where the driver prepares to get off the vehicle, and the locked state in which the vehicle is locked are determined. Based on the combination of the determination results of these states, the state before the passenger including the driver gets on the vehicle and the state where the driver gets off and the vehicle is locked are determined. Thereby, the various processes mentioned above performed according to the state of a vehicle can be implemented exactly.
  • the vehicle state determination unit 23 When it is determined by the vehicle state determination unit 23 that the vehicle has been unlocked and traveling has started, and when it has been determined that the vehicle has been unlocked and the image captured by the camera 9 has been determined that traveling has started. In addition, it is determined whether or not an occupant other than the driver is present in the vehicle interior by comparing the image captured by the camera 9. Thereby, the passenger
  • each predetermined value can be obtained by, for example, an experiment using a real machine or a desk so that a good determination result can be obtained without causing erroneous determination in a determination process using the predetermined value. It can be determined in advance according to the result of simulation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

An image captured by a camera (9) when an occupant other than a driver is present in the vehicle interior and a vehicle status determination unit (12) has determined that the driver has prepared to get off and is getting off the vehicle, is used as a reference image. An image captured by the camera (9) when the vehicle status determination unit (12) has determined that the driver has gotten off and locked the vehicle is used as a comparison image. A determination is made as to whether or not the occupant remains in the vehicle interior on the basis of the results of a comparison between the reference image and the comparison image by a remaining occupant determination unit (17).

Description

車室内残存乗員検出装置及び車室内残存乗員検出方法Vehicle interior remaining passenger detection device and vehicle interior residual passenger detection method
　本発明は、ドライバが降車後に車室内に残存する乗員を検出して報知する車室内残存乗員検出装置及び車室内残存乗員検出方法に関する。 The present invention relates to a vehicle interior residual occupant detection device and a vehicle interior residual occupant detection method in which a driver detects and notifies passengers remaining in the vehicle interior after getting off.
　従来、この種の技術としては、例えば以下に示す文献に記載されたものが知られている（特許文献１参照）。この文献には、幼児などの車室内への閉じ込めを防止するために、ドアロック操作時に車室内に人を閉じ込めたことが検知された場合に、閉じ込めが発生していることを報知したり、強制的にドアロックを禁止する技術が記載されている。この従来技術では、車室内に設けられたカメラで撮像された画像によって車室内に人を閉じ込めたか否かを検知している。 Conventionally, as this type of technology, for example, those described in the following documents are known (see Patent Document 1). In this document, in order to prevent confinement in the vehicle interior of an infant or the like, when it is detected that a person is confined in the vehicle interior at the time of door lock operation, it is informed that confinement has occurred, A technique for forcibly prohibiting door locking is described. In this prior art, it is detected whether or not a person is confined in the vehicle interior based on an image captured by a camera provided in the vehicle interior.
特開２００４－３４６７０８号公報JP 2004-346708 A
　上記従来の技術においては、車室内に設けられたカメラで撮像された画像を用いて乗員の有無を判断している。このため、例えば車室内の照度や温度等の車室内環境の変化によっては乗員の有無の判断に適した画像が得られず、正確に乗員の有無を判断できず、的確に報知を行うことができないおそれがあった。 In the above conventional technique, the presence or absence of a passenger is determined using an image captured by a camera provided in the passenger compartment. For this reason, for example, depending on changes in the vehicle interior environment such as illuminance and temperature in the vehicle interior, an image suitable for the determination of the presence or absence of the occupant cannot be obtained, and the presence or absence of the occupant cannot be determined accurately, and accurate notification can be made. There was a risk of not being able to.
　本発明は、上記の課題を鑑みてなされたものであり、ドライバが降車施錠後に車室内に残存する乗員の有無を検出する精度を向上した車室内残存乗員検出装置及び車室内残存乗員検出方法を提供することを目的とする。 The present invention has been made in view of the above problems, and provides a vehicle interior residual occupant detection device and a vehicle interior residual occupant detection method that improve the accuracy with which a driver detects the presence or absence of an occupant remaining in the vehicle interior after locking. The purpose is to provide.
　上記目的を達成するために、本発明の一態様に係る車室内残存乗員検出装置は、車両に設けられて、車室内を撮像するカメラと、前記車室内に乗員が存在しているか否かを判定する乗員有無判定手段と、所定の車両状態信号に基づいて、車両状態が、ドライバが降車を準備している降車準備状態であるか、又は前記車両のドアをロックした施錠状態であるかを判定する車両状態判定手段と、前記車両状態判定手段により、前記車両状態が前記降車準備状態であると判定されたときに、前記カメラにより撮像された画像を第１の基準画像として記憶する第１の基準画像記憶手段と、前記車両状態判定手段により前記車両状態が前記施錠状態であると判定され、かつ、前記乗員有無判定手段により前記車室内に乗員が存在していると判定されたとき、前記カメラにより撮像された画像を第１の比較画像として取得する比較画像取得手段と、前記第１の基準画像と前記第１の比較画像とを比較し、前記施錠状態において前記車室内に乗員が残存しているか否かを判定する残存乗員判定手段と、前記残存乗員判定手段により乗員が残存していると判定された場合に、報知する報知制御手段と、を備える。
また、本発明の一態様に係る車室内残存乗員検出方法は、車室内に乗員が存在しているか否かを判定する工程と、所定の車両状態信号に基づいて、車両状態が、ドライバが降車を準備している降車準備状態であるか、又は前記車両のドアをロックした施錠状態であるかを判定する工程と、前記車両状態が前記降車準備状態であると判定されたときに、前記車室内を撮像して基準画像として記憶する工程と、前記車両状態が前記施錠状態であると判定され、かつ、前記車室内に乗員が存在していると判定されたときに、前記車室内を撮像して比較画像として取得する工程、前記基準画像と前記比較画像とを比較し、前記施錠状態において前記車室内に乗員が残存しているか否かを判定する工程と、前記車室内に乗員が残存していると判定された場合に、報知する工程と、を備える。 In order to achieve the above object, a vehicle interior remaining occupant detection device according to an aspect of the present invention is provided in a vehicle, and includes a camera that captures an image of the vehicle interior and whether or not an occupant is present in the vehicle interior. Based on the occupant presence / absence determining means and the predetermined vehicle state signal, whether the vehicle state is a getting-off preparation state where the driver is preparing to get off, or is a locked state where the door of the vehicle is locked A first vehicle image is stored as a first reference image when the vehicle state determination unit and the vehicle state determination unit determine that the vehicle state is the getting-off preparation state. And the vehicle state determining means determines that the vehicle state is the locked state, and the occupant presence / absence determining means determines that an occupant is present in the vehicle interior. The comparison image acquisition means for acquiring the image captured by the camera as a first comparison image, the first reference image and the first comparison image are compared, and the passenger is in the vehicle compartment in the locked state. Remaining occupant determination means for determining whether or not the vehicle remains, and notification control means for informing when the remaining occupant determination means determines that an occupant remains.
Further, the vehicle interior remaining passenger detection method according to one aspect of the present invention includes a step of determining whether there is a passenger in the vehicle interior and a vehicle state based on a predetermined vehicle state signal, and the driver getting off the vehicle. Determining whether the vehicle is ready to get off or being locked with the door of the vehicle locked, and when it is determined that the vehicle state is the ready to get off Imaging the vehicle interior and storing it as a reference image; and imaging the vehicle interior when it is determined that the vehicle state is the locked state and an occupant is present in the vehicle interior Obtaining a comparison image, comparing the reference image with the comparison image, determining whether or not an occupant remains in the vehicle interior in the locked state, and an occupant remaining in the vehicle interior Determined to be The case, and a step of notifying, the.
図１は、本発明の実施形態１に係る車室内残存乗員検出装置の構成を示す図である。FIG. 1 is a diagram illustrating a configuration of a vehicle interior remaining occupant detection device according to Embodiment 1 of the present invention. 図２は、カメラの取り付け位置の一例を示す図である。FIG. 2 is a diagram illustrating an example of the attachment position of the camera. 図３は、本発明の実施形態１に係る動作処理の手順を示すフローチャートである。FIG. 3 is a flowchart showing a procedure of operation processing according to the first embodiment of the present invention. 図４は、本発明の実施形態１に係る残存乗員の有無の判定処理における、基準画像と比較画像ならびに判定結果の一例を示す図である。FIG. 4 is a diagram illustrating an example of a reference image, a comparative image, and a determination result in the determination process for the presence or absence of a remaining occupant according to the first embodiment of the present invention. 図５は、本発明の実施形態２に係る車室内残存乗員検出装置の構成を示す図である。FIG. 5 is a diagram showing a configuration of a vehicle interior remaining occupant detection device according to Embodiment 2 of the present invention. 図６は、本発明の実施形態２に係る動作処理の手順を示すフローチャートである。FIG. 6 is a flowchart showing a procedure of operation processing according to the second embodiment of the present invention. 図７は、本発明の実施形態２に係る乗員の有無の判定処理における、基準画像と比較画像ならびに判定結果の一例を示す図である。FIG. 7 is a diagram illustrating an example of a reference image, a comparative image, and a determination result in the determination process for the presence or absence of an occupant according to the second embodiment of the present invention.
　以下、図面を用いて本発明を実施するための実施形態を説明する。
（実施形態１）
　図１は本発明の実施形態１に係る車室内残存乗員検出装置１の構成を示す図である。図１に示す実施形態１の車室内残存乗員検出装置１は、ＥＣＵ（Electronic Control Unit）２を備えている。このＥＣＵ２は、装置全体を統合的に制御する機能を担っており、制御プログラムに従って動作することにより、装置の処理動作を制御する。ＥＣＵ２としては、ＣＰＵ、ＲＯＭ、ＲＡＭ、Ｉ／Ｏインターフェースを主体に構成されたマイクロコンピュータを主要構成部品として構成することができる。このＥＣＵ２は、各種の演算処理を行い、この演算処理の結果に基づいて車室内に残存するペット（動物）などを含む乗員を検出して報知する。 Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a vehicle interior remaining passenger detection device 1 according to Embodiment 1 of the present invention. A vehicle interior remaining occupant detection device 1 of Embodiment 1 shown in FIG. 1 includes an ECU (Electronic Control Unit) 2. The ECU 2 has a function of controlling the entire apparatus in an integrated manner, and controls the processing operation of the apparatus by operating according to a control program. As the ECU 2, a microcomputer mainly composed of a CPU, a ROM, a RAM, and an I / O interface can be configured as a main component. The ECU 2 performs various arithmetic processes, and detects and notifies passengers including pets (animals) remaining in the vehicle interior based on the results of the arithmetic processes.
　このＥＣＵ２には、車両のドアロックが施錠状態（ロック状態）である場合に出力信号（施錠信号）を出力するドアロック検知センサ３が接続されている。ＥＣＵ２には、車両のドアロックが解錠状態（アンロック状態）である場合に出力信号（解錠信号）を出力するアンロック検知センサ４が接続されている。ＥＣＵ２には、シフトレバーの位置を示す信号（シフトポジション信号）を出力するシフトポジションセンサ５が接続されている。ＥＣＵ２には、上記各センサからの出力信号が入力される。また、ＥＣＵ２には、車両の速度を検出する車速センサ６から出力される車速信号、ならびにシートベルトが装着されている場合に出力信号を出力するシートベルト装着センサ７からの信号（シートベルト信号）が入力される。なお、以下にて、ドアロック検知センサ３、アンロック検知センサ４、シフトポジションセンサ５、車速センサ６、シートベルト装着センサ７の少なくとも１つ以上のセンサから出力される信号を車両状態センサと称する場合もある。 The ECU 2 is connected to a door lock detection sensor 3 that outputs an output signal (locking signal) when the door lock of the vehicle is locked (locked). The ECU 2 is connected to an unlock detection sensor 4 that outputs an output signal (unlock signal) when the door lock of the vehicle is unlocked (unlocked). A shift position sensor 5 that outputs a signal (shift position signal) indicating the position of the shift lever is connected to the ECU 2. The ECU 2 receives output signals from the sensors. The ECU 2 also has a vehicle speed signal output from the vehicle speed sensor 6 that detects the vehicle speed, and a signal (seat belt signal) from the seat belt mounting sensor 7 that outputs an output signal when the seat belt is mounted. Is entered. Hereinafter, a signal output from at least one of the door lock detection sensor 3, the unlock detection sensor 4, the shift position sensor 5, the vehicle speed sensor 6, and the seat belt mounting sensor 7 is referred to as a vehicle state sensor. In some cases.
　さらに、ＥＣＵ２には、車室内で乗員が着座した位置を入力する乗員位置入力スイッチ８ならびに車室内に設置されたカメラ９が接続されている。また、ＥＣＵ２には、ハザードランプ１０およびブザー１１が接続されている。 Further, the ECU 2 is connected to an occupant position input switch 8 for inputting a position where the occupant is seated in the vehicle interior and a camera 9 installed in the vehicle interior. Also, a hazard lamp 10 and a buzzer 11 are connected to the ECU 2.
　乗員位置入力スイッチ８は、乗員がナビゲーション装置のメニュー画面から操作することで、ドライバ以外の乗員の着座位置を入力できるようになっている。 The occupant position input switch 8 allows the occupant to input a seating position of an occupant other than the driver by operating from the menu screen of the navigation device.
　カメラ９は、図２（ａ），２（ｂ）（図２（ａ）は上面図、図２（ｂ）は側面図）に示すように、車室内の全座席を撮像可能となるように、車室内の天井部分に設けられ、撮像した画像をＥＣＵ２に出力する。 As shown in FIGS. 2 (a) and 2 (b) (FIG. 2 (a) is a top view and FIG. 2 (b) is a side view), the camera 9 can image all seats in the vehicle interior. , Provided in the ceiling of the vehicle interior, and outputs the captured image to the ECU 2.
　ハザードランプ１０ならびにブザー１１は、車室内に残存する乗員が検出されたときにドライバにその旨報知する報知手段として機能する。ハザードランプ１０は、例えば点滅動作により視覚的に警報報知し、車室内に設けられたブザー１１は、例えば車外のドライバが認識できる程度の音量の音により警報報知する。 The hazard lamp 10 and the buzzer 11 function as notifying means for notifying the driver when an occupant remaining in the passenger compartment is detected. The hazard lamp 10 visually alerts the user by, for example, a blinking operation, and the buzzer 11 provided in the passenger compartment alerts the user with a sound having a volume that can be recognized by a driver outside the vehicle, for example.
　ＥＣＵ２は、車両状態判定部１２、乗員有無判定部１３、基準画像記憶部１４、基準画像診断部１５、残存乗員判定部１７および報知制御部１８とを有している。これらの諸部が担っている機能は、上述したコンピュータが備えるソフトウェアとハードウェア資源とが協働した具体的な手段によって実現される。 The ECU 2 includes a vehicle state determination unit 12, an occupant presence determination unit 13, a reference image storage unit 14, a reference image diagnosis unit 15, a remaining occupant determination unit 17, and a notification control unit 18. The functions performed by these units are realized by specific means in which software and hardware resources included in the computer described above cooperate.
　車両状態判定部１２は、車両状態信号に基づいて車両の状態を判定する車両状態判定手段として機能する。車両状態判定部１２は、車両状態信号である施錠信号、解錠信号、シフトポジション信号、車速信号から、車両状態として、解錠状態、走行状態、降車準備状態、施錠状態のいずれの状態であるかを判定する。車両状態判定部１２は、車両のドアがロック解除された後にシフトポジションがパーキング位置以外であり、車速が０ｋｍ／ｈでない場合には、車両は走行状態であると判定する。続いて、車両状態判定部１２は、車速が０ｋｍ／ｈとなり、シフトポジションがパーキング位置となった場合には、ドライバは降車を準備して降車する状態（降車準備状態）であると判定する。車両状態判定部１２は、車両のドアがロックされると施錠状態と判定する。 The vehicle state determination unit 12 functions as a vehicle state determination unit that determines the state of the vehicle based on the vehicle state signal. The vehicle state determination unit 12 is in any of the unlocked state, the traveling state, the unloading preparation state, and the locked state as the vehicle state from the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals. Determine whether. The vehicle state determination unit 12 determines that the vehicle is in the traveling state when the shift position is other than the parking position after the vehicle door is unlocked and the vehicle speed is not 0 km / h. Subsequently, when the vehicle speed becomes 0 km / h and the shift position becomes the parking position, the vehicle state determination unit 12 determines that the driver is in a state of preparing to get off and getting off (a getting-off preparation state). The vehicle state determination unit 12 determines that the vehicle is locked when the vehicle door is locked.
　乗員有無判定部１３は、車両の走行中に、車室内に乗員が存在しているか否かを判定する乗員有無判定手段として機能する。乗員有無判定部１３は、車両状態信号であるシートベルト信号に基づくシートベルトの装着状況、ならびに乗員位置入力スイッチ８から入力された乗員の着座位置に基づいて、ドライバ以外の乗員の有無を判定する。なお、この乗員位置入力スイッチ８は、ドライバによって必ずしも入力されるとは限らないために、ドライバ以外の乗員の有無は、シートベルト信号のみによって、判定される場合もある。 The occupant presence / absence determination unit 13 functions as an occupant presence / absence determination unit that determines whether an occupant is present in the vehicle interior while the vehicle is traveling. The occupant presence / absence determination unit 13 determines the presence / absence of an occupant other than the driver based on the seat belt wearing state based on the seat belt signal that is a vehicle state signal and the seating position of the occupant input from the occupant position input switch 8. . Since the occupant position input switch 8 is not necessarily input by the driver, the presence or absence of an occupant other than the driver may be determined only by the seat belt signal.
　基準画像記憶部１４は、車両状態判定部１２により、降車準備状態であると判定されたとき、カメラ９により撮像された画像を基準画像１６（第１の基準画像）として記憶する第１の基準画像記憶手段として機能する。 The reference image storage unit 14 stores the image captured by the camera 9 as a reference image 16 (first reference image) when the vehicle state determination unit 12 determines that the vehicle is ready to get off. Functions as image storage means.
　基準画像診断部１５は、基準画像記憶部１４に基準画像１６が記憶される際に、記憶する画像が比較画像と比較するのに適しているか否かを診断する第１の基準画像診断手段として機能する。基準画像診断部１５は、記憶する画像が比較画像と比較するのに適していないものと判定した場合には、カメラ９により再度撮像した新たな画像で再診断する第１の基準画像診断手段として機能する。　 The reference image diagnosis unit 15 is a first reference image diagnosis unit that diagnoses whether or not the stored image is suitable for comparison with the comparison image when the reference image 16 is stored in the reference image storage unit 14. Function. When the reference image diagnosis unit 15 determines that the stored image is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 serves as a first reference image diagnosis unit that performs re-diagnosis with a new image captured again by the camera 9. Function. *
　具体的には、例えば画面全体の平均濃淡値が明るすぎたり暗すぎたりした場合には不適と診断し、再度画像を取り直す。すなわち、基準画像診断部１５は、良好は判定結果得られる程度に平均濃淡値が所定の範囲内に収まる明るさであるか否かを診断する。ここで、所定の範囲とは、例えば以降の判定処理で正確な判定ができる程度の値に設定される。基準画像診断部１５は、基準画像１６として適しているものと診断した基準画像１６を残存乗員判定部１７に出力する。 Specifically, for example, if the average gray value of the entire screen is too bright or too dark, it is diagnosed as inappropriate and the image is taken again. That is, the reference image diagnostic unit 15 diagnoses whether the average gray value is within a predetermined range so that the determination result is obtained as good. Here, the predetermined range is set, for example, to a value that allows accurate determination in subsequent determination processing. The reference image diagnosis unit 15 outputs the reference image 16 diagnosed as being suitable as the reference image 16 to the remaining occupant determination unit 17.
　残存乗員判定部１７は、乗員有無判定部１３により車室内に乗員が存在していると判定した場合に、基準画像１６と比較画像とを比較し、施錠状態において車室内に乗員が残存しているか否かを判定する残存乗員判定手段として機能する。また、残存乗員判定部１７は、車両状態判定部１２により施錠状態と判定されたとき、カメラ９により撮像された画像を比較画像（第１の比較画像）として取得する比較画像取得手段として機能する。 The remaining occupant determination unit 17 compares the reference image 16 with the comparison image when the occupant presence / absence determination unit 13 determines that an occupant is present in the vehicle interior, and the occupant remains in the vehicle interior in the locked state. It functions as a remaining occupant determination means for determining whether or not there is. The remaining occupant determination unit 17 functions as a comparison image acquisition unit that acquires an image captured by the camera 9 as a comparison image (first comparison image) when the vehicle state determination unit 12 determines that the vehicle is locked. .
　残存乗員判定部１７は、車両状態判定部１２で車両が施錠状態と判定された場合に、ドライバ以外の乗員が車室内に残っているか否かの判定を行う。具体的には、残存乗員判定部１７は、施錠状態と判定されたタイミングでカメラ９で撮像
した画像を比較画像として取得し、この比較画像と基準画像１６との画素毎の濃淡差分処理により画像間の濃淡差を比較する。残存乗員判定部１７は、比較の結果、所定値以上の濃淡差がある画素数が予め設定された所定値以下であれば、乗員が車室内に残存しているものと判定する。残存乗員判定部１７は、比較の結果、所定値以上の濃淡差がある画素数が予め設定された所定値より大きい場合には、カメラ９により撮像された新たな画像を比較画像として取り込み、再判定を行う。 When the vehicle state determination unit 12 determines that the vehicle is in a locked state, the remaining occupant determination unit 17 determines whether or not an occupant other than the driver remains in the passenger compartment. Specifically, the remaining occupant determination unit 17 acquires, as a comparative image, an image captured by the camera 9 at a timing determined to be in the locked state, and performs image difference processing for each pixel between the comparative image and the reference image 16. Compare the shading difference between. As a result of the comparison, the remaining occupant determination unit 17 determines that the occupant remains in the passenger compartment if the number of pixels having a light and shade difference equal to or greater than a predetermined value is equal to or less than a predetermined value. As a result of the comparison, if the number of pixels having a difference in density equal to or greater than a predetermined value is greater than a predetermined value, the remaining occupant determination unit 17 captures a new image captured by the camera 9 as a comparative image, Make a decision.
　報知制御部１８は、残存乗員判定部１７により乗員が残存していると判定された場合に、報知する報知手段として機能する。報知制御部１８は、残存乗員判定部１７で車室内に乗員が残存すると判定された場合に、ハザードランプ１０の点滅やブザー１１の音によりその旨ドライバに報知する際に、ハザードランプ１０やブザー１１を駆動制御する。 The notification control unit 18 functions as a notification unit that notifies when the remaining occupant determination unit 17 determines that an occupant remains. When the remaining occupant determination unit 17 determines that the occupant remains in the passenger compartment, the notification control unit 18 notifies the driver of the hazard lamp 10 or the buzzer when the hazard lamp 10 blinks or the buzzer 11 sounds. 11 is driven and controlled.
　次に、ＥＣＵ２が実行処理する図３に示すフローチャートを参照して、この実施形態１の動作について説明する。 Next, the operation of the first embodiment will be described with reference to the flowchart shown in FIG.
　図３において、まず車両のイグニッションスイッチ（不図示）がオンとなると、本フローチャートが起動し、ステップＳ１０１において、以降の処理の実行に使用するタイマーやカウンタ、フラグを初期設定する初期化の処理を実行する。続いて、ステップＳ１０２において、車両状態信号である施錠信号、解錠信号、シフトポジション信号、車速信号に基づいて、現在の車両状態の判定を行う。この判定では、ドアがロック解除された後にシフトポジションがパーキング位置以外であり、車速が０ｋｍ／ｈでない場合には、走行状態と判定する。続いて、車速が０ｋｍ／ｈとなり、シフトポジションがパーキング位置となった場合には、ドライバが降車準備状態と判定し、車両のドアがロックされると施錠状態と判定する。 In FIG. 3, first, when a vehicle ignition switch (not shown) is turned on, this flowchart is started. In step S101, initialization processing for initializing timers, counters, and flags used for subsequent processing is performed. Execute. Subsequently, in step S102, the current vehicle state is determined based on the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals. In this determination, if the shift position is other than the parking position after the door is unlocked and the vehicle speed is not 0 km / h, it is determined that the vehicle is traveling. Subsequently, when the vehicle speed is 0 km / h and the shift position is the parking position, the driver determines that the vehicle is in the preparation state for getting off, and determines that the vehicle is locked when the vehicle door is locked.
　ステップＳ１０３において、ステップＳ１０２で判定された車両状態に基づいて分岐処理を行う。ステップＳ１０２において、車両状態が走行状態であると判定された場合には、ステップＳ１０４に進み、走行状態でなければステップＳ１０８へ進む。 In step S103, a branching process is performed based on the vehicle state determined in step S102. If it is determined in step S102 that the vehicle state is the traveling state, the process proceeds to step S104, and if not, the process proceeds to step S108.
　次に、ステップＳ１０４において、車室内のドライバ以外の乗員の有無を判定する。この乗員有無判定では、車両状態信号であるシートベルト装着センサ７および乗員位置入力スイッチ８からの入力信号に基づいて、ドライバ以外の乗員の有無を判定する。 Next, in step S104, the presence or absence of a passenger other than the driver in the passenger compartment is determined. In this occupant presence / absence determination, the presence / absence of an occupant other than the driver is determined based on input signals from the seat belt mounting sensor 7 and the occupant position input switch 8 which are vehicle state signals.
　続いて、ステップＳ１０５において、ステップＳ１０４の判定結果であるドライバ以外の乗員の有無に基づいて分岐処理を行う。ステップＳ１０４において、ドライバ以外の乗員がいると判定されている場合には、ステップＳ１０６に進み、いない場合にはステップＳ１０７へ進む。 Subsequently, in step S105, a branching process is performed based on the presence or absence of an occupant other than the driver, which is the determination result in step S104. If it is determined in step S104 that there is an occupant other than the driver, the process proceeds to step S106, and if not, the process proceeds to step S107.
　次に、ステップＳ１０６において、これまでの処理でドライバ以外の乗員がいると判定された場合は、以降の処理のために乗員フラグに１をセットし、先のステップＳ１０２へ戻り車両状態の判定を実施する。一方、ステップＳ１０７において、これまでの処理でドライバ以外の乗員はいないと判定された場合には、以降の処理のために乗員フラグに０をセットし、ステップＳ１０２へ戻り車両状態の判定を実施する。 Next, if it is determined in step S106 that there is an occupant other than the driver in the process so far, 1 is set in the occupant flag for the subsequent process, and the process returns to step S102 to determine the vehicle state. carry out. On the other hand, if it is determined in step S107 that there is no occupant other than the driver in the processing so far, the occupant flag is set to 0 for the subsequent processing, and the process returns to step S102 to determine the vehicle state. .
　次に、先のステップＳ１０３で車両が走行状態ではないと判定された場合には、続いてステップＳ１０８において、ステップＳ１０２で判定した車両状態に基づいて分岐処理を行う。先のステップＳ１０２において、車両状態が降車準備状態であると判定された場合には、ステップＳ１０９に進み、降車準備状態でないと判定された場合には、ステップＳ１１４へ進む。 Next, when it is determined in the previous step S103 that the vehicle is not in a traveling state, a branch process is subsequently performed in step S108 based on the vehicle state determined in step S102. If it is determined in the previous step S102 that the vehicle state is the preparation for getting off, the process proceeds to step S109, and if it is determined that the vehicle is not in the preparation for getting off, the process proceeds to step S114.
　次に、ステップＳ１０９において、ステップＳ１０６またはステップＳ１０７で設定した乗員フラグに基づいて分岐処理を行う。乗員フラグが１、すなわちドライバ以外の乗員がいる場合にはステップＳ１１０に進み、乗員フラグが０、すなわちドライバ以外の乗員がいない場合には、残存乗員の存在を報知する必要がないと判断して一連の処理を終了する。 Next, in step S109, branch processing is performed based on the occupant flag set in step S106 or step S107. If the occupant flag is 1, that is, if there is an occupant other than the driver, the process proceeds to step S110. If the occupant flag is 0, that is, if there is no occupant other than the driver, it is determined that it is not necessary to notify the presence of the remaining occupant. A series of processing ends.
　ステップＳ１１０では、カメラ９で撮像した画像が基準画像１６としてＥＣＵ２のマイクロコンピュータに備えられた記憶装置の処理領域に取り込まれる。このとき、カメラ９の設置位置により、助手席や後席を含めたすべての座席を一度に撮像して各座席ごとのドライバ以外の乗員の残存を以降の処理で判定することも可能であるが、本実施形態１では、残存の可能性が高い後席について説明するものとする。また、ここでは、図２（ａ），２（ｂ）で示すように車両天井に設置されたカメラ９で撮像した後席の画像のうち、図４に示すような左右いずれか片側の座席の処理について説明するが、後席全体及び助手席に対しても同様な処理により残存判定することができる。 In step S110, an image captured by the camera 9 is taken as a reference image 16 into a processing area of a storage device provided in the microcomputer of the ECU 2. At this time, depending on the installation position of the camera 9, it is possible to image all the seats including the passenger seat and the rear seat at a time, and determine the remaining of the passengers other than the driver for each seat in the subsequent processing. In the first embodiment, a rear seat that has a high possibility of remaining will be described. Also, here, as shown in FIGS. 2 (a) and 2 (b), among the images of the rear seats taken by the camera 9 installed on the vehicle ceiling, the left or right side seat as shown in FIG. Although the process will be described, it is possible to determine the remaining of the entire rear seat and the passenger seat by the same process.
　次に、ステップＳ１１１において、ステップＳ１１０で基準画像１６として処理領域に取り込んだ画像が基準画像１６として適しているか否かの診断を行う。診断方法としては、例えば画像全体の平均濃淡値が予め設定された基準画像１６として適した明るさの範囲に対して明るすぎたり暗すぎたりした場合には、基準画像１６としては不適と診断し、それ以外の画像を基準画像１６に適しているものと診断する。 Next, in step S111, it is diagnosed whether or not the image captured in the processing area as the reference image 16 in step S110 is suitable as the reference image 16. As a diagnostic method, for example, when the average gray value of the entire image is too bright or too dark for a brightness range suitable for the preset reference image 16, it is diagnosed as inappropriate as the reference image 16. The other images are diagnosed as being suitable for the reference image 16.
　続いて、ステップＳ１１２において、ステップＳ１１１での診断結果に基づいて分岐処理を行う。ステップＳ１１０で処理領域に取り込んだ画像に対してステップＳ１１１で適していると診断された場合にはステップＳ１１３に進み、不適と診断された場合には、ステップＳ１１０に戻り、再度画像を取り込んで再度診断を実施する。なお、本実施形態１では、適していると診断されるまで処理を繰り返すことにしているが、所定回数以上繰り返しても診断結果が不適の場合には、以降の残存判定が不能ということでその旨ドライバに通知するようにしてもよい。 Subsequently, in step S112, branch processing is performed based on the diagnosis result in step S111. If it is determined that the image captured in the processing area in step S110 is suitable in step S111, the process proceeds to step S113. If it is diagnosed as inappropriate, the process returns to step S110, and the image is captured again. Make a diagnosis. In the first embodiment, the process is repeated until it is diagnosed as being suitable. However, if the diagnosis result is unsuitable even if it is repeated a predetermined number of times or more, the remaining determination cannot be made. The driver may be notified.
　次に、ステップＳ１１３において、ステップＳ１１０で処理領域に取り込んだ画像を以降の処理のため基準画像１６として、ＥＣＵ２のマイクロコンピュータに備えられたメモリに記憶し、ステップＳ１０２へ戻り車両状態の判定を実施する。ここで記憶される基準画像１６の一例を図４（ａ）に示す。これまでの処理でドライバ以外の乗員がいると判定されているので、基準画像１６は乗員がいる状態となる。 Next, in step S113, the image captured in the processing area in step S110 is stored as a reference image 16 for subsequent processing in a memory provided in the microcomputer of the ECU 2, and the process returns to step S102 to determine the vehicle state. To do. An example of the reference image 16 stored here is shown in FIG. Since it is determined that there is an occupant other than the driver in the processing so far, the reference image 16 is in a state where there is an occupant.
　次に、ステップＳ１１４において、ステップＳ１０２で判定した車両状態に基づいて分岐処理を行う。ここでは、ステップＳ１０３で車両状態は走行状態ではない、かつステップＳ１０８で降車準備状態ではないと判定されている。ステップＳ１０２において車両状態が施錠状態であると判定されている場合には、ドライバは降車して車両が施錠されたものと判定し、ステップＳ１１５に進み、施錠状態でなければステップＳ１０２へ戻り車両状態の判定を実施する。 Next, in step S114, branch processing is performed based on the vehicle state determined in step S102. Here, it is determined in step S103 that the vehicle state is not the traveling state and in step S108 that the vehicle is not in the getting-off preparation state. If it is determined in step S102 that the vehicle state is locked, the driver determines that the vehicle is getting off and the vehicle is locked, and proceeds to step S115. If the vehicle state is not locked, the driver returns to step S102 to return to the vehicle state. Perform the determination.
　続いて、ステップＳ１１５において、ステップＳ１０９と同様にステップＳ１０６またはステップＳ１０７で設定した乗員フラグに基づいて分岐処理を行う。乗員フラグが１、すなわちドライバ以外の乗員がいる場合には、ステップＳ１１６に進み、乗員フラグが０、すなわちドライバ以外の乗員がいない場合には、残存乗員の存在を報知する必要がないと判断して一連の処理を終了する。 Subsequently, in step S115, branch processing is performed based on the occupant flag set in step S106 or step S107, as in step S109. If the occupant flag is 1, that is, if there is an occupant other than the driver, the process proceeds to step S116, and if the occupant flag is 0, that is, if there is no occupant other than the driver, it is determined that there is no need to notify the presence of the remaining occupant. To end the series of processing.
　次に、ステップＳ１１６において、カメラ９で撮像した画像を比較画像として処理領域に取り込み、ステップＳ１１７に進む。ここで取得される比較画像の一例を図４（ｂ）、（ｃ）に示す。図４（ｂ）は乗員がいない場合、図４（ｃ）は乗員がいる場合の一例である。 Next, in step S116, the image captured by the camera 9 is taken into the processing area as a comparison image, and the process proceeds to step S117. An example of the comparative image acquired here is shown in FIGS. FIG. 4B is an example when there is no occupant, and FIG. 4C is an example when there is an occupant.
　次に、ステップＳ１１７において、ステップＳ１１６で取り込んだ比較画像とステップＳ１１３で記憶した基準画像１６とを比較する。ここで比較方法としては、例えば比較画像と基準画像１６との対応する画素毎の濃淡差分を算出し、予め設定された所定値以上の濃度差がある画素の数をカウントする。 Next, in step S117, the comparison image captured in step S116 is compared with the reference image 16 stored in step S113. Here, as a comparison method, for example, the density difference for each corresponding pixel between the comparison image and the reference image 16 is calculated, and the number of pixels having a density difference equal to or greater than a predetermined value set in advance is counted.
　続いて、ステップＳ１１８において、ステップＳ１１７の処理結果に基づいて分岐処理を行う。ステップＳ１１７で算出した比較画像と基準画像１６との差であるカウント値（すなわち画像中の面積に相当）が、予め設定された所定値ＴＨ＿ＮＧ（例えば画像全体の３／４）以下であれば、ステップＳ１１９に進む。一方、所定値ＴＨ＿ＮＧより大きければ両者の差が大きすぎて、例えば他車両のライトなどで瞬間的に照射されたような瞬間的な照度変化の可能性が考えられる。このため、先のステップＳ１１６に戻り再度比較画像を取り込む。 Subsequently, in step S118, branch processing is performed based on the processing result in step S117. If the count value that is the difference between the comparison image calculated in step S117 and the reference image 16 (that is, the area in the image) is equal to or smaller than a predetermined value TH_NG (for example, 3/4 of the entire image), The process proceeds to step S119. On the other hand, if it is larger than the predetermined value TH_NG, the difference between the two is too large, and for example, there is a possibility of an instantaneous change in illuminance as if it was instantaneously irradiated with a light of another vehicle. Therefore, the process returns to the previous step S116 to capture the comparison image again.
　本実施形態１では、比較画像と基準画像１６の差が所定値ＴＨ＿ＮＧ以下になるまで処理を繰り返すことにしている。しかし、所定回数以上繰り返しても所定値ＴＨ＿ＮＧ以下にならない場合には、残存乗員の判定不能ということでその旨ドライバに通知するようにしてもよい。 In the first embodiment, the process is repeated until the difference between the comparison image and the reference image 16 becomes equal to or smaller than a predetermined value TH_NG. However, if it does not become the predetermined value TH_NG or less even after being repeated a predetermined number of times, the driver may be notified that the remaining occupant cannot be determined.
　次に、ステップＳ１１９において、ステップＳ１１７の処理結果に基づいて分岐処理を行う。ステップＳ１１７で算出した比較画像と基準画像１６との差であるカウント値（すなわち画像中の面積に相当）が、予め設定された所定値ＴＨ＿Ｒ（例えば画像全体の１／８）以下であれば基準画像１６との差がないと判定する。すなわち、乗員がいるものと推定し、ステップＳ１２０に進む。一方、所定値ＴＨ＿Ｒよりも大きければ基準画像１６との差があり、すなわち乗員はいないものと推定し、残存乗員の存在を報知する必要がないと判断して一連の処理を終了する。ここで判定画像の一例を図４（ｄ）、（ｅ）に示す。図４（ｄ）は乗員が存在していない場合、図４（ｅ）は乗員が存在している場合を示している。 Next, in step S119, branch processing is performed based on the processing result in step S117. If the count value that is the difference between the comparison image calculated in step S117 and the reference image 16 (that is, the area in the image) is equal to or less than a predetermined value TH_R (for example, 1/8 of the entire image), the reference It is determined that there is no difference from the image 16. That is, it is estimated that there is a passenger, and the process proceeds to step S120. On the other hand, if it is larger than the predetermined value TH_R, there is a difference from the reference image 16, that is, it is estimated that there is no occupant, and it is determined that there is no need to notify the presence of the remaining occupant, and the series of processing ends. An example of the determination image is shown in FIGS. 4 (d) and 4 (e). FIG. 4D shows a case where no occupant exists, and FIG. 4E shows a case where an occupant exists.
　最後に、ステップＳ１２０において、ステップＳ１１９でドライバ以外の乗員が車室内に残っていると判定された場合には、ハザードランプ１０の点滅やブザー１１の音によりその旨ドライバに報知して、一連の処理を終了する。 Finally, in step S120, if it is determined in step S119 that an occupant other than the driver remains in the passenger compartment, the driver is notified by blinking the hazard lamp 10 or the sound of the buzzer 11, and a series of The process ends.
　なお、本実施形態１では、残存乗員の存在を報知して直ぐ一連の処理を終了するようにしている。一方、残存乗員の放置が解消されるまで、もしくはドライバの意志、例えば再度の施錠動作といった意志が確認できるまで、報知動作を継続するようにしてもよい。 In the first embodiment, a series of processing is terminated immediately after the presence of the remaining occupant is notified. On the other hand, the notification operation may be continued until the remaining occupant is canceled or until the driver's intention, for example, the intention of the re-locking operation can be confirmed.
　このように、上記実施形態１においては、カメラで撮像された画像に基づいて乗員の有無を判定する際に、基準画像１６と比較画像とを取得する時間間隔を従来に比べて短く設定している。これにより、時間の変化等に起因する車室内環境の変化を受けにくくなり、ドライバが降車後に車室内の残存乗員の有無を判定する判定処理の精度を向上することができる。この結果、車室内に残存する乗員を精度よく検出することができ、乗員が残存する場合には、その旨的確に報知することができる。これにより、幼児などが車室内に取り残されて閉じ込められるといった事態を防止できる可能性が高くなる。 As described above, in the first embodiment, when determining the presence or absence of the occupant based on the image captured by the camera, the time interval for acquiring the reference image 16 and the comparison image is set shorter than in the past. Yes. Thereby, it becomes difficult to receive the change of the vehicle interior environment resulting from the change of time etc., and the accuracy of the determination process which determines the presence or absence of the passenger | crew in the vehicle interior after a driver gets off can be improved. As a result, the occupant remaining in the passenger compartment can be detected with high accuracy, and when the occupant remains, it can be accurately notified. This increases the possibility of preventing a situation where an infant or the like is left behind and confined in the vehicle interior.
　車両の状態として、車両が走行している状態、ドライバが降車を準備して降車する状態、車両が施錠された状態を判定し、これらの状態の判定結果の組み合わせに基づいて、ドライバが降車して車両が施錠された状態を判定している。これにより、車両の状態に応じて実施する上述した各種処理を的確に実施することができる。 The state of the vehicle is determined such that the vehicle is traveling, the driver prepares to get off the vehicle, the vehicle is locked, and the vehicle is locked, and the driver gets off based on the combination of the determination results of these states. The vehicle is determined to be locked. Thereby, the various processes mentioned above performed according to the state of a vehicle can be implemented exactly.
　基準画像記憶部１４に基準画像１６が記憶される際に、予め設定された適否要件に基づいて、記憶する画像が比較画像と比較するのに適しているか否かを診断する基準画像診断部１５を備えている。基準画像診断部１５は、基準画像記憶部１４に記憶する画像が比較画像と比較するのに適していないものと診断した場合には、カメラ９により再度撮像した新たな画像で再診断する。基準画像記憶部１４は、基準画像診断部１５により記憶する画像が比較画像と比較するのに適しているものと診断された画像を基準画像１６として記憶する。これにより、適切でない基準画像１６を用いることによる車室内の残存乗員有無の誤判定を排除することができる。 When the reference image 16 is stored in the reference image storage unit 14, a reference image diagnosis unit 15 for diagnosing whether or not the stored image is suitable for comparison with the comparison image, based on preset suitability requirements. It has. If the reference image diagnosis unit 15 diagnoses that the image stored in the reference image storage unit 14 is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 performs re-diagnosis with a new image taken again by the camera 9. The reference image storage unit 14 stores, as the reference image 16, an image diagnosed as an image stored by the reference image diagnostic unit 15 suitable for comparison with a comparison image. Thereby, it is possible to eliminate erroneous determination of the presence or absence of a remaining occupant in the vehicle interior due to the use of the inappropriate reference image 16.
　基準画像１６と比較画像との対応する画素の濃淡差を算出し、所定値以上の濃淡差がある画素数が所定値以上である場合には、カメラ９で撮像された新たな画像を比較画像として再判定を行う。これにより、例えば他車のライトを瞬間的に照らされたような瞬間的な照度変化に対しても適切な画像を比較画像として用いることができ、車室内の残存乗員有無の誤判定を排除することができる。
（実施形態２）
　図５は本発明の実施形態２に係る車室内残存乗員検出装置２１の構成を示す図である。なお、図５において、先の実施形態１の構成を示す図１と同一または同等の機能を有する構成要件には同じ符号を付与し、その説明を省略する。 The contrast difference of the corresponding pixels between the reference image 16 and the comparison image is calculated, and if the number of pixels having a contrast difference equal to or greater than a predetermined value is equal to or greater than the predetermined value, a new image captured by the camera 9 is compared with the comparison image. Re-determination is performed. As a result, for example, an appropriate image can be used as a comparison image even for a momentary change in illuminance such as when the light of another vehicle is momentarily illuminated, and an erroneous determination of the presence or absence of a remaining passenger in the passenger compartment is eliminated. be able to.
(Embodiment 2)
FIG. 5 is a diagram showing a configuration of the vehicle interior remaining passenger detection device 21 according to Embodiment 2 of the present invention. In FIG. 5, constituent elements having the same or equivalent functions as those in FIG. 1 showing the configuration of the first embodiment are given the same reference numerals, and descriptions thereof are omitted.
　図５において、本実施形態２の車室内残存乗員検出装置２１は、実施形態１におけるＥＣＵ２に代えて新たなＥＣＵ２２を備えている。このＥＣＵ２２には、実施形態１と同様のドアロック検知センサ３から施錠信号、アンロック検知センサ４から解錠信号、シフトポジションセンサ５からシフトポジション信号、車速センサ６から車速信号が入力され、実施形態１と同様のカメラ９が接続されている。 In FIG. 5, the vehicle interior remaining passenger detection device 21 according to the second embodiment includes a new ECU 22 instead of the ECU 2 according to the first embodiment. The ECU 22 receives the lock signal from the door lock detection sensor 3, the unlock signal from the unlock detection sensor 4, the shift position signal from the shift position sensor 5, and the vehicle speed signal from the vehicle speed sensor 6 as in the first embodiment. A camera 9 similar to that in Embodiment 1 is connected.
　また、ＥＣＵ２２は、実施形態１の車両状態判定部１２に代えて新たな車両状態判定部２３を備え、実施形態１の乗員有無判定部１３に代えて新たな乗員有無判定部２４を備えている。また、ＥＣＵ２２は、実施形態１の基準画像記憶部１４に代えて新たな基準画像記憶部２５を備え、実施形態１の残存乗員判定部１７に代えて新たな残存乗員判定部２６を備えている。 The ECU 22 includes a new vehicle state determination unit 23 instead of the vehicle state determination unit 12 of the first embodiment, and includes a new passenger presence determination unit 24 instead of the occupant presence / absence determination unit 13 of the first embodiment. . The ECU 22 includes a new reference image storage unit 25 instead of the reference image storage unit 14 of the first embodiment, and includes a new remaining occupant determination unit 26 instead of the remaining occupant determination unit 17 of the first embodiment. .
　車両状態判定部２３は、施錠信号、解錠信号、シフトポジション信号、車速信号から、車両状態として解錠状態、走行開始状態、降車準備状態、施錠状態のいずれの状態であるかを判定する。車両状態判定部２３は、解錠信号にしたがって解錠状態と判定し、ＥＣＵを構成するマイクロコンピュータに備えられたカウンタやフラグが初期化されている場合には、ドライバを含めた乗員は乗車前であると判定する。車両状態判定部２３は、ドアロック解除された後にシフトポジションがパーキング位置以外であり、車速が０ｋｍ／ｈでなくなった場合に走行開始状態と判定する。車両状態判定部２３は、車速が０ｋｍ／ｈとなり、シフトポジションがパーキング位置となった場合には、ドライバは降車を準備して降車する状態（降車準備状態）であると判定する。車両状態判定部２３は、車両のドアがロックされると施錠状態と判定する。車両状態判定部２３は、車両の解錠後、車両が走行中でなくかつ降車準備状態でないと判定され、かつ、車両が施錠されたと判定された場合には、ドライバは降車して車両が施錠されたものと判定する。 The vehicle state determination unit 23 determines, from the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal, whether the vehicle state is an unlocked state, a travel start state, a getting-off preparation state, or a locked state. The vehicle state determination unit 23 determines that the vehicle is unlocked according to the unlock signal, and if the counters and flags provided in the microcomputer constituting the ECU have been initialized, the occupant including the driver is It is determined that The vehicle state determination unit 23 determines that the vehicle is in the travel start state when the shift position is other than the parking position after the door lock is released and the vehicle speed is no longer 0 km / h. When the vehicle speed is 0 km / h and the shift position is at the parking position, the vehicle state determination unit 23 determines that the driver is in a state of preparing to get off and getting off (preparation for getting off). The vehicle state determination unit 23 determines that the vehicle is locked when the vehicle door is locked. When it is determined that after the vehicle is unlocked, the vehicle is not running and is not ready to get off, and the vehicle state determination unit 23 determines that the vehicle is locked, the driver gets off and the vehicle is locked. It is determined that
　乗員有無判定部２４は、車両状態判定部２３により車両が解錠されて走行が開始されたと判定されたときに、車室内にドライバ以外の乗員が存在しているかどうかを判定する。すなわち、乗員有無判定部２４は、解錠状態と判定されたときにカメラ９で撮像された画像と、走行が開始されたと判定されたときにカメラ９で撮像された画像とを比較することで、車室内にドライバ以外の乗員が存在しているかどうかを判定する。 The occupant presence / absence determination unit 24 determines whether there is an occupant other than the driver in the passenger compartment when the vehicle state determination unit 23 determines that the vehicle has been unlocked and travel has started. In other words, the occupant presence / absence determination unit 24 compares the image captured by the camera 9 when it is determined to be unlocked with the image captured by the camera 9 when it is determined that traveling has started. Then, it is determined whether there are passengers other than the driver in the passenger compartment.
　乗員有無判定部２４は、乗員が乗車後車両が走行を開始したと判定されたタイミングでカメラ９で撮像した画像を比較画像（第２の比較画像）として取得する。乗員有無判定部２４は、取得した比較画像と、ドライバを含む乗員が乗車前で解錠状態と判定されたときにカメラ９で撮像された画像の基準画像１６（第２の基準画像）との画素毎の濃淡差分処理により画像間の違いを比較する。比較の結果、乗員がいない場合の基準画像１６に対して、比較画像と比較した結果に違いがあれば乗員がいるものと推定し、ドライバ以外の乗員がいると判定する。 The passenger presence / absence determination unit 24 acquires, as a comparative image (second comparative image), an image captured by the camera 9 at a timing when it is determined that the vehicle has started running after the passenger has boarded the vehicle. The occupant presence / absence determination unit 24 compares the acquired comparison image and the reference image 16 (second reference image) of the image captured by the camera 9 when the occupant including the driver is determined to be unlocked before boarding. The difference between the images is compared by gradation difference processing for each pixel. As a result of the comparison, if there is a difference between the comparison image and the reference image 16 when there is no occupant, it is estimated that there is an occupant and it is determined that there is an occupant other than the driver.
　乗員有無判定部２４は、基準画像１６とカメラで撮像された比較画像との対応する画素の濃淡差を算出する。乗員有無判定部２４は、予め設定された所定値以上の濃淡差がある画素数が予め設定された所定値より大きい場合には、カメラ９により撮像された新たな画像を、解錠状態と判定されたときに前記カメラ９で撮像した画像と比較し、再判定を行う。 The occupant presence / absence determination unit 24 calculates a difference in shade of corresponding pixels between the reference image 16 and a comparison image captured by the camera. The occupant presence / absence determination unit 24 determines that the new image captured by the camera 9 is in an unlocked state when the number of pixels having a difference in density equal to or greater than a predetermined value is greater than a predetermined value. When this is done, it is compared with the image captured by the camera 9 and re-determined.
　基準画像記憶部２５は、車両状態判定部２３で降車準備中であると判定され、かつ、乗員有無判定部２４でドライバ以外の乗員がいると判定された場合に、カメラ９で撮像した画像を基準画像１６（第１の基準画像）として記憶する第１の基準画像記憶手段として機能し、車両状態判定部２３で解錠状態と判定された場合に、カメラ９で撮像した画像を基準画像１６（第２の基準画像）として記憶する第２の基準画像記憶手段として機能する。基準画像診断部１５は、実施形態１で説明した機能に加えて、車両状態判定部２３で走行開始状態と判定された場合に、基準画像記憶部２５に基準画像１６が記憶される際に、記憶する画像が比較画像と比較するのに適しているか否かを診断する第２の基準画像診断手段として機能する。基準画像診断部１５は、記憶する画像が比較画像と比較するのに適していないものと判定した場合には、カメラ９により再度撮像した新たな画像で再診断する第２の基準画像診断手段として機能する。 The reference image storage unit 25 determines an image captured by the camera 9 when it is determined that the vehicle state determination unit 23 is preparing to get off and the passenger presence determination unit 24 determines that there is an occupant other than the driver. It functions as a first reference image storage unit that stores the reference image 16 (first reference image), and when the vehicle state determination unit 23 determines that the vehicle is unlocked, the image captured by the camera 9 is used as the reference image 16. It functions as second reference image storage means for storing (second reference image). In addition to the functions described in the first embodiment, the reference image diagnosis unit 15 is configured to store the reference image 16 in the reference image storage unit 25 when the vehicle state determination unit 23 determines that the vehicle is in the travel start state. It functions as a second reference image diagnostic means for diagnosing whether or not the stored image is suitable for comparison with the comparative image. When the reference image diagnosis unit 15 determines that the stored image is not suitable for comparison with the comparison image, the reference image diagnosis unit 15 serves as a second reference image diagnosis unit that performs re-diagnosis with a new image captured again by the camera 9. Function.
　次に、ＥＣＵ２２が実行処理する図６に示すフローチャートを参照して、この実施形態２の動作について、実施形態１と異なる処理を主に説明する。なお、図６において、実施形態１の処理動作を示す図３と同等の処理動作には同じ符号を付与し、その説明を省略する。 Next, with reference to the flowchart shown in FIG. 6 executed by the ECU 22, the operation of the second embodiment will be described mainly with respect to the processing different from the first embodiment. In FIG. 6, the same reference numerals are given to the processing operations equivalent to those in FIG. 3 showing the processing operations of the first embodiment, and the description thereof is omitted.
　図６において、図３に示すステップＳ１０１の処理を実行した後、続いてステップＳ２０１において、車両状態信号である施錠信号、解錠信号、シフトポジション信号、車速信号に基づいて、現在の車両状態の判定を行う。この判定では、解錠信号で解錠状態を判定し、ロック解除された後にシフトポジションがパーキング位置以外であり、車速が０ｋｍ／ｈでなくなった場合に走行開始状態と判定する。続いて、車速が０ｋｍ／ｈとなり、シフトポジションがパーキング位置となった場合には、降車準備状態と判定し、施錠信号に基づいて施錠状態と判定する。 In FIG. 6, after performing the process of step S101 shown in FIG. 3, in step S201, based on the lock signal, the unlock signal, the shift position signal, and the vehicle speed signal that are vehicle state signals, Make a decision. In this determination, the unlocked state is determined based on the unlock signal, and the travel start state is determined when the shift position is other than the parking position and the vehicle speed is no longer 0 km / h after unlocking. Subsequently, when the vehicle speed becomes 0 km / h and the shift position becomes the parking position, it is determined that the vehicle is in the preparation state for getting off, and the locked state is determined based on the locking signal.
　次に、ステップＳ２０２において、ステップＳ２０１で判定した車両状態に基づいて分岐処理を行う。ステップＳ２０１で初期化後に車両状態が解錠状態であると判定された場合には、ドライバを含む乗員が車両に乗車する前であると判定し、ステップＳ２０３に進み、解錠状態でなければステップＳ２０７へ進む。 Next, in step S202, branch processing is performed based on the vehicle state determined in step S201. If it is determined in step S201 that the vehicle state is an unlocked state after initialization, it is determined that an occupant including a driver is not in the vehicle, and the process proceeds to step S203. The process proceeds to S207.
　次に、ステップＳ２０３において、カメラ９で撮像した画像を基準画像１６として処理領域に取り込み、ステップＳ２０４に進む。 Next, in step S203, the image captured by the camera 9 is taken into the processing area as the reference image 16, and the process proceeds to step S204.
　次に、ステップＳ２０４において、ステップＳ２０３で基準画像１６として処理領域に取り込んだ画像が基準画像１６として適しているか否かの診断を行う。診断方法としては、例えば画像全体の平均濃淡値が予め設定された基準画像１６として適した明るさの範囲に対して明るすぎたり暗すぎたりした場合には、基準画像１６としては不適と診断し、それ以外の画像を基準画像１６に適しているものと診断する。 Next, in step S204, a diagnosis is made as to whether or not the image captured in the processing area as the reference image 16 in step S203 is suitable as the reference image 16. As a diagnostic method, for example, when the average gray value of the entire image is too bright or too dark for a brightness range suitable for the preset reference image 16, it is diagnosed as inappropriate as the reference image 16. The other images are diagnosed as being suitable for the reference image 16.
　続いて、ステップＳ２０５において、ステップＳ２０４での診断結果に基づいて分岐処理を行う。ステップＳ２０３で処理領域に取り込んだ画像に対してステップＳ２０４で適していると診断された場合にはステップＳ２０６に進み、不適と診断された場合には、ステップＳ２０３に戻り、再度画像を取り込んで再度診断を実施する。なお、本実施形態２では、適していると診断されるまで処理を繰り返すことにしているが、所定回数以上繰り返しても診断結果が不適の場合には、以降の残存判定が不能ということでその旨ドライバに通知するようにしてもよい。 Subsequently, in step S205, branch processing is performed based on the diagnosis result in step S204. If it is determined in step S204 that the image captured in the processing area is suitable in step S204, the process proceeds to step S206. If it is diagnosed as inappropriate, the process returns to step S203, and the image is captured again. Make a diagnosis. In the second embodiment, the process is repeated until it is diagnosed as being suitable. However, if the diagnosis result is unsuitable even if it is repeated a predetermined number of times or more, it is impossible to perform the remaining determination. The driver may be notified.
　次に、ステップＳ２０６において、ステップＳ２０３で処理領域に取り込んだ画像を以降の処理のため基準画像１６として、ＥＣＵ２のマイクロコンピュータに備えられたメモリに記憶し、ステップＳ２０１へ戻り車両状態の判定を実施する。ここで記憶される基準画像１６の一例を図７（ａ）に示す。図７（ａ）に示す状態は、車両の解錠時なので、基準画像１６には乗員がいない状態となる。 Next, in step S206, the image captured in the processing area in step S203 is stored in the memory provided in the microcomputer of the ECU 2 as a reference image 16 for subsequent processing, and the process returns to step S201 to determine the vehicle state. To do. An example of the reference image 16 stored here is shown in FIG. Since the state shown in FIG. 7A is when the vehicle is unlocked, the reference image 16 has no occupant.
　次に、ステップＳ２０７において、ステップＳ２０１で判定した車両状態に基づいて分岐処理を行う。ここでは、ステップＳ２０２で車両状態は解錠状態ではないと判定されており、ステップＳ２０１において車両状態が走行開始状態であると判定されている場合には、ステップＳ２０８に進み、走行開始状態でなければステップＳ１０８へ進む。 Next, in step S207, branch processing is performed based on the vehicle state determined in step S201. Here, it is determined in step S202 that the vehicle state is not an unlocked state, and if it is determined in step S201 that the vehicle state is a travel start state, the process proceeds to step S208 and must be in a travel start state. If so, the process proceeds to step S108.
　次に、ステップＳ２０８において、カメラ９で撮像した画像を比較画像として処理領域に取り込み、ステップＳ２０９に進む。ここで取得される比較画像の一例を図７（ｂ）、（ｃ）に示す。図７（ｂ）は乗員がいない場合、図７（ｃ）は乗員がいる場合を示している。 Next, in step S208, the image captured by the camera 9 is taken into the processing area as a comparative image, and the process proceeds to step S209. An example of the comparative image acquired here is shown in FIGS. 7B and 7C. FIG. 7B shows a case where there is no passenger, and FIG. 7C shows a case where there is a passenger.
　次に、ステップＳ２０９において、ステップＳ２０８で取り込んだ比較画像とステップＳ２０６で記憶した基準画像１６とを比較する。ここで比較方法としては、例えば比較画像と基準画像１６との対応する画素毎の濃淡差分を算出し、予め設定された所定値以上の濃度差がある画素の数をカウントする。 Next, in step S209, the comparison image captured in step S208 is compared with the reference image 16 stored in step S206. Here, as a comparison method, for example, the density difference for each corresponding pixel between the comparison image and the reference image 16 is calculated, and the number of pixels having a density difference equal to or greater than a predetermined value set in advance is counted.
　続いて、ステップＳ２１０において、ステップＳ２０９の処理結果に基づいて分岐処理を行う。ステップＳ２０９で算出した比較画像と基準画像１６との差であるカウント値（すなわち画像中の面積に相当）が、予め設定された所定値ＴＨ＿ＮＧ（例えば画像全体の３／４）以下であれば、ステップＳ２１１に進む。一方、所定値ＴＨ＿ＮＧより大きければ両者の差が大きすぎて、例えば他車両のライトなどで瞬間的に照射されたような瞬間的な照度変化の可能性が考えられる。このため、先のステップＳ２０８に戻り再度比較画像を取り込む。 Subsequently, in step S210, branch processing is performed based on the processing result of step S209. If the count value that is the difference between the comparison image calculated in step S209 and the reference image 16 (that is, the area in the image) is equal to or smaller than a predetermined value TH_NG (for example, 3/4 of the entire image), The process proceeds to step S211. On the other hand, if it is larger than the predetermined value TH_NG, the difference between the two is too large, and for example, there is a possibility of an instantaneous change in illuminance as if it was instantaneously irradiated with a light of another vehicle. Therefore, the process returns to the previous step S208 to capture the comparison image again.
　本実施形態２では、比較画像と基準画像１６の差が所定値ＴＨ＿ＮＧ以下になるまで処理を繰り返すことにしている。しかし、所定回数以上繰り返しても所定値ＴＨ＿ＮＧ以下にならない場合には、残存乗員の判定不能ということでその旨ドライバに通知するようにしてもよい。 In the second embodiment, the process is repeated until the difference between the comparison image and the reference image 16 becomes equal to or smaller than a predetermined value TH_NG. However, if it does not become the predetermined value TH_NG or less even after being repeated a predetermined number of times, the driver may be notified that the remaining occupant cannot be determined.
　次に、ステップＳ２１１において、ステップＳ２０９の処理結果に基づいて分岐処理を行う。ステップＳ２０９で算出した比較画像と基準画像１６との差であるカウント値（すなわち画像中の面積に相当）が、予め設定された所定値ＴＨ＿Ｒ（例えば画像全体の１／８）以下であれば基準画像１６との差がないと判定する。すなわち、乗員がいないものと推定して、ステップＳ２１２に進む。一方、所定値ＴＨ＿Ｒよりも大きければ基準画像１６との差がある、すなわち乗員はいるものと推定し、ステップＳ２１３に進む。ここで判定画像の一例を図７（ｄ）、（ｅ）に示す。図７（ｄ）は乗員がいない場合、図７（ｅ）は乗員がいる場合を示している。このように、図５に示す構成において、上記ステップＳ２０７～ステップＳ２１１に示す処理を実行することで、乗員の有無を判定し、乗員有無判定手段として機能している。 Next, in step S211, branch processing is performed based on the processing result of step S209. If the count value (that is, the area in the image) that is the difference between the comparison image calculated in step S209 and the reference image 16 is equal to or less than a predetermined value TH_R (for example, 1/8 of the entire image), the reference It is determined that there is no difference from the image 16. That is, it is estimated that no passenger is present, and the process proceeds to step S212. On the other hand, if it is larger than the predetermined value TH_R, it is estimated that there is a difference from the reference image 16, that is, there is an occupant, and the process proceeds to step S213. An example of the determination image is shown in FIGS. 7D and 7E. FIG. 7D shows a case where there is no passenger, and FIG. 7E shows a case where there is a passenger. In this way, in the configuration shown in FIG. 5, by performing the processing shown in steps S207 to S211, the presence / absence of an occupant is determined and functions as an occupant presence / absence determination means.
　続いて、ステップＳ２１２において、これまでの処理でドライバ以外の乗員がいないと判定されているので、以降の処理のために乗員フラグに０をセットし、ステップＳ２０１へ戻り車両状態の判定を実施する。一方、ステップＳ２１３において、これまでの処理でドライバ以外の乗員はいると判定されているので、以降の処理のために乗員フラグに１をセットし、ステップＳ２０１へ戻り車両状態の判定を実施する。 Subsequently, in step S212, it is determined that there is no occupant other than the driver in the processing so far, so the occupant flag is set to 0 for the subsequent processing, and the process returns to step S201 to determine the vehicle state. . On the other hand, since it is determined in step S213 that there is an occupant other than the driver in the process so far, 1 is set in the occupant flag for the subsequent process, and the process returns to step S201 to determine the vehicle state.
　その後、ステップＳ２０１で車両状態を判定する。判定の結果、車両状態は解錠状態ではなく、かつ車両状態が走行開始状態でないと判定された場合には、先の図３に示すフローチャートに示す一連のステップＳ１０８～Ｓ１２０と同様の処理を実行する。この一連の処理を実行することで、先の実施形態１と同様に車室内の残存乗員の有無を判定し、車室内に乗員が残存しているものと判定された場合にはその旨報知する。 Thereafter, the vehicle state is determined in step S201. As a result of the determination, if it is determined that the vehicle state is not unlocked and the vehicle state is not the running start state, the same processing as the series of steps S108 to S120 shown in the flowchart of FIG. 3 is executed. To do. By executing this series of processing, it is determined whether there is a remaining passenger in the vehicle interior as in the first embodiment, and if it is determined that there is a passenger remaining in the vehicle interior, a notification to that effect is given. .
　このように、本実施形態２においては、実施形態１と同様の効果を得ることができることに加えて、以下に説明するような効果を得ることができる。 Thus, in the second embodiment, in addition to being able to obtain the same effects as those of the first embodiment, the effects described below can be obtained.
　基準画像記憶部２５は、車両状態判定部２３によりドライバを含めた乗員が車両に乗車する前であると判定されたときに、カメラ９により撮像された画像を基準画像１６として記憶する。乗員有無判定部２４は、基準画像記憶部２５に記憶された基準画像１６と、乗員が車両に乗車後にカメラ９で撮像された画像である比較画像とを比較し、車室内にドライバ以外の乗員が存在しているか否かを判定する。これにより、残存乗員の有無の判定と同様に基準画像１６と比較画像との比較結果に基づいて、乗員の有無を判定することが可能となり、残存乗員の有無の判定と同様の処理により乗員の有無を判定することが可能となる。この結果、乗員の有無と残存乗員の有無を判定する一部処理の共通化を図ることが可能となり、装置の構成を大型化することなく乗員の有無を判定することができる。 The reference image storage unit 25 stores the image captured by the camera 9 as the reference image 16 when the vehicle state determination unit 23 determines that the occupant including the driver is before getting on the vehicle. The occupant presence / absence determination unit 24 compares the reference image 16 stored in the reference image storage unit 25 with a comparative image that is an image captured by the camera 9 after the occupant gets into the vehicle, and occupants other than the driver are placed in the passenger compartment. It is determined whether or not exists. This makes it possible to determine the presence or absence of an occupant on the basis of the comparison result between the reference image 16 and the comparison image in the same manner as the determination of the presence or absence of a remaining occupant. The presence or absence can be determined. As a result, it is possible to share part of the processing for determining the presence / absence of the occupant and the presence / absence of the remaining occupant, and the presence / absence of the occupant can be determined without increasing the size of the apparatus.
　乗員有無判定部２４は、基準画像１６と比較画像との対応する画素の濃淡差を算出し、所定値以上の濃淡差がある画素数が所定値以上である場合には、カメラ９により撮像された新たな画像を比較画像として再判定を行う。これにより、例えば他車のライトを瞬間的に照らされたような瞬間的な照度変化に対しても適切な画像を比較画像として用いることができ、乗員有無の誤判定を排除することができる。 The occupant presence / absence determination unit 24 calculates the density difference between the corresponding pixels of the reference image 16 and the comparison image. The new image is re-determined as a comparison image. Thereby, for example, an appropriate image can be used as a comparative image even for an instantaneous illuminance change such as when a light of another vehicle is instantaneously illuminated, and an erroneous determination of the presence or absence of an occupant can be eliminated.
　車両の状態として、車両が解錠された解錠状態、車両が走行を開始した状態、ドライバが降車を準備して降車する降車準備状態、車両が施錠された施錠状態を判定する。これらの状態の判定結果の組み合わせに基づいて、ドライバを含めた乗員が車両に乗車する前である状態、ドライバが降車して車両が施錠された状態を判定している。これにより、車両の状態に応じて実施する上述した各種処理を的確に実施することができる。 As the vehicle state, the unlocked state in which the vehicle is unlocked, the state in which the vehicle has started traveling, the preparation state for getting off the vehicle where the driver prepares to get off the vehicle, and the locked state in which the vehicle is locked are determined. Based on the combination of the determination results of these states, the state before the passenger including the driver gets on the vehicle and the state where the driver gets off and the vehicle is locked are determined. Thereby, the various processes mentioned above performed according to the state of a vehicle can be implemented exactly.
　車両状態判定部２３により車両が解錠されて走行が開始されたと判定されたときに、解錠状態と判定されたときにカメラ９で撮像された画像と、走行が開始されたと判定されたときにカメラ９で撮像された画像とを比較することで、車室内にドライバ以外の乗員が存在しているかどうかを判定している。これにより、車室内の乗員は着座した状態となり各座席でのドライバ以外の乗員の有無を確実に判定することができる。 When it is determined by the vehicle state determination unit 23 that the vehicle has been unlocked and traveling has started, and when it has been determined that the vehicle has been unlocked and the image captured by the camera 9 has been determined that traveling has started. In addition, it is determined whether or not an occupant other than the driver is present in the vehicle interior by comparing the image captured by the camera 9. Thereby, the passenger | crew in a vehicle interior will be in the seated state, and the presence or absence of passengers other than the driver in each seat can be determined reliably.
　なお、上記実施形態１、２において、各所定値は、それぞれの所定値を用いた判定処理などで誤判定を招くことなく良好な判定結果が得られるように、例えば実機を用いた実験や机上のシミュレーションなどの結果によって予め定めることができる。
本出願は、２０１２年１月２４日に出願された日本国特許願第２０１２－０１１９４２号に基づく優先権を主張しており、これらの出願の全内容がここに援用される。
以上、本実施形態に沿って本発明の内容を説明したが、本発明はこれらの記載に限定されるものではなく、種々の変形及び改良が可能であることは、当業者には自明である。 In the first and second embodiments, each predetermined value can be obtained by, for example, an experiment using a real machine or a desk so that a good determination result can be obtained without causing erroneous determination in a determination process using the predetermined value. It can be determined in advance according to the result of simulation.
This application claims priority based on Japanese Patent Application No. 2012-011942 filed on January 24, 2012, the entire contents of which are incorporated herein by reference.
The contents of the present invention have been described above according to the present embodiment, but the present invention is not limited to these descriptions, and it is obvious to those skilled in the art that various modifications and improvements can be made. .
　１，２１　車室内残存乗員検出装置
　２，２２　ＥＣＵ
　３　ドアロック検知センサ
　４　アンロック検知センサ
　５　シフトポジションセンサ
　６　車速センサ
　７　シートベルト装着センサ
　８　乗員位置入力スイッチ
　９　カメラ
　１０　ハザードランプ
　１１　ブザー
　１２，２３　車両状態判定部
　１３，２４　乗員有無判定部
　１４，２５　基準画像記憶部
　１５　基準画像診断部
　１６　基準画像
　１７，２６　残存乗員判定部
　１８　報知制御部 1,21 Vehicle interior remaining occupant detection device 2,22 ECU
DESCRIPTION OF SYMBOLS 3 Door lock detection sensor 4 Unlock detection sensor 5 Shift position sensor 6 Vehicle speed sensor 7 Seat belt installation sensor 8 Passenger position input switch 9 Camera 10 Hazard lamp 11 Buzzer 12, 23 Vehicle state determination part 13, 24 Passenger presence determination part 14, 25 Reference Image Storage Unit 15 Reference Image Diagnosis Unit 16 Reference Image 17, 26 Remaining Occupant Determination Unit 18 Notification Control Unit

Claims (9)

  1. 　車両に設けられて、車室内を撮像するカメラと、
    　前記車室内に乗員が存在しているか否かを判定する乗員有無判定手段と、
    　所定の車両状態信号に基づいて、車両状態が、ドライバが降車を準備している降車準備状態であるか、又は前記車両のドアをロックした施錠状態であるかを判定する車両状態判定手段と、
    　前記車両状態判定手段により、前記車両状態が前記降車準備状態であると判定されたときに、前記カメラにより撮像された画像を第１の基準画像として記憶する第１の基準画像記憶手段と、
    　前記車両状態判定手段により前記車両状態が前記施錠状態であると判定され、かつ、前記乗員有無判定手段により前記車室内に乗員が存在していると判定されたとき、前記カメラにより撮像された画像を第１の比較画像として取得する比較画像取得手段と、
    　前記第１の基準画像と前記第１の比較画像とを比較し、前記施錠状態において前記車室内に乗員が残存しているか否かを判定する残存乗員判定手段と、
    　前記残存乗員判定手段により乗員が残存していると判定された場合に、報知する報知制御手段と、
    　を備えたことを特徴とする車室内残存乗員検出装置。     A camera provided in the vehicle for imaging the interior of the vehicle;
    Occupant presence / absence determining means for determining whether an occupant is present in the vehicle interior;
    Vehicle state determination means for determining, based on a predetermined vehicle state signal, whether the vehicle state is an unloading preparation state in which the driver is preparing to get off or a locked state in which the door of the vehicle is locked;
    A first reference image storage unit that stores an image captured by the camera as a first reference image when the vehicle state determination unit determines that the vehicle state is the getting-off preparation state;
    An image captured by the camera when the vehicle state determination means determines that the vehicle state is the locked state and the occupant presence determination means determines that an occupant is present in the vehicle compartment. A comparative image acquisition means for acquiring a first comparative image;
    A remaining occupant determination means that compares the first reference image with the first comparison image and determines whether or not an occupant remains in the passenger compartment in the locked state;
    Notification control means for notifying when it is determined by the remaining occupant determination means that an occupant remains;
    A vehicle interior remaining occupant detection device.
  2. 　前記車両状態判定手段は、前記降車準備状態、前記施錠状態、前記車両のドアロックが解錠された解錠状態、前記車両が走行を開始した走行開始状態、及び前記車両が走行している走行状態のうち、前記車両状態がどの状態にあるのかを判定することを特徴とする請求項１に記載の車室内残存乗員検出装置。 The vehicle state determination means includes the getting-off preparation state, the locked state, the unlocked state in which the door lock of the vehicle is unlocked, the traveling start state in which the vehicle has started traveling, and the traveling in which the vehicle is traveling. The vehicle interior remaining occupant detection device according to claim 1, wherein the vehicle state is determined from among the states.
  3. 　前記車両状態判定手段は、前記車両状態の判定結果の組み合わせに基づいて、前記ドライバを含めた乗員が車両に乗車する前の状態であるか、又は前記ドライバが降車して前記車両のドアをロックした状態であるかを判定することを特徴とする請求項２に記載の車室内残存乗員検出装置。 The vehicle state determination means is in a state before an occupant including the driver gets on the vehicle based on a combination of the determination results of the vehicle state, or the driver gets off and locks the door of the vehicle. 3. The vehicle interior remaining passenger detection device according to claim 2, wherein it is determined whether the vehicle is in a closed state.
  4. 　前記残存乗員判定手段は、
    　前記第１の基準画像と前記第１の比較画像との対応する画素の濃淡差を算出し、
    　所定値以上の濃淡差がある画素数が所定値より大きい場合には、前記カメラにより撮像された新たな画像を前記第１の比較画像として用いて、前記判定を再度行うことを特徴とする請求項１乃至請求項３のいずれか１項に記載の車室内残存乗員検出装置。     The remaining occupant determination means
    Calculating shade differences of corresponding pixels between the first reference image and the first comparison image;
    When the number of pixels having a gray level difference equal to or greater than a predetermined value is greater than a predetermined value, the determination is performed again using a new image captured by the camera as the first comparison image. The vehicle interior remaining occupant detection device according to any one of claims 1 to 3.
  5. 　前記第１の基準画像記憶手段に前記第１の基準画像を記憶する際に、予め設定された適否要件に基づいて、前記第１の基準画像は、前記残存乗員判定手段による前記判定に用いるのに適しているか否かを診断する第１の基準画像診断手段を更に備え、
    　前記第１の基準画像診断手段は、前記第１の基準画像が、前記残存乗員判定手段による前記判定に用いるのに適していないと判定した場合には、前記カメラにより撮像された新たな画像を前記第１の基準画像として取得して、前記診断を再度行ない、
    　前記第１の基準画像記憶手段は、前記第１の基準画像が、前記残存乗員判定手段による前記判定に用いるのに適していると診断した場合には、前記第１の基準画像記憶手段に前記第１の基準画像を記憶することを特徴とする請求項１乃至請求項４のいずれか１項に記載の車室内残存乗員検出装置。     When the first reference image is stored in the first reference image storage unit, the first reference image is used for the determination by the remaining occupant determination unit based on a predetermined suitability requirement. A first reference image diagnostic means for diagnosing whether or not it is suitable for
    When the first reference image diagnosis unit determines that the first reference image is not suitable for use in the determination by the remaining occupant determination unit, a new image captured by the camera is used. Acquiring as the first reference image, performing the diagnosis again,
    When the first reference image storage unit diagnoses that the first reference image is suitable for use in the determination by the remaining occupant determination unit, the first reference image storage unit stores the first reference image storage unit in the first reference image storage unit. The vehicle interior remaining occupant detection device according to any one of claims 1 to 4, wherein the first reference image is stored.
  6. 　前記車両状態判定手段により、前記車両状態が前記解錠状態であると判定されたときに、前記カメラで撮像された画像を第２の基準画像として記憶する第２の基準画像記憶手段を更に備え、
    　前記乗員有無判定手段は、
    　前記車両状態判定手段により、前記車両状態が前記走行開始状態であると判定されたときに、前記カメラで撮像された画像を第２の比較画像として取得して、前記第２の基準画像と前記第２の比較画像とを比較し、前記走行開始状態において前記車室内に乗員が存在しているか否かを判定することを特徴とする請求項２乃至請求項５のいずれか１項に記載の車室内残存乗員検出装置。     When the vehicle state determination unit determines that the vehicle state is the unlocked state, the vehicle state determination unit further includes a second reference image storage unit that stores an image captured by the camera as a second reference image. ,
    The passenger presence determination means
    When the vehicle state determination unit determines that the vehicle state is the travel start state, an image captured by the camera is acquired as a second comparison image, and the second reference image and the The second comparison image is compared with the second comparison image to determine whether or not an occupant is present in the vehicle interior in the traveling start state. A passenger occupant detection device.
  7. 　前記乗員有無判定手段は、
    　前記第２の基準画像と前記第２の比較画像との対応する画素の濃淡差を算出し、
    　所定値以上の濃淡差がある画素数が所定値より大きい場合には、前記カメラにより撮像された新たな画像を前記第２の比較画像として用いて、前記判定を再度行うことを特徴とする請求項６に記載の車室内残存乗員検出装置。     The passenger presence determination means
    Calculating shade differences of corresponding pixels between the second reference image and the second comparison image;
    When the number of pixels having a gray level difference equal to or greater than a predetermined value is greater than a predetermined value, the determination is performed again using a new image captured by the camera as the second comparison image. Item 7. A passenger occupant detection device according to item 6.
  8. 　前記第２の基準画像記憶手段に前記第２の基準画像を記憶する際に、予め設定された適否要件に基づいて、前記第２の基準画像は、前記乗員有無判定手段による前記判定に用いるのに適しているか否かを診断する第２の基準画像診断手段を更に備え、
    　前記第２の基準画像診断手段は、前記第２の基準画像が、前記乗員有無判定手段による前記判定に用いるのに適していないと判定した場合には、前記カメラにより撮像された新たな画像を前記第２の基準画像として取得して、前記診断を再度行ない、
    　前記第２の基準画像記憶手段は、前記第２の基準画像が、前記乗員有無判定手段による前記判定に用いるのに適していると診断した場合には、前記第２の基準画像記憶手段に前記第２の基準画像を記憶することを特徴とする請求項６又は請求項７に記載の車室内残存乗員検出装置。     When the second reference image is stored in the second reference image storage unit, the second reference image is used for the determination by the occupant presence / absence determination unit based on a predetermined suitability requirement. Further comprising a second reference image diagnostic means for diagnosing whether or not it is suitable for
    When the second reference image diagnostic unit determines that the second reference image is not suitable for use in the determination by the occupant presence determination unit, a new image captured by the camera is used. Obtain as the second reference image, perform the diagnosis again,
    When the second reference image storage unit diagnoses that the second reference image is suitable for use in the determination by the occupant presence determination unit, the second reference image storage unit stores the second reference image storage unit in the second reference image storage unit. The vehicle interior remaining occupant detection device according to claim 6 or 7, wherein the second reference image is stored.
  9. 　車室内に乗員が存在しているか否かを判定する工程と、
    　所定の車両状態信号に基づいて、車両状態が、ドライバが降車を準備している降車準備状態であるか、又は前記車両のドアをロックした施錠状態であるかを判定する工程と、
    　前記車両状態が前記降車準備状態であると判定されたときに、前記車室内を撮像して基準画像として記憶する工程と、
    　前記車両状態が前記施錠状態であると判定され、かつ、前記車室内に乗員が存在していると判定されたときに、前記車室内を撮像して比較画像として取得する工程、
    　前記基準画像と前記比較画像とを比較し、前記施錠状態において前記車室内に乗員が残存しているか否かを判定する工程と、
    　前記車室内に乗員が残存していると判定された場合に、報知する工程と、
    　を備えたことを特徴とする車室内残存乗員検出方法。     Determining whether an occupant is present in the passenger compartment;
    Based on a predetermined vehicle state signal, determining whether the vehicle state is a getting-off preparation state where the driver is preparing to get off, or a locked state where the door of the vehicle is locked;
    Capturing the vehicle interior and storing it as a reference image when it is determined that the vehicle state is the getting-off preparation state;
    When it is determined that the vehicle state is the locked state and it is determined that an occupant is present in the vehicle interior, the vehicle interior is imaged and acquired as a comparison image;
    Comparing the reference image with the comparative image and determining whether or not an occupant remains in the vehicle interior in the locked state;
    A step of notifying when it is determined that an occupant remains in the passenger compartment;
    A vehicle interior residual occupant detection method comprising:
PCT/JP2013/051185 2012-01-24 2013-01-22 Device and method for detecting remaining occupant in vehicle interior WO2013111745A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012-011942 2012-01-24
JP2012011942 2012-01-24

Publications (1)

Publication Number Publication Date
WO2013111745A1 true WO2013111745A1 (en) 2013-08-01

Family

ID=48873465

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/051185 WO2013111745A1 (en) 2012-01-24 2013-01-22 Device and method for detecting remaining occupant in vehicle interior

Country Status (1)

Country Link
WO (1) WO2013111745A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017122350A (en) * 2016-01-07 2017-07-13 株式会社東海理化電機製作所 Door lock device for vehicle
WO2021084816A1 (en) * 2019-10-31 2021-05-06 株式会社Ｊｖｃケンウッド Recording system, recording method, and program
EP3819171A4 (en) * 2018-07-04 2021-07-14 Mitsubishi Electric Corporation Notification target detection device, alarm system, and notification target detection method
US11361574B2 (en) 2019-10-23 2022-06-14 Bendix Commercial Vehicle Systems Llc System and method for monitoring for driver presence and position using a driver facing camera

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000289459A (en) * 1999-04-09 2000-10-17 Matsushita Electric Ind Co Ltd Cabin abnormality detection device
JP2002063668A (en) * 2000-06-07 2002-02-28 Toshiba Corp Device for detecting and reporting person inside vehicle and dangerous state avoiding device
JP2004346708A (en) * 2003-05-26 2004-12-09 Sumitomo Electric Ind Ltd Lock-in prevention system
JP2005050067A (en) * 2003-07-31 2005-02-24 Fujitsu Ten Ltd Vehicle controller
WO2011014554A1 (en) * 2009-07-29 2011-02-03 Davy Zide Qian Driver alarm for preventing children from being left in car

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000289459A (en) * 1999-04-09 2000-10-17 Matsushita Electric Ind Co Ltd Cabin abnormality detection device
JP2002063668A (en) * 2000-06-07 2002-02-28 Toshiba Corp Device for detecting and reporting person inside vehicle and dangerous state avoiding device
JP2004346708A (en) * 2003-05-26 2004-12-09 Sumitomo Electric Ind Ltd Lock-in prevention system
JP2005050067A (en) * 2003-07-31 2005-02-24 Fujitsu Ten Ltd Vehicle controller
WO2011014554A1 (en) * 2009-07-29 2011-02-03 Davy Zide Qian Driver alarm for preventing children from being left in car

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017122350A (en) * 2016-01-07 2017-07-13 株式会社東海理化電機製作所 Door lock device for vehicle
WO2017119444A1 (en) * 2016-01-07 2017-07-13 株式会社東海理化電機製作所 Door lock device for vehicles
EP3819171A4 (en) * 2018-07-04 2021-07-14 Mitsubishi Electric Corporation Notification target detection device, alarm system, and notification target detection method
US11361574B2 (en) 2019-10-23 2022-06-14 Bendix Commercial Vehicle Systems Llc System and method for monitoring for driver presence and position using a driver facing camera
WO2021084816A1 (en) * 2019-10-31 2021-05-06 株式会社Ｊｖｃケンウッド Recording system, recording method, and program
JP2021072559A (en) * 2019-10-31 2021-05-06 株式会社Ｊｖｃケンウッド Recording system, recording method and program
JP7354771B2 (en) 2019-10-31 2023-10-03 株式会社Ｊｖｃケンウッド Recording system, recording method and program
US11877099B2 (en) 2019-10-31 2024-01-16 Jvckenwood Corporation Recording system, recording method, and program

Similar Documents

Publication Publication Date Title
US11473353B2 (en) Vehicle door apparatus to restrict opening of a door
JP4858381B2 (en) Car left behind prevention device
US20070195990A1 (en) Vision-Based Seat Belt Detection System
JP5059551B2 (en) Vehicle occupant detection device
WO2013111745A1 (en) Device and method for detecting remaining occupant in vehicle interior
JP2009241857A (en) Reporting system for vehicle
JP6772786B2 (en) Crew detection device and occupant detection program
US20140297133A1 (en) Automatic child lock device
US11498457B2 (en) Vehicle mounted system, vehicle control system and method for monitoring an infant in a vehicle cabin
US20210053528A1 (en) Wheelchair securing structure
KR20210032766A (en) Vehicle and control method for the same
JP2016199207A (en) Seat belt warning device
JP4144448B2 (en) Vehicle rear display device
JP2017165218A (en) Display control device for vehicle, display system for vehicle, display control method for vehicle, and program
JP2014040198A (en) Occupant detection device for vehicle and occupant detection method for vehicle
US7463166B2 (en) Control system for the operation of presence detection devices and method therefor
JP2014223886A (en) Vehicle cabin monitoring device
US11146784B2 (en) Abnormality detection device and abnormality detection method
CN108482302B (en) Safety belt state reminding system, reminding control method and vehicle
JP2018134885A (en) Vehicular monitoring device
KR20200038097A (en) Neglect prevention system of Infant and stuff in vehicle and neglect prevention method thereof
JP2019074965A (en) Device and system for detecting driving incapacity
JP2019074964A (en) Driving disabled condition prediction device and driving disabled condition prediction system
WO2013133075A1 (en) Vehicle-use reporting device and vehicle-use reporting method
KR20200087107A (en) Neglect prevention system of Infant and stuff in vehicle and neglect prevention method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13741469

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13741469

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP