WO2024115067A2 - A 360-degrees motor vehicle monitoring system - Google Patents

A 360-degrees motor vehicle monitoring system Download PDF

Info

Publication number
WO2024115067A2
WO2024115067A2 PCT/EP2023/081237 EP2023081237W WO2024115067A2 WO 2024115067 A2 WO2024115067 A2 WO 2024115067A2 EP 2023081237 W EP2023081237 W EP 2023081237W WO 2024115067 A2 WO2024115067 A2 WO 2024115067A2
Authority
WO
WIPO (PCT)
Prior art keywords
camera assembly
passenger cabin
operable
vehicle
motor vehicle
Prior art date
Application number
PCT/EP2023/081237
Other languages
French (fr)
Inventor
Weiquan CONG
Original Assignee
Continental Automotive Technologies GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive Technologies GmbH filed Critical Continental Automotive Technologies GmbH
Publication of WO2024115067A2 publication Critical patent/WO2024115067A2/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/29—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/102—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using 360 degree surveillance camera system
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/103—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using camera systems provided with artificial illumination device, e.g. IR light source
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8006—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo

Definitions

  • This disclosure relates to a motor vehicle monitoring system, and more in particular a 360 degrees onboard monitoring system.
  • a purpose of this disclosure is to ameliorate the problem of multiple monitoring systems in a motor vehicle, by providing the subject-matter of the independent claims. Further purposes of this disclosure are set out in the accompanying dependent claims.
  • a 360-degrees image sensor system for monitoring at least an area of a passenger cabin and simultaneously at least partially an area outside the passenger cabin.
  • the image sensor monitoring system may comprise a camera assembly.
  • the camera assembly may comprise a fisheye lens.
  • the camera assembly may further comprise a RGB-IR sensor.
  • An RGB-IR image sensor is a sensor which is operable to generate video images from electromagnetic waves in the RGB spectrum and IR spectrum. Such sensors are known in the prior art.
  • the camera assembly may further comprise an illumination source operable to illuminate in near infrared (NIR) spectrum.
  • NIR near infrared
  • the motor vehicle monitoring system disclosed herein eliminates the need for additional dash camera and/or surround-view monitoring system, thereby achieving reduction of number of monitoring systems in a motor vehicle.
  • the use of an illumination light source operable in a NIR spectrum provide unobtrusive, remote sensing of a person’s presence, pupil position and gaze, eyelid movement, and other indicators of driver alertness and occupant safety, without distracting the driver.
  • the system may further comprise a processing unit operable to convert fisheye projection images into rectilinear projection video images.
  • the system may further comprise a processing unit operable to perform segmentation of the video images. Segmentation of video images is a process that saves videos at a pre-determined set of lengths, for example in sets of 3 minutes, 5 minutes, 10 minutes, etc, instead of one continues video clip. The advantage of performing segmentation of video images reduces data and computational power.
  • the system may further comprise a processing unit operable to execute processing steps for auxiliary applications or combination thereof.
  • vehicle monitoring system is improved through sensing and data fusion using a single monitoring system for multiple automotive applications.
  • the system may further comprise an inertial measurement unit operable to measure a tilting angle of the integrated camera assembly.
  • the system may further comprise a display device operable to display video images.
  • the camera assembly may be integrated with display device as a single monitoring system.
  • the camera assembly may be incorporated into a display device of a single integrated monitoring system, to solve issue of space constrains.
  • the display device may be used to display image videos and information for other vehicular applications, for example displaying a field of view outside of the motor vehicle.
  • the camera assembly may be mounted on a windshield of a motor vehicle.
  • the camera assembly may be mounted on at least a side of a rearview mirror of a motor vehicle.
  • FIG. 1A shows an exemplary aspect of a 360-degree motor vehicle monitoring system implemented on a motor vehicle.
  • FIG. 1 B shows an exemplary aspect of a 360-degree motor vehicle monitoring system implemented on a motor vehicle.
  • FIG. 2A shows an exemplary integrated 360-degree motor vehicle monitoring system.
  • FIG. 2B shows an exemplary 360-degree motor vehicle monitoring system.
  • FIG. 1A and FIG. 1 B shows a 360-degree image sensor system 100a, 100bin accordance with a preferred aspect of this disclosure.
  • the system 100a shown in FIG. 1A includes a camera assembly 102, which is mounted on at least one side of a rearview mirror 104 of a motor vehicle 120. In a typical set up within the rearview mirror is mounted on a windshield 108.
  • the camera assembly 102 includes a fisheye lens (FIG. 2A and FIG. 2B referred) having a field of view (FOV) of more than 180° with an optical axis direction towards at least an area of the passenger cabin 122 and simultaneously at least partially outside a radius of a motor vehicle 120.
  • FOV field of view
  • the fisheye lens has a FOV of 120° or more, preferable 170° or more, more preferable 180° to 360°, even more preferable 190° to 270°, with a suitable image sensor, such as a RGB-IR image sensor, the camera assembly 102 will be able to achieve 360° detection, to double up as a surround-view camera to monitoring surrounding of the motor vehicle 120 and/or a dash camera (dashcam) for surveillance of the motor vehicle 120.
  • a suitable image sensor such as a RGB-IR image sensor
  • An advantage of using a RGB-IR sensor allows a single image sensor system to capture a single image and process the single image captured to produce two separate images, one video image in a RGB spectrum and one video image in an IR spectrum. More advantageously, producing images in IR spectrum allows the monitoring system to identify thermal areas in the video images, thus distinguishing between living beings seated within the passenger cabin 122 and objects, for example chair
  • FIG. 1 B shows an exemplary aspect of a 360-degree motor vehicle monitoring system 100b implemented on a motor vehicle 120.
  • the camera assembly 102 is mounted on a windshield 108.
  • the camera assembly 102 includes a fisheye lens (FIG. 2A and FIG. 2B referred) having a field of view a field of view towards at least an area of the passenger cabin and at least partially outside a radius of a motor vehicle 120.
  • the fisheye lens has a field of view of 120° or more, preferable 170° or more, more preferable 180° to 360°, even more preferable 190° to 270°.
  • the camera assembly 102 is operable to capture images for different zones. This may be achieved by differentiating field of illumination for different zones.
  • the camera assembly 102 has a working wavelength range including visible light and near infrared light. This working wavelength range may be between 380nm to 1000 nm. Referring back to FIG. 1A and FIG.
  • the zones include a dashcam zone 110 operable to surveillance a front view of the motor vehicle 120, an exterior monitoring zone 112 operable to function as a surround-view monitoring camera relative to a motor vehicle 120, a driver monitoring zone 114 operable to monitor a states of the driver within a motor vehicle 120 and a cabin monitoring zone 114 operable to monitor a passenger cabin 122 of a motor vehicle 120.
  • the camera assembly 102 may further include a processing unit 106 (FIG. 2A, FIG. 2B referred).
  • the processing unit 106 may be operable to convert fisheye projection images into rectilinear projection video images.
  • the processing unit 106 may be operable to perform segmentation of the video images.
  • processing unit 106 may be operable to execute processing steps for auxiliary applications.
  • An advantage of using processing unit 106 to execute such functions improves the system 100a through sensing and data fusion using a single monitoring system for multiple automotive applications.
  • an electronic control unit 214 may be embedded in the rearview mirror 108.
  • Applicable auxiliary applications may include video images captured by the camera assembly 102 in the dashcam zone 110 and/or the exterior monitoring zone 112 in the format of converted rectilinear projection video images captured by the camera assembly 102.
  • FIG. 2A shows an exemplary integrated 360-degree motor vehicle monitoring system 200a in accordance with a preferred aspect of this disclosure.
  • the system 200a shown here incorporate the camera assembly 102 with a display device 202, connected by a display connection 204.
  • An advantage of integrating the camera assembly within a display device 202 has the advantage of a field of view of the entire passenger cabin.
  • the display connection 204 may be a hardware element, for example physical wire connections, or software element, for example transmission of data information between different hardware modules.
  • the camera assembly includes a fisheye lens 212 and an image sensor 210.
  • a suitable image sensor 210 will be RGB-IR sensor, operable to generate video images in RGB spectrum and video images in IR spectrum.
  • the camera assembly 102 further includes an illumination source 206 operable to illuminate in a near infrared (NIR) spectrum.
  • NIR near infrared
  • Suitable illumination source may be vertical cavity surface emitting laser (VCSEL) or light emitting diode (LED).
  • the illumination source 206 may be driven by an illumination driver 208.
  • the illumination source 206 may include an illumination lens to adjust the field of illumination and illumination profile, which provides a map of the intensity of light across the entire field of view of the lens.
  • FIG. 2B shows an exemplary 360-degree motor vehicle monitoring system 200b in accordance with a preferred aspect of this disclosure.
  • an electronic control unit (ECU) 214 is connected to the camera assembly 102 and display device 202.
  • Communication between ECU 214 uses Serializer/Deserializer high-speed communications.
  • Serializer/Deserializer is a pair of functional blocks used to convert data between serial data and parallel interfaces in each direction.
  • the 360-degrees motor vehicle monitoring system may include an inertial measurement unit (IMU) operable to measure the tilting angle of the camera assembly 102 for measurement of potential compensation value.
  • IMU inertial measurement unit
  • the 360-degrees motor vehicle monitoring system may further include a read-only-memory (ROM) for image sensor 210 and lens calibration data storage.
  • ROM read-only-memory
  • An example of data to be stored may be lens parameters.
  • the camera assembly may include an optical filter as cover lens.
  • the visible light neural density filter and long pass filter may be operable to compensate mismatch between active NIR illumination and ambient illumination. It reduces the visibility to inner side of the cover, to have better appearance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)

Abstract

A 360-degrees motor vehicle monitoring system is disclosed herein. The motor vehicle monitoring system comprises a camera assembly. The camera assembly comprises a fisheye lens. The camera assembly further comprises an image sensor, more specifically a RGB-IR sensor, operable to generate video images in RGB spectrum and video images in IR spectrum. The camera assembly further comprises an illumination source operable to illuminate in near infrared (NIR) spectrum. In particular, the camera assembly has a field of view operable to monitor at least an area of a passenger cabin and at least partially outside a radius of a motor vehicle.

Description

A 360-DEGREES MOTOR VEHICLE MONITORING SYSTEM
TECHNICAL FIELD
This disclosure relates to a motor vehicle monitoring system, and more in particular a 360 degrees onboard monitoring system.
BACKGROUND
Increasingly, the automotive industry is using camera systems to either monitor a surrounding of the vehicle, or for in-vehicle monitoring purposes. Conventional vehicle monitoring systems either surveillance within a passenger compartment for driver or cabin monitoring purposes or surveillance outside the motor vehicle. Some examples include dash cameras and surround-view monitoring.
However, separate camera solutions are required for a single vehicle to have both interior and exterior surveillance purposes. Having multiple monitoring systems increases the difficulty of fusion of images for different types of automotive applications.
The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.
SUMMARY
A purpose of this disclosure is to ameliorate the problem of multiple monitoring systems in a motor vehicle, by providing the subject-matter of the independent claims. Further purposes of this disclosure are set out in the accompanying dependent claims.
In an aspect of this disclosure, a 360-degrees image sensor system for monitoring at least an area of a passenger cabin and simultaneously at least partially an area outside the passenger cabin is provided. The image sensor monitoring system may comprise a camera assembly. The camera assembly may comprise a fisheye lens. The camera assembly may further comprise a RGB-IR sensor. An RGB-IR image sensor is a sensor which is operable to generate video images from electromagnetic waves in the RGB spectrum and IR spectrum. Such sensors are known in the prior art. The camera assembly may further comprise an illumination source operable to illuminate in near infrared (NIR) spectrum. An advantage of the above-described aspect of this disclosure yields a single 360-degree camera assembly operable to monitor both an interior of a motor vehicle and simultaneously at least partially an area outside the passenger cabin of the motor vehicle. The motor vehicle monitoring system disclosed herein eliminates the need for additional dash camera and/or surround-view monitoring system, thereby achieving reduction of number of monitoring systems in a motor vehicle. The use of an illumination light source operable in a NIR spectrum provide unobtrusive, remote sensing of a person’s presence, pupil position and gaze, eyelid movement, and other indicators of driver alertness and occupant safety, without distracting the driver.
The system may further comprise a processing unit operable to convert fisheye projection images into rectilinear projection video images. The system may further comprise a processing unit operable to perform segmentation of the video images. Segmentation of video images is a process that saves videos at a pre-determined set of lengths, for example in sets of 3 minutes, 5 minutes, 10 minutes, etc, instead of one continues video clip. The advantage of performing segmentation of video images reduces data and computational power. The system may further comprise a processing unit operable to execute processing steps for auxiliary applications or combination thereof. Advantageously, vehicle monitoring system is improved through sensing and data fusion using a single monitoring system for multiple automotive applications. The system may further comprise an inertial measurement unit operable to measure a tilting angle of the integrated camera assembly.
The system may further comprise a display device operable to display video images. The camera assembly may be integrated with display device as a single monitoring system. Advantageously, the camera assembly may be incorporated into a display device of a single integrated monitoring system, to solve issue of space constrains. The display device may be used to display image videos and information for other vehicular applications, for example displaying a field of view outside of the motor vehicle.
The camera assembly may be mounted on a windshield of a motor vehicle. The camera assembly may be mounted on at least a side of a rearview mirror of a motor vehicle.
Other objects, features and characteristics, as well as the methods of operation and the functions of the related elements of the structure, the combination of parts and economics of manufacture will become more apparent upon consideration of the following detailed description and appended claims with reference to the accompanying drawings, all of which form a part of this specification. It should be understood that the detailed description and specific examples, while indicating the non-limiting embodiments of the disclosure, are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.
BRIEF DESCRIPTION OF DRAWINGS
The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein:
FIG. 1A shows an exemplary aspect of a 360-degree motor vehicle monitoring system implemented on a motor vehicle. FIG. 1 B shows an exemplary aspect of a 360-degree motor vehicle monitoring system implemented on a motor vehicle.
FIG. 2A shows an exemplary integrated 360-degree motor vehicle monitoring system.
FIG. 2B shows an exemplary 360-degree motor vehicle monitoring system.
DETAILED DESCRIPTION
It should be understood that like reference numerals identify corresponding or similar elements throughout the several drawings. It should be understood that although a particular component arrangement is disclosed and illustrated in these exemplary embodiments, other arrangements could also benefit from the teachings of this disclosure.
FIG. 1A and FIG. 1 B shows a 360-degree image sensor system 100a, 100bin accordance with a preferred aspect of this disclosure. The system 100a shown in FIG. 1A includes a camera assembly 102, which is mounted on at least one side of a rearview mirror 104 of a motor vehicle 120. In a typical set up within the rearview mirror is mounted on a windshield 108. The camera assembly 102 includes a fisheye lens (FIG. 2A and FIG. 2B referred) having a field of view (FOV) of more than 180° with an optical axis direction towards at least an area of the passenger cabin 122 and simultaneously at least partially outside a radius of a motor vehicle 120. The fisheye lens has a FOV of 120° or more, preferable 170° or more, more preferable 180° to 360°, even more preferable 190° to 270°, with a suitable image sensor, such as a RGB-IR image sensor, the camera assembly 102 will be able to achieve 360° detection, to double up as a surround-view camera to monitoring surrounding of the motor vehicle 120 and/or a dash camera (dashcam) for surveillance of the motor vehicle 120. An advantage of using a RGB-IR sensor allows a single image sensor system to capture a single image and process the single image captured to produce two separate images, one video image in a RGB spectrum and one video image in an IR spectrum. More advantageously, producing images in IR spectrum allows the monitoring system to identify thermal areas in the video images, thus distinguishing between living beings seated within the passenger cabin 122 and objects, for example chair
FIG. 1 B shows an exemplary aspect of a 360-degree motor vehicle monitoring system 100b implemented on a motor vehicle 120. In this configuration, the camera assembly 102 is mounted on a windshield 108. Similar to the system 100a shown in FIG. 1A, the camera assembly 102 includes a fisheye lens (FIG. 2A and FIG. 2B referred) having a field of view a field of view towards at least an area of the passenger cabin and at least partially outside a radius of a motor vehicle 120. Preferably, the fisheye lens has a field of view of 120° or more, preferable 170° or more, more preferable 180° to 360°, even more preferable 190° to 270°.
As shown in FIG. 1A and FIG. 1 B, the camera assembly 102 is operable to capture images for different zones. This may be achieved by differentiating field of illumination for different zones. Preferably, the camera assembly 102 has a working wavelength range including visible light and near infrared light. This working wavelength range may be between 380nm to 1000 nm. Referring back to FIG. 1A and FIG. 1 B, the zones include a dashcam zone 110 operable to surveillance a front view of the motor vehicle 120, an exterior monitoring zone 112 operable to function as a surround-view monitoring camera relative to a motor vehicle 120, a driver monitoring zone 114 operable to monitor a states of the driver within a motor vehicle 120 and a cabin monitoring zone 114 operable to monitor a passenger cabin 122 of a motor vehicle 120.
Thus, it can be seen, a main advantage of the aforesaid configuration yields a single 360-degree camera assembly operable to monitor both an interior of a motor vehicle and an exterior of the motor vehicle. The motor vehicle monitoring system disclosed herein eliminates the need for additional dash camera and/or surround-view monitoring system, thereby achieving reduction of number of monitoring systems in a motor vehicle. Therefore, an image sensor system proposed herein allows monitoring a vehicular environment with less power consumption. The camera assembly 102 may further include a processing unit 106 (FIG. 2A, FIG. 2B referred). The processing unit 106 may be operable to convert fisheye projection images into rectilinear projection video images. The processing unit 106 may be operable to perform segmentation of the video images. Further, the processing unit 106 may be operable to execute processing steps for auxiliary applications. An advantage of using processing unit 106 to execute such functions improves the system 100a through sensing and data fusion using a single monitoring system for multiple automotive applications. Optionally an electronic control unit 214 may be embedded in the rearview mirror 108. Applicable auxiliary applications may include video images captured by the camera assembly 102 in the dashcam zone 110 and/or the exterior monitoring zone 112 in the format of converted rectilinear projection video images captured by the camera assembly 102.
FIG. 2A shows an exemplary integrated 360-degree motor vehicle monitoring system 200a in accordance with a preferred aspect of this disclosure. The system 200a shown here incorporate the camera assembly 102 with a display device 202, connected by a display connection 204. An advantage of integrating the camera assembly within a display device 202 has the advantage of a field of view of the entire passenger cabin. The display connection 204 may be a hardware element, for example physical wire connections, or software element, for example transmission of data information between different hardware modules. The camera assembly includes a fisheye lens 212 and an image sensor 210. A suitable image sensor 210 will be RGB-IR sensor, operable to generate video images in RGB spectrum and video images in IR spectrum. The camera assembly 102 further includes an illumination source 206 operable to illuminate in a near infrared (NIR) spectrum. The selection of illumination in a NIR spectrum allows the 360-degrees motor vehicle monitoring system to switch between driver monitoring systems application and/or cabin monitoring systems application by controlling the field of illumination, to differentiate the driver monitoring zone 114 and the cabin monitoring zone 116. Suitable illumination source may be vertical cavity surface emitting laser (VCSEL) or light emitting diode (LED). The illumination source 206 may be driven by an illumination driver 208. Optionally, the illumination source 206 may include an illumination lens to adjust the field of illumination and illumination profile, which provides a map of the intensity of light across the entire field of view of the lens.
FIG. 2B shows an exemplary 360-degree motor vehicle monitoring system 200b in accordance with a preferred aspect of this disclosure. In this system 200n, an electronic control unit (ECU) 214 is connected to the camera assembly 102 and display device 202. Communication between ECU 214 uses Serializer/Deserializer high-speed communications. Serializer/Deserializer is a pair of functional blocks used to convert data between serial data and parallel interfaces in each direction.
Optionally, the 360-degrees motor vehicle monitoring system may include an inertial measurement unit (IMU) operable to measure the tilting angle of the camera assembly 102 for measurement of potential compensation value.
Optionally, the 360-degrees motor vehicle monitoring system may further include a read-only-memory (ROM) for image sensor 210 and lens calibration data storage. An example of data to be stored may be lens parameters.
Optionally, the camera assembly may include an optical filter as cover lens. The visible light neural density filter and long pass filter (LPF) may be operable to compensate mismatch between active NIR illumination and ambient illumination. It reduces the visibility to inner side of the cover, to have better appearance.
The foregoing description shall be interpreted as illustrative and not be limited thereto. One of ordinary skill in the art would understand that certain modifications may come within the scope of this disclosure. Although the different non-limiting embodiments are illustrated as having specific components or steps, the embodiments of this disclosure are not limited to those combinations. Some of the components or features from any of the non-limiting embodiments may be used in combination with features or components from any of the other non-limiting embodiments. For these reasons, the appended claims should be studied to determine the true scope and content of this disclosure. List of Reference Signs
Figure imgf000010_0001

Claims

Patent claims
1 . A 360-degrees image sensor system (100a, 100b) for monitoring at least an area of a passenger cabin (122) and simultaneously at least partially an area outside the passenger cabin (122), comprising: a camera assembly (102) comprising: a fisheye lens (212); a RGB-IR image sensor (210); and an illumination source (206) operable to illuminate in near infrared (NIR) spectrum.
2. The system (100a, 100b) of claim 1 , wherein the system (100a, 100b) further comprises a processing unit (106) operable to: convert fisheye projection images into rectilinear projection video images; perform segmentation of the video images execute processing steps for auxiliary applications or combination thereof.
3. The system (100a, 100b) of claims 1 - 2, wherein the system (100a, 100b) further comprises an inertial measurement unit operable to measure a tilting angle of the integrated camera assembly.
4. The system (100a, 100b) of claims 1 -3, wherein the system (100a, 100b) further comprises a display device (202) operable to display video images.
5. The system (100a, 100b) of claims 1 - 4, wherein the camera assembly (102) is operable to monitor one or more occupants within the passenger cabin (122) in response to a field of illumination.
6. The system (100a, 100b) according to any one of the preceding claims, wherein the camera assembly (102) is integrated with the display device (202) as a single monitoring system.
7. The system (100a, 100b) according to any one of the preceding claims, wherein the fisheye lens (212) has a field of view of 120° or more, preferable 170° or more, more preferable 180° to 360°, even more preferable 190° to 270°.
8. The system (100a, 100b) according to any one of the preceding claims, wherein the auxiliary applications comprises: dashcam zone (110); and/or exterior monitoring zone (112).
9. A vehicle (120) comprising a passenger cabin (122) and a system (100a, 100b) according to any one of the preceding claims, wherein the camera assembly (102) has a field of view of at least an area of a passenger cabin (122); and at least partially outside the vehicle (120).
10. The vehicle (120) according to the preceding claim 8, wherein the camera assembly (102) is mounted on a windshield of the vehicle (120).
11. The vehicle (120) according to claim 8, wherein the camera assembly (102) is mounted on at least a side of rearview mirror of a vehicle (120).
12. A method for 360-degrees monitoring for monitoring at least an area of a passenger cabin (122) and simultaneously at least partially an area outside the passenger cabin (122), comprising the steps of: a. capturing images with the system (100a, 100b) according to any one of the preceding claims 1-7 while the system (100a, 100b) is mounted in a passenger cabin (122) of the vehicle; and b. detect objects inside the passenger cabin (122) and/or outside the vehicle (120).
PCT/EP2023/081237 2022-11-28 2023-11-09 A 360-degrees motor vehicle monitoring system WO2024115067A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB2217773.7A GB2624689A (en) 2022-11-28 2022-11-28 A 360-degrees motor vehicle monitoring system
GB2217773.7 2022-11-28

Publications (1)

Publication Number Publication Date
WO2024115067A2 true WO2024115067A2 (en) 2024-06-06

Family

ID=84889458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2023/081237 WO2024115067A2 (en) 2022-11-28 2023-11-09 A 360-degrees motor vehicle monitoring system

Country Status (2)

Country Link
GB (1) GB2624689A (en)
WO (1) WO2024115067A2 (en)

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6704434B1 (en) * 1999-01-27 2004-03-09 Suzuki Motor Corporation Vehicle driving information storage apparatus and vehicle driving information storage method
JP2001045438A (en) * 1999-08-04 2001-02-16 Suzuki Motor Corp Image processing method and image processor
JP3945098B2 (en) * 1999-10-29 2007-07-18 スズキ株式会社 Drive recorder
JP2002053080A (en) * 2000-06-01 2002-02-19 Nippon Lsi Card Co Ltd Device and system for monitoring internal and external situation of automobile, and safe driving attesting method using it
JP2002203254A (en) * 2000-08-30 2002-07-19 Usc Corp Curved surface image transforming method and recording medium with the curved surface image transforming method recorded thereon
JP2013082354A (en) * 2011-10-11 2013-05-09 Koito Mfg Co Ltd Interior light unit for vehicle

Also Published As

Publication number Publication date
GB2624689A (en) 2024-05-29
GB202217773D0 (en) 2023-01-11

Similar Documents

Publication Publication Date Title
US11465561B2 (en) Interior rearview mirror assembly with driver monitoring system
JP5235876B2 (en) Vehicle vision system
US20180015879A1 (en) Side-view mirror camera system for vehicle
CN100413324C (en) On-vehicle night vision camera system, display device and display method
US20150294169A1 (en) Vehicle vision system with driver monitoring
KR102414041B1 (en) Rearview mirror assembly with occupant detection
JP6441360B2 (en) Display system for displaying an image acquired by a camera system on a rearview mirror assembly of a vehicle
US9168869B1 (en) Vehicle with a multi-function auxiliary control system and heads-up display
JP5093611B2 (en) Vehicle periphery confirmation device
JP2014208529A (en) Mirror alternative system for vehicle
EP1386786A1 (en) In-vehicle occupant image pickup apparatus
US20150124097A1 (en) Optical reproduction and detection system in a vehicle
US20230078512A1 (en) Overhead console accessory system with shared controls, cameras, and lighting
CN108136964B (en) Shooting device with mirror
US20220089092A1 (en) Mirror assembly with passenger camera
EP3166307B1 (en) Capturing device for a motor vehicle, driver assistance system as well as motor vehicle
JP2019110518A (en) Imaging device and imaging system
WO2024115067A2 (en) A 360-degrees motor vehicle monitoring system
US11590893B2 (en) Vehicle electronic mirror system
CN110789455B (en) Outside rear-view mirror and vehicle
JP7037896B2 (en) Visual assistance device
US20240217437A1 (en) Vehicular driver monitoring system with driver monitoring camera and near ir light emitter at interior rearview mirror assembly
CN108859964A (en) A kind of vehicle mirrors warning system
US20180065452A1 (en) Using vision devices and techniques to control sunlight blocking devices for vehicles
US20240190351A1 (en) Vehicular driver monitoring system