WO2022227028A1 - 车载显示装置的控制方法、控制装置、电子设备、车辆 - Google Patents
车载显示装置的控制方法、控制装置、电子设备、车辆 Download PDFInfo
- Publication number
- WO2022227028A1 WO2022227028A1 PCT/CN2021/091580 CN2021091580W WO2022227028A1 WO 2022227028 A1 WO2022227028 A1 WO 2022227028A1 CN 2021091580 W CN2021091580 W CN 2021091580W WO 2022227028 A1 WO2022227028 A1 WO 2022227028A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- passenger
- passenger seat
- vehicle
- occupant
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000012545 processing Methods 0.000 claims description 55
- 238000004590 computer program Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 3
- 230000001953 sensory effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 56
- 230000006870 function Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 22
- 230000004927 fusion Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 238000013475 authorization Methods 0.000 description 7
- 230000009977 dual effect Effects 0.000 description 7
- 230000000052 comparative effect Effects 0.000 description 6
- 230000001815 facial effect Effects 0.000 description 5
- 239000006260 foam Substances 0.000 description 4
- 238000004378 air conditioning Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
- B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/80—Arrangements for controlling instruments
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
- B60K35/85—Arrangements for transferring vehicle- or driver-related data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/16—Type of output information
- B60K2360/164—Infotainment
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/16—Type of output information
- B60K2360/166—Navigation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/18—Information management
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/55—Remote control arrangements
- B60K2360/56—Remote control arrangements using mobile devices
- B60K2360/563—Vehicle displaying mobile device information
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/55—Remote control arrangements
- B60K2360/56—Remote control arrangements using mobile devices
- B60K2360/566—Mobile devices displaying vehicle information
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/589—Wireless data transfers
- B60K2360/5911—Bluetooth
Definitions
- the present application relates to the technical field of smart cars, and in particular, to a control method, a control device, an electronic device, a vehicle, a computing device, a computer-readable storage medium, and the like of a vehicle-mounted display device.
- the existing in-vehicle display methods are mainly designed for the needs of the driver, and it is difficult to take into account the experience of other passengers in the vehicle.
- the camera is used to photograph the interior of the car to obtain image information in the car, and faces are detected according to the image information in the car to detect the occupants in the car.
- the display screen of the vehicle-mounted display device is divided into two for display (ie, split screen), so that one display device can be used by both the driver and the passenger, and the two can be used independently of each other. Influence, improve the use experience of the vehicle.
- Embodiments of the present application provide a control method, a control device, an electronic device, a vehicle, a computing device, a computer-readable storage medium, and the like for an in-vehicle display device that can more appropriately perform screen splitting.
- a first aspect of the present application provides a method for controlling a vehicle-mounted display device, including: acquiring first sensing information and acquiring second sensing information; the first sensing information and the second sensing information are respectively used to determine the seat of the first passenger. Information and the second passenger's riding information; when the first passenger's riding information indicates that there is an occupant in the passenger's seat, and the second passenger's riding information indicates that there is a passenger in the passenger's seat, a split-screen display instruction is sent.
- sending a split-screen display command can reduce the occurrence of split-screen misexecution caused by using a single sensor information, so that split-screen display can be performed more appropriately.
- the first sensing information includes in-vehicle image information
- the second sensing information includes pressure sensing information, infrared sensing information, millimeter wave sensing information, and sound sensing information. one or more of.
- the in-vehicle image information, pressure sensing information, infrared sensing information, millimeter wave sensing information and sound sensing information here can be understood as information obtained through image sensors, pressure sensors, infrared sensors, millimeter wave radars, and sound sensors.
- the first sensing information is in-vehicle image information
- the above-mentioned control method is adopted.
- the first passenger seat information indicates that there is an occupant in the passenger seat.
- the sensor in the passenger seat does not detect the corresponding sensing information.
- the pressure sensor cannot detect pressure. Therefore, , the second passenger riding information indicating that there is an occupant in the passenger seat will not be generated, so it is finally determined that there is no passenger in the passenger seat, and the display device will not be displayed in a split screen.
- the second passenger seat information indicates that there is an occupant in the passenger seat due to a heavy object placed on the passenger seat
- the passenger seat is not detected through the in-vehicle image information. Since there is an occupant, the first passenger ride information indicating that there is an occupant in the passenger seat is not generated, so it is finally determined that there is no occupant in the passenger seat, and the display device is not split.
- the first passenger's riding information obtained according to the first sensing information firstly, whether there is a person can be detected by the face feature in the first sensing information, and whether there is a person can also be detected by the body feature.
- the first sensing information may be in-vehicle image information obtained through an in-vehicle camera or other sensors, which is not limited in this application.
- the interior space of the car can be pre-divided to define the passenger area, and when a person is detected by an image sensor such as a camera, it is determined whether the person is located in the pre-divided passenger area.
- the first passenger seat information an image sensor such as a camera facing the passenger seat can also be specially provided, and when a person is detected through the camera, the first passenger seat information indicating that there is an occupant in the passenger seat is generated.
- the interior space can also define other areas, such as the main driver area, the left area of the rear row, and the right area of the rear row.
- the second sensing information may be, for example, pressure detection information of a pressure sensor of the passenger's seat, and the pressure sensor may be typically provided in the seat cushion, and in addition It can also be provided in the seat back. Specifically, it can be arranged in the foam pad, or between the foam pad and the skin.
- the specific content of the pressure detection information is also different.
- the pressure sensing information includes the detected pressure value.
- the pressure threshold is set for comparison, and when the pressure value is greater than the preset pressure threshold, it is considered that the passenger seat is occupied.
- the pressure-sensing information is on-off information of the pressure-sensitive switch, and the on-off information indicates the pressure When the sensitive switch is turned on, it is assumed that there is someone in the passenger seat.
- the specific form of the second passenger riding information may be, for example, information indicating whether the pressure value is greater than a preset threshold, or information indicating whether the pressure-sensitive switch is turned on.
- other equivalent information is also possible.
- an indication information directly indicating that there is an occupant in the passenger seat is generated.
- instruction information indicating that there is no passenger in the passenger seat is generated, and this instruction information also belongs to the "second passenger seat information".
- the pressure sensor is used as an example to introduce the above.
- sensors such as infrared sensors (infrared cameras), millimeter-wave sensors (millimeter-wave radar), and sound sensors (microphones), can also be used to obtain the second sensor.
- infrared sensors infrared cameras
- millimeter-wave sensors millimeter-wave radar
- sound sensors microphones
- the first passenger riding information is determined in response to the door closing information. That is, in response to the detected or received door closing signal, the process of determining the riding information of the first passenger according to the first sensing information is performed.
- the first passenger's riding information is determined after the door is closed, that is, the split-screen display command is sent after the door is closed, and the screen is not split when the door is open. Inappropriate split screen when picking up items in a school bag placed on the passenger seat.
- the method further includes: acquiring in-vehicle image information; acquiring a face image according to the in-vehicle image information; when there is a face image in the pre-defined passenger area, the first passenger riding information indicates There is a passenger in the passenger seat.
- the above method can use the existing sensors in the cockpit to obtain the in-vehicle image information to detect whether there is an occupant in the passenger seat, which can avoid the additional increase in the number of sensors and the resulting problems. cost increase.
- the method further includes: under the condition that the ratio of the number of frames of face images existing in the passenger area to the total number of sampled frames is greater than or equal to a preset value, the first passenger riding information indicates There is a passenger in the passenger seat.
- the "proportion of the number of image frames of the face image in the total number of sampling frames" can be a ratio value, or it can refer to the number of images from which face information is obtained, and the two can be substantially the same. Effect.
- the method further includes: when a preset operation is detected, sending an instruction to close the split-screen display.
- the split-screen display can be turned off through active operation.
- the preset operation may be, for example, an operation on a physical button, a touch operation on a human-computer interface, or a voice control operation (input voice control instruction) input through a microphone.
- the method further includes: when at least one of the first passenger's riding information or the second passenger's riding information indicates that there is no passenger in the passenger's seat, sending an instruction to close the split-screen display.
- the first sensing information is image information
- the second sensing information is pressure sensing information
- the above control method further includes: acquiring the riding information of the third passenger obtained according to the sound information; When one of the first passenger information and the second passenger information indicates that there is a passenger in the passenger seat and the other indicates that there is no passenger in the passenger seat, and the third passenger information indicates that there is a passenger in the passenger seat, the first passenger passenger information is repeatedly determined. Or the processing of the second passenger's occupancy information.
- a second aspect of the present application provides a control device for a vehicle-mounted display device, which includes an acquisition module and a processing module, the acquisition module is used to acquire first sensing information, and the first sensing information is used to determine the riding information of the first passenger; the acquisition module It is also used to obtain the second sensing information, and the second sensing information is used to determine the seating information of the second passenger; the processing module is used for when the first passenger riding information indicates that there is an occupant in the passenger seat, and the second passenger riding information indicates the passenger seat. When there is an occupant, send a split-screen display command.
- the first sensing information includes in-vehicle image information
- the second sensing information includes pressure sensing information, infrared sensing information, millimeter wave sensing information, and sound sensing information. one or more of.
- the processing module is further configured to determine the first passenger riding information in response to the door closing information.
- the acquisition module is further used to acquire image information in the vehicle; the processing module is further used to acquire a face image according to the image information in the vehicle; when there is a human face in the pre-defined passenger area When the image is created, first passenger ride information indicating that there is an occupant in the passenger seat is generated.
- the above method can use the in-vehicle image information obtained by the existing sensors in the cockpit to detect whether there is an occupant in the passenger seat, which can reduce the deployment of additional sensors and reduce costs.
- the processing module is further configured to generate an instruction for the co-pilot under the condition that the ratio of the number of frames of the face image existing in the co-pilot area to the total number of sampled frames is greater than or equal to a preset value
- the seat has the passenger's first passenger ride information.
- the "proportion of the number of image frames of the face image in the total number of sampling frames" can be a ratio value, or it can refer to the number of images from which face information is obtained. same effect.
- a third aspect of the present application provides an electronic device, which includes a display device and a control device of any of the above structures.
- a fourth aspect of the present application provides a vehicle including the control device of any of the above structures or the above electronic device.
- a fifth aspect of the present application provides a computing device, which includes one or more processors, the processors are coupled to a memory, the memory stores program instructions, and when executed by the processor, the program instructions cause the processor to execute any one of the above control methods .
- a sixth aspect of the present application provides a computer-readable storage medium on which program instructions are stored, and when executed by a computer, the program instructions cause the computer to execute any one of the above control methods.
- a seventh aspect of the present application provides a computer program product, comprising: computer program code, which implements any one of the foregoing control methods when the computer program code is run on a computer.
- FIG. 1 is a flowchart illustrating a control method involved in an embodiment of the present application
- FIG. 2 is a schematic block diagram of an electronic device involved in an embodiment of the present application.
- FIG. 3 is an explanatory diagram of a logical structure of an electronic device involved in an embodiment of the application.
- FIG. 4 is a schematic explanatory diagram of a method for dividing a vehicle interior area involved in an embodiment of the application
- FIG. 5 is an explanatory diagram of a system structure to which a control method and a control device according to an embodiment of the present application are applied;
- FIG. 6 is a processing flow chart of a control method in an embodiment of the present application.
- Fig. 7 is another processing flow chart of the control method in an embodiment of the present application.
- Fig. 8 is another processing flow chart of the control method in an embodiment of the present application.
- FIG. 9 is a schematic illustration diagram of a control device and a control method involved in an embodiment of the application.
- FIG. 10 is a processing flow chart of a control method involved in an embodiment of the application.
- FIG. 11 is another processing flow chart of the control method involved in an embodiment of the application.
- FIG. 12 is a schematic explanatory diagram of a permission granting process involved in an embodiment of the application.
- FIG. 13 is a schematic diagram illustrating a control device and a control method involved in an embodiment of the application
- FIG. 14 is another schematic diagram illustrating the control device and the control method according to an embodiment of the present application.
- 15 is a schematic structural diagram of an electronic control unit provided by an embodiment of the present application.
- FIG. 16 shows a schematic diagram of a vehicle according to an embodiment of the present application.
- FIG. 17 is an explanatory diagram of an application scenario involved in an embodiment of the present application.
- FIG. 18 is an explanatory diagram of another application scenario involved in the embodiment of the present application.
- a comparative example as an embodiment of the present application will be described.
- a camera is used to photograph the interior of the vehicle to obtain image information in the vehicle, and faces are detected according to the image information in the vehicle to detect the occupants in the vehicle.
- one display area of the display screen of the vehicle-mounted display device is divided into two display areas to display the corresponding content respectively (ie, split screen), so that one display device can be used by the driver and can be used for It can be used by the co-pilot and occupants without affecting each other, which improves the experience of the occupants.
- this comparative example may have a problem of the appropriateness of the split screen.
- a person outside the vehicle 410 sticks his head into the vehicle from the window on the passenger side of the vehicle 401 to communicate with the driver, or is in a storage box (installed on the dashboard) in front of the passenger seat )
- the camera detects that the face of the person outside the vehicle 410 is located in the passenger area, it is considered that there is an occupant in the passenger seat, so that the display device is displayed in a split screen.
- a pressure sensor is provided on the passenger seat, and the pressure sensor detects the pressure when an occupant sits on the passenger seat, so that the control device on the vehicle considers that there is an occupant in the passenger seat, so that the The display device performs split-screen display.
- this comparative example may also have problems with split screen suitability. Specifically, for example, referring to FIG. 18 , when a heavy object such as a backpack 411 is placed on the passenger seat, the pressure sensor detects the pressure. When the heavy object is heavy, the control device will misjudge the passenger seat according to the information detected by the pressure sensor. There is an occupant, and the display device is further displayed in a split screen.
- the embodiments of the present application provide a control method, control device, electronic device, vehicle, computing device, computer-readable storage medium, and computer program product of a vehicle-mounted display device capable of performing split-screen display more appropriately.
- FIG. 16 is a schematic diagram of a vehicle according to an embodiment of the present application.
- a vehicle 400 includes a driver's seat 401 , a passenger's seat 402 , an electronic device 403 , and a camera 404 .
- the passenger seat 402 includes a pressure sensor 402a, which is disposed in the seat cushion. When the passenger sits on the passenger seat 402, the pressure sensor 402a detects a certain pressure.
- the pressure sensor 402a may be in the form of a sensor that detects a pressure value, or may be a pressure-sensitive switch that is turned on when a pressure of a certain value or more is received.
- a pressure sensor may be provided on the driver's seat 401 .
- the electronic device 403 is a central control display with a display device and a control device.
- the display device is used to display navigation images, video images, and human-computer interaction interfaces, and the control device is used to control the display device.
- the camera 404 is disposed near the rear-view mirror in the vehicle, and is used to photograph the interior of the vehicle to obtain image information in the vehicle, and the image information in the vehicle is transmitted to the control device of the electronic device 403 .
- the camera 404 can also be used for taking pictures and making video calls.
- a camera may also be arranged near the upper left corner or the upper right corner of the front windshield.
- the embodiment provides a control method of a vehicle-mounted display device, a control device for executing the control method, electronic equipment, computing equipment, computer-readable storage medium, and computer program products. .
- FIG. 1 is an explanatory diagram of a flow chart of the control method involved in this embodiment.
- FIG. 2 is a schematic block diagram of an electronic device involved in this embodiment.
- the control method includes: S1, acquiring first sensing information; S2, determining the riding information of the first passenger according to the first sensing information; S3, acquiring second sensing information; S4, according to the second sensing information
- the sensory information determines the riding information of the second co-driver; S5, when both the riding information of the first co-driver and the riding information of the second co-driver indicate that there is an occupant in the passenger seat, send a split-screen display instruction.
- the comprehensive logic determines that there is no passenger in the passenger seat, and does not send a split-screen display instruction.
- the first sensing information is, for example, image information in the vehicle, typically image information captured by a color camera. In addition, it may also be image information obtained by other image sensors such as lidar, millimeter-wave radar, and infrared cameras.
- the interior space of the car can be pre-divided to define the passenger area, and when the camera detects a person, it is determined whether the person is located in the passenger area, and when the person is located in the passenger area (or there is a person image in the passenger area), an indicator is generated indicating that the passenger seat has a seat.
- the first passenger's riding information of the passenger; a camera facing the passenger's seat may also be specially set up, and when a person is detected through the camera, the first passenger's riding information indicating that there is an occupant in the passenger's seat is generated.
- the second sensing information may be the sensing information of the pressure sensor arranged on the passenger seat, or may also be the infrared sensing information of the infrared camera, the infrared sensing information of the millimeter-wave radar Millimeter wave sensing information, sound sensing information of a sound sensor (microphone), etc.
- the pressure sensor can typically be provided in the seat cushion, but also in the seat back. Specifically, it can be arranged in the foam pad, or between the foam pad and the skin.
- the specific content of the pressure detection information is also different.
- the pressure detection information when a pressure sensor for detecting a pressure value is set, the pressure detection information includes the detected pressure value.
- the pressure value can be compared with the preset pressure value.
- the pressure threshold value is compared, and when the pressure value is greater than the preset pressure threshold value, it is considered that the passenger seat is occupied.
- the pressure detection information when a pressure sensor (pressure-sensitive switch) that switches on/off states in response to sitting/leaving of an occupant is provided, the pressure detection information is on-off information of the pressure-sensitive switch, and the on-off information indicates the pressure-sensitive switch When the switch is turned on, it is assumed that the passenger seat is occupied.
- the specific form of the second passenger's riding information may, for example, indicate whether the pressure value is greater than the predetermined value.
- the information for setting the threshold value may also be information indicating whether the pressure-sensitive switch is turned on.
- other equivalent information is also possible. For example, in the case where there are multiple pressure-sensitive switches distributed in different positions of the passenger seat, when multiple pressure-sensitive switches all generate ON information, an indication information directly indicating that there is an occupant in the passenger seat is generated. When the ON information is not generated for some of the switches, instruction information indicating that there is no passenger in the passenger seat is generated, and this instruction information also belongs to the "second passenger seat information".
- execution order of the above S1 and S3 is not limited, and S3 may be executed before S1, or both may be executed in parallel.
- S1-S5 can be executed by one execution body, and can also be executed by multiple execution bodies.
- S1 and S2 are executed by the first execution body
- S3 and S4 are executed by the second execution body.
- the third executive body obtains the results output by the first executive body and the second executive body (the first passenger's riding information, the second passenger's riding information), and executes S5 accordingly.
- the display device When the display device receives a split-screen display instruction, it switches from the normal display mode (default display mode) to the split-screen display mode, and divides a display area in the normal display mode into a driver-side display area and a passenger-side display area.
- the normal display mode default display mode
- the split screen display in the process of executing the split screen display, if it is determined that there is no passenger in the passenger seat according to the first passenger seat information and the second passenger seat information, the split screen display is turned off and switched to the normal display state.
- the processing can be triggered by the detection of the door opening signal or the door closing signal, that is, when the door opening signal or the door closing signal is detected, the same processing as S1-S4 is performed.
- the split-screen display is turned off.
- an instruction to close the split-screen display may be sent to turn off the split-screen display.
- the preset operation here may be an operation on a physical button, a voice control operation (inputting a voice control command) through a microphone, a touch operation on a human-computer interaction interface, and the like.
- the so-called “split-screen display” in this application refers to dividing a display area in the normal display state of the display device into two or more display areas to display corresponding content respectively, and the divided multiple display areas
- the entire range of the area roughly corresponds to the range of one display area before division.
- the display content between the divided display areas may be the same (for example, the same application icon is displayed) or different.
- the range (size) of the divided display areas may be the same or different. For example, it may be allocated to the driver. A larger display area, while the other occupants are allocated a relatively small display area.
- two display areas can be divided, and when there are also rear occupants, three or more display areas can be divided. In addition, it can be divided into left and right or up and down.
- This embodiment also provides an electronic device for executing the above-mentioned control method and a control device possessed by the electronic device.
- the electronic device 301 includes a display device 201 and a control device 101
- the control device 101 includes an acquisition module 101a and a processing module 101b.
- the electronic device 301 is, for example, an electronic device of an in-vehicle system (in-vehicle infotainment system), and may also be an interior mirror having a display function, or the like.
- the display device 201 may also be referred to as a display screen, and is used to display a navigation screen, a video screen, a music playing screen, a human-computer interaction interface, and the like.
- the control device 101 can typically be an electronic control unit (Electronic Control Unit, ECU), and the acquisition module 101a and the processing module 101b are software function modules.
- the acquiring module 101a and the processing module 101b can also be implemented by hardware.
- the electronic control unit here refers to a control device including an integrated circuit for realizing a series of functions such as analysis, processing and transmission of data.
- an embodiment of the present application provides an electronic control unit ECU, which includes a micro-computer (micro-computer), an input circuit, an output circuit, and an analog-to-digital (A/D) converter.
- micro-computer micro-computer
- A/D analog-to-digital
- the main function of the input circuit includes preprocessing the input signal (such as the signal from the sensor), and the processing method is different for different input signals.
- the input circuit may include an input circuit processing analog signals and an input circuit processing digital signals.
- the main function of the A/D converter includes converting the analog signal into a digital signal.
- the analog signal is preprocessed by the corresponding input circuit and then input to the A/D converter for processing and converting into a digital signal accepted by the microcomputer.
- the output circuit is a device that establishes the connection between the microcomputer and the actuator. Its functions include converting the processing results sent by the microcomputer into control signals to drive the actuators to work.
- the output circuit generally uses a power transistor, which controls the electronic circuit of the actuator by turning on or off according to the instructions of the microcomputer.
- the microcomputer includes a central processing unit (CPU), a memory and an input/output (I/O) interface.
- the CPU is connected to the memory and I/O interface through a bus, and information can be communicated between each other through the bus. exchange.
- the memory can be a read-only memory (ROM) or a random access memory (RAM) or other memory.
- the I/O interface is the connection circuit for exchanging information between the central processing unit (CPU) and the input circuit, the output circuit or the A/D converter. Specifically, the I/O interface can be divided into a bus interface and a communication interface .
- the memory stores a program, and the CPU calls the program in the memory to execute the control method described in the embodiment corresponding to FIG. 1 and the following embodiments.
- the split-screen display command is sent to the in-vehicle display device. Therefore, even in the case shown in FIG. 17 and FIG. 18 , the display device can be appropriately split.
- the first passenger seat information indicates that there is an occupant in the passenger seat
- the pressure sensor of the passenger seat cannot detect pressure, so the second passenger riding information indicating that there is an occupant in the passenger seat will not be generated, so it is finally determined that the passenger is the passenger seat. There is no occupant in the seat, and the display device will not be split.
- the second passenger seat information indicates that there is an occupant in the passenger seat
- the instruction for the passenger is not generated.
- the seat has the passenger's first passenger's riding information, so it is finally determined that there is no passenger in the passenger's seat, and the display device will not be displayed in a split screen.
- the occurrence of inappropriate screen splitting can be reduced, a more appropriate split screen display function can be provided, and the occupant's sense of experience can be improved.
- FIGS. 3 to 8 An embodiment of the present application will be described below with reference to FIGS. 3 to 8 , and the embodiment provides a control method for a vehicle-mounted display device, a control device for executing the control method, an electronic device, and the like.
- FIG. 3 is an explanatory diagram of a logical structure of the electronic device involved in this embodiment.
- FIG. 4 is a schematic explanatory diagram of a method for dividing a vehicle interior area involved in this embodiment.
- FIG. 5 is an explanatory diagram of a system structure to which the control method and the control device of the present embodiment are applied.
- the electronic device 300 includes a display device 200 and a control device 100 .
- the display device 200 may also be referred to as a display screen. In this embodiment, it is the central control display screen of the vehicle, for example, used to display the navigation screen, the music playback screen, the video playback screen, and the content captured by the camera. to display the screen sent from the mobile phone.
- the display device 200 is a touch screen, and is further configured to display an operation interface and the like for the operator to perform a touch operation.
- the display device 200 may also be a display screen integrated on the rearview mirror of the vehicle.
- the control device 100 is used to control the display content of the display device 200 , and may be provided in a housing with the display device 200 , or may be provided separately from the display device 200 .
- the control device 100 is also a control device of the vehicle machine (vehicle infotainment system). As other embodiments, the control device 100 may be set independently of the control device of the vehicle.
- the control device 100 includes an image data stream module 10 , a face detection module 20 , an area division module 30 , a co-pilot area fusion module 40 , a vehicle-machine signal fusion module 50 and a split-screen processing module 60 .
- the image data stream module 10 is used to acquire the in-vehicle image data stream obtained by the camera of the camera monitoring system (Camera Monitor System, CMS) photographing the in-vehicle, and to analyze the data format of the acquired in-vehicle image data stream. This process starts to be executed after the control device 100 is started (or the vehicle is started), in preparation for the subsequent face detection algorithm recognition.
- CMS Camera Monitor System
- the CMS camera is, for example, a color camera (RGB camera), which is used for taking pictures and making video calls.
- the data format analysis performed by the image data stream module 10 is to convert the camera data stream from YUV to Red Green Blue (RGB) format (YUV and RGB are both color codes). method).
- the image data stream module 10 can also obtain the in-vehicle image data stream captured by other cameras or other image sensors (optical sensors).
- the camera can be a camera dedicated to photographing the passenger seat, and other image sensors are, for example, lidar.
- the face detection module 20 is used to process the in-vehicle image data from cameras, etc. using a face recognition algorithm model to perform face detection and obtain face information (including face feature information and face position information).
- the processing of the face detection module 20 is triggered by detecting a door closing signal (door closing information).
- a door closing signal door closing information
- the energy consumption caused by the algorithm recognition can be reduced.
- starting the processing of the face detection module 20 with the detection of the door closing signal as a triggering condition can also obtain the technical effect of reducing inappropriate split screen processing, which will be described in detail later.
- the face recognition algorithm model here is, for example, an algorithm model based on deep learning. It goes without saying that the face recognition algorithm model can also be a model other than deep learning.
- the area division module 30 is configured to perform area division on the face detected by the face detection module 20 to determine which area of the vehicle interior space the detected face belongs to.
- the interior space of the vehicle is divided into 4 areas in advance, namely the main driver area 1, the auxiliary driver area 2, the rear row area 3 and the rear row area 4, the main driver area 1, the rear row area
- the area 3, the passenger area 2, and the rear area 4 are demarcated by a straight line L, and these area division information is stored in the memory of the control device 100 in advance.
- the area division module 30 determines which area the detected face belongs to according to the face position information and the area division information. It can be understood that the area division of the interior space of the vehicle is only exemplary, and the manner of area division is also adjusted accordingly according to the different relative positions of different main driver's seats in the vehicle.
- the co-pilot area fusion module 40 is used to perform statistics on the faces of the co-pilot areas obtained by the area division module 30, for example, using a sliding window algorithm (Sliding Window Algorithm) to maintain a sliding window for recording the historical detection results, by counting people in the co-pilot area (faces).
- a sliding window algorithm sliding Window Algorithm
- the vehicle-machine signal fusion module 50 is used to detect door opening and closing information and determine whether there is an occupant in the passenger seat according to the detection information of the pressure sensor on the passenger seat.
- the pressure sensor is used to detect the pressure on the passenger seat.
- the vehicle-machine signal fusion module 50 considers that there is an occupant in the passenger seat, and can generate pressure detection result information indicating that there is an occupant in the passenger seat, and vice versa. Then, it is considered that there is no occupant in the passenger seat, and pressure detection result information indicating that there is no occupant in the passenger seat can be generated.
- the split-screen processing module 60 is used to determine whether to perform split-screen display according to the output result of the co-pilot area fusion module 40 and the output result of the vehicle-machine signal fusion module 50. Specifically, the image detection result information output by the co-pilot area fusion module 40 indicates that When there is an occupant in the passenger seat and the pressure detection result information output by the vehicle-machine signal fusion module 50 indicates that there is an occupant in the passenger seat, the split-screen processing module 60 generates a split-screen display instruction, and sends the split-screen display instruction to the image display device 200 .
- the split-screen display instruction is used to perform split-screen display on the display screen of the image display device 200 , and the split-screen display here means dividing an originally displayed display area into two sub-display areas for display.
- the two sub-display areas behind the split screen one is the main driver sub-display area for the driver to watch and use, and the other is the co-driver sub-display area for the co-driver to watch and use.
- the display content of the main driver display area can be the same as the display content before the split screen (however, according to the change of the size of the display area, the arrangement of the display icons can be adjusted, etc.).
- the display content of the sub-display area of the passenger driver may be the same as the display content (for example, the application icon) of the sub-display area of the main driver, or may be different, or partially the same.
- the split-screen processing module 60 comprehensively determines that there is no passenger in the passenger seat, and does not generate Split screen command or generate split screen display close command.
- the above-mentioned functional modules can be implemented by hardware such as Large Scale Integration (LSI) and Application Specific Integrated Circuit (ASIC), or by software (that is, by executing the program stored in the memory by the processor. implementation), or one part is implemented in hardware and the other part is implemented in software.
- these functional modules are implemented by software, and the system structure of their application is shown in FIG. 5 , for example.
- the control device 100 includes an electronic control unit.
- the electronic control unit includes a processor as hardware and a memory.
- the memory stores system software (operating system, compiler, etc.), support software (interface software, tool software, and compiler). etc.) and application software, these functional modules in this embodiment and the control methods executed by these functional modules correspond to application software, and when the application software is executed by the processor, the functions of these functional modules in this embodiment are played or the control method is executed method.
- FIG. 6 is a processing flow chart of the control method in this embodiment.
- an in-vehicle image data stream obtained by photographing the inside of the vehicle is obtained from the camera, and then in S20 , the format conversion is performed on the acquired data.
- the door may be monitored for one opening and closing within a preset time (door opening and closing signal), or only the door closing signal may be monitored.
- the face recognition model is loaded and the algorithm PipeLine (linear communication model of the pipeline segment for exchanging data between the add-in program and its host) is initialized.
- face detection is performed on the in-vehicle image data by using the algorithm model, and the detection result (face information) is output.
- S60 when a human face is detected, the area to which the human face belongs in the vehicle interior space is determined (FIG. 4). In S50, it is possible to detect a plurality of images containing human faces. At this time, in S60, each identified human face is determined to which area in the interior space of the vehicle it belongs to.
- the sliding window algorithm is used to count the proportion of the number of image frames of people (faces) in the co-pilot area in the total number of frames of the statistical sample.
- S80 according to the statistical result of S70, it is determined whether there is an occupant in the passenger seat. For example, when a human face in the passenger area is detected in 20 frames within 70 frames, it is considered that there is an occupant in the passenger seat.
- an instruction is generated to indicate the passenger seat If there is image detection result information of the occupant, otherwise, it is considered that there is no occupant in the passenger seat, and image detection result information indicating that there is no occupant in the passenger seat can be generated, and in S40, the door closing signal is monitored.
- the pressure detection information obtained by the pressure sensor of the passenger seat is acquired, and when the pressure detection information indicates that there is an occupant in the passenger seat, in S110, a split screen command for the display device 200 is generated and output to the display device 200. .
- the dual-screen operation processing is started, and the corresponding processing is performed according to the operation of the display device 200 by the driver and the passenger.
- the dual-channel voice control (dual microphone control screen) function is awakened, and corresponding controls are performed according to the voices of the driver and the passenger, for example, the control of turning on the air conditioner.
- the control device sends a message to close the split screen. Display the command to turn off (cancel) the split-screen display of the display device, and at this time, return to the previous normal display state.
- the closing of the split-screen display can also be triggered by the active operation of the driver or co-driver, etc.
- the active operation includes the operation of the physical button, the touch operation of the human-computer interface, or the voice control through the microphone. operate.
- FIG. 7 is another processing flow chart of the control method in this embodiment.
- the scene in FIG. 7 is a scene in which it is determined that there is an occupant in the passenger seat based on the in-vehicle image data captured by the camera, and it is determined that there is an occupant in the passenger seat based on the pressure detection information of the pressure sensor of the passenger seat.
- an in-vehicle image data stream obtained by photographing the inside of the vehicle is obtained from the camera, and then, in S20 , format conversion is performed on the acquired data.
- the face recognition model is loaded and the algorithm PipeLine is initialized.
- the in-vehicle image data stream is sent to other processing modules.
- the door closing signal is also monitored in S32.
- the algorithm model is used to perform face detection on the image data in the vehicle, and the detection result (face information) is output.
- S60 when a human face is detected, it is determined in which area of the vehicle interior space the human face is located (FIG. 4).
- the sliding window algorithm is used to count the proportion of the number of image frames of people (faces) in the co-pilot area in the total number of frames of the statistical sample.
- the ratio exceeds a predetermined threshold, in S90, it is determined that there is an occupant in the passenger seat, and image detection result information indicating that there is an occupant in the passenger seat is generated.
- S102 it is determined whether there is a passenger (heavy object) in the passenger seat according to the pressure detection information obtained by the pressure sensor of the passenger seat, and when it is determined that there is an occupant in the passenger seat, in S112, it is finally determined that there is a passenger in the passenger seat (a passenger), A split screen command for the display device 200 is generated and output to the display device 200 .
- the processes in S120 and S130 are the same as those in FIG. 6 , and their detailed descriptions are omitted here.
- control device may not be directly connected with the display device 200, for example, the control device is set in other vehicle-mounted controllers, at this time, the other vehicle-mounted controllers send a split-screen display instruction to the vehicle-mounted controller, and the vehicle-mounted control The controller then sends a split-screen display instruction to the display device 200 .
- FIG. 8 is another processing flow chart of the control method in this embodiment.
- it is determined that there is no occupant (face) in the passenger seat according to the in-vehicle image data, and that there is an occupant (heavy object) in the passenger seat according to the pressure detection information of the pressure sensor.
- the main difference between FIG. 8 and FIG. 7 is the processing after S60.
- S60 As shown in FIG. 8 , it is determined in S60 which area the detected face belongs to.
- the sliding window algorithm is used to count the proportion of the number of image frames in which there is an occupant in the passenger seat in the total number of statistical frames, and the obtained proportion does not reach a preset threshold. Therefore, in S92, image detection result information indicating that there is no occupant (face) in the passenger seat is generated.
- S102 it is determined from the detection information of the pressure sensor that there is an occupant (heavy object) in the passenger seat.
- S114 it is comprehensively determined that there is no occupant in the passenger seat, and the processing of this flow is ended.
- the processing of this flow may be directly ended, and the processing in S102 may be omitted.
- the door closing signal may be monitored before S32, and when the door closing signal is detected, it is judged according to the pressure detection information of the pressure sensor whether there is a passenger (heavy object) as a passenger, and when it is judged that there is no passenger, The process of this flow is ended, and when it is determined that there is an occupant, the processes of S32 and subsequent are executed.
- the comprehensive logic determines that there is no occupant in the passenger seat, and no split-screen instruction is generated, thereby reducing the occurrence of inappropriate screen splitting.
- face recognition is performed after the door closing signal is detected, and the pressure detection information is used to determine whether to split the screen, that is, the split screen command is issued on the condition that the door status information indicates that the door is closed. Therefore, , the screen can be split more appropriately.
- the above-mentioned comparative example is used to detect whether there is an occupant in the passenger seat by simply using image information, for example, when someone opens the door and leans into the car to retrieve the items in the schoolbag placed on the passenger seat, the Do a split screen display.
- the algorithm processing of the in-vehicle image is performed to recognize the face, which can reduce the processing burden of the system.
- FIG. 9 is a schematic diagram illustrating a control device and a control method according to an embodiment of the present application.
- FIG. 10 is a processing flow chart of the control method involved in this embodiment.
- FIG. 12 is a schematic explanatory diagram of the authorization granting process involved in this embodiment.
- control device 100A includes a situational intelligence module 80 , and other similarities are given the same reference numerals and their detailed descriptions are omitted.
- the situational intelligence module 80 in this embodiment is used to obtain Bluetooth connection information when a split-screen command is issued. An inquiry message is sent to the driver's mobile phone through Bluetooth to inquire whether the driver enters the privacy mode, and, optionally, after the driver makes a selection, the driver is prompted to grant permission to the passenger display area.
- other wireless communication connections such as Universal Serial Bus (Universal Serial Bus, USB) connection or Wi-Fi connection may be used instead of Bluetooth connection.
- the driver when it is decided to perform the split-screen display, when it is detected that the driver's mobile phone is connected to the car machine via Bluetooth, the driver is asked whether to enter the privacy mode through the mobile phone (instead of the display device of the car machine), if there is no Bluetooth Connect without asking to enter privacy mode.
- the driver when performing a Bluetooth connection after the split-screen display, it will also ask whether to enter the privacy mode, so that the driver's privacy can be well protected.
- the question about whether to enter privacy mode is displayed on the driver's mobile phone, not on the in-vehicle display device.
- the privacy mode When the driver chooses to enter the privacy mode, for example, incoming calls and text messages received by the mobile phone and notification information of privacy applications will not pop up on the display screen of the in-vehicle display device (this function can be customized by the user). Apps that are opened by operating the display area will not display the driver's usage history (for example, historical locations will not be displayed when the Maps app is opened).
- the privacy mode also includes, for example, the following controls: hiding the schedule in the privacy area of the car owner's account; hiding voice notes; hiding Bluetooth voice messages; and entering the incognito mode for the in-vehicle application.
- the display device when the display device receives the split-screen display instruction, it divides a screen display area into two sub-display areas, and the divided two sub-display areas are called the first display area and the second display area, which are respectively corresponding to the main driver.
- the screen area corresponds to the screen area of the co-pilot.
- a "space" is allocated to each account, and the space refers to a set of electronic equipment software environment (in this embodiment, the vehicle software environment), which is composed of various vehicle applications.
- the car system Before the permission granting stage, the car system has only one account, the owner account, and only one set of "spaces", called the default space or the first space, which is operated by the driver.
- the second space a set of application spaces (called the second space) that is independent of the default space is formed.
- the dual space is realized through the application of dual opening. All applications that support dual opening in the default space can appear in the second space for the co-pilot. interactive operation.
- the main and co-pilot operations of some applications are mutually exclusive, that is, for the same application, the co-pilot and the main driver can only be interacted with by one person at the same time.
- the main driver and the co-pilot are the same application operations will cause mutual preemption.
- the authority there are three kinds in this embodiment, namely, the vehicle owner's exclusive authority, the authorized account authority and the temporary account authority.
- the specific authority content is that the following operations can be performed under the exclusive authority of the vehicle owner: control functions such as the air-conditioning sunroof (which can be controlled by voice), add or delete authorized account schedules, add or delete vehicle owner account schedules, vehicle Map manipulation, control of in-vehicle entertainment functions (multimedia and games), voice memo function, and mobile phone screen projection, etc.; the following operations can be performed under the authorization account authority: adding and deleting authorized account schedules, adding and deleting car owner account schedules, vehicle map control, control of air-conditioning sunroof and other functions (Voice control is available), in-vehicle entertainment (multimedia and games), and mobile phone screencasting; the following operations can be performed under temporary account permissions: in-vehicle entertainment functions (multimedia and games) and mobile phone screencasting.
- control functions such as the air-conditioning sunroof (which can be controlled by voice), add or delete authorized account schedules, add or delete vehicle owner account schedules, vehicle Map manipulation, control of in-vehicle entertainment
- driving control driving mode, chassis system adjustment, steering system adjustment, power system adjustment
- driving control can be performed under the exclusive authority of the owner, while driving control cannot be performed under the authorization account authority and the temporary account authority.
- the assistant driver and the main driver share (or share) a set of application space, and the split-screen control state is dual-screen In a single space, for example, if both parties operate multimedia at the same time, they will interrupt each other.
- the display content of the second display area is synchronized with the content of the first display area. This mode ensures that the co-pilot has a high enough authority to replace the driver's operation.
- the co-pilot can complete the corresponding operation without encountering authority obstruction.
- the control of the display area of the assistant driver is independent of that of the main driver, and the in-car multimedia and games are realized by the dual application of the application (that is, through the manipulation in the display area of the assistant driver, music can be played independently, independent Control the game), the independent sound field in the car is transmitted through the sound transmission characteristics of the array speakers, and the speakers in different positions (such as headrest speakers) to achieve the dual sound fields of the main and auxiliary drivers without interfering with each other.
- the system will set a "second space” for them, and at the same time assign an account different from the owner's account in the second space.
- the first display area displays the corresponding content of the default space
- the second screen displays the corresponding content of the second space.
- the two accounts are different, the two can realize the operation of independent accounts in the independent space without interfering with each other, and the main driver can play song A, The co-pilot can play song B, and the two can be played at the same time.
- the dual-play sound fields of the main and co-pilots do not interfere with each other.
- the authorized account has enough permissions to operate the owner's schedule, on-board map navigation and other applications related to the driver's strength.
- the system will open up a "second space" for him, and assign an account with temporary account authority to the co-pilot occupant in the second space. Operations are not mutually exclusive with the default space.
- the first display area displays the corresponding content of the default space
- the second display area displays the corresponding content of the second space.
- the above description is based on the example that the screen display area is divided into two display areas: the main driver display area and the auxiliary driver display area. Those skilled in the art can also understand the control mechanism when the screen display area is divided into more display areas. .
- the co-pilot occupant can be identified through facial information and voice information, remember this person (co-pilot occupant), and automatically authorize it next time.
- the single-channel (by facial information or voice information) recognition is successful, ask the driver whether to grant permission.
- both channels (through facial information and voice information) are successfully recognized, the corresponding permissions are automatically granted.
- the difference between this embodiment and the above-mentioned embodiment also includes that, in this embodiment, the vehicle speed information is obtained, and when the vehicle speed exceeds the high-speed threshold, the control device restricts the driver to control the vehicle-mounted system through the man-machine interface operation, and only opens the voice control channel.
- a split-screen signal (signal indicating the decision to display split-screen display) is acquired, and when a split-screen signal is detected, in S210, it is confirmed whether the vehicle has established a connection with the driver's mobile phone via Bluetooth.
- an inquiry message is sent to the driver's mobile phone to ask the driver whether to enter the display device 200 into the privacy mode.
- the driver's mobile phone sends out an inquiry message to ask the driver what kind of permission is granted to the co-pilot occupant.
- the display of the display device 200 is made to enter the privacy mode. Hide Bluetooth voice and SMS messages, and enter incognito mode for car apps.
- the temporary account mode is entered.
- the facial information and voice information of the co-pilot occupant are recognized, and the recognized facial information and voice information are stored in the memory for the next time. Prepare for automatic authorization. Afterwards, in S260, the authorized account mode is entered.
- FIG. 11 is another processing flow chart of the control method involved in this embodiment.
- the processes in S200 , S210 , and S220 in FIG. 11 are the same as those in FIG. 8 , and detailed descriptions thereof are omitted here.
- S270 after the driver selects whether to enter the privacy mode by operating on the mobile phone, in S270, obtain the face recognition result of the co-pilot occupant by the image face recognition module of the control device, and obtain the control device.
- the voice recognition module of the device recognizes the voice recognition result of the co-pilot occupant, determines the identity of the co-pilot occupant according to the face recognition result and the voice recognition result, and queries the authorization history record information from the memory according to the identity to determine the co-pilot occupant.
- the owner's authority account mode can be entered in S280, the authorized account mode can be entered in S282, or the temporary account mode can be entered in S284.
- the driver can enter the privacy mode through the driver's selection on the mobile phone, so that the driver's experience can be improved.
- FIG. 13 is a schematic diagram illustrating a control device and a control method according to an embodiment of the present application.
- FIG. 14 is another schematic explanatory diagram of the control device and the control method according to this embodiment.
- control device 100B further includes a voice localization module 70
- the voice localization module 70 has a sound ranging function and a voice area division function, and takes the detection of a door closing signal as a trigger condition, Activate the sound ranging function, that is, measure the distance from the microphone according to the sound information of the occupant in the car, and at the same time, according to the position of the microphone (for example, when there are multiple microphones), the position of the occupant in the car who makes the sound can be determined.
- the position of the occupant determines the area to which it belongs ( Figure 2), and then it can determine whether there is an occupant in the passenger seat, and generate corresponding sound area division result information according to the judgment result, which is used to indicate whether there is an occupant (speaker) in the passenger seat, refer to 14 , when the sound location recognition result information, the pressure detection result information, and the image detection result information all indicate that there is an occupant in the passenger seat, a split-screen display instruction is generated to cause the display device 200 to perform split-screen display.
- Table 1 Example of the method for determining whether there is an occupant in the passenger seat
Landscapes
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
Abstract
一种车载显示装置(200、201)的控制方法,包括:获取第一传感信息,例如通过座舱内部摄像头(404)获得的车内图像信息;获取第二传感信息,例如副驾座椅(402)的压力传感器信息;第一传感信息和第二传感信息分别用于确定第一副驾乘坐信息和第二副驾乘坐信息;当第一副驾乘坐信息指示副驾座位有乘坐者,并且第二副驾乘坐信息指示副驾座位有乘坐者时,发送分屏显示指令,使车载显示装置(200、201)进行分屏显示。
Description
本申请涉及智能车技术领域,特别涉及一种车载显示装置的控制方法、控制装置、电子设备、车辆、计算设备、计算机可读存储介质等。
随着汽车智能化的发展,在座舱内布置显示屏的车型越来越多。现有的车载显示方式主要针对驾驶者的需求进行设计,难以兼顾到车内其他乘坐人员的体验。在一些车型中,利用摄像头对车厢内部进行拍摄,得到车内图像信息,根据车内图像信息检测人脸来检测车内的乘员,当在副驾区域检测到人脸时,认为副驾座位上有乘员,此时,将车载显示装置的显示画面一分为二进行显示(即分屏),如此,一个显示装置既能为驾驶员使用,又能为副驾乘员使用,且可以二者互不影响,提高了车辆的使用体验。
然而,仅利用车内图像信息来检测副驾座位是否有人,有时会存在一些分屏适当性的问题。具体而言,例如参照图17,当有车外人员410从车辆的副驾侧的车窗将头部伸入车内与驾驶员进行交流或者在副驾座位前的收纳箱中进行取物时,通过摄像头检测到车外人员410的脸部,认为副驾座位有乘员,从而进行分屏显示。
发明内容
本申请实施例提供一种能够更适当地进行分屏的车载显示装置的控制方法、控制装置、电子设备、车辆、计算设备、计算机可读存储介质等。
本申请第一方面提供一种车载显示装置的控制方法,包括:获取第一传感信息,获取第二传感信息;第一传感信息和第二传感信息分别用于确定第一副驾乘坐信息和第二副驾乘坐信息;当第一副驾乘坐信息指示副驾座位有乘坐者,并且第二副驾乘坐信息指示副驾座位有乘坐者时,发送分屏显示指令。
采用本申请第一方面的控制方法,当根据第一传感信息得到的第一副驾乘坐信息指示车辆的副驾座位有乘坐者,并且根据第二传感信息得到的第二副驾乘坐信息指示副驾座位有乘坐者时,发送分屏显示指令,能够减少因采用单一传感信息导致分屏误执行的发生,从而可以更适当地进行分屏显示。
作为第一方面一个可能的实现方式,第一传感信息包括车内图像信息,所述第二传感信息包括压力传感信息、红外传感信息、毫米波传感信息与声音传感信息中的一种或多种。
这里的车内图像信息、压力传感信息、红外传感信息、毫米波传感信息与声音传感信息可以理解为通过图像传感器、压力传感器、红外传感器、毫米波雷达、声音传感器获得的信息。
例如当第一传感信息是车内图像信息时,采用上述控制方法,例如在图17所示 的情景下,虽然根据车内图像信息检测到车外人员410的脸部位于副驾区域,即第一副驾乘坐信息指示副驾座位有乘坐者,但是,由于车外人员410实际上没有坐在副驾座位上,副驾座位的传感器未检测到相应的传感信息,例如压力传感器检测不到有压力,因此,并不会产生指示副驾座位有乘坐者的第二副驾乘坐信息,从而,最终判断为副驾座位无乘坐者,不会使显示装置进行分屏显示。
另外,采用上述控制方法,例如在图18所示的情景下,虽然副驾座位上放置的重物造成第二副驾乘坐信息指示副驾座位有乘坐者,但是由于没有通过车内图像信息检测到副驾座位有乘坐者,因而不会生成指示副驾座位有乘坐者的第一副驾乘坐信息,从而最终判断为副驾座位无乘坐者,不会使显示装置分屏。
关于“根据第一传感信息得到的第一副驾乘坐信息”,首先,可以通过第一传感信息中的人脸特征来检测是否有人,也可以通过身体特征来检测是否有人。可以理解的,第一传感信息可以是通过车内摄像头或其它传感器获得的车内图像信息,本申请对此不做限定。
另外,可以预先划分车内空间以定义副驾区域,在通过摄像头等图像传感器检测到人物时判断该人物是否位于预先划分的副驾区域,在该人物位于副驾区域时,生成指示副驾座位有乘坐者的第一副驾乘坐信息;也可以专门设置朝向副驾座位的摄像头等图像传感器,通过该摄像头检测到人物时,生成指示副驾座位有乘坐者的第一副驾乘坐信息。进一步的,车内空间也可以定义其它区域,例如主驾区域、后排左侧区域、后排右侧区域。
关于“根据第二传感信息得到的第二副驾乘坐信息”,首先,第二传感信息例如可以是副驾座位的压力传感器的压力检测信息,压力传感器典型地可以设置在座椅坐垫中,另外也可以设置在座椅靠背中。具体可以设置在泡沫垫材中,也可以设置在泡沫垫材与蒙皮之间。
另外,根据压力传感器类型不同,压力检测信息的具体内容也不同,例如,当设置检测压力值的压力传感器时,压力传感信息包括检测到的压力值,此时,可以将该压力值与预设压力阈值进行比较,在该压力值大于预设压力阈值时,认为副驾座位有人乘坐。另外,当设置对应于乘员的就座/离开而切换接通/断开状态的压力传感器(压敏开关)时,压力传感信息是压敏开关的通断信息,在该通断信息指示压敏开关接通时,认为副驾座位有人乘坐。
由此可以理解,第二副驾乘坐信息的具体形式例如可以是指示压力值是否大于预设阈值的信息,也可以是指示压敏开关是否接通的信息。另外,还可以是其他等同信息。例如,在副驾座位的不同位置分布有多个压敏开关的情况下,当多个压敏开关都产生了接通信息时,生成直接指示副驾座位有乘坐者的指示信息,当多个压敏开关中有一部分没有产生接通信息时,生成指示副驾座位没有乘坐者的指示信息,该指示信息也属于“第二副驾乘坐信息”。以上以压力传感器为例进行介绍,可以理解的,采用其它类型的传感器,例如红外传感器(红外摄像头)、毫米波传感器(毫米波雷达)与声音传感器(麦克风),也可以实现获取第二传感信息的目的,本申请对第二传感器和第二传感信息的种类不做限定。
作为第一方面的一个可能的实现方式,确定第一副驾乘坐信息,响应于车门关闭 信息。即响应于检测到或接收到的车门关闭信号,执行根据第一传感信息确定第一副驾乘坐信息的处理。
如此,能够降低确定第一副驾乘坐信息所需的运算负担,降低***处理能耗。
另外,在车门关闭后确定第一副驾乘坐信息,那么也就是在车门关闭后发送分屏显示指令,车门打开的状态下并不分屏,如此能够避免例如在有人打开车门将身体探入车内取放置在副驾座位上的书包内的物品时不适当地进行分屏。
作为第一方面的一个可能的实现方式,还包括:获取车内图像信息;根据车内图像信息获得人脸图像;当预先划定的副驾区域内存在人脸图像时,第一副驾乘坐信息指示副驾座位有乘坐者。
与专门设置对副驾座位进行拍摄的传感器相比,采用如上方式,可以利用座舱内现有的传感器得到车内图像信息来检测副驾座位是否有乘坐者,能够避免传感器数量的额外增加以及由此带来的成本提高。
作为第一方面的一个可能的实现方式,还包括:在副驾区域存在的人脸图像的帧数在取样总帧数中的占比大于或等于预设值的条件下,第一副驾乘坐信息指示副驾座位有乘坐者。
这里的“人脸图像的图像帧数在取样总帧数中的占比”,可以是得到一个比例值,也可以是指从中获取到人脸信息的图像的数量,二者实质上可达到相同的效果。
作为第一方面的一个可能的实现方式,还包括:当检测到预设操作时,发送关闭分屏显示指令。
如此,当驾驶员、副驾驶员或其他乘员不希望进行分屏显示时,可以通过主动操作来关闭分屏显示。
这的预设操作例如可以是对实体按键的操作、对人机交互界面的触控操作或通过麦克风输入的语音控制操作(输入语音控制指令)。
作为第一方面的一个可能的实现方式,还包括:当第一副驾乘坐信息或第二副驾乘坐信息中的至少一方指示副驾座位没有乘坐者时,发送关闭分屏显示指令。
如此,当第一副驾乘坐信息或第二副驾乘坐信息中的至少一方指示副驾座位没有乘坐者时,可以认为副驾座位没有乘坐者,此时关闭分屏显示能够提高用户体验。
作为第一方面的一个可能的实现方式,第一传感信息是图像信息,第二传感信息是压力传感信息,上述控制方法还包括:获取根据声音信息得到的第三副驾乘坐信息;当第一副驾乘坐信息与第二副驾乘坐信息中的一方指示副驾座位有乘坐者而另一方指示副驾座位没有乘坐者、第三副驾乘坐信息指示副驾座位有乘坐者时,重复确定第一副驾乘坐信息或第二副驾乘坐信息的处理。
如此,能够提高鲁棒性,更加可靠地执行分屏操作。
本申请第二方面提供一种车载显示装置的控制装置,其包括获取模块与处理模块,获取模块用于获取第一传感信息,第一传感信息用于确定第一副驾乘坐信息;获取模块还用于获取第二传感信息,第二传感信息用于确定第二副驾乘坐信息;处理模块用于当第一副驾乘坐信息指示副驾座位有乘坐者,并且第二副驾乘坐信息指示副驾座位有乘坐者时,发送分屏显示指令。
采用如上的控制装置,能够执行更适当的分屏。
作为第二方面的一种可能的实现方式,第一传感信息包括车内图像信息,第二传感信息包括压力传感信息、红外传感信息、毫米波传感信息与声音传感信息中的一种或多种。
作为第二方面的一种可能的实现方式,处理模块还用于响应于车门关闭信息确定第一副驾乘坐信息。
采用如上方式,能够降低***的处理负担。
作为第二方面的一种可能的实现方式,获取模块还用于获取车内图像信息;处理模块还用于,根据车内图像信息获得人脸图像;当预先划定的副驾区域内存在人脸图像时,生成指示副驾座位有乘坐者的第一副驾乘坐信息。
与专门设置对副驾座位进行拍摄的传感器相比,采用如上方式,可以利用座舱内已有的传感器得到的车内图像信息来检测副驾座位是否有乘坐者,能够减少额外的传感器部署,降低成本。
作为第二方面的一种可能的实现方式,处理模块还用于在副驾区域存在的人脸图像的帧数在取样总帧数中的占比大于或等于预设值的条件下,生成指示副驾座位有乘坐者的第一副驾乘坐信息。
这里的“人脸图像的的图像帧数在取样总帧数中的占比”,可以是得到一个比例值,也可以是指从中获取到人脸信息的图像的数量,二者实质上可以达到相同的效果。
本申请第三方面提供一种电子设备,其包括显示装置与上述任一结构的控制装置。
本申请第四方面提供一种车辆,其包括上述任一结构的控制装置或上述电子设备。
本申请第五方面提供一种计算设备,其包括一个或多个处理器,处理器与存储器耦合,存储器存储有程序指令,程序指令当被处理器执行时使得处理器执行上述任一种控制方法。
本申请第六方面提供一种计算机可读存储介质,其上存储有程序指令,程序指令当被计算机执行时使得计算机执行上述任一种控制方法。
本申请第七方面提供一种计算机程序产品,包括:计算机程序代码,当计算机程序代码在计算机上运行时,实现前述任一种控制方法。
本申请的这些和其它方面在以下(多个)实施例的描述中会更加简明易懂。
以下参照附图来进一步说明本申请的各个特征和各个特征之间的联系。附图均为示例性的,一些特征并不以实际比例示出,并且一些附图中可能省略了本申请所涉及领域的惯常的且对于本申请非必要的特征,或是额外示出了对于本申请非必要的特征,附图所示的各个特征的组合并不用以限制本申请。另外,在本说明书全文中,相同的附图标记所指代的内容也是相同的。具体的附图说明如下:
图1是本申请一个实施例涉及的控制方法的一种流程说明图;
图2是本申请一个实施例涉及的电子设备的一种示意框图;
图3为本申请一个实施例涉及的电子设备的一种逻辑结构说明图;
图4为本申请一个实施例中涉及的车内区域划分方法的一种示意说明图;
图5为本申请一个实施例的控制方法和控制装置所应用的一种***结构说明图;
图6为本申请一个实施例中的控制方法的一种处理流程图;
图7为本申请一个实施例中的控制方法的另一种处理流程图;
图8为本申请一个实施例中的控制方法的又一种处理流程图;
图9为本申请一个实施例涉及的控制装置与控制方法的一种原理说明图;
图10为本申请一个实施例涉及的控制方法的一种处理流程图;
图11为本申请一个实施例涉及的控制方法的另一种处理流程图;
图12为本申请一个实施例中涉及的权限授予处理的一种示意说明图;
图13为本申请一个实施例涉及的控制装置与控制方法的一种原理说明图;
图14为本申请一个实施例涉及的控制装置与控制方法的另一种原理说明图;
图15所示,本申请一个实施例提供的一种电子控制单元的结构示意图;
图16所示为本申请一个实施例提供的一种车辆的示意图;
图17所示为本申请实施例涉及的一种应用场景的说明图;
图18所示为本申请实施例涉及的另一种应用场景的说明图。
在对本申请的实施例进行描述之前,先描述一下作为本申请的实施例的比较例。在一个比较例中,利用摄像头对车厢内部进行拍摄,得到车内图像信息,根据车内图像信息检测人脸来检测车内的乘员,当在副驾区域检测到人脸时,认为副驾座位有乘坐者,此时,将车载显示装置的显示画面的一个显示区域分为二个显示区域来分别进行相应内容的显示(即分屏),如此,一个显示装置既能为驾驶员使用,又能为副驾乘员使用,且可以二者互不影响,提高了乘员的体验感。
然而,该比较例有可能存在分屏适当性的问题。具体而言,例如参照图17,当有车外人员410从车辆401的副驾侧的车窗将头部伸入车内与驾驶员进行交流或者在副驾座位前的收纳箱(设置在仪表板上)中进行取放物品时,通过摄像头检测到车外人员410的脸部位于副驾区域,认为副驾座位有乘员,从而使显示装置分屏显示。
另外,在另一个比较例中,在副驾座椅上设置压力传感器,当有乘员坐在副驾座位上时,该压力传感器检测到压力,从而,车辆上的控制装置认为副驾座位有乘坐者,使显示装置进行分屏显示。
然而,该比较例也可能存在有分屏适当性的问题。具体而言,例如参照图18,当副驾座位上放有背包411等重物时,压力传感器检测到压力,当重物较重时,控制装置会根据压力传感器检测到的信息误判副驾座位上有乘员,进而使显示装置进行分屏显示。
为此,本申请实施例提供一种能够更适当地进行分屏显示的车载显示装置的控制方法、控制装置、电子设备、车辆、计算设备、计算机可读存储介质、计算机程序产品。
先对本申请实施例的车载显示装置的控制方法、控制装置、电子设备等所应用的车辆的一例进行说明。图16所示为本申请实施例提供的一种车辆的示意图。如图16所示,车辆400具有主驾座椅401、副驾座椅402、电子设备403、摄像头404。
副驾座椅402包括压力传感器402a,压力传感器402a设置在座椅坐垫内,当乘 员坐在副驾座椅402上时,该压力传感器402a检测到一定的压力。该压力传感器402a的形式可以是检测压力值的传感器,也可以是受到一定值以上的压力时接通的压敏开关。另外,也可以在主驾座椅401上设置压力传感器。
在本实施例中,电子设备403是中控显示器,具有显示装置与控制装置,显示装置用于进行导航画面、视频画面、人机交互界面等的显示,控制装置用于对显示装置进行控制。
在本实施例中,摄像头404设置在车内后视镜附近,用于对车厢内部进行拍摄而得到车内图像信息,该车内图像信息被传送给电子设备403的控制装置。另外,摄像头404还可以用于拍照与视频通话等。另外,作为其他实施例,也可以在前挡风玻璃的左上角或右上角附近设置摄像头。
下面参照图1、图2对本申请一个实施例进行说明,该实施例提供一种车载显示装置的控制方法、执行控制方法的控制装置、电子设备、计算设备、计算机可读存储介质、计算机程序产品。
图1是本实施例涉及的控制方法的一种流程说明图。图2是本实施例涉及的电子设备的一种示意框图。
如图1所示,该控制方法包括:S1、获取第一传感信息;S2、根据第一传感信息确定第一副驾乘坐信息;S3、获取第二传感信息;S4、根据第二传感信息确定第二副驾乘坐信息;S5、当第一副驾乘坐信息与第二副驾乘坐信息皆指示副驾座位有乘坐者时,发送分屏显示指令。另外,当第一副驾乘坐信息与第二副驾乘坐信息中的一方或两方指示副驾座位无乘坐者时,综合逻辑判断为副驾座位无乘坐者,并不发送分屏显示指令。
这里,关于“S1、获取第一传感信息”,第一传感信息例如是车内图像信息,典型的是彩色摄像头拍摄到的图像信息。另外,还可以是通过激光雷达、毫米波雷达、红外摄像头等其他图像传感器获得的图像信息。
关于“S2、根据车内图像信息获得第一副驾乘坐信息”,首先,可以通过车内图像信息中的人脸特征(人脸图像)来检测是否有人(乘坐者),也可以通过人物身体特征来检测是否有人(乘坐者)。
另外,可以预先划分车内空间以定义副驾区域,在摄像头检测到人物时判断该人物是否位于副驾区域,在该人物位于副驾区域(或者说副驾区域有人物图像)时,生成指示副驾座位有乘坐者的第一副驾乘坐信息;也可以专门设置朝向副驾座位的摄像头,通过该摄像头检测到人物时,生成指示副驾座位有乘坐者的第一副驾乘坐信息。
关于“S3、获取第二传感信息”,首先,第二传感信息可以是设置在副驾座位上的压力传感器的传感信息,或者还可以是红外摄像头的红外传感信息、毫米波雷达的毫米波传感信息、声音传感器(麦克风)的声音传感信息等。压力传感器典型地可以设置在座椅坐垫中,另外也可以设置在座椅靠背中。具体可以设置在泡沫垫材中,也可以设置在泡沫垫材与蒙皮之间。
另外,根据压力传感器类型不同,压力检测信息的具体内容也不同,例如,当设置检测压力值的压力传感器时,压力检测信息包括检测到的压力值,此时,可以将该压力值与预设压力阈值进行比较,在该压力值大于预设压力阈值时,认为副驾座位有 人乘坐。另外,当设置对应于乘员的就座/离开而切换接通/断开状态的压力传感器(压敏开关)时,压力检测信息是压敏开关的通断信息,在该通断信息指示压敏开关接通时,认为副驾座位有人乘坐。
关于“S4、根据第二传感信息获取第二副驾乘坐信息”,在第二传感信息是压力传感器的压力检测信息时,第二副驾乘坐信息的具体形式例如可以是指示压力值是否大于预设阈值的信息,也可以是指示压敏开关是否接通的信息。另外,还可以是其他等同信息。例如,在副驾座位的不同位置分布有多个压敏开关的情况下,当多个压敏开关都产生了接通信息时,生成直接指示副驾座位有乘坐者的指示信息,当多个压敏开关中有一部分没有产生接通信息时,生成指示副驾座位没有乘坐者的指示信息,该指示信息也属于“第二副驾乘坐信息”。
另外,上述S1与S3的执行顺序没有限制,可以S3在S1之前执行,也可以二者并行执行。
另外,上述S1-S5可以由一个执行主体执行,也可由多个执行主体执行,在由多个执行主体执行时,例如,S1、S2由第一执行主体执行,S3、S4由第二执行主体执行,第三执行主体获取第一执行主体与第二执行主体输出的结果(第一副驾乘坐信息、第二副驾乘坐信息),据此执行S5。
显示装置在接到分屏显示指令时,由通常的显示模式(默认显示模式)切换为分屏显示模式,将通常显示模式下的一个显示区域划分为主驾侧显示区域与副驾侧显示区域。
另外,在本实施例中,在执行分屏显示的过程中,如果根据第一副驾乘坐信息与第二副驾乘坐信息确定副驾座位没有乘坐者时,关闭分屏显示,切换到通常的显示状态。该处理可以以检测到开门信号或关门信号为触发条件,即当检测到开门信号或关门信号时,执行与S1-S4相同的处理,之后,如果根据第一副驾乘坐信息与第二副驾乘坐信息确定副驾座位没有乘坐者时,则关闭分屏显示。可选地,在本申请实施例中,也可以在检测到驾驶员、副驾乘员或其他乘员做出预设操作时,发送关闭分屏显示指令,使分屏显示关闭。这里的预设操作可以是对实体按键的操作、通过麦克风进行的语音控制操作(输入语音控制指令)、对人机交互界面进行的触控操作等。
另外,本申请中所谓的“分屏显示”是指将显示装置的通常显示状态下的一个显示区域划分为二个或更多个显示区域来分别进行相应内容的显示,划分后的多个显示区域的整体范围大致相当于划分前的一个显示区域的范围。划分后的多个显示区域间的显示内容可以相同(例如显示相同的应用图标),也可以不同,另外,划分后的显示区域的范围(尺寸)可以相同也可以不同,例如可以对驾驶员分配较大的显示区域,而对其他乘员分配相对较小的显示区域。当仅有驾驶员和副驾乘员时,可以划分两个显示区域,当还有后排乘员时,可以划分三个或更多个显示区域。另外,可以左右划分也可以上下划分。
本实施例还提供用于执行上述控制方法的电子设备以及电子设备具有的控制装置。如图2所示,该电子设备301包括显示装置201与控制装置101,控制装置101包括获取模块101a与处理模块101b。
电子设备301例如是车机***(车载信息娱乐***)的电子设备,还可以是具备显 示功能的车内后视镜等。显示装置201也可以称为显示屏,用于显示导航画面、视频画面、音乐播放画面、人机交互界面等。控制装置101典型地可以是电子控制单元(Electronic Control Unit,ECU),获取模块101a与处理模块101b是软件功能模块。另外,获取模块101a与处理模块101b也可由硬件实现。
另外,这里的电子控制单元是指包括集成电路的用于实现对数据的分析处理发送等一系列功能的控制装置。如图15所示,本申请实施例提供了一种电子控制单元ECU,该ECU包括微型计算机(micro-computer)、输入电路、输出电路和模/数(analog-to-digital,A/D)转换器。
输入电路的主要功能包括对输入信号(例如来自传感器的信号)进行预处理,输入信号不同,处理方法也不同。具体地,因为输入信号有两类:模拟信号和数字信号,所以输入电路可以包括处理模拟信号的输入电路和处理数字信号的输入电路。
A/D转换器的主要功能包括将模拟信号转变为数字信号,模拟信号经过相应输入电路预处理后输入A/D转换器进行处理转换为微型计算机接受的数字信号。
输出电路是微型计算机与执行器之间建立联系的一个装置。它的功能包括将微型计算机发出的处理结果转变成控制信号,以驱动执行器工作。输出电路一般采用的是功率晶体管,根据微型计算机的指令通过导通或截止来控制执行元件的电子回路。
微型计算机包括中央处理器(central processing unit,CPU)、存储器和输入/输出(input/output,I/O)接口,CPU通过总线与存储器、I/O接口相连,彼此之间可以通过总线进行信息交换。存储器可以是只读存储器(read-only memory,ROM)或随机存取存储器(random access memory,RAM)等存储器。I/O接口是中央处理单元(central processor unit,CPU)与输入电路、输出电路或A/D转换器之间交换信息的连接电路,具体的,I/O接口可以分为总线接口和通信接口。存储器存储有程序,CPU调用存储器中的程序可以执行图1对应实施例以及后述实施例中描述的控制方法。
采用如上所述的本实施例,当根据车内图像信息得到的第一副驾乘坐信息指示车辆的副驾座位有乘坐者,并且根据压力检测信息得到的第二副驾乘坐信息指示副驾座位有乘坐者时,向车载显示装置发送分屏显示指令,因此,即使是例如在图17、图18所示的情况下也可以适当地使显示装置进行分屏。
具体而言,采用本实施例,在图17所示的情况下,虽然根据车内图像信息检测到车外人员410的脸部位于副驾区域,即第一副驾乘坐信息指示副驾座位有乘坐者,但是,由于车外人员410没有坐在副驾座位上,副驾座位的压力传感器检测不到有压力,因此,并不会产生指示副驾座位有乘坐者的第二副驾乘坐信息,从而,最终判断为副驾座位无乘坐者,不会使显示装置分屏。
另外,采用本实施例,在图18所示的情况下,虽然第二副驾乘坐信息指示副驾座位有乘坐者,但是由于没有通过车内图像信息检测到副驾区域有人脸,因而不会生成指示副驾座位有乘坐者的第一副驾乘坐信息,从而最终判断为副驾座位无乘坐者,不会使显示装置分屏显示。
综上,采用本实施例,能够减少不适当分屏情况的发生,提供更恰当的分屏显示功能,提高乘员的体验感。
下面参照图3-图8对本申请一个实施例进行说明,该实施例提供一种车载显示 装置的控制方法、执行控制方法的控制装置、电子设备等。
图3为本实施例涉及的电子设备的一种逻辑结构说明图。图4为本实施例中涉及的车内区域划分方法的一种示意说明图。图5为本实施例的控制方法和控制装置所应用的一种***结构说明图。
如图3所示,电子设备300包括显示装置200与控制装置100。显示装置200也可以称之为显示屏,在本实施例中,为车辆的中控显示屏,例如用于显示导航画面、音乐播放画面、视频播放画面以及摄像头拍摄到的内容等,还可以用于显示手机传送来的画面。可选地,显示装置200为触控屏,还用于显示供操作者进行触控操作的操作界面等。另外,作为其他实施例,显示装置200也可以是集成在车内后视镜上的显示屏。
控制装置100用于控制显示装置200的显示内容,可以与显示装置200设置在一个壳体中,也可以与显示装置200分开设置。另外,在本实施例中,控制装置100同时也是车机(车载信息娱乐***)的控制装置。作为其他实施例,控制装置100可以独立于车机的控制装置设置。
如图3所示,控制装置100包括图像数据流模块10、人脸检测模块20、区域划分模块30、副驾区域融合模块40、车机信号融合模块50与分屏处理模块60。
下面对这些功能模块的具体作用进行描述。
图像数据流模块10用于获取摄像头监控***(Camera Monitor System,CMS)摄像头对车内进行拍摄得到的车内图像数据流,并对获取到的车内图像数据流进行数据格式解析。这一处理在控制装置100启动(或者说车机启动)后即开始执行,为后面的人脸检测算法识别做准备。
CMS摄像头例如是彩色摄像头(RGB摄像头),用于拍照与视频通话等。与此相应,在本实施例中,图像数据流模块10所进行的数据格式解析为,将摄像头数据流进行YUV到红绿蓝(Red Green Blue,RGB)格式转换(YUV和RGB均为颜色编码方法)。
另外,作为其他实施例,图像数据流模块10还可以获取其他摄像头或者其他图像传感器(光学传感器)拍摄到的车内图像数据流,其他摄像头例如是设置在前挡风玻璃左上与右上区域附近的摄像头,可以是专门对副驾座位进行拍摄的摄像头,其他图像传感器例如是激光雷达。
人脸检测模块20用于对来自摄像头等的车内图像数据利用人脸识别算法模型进行处理,以进行人脸检测,而获取人脸信息(包括人脸特征信息与人脸位置信息)。在本实施例中,该人脸检测模块20的处理以检测到关门信号(关门信息)为触发条件。如此,例如与车机开机后人脸检测模块20立即执行处理的方式相比,能够降低算法识别带来的能耗。另外,以检测到关门信号为触发条件来开始人脸检测模块20的处理,还能够获得减少不适当地进行分屏处理的技术效果,具体将在后面描述。
这里的人脸识别算法模型例如是基于深度学习的算法模型。不言而喻,人脸识别算法模型还可以是深度学习以外的模型。
区域划分模块30用于对人脸检测模块20检测到的人脸进行区域划分,以确定检测到的人脸属于车内空间的哪个区域。在本实施例中,如图4所示,预先将车内空间划分为4个区域,即主驾区域1、副驾区域2、后排区域3与后排区域4,主驾区域1、 后排区域3与副驾区域2、后排区域4之间以直线L分界,这些区域划分信息预先存储在控制装置100的存储器中。区域划分模块30根据人脸位置信息与区域划分信息判断检测出的人脸属于哪个区域。可以理解的,该车内空间的区域划分仅为示例性的,根据不同主驾驶位在车内相对位置的不同,区域划分的方式也相应进行调整。
副驾区域融合模块40用于针对区域划分模块30得到的副驾区域人脸进行统计,例如利用滑窗算法(Sliding Window Algorithm),维护一个记录历史检测结果的滑动窗口,通过统计副驾区域有人(人脸)的图像帧数(人脸图像的帧数)在统计样本的总帧数中的占比来判断副驾座位是否有乘坐者,例如当70帧内有20帧或者30帧(具体数值本领域技术人员可以根据经验进行设定)检测到副驾区域有人脸时,则认为副驾座位有乘坐者,生成指示副驾座位有乘坐者的图像检测结果信息,反之则认为副驾座位无乘坐者,生成指示副驾座位无乘坐者的图像检测结果信息。
车机信号融合模块50用于检测开关门信息以及根据副驾座位上的压力传感器的检测信息确定副驾座位是否有乘坐者。该压力传感器用于检测副驾座位受到的压力,当检测到一定值以上的压力时,车机信号融合模块50认为副驾座位有乘坐者,可以生成指示副驾座位有乘坐者的压力检测结果信息,反之则认为副驾座位无乘坐者,可以生成指示副驾座位无乘坐者的压力检测结果信息。
分屏处理模块60用于根据副驾区域融合模块40的输出结果与车机信号融合模块50的输出结果确定是否进行分屏显示,具体而言,在副驾区域融合模块40输出的图像检测结果信息指示副驾座位有乘坐者并且车机信号融合模块50输出的压力检测结果信息指示副驾座位有乘坐者时,分屏处理模块60生成分屏显示指令,并将该分屏显示指令发送给图像显示装置200。该分屏显示指令用于使图像显示装置200的显示画面进行分屏显示,这里的分屏显示的意思是将原来显示的一个显示区域划分成两个子显示区域进行显示。分屏后的两个子显示区域中,一个是主驾子显示区域,供驾驶员观看与使用,另一个是副驾子显示区域,供副驾乘员观看与使用。主驾显示区域的显示内容可以与分屏前的显示内容相同(但是,相应于显示区域尺寸的变化,可以调整显示图标的排列方式等)。副驾子显示区域的显示内容可以与主驾子显示区域的显示内容(例如应用图标)相同,也可以不同,或者有一部分相同。
另外,当副驾区域融合模块40和车机信号融合模块50中的任一方或者双方输出的结果信息指示副驾座位无乘坐者时,分屏处理模块60综合判断为副驾座位无乘坐者,并不生成分屏指令或者生成分屏显示关闭指令。
上述这些功能模块可以由大规模集成电路(Large Scale Integration,LSI)和专用集成电路(Application Specific Integrated Circuit,ASIC)等硬件实现,也可以由软件实现(即通过处理器执行存储器中存储的程序来实现),或者一部分由硬件实现,另一部分由软件实现。在本实施例中,这些功能模块由软件实现,其应用的***结构例如图5所示。具体而言,控制装置100包括电子控制单元,电子控制单元包括作为硬件的处理器和存储器,存储器中存储有***软件(操作***、编译程序等)、支持软件(接口软件、工具软件与编译程序等)与应用软件,本实施例中的这些功能模块及这些功能模块所执行的控制方法对应于应用软件,应用软件被处理器执行时发挥本实施例中的这些功能模块的作用或者说执行控制方法。
图6为本实施例中的控制方法的一种处理流程图。
如图6所示,在S10中,从摄像头获取其对车内进行拍摄而得到的车内图像数据流,之后,S20中,对获取到的数据进行格式转化。
之后,在S40中监测是否有关门信号。这里可以是监测车门在预设时间内的一开一关(开关门信号),也可以仅监测关门信号。
当监测到关门信号时,在S30中,加载人脸识别模型以及进行算法PipeLine(在外接程序与其主机之间交换数据的管线段的线性通信模型)初始化。之后,在S50中,利用算法模型对车内图像数据进行人脸检测,并输出检测结果(人脸信息)。
在S60中,当检测到人脸时,确定人脸在车内空间中所属的区域(图4)。在S50中有可能检测到多个包含人脸的图像,此时,在S60中对识别到的每个人脸判断其属于车内空间中的哪个区域。在S70中,利用滑窗算法,统计副驾区域有人(人脸)的图像帧数在统计样本的总帧数中的占比。在S80中,根据S70的统计结果,判断副驾座位是否有乘坐者,例如当70帧内有20帧检测到副驾区域人脸时,则认为副驾座位有乘坐者,在S90中,生成指示副驾座位有乘坐者的图像检测结果信息,反之则认为副驾座位无乘坐者,可以生成指示副驾座位无乘坐者的图像检测结果信息,在S40中,监视关门信号。
另外,在S100中,获取副驾座位的压力传感器得到的压力检测信息,在压力检测信息指示副驾座位有乘坐者时,在S110中,生成针对显示装置200的分屏指令,并输出给显示装置200。
之后,在S130中,启动双屏幕操作处理,根据驾驶员与副驾乘员对显示装置200的操作进行相应的处理。在S120中,唤醒双通道语音控制(双麦克控屏)功能,根据驾驶员与副驾乘员的语音分别进行相应的控制,例如打开空调的控制等。
另外,在执行S120、S130的过程中,如果通过车内图像数据检测到副驾座位无乘坐者或者通过副驾座位的压力传感器的压力检测信息检测到副驾座位无乘坐者时,控制装置发送关闭分屏显示指令,使显示装置的分屏显示关闭(取消),此时,可以返回之前的通常显示状态。另外,分屏显示的关闭还可以通过驾驶员或副驾乘员等的主动操作来触发,该主动操作包括对实体按键的操作、对人机交互界面的触控操作或通过麦克风进行的语音控制操作。
图7为本实施例中的控制方法的另一种处理流程图。该图7的场景是根据摄像头拍摄到的车内图像数据判断为副驾座位有乘坐者,且根据副驾座位的压力传感器的压力检测信息判断为副驾座位有乘坐者的情景。
如图7所示,在S10中,从摄像头获取其对车内进行拍摄得到的车内图像数据流,之后,S20中,对获取到的数据进行格式转化。在S32中加载人脸识别模型以及进行算法PipeLine初始化,之后,在S34中,将车内图像数据流传送到其他处理模块。另外,在S32中还监测关门信号。当监测到关门信号时,在S50中,利用算法模型对车内图像数据进行人脸检测,并输出检测结果(人脸信息)。
在S60中,当检测到人脸时,确定人脸位于车内空间中的哪个区域(图4)。在S70中,利用滑窗算法,统计副驾区域有人(人脸)的图像帧数在统计样本的总帧数中的占比。当该占比达到规定阈值以上时,在S90中,确定为副驾座位有乘坐者,生成指示 副驾座位有乘坐者的图像检测结果信息。
另外,在S102中,根据副驾座位的压力传感器得到的压力检测信息判断副驾是否有人(重物),当判断为副驾座位有乘坐者时,在S112中,最终确定副驾座位有人(乘坐者),生成针对显示装置200的分屏指令,并输出给显示装置200。S120与S130中的处理与图6中相同,这里省略对其的详细描述。另外,作为其他实施例,控制装置可以不与显示装置200直接连接,例如控制装置设置在其它车载控制器中,此时,其他车载控制器向车机控制器发送分屏显示指令,车机控制器再向显示装置200发送分屏显示指令。
图8为本实施例中的控制方法的又一种处理流程图。图8的情景为根据车内图像数据判断副驾座位无乘坐者(人脸),根据压力传感器的压力检测信息判断为副驾座位有乘坐者(重物)。图8与图7相比主要区别在于S60之后的处理。如图8所示,在S60中确定检测到的人脸属于哪个区域。在S72中,利用滑窗算法统计副驾座位有乘坐者的图像帧数在统计总帧数中的占比,并且得到的该占比没有达到预设阈值。因此,在S92中,生成指示副驾座位无乘坐者(人脸)的图像检测结果信息。
在S102中,根据压力传感器的检测信息判断为副驾座位有乘坐者(重物)。在S114中,综合判断为副驾座位无乘坐者,结束本流程的处理。作为其他实施例,在S92中判断为副驾座位无乘坐者时,可以直接结束本流程的处理,而省略S102中的处理。另外,作为另一个实施例,可以在S32之前,监测关门信号,当监测到关门信号时,根据压力传感器的压力检测信息判断副驾作为是否有乘坐者(重物),判断为无乘坐者时,结束本流程的处理,判断为有乘坐者时,执行S32及之后的处理。
采用本实施例,当图像检测结果信息指示副驾座位有乘坐者,并且压力检测结果信息指示副驾座位有乘坐者时,生成分屏指令,而当图像检测结果信息与压力检测结果信息中的任意一方或双方指示副驾座位无乘坐者时,综合逻辑判断为副驾座位无乘坐者,并不生成分屏指令,从而能够减少不适当分屏的情况发生。
另外,采用本实施例,由于是在检测到关门信号后执行人脸识别,并结合压力检测信息确定是否分屏,即分屏指令的发出是以车门状态信息指示车门关闭为条件进行的,因此,能够更加适当地进行分屏。具体而言,假如采用上述比较例,单纯利用图像信息来检测副驾座位是否有乘坐者,那么,例如在有人打开车门将身体探入车内取放置在副驾座位上的书包内的物品时,会进行分屏显示。而采用本实施例,即使在有人打开车门将身体探入车内取放置在副驾座位上的书包内的物品的情景下,也不会不适当地分屏。另外,在检测到关门信号后,才执行对车内图像的算法处理以识别人脸,如此,能够降低***的处理负担。
图9为本申请一个实施例涉及的控制装置与控制方法的一种原理说明图。图10为本实施例涉及的控制方法的一种处理流程图。图12为本实施例中涉及的权限授予处理的一种示意说明图。
本实施例与上述实施例的主要区别在于,控制装置100A包括情景智能模块80,其他相同之处采用相同的附图标记并省略了对其的详细描述。本实施例中的情景智能模块80用于,在发出分屏指令时,获取蓝牙连接信息,如果蓝牙连接信息指示与驾驶员的手机通过蓝牙而与控制装置(车端)建立了通信连接时,通过蓝牙向驾驶员的手机 发出询问消息,以询问驾驶员是否进入隐私模式,并且,可选地,在驾驶员进行选择后,提示驾驶员对副驾显示区域进行权限授予。另外,作为蓝牙连接的代替,也可以是通用串行总线(Universal Serial Bus,USB)连接或Wi-Fi连接等其他无线通信连接。
进一步的,在决定执行分屏显示时,当检测到驾驶员的手机通过蓝牙连接到车机的情况下,通过手机(而不是车机的显示装置)询问驾驶员是否进入隐私模式,若没有蓝牙连接则无需询问是否进入隐私模式。另外,可选地,在分屏显示后执行蓝牙连接时也会询问是否进入隐私模式,如此,可以很好地保护驾驶员的隐私。另外,是否进入隐私模式的询问是在驾驶员的手机上显示,而不是在车载显示装置上显示。
在驾驶员选择进入隐私模式时,例如手机接受的来电与短信以及隐私应用通知信息不会在车载显示装置的显示画面上弹出通知(此功能可由用户自定义),另外,副驾乘员对副驾显示区域进行操作所打开的应用不会显示驾驶员使用历史记录(比如在打开地图应用时不会显示历史地点)。另外,隐私模式下例如还包括如下控制:隐藏车主账户隐私区域日程;隐藏语音便签;隐藏蓝牙语音消息;车载应用进入无痕模式。
另外,在决定分屏显示时,询问驾驶员以进行权限授予(建立不同权限的账户),适当的释放控制权给副驾乘员,实现双通道控屏,双用户操作车载***。
综上,在显示装置接收到分屏显示指令时,将一个屏幕显示区域分成两个子显示区域,划分后的两个子显示区域称为第一显示区域与第二显示区域,分别为主驾驶对应的屏幕区域与副驾驶对应的屏幕区域。在控制装置侧,对每个账户分配一个“空间”,空间指的是一套电子设备软件环境(本实施例中是车机软件环境),由各车载应用组成。权限授予阶段之前,车机***只有一个账户,即车主账户,也只有一套“空间”,称其为默认空间或第一空间,由驾驶员操作。在授予权限之后,形成与默认空间相互独立的一套应用空间(称为第二空间),双空间通过应用双开实现,默认空间内所有支持双开的应用可以出现在第二空间,供副驾驶进行交互操作。
有时,有的应用的主副驾操作是互斥的,即,对同一个应用来说,副驾和主驾同时只能由一个人与其交互,比如,对于音乐应用,主驾驶和副驾驶对该应用的操作会造成相互抢占。关于权限,在本实施例中有三种,即车主独有权限、授权账户权限与临时账户权限。如图12所示,在本实施例中,具体的权限内容为,车主独有权限下可执行如下操作:控制空调天窗等功能(可语音控制)、增删授权账户日程、增删车主账户日程、车载地图操控、控制车载娱乐功能(多媒体与游戏)、语音便签功能与手机投屏等;授权账户权限下可执行如下操作:增删授权账户日程、增删车主账户日程、车载地图操控、控制空调天窗等功能(可语音控制)、车载娱乐功能(多媒体与游戏)与手机投屏;临时账户权限下可执行如下操作:车载娱乐功能(多媒体与游戏)与手机投屏。
也就是说,相比于车主独有权限,授权账户权限下不能实现控制空调天窗等功能。临时账户权限下,除了能够实现车载娱乐功能(多媒体与游戏)与手机投屏外,车主独有权限与授权账户权限所具有的其他功能不能实现。
另外,在车主独有权限下,可执行驾驶控制(驾驶模式,底盘***调节,转向***调节,动力***调节),而授权账户权限下与临时账户权限下不能执行驾驶控制。
另外,当授予权限为车主权限时候,副驾显示区域所有的操控和主驾显示区域的 操控互斥,此时副驾与主驾共用(或者说共享)一套应用空间,分屏控制状态是双屏单空间,比如双方同时操作多媒体会互相打断。此时,第二显示区域的显示内容与第一显示区域容同步。这种模式保证了副驾有足够高的权限来替代驾驶员操作,当驾驶员不方便操作屏幕时,可由副驾完成相应操作而不会遇到权限受阻。
当授予权限为授权账户权限或者临时账户权限时候,副驾显示区域的操控与主驾独立,车内多媒体与游戏通过应用双开的方式实现(即,通过在副驾显示区域的操控可以独立播放音乐、独立操控游戏),车内音场的独立通过阵列扬声器传声特性,以及不同位置的扬声器(例如头枕扬声器)来实现主副驾双音场互不干扰。
进一步的,副驾乘员被授予“授权账户权限”后,***将为其设置“第二空间”,同时在第二空间为其分配一个有别于车主账户的账户。第一显示区域显示默认空间相应内容,第二屏幕显示第二空间相应内容。此时副驾乘员与主驾操作是不互斥的,双屏双空间通过应用双开来实现,比如主驾合副驾可同时在第一显示区域和第二显示区域上打开“音乐”应用,此时默认空间和第二空间的“音乐”应用其实是两个独立的应用,由于二者账户不同,二者可以实现在独立空间下独立账户的操作互不干扰,主驾可以播放歌曲A,副驾可以播放歌曲B,二者可以同时播放,通过车内阵列扬声器的传声特性,以及头枕扬声器来实现主副驾驶的双播放音场互不干扰。同时,授权账户有足够的权限来操作车主日程,车载地图导航等跟主驾强相关的应用。
另外,副驾乘员被授予“临时账户权限”后,***将为其开辟“第二空间”,在第二空间中分配给副驾乘员一个具有临时账户权限的账号,第二空间内的操作与默认空间是不互斥的。第一显示区域显示默认空间相应内容,第二显示区域显示第二空间相应内容。
另外,上面是以屏幕显示区域被划分为主驾显示区域与副驾显示区域这两个显示区域为例进行的描述,本领域技术人员由此也可以理解划分为更多个显示区域时的控制机制。
另外,关于授权账户,可以通过面部信息与语音信息识别副驾乘员,记住此人(副驾乘员),下次对其自动授权。当单通道(通过面部信息或语音信息)识别成功时,询问驾驶员是否进行权限授予。当双通道(通过面部信息和语音信息)均识别成功时,自动授予相应权限。
另外,本实施例与上述实施例的区别还包括,在本实施例中,获取车速信息,当车速超过高速阈值时,控制装置限制驾驶员通过人机交互界面操作来控制车载***,只开放语音控制通道。
下面参照图10对本实施例的一种控制流程进行描述。
在S200中,获取分屏信号(指示决定分屏显示的信号),当检测到分屏信号时,在S210中,确认车机是否通过蓝牙与驾驶员的手机建立了连接,当车机通过蓝牙与驾驶员的手机建立了连接时,向驾驶员手机发出询问信息,以询问驾驶员是否使显示装置200进入隐私模式,当驾驶员在手机上进行操作而做出选择时,在S230中向驾驶员手机发出询问信息,以询问驾驶员对副驾乘员授予哪种权限。另外,在S210中,当驾驶员选择进入隐私模式时,在S220中,使显示装置200的显示进入隐私模式,具体而言,在本实施例中,隐藏车主账户隐私区域日程、隐藏语音便签、隐藏蓝牙语音以及 短信消息、车载应用进入无痕模式。
另外,在S230中,当驾驶员在手机上进行操作,选择授予副驾乘员临时账户权限时,在S250中,进入临时账户模式。另外,当驾驶员选择授予副驾乘员授权账户权限时,在S240中,对副驾乘员进行面部信息与语音信息识别,将识别到的面部信息与语音信息存储在存储器中,为下次自动授权做准备。之后,在S260中,进入授权账户模式。
图11为本实施例涉及的控制方法的另一种处理流程图。图11中的S200、S210、S220中的处理与图8相同,在此省略对其的详细描述。如图11所示,当驾驶员在手机上进行操作而选择是否进入隐私模式后,在S270中,获取控制装置所具有的图像人脸识别模块对副驾乘员的人脸识别结果,获取控制装置所具有的语音识别模块对副驾乘员的语音识别结果,根据人脸识别结果与语音识别结果确定副驾乘员的身份,根据该身份从存储器中查询授权历史记录信息,确定副驾乘员的权限,根据查询结果,可以在S280中进入车主权限账户模式,在S282中进入授权账户模式,或者在S284中进入临时账户模式。
采用本实施例,考虑到驾驶员的隐私性需求,通过驾驶员在手机上的选择,可以进入隐私模式,从而能够提升驾驶员的体验。
图13为本申请一个实施例涉及的控制装置与控制方法的一种原理说明图。图14为本实施例涉及的控制装置与控制方法的另一种原理说明图。
参照图13,本实施例与上述实施例的主要区别在于,控制装置100B还包括语音定位模块70,语音定位模块70具有声音测距功能与语音区域划分功能,以检测到关门信号为触发条件,激活声音测距功能,即根据车内乘员的声音信息测量其与麦克风的距离,同时根据麦克风(例如多麦克风时)的位置等可以确定发出声音的车内乘员的位置,如此,能够根据车内乘员的位置确定其所属的区域(图2),进而能够判断副驾座位是否有乘坐者,根据判断结果生成相应的声音区域划分结果信息,用于指示副驾座位是否有乘坐者(说话人),参照图14,当声音定位识别结果信息、压力检测结果信息、图像检测结果信息这三者皆指示副驾座位有乘坐者时,生成分屏显示指令,使显示装置200进行分屏显示。
另外,可选地,还可以采用下表1中所示的方式:
表1副驾座位有无乘坐者判定方式例
场景1 | 场景2 | 场景3 | 场景4 | 场景5 | 场景6 | 场景7 | 场景8 | |
人脸区域检测 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
压力感应 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 |
语音区域划分 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
副驾检测结果 | 0 | 0 | redo | redo | redo | redo | 1 | 1 |
注:1代表成功检出有人,0代表没有检出;redo表示重新触发检测(获取新的车内图像信息与压力检测信息,据此进行检测),当连续重新触发超过5次,停止检测,认为副驾没人。
本实施例中,“人脸区域检测”与“压力感应”一定会产生有效结果,“语音区域划分”不一定会得到有效结果(因为有可能副驾驶不发出声音)。增加“语音区域 划分”功能的作用在于,为了增加区域划分的鲁棒性(触发重新检测)。另外,作为其他实施例,也可以在上表中的场景4和场景6的情况下,生成副驾座位有乘坐者的最终结果。
注意,上述仅为本申请的较佳实施例及所运用的技术原理。本领域技术人员会理解,本申请不限于这里所述的特定实施例,对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本申请的保护范围。因此,虽然通过以上实施例对本申请进行了较为详细的说明,但是本申请不仅仅限于以上实施例,在不脱离本申请的构思的情况下,还可以包括更多其他等效实施例,均属于本申请的保护范畴。
Claims (19)
- 一种车载显示装置的控制方法,其特征在于,包括:获取第一传感信息,获取第二传感信息;所述第一传感信息和第二传感信息分别用于确定第一副驾乘坐信息和第二副驾乘坐信息;当所述第一副驾乘坐信息指示副驾座位有乘坐者,并且所述第二副驾乘坐信息指示所述副驾座位有乘坐者时,发送分屏显示指令。
- 根据权利要求1所述的控制方法,其特征在于,所述第一传感信息包括车内图像信息,所述第二传感信息包括压力传感信息、红外传感信息、毫米波传感信息与声音传感信息中的一种或多种。
- 根据权利要求1或2所述的控制方法,其特征在于,所述确定第一副驾乘坐信息,响应于车门关闭信息。
- 根据权利要求2或3所述的控制方法,其特征在于,还包括:获取所述车内图像信息;根据所述车内图像信息获得人脸图像;当预先划定的副驾区域内存在所述人脸图像时,所述第一副驾乘坐信息指示所述副驾座位有乘坐者。
- 根据权利要求4所述的控制方法,其特征在于,还包括:在所述副驾区域存在的所述人脸图像的帧数在取样总帧数中的占比大于或等于预设值的条件下,第一副驾乘坐信息指示所述副驾座位有乘坐者。
- 根据权利要求1-5中任一项所述的控制方法,其特征在于,还包括:当检测到预设操作时,发送关闭分屏显示指令。
- 根据权利要求1-5中任一项所述的控制方法,其特征在于,还包括:当所述第一副驾乘坐信息或所述第二副驾乘坐信息中的至少一方指示所述副驾座位没有乘坐者时,发送关闭分屏显示指令。
- 一种车载显示装置的控制装置,其特征在于,包括获取模块与处理模块,所述获取模块用于获取第一传感信息,所述第一传感信息用于确定第一副驾乘坐信息;所述获取模块还用于获取第二传感信息,所述第二传感信息用于确定第二副驾乘坐信息;所述处理模块用于当所述第一副驾乘坐信息指示所述副驾座位有乘坐者,并且所述第二副驾乘坐信息指示所述副驾座位有乘坐者时,发送分屏显示指令。
- 根据权利要求8所述的控制装置,其特征在于,所述第一传感信息包括车内图像信息,所述第二传感信息包括压力传感信息、红外传感信息、毫米波传感信息与声音传感信息中的一种或多种。
- 根据权利要求8或9所述的控制装置,其特征在于,所述处理模块还用于:响应于车门关闭信息,确定所述第一副驾乘坐信息。
- 根据权利要求9或10所述的控制装置,其特征在于,还包括:所述获取模块还用于获取所述车内图像信息;所述处理模块还用于:根据所述车内图像信息获得人脸图像;当预先划定的副驾区域内存在所述人脸图像时,生成指示副驾座位有乘坐者的所述第一副驾乘坐信息。
- 根据权利要求11所述的控制装置,其特征在于,所述处理模块还用于在所述副驾区域存在的所述人脸图像的帧数在取样总帧数中的占比大于或等于预设值的条件下,生成指示所述副驾座位有乘坐者的所述第一副驾乘坐信息。
- 根据权利要求8-12中任一项所述的控制装置,其特征在于,所述处理模块还用于,当检测到预设操作时,发送关闭分屏显示指令。
- 根据权利要求8-12中任一项所述的控制装置,其特征在于,所述处理模块还用于,当所述第一副驾乘坐信息或所述第二副驾乘坐信息中的至少一方指示所述副驾座位没有乘坐者时,发送关闭分屏显示指令。
- 一种电子设备,其特征在于,包括显示装置与权利要求8-14中任一项所述的控制装置。
- 一种车辆,其特征在于,包括权利要求8-14任一项所述的控制装置或权利要求15所述的电子设备。
- 一种计算设备,其特征在于,包括:一个或多个处理器,所述处理器与存储器耦合,所述存储器存储有计算机程序,所述计算机程序当被所述处理器执行时使得所述处理器执行权利要求1至7中任一项所述的控制方法。
- 一种计算机程序产品,其特征在于,所述计算机程序产品包括:计算机程序代码,当所述计算机程序代码在计算机上运行时,实现如权利要求1至7中任一项所述的控制方法。
- 一种计算机可读存储介质,其上存储有程序指令,其特征在于,所述程序指令当被计算机执行时使得所述计算机执行权利要求1至7中任一项所述的控制方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202180001996.6A CN113423597B (zh) | 2021-04-30 | 2021-04-30 | 车载显示装置的控制方法、控制装置、电子设备、车辆 |
PCT/CN2021/091580 WO2022227028A1 (zh) | 2021-04-30 | 2021-04-30 | 车载显示装置的控制方法、控制装置、电子设备、车辆 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2021/091580 WO2022227028A1 (zh) | 2021-04-30 | 2021-04-30 | 车载显示装置的控制方法、控制装置、电子设备、车辆 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022227028A1 true WO2022227028A1 (zh) | 2022-11-03 |
Family
ID=77719484
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/091580 WO2022227028A1 (zh) | 2021-04-30 | 2021-04-30 | 车载显示装置的控制方法、控制装置、电子设备、车辆 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113423597B (zh) |
WO (1) | WO2022227028A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115033155A (zh) * | 2022-06-08 | 2022-09-09 | 上海集度汽车有限公司 | 显示设备的显示控制方法、装置、车辆及介质 |
CN115190199B (zh) * | 2022-07-08 | 2023-07-21 | 润芯微科技(江苏)有限公司 | 一种通过移动端扫码操作车机功能的方法 |
CN115691490A (zh) * | 2022-10-09 | 2023-02-03 | 蔚来汽车科技(安徽)有限公司 | 动态切换音区的方法、语音交互方法、设备、介质及车辆 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103359038A (zh) * | 2013-08-05 | 2013-10-23 | 北京汽车股份有限公司 | 一种识别儿童坐副驾驶位置的方法、***及汽车 |
CN105966307A (zh) * | 2015-11-02 | 2016-09-28 | 乐卡汽车智能科技(北京)有限公司 | 一种在车辆内部进行安全检测的方法及装置 |
DE102017122329A1 (de) * | 2017-09-26 | 2019-03-28 | Borgward Trademark Holdings Gmbh | Mehrbildschirminteraktionseinrichtung und Fahrzeug |
CN109614001A (zh) * | 2018-11-05 | 2019-04-12 | 广州小鹏汽车科技有限公司 | 一种车载屏幕触控操作权限的控制方法、控制装置以及控制设备 |
CN209103270U (zh) * | 2018-12-18 | 2019-07-12 | 扬州晶视科技发展有限公司 | 一种车载影音分屏显示*** |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107845256A (zh) * | 2017-11-23 | 2018-03-27 | 何世容 | 一种监控拼车***及监控拼车方法 |
CN108973853B (zh) * | 2018-06-15 | 2021-01-15 | 威马智慧出行科技(上海)有限公司 | 一种车辆警示装置及车辆警示方法 |
CN111891001A (zh) * | 2020-07-29 | 2020-11-06 | 中国第一汽车股份有限公司 | 一种副驾驶座椅自适应调节方法及调节*** |
-
2021
- 2021-04-30 WO PCT/CN2021/091580 patent/WO2022227028A1/zh active Application Filing
- 2021-04-30 CN CN202180001996.6A patent/CN113423597B/zh active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103359038A (zh) * | 2013-08-05 | 2013-10-23 | 北京汽车股份有限公司 | 一种识别儿童坐副驾驶位置的方法、***及汽车 |
CN105966307A (zh) * | 2015-11-02 | 2016-09-28 | 乐卡汽车智能科技(北京)有限公司 | 一种在车辆内部进行安全检测的方法及装置 |
DE102017122329A1 (de) * | 2017-09-26 | 2019-03-28 | Borgward Trademark Holdings Gmbh | Mehrbildschirminteraktionseinrichtung und Fahrzeug |
CN109614001A (zh) * | 2018-11-05 | 2019-04-12 | 广州小鹏汽车科技有限公司 | 一种车载屏幕触控操作权限的控制方法、控制装置以及控制设备 |
CN209103270U (zh) * | 2018-12-18 | 2019-07-12 | 扬州晶视科技发展有限公司 | 一种车载影音分屏显示*** |
Also Published As
Publication number | Publication date |
---|---|
CN113423597A (zh) | 2021-09-21 |
CN113423597B (zh) | 2023-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022227028A1 (zh) | 车载显示装置的控制方法、控制装置、电子设备、车辆 | |
US10821988B2 (en) | Vehicle control system based on face recognition | |
US11169618B2 (en) | Systems and methods for distinguishing gestures | |
US9977593B2 (en) | Gesture recognition for on-board display | |
US10032453B2 (en) | System for providing occupant-specific acoustic functions in a vehicle of transportation | |
CN108725357B (zh) | 基于人脸识别的参数控制方法、***与云端服务器 | |
US9550419B2 (en) | System and method for providing an augmented reality vehicle interface | |
US9942522B2 (en) | In-vehicle camera system | |
JP4305289B2 (ja) | 車両用制御装置およびその装置を備える車両用制御システム | |
US9560387B2 (en) | Interface for wireless data transmission in a motor vehicle, and computer program product | |
US10861457B2 (en) | Vehicle digital assistant authentication | |
US20160041562A1 (en) | Method of controlling a component of a vehicle with a user device | |
US20130096771A1 (en) | Apparatus and method for control of presentation of media to users of a vehicle | |
WO2016145100A1 (en) | Vehicle supervising of occupant applications | |
US10674003B1 (en) | Apparatus and system for identifying occupants in a vehicle | |
US10862764B2 (en) | Universal console chassis for the car | |
CN111204219A (zh) | 车辆用显示装置、车辆用显示方法及存储介质 | |
GB2525039A (en) | An audio-visual communications system integrated into a vehicle | |
GB2549582A (en) | An audio-visual communications system integrated into a vehicle | |
CN116249635A (zh) | 用于激活包括数字后视镜***的车辆中的须经授权功能的激活方法和激活装置 | |
GB2555088A (en) | Interface apparatus and method | |
WO2024139737A1 (zh) | 车机***的消息展示方法、电子设备及车辆 | |
US20230242039A1 (en) | Method and device for adjusting comfort and/or safety functions of a vehicle | |
US20230339369A1 (en) | Vehicle seat auto-adjust based on rear seat occupancy | |
GB2583890A (en) | Controller for an infotainment system of a vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21938502 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21938502 Country of ref document: EP Kind code of ref document: A1 |