CN117485357A - Vehicle-mounted function key recommending method and device, electronic equipment and storage medium - Google Patents

Vehicle-mounted function key recommending method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117485357A
CN117485357A CN202311444928.4A CN202311444928A CN117485357A CN 117485357 A CN117485357 A CN 117485357A CN 202311444928 A CN202311444928 A CN 202311444928A CN 117485357 A CN117485357 A CN 117485357A
Authority
CN
China
Prior art keywords
driver
vehicle
function key
sight
sight line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311444928.4A
Other languages
Chinese (zh)
Inventor
顾秀颖
朱诚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Changan Automobile Co Ltd
Original Assignee
Chongqing Changan Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Changan Automobile Co Ltd filed Critical Chongqing Changan Automobile Co Ltd
Priority to CN202311444928.4A priority Critical patent/CN117485357A/en
Publication of CN117485357A publication Critical patent/CN117485357A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0098—Details of control systems ensuring comfort, safety or stability not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The invention provides a vehicle-mounted function key recommending method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: by monitoring the sight line of the driver, when the total time of the sight line of the driver leaving the preset sight line range exceeds a preset time threshold value; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a preset frequency threshold value, screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching out vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver, wherein the driver needs to concentrate on driving in the process of driving the automobile, and the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver in the driving process, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.

Description

Vehicle-mounted function key recommending method and device, electronic equipment and storage medium
Technical Field
The invention relates to the field of Internet of vehicles, in particular to a vehicle-mounted function key recommending method and device, electronic equipment and a storage medium.
Background
In recent years, the technology of the automobile industry has rapidly developed, and the loading amount of intelligent driving systems has rapidly increased. Intelligent driving should provide a more comfortable and secure riding experience, which is also a recognized technological development direction.
When a driver needs to concentrate on driving an automobile, in the actual driving process, if the driver needs to control other function keys, the driver needs to be distracted to find the corresponding function keys, so that the driver is distracted to drive, and safety accidents are caused.
Disclosure of Invention
In view of the above drawbacks of the prior art, the present invention provides a method, an apparatus, an electronic device, and a storage medium for recommending a vehicle-mounted function key, so as to solve the above technical problems.
The invention provides a vehicle-mounted function key recommending method, which comprises the following steps: monitoring the sight of a driver during the running process of the vehicle; when the total time that the sight line of the driver leaves the preset sight line range exceeds a first threshold value; or, the frequency of the sight line of the driver leaving the preset sight line range in the preset time period is greater than a second threshold value; splicing all sensor data representing the running process of the vehicle to form a data vector; screening driver operation records with the similarity to the data vector being greater than or equal to a third threshold value from a preset database, and searching vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver.
In an embodiment of the present invention, before screening out the driver operation records with the similarity to the data vector greater than or equal to the third threshold from the preset database, the method further includes: acquiring reference sensor data corresponding to all vehicle function keys, wherein the reference sensor data comprise temperature, vehicle speed, road type and congestion condition; converting the reference sensor data into vectors, and generating database record vectors, wherein the database record vectors correspond to driver operation records; and storing the database record vector in a preset database.
In an embodiment of the present invention, finding a vehicle function key corresponding to a driver operation record includes: determining a region with the longest line-of-sight residence time period or a region with the line-of-sight residence frequency larger than a fourth threshold value as an alternative region; and sequencing the function keys in the alternative area according to the use frequency of the driver from high to low, and determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record.
In an embodiment of the present invention, determining a region in which a line of sight dwell period is longest, or a region in which a line of sight dwell frequency is greater than a fourth threshold value, as the candidate region includes: acquiring sight line data of a driver, wherein the sight line data comprises a sight line stay area and single stay time; dividing the line-of-sight stay region into a plurality of regions; the sight line stay time of the driver in each area is counted to obtain the total sight line stay time of the driver in each area; determining the area with the longest stay time of the sight line as an alternative area; or, obtaining the sight line residence frequency by counting the single sight line residence time of the driver in each area; and determining a region with the sight-line retention frequency being greater than or equal to a fourth threshold value as an alternative region.
In an embodiment of the present invention, the step of screening the driver operation records having a similarity with the data vector greater than or equal to the third threshold from the predetermined database includes: calculating the similarity between the data vector and the database record vector through a cosine similarity formula; when the similarity is larger than or equal to a third threshold value, screening out driver operation records corresponding to the data vectors larger than or equal to the third threshold value; and when the similarity is smaller than a third threshold value, comparing similarity values of other data vectors and the database record vectors until the driver operation record is screened out.
In an embodiment of the present invention, recommending the found vehicle function key to the driver includes: recommending the found vehicle function keys to the driver, including: inquiring whether the driver uses the vehicle function keys corresponding to the driver operation record or not, and monitoring voice data of the driver; when the voice data is monitored to be the voice data which is replied by the target object based on the inquiry and is used for starting the vehicle function key corresponding to the driver operation record, starting the function corresponding to the vehicle function key corresponding to the driver operation record; and when the voice data is not monitored as the voice data replied by the target object based on the inquiry and is used for starting the target function key, inquiring the function key with the highest use frequency of the history record, determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record, and starting the function corresponding to the vehicle function key corresponding to the driver operation record.
In one embodiment of the present invention, a method for monitoring a driver's line of sight during driving of a vehicle includes: obtaining image information of the driver through a camera of the vehicle; and identifying the image information, and monitoring the sight of the driver according to an image identification result.
The invention provides a vehicle-mounted function key recommending device, which comprises: the monitoring module is used for monitoring the sight of a driver in the running process of the vehicle; the splicing module is used for enabling the total time of the sight line of the driver leaving the preset sight line range to exceed a first threshold value; or, the frequency of the sight line of the driver leaving the preset sight line range in the preset time period is greater than a second threshold value; splicing all sensor data representing the running process of the vehicle to form a data vector; the function recommendation module is used for screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver.
The invention has the beneficial effects that: by monitoring the sight line of the driver, when the total time of the sight line of the driver leaving the preset sight line range exceeds a preset time threshold value; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a preset frequency threshold value, screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching out vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver, wherein the driver needs to concentrate on driving in the process of driving the automobile, and the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver in the driving process, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:
FIG. 1 is a system architecture diagram of an in-vehicle function key recommendation method shown in an exemplary embodiment of the present application;
FIG. 2 is a system diagram illustrating an in-vehicle function key recommendation method according to an exemplary embodiment of the present application;
FIG. 3 is a block diagram of a cloud and end software for vehicle-mounted function key recommendation, as shown in an exemplary embodiment of the present application;
FIG. 4 is a flowchart illustrating an on-board function key recommendation method according to an exemplary embodiment of the present application;
FIG. 5 is a schematic view of a line-of-sight zone of a vehicle, as shown in an exemplary embodiment of the present application;
FIG. 6 is a timing diagram of an in-vehicle function key recommendation method shown in an exemplary embodiment of the present application;
FIG. 7 is a block diagram of an in-vehicle function key recommending apparatus shown in an exemplary embodiment of the present application;
fig. 8 shows a schematic diagram of a computer system suitable for use in implementing the electronic device of the embodiments of the present application.
Detailed Description
Further advantages and effects of the present invention will become readily apparent to those skilled in the art from the disclosure herein, by referring to the accompanying drawings and the preferred embodiments. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be understood that the preferred embodiments are presented by way of illustration only and not by way of limitation.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
In the following description, numerous details are set forth in order to provide a more thorough explanation of embodiments of the present invention, it will be apparent, however, to one skilled in the art that embodiments of the present invention may be practiced without these specific details, in other embodiments, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the embodiments of the present invention.
Referring to fig. 1, fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solution of the embodiment of the present invention may be applied. As shown in fig. 1, the system architecture may include a camera module 110, a vehicle-mounted host 120, and a cloud 130, where the camera module 110 is configured to obtain image information of a driver in a driving area, identify the image information, monitor a line of sight of the driver, and when it is monitored that a total time when the line of sight of the driver leaves a preset line of sight range exceeds a first threshold; or, if the frequency of the driver's sight leaving the preset sight range within the preset time period is greater than the second threshold, all the sensor data representing the running process of the vehicle are spliced by the vehicle-mounted host computer 120 to form a data vector, the driver operation records with the similarity to the data vector being greater than or equal to the third threshold are screened out from the preset database, and the vehicle function keys corresponding to the driver operation records are found out; and recommending the found vehicle function key to a driver, inquiring whether the driver uses the target function key or not through the vehicle-mounted host 130, monitoring voice data of the driver, and starting a function corresponding to the target function key when the voice data is monitored to be the voice data which is replied by the target object based on the inquiry and is used for starting the target function key. In one embodiment of the present application, the on-board host module begins the recommended decision process when the driver achieves a long or frequent exit from the road. The vehicle-mounted host computer 120 judges whether the driver is looking for the relevant function keys or not through the sight area and the change frequency of the driver in a period of time detected by the camera module 110. If so, acquiring data of the current vehicle sensor, uploading the data to the cloud 130, and requesting the cloud 130 for the function sequencing of the driver operation under the similar condition. According to the function division of the area with the longest or most frequent stay of the driver's sight, the function sequence recommended by the cloud 130 is combined, and finally, the function keys recommended by decision making are recommended. By monitoring the sight line of the driver, when the total time of the sight line of the driver leaving the preset sight line range exceeds a preset time threshold value; or, if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than the preset frequency threshold value, triggering the function key recommendation. In the process that a driver drives an automobile, the driver needs to concentrate on driving, the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver during driving, and the screened function keys are confirmed with the driver in a voice interaction inquiry mode, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
In an embodiment of the present application, fig. 2 shows a system diagram of a vehicle-mounted function key recommending method applied to an exemplary embodiment of the present invention, where, as shown in fig. 2, a camera module is a camera, and a plurality of sensors, namely, a sensor 1, a sensor 2 and a sensor 3, are installed on a vehicle-mounted host, and a sight area and a change frequency of a driver are monitored by the camera within a period of time, so as to determine whether the driver is looking for a related function key. If so, acquiring data of the current vehicle sensor, uploading the data to the cloud, and requesting the function keys operated by the driver under the similar condition to the cloud for sorting. And according to the function division of the area with the longest or most frequent stay of the driver's sight, combining the function key sequence recommended by the cloud, and finally recommending the function keys recommended by decision making. During driving, the vision of the driver is monitored, the intention function keys of the current driver are automatically recommended, and the screened function keys are confirmed with the driver in a voice interaction inquiry mode, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
In an embodiment of the application, fig. 3 shows a vehicle-mounted function key recommending cloud and a vehicle-end software module diagram shown in an exemplary embodiment of the invention, and as shown in fig. 3, the cloud is used for providing data services, and a vehicle-mounted host includes an image algorithm unit, a recommending and recommending decision unit, a communication unit and a voice interaction unit. The image algorithm module is used for realizing that the detection module continuously detects the sight of a driver after the vehicle is started and is in a non-parking gear, and notifying the recommendation decision module of the detection result. The recommendation decision making module is used for starting a recommendation decision making process when the driver leaves the road surface for a long time or frequently. The recommendation decision module is used for judging whether the driver is looking for related function keys or not through the sight area and the change frequency of the driver in a period of time detected by the image detection module. If so, acquiring data of the current vehicle sensor, uploading the data to the cloud, and requesting the function keys operated by the driver under the similar condition to the cloud for sorting. And according to the function division of the area with the longest or most frequent stay of the driver's sight, combining the function sequencing of cloud recommendation, and finally recommending the function of decision recommendation. The voice interaction module is used for reminding a driver of paying attention to driving safety through voice and inquiring whether the driver needs to use the function recommended by the recommendation decision module. After the driver is negated, the first few functions recommended by the cloud can be broadcasted to the driver for selection. Thus, by monitoring the sight line of the driver, when the time that the sight line of the driver is not in the preset range exceeds the preset time threshold value, the function key recommendation is triggered. In the process that a driver drives an automobile, the driver needs to concentrate on driving, the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver during driving, and the screened function keys are confirmed with the driver in a voice interaction inquiry mode, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
Referring to fig. 4, fig. 4 is a flowchart illustrating an on-board function key recommendation method according to an exemplary embodiment of the present application. As shown in fig. 4, in an exemplary embodiment, the vehicle-mounted function key recommending method at least includes steps S410 to S430, which are described in detail as follows:
step S410 monitors the line of sight of the driver during the running of the vehicle.
In one embodiment of the application, the line of sight of a driver during driving of the vehicle is monitored by installing a high-definition camera inside the vehicle.
Step S420, when the total time of the sight line of the driver leaving the preset sight line range exceeds a first threshold value; or, the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a second threshold value; and splicing all sensor data representing the running process of the vehicle to form a data vector.
In one embodiment of the present application, sensor data during vehicle travel is stitched into data vectors by a concatate_sensor_data function.
Step S430, screening driver operation records with the similarity to the data vector being greater than or equal to a third threshold value from a preset database, and searching out vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to a driver.
In one embodiment of the present application, the similarity between the sensor data and the database record vector is calculated by the Euclidean distance formula.
In one embodiment of the present application, the total time when the driver's line of sight leaves the preset line of sight range exceeds a first threshold; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a second threshold value, acquiring current vehicle sensor data, uploading the current vehicle sensor data to the cloud, requesting the function sequencing of the driver operation under the similar condition to the cloud, and recommending and deciding a recommendation function according to the sequencing list.
In fig. 4, by monitoring the driver's line of sight, when the total time when the driver's line of sight leaves the preset line of sight range exceeds the preset time threshold; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a preset frequency threshold value, screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching out vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver, wherein the driver needs to concentrate on driving in the process of driving the automobile, and the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver in the driving process, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
In an embodiment of the present application, fig. 5 is a schematic view area division diagram of an exemplary vehicle according to the present invention, as shown in fig. 5, 2.1 in fig. 5 is an area where a line of sight should be gazed during normal driving, 2.2, 2.3, and 2.4 are areas where function keys exist, each area is divided into areas that can be clearly defined according to a line of sight algorithm, and a function key or a software formation list existing in each area is stored in an on-board host in advance.
In one embodiment of the present application, before screening the driver operation record with the similarity to the data vector greater than or equal to the third threshold from the preset database, the method further includes: acquiring reference sensor data corresponding to all vehicle function keys, wherein the reference sensor data comprises temperature, vehicle speed, road type and congestion condition; converting the reference sensor data into vectors, and generating database record vectors, wherein the database record vectors correspond to driver operation records; the database record vector is stored in a preset database. Through the database record vector and the driver operation record that the driver used the function button to correspond are stored in the database, when being convenient for follow-up car to travel and start, can predict the intention function button of driver based on current vehicle sensor data to guarantee to recommend the intention function button of current driver automatically through monitoring driver's sight during driving, just so need not the driver to scatter energy in addition and look for corresponding function button, improved the security of driving.
In one embodiment of the present application, the reference sensor data is vehicle sensor data, and after the driver manually uses the function key, the vehicle sensor data corresponding to the function key at the same time is acquired, for example: the method comprises the steps of (1) uploading vehicle sensor data to a cloud end, converting reference sensor data into vectors by the cloud end, generating database record vectors, wherein the database record vectors correspond to driver operation records; the database record vector is stored in a preset database. Through the database record vector and the driver operation record that the driver used the function button to correspond are stored in the database, when being convenient for follow-up car to travel and start, can predict the intention function button of driver based on current vehicle sensor data to guarantee to recommend the intention function button of current driver automatically through monitoring driver's sight during driving, just so need not the driver to scatter energy in addition and look for corresponding function button, improved the security of driving.
In one embodiment of the present application, locating a vehicle function key corresponding to a driver operation record includes: determining a region with the longest line-of-sight residence time period or a region with the line-of-sight residence frequency larger than a fourth threshold value as an alternative region; and sequencing the function keys in the alternative area according to the use frequency of the driver from high to low, and determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record. When the driver wants to turn on the function key, the driver is likely to find the corresponding function key through the sight line and stop the sight line in the area corresponding to the function key, so that the area with the longest stay time of the sight line is determined as an alternative area, or the area with the stay frequency of the sight line larger than the fourth threshold value is determined as an alternative area, and the function key with the highest use frequency is determined as the vehicle function key corresponding to the driver operation record, thereby accurately determining the intention function key of the driver, avoiding the driver to separately and dispersedly find the corresponding function key, and improving the driving safety.
In one embodiment of the application, determining the longest line-of-sight dwell period area, or an area where the line-of-sight dwell frequency is greater than a fourth threshold, as the candidate area includes: acquiring sight line data of a driver, wherein the sight line data comprises a sight line stay area and single stay time; dividing the sight line stay area into a plurality of areas; the sight line stay time of the driver in each area is counted to obtain the total sight line stay time of the driver in each area; determining the area with the longest stay time of the sight line as an alternative area; or, obtaining the sight line residence frequency by counting the single sight line residence time of the driver in each area; and determining a region with the sight-line retention frequency being greater than or equal to a fourth threshold value as an alternative region. When the driver wants to turn on the function key, the driver is likely to find the corresponding function key through the sight line and stop the sight line in the area corresponding to the function key, so that the area with the longest stay time of the sight line is determined as an alternative area, or the area with the stay frequency of the sight line larger than the fourth threshold value is determined as an alternative area, and the function key with the highest use frequency is determined as the vehicle function key corresponding to the driver operation record, thereby accurately determining the intention function key of the driver, avoiding the driver to separately and dispersedly find the corresponding function key, and improving the driving safety.
In one embodiment of the present application, a region where the driver's line of sight stays longest for a period of time is calculated, and the operable function keys of the region are compared with a recommended function key list returned by the cloud, and then a voice interaction module is invoked to recommend a function with the highest history use frequency in the functions of the region, in this embodiment, by collecting line of sight data of the driver: eye tracking devices or sensors are used to collect driver gaze data. The gaze data may include a gaze location and a single dwell time. The collected line of sight dwell position data is divided into different regions. The regions may be divided according to a dashboard layout or screen of the vehicle, such as a left-side dashboard region, a right-side dashboard region, a center screen region, and the like. For each zone, the total time the driver stays in that zone is calculated. The line of sight dwell position may be combined with the single dwell time to calculate the dwell time for each zone. Determining the area with the longest residence time: the area with the longest residence time is found from all the areas and is determined as an alternative area. Comparing the region operable function with the cloud recommendation list: and comparing the operable function keys of the alternative area with the recommended list function keys returned by the cloud, and determining the common function keys as vehicle function keys corresponding to the driver operation record. Calling a voice interaction module to recommend: and suggesting the common function keys to the driver, and recommending through the voice interaction module. By monitoring the sight line of the driver, when the time that the sight line of the driver is not in the preset range exceeds the preset time threshold value, function key recommendation is triggered, the area with the longest sight line stay time is determined as an alternative area, or the area with the sight line stay frequency larger than the fourth threshold value is determined as an alternative area, and the function key with the highest use frequency is determined as the vehicle function key corresponding to the driver operation record, so that the intention function key of the driver is accurately determined. In the process that the driver drives the automobile, the driver needs to concentrate on driving, and the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver during driving, so that the driver does not need to disperse energy to search for the corresponding function keys, and the driving safety is improved.
In the above embodiment, if there is no common function key in the operable function keys of the candidate area and the recommended list function keys returned by the cloud, the function with the highest use frequency of the history record is queried: and according to the operable functions in the area with the longest stay time, querying the history record to find the function with the highest use frequency. Calling a voice interaction module to recommend: and suggesting the function with the highest history record use frequency to a driver, and recommending through the voice interaction module. After being queried through the voice interaction module, if the target function key is not determined, the target function key can be obtained through querying the history record, so that the intention function key of the driver can be obtained more accurately, and the user experience is improved.
In one embodiment of the present application, screening driver operation records from a preset database, the similarity to the data vector being greater than or equal to a third threshold value, includes: calculating the similarity between the data vector and the database record vector through a cosine similarity formula; when the similarity is greater than or equal to a third threshold value, screening out driver operation records corresponding to the data vectors greater than or equal to the third threshold value; and when the similarity is smaller than a third threshold value, comparing the similarity values of other data vectors and the database record vectors until the driver operation record is screened out. The data vector corresponding to the current vehicle sensor data is compared with the database record vector in similarity, so that the intention function key of the driver at the current moment can be accurately predicted, the driver does not need to disperse energy to find the corresponding function key, and the driving safety is improved.
In one embodiment of the present application, recommending the found vehicle function key to the driver includes: inquiring whether a driver uses a vehicle function key corresponding to the driver operation record, and monitoring voice data of the driver; when the voice data is monitored as the voice data of the vehicle function keys corresponding to the driver operation records and replied to by the target object based on the inquiry, starting the functions corresponding to the vehicle function keys corresponding to the driver operation records; and when the voice data is not monitored as the voice data which is replied by the target object based on the inquiry and is used for starting the target function key, inquiring the function key with the highest use frequency of the history record, determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record, and starting the function corresponding to the vehicle function key corresponding to the driver operation record. Therefore, not only the intention function key of the driver can be determined by detecting the sight of the driver, but also the intention function key can be determined by the history record, so that the intention function key of the driver at the current moment can be accurately predicted, the driver does not need to disperse energy to find the corresponding function key, and the driving safety is improved.
In one embodiment of the present application, monitoring a driver's line of sight during travel of a vehicle includes: obtaining image information of a driver through a camera of the vehicle; and identifying image information, and monitoring the sight of the driver according to the image identification result.
In one embodiment of the present application, fig. 6 shows a timing diagram of a method for recommending function keys on board an exemplary vehicle of the present invention. As shown in fig. 6, when the driver uses the function key, the communication unit obtains the corresponding vehicle sensor data when the driver uses the function key, converts the vehicle sensor data into a database record vector, stores the database record vector and the driver operation record corresponding to the vehicle sensor data in a preset database, and after the vehicle is started and the gear is not a stop gear, the image algorithm unit identifies the sight of the driver through the image identification algorithm and transmits the image identification result to the recommendation decision unit, and the recommendation decision unit judges according to the image identification result that the total time when the sight of the driver leaves the preset sight range exceeds a first threshold; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a second threshold value, acquiring current vehicle sensor data, uploading the vehicle sensor data corresponding to the current vehicle to a cloud, initiating a communication request to the cloud, inquiring vehicle type data after the cloud receives the request information, splicing the vehicle sensor data corresponding to the current vehicle to form a data vector, and calculating the similarity between the data vector and a database record vector through a cosine similarity formula; when the similarity is greater than or equal to a third threshold value, screening driver operation records corresponding to data vectors greater than or equal to the third threshold value, arranging functions corresponding to the screened driver operation records according to the order of the use frequency from high to low, and forming a function key list. The recommendation decision unit acquires sight line data of a driver and divides a sight line stay area into a plurality of areas; the sight line stay time of the driver in each area is counted to obtain the total sight line stay time of the driver in each area; determining the area with the longest stay time of the sight line as an alternative area; comparing the function keys in the alternative area with a function key list returned by the cloud to obtain target function keys, inquiring whether a driver uses the target function keys or not through a voice interaction unit, and monitoring voice data of the driver; and when the voice data is monitored as the voice data which is replied by the target object based on the inquiry and is used for starting the target function key, starting the function corresponding to the target function key. During the driving process of the automobile, the driver needs to concentrate on driving, the sight line of the driver is monitored, and when the total time of the sight line of the driver leaving the preset sight line range exceeds a preset time threshold value; or, if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than the preset frequency threshold, the function key recommendation is triggered, the intention function key of the current driver is automatically recommended, and the screened function key is confirmed with the driver in a voice interaction inquiry mode, so that the driver does not need to disperse efforts to find the corresponding function key additionally, and the driving safety is improved.
Fig. 7 is a block diagram of an in-vehicle function key recommending apparatus shown in an exemplary embodiment of the present application. As shown in fig. 7, the exemplary in-vehicle function key recommending system includes:
the system comprises a monitoring module 710, a splicing module 720 and a function recommending module 730, wherein the monitoring module 710 is used for monitoring the sight of a driver in the running process of the vehicle; a stitching module 720, configured to, when a total time when the line of sight of the driver leaves the preset line of sight range exceeds a first threshold; or, the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a second threshold value; splicing all sensor data representing the running process of the vehicle to form a data vector; the function recommendation module 730 is configured to screen out driver operation records with a similarity to the data vector greater than or equal to a third threshold from a preset database, and find out a vehicle function key corresponding to the driver operation record; and recommending the found vehicle function keys to a driver. By monitoring the sight line of the driver, when the total time of the sight line of the driver leaving the preset sight line range exceeds a preset time threshold value; or if the frequency of the sight line of the driver leaving the preset sight line range within the preset time period is greater than a preset frequency threshold value, screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching out vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver, wherein the driver needs to concentrate on driving in the process of driving the automobile, and the intention function keys of the current driver are automatically recommended by monitoring the sight of the driver in the driving process, so that the driver does not need to disperse energy to find the corresponding function keys, and the driving safety is improved.
In an embodiment of the present invention, the function recommendation module 730 is configured to obtain reference sensor data corresponding to all vehicle function keys, where the reference sensor data includes temperature, speed, road type and congestion; converting the reference sensor data into vectors, and generating database record vectors, wherein the database record vectors correspond to driver operation records; the database record vector is stored in a preset database.
In an embodiment of the present invention, the function recommendation module 730 is configured to determine, as the candidate area, an area with the longest line of sight residence time period or an area with a line of sight residence frequency greater than a fourth threshold; and sequencing the function keys in the alternative area according to the use frequency of the driver from high to low, and determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record.
In an embodiment of the present invention, the function recommendation module 730 is configured to determine, as an alternative area, an area with a longest line of sight dwell period or an area with a line of sight dwell frequency greater than a fourth threshold, including: acquiring sight line data of a driver, wherein the sight line data comprises a sight line stay area and single stay time; dividing the sight line stay area into a plurality of areas; the sight line stay time of the driver in each area is counted to obtain the total sight line stay time of the driver in each area; determining the area with the longest stay time of the sight line as an alternative area; or, obtaining the sight line residence frequency by counting the single sight line residence time of the driver in each area; and determining a region with the sight-line retention frequency being greater than or equal to a fourth threshold value as an alternative region.
In an embodiment of the present invention, the function recommendation module 730 is configured to screen driver operation records with similarity to the data vector greater than or equal to a third threshold from a preset database, and includes: calculating the similarity between the data vector and the database record vector through a cosine similarity formula; when the similarity is greater than or equal to a third threshold value, screening out driver operation records corresponding to the data vectors greater than or equal to the third threshold value; and when the similarity is smaller than a third threshold value, comparing the similarity values of other data vectors and the database record vectors until the driver operation record is screened out.
In an embodiment of the present invention, the vehicle-mounted function key recommending device further includes a voice interaction module 740 for querying whether the driver uses the vehicle function key corresponding to the driver operation record and monitoring the voice data of the driver; when the voice data is monitored as the voice data of the vehicle function keys corresponding to the driver operation records and replied to by the target object based on the inquiry, starting the functions corresponding to the vehicle function keys corresponding to the driver operation records; and when the voice data is not monitored as the voice data which is replied by the target object based on the inquiry and is used for starting the target function key, inquiring the function key with the highest use frequency of the history record, determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record, and starting the function corresponding to the vehicle function key corresponding to the driver operation record.
In an embodiment of the invention, the monitoring module is used for obtaining image information of a driver through a camera of the vehicle; and identifying image information, and monitoring the sight of the driver according to the image identification result.
It should be noted that, the vehicle-mounted function key recommending system provided in the above embodiment and the vehicle-mounted function key recommending method provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit execute the operation has been described in detail in the method embodiment, which is not repeated here. In practical application, the road condition refreshing device provided in the above embodiment may distribute the functions to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above, which is not limited herein.
The embodiment of the application also provides electronic equipment, which comprises: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic equipment realizes the vehicle-mounted function key recommending method provided in each embodiment.
Fig. 8 shows a schematic diagram of a computer system suitable for use in implementing the electronic device of the embodiments of the present application. It should be noted that, the computer system 800 of the electronic device shown in fig. 8 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.
As shown in fig. 8, the computer system 800 includes a central processing unit (Central Processing Unit, CPU) 801 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 802 or a program loaded from a storage section 808 into a random access Memory (Random Access Memory, RAM) 803. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other by a bus 804. An Input/Output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and the like, and a speaker, and the like; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN (Local Area Network ) card, modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage portion 808 as needed.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. When executed by a Central Processing Unit (CPU) 801, the computer program performs the various functions defined in the system of the present application.
It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by means of software, or may be implemented by means of hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
Another aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform the in-vehicle function key recommendation method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment or may exist alone without being incorporated in the electronic device.
Another aspect of the present application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the in-vehicle function key recommendation method provided in the above-described respective embodiments.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. It is therefore intended that all equivalent modifications and changes made by those skilled in the art without departing from the spirit and technical spirit of the present invention shall be covered by the appended claims.

Claims (10)

1. The vehicle-mounted function key recommending method is characterized by comprising the following steps of:
monitoring the sight of a driver during the running process of the vehicle;
when the total time that the sight line of the driver leaves the preset sight line range exceeds a first threshold value; or, the frequency of the sight line of the driver leaving the preset sight line range in the preset time period is greater than a second threshold value; splicing all sensor data representing the running process of the vehicle to form a data vector;
screening driver operation records with the similarity to the data vector being greater than or equal to a third threshold value from a preset database, and searching vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver.
2. The vehicle-mounted function key recommending method according to claim 1, wherein before screening out driver operation records having a similarity with the data vector greater than or equal to a third threshold value from a preset database, further comprising:
acquiring reference sensor data corresponding to all vehicle function keys, wherein the reference sensor data comprise temperature, vehicle speed, road type and congestion condition;
Converting the reference sensor data into vectors, and generating database record vectors, wherein the database record vectors correspond to driver operation records;
and storing the database record vector in a preset database.
3. The vehicle-mounted function key recommending method according to claim 1, wherein finding a vehicle function key corresponding to the driver operation record comprises:
determining a region with the longest line-of-sight residence time period or a region with the line-of-sight residence frequency larger than a fourth threshold value as an alternative region;
and sequencing the function keys in the alternative area according to the use frequency of the driver from high to low, and determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record.
4. The vehicle-mounted function key recommending method according to claim 3, wherein determining a region in which a line-of-sight stay period is longest or a region in which a line-of-sight stay frequency is greater than a fourth threshold value as the candidate region includes:
acquiring sight line data of a driver, wherein the sight line data comprises a sight line stay area and single stay time;
dividing the line-of-sight stay region into a plurality of regions;
The sight line stay time of the driver in each area is counted to obtain the total sight line stay time of the driver in each area;
determining the area with the longest stay time of the sight line as an alternative area; or,
obtaining the sight line residence frequency by counting the single sight line residence time of the driver in each area;
and determining a region with the sight-line retention frequency being greater than or equal to a fourth threshold value as an alternative region.
5. The vehicle-mounted function key recommending method according to claim 2, wherein the step of screening out driver operation records having a similarity with the data vector greater than or equal to a third threshold value from a preset database comprises:
calculating the similarity between the data vector and the database record vector through a cosine similarity formula;
when the similarity is larger than or equal to a third threshold value, screening out driver operation records corresponding to the data vectors larger than or equal to the third threshold value;
and when the similarity is smaller than a third threshold value, comparing similarity values of other data vectors and the database record vectors until the vehicle function keys corresponding to the driver operation records are screened out.
6. The vehicle-mounted function key recommending method according to claim 1, wherein recommending the found vehicle function key to the driver includes:
inquiring whether the driver uses the vehicle function keys corresponding to the driver operation record or not, and monitoring voice data of the driver;
when the voice data is monitored to be the voice data which is replied by the target object based on the inquiry and is used for starting the vehicle function key corresponding to the driver operation record, starting the function corresponding to the vehicle function key corresponding to the driver operation record;
and when the voice data is not monitored as the voice data replied by the target object based on the inquiry and is used for starting the target function key, inquiring the function key with the highest use frequency of the history record, determining the function key with the highest use frequency as the vehicle function key corresponding to the driver operation record, and starting the function corresponding to the vehicle function key corresponding to the driver operation record.
7. The vehicle-mounted function key recommending method according to claim 1, wherein monitoring the line of sight of the driver during the running of the vehicle comprises:
Obtaining image information of the driver through a camera of the vehicle;
and identifying the image information, and monitoring the sight of the driver according to an image identification result.
8. An in-vehicle function key recommending apparatus, characterized in that the in-vehicle function key recommending apparatus includes:
the monitoring module is used for monitoring the sight of a driver in the running process of the vehicle;
the splicing module is used for enabling the total time of the sight line of the driver leaving the preset sight line range to exceed a first threshold value; or, the frequency of the sight line of the driver leaving the preset sight line range in the preset time period is greater than a second threshold value; splicing all sensor data representing the running process of the vehicle to form a data vector;
the function recommendation module is used for screening driver operation records with the similarity with the data vector being greater than or equal to a third threshold value from a preset database, and searching vehicle function keys corresponding to the driver operation records; and recommending the found vehicle function keys to the driver.
9. An electronic device, the electronic device comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the in-vehicle function key recommendation method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform the in-vehicle function key recommendation method according to any one of claims 1 to 7.
CN202311444928.4A 2023-11-01 2023-11-01 Vehicle-mounted function key recommending method and device, electronic equipment and storage medium Pending CN117485357A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311444928.4A CN117485357A (en) 2023-11-01 2023-11-01 Vehicle-mounted function key recommending method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311444928.4A CN117485357A (en) 2023-11-01 2023-11-01 Vehicle-mounted function key recommending method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117485357A true CN117485357A (en) 2024-02-02

Family

ID=89684146

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311444928.4A Pending CN117485357A (en) 2023-11-01 2023-11-01 Vehicle-mounted function key recommending method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117485357A (en)

Similar Documents

Publication Publication Date Title
EP3786018A1 (en) Electronic device for vehicle and operating method thereof
US10776735B2 (en) Risk information processing method and server device
CN113570013A (en) Service execution method and device
EP2924624A1 (en) Method of diagnosing operating characteristics
CN110942332A (en) Information processing apparatus and information processing method
US9529889B2 (en) Information terminal with application prioritization
US20050169502A1 (en) Method and device for mobile object information management, and computer product
CN107004353B (en) Traffic violation management system and traffic violation management method
CN110930765B (en) Early warning priority determination method, device, storage medium and device
WO2020008556A1 (en) Vehicle number identification device, vehicle number identification method, and program
US20200134729A1 (en) Information processing device, information processing system, information processing method, and program
JP2019067201A (en) Vehicle search system, vehicle search method, and vehicle and program employed in the same
CN112758088A (en) Dangerous source reminding method and advanced driving assistance system
CN111798661A (en) Overtaking early warning method and device during vehicle running
WO2016201867A1 (en) M2m car networking identification method and apparatus
Re et al. Testing the robustness of commercial lane departure warning systems
US9821812B2 (en) Traffic complexity estimation
CN117485357A (en) Vehicle-mounted function key recommending method and device, electronic equipment and storage medium
US20220317683A1 (en) Remote operator recommendation system and remote operator recommendation method
KR102385191B1 (en) Real-time dangerous vehicle tracking system and tracking method
JP6906574B2 (en) In-vehicle device and vehicle management system
CN114666765A (en) Method and device for seeking vehicle use help from inside to outside of vehicle
CN112700138A (en) Method, device and system for road traffic risk management
Pai et al. Realization of Internet of vehicles technology integrated into an augmented reality system
JP7318438B2 (en) Driving support device, control method for driving support device, and control program for driving support device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination