CN116161051A - Warning method, device, equipment, medium and vehicle for vehicle driver - Google Patents
Warning method, device, equipment, medium and vehicle for vehicle driver Download PDFInfo
- Publication number
- CN116161051A CN116161051A CN202211690342.1A CN202211690342A CN116161051A CN 116161051 A CN116161051 A CN 116161051A CN 202211690342 A CN202211690342 A CN 202211690342A CN 116161051 A CN116161051 A CN 116161051A
- Authority
- CN
- China
- Prior art keywords
- warning
- vehicle
- content
- information
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000005236 sound signal Effects 0.000 claims abstract description 112
- 238000012549 training Methods 0.000 claims description 66
- 238000000605 extraction Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 16
- 230000003993 interaction Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000004075 alteration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006855 networking Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000007789 sealing Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/143—Alarm means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/146—Display means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Theoretical Computer Science (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Transportation (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Mechanical Engineering (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a warning method, a warning device, warning equipment, warning media and a vehicle for a vehicle driver, which are used for solving the technical problem that the driving experience of the driver is poor due to potential safety hazards in the automobile driving process in the prior art, and the method comprises the following steps: acquiring an audio signal outside the vehicle in real time; determining alert content contained in the audio signal based on the audio signal; outputting warning information to a vehicle driver according to the warning content; and outputting the warning information to the terminal equipment in the source direction so that the terminal equipment in the source direction warns a driver in the source direction.
Description
Technical Field
The invention relates to the technical fields of automatic driving, man-machine interaction and the like, in particular to a warning method, a warning device, warning equipment, warning media and a vehicle for a vehicle driver.
Background
The automatic driving of the automobile is a technology adopting environment information perception and identification, artificial intelligence and intelligent decision control of the automobile network system. The automatic driving technology integrates automatic control, a complex system, artificial intelligence, machine vision and the like, collects information such as vehicle networking data, geographic data, environment sensing data and the like of cloud and vehicle-mounted sensors, identifies the environment characteristics of a vehicle driving area, and performs task setting and control planning.
In the related art, due to the better cabin sealing effect of some vehicles or the worse hearing of some drivers, the drivers may not recognize whether vehicles or people in the surrounding send warning information to themselves, such as whistling, calling for help, etc. Meanwhile, in the current automatic driving, the vehicle machine cannot sense the warning information of other vehicles, pedestrians and the like. The potential safety hazard exists in the driving process of the automobile, and the driving experience of a driver is poor.
Accordingly, a method is needed to alert the driver of the vehicle.
Disclosure of Invention
The invention provides a warning method, a warning device, warning equipment, warning media and a vehicle for a vehicle driver, which are used for solving the technical problem that in the prior art, potential safety hazards exist in the driving process of an automobile, so that the driving experience of the driver is poor.
In a first aspect, an embodiment of the present invention provides a warning method for a driver of a vehicle, including:
acquiring an audio signal outside the vehicle in real time;
determining alert content contained in the audio signal based on the audio signal;
outputting warning information to a vehicle driver according to the warning content;
and outputting warning information to the terminal equipment in the source direction so that the terminal equipment in the source direction warns a driver in the source direction.
In a possible implementation manner, in the method provided by the embodiment of the present invention, an audio signal outside a vehicle is obtained in real time, including;
a plurality of audio signals outside the vehicle are acquired by a plurality of audio sensors provided at different positions of the vehicle.
In a possible implementation manner, in the method provided by the embodiment of the present invention, the determining, based on the audio signal, the alert content included in the audio signal includes:
analyzing the plurality of audio signals, and judging whether the plurality of audio signals contain warning contents of preset types or not;
if the plurality of audio signals contain warning contents, determining the source direction of the warning contents according to the waveforms of the warning contents in the plurality of audio signals.
In a possible implementation manner, in the method provided by the embodiment of the present invention, a plurality of audio signals are parsed, and whether the plurality of audio signals include a preset type of warning content is determined, including:
analyzing the plurality of audio signals by utilizing a pre-trained warning content extraction model, and judging whether the plurality of audio signals contain warning contents of preset types or not;
the warning content extraction model is generated by training in the following way:
acquiring a plurality of training sample data containing audio signals and labeling results;
and comparing the warning type corresponding to each training sample data with the labeling result of the training sample data by taking each audio signal in the plurality of training sample data as input, generating a warning content extraction model based on the difference training of the warning type corresponding to each training sample data and the labeling result of the training sample data, wherein the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the labeling result is the type of the warning content labeled in the training sample data in advance.
In a possible implementation manner, before the method provided by the embodiment of the present invention obtains the audio signal outside the vehicle in real time, the method further includes:
a video signal outside the vehicle is acquired by a video sensor provided to the vehicle.
In a possible implementation manner, in the method provided by the embodiment of the present invention, if the plurality of audio signals include warning content, after determining the source direction of the warning content according to the waveform of the warning content in the plurality of audio signals, the method further includes:
and correcting the source direction of the warning content according to the video signal.
In a possible implementation manner, in the method provided by the embodiment of the present invention, outputting warning information to a driver of a vehicle according to warning content includes:
determining an alert level of the alert content based on the alert content;
generating warning information according to the warning level of the warning content, wherein the warning information comprises sound prompt information and flash prompt information;
outputting audible prompts through vehicle built-in speakers, and/or
Outputting flash prompt information through vehicle-mounted man-machine interaction equipment and/or
And outputting sound prompt information and flash prompt information through terminal equipment connected with the vehicle.
In a possible implementation manner, in the method provided by the embodiment of the present invention, warning information is output to a terminal device in a source direction, so that the terminal device in the source direction warns a driver in the source direction, including:
generating travel information of the vehicle based on the navigation path of the vehicle;
and outputting the warning information and the driving information to the terminal equipment in the source direction.
In a second aspect, an embodiment of the present invention provides a warning device for a driver of a vehicle, including:
an acquisition unit configured to acquire an audio signal outside the vehicle in real time;
a determination unit configured to determine alert content contained in the audio signal based on the audio signal;
the output unit is used for outputting warning information to a vehicle driver according to the warning content;
and the sending unit is used for outputting warning information to the terminal equipment in the source direction so as to enable the terminal equipment in the source direction to warn a driver in the source direction.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is specifically configured to;
a plurality of audio signals outside the vehicle are acquired by a plurality of audio sensors provided at different positions of the vehicle.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the determining unit is specifically configured to:
analyzing the plurality of audio signals, and judging whether the plurality of audio signals contain warning contents of preset types or not;
if the plurality of audio signals contain warning contents, determining the source direction of the warning contents according to the waveforms of the warning contents in the plurality of audio signals.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the determining unit is specifically configured to:
analyzing the plurality of audio signals by utilizing a pre-trained warning content extraction model, and judging whether the plurality of audio signals contain warning contents of preset types or not;
the warning content extraction model is generated by training in the following way:
acquiring a plurality of training sample data containing audio signals and labeling results;
and comparing the warning type corresponding to each training sample data with the labeling result of the training sample data by taking each audio signal in the plurality of training sample data as input, generating a warning content extraction model based on the difference training of the warning type corresponding to each training sample data and the labeling result of the training sample data, wherein the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the labeling result is the type of the warning content labeled in the training sample data in advance.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is further configured to:
a video signal outside the vehicle is acquired by a video sensor provided to the vehicle.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the determining unit is further configured to:
and correcting the source direction of the warning content according to the video signal.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the output unit is specifically configured to:
determining an alert level of the alert content based on the alert content;
generating warning information according to the warning level of the warning content, wherein the warning information comprises sound prompt information and flash prompt information;
outputting audible prompts through vehicle built-in speakers, and/or
Outputting flash prompt information through vehicle-mounted man-machine interaction equipment and/or
And outputting sound prompt information and flash prompt information through terminal equipment connected with the vehicle.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the sending unit is further configured to:
generating travel information of the vehicle based on the navigation path of the vehicle;
and outputting the warning information and the driving information to the terminal equipment in the source direction.
In a third aspect, an embodiment of the present invention provides an electronic device, including: at least one processor, at least one memory and computer program instructions stored in the memory, which when executed by the processor implement the method as provided by the first aspect of the embodiments of the invention.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as provided by the first aspect of embodiments of the present invention.
In a fifth aspect, embodiments of the present invention provide a vehicle comprising a warning device for a driver of the vehicle as mentioned in the second aspect and an electronic apparatus as mentioned in the third aspect.
In the embodiment of the invention, firstly, an audio signal outside the vehicle is obtained in real time, then, the warning content contained in the audio signal is determined based on the audio signal, then, warning information is output to a vehicle driver according to the warning content, and finally, the warning information is output to terminal equipment in the source direction. Compared with the prior art, the method solves the problem that potential safety hazards exist in the automobile driving process, and further the driving experience of a driver is poor, can utilize signals outside the automobile, reminds the driver according to the signal content, reminds the driver in the source direction, is beneficial to safe driving of the automobile, and improves the driving experience of the driver.
Drawings
Fig. 1 is a flow chart of a warning method for a vehicle driver according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a warning method for a vehicle driver according to an embodiment of the present invention;
FIG. 3 is a schematic view of a sensor mounting location according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a warning device for a driver of a vehicle according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Some words appearing hereinafter are explained:
1. in the embodiment of the invention, the term "and/or" describes the association relation of the association objects, which means that three relations can exist, for example, a and/or B can be expressed as follows: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
The automatic driving of the automobile is a technology adopting environment information perception and identification, artificial intelligence and intelligent decision control of the automobile network system. The automatic driving technology integrates automatic control, a complex system, artificial intelligence, machine vision and the like, collects information such as vehicle networking data, geographic data, environment sensing data and the like of cloud and vehicle-mounted sensors, identifies the environment characteristics of a vehicle driving area, and performs task setting and control planning.
In the related art, due to the better cabin sealing effect of some vehicles or the worse hearing of some drivers, the drivers may not recognize whether vehicles or people in the surrounding send warning information to themselves, such as whistling, calling for help, etc. Meanwhile, in the current automatic driving, the vehicle machine cannot sense the warning information of other vehicles, pedestrians and the like. The potential safety hazard exists in the driving process of the automobile, and the driving experience of a driver is poor.
The present invention provides a warning method for a vehicle driver, and the warning method, device, equipment, medium and vehicle for a vehicle driver provided by the present invention are described in more detail below with reference to the accompanying drawings and embodiments.
An embodiment of the present invention provides a warning method for a driver of a vehicle, as shown in fig. 1, including:
step S101, acquiring an audio signal outside the vehicle in real time.
In specific implementation, the audio signal outside the vehicle is acquired through the audio sensor arranged outside the vehicle, and of course, a plurality of audio sensors may be arranged, or one or more video sensors may be arranged to acquire the video signal outside the vehicle to assist in determining the source position of the sound information in the audio information, or to assist in using a distance sensor, such as a Laser Radar (Lidar) or the like.
Step S102, warning content contained in the audio signal is determined based on the audio signal.
When the method is implemented, the warning content contained in the audio signals is determined based on one or more audio signals, the one or more audio signals are analyzed during the specific determination, whether the plurality of audio signals contain preset types of warning content or not is judged, and the preset types of warning content such as automobile whistle, traffic light prompt tone, traffic police whistle, calling sound of pedestrians, vehicle collision sound and the like can be set according to specific requirements. If the plurality of audio signals contain warning contents, determining the source direction of the warning contents according to the waveforms of the warning contents in the plurality of audio signals. Meanwhile, the source direction of the warning content can be assisted and corrected according to the video signal acquired by the video sensor and the information acquired by the distance sensor.
In the judging process, the warning content can be determined through the type and the shape of the waveform, or the pre-trained warning content extraction model can be utilized to analyze the plurality of audio signals, so as to judge whether the plurality of audio signals contain the preset type of warning content.
The warning content extraction model is generated by training in the following way:
firstly, a plurality of training sample data containing audio signals and marking results are obtained, then each audio signal in the plurality of training sample data is used as input, the warning type corresponding to each training sample data is compared with the marking results of the training sample data, a warning content extraction model is generated based on the difference training of the warning type corresponding to each training sample data and the marking results of the training sample data, the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the marking results are the types of the warning content marked in the training sample data in advance.
Step S103, outputting warning information to a vehicle driver according to the warning content.
When the method is implemented, the warning grade of the warning content is determined based on the warning content, the warning grade can be set according to requirements, for example, the warning grade is divided according to the volume of audio frequency, the distance of the source of the warning content and the like, and then warning information is generated according to the warning grade of the warning content, and the warning information can comprise sound prompt information, flash prompt information and the like. And then carrying out one or more of the following warning methods according to the warning grade and the warning information: the vehicle-mounted human-computer interaction device outputs sound prompt information through a vehicle-mounted loudspeaker, outputs flash prompt information through a vehicle-mounted human-computer interaction device, and outputs sound prompt information and flash prompt information through a terminal device connected with the vehicle. Of course, the auxiliary prompt may also be performed by vibration of the driver seat, which is not limited in the embodiment of the present invention.
Step S104, outputting warning information to the terminal equipment in the source direction so that the terminal equipment in the source direction warns the driver in the source direction.
The driver is prompted, and meanwhile, warning information can be output to the source direction of the warning content. If the source direction is another vehicle, outputting warning information to terminal equipment, a vehicle-mounted system and the like in the source direction so as to enable a driver in the source direction to receive warning; if the source direction is a pedestrian, the prompting information can be output to the terminal device of the pedestrian, and the specific prompting mode is not limited.
When the source direction is the vehicle, the driving information generated based on the driving vehicle navigation path can be output to the terminal equipment of the vehicle driver or the vehicle-mounted system of the vehicle, so that the vehicle in the source direction can clearly know the driving intention of the vehicle, and further the path, the vehicle speed and the like can be planned more effectively, and the driving safety is improved.
The following describes the warning method for the vehicle driver in the embodiment of the present invention in detail with reference to fig. 2, and as shown in fig. 2, the warning method for the vehicle driver provided in the embodiment of the present invention may include the following specific steps:
in step S201, a video signal outside the vehicle is acquired by a video sensor provided in the vehicle.
In practice, a video signal outside the vehicle is acquired by a video sensor provided in the vehicle, and a distance sensor such as a Laser Radar (Lidar) may be used as an auxiliary sensor. An object outside the vehicle is detected.
Step S202, an audio signal outside the vehicle is acquired by an audio sensor provided outside the vehicle.
In particular, an audio signal outside the vehicle is acquired by an audio sensor disposed outside the vehicle. The step S201 and the step S202 may be performed simultaneously, or may be performed first.
In an example, step S201 is performed first, then when a moving object is detected around the vehicle in the video signal acquired in S201, the detection sensitivity of the audio sensor is improved, or when an audio signal with a certain volume is detected, the audio signal is acquired through one or more audio sensors, specifically, the setting of a plurality of audio sensors may be performed in a manner as shown in fig. 3, where 301, 302, 303, and 304 may be the positions of the audio sensors, 311 may be the positions of the video sensors in S201, and may be other positions, which may of course be set in other positions.
Step S203, analyzing the plurality of audio signals, and judging whether the plurality of audio signals contain warning contents of preset types.
In specific implementation, one or more audio signals are analyzed, whether the audio signals contain preset type warning contents or not is judged, and the preset type warning contents such as automobile whistle, traffic light prompt tone, traffic police whistle, pedestrian calling sound, vehicle collision sound and the like can be set according to specific requirements. In the judging process, the warning content can be determined through the type and the shape of the waveform, or the pre-trained warning content extraction model can be utilized to analyze the plurality of audio signals, so as to judge whether the plurality of audio signals contain the preset type of warning content. The warning content extraction model is generated by training in the following way:
firstly, a plurality of training sample data containing audio signals and marking results are obtained, then each audio signal in the plurality of training sample data is used as input, the warning type corresponding to each training sample data is compared with the marking results of the training sample data, a warning content extraction model is generated based on the difference training of the warning type corresponding to each training sample data and the marking results of the training sample data, the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the marking results are the types of the warning content marked in the training sample data in advance.
Still using the above example, for example, in a plurality of audio signals, if a whistle is detected by the warning content extraction model, the whistle is determined to be the warning content.
In step S204, if the plurality of audio signals include warning content, a source direction of the warning content is determined according to waveforms of the warning content in the plurality of audio signals.
In the implementation, if the plurality of audio signals include warning content, the source direction of the warning content is determined according to the waveform of the warning content in the plurality of audio signals. Meanwhile, the source direction of the warning content can be assisted and corrected according to the video signal acquired by the video sensor and the information acquired by the distance sensor.
Still using the above example, for four audio signals acquired by four different audio sensors 301, 302, 303 and 304, the location of the source of the whistling sound is determined according to the phase and amplitude of the sound wave of the whistling sound in the audio signals, respectively.
In step S205, the source direction of the warning content is modified according to the video signal.
In practice, the source direction of the warning content is corrected according to the video signal
Still using the above example, the location of the source of the whistle is corrected by the video signal captured at 311, although the location of the source of the whistle may also be corrected simultaneously by information acquired by a distance sensor.
Step S206, determining the warning level of the warning content based on the warning content.
When the method is implemented, the warning level can be set according to the requirements, such as dividing the warning level according to the volume of the audio, the distance of the source of the warning content and the like, and then generating warning information according to the warning level of the warning content, wherein the warning information can comprise voice prompt information, flash prompt information and the like.
Still using the above example, the volume and source position of the whistle determine the warning level of the warning content, if it is determined that the source distance of the whistle is far, the warning level is low.
Step S207, generating warning information according to the warning level of the warning content, and outputting the warning information to the driver.
In specific implementation, one or more of the following warning methods are performed according to the warning level and the warning information: the vehicle-mounted human-computer interaction device outputs sound prompt information through a vehicle-mounted loudspeaker, outputs flash prompt information through a vehicle-mounted human-computer interaction device, and outputs sound prompt information and flash prompt information through a terminal device connected with the vehicle. Of course, the auxiliary prompt can also be performed by means of vibration of the driver seat or the like.
Still further, with the above example, since the warning level of the whistle is low, the vehicle-mounted speaker is designated to play the whistle to prompt the driver, or the vehicle-mounted speaker may be designated to play a prompt such as "vehicle whistle in front" to prompt the driver.
Step S208, outputting warning information to the terminal equipment in the source direction.
In the implementation, the warning information can be output to the source direction of the warning content. If the source direction is another vehicle, the warning information is output to the terminal equipment, the vehicle-mounted system and the like in the source direction, so that the driver in the source direction receives the warning.
When the source direction is the vehicle, the driving information generated based on the driving vehicle navigation path can be output to the terminal equipment of the vehicle driver or the vehicle-mounted system of the vehicle, so that the vehicle in the source direction can clearly know the driving intention of the vehicle, and further the path, the vehicle speed and the like can be planned more effectively, and the driving safety is improved.
As shown in fig. 4, based on the same inventive concept as the warning method for the vehicle driver, the present invention further provides a warning device for the vehicle driver, including:
an acquisition unit 401 for acquiring an audio signal outside the vehicle in real time;
a determining unit 402, configured to determine alert content included in the audio signal based on the audio signal;
an output unit 403 for outputting warning information to a vehicle driver according to the warning content;
and the sending unit 404 is configured to output the warning information to the terminal device in the source direction, so that the terminal device in the source direction warns the driver in the source direction.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit 401 is specifically configured to;
a plurality of audio signals outside the vehicle are acquired by a plurality of audio sensors provided at different positions of the vehicle.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the determining unit 402 is specifically configured to:
analyzing the plurality of audio signals, and judging whether the plurality of audio signals contain warning contents of preset types or not;
if the plurality of audio signals contain warning contents, determining the source direction of the warning contents according to the waveforms of the warning contents in the plurality of audio signals.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the determining unit 402 is specifically configured to:
analyzing the plurality of audio signals by utilizing a pre-trained warning content extraction model, and judging whether the plurality of audio signals contain warning contents of preset types or not;
the warning content extraction model is generated by training in the following way:
acquiring a plurality of training sample data containing audio signals and labeling results;
and comparing the warning type corresponding to each training sample data with the labeling result of the training sample data by taking each audio signal in the plurality of training sample data as input, generating a warning content extraction model based on the difference training of the warning type corresponding to each training sample data and the labeling result of the training sample data, wherein the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the labeling result is the type of the warning content labeled in the training sample data in advance.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 401 is further configured to:
a video signal outside the vehicle is acquired by a video sensor provided to the vehicle.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the determining unit 402 is further configured to:
and correcting the source direction of the warning content according to the video signal.
In one possible implementation manner, in the device provided by the embodiment of the present invention, the output unit 403 is specifically configured to:
determining an alert level of the alert content based on the alert content;
generating warning information according to the warning level of the warning content, wherein the warning information comprises sound prompt information and flash prompt information;
outputting audible prompts through vehicle built-in speakers, and/or
Outputting flash prompt information through vehicle-mounted man-machine interaction equipment and/or
And outputting sound prompt information and flash prompt information through terminal equipment connected with the vehicle.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the sending unit 403 is further configured to:
generating travel information of the vehicle based on the navigation path of the vehicle;
and outputting the warning information and the driving information to the terminal equipment in the source direction.
In addition, the warning method and apparatus for a vehicle driver of the embodiments of the present application described in connection with fig. 1-4 may be implemented by an electronic device. Fig. 5 shows a schematic hardware structure of an electronic device according to an embodiment of the present application.
As shown in fig. 5, the electronic device 500 may include a processing means (e.g., a central processor, a graphics processor, etc.) 501 that may perform various suitable actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503 to implement the warning method for a vehicle driver according to the embodiments described in the present disclosure. In the RAM 503, various programs and data required for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
In general, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 507 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 508 including, for example, magnetic tape, hard disk, etc.; and communication means 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 shows an electronic device 500 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts, thereby implementing the speech control method as described above. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or from the storage means 508, or from the ROM 502. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 501.
The embodiment of the disclosure also provides a vehicle, which comprises the warning device and the electronic equipment for the vehicle driver.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
acquiring an audio signal outside the vehicle in real time;
determining alert content contained in the audio signal based on the audio signal;
and outputting warning information to a vehicle driver according to the warning content.
Alternatively, the electronic device may perform other steps described in the above embodiments when the above one or more programs are executed by the electronic device.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the embodiment of the invention, firstly, the audio signal outside the vehicle is obtained in real time, then the warning content contained in the audio signal is determined based on the audio signal, and finally, the warning information is output to the driver of the vehicle according to the warning content. Compared with the prior art, the method solves the problem that potential safety hazards exist in the automobile driving process, so that the driving experience of a driver is poor, can utilize signals outside the automobile, reminds the driver according to the signal content, is beneficial to safe driving of the automobile, and improves the driving experience of the driver.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (12)
1. A warning method for a driver of a vehicle, comprising:
acquiring an audio signal outside the vehicle in real time;
determining alert content contained in the audio signal based on the audio signal;
outputting warning information to a vehicle driver according to the warning content;
and outputting the warning information to the terminal equipment in the source direction so that the terminal equipment in the source direction warns a driver in the source direction.
2. The method of claim 1, wherein the acquiring in real time an audio signal external to the vehicle comprises;
a plurality of audio signals outside the vehicle are acquired by a plurality of audio sensors provided at different positions of the vehicle.
3. The method of claim 2, wherein the determining alert content contained in the audio signal based on the audio signal comprises:
analyzing the plurality of audio signals, and judging whether the plurality of audio signals contain warning contents of preset types or not;
if the plurality of audio signals contain the warning content, determining the source direction of the warning content according to the waveform of the warning content in the plurality of audio signals.
4. The method of claim 3, wherein the parsing the plurality of audio signals to determine whether the plurality of audio signals include a preset type of alert content comprises:
analyzing the plurality of audio signals by utilizing a pre-trained warning content extraction model, and judging whether the plurality of audio signals contain warning contents of preset types or not;
the warning content extraction model is generated through training in the following mode:
acquiring a plurality of training sample data containing audio signals and labeling results;
and comparing the warning type corresponding to each training sample data with the labeling result of the training sample data by taking each audio signal in a plurality of training sample data as input, generating the warning content extraction model based on the difference training of the warning type corresponding to each training sample data and the labeling result of the training sample data, wherein the warning type corresponding to each training sample data is the type representing the warning content in each training sample data, and the labeling result is the type of the warning content labeled in the training sample data in advance.
5. The method of claim 4, wherein prior to the acquiring the audio signal external to the vehicle in real time, the method further comprises:
a video signal outside the vehicle is acquired by a video sensor provided to the vehicle.
6. The method of claim 5, wherein if the plurality of audio signals includes the alert content, after determining the direction of origin of the alert content from the waveform of the alert content in the plurality of audio signals, the method further comprises:
and correcting the source direction of the warning content according to the video signal.
7. The method of claim 6, wherein outputting the alert information to the driver of the vehicle based on the alert content comprises:
determining an alert level of the alert content based on the alert content;
generating warning information according to the warning content and the warning grade, wherein the warning information comprises sound prompt information and flash prompt information;
outputting the sound prompt information through a built-in speaker of the vehicle, and/or
Outputting the flash prompt information through the vehicle-mounted man-machine interaction equipment, and/or
And outputting the sound prompt information and the flash prompt information through terminal equipment connected with the vehicle.
8. The method according to claim 7, wherein the outputting the warning information to the source direction terminal device to cause the source direction terminal device to warn the source direction driver includes:
generating travel information of the vehicle based on a navigation path of the vehicle;
and outputting the warning information and the driving information to the terminal equipment in the source direction.
9. A warning device for a driver of a vehicle, comprising:
an acquisition unit configured to acquire an audio signal outside the vehicle in real time;
a determining unit configured to determine alert content contained in the audio signal based on the audio signal;
and the output unit is used for outputting warning information to a vehicle driver according to the warning content.
10. An electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which when executed by the processor, implement the method of any one of claims 1-8.
11. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-8.
12. A vehicle comprising a warning device for a driver of the vehicle according to claim 9, and an electronic apparatus according to claim 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211690342.1A CN116161051A (en) | 2022-12-27 | 2022-12-27 | Warning method, device, equipment, medium and vehicle for vehicle driver |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211690342.1A CN116161051A (en) | 2022-12-27 | 2022-12-27 | Warning method, device, equipment, medium and vehicle for vehicle driver |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116161051A true CN116161051A (en) | 2023-05-26 |
Family
ID=86417440
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211690342.1A Pending CN116161051A (en) | 2022-12-27 | 2022-12-27 | Warning method, device, equipment, medium and vehicle for vehicle driver |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116161051A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109606260A (en) * | 2018-12-26 | 2019-04-12 | 北京蓦然认知科技有限公司 | A kind of method and device of the outer auditory tone cues of vehicle |
| CN109658740A (en) * | 2019-02-19 | 2019-04-19 | 百度在线网络技术（北京）有限公司 | Exchange method and device between vehicle |
| CN109891262A (en) * | 2016-10-27 | 2019-06-14 | 株式会社电装 | Object detection device |
| CN111994000A (en) * | 2020-09-01 | 2020-11-27 | 成都市骐智科技有限责任公司 | Auxiliary driving method for traffic warning tone |
| US20210082273A1 (en) * | 2019-09-13 | 2021-03-18 | Veebar Tech | System and method for alerting a user to the presence of environmental sounds |
| CN112951271A (en) * | 2020-11-27 | 2021-06-11 | 长沙理工大学 | Tunnel traffic safety warning method and system based on acoustic assistance |
| CN113232659A (en) * | 2020-02-06 | 2021-08-10 | 丰田自动车株式会社 | Blind spot information acquisition device and method, vehicle, and recording medium having program recorded thereon |
| US20220223170A1 (en) * | 2021-01-13 | 2022-07-14 | Baidu Usa Llc | Sound source detection and localization for autonomous driving vehicle |
-
2022
- 2022-12-27 CN CN202211690342.1A patent/CN116161051A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109891262A (en) * | 2016-10-27 | 2019-06-14 | 株式会社电装 | Object detection device |
| CN109606260A (en) * | 2018-12-26 | 2019-04-12 | 北京蓦然认知科技有限公司 | A kind of method and device of the outer auditory tone cues of vehicle |
| CN109658740A (en) * | 2019-02-19 | 2019-04-19 | 百度在线网络技术（北京）有限公司 | Exchange method and device between vehicle |
| US20210082273A1 (en) * | 2019-09-13 | 2021-03-18 | Veebar Tech | System and method for alerting a user to the presence of environmental sounds |
| CN113232659A (en) * | 2020-02-06 | 2021-08-10 | 丰田自动车株式会社 | Blind spot information acquisition device and method, vehicle, and recording medium having program recorded thereon |
| CN111994000A (en) * | 2020-09-01 | 2020-11-27 | 成都市骐智科技有限责任公司 | Auxiliary driving method for traffic warning tone |
| CN112951271A (en) * | 2020-11-27 | 2021-06-11 | 长沙理工大学 | Tunnel traffic safety warning method and system based on acoustic assistance |
| US20220223170A1 (en) * | 2021-01-13 | 2022-07-14 | Baidu Usa Llc | Sound source detection and localization for autonomous driving vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112590813B (en) | Method, device, electronic device and medium for generating information of automatic driving vehicle | |
| US11610411B2 (en) | Driver assistance system and method for displaying traffic information | |
| CN111402925B (en) | Voice adjustment method, device, electronic equipment, vehicle-mounted system and readable medium | |
| US10741076B2 (en) | Cognitively filtered and recipient-actualized vehicle horn activation | |
| CN109795407B (en) | Vehicle-mounted intelligent reminding method and device and vehicle | |
| US10970899B2 (en) | Augmented reality display for a vehicle | |
| CN113386785B (en) | Method and device for displaying augmented reality warning information | |
| CN113205088B (en) | Obstacle image presentation method, electronic device, and computer-readable medium | |
| CN113581195B (en) | Special vehicle identification method, electronic device and computer readable medium | |
| CN111009146A (en) | Server, information processing method, and non-transitory storage medium storing program | |
| CN115731526B (en) | Lane line identification method, lane line identification device, electronic equipment and computer readable medium | |
| CN116161051A (en) | Warning method, device, equipment, medium and vehicle for vehicle driver | |
| CN116312052A (en) | Vehicle lane change prompting method, system, equipment and vehicle | |
| CN112677985B (en) | Method and device for determining activation level of central control function of vehicle, electronic equipment and medium | |
| US20210216775A1 (en) | Roadway alert system using video stream from a smart mirror | |
| CN114103944A (en) | Workshop time distance adjusting method, device and equipment | |
| CN113879313A (en) | Driver fatigue detection method and device | |
| CN115205501B (en) | Road surface condition display method, device, equipment and medium | |
| CN118220130A (en) | Reminding processing method, device, equipment and medium | |
| CN115577145B (en) | Transportation information storage method, apparatus, electronic device, medium, and program product | |
| CN113888892B (en) | Road information prompting method and device, electronic equipment and computer readable medium | |
| WO2020221023A1 (en) | Message sending control method and apparatus in sensing base station | |
| CN117709832A (en) | Transportation information generation method, device, equipment and computer readable medium | |
| CN118053447A (en) | Sound signal processing method and device, electronic equipment and storage medium | |
| CN116101321A (en) | Method and device for judging and early warning of non-drivable lane |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |