CN117351754A - Road condition prompting method, electronic equipment and computer readable medium - Google Patents
Road condition prompting method, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN117351754A CN117351754A CN202210749766.4A CN202210749766A CN117351754A CN 117351754 A CN117351754 A CN 117351754A CN 202210749766 A CN202210749766 A CN 202210749766A CN 117351754 A CN117351754 A CN 117351754A
- Authority
- CN
- China
- Prior art keywords
- road condition
- target vehicle
- information
- sound field
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000003190 augmentative effect Effects 0.000 claims abstract description 28
- 230000000877 morphologic effect Effects 0.000 claims description 33
- 230000000694 effects Effects 0.000 claims description 20
- 230000005669 field effect Effects 0.000 claims description 19
- 210000000988 bone and bone Anatomy 0.000 claims description 9
- 238000010521 absorption reaction Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 210000005069 ears Anatomy 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000003993 interaction Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 210000000624 ear auricle Anatomy 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 210000003128 head Anatomy 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000004800 psychological effect Effects 0.000 description 3
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 2
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 210000000860 cochlear nerve Anatomy 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000003094 ear ossicle Anatomy 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/09623—Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Mechanical Engineering (AREA)
- General Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
- Traffic Control Systems (AREA)
- Stereophonic System (AREA)
Abstract
The disclosure provides a road condition prompting method, comprising the following steps: generating a sound field signal according to road condition information, wherein the sound field signal is used for simulating the road condition represented by the road condition information; and generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality mode. The present disclosure also provides an electronic device, a computer-readable medium. The road condition prompt method and device based on the hearing of the driver eliminates potential safety hazards and is beneficial to improving driving experience.
Description
Technical Field
The disclosure relates to the technical field of augmented reality, and features relate to a road condition prompting method based on audio augmented reality, electronic equipment and a computer readable medium.
Background
Along with the trend of intelligent and networking of automobiles, the digitalization of the automobile industry is deepened continuously, the relationship among people, automobiles and environments is also continuously developed and changed, and the man-machine interaction design becomes a core element of intelligent automobile development and innovation. In the future, intelligent automobiles are used as platforms to integrate a large number of functions, and man-machine interaction plays a key role in automobile safety and operation efficiency. The working efficiency, safety and convenience of the man-machine interaction system can directly influence the running state and safety performance of the automobile. Common man-machine interaction system of automobile includes panel board, touch-sensitive screen and Head Up Display (HUD) etc. and the driver mainly relies on the vision when using above-mentioned man-machine interaction system, has to often remove the sight in the place ahead, causes the hidden danger for driving safety, and frequent Head, eye action also can lead to the driver tired moreover, influences driving experience.
The automobile man-machine interaction system can be used for prompting road conditions, however, common road conditions are realized through signal lamps, display screens, radar images and the like, potential safety hazards can be brought due to the fact that a driver moves a sight, and road condition contents capable of prompting are limited.
Disclosure of Invention
The embodiment of the disclosure provides a road condition prompting method based on audio augmented reality, electronic equipment and a computer readable medium.
In a first aspect, an embodiment of the present disclosure provides a road condition prompting method, including:
generating a sound field signal according to road condition information, wherein the sound field signal is used for simulating the road condition represented by the road condition information;
and generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality mode.
In some embodiments, generating the sound field signal from the traffic information includes:
determining a virtual sound image according to the morphological information representing the morphological characteristics of the target vehicle in the road condition information, wherein the virtual sound image is used for simulating the morphological characteristics of the target vehicle through sound;
and adding a sound field effect to the virtual sound image according to the motion information representing the motion state of the target vehicle in the road condition information so as to simulate the motion state of the target vehicle and obtain the sound field signal.
In some embodiments, determining the virtual sound image based on the morphological information representing the morphological feature of the target vehicle in the road condition information includes:
determining the tonnage level of the target vehicle according to the morphological information;
determining at least one voiceprint feature according to the tonnage level of the target vehicle, wherein the voiceprint features corresponding to different tonnage levels are different;
and generating the virtual sound image according to the at least one voiceprint feature.
In some embodiments, the morphology information includes a type and a size of the target vehicle; determining a tonnage level of the target vehicle according to the morphological information, including:
and determining the tonnage level of the target vehicle according to the type and the size of the target vehicle.
In some embodiments, the motion information includes position information and speed information of the target vehicle relative to a current vehicle; according to the motion information representing the motion state of the target vehicle in the road condition information, adding a sound field effect to the virtual sound image, including:
and adding a sound field effect to the virtual sound image according to the position information and the speed information of the target vehicle relative to the current vehicle.
In some embodiments, the sound field effect comprises at least one of a binaural sound difference, a binaural delay, an auricle effect, a reflection effect, an absorption effect, a doppler effect.
In some embodiments, before generating the sound field signal according to the road condition information, the road condition prompting method further includes:
and acquiring the road condition information.
In some embodiments, obtaining the traffic information includes:
monitoring road conditions and judging whether an event to be prompted exists or not;
and when an event to be prompted exists, acquiring the road condition information.
In some embodiments, when there is an event to be prompted, acquiring the road condition information includes:
acquiring position information and speed information of a target vehicle relative to a current vehicle;
acquiring video data of the target vehicle;
and determining the type and the size of the target vehicle according to the position information of the target vehicle relative to the current vehicle and the video data.
In some embodiments, generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality manner includes:
and playing stereo audio according to the sound field signal through the audio playing equipment to prompt road conditions.
In some embodiments, the audio playback device is a bone conduction headset.
In a second aspect, embodiments of the present disclosure provide an electronic device, including:
one or more processors;
and the memory is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the road condition prompting method according to the first aspect of the embodiment of the disclosure.
In a third aspect, an embodiment of the present disclosure provides a computer readable medium having a computer program stored thereon, where the program when executed by a processor implements the road condition prompting method according to the first aspect of the embodiment of the present disclosure.
According to the road condition prompting method based on the audio augmented reality, the sound field signal can be generated according to the road condition information, the road condition represented by the road condition information is simulated, and then the virtual sound field is created according to the sound field signal, so that a driver can perceive the road condition by sound in the virtual sound field, and the road condition prompting of the driver in an audio augmented reality mode is realized. According to the embodiment of the disclosure, the road condition prompt is performed based on the hearing of the driver, the driver does not need to move the sight, potential safety hazards are eliminated, and the driving experience is improved based on the rich road condition content of the audio augmented reality prompt.
Drawings
FIG. 1 is a flow chart of a road condition prompting method in an embodiment of the disclosure;
FIG. 2 is a flowchart illustrating a portion of steps in another method for prompting road conditions according to an embodiment of the present disclosure;
FIG. 3 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 5 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 6 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 7 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 8 is a flowchart illustrating a portion of steps in a road condition prompting method according to another embodiment of the present disclosure;
FIG. 9 is a block diagram of an electronic device in an embodiment of the present disclosure;
FIG. 10 is a block diagram of one component of a computer-readable medium in an embodiment of the present disclosure;
fig. 11 is a schematic diagram of a system architecture of a road condition prompting in an embodiment of the disclosure.
Detailed Description
For better understanding of the technical solutions of the present disclosure, the following describes in detail a road condition prompting method, an electronic device, and a computer readable medium based on audio augmented reality (AR, augmented Reality) provided by the present disclosure with reference to the accompanying drawings.
Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Embodiments of the disclosure and features of embodiments may be combined with each other without conflict.
As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
In a first aspect, referring to fig. 1, an embodiment of the present disclosure provides a road condition prompting method, including:
s1, generating a sound field signal according to road condition information, wherein the sound field signal is used for simulating road conditions represented by the road condition information.
S2, generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality mode.
The road condition prompting method provided by the embodiment of the disclosure can be executed in a central-control advanced driving assistance system (ADAS, advanced Driving Assistance System) of the automobile, and also can be executed in intelligent wearing equipment (such as an AR head-mounted display device) worn by a driver of the automobile. The embodiments of the present disclosure are not particularly limited thereto.
In the embodiment of the disclosure, the road condition information is information which is obtained by monitoring the road condition around the automobile and can reflect the real-time road condition. In some embodiments, the vehicle monitors road conditions around the vehicle through sensors such as millimeter wave radar, cameras and the like and obtains road condition information.
In the embodiment of the disclosure, generating the virtual sound field according to the sound field signal means that the virtual sound field is created according to the sound field signal reason surrounding stereo technology, and the road condition is simulated in an audio augmented reality mode, so that the purpose of prompting the driver of the road condition is achieved.
It should be noted that, the virtual sound field is created according to the sound field signal, which is triggered from auditory psychology, and a three-dimensional sound field is simulated, so that the driver feels to put in the body and three-dimensional space, and the driver can perceive road conditions according to the sound in the virtual sound field. For example, the driver can determine the position of the sound source according to the sound in the virtual sound field, and the position of the sound source in the virtual sound field represents the relative position relationship between the automobile and other target vehicles on the road, so that the driver can determine the relative position relationship between the automobile and other target vehicles on the road according to the sound in the virtual sound field, that is, the road condition prompt is obtained.
In the embodiment of the disclosure, the process of generating the sound field signal according to the road condition information is a process of constructing a three-dimensional sound field to simulate road conditions, and the sound field signal is a data description of the constructed three-dimensional sound field.
According to the road condition prompting method provided by the embodiment of the disclosure, the sound field signal can be generated according to the road condition information, the road condition represented by the road condition information is simulated, and then the virtual sound field is created according to the sound field signal, so that a driver can perceive the road condition by sound in the virtual sound field, and the road condition prompting of the driver in an audio augmented reality mode is realized. According to the embodiment of the disclosure, the road condition prompt is performed based on the hearing of the driver, the driver does not need to move the sight, potential safety hazards are eliminated, and the driving experience is improved based on the rich road condition content of the audio augmented reality prompt.
The embodiment of the disclosure does not limit the road condition content prompted in an audio augmented reality mode. In some embodiments, morphological features of the target vehicle surrounding the car, such as the size, shape, type, etc., of the target vehicle can be prompted in an audio augmented reality manner. In some embodiments, the motion state of a target vehicle around a car relative to the car, such as the position, speed, etc. of the target vehicle relative to the car, can be prompted in an audio augmented reality manner. In some embodiments, the morphological features of the target vehicles around the automobile can be prompted in an audio augmented reality manner, and the motion state of the target vehicles around the automobile relative to the automobile can be prompted.
The embodiment of the disclosure does not particularly limit how to generate the sound field signal according to the road condition information.
It should be noted that, in the embodiment of the present disclosure, the sound field signal generated according to the road condition information determines the road condition content prompted in the audio augmented reality manner. For example, if the generated sound field signal includes information simulating the morphological feature of the target vehicle, the virtual sound field generated from the sound field signal includes sound features capable of reflecting the morphological feature of the target vehicle; when the generated sound field signal includes information simulating the motion state of the target vehicle, the virtual sound field generated from the sound field signal includes sound features reflecting the motion state of the target vehicle. The embodiments of the present disclosure are not particularly limited thereto.
Accordingly, in some embodiments, referring to fig. 2, generating a sound field signal from road condition information includes:
s11, determining a virtual sound image according to the morphological information representing the morphological characteristics of the target vehicle in the road condition information, wherein the virtual sound image is used for simulating the morphological characteristics of the target vehicle through sound.
And S12, adding a sound field effect to the virtual sound image according to the motion information representing the motion state of the target vehicle in the road condition information so as to simulate the motion state of the target vehicle and obtain the sound field signal.
In the embodiment of the present disclosure, the virtual sound image is a virtual sound source corresponding to the target vehicle, and the virtual sound image is used to simulate sound emitted by the target vehicle. For example, the virtual sound image is used to simulate a sound in which at least one of a tire noise, a wind noise, an engine sound, and the like is mixed when the target vehicle is traveling.
The embodiment of the present disclosure does not particularly limit morphological features of the vehicle. For example, the morphological features of the vehicle may include the type of vehicle, such as a car, truck, off-road vehicle, etc.; morphological features of the vehicle may also include the size of the vehicle; the morphological characteristics of the vehicle may also include the weight of the vehicle.
It should be noted that, the sounds made by the vehicles in different forms during running are different, and in the embodiment of the disclosure, the sounds made by the vehicles in different forms during running can be simulated through different virtual sound images, so that a driver can distinguish the vehicles in different forms according to the virtual sound images.
It should also be noted that, in the embodiment of the present disclosure, a sound field effect is added to the virtual sound image, so that the driver can perceive the position, the motion state, and the like of the virtual sound source in the virtual sound field, and the position, the motion state, and the like of the virtual sound source in the virtual sound field are consistent with the position and the motion state of the target vehicle relative to the current vehicle, so that the driver can perceive the position and the motion state of the target vehicle relative to the current vehicle.
In some embodiments, the virtual sound image is processed according to auditory psychological effects based on a sound repair system (SRS, sound Retrieval System) to add a sound field effect to the virtual sound image, creating a virtual sound field.
In the embodiment of the disclosure, the morphological characteristics of the target vehicle are simulated by the virtual sound image, and the sound field effect is added for the virtual sound image, so that the morphological characteristics, the movement state and other road condition information of the target vehicle can be prompted to the driver, and the content of road condition prompt is enriched.
The embodiment of the present disclosure is not particularly limited as to how to determine the virtual sound image from the morphological information of the target vehicle.
In some embodiments, referring to fig. 3, determining the virtual sound image according to the morphological information representing the morphological feature of the target vehicle in the road condition information includes:
s111, determining the tonnage level of the target vehicle according to the morphological information;
s112, determining at least one voiceprint characteristic according to the tonnage level of the target vehicle, wherein voiceprint characteristics corresponding to different tonnage levels are different;
s113, generating the virtual sound image according to the at least one voiceprint feature.
In the embodiment of the disclosure, the vehicle is divided into a plurality of tonnage levels, and the voiceprint features including the tire noise, wind noise, engine sound, and the like of the vehicle of the tonnage level are recorded or extracted in advance corresponding to each tonnage level.
In the embodiment of the disclosure, a virtual sound image is generated according to at least one sound print characteristic, namely, the sound print characteristic corresponding to at least one sound such as tire noise, wind noise, engine sound and the like when a vehicle with a corresponding tonnage level runs is modulated, and the virtual sound image is generated.
It should be noted that, in the embodiment of the present disclosure, considering that the influence of vehicles with different tonnages on the running safety of the current vehicle is different, for example, the larger the tonnages of the vehicles, the larger the influence on the running safety of the current vehicle is, so that the tonnages information of the target vehicle is prompted to the driver in an audio augmented reality manner, which is beneficial for the driver to execute corresponding driving actions according to the tonnages of the target vehicle, thereby improving the driving safety.
The embodiments of the present disclosure are not particularly limited as to how the tonnage level of the target vehicle is determined.
In some embodiments, the morphology information includes a type and a size of the target vehicle; determining a tonnage level of the target vehicle according to the morphological information, including: and determining the tonnage level of the target vehicle according to the type and the size of the target vehicle.
In the embodiment of the present disclosure, the current vehicle can acquire the size of the target vehicle through sensors such as millimeter wave radar, cameras, and the like around the vehicle, and can recognize the type of the target vehicle, such as a car, a truck, an off-road vehicle, and the like. Based on the size and type of the target vehicle, the tonnage level of the target vehicle can be estimated.
In some embodiments, the size of the target vehicle is represented by the cross-sectional area of the target vehicle in the vertical plane.
The embodiment of the present disclosure is not particularly limited as to how to obtain the cross-sectional area of the target vehicle on the vertical plane.
In some embodiments, the current vehicle detects the distance between the target vehicle and the current vehicle by using millimeter wave radar, captures an image of the target vehicle by using a camera, then determines the position of the target vehicle in the image by using a target detection algorithm, extracts an image block of the target vehicle, and measures the number of pixels corresponding to the length and width of the image block; and then calculating the actual width and height of the target vehicle according to the distance of the target vehicle obtained by the millimeter wave radar and the lens focal length of the camera, and further calculating the cross-sectional area of the target vehicle on the vertical plane.
In some embodiments, referring to fig. 4, the motion information includes position information and speed information of the target vehicle relative to a current vehicle; according to the motion information representing the motion state of the target vehicle in the road condition information, adding a sound field effect to the virtual sound image, including:
s121, adding a sound field effect to the virtual sound image according to the position information and the speed information of the target vehicle relative to the current vehicle.
In the embodiment of the disclosure, a sound field effect is added to the virtual sound image, so that a driver can perceive the position and the moving speed of the virtual sound source in the virtual sound field, and the position and the moving speed of the virtual sound source in the virtual sound field are consistent with the position and the moving speed of the target vehicle relative to the current vehicle, so that the driver can perceive the position and the moving speed of the target vehicle relative to the current vehicle.
The embodiments of the present disclosure do not particularly limit the sound field effect.
In some embodiments, the sound field effect comprises at least one of a binaural sound difference, a binaural delay, an auricle effect, a reflection effect, an absorption effect, a doppler effect.
In some embodiments, the virtual sound image is processed according to auditory psychological effects based on a sound restoration system (SRS, sound Retrieval System), a sound field effect is added to the virtual sound image, and at least one of a difference in binaural sound, a binaural delay, a pinna effect, a reflection effect, an absorption effect, a doppler effect is simulated, thereby creating a virtual sound field.
In some embodiments, referring to fig. 5, before generating the sound field signal according to the road condition information, the road condition prompting method further includes:
s3, acquiring the road condition information.
In some embodiments, the road condition information is obtained through sensors such as millimeter wave radars, cameras and the like around the vehicle.
In the embodiment of the disclosure, when an event requiring prompting occurs, the road condition prompting is performed to the driver.
Accordingly, in some embodiments, referring to fig. 6, obtaining the road condition information includes:
s31, monitoring road conditions and judging whether an event to be prompted exists or not;
s32, when an event to be prompted exists, acquiring the road condition information.
The embodiment of the disclosure does not limit the event to be prompted in particular. For example, the event to be prompted may be the target vehicle entering a mirror/mirror blind zone of the current vehicle; the event to be prompted may also be that the distance between the target vehicle and the current vehicle is smaller than a specific threshold value; the event to be prompted may also be that the target vehicle is overtaking.
In the embodiment of the disclosure, road condition prompt is carried out to the driver only when the event requiring prompt occurs, so that the interference of unnecessary prompt information to the driver is avoided, and driving experience is improved.
The embodiment of the disclosure does not limit the acquisition of road condition information in particular.
In some embodiments, referring to fig. 7, when there is an event to be prompted, acquiring the road condition information includes:
s321, acquiring position information and speed information of a target vehicle relative to a current vehicle;
s322, acquiring video data of the target vehicle;
s323, determining the type and the size of the target vehicle according to the position information of the target vehicle relative to the current vehicle and the video data.
In some embodiments, the types of vehicles include cars, trucks, off-road vehicles, and the like.
In some embodiments, the size of the target vehicle is represented by the cross-sectional area of the target vehicle in the vertical plane.
The embodiment of the present disclosure is not particularly limited as to how to obtain the cross-sectional area of the target vehicle on the vertical plane.
In some embodiments, the current vehicle detects the distance between the target vehicle and the current vehicle by using millimeter wave radar, captures an image of the target vehicle by using a camera, then determines the position of the target vehicle in the image by using a target detection algorithm, extracts an image block of the target vehicle, and measures the number of pixels corresponding to the length and width of the image block; and then calculating the actual width and height of the target vehicle according to the distance of the target vehicle obtained by the millimeter wave radar and the lens focal length of the camera, and further calculating the cross-sectional area of the target vehicle on the vertical plane.
In some embodiments, referring to fig. 8, generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality manner includes:
s21, playing stereo audio according to the sound field signal through the audio playing device, and prompting road conditions.
In the embodiment of the disclosure, the audio playing device plays stereo audio according to the sound field signal, that is, creates a virtual sound field around the driver, thereby realizing road condition prompting in an audio augmented reality manner.
The embodiment of the present disclosure does not particularly limit the audio playback apparatus. For example, the audio playback device may be a sound system of a vehicle; the audio playback device may also be a playback system of a smart wearable device worn by the driver (e.g., an AR head mounted display device); the audio playing device may also be a headset connected to a central control system of the vehicle or to a smart wearable device worn by the driver.
In some embodiments, the audio playback device is a bone conduction headset.
In the embodiment of the disclosure, the bone conduction earphone is utilized to construct a virtual sound field to prompt the driver of road conditions, the bone conduction earphone can conduct sound information to the auditory nerve of the driver through the auditory ossicles, and the driver can receive other sounds through the ears, such as listening to music, navigating and the like, so that driving experience is improved.
In a second aspect, referring to fig. 9, an embodiment of the present disclosure provides an electronic device, including:
one or more processors 101;
a memory 102, on which one or more programs are stored, which when executed by one or more processors, cause the one or more processors to implement the road condition prompting method according to the first aspect of the embodiments of the present disclosure;
one or more I/O interfaces 103, coupled between the processor and the memory, are configured to enable information interaction of the processor with the memory.
Wherein the processor 101 is a device having data processing capabilities, including but not limited to a Central Processing Unit (CPU) or the like; memory 102 is a device with data storage capability including, but not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically charged erasable programmable read-only memory (EEPROM), FLASH memory (FLASH); an I/O interface (read/write interface) 103 is connected between the processor 101 and the memory 102 to enable information interaction between the processor 101 and the memory 102, including but not limited to a data Bus (Bus) or the like.
In some embodiments, processor 101, memory 102, and I/O interface 103 are connected to each other via bus 104, and thus to other components of the computing device.
In some embodiments, the electronic device is a head mounted display device. For example, augmented reality (AR, augmented Reality) head mounted display devices.
In some embodiments, an audio playback device is included in the electronic device. For example, the audio playback device is a bone conduction headset.
In a third aspect, referring to fig. 10, an embodiment of the present disclosure provides a computer readable medium having a computer program stored thereon, where the program when executed by a processor implements the road condition prompting method according to the first aspect of the embodiment of the present disclosure.
In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the following details of the technical solutions provided by the embodiments of the present disclosure are described by specific embodiments:
examples
The system architecture of the road condition prompt based on audio reality enhancement in this embodiment is shown in fig. 11. The system architecture is centered on an enhanced display (AR) head-mounted display device, and comprises a plurality of automobile sensors and an automobile central control system, wherein the AR head-mounted display device comprises a Sound Repair System (SRS) processing engine and a bone conduction earphone.
Wherein:
the bone conduction earphone is used for prompting the road condition to the driver in an audio augmented reality mode.
The SRS processing engine is used for simulating a three-dimensional sound field from auditory psychology, so that a driver feels to be in the three-dimensional sound field. In the three-dimensional sound field, the sound can come from any place in the spherical space around the driver, and the driver can accurately judge the position of the sound source. The SRS processing engine psychologically and subjectively restores the sound wave states (direct sound, reflected sound and reverberant sound) caused by the sound source in reality at two ears, reproduces the azimuth and the spatial distribution of the sound source in reality, and enables people to feel as if they were in the scene.
In addition, the system architecture also comprises a communication module, wherein the communication module can selectively support wireless communication protocols such as Bluetooth, wifi and the like or wired communication protocols such as USB and the like, and is used for receiving information transmitted by the automobile central control system, the instrument, the ADAS and the physical keys and transmitting the information to the AR head-mounted display equipment; the battery and power management module is used for supplying power to the AR head-mounted display device; and the operating system is responsible for software and hardware management of the system.
In this embodiment, the flow of road condition prompting based on audio reality enhancement includes:
1. the automobile central control system monitors road conditions in real time through sensors arranged at all directions of the current automobile, and when an event needing to be prompted is found and a driver needs to be prompted, road condition information is extracted and transmitted to the AR headset display device through the communication module. For example, if the millimeter wave radar irradiating the rear side finds that the target vehicle invades the blind area of the rearview mirror of the current vehicle, the detected relative position and speed information of the target vehicle is transmitted to the automobile central control system of the current vehicle, the automobile central control system selectively retrieves video data of the rear side camera according to the type of the information and identifies the target vehicle in the video, such as distinguishing a car, an off-road vehicle, a truck and the like, and measures the cross-sectional area of the invaded target vehicle on a vertical plane, and then the information is transmitted to the AR headset display device in a digital form in real time. The cross-sectional area may be obtained by: detecting the distance between the target vehicle and the current vehicle by using a millimeter wave radar, capturing an image of the target vehicle by using a camera, determining the position of the target vehicle in the image by using a target detection algorithm, extracting an image block of the target vehicle, and measuring the number of pixels corresponding to the length and the width of the image block; and then calculating the actual width and height of the target vehicle according to the distance of the target vehicle obtained by the millimeter wave radar and the lens focal length of the camera, and further calculating the cross-sectional area of the target vehicle on the vertical plane.
2. After the AR head-mounted display equipment receives the road condition information transmitted by the automobile central control system, the road condition information is subjected to secondary processing: estimating tonnage of the target vehicle according to the type of the target vehicle and the cross-sectional area of the target vehicle on a vertical plane; the tonnage of the vehicle is divided into a plurality of tonnage levels in advance, each tonnage level corresponds to a voiceprint feature, the voiceprint feature is prerecorded and extracted, the voiceprint feature is approximately equal to the mixture of at least one of tire noise, wind noise, engine sound and the like when the vehicle with the tonnage level runs, and a virtual sound image is modulated by utilizing the voiceprint feature corresponding to the target vehicle to simulate the target vehicle.
The SRS processing engine builds a virtual sound field according to the auditory psychological effect, and then utilizes the position and the moving speed of the virtual sound image in the sound field to simulate the azimuth and the relative speed of the rear vehicle, so as to achieve the purpose of reminding a driver. The principle is as follows:
since there is a certain distance between the ears of a person, the distance from the sound source to the ears differs as long as the sound is not emitted from the mid-plane of the line connecting the ears. The sound needs time to travel in the air, and different distances can cause different travel times, and meanwhile, because the sound can be attenuated along with the increase of the distance, the loudness of the sound reaching two ears is different. These two factors allow us to discern the orientation of the sound in the horizontal direction. While sound propagates in the medium with amplitude decreasing with increasing distance and generally with attenuation of high frequency components to a greater degree than low frequency components. Our brain can calculate the approximate distance of the sound source from our based on such characteristics. However, since we cannot know what the sound source should be, we can only infer the distance of the sound empirically, and have no way to make an accurate judgment. Only when the spectral distribution and amplitude of the sound source can be known exactly, the distance of the sound source can be calculated exactly, knowing the attenuation characteristics of the medium with respect to frequency. But when the sound source is on the bisector of the two earlobes, the two ears receive sound without phase difference and loudness difference, how does we determine the position of the sound? Our ears have very specific shapes and complex folds and projections allow the original sound to be superimposed with the sound reflected by the pinna after reaching the pinna. Due to the phase difference, some frequency components are enhanced and some frequency components are weakened. In this way we can more specifically discern the orientation of the sound by the sound being altered by the pinna. For the pinna we can build a function describing its effect on the sound frequency characteristics, this function being the Head related transfer function (HRTF, head-related Transfer Function). The HRTF is a function of the sound location, and sounds in different places have different frequency response characteristics after passing through the HRTF, so that we can distinguish sounds in different places.
The HRTF synthesis simulation circuit can be divided into two main parts: digital filters and time delays.
The application first sends the HRTF the preferred synthesized coordinates to find the appropriate parameters, which he then returns to the digital filter. The left and right channel filters calculate a frequency profile that is easily heard by the person-the difference in volume of both ears (IAD) and the auricle effect of this frequency.
After modeling the 3D characteristics of the frequency bins, the only thing left is the two-ear time delay difference (ITD). The synthesized ITD is only needed to reconstruct the approximate estimated delay according to the digital delay circuit.
The time delay, wall absorption effects, field effects and doppler effects of the reflection can be implemented through an application programming interface.
3. The central processing unit of the AR headset display device transmits the processed modulation signals to the bone conduction earphone in real time, and the earphone restores sound field signals through a stereo technology to remind a driver of road conditions.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, it will be apparent to one skilled in the art that features, characteristics, and/or elements described in connection with a particular embodiment may be used alone or in combination with other embodiments unless explicitly stated otherwise. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as set forth in the appended claims.
Claims (13)
1. A road condition prompting method, comprising:
generating a sound field signal according to road condition information, wherein the sound field signal is used for simulating the road condition represented by the road condition information;
and generating a virtual sound field according to the sound field signal, and prompting road conditions in an audio augmented reality mode.
2. The road condition prompting method according to claim 1, wherein generating a sound field signal according to road condition information comprises:
determining a virtual sound image according to the morphological information representing the morphological characteristics of the target vehicle in the road condition information, wherein the virtual sound image is used for simulating the morphological characteristics of the target vehicle through sound;
and adding a sound field effect to the virtual sound image according to the motion information representing the motion state of the target vehicle in the road condition information so as to simulate the motion state of the target vehicle and obtain the sound field signal.
3. The road condition prompting method according to claim 2, wherein determining a virtual sound image from morphological information representing morphological features of a target vehicle in the road condition information comprises:
determining the tonnage level of the target vehicle according to the morphological information;
determining at least one voiceprint feature according to the tonnage level of the target vehicle, wherein the voiceprint features corresponding to different tonnage levels are different;
and generating the virtual sound image according to the at least one voiceprint feature.
4. The road condition prompting method according to claim 3, wherein the morphological information includes a type and a size of the target vehicle; determining a tonnage level of the target vehicle according to the morphological information, including:
and determining the tonnage level of the target vehicle according to the type and the size of the target vehicle.
5. The road condition prompting method according to claim 2, wherein the motion information includes position information and speed information of the target vehicle with respect to a current vehicle; according to the motion information representing the motion state of the target vehicle in the road condition information, adding a sound field effect to the virtual sound image, including:
and adding a sound field effect to the virtual sound image according to the position information and the speed information of the target vehicle relative to the current vehicle.
6. The road condition prompting method according to claim 5, wherein the sound field effect includes at least one of a binaural sound difference, a binaural time delay, a auricle effect, a reflection effect, an absorption effect, a doppler effect.
7. The traffic prompt method according to any one of claims 1 to 6, wherein before generating the sound field signal from the traffic information, the traffic prompt method further comprises:
and acquiring the road condition information.
8. The road condition prompting method according to claim 7, wherein obtaining the road condition information comprises:
monitoring road conditions and judging whether an event to be prompted exists or not;
and when an event to be prompted exists, acquiring the road condition information.
9. The traffic prompt method according to claim 8, wherein, when there is an event to be prompted, acquiring the traffic information includes:
acquiring position information and speed information of a target vehicle relative to a current vehicle;
acquiring video data of the target vehicle;
and determining the type and the size of the target vehicle according to the position information of the target vehicle relative to the current vehicle and the video data.
10. The road condition prompting method according to any one of claims 1 to 6, wherein generating a virtual sound field from the sound field signal, and performing road condition prompting in an audio augmented reality manner, comprises:
and playing stereo audio according to the sound field signal through the audio playing equipment to prompt road conditions.
11. The road condition prompting method according to claim 10, wherein the audio playing device is a bone conduction headset.
12. An electronic device, comprising:
one or more processors;
a memory having one or more programs stored thereon, which when executed by the one or more processors cause the one or more processors to implement the road condition prompting method according to any one of claims 1 to 11.
13. A computer readable medium having stored thereon a computer program which when executed by a processor implements a road condition prompting method according to any one of claims 1 to 11.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210749766.4A CN117351754A (en) | 2022-06-29 | 2022-06-29 | Road condition prompting method, electronic equipment and computer readable medium |
| PCT/CN2023/101492 WO2024001884A1 (en) | 2022-06-29 | 2023-06-20 | Road condition prompting method, and electronic device and computer-readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210749766.4A CN117351754A (en) | 2022-06-29 | 2022-06-29 | Road condition prompting method, electronic equipment and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117351754A true CN117351754A (en) | 2024-01-05 |
Family
ID=89360023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210749766.4A Pending CN117351754A (en) | 2022-06-29 | 2022-06-29 | Road condition prompting method, electronic equipment and computer readable medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN117351754A (en) |
| WO (1) | WO2024001884A1 (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5052241B2 (en) * | 2007-07-19 | 2012-10-17 | クラリオン株式会社 | On-vehicle voice processing apparatus, voice processing system, and voice processing method |
| JP2013143744A (en) * | 2012-01-12 | 2013-07-22 | Denso Corp | Sound image presentation device |
| US20150365743A1 (en) * | 2014-06-14 | 2015-12-17 | GM Global Technology Operations LLC | Method and apparatus for including sound from an external environment into a vehicle audio system |
| JP6564576B2 (en) * | 2015-02-16 | 2019-08-21 | 修一 田山 | Proximity alarm device for automobiles |
| US10403141B2 (en) * | 2016-08-19 | 2019-09-03 | Sony Corporation | System and method for processing traffic sound data to provide driver assistance |
| CN112882568A (en) * | 2021-01-27 | 2021-06-01 | 深圳市慧鲤科技有限公司 | Audio playing method and device, electronic equipment and storage medium |
-
2022
- 2022-06-29 CN CN202210749766.4A patent/CN117351754A/en active Pending
-
2023
- 2023-06-20 WO PCT/CN2023/101492 patent/WO2024001884A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024001884A1 (en) | 2024-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3285241B1 (en) | System and method for processing traffic sound data to provide driver assistance | |
| US10343602B2 (en) | Spatial auditory alerts for a vehicle | |
| US5979586A (en) | Vehicle collision warning system | |
| JP2013198065A (en) | Sound presentation device | |
| JP5668765B2 (en) | In-vehicle acoustic device | |
| JP2007328603A (en) | Vehicle warning device | |
| US20180022357A1 (en) | Driving recorder system | |
| CN114394109A (en) | Driving assistance method, device, equipment, medium and program product | |
| Bellotti et al. | Using 3d sound to improve the effectiveness of the advanced driver assistance systems | |
| CN113619483A (en) | Reminding method for vehicle, corresponding reminding device and vehicle | |
| JP2020086907A (en) | Careless driving determination device | |
| CN117351754A (en) | Road condition prompting method, electronic equipment and computer readable medium | |
| JP2020154795A (en) | Output control device, output control method, and output control program | |
| CN110626350A (en) | Side and rear blind spot vehicle identification anti-collision early warning system and method based on 3D sound field | |
| CN110139205B (en) | Method and device for auxiliary information presentation | |
| US20230162609A1 (en) | Reporting device, vehicle, and reporting control method | |
| CN220594782U (en) | Alarm system for vehicle | |
| US11943581B2 (en) | Transparent audio mode for vehicles | |
| WO2021124918A1 (en) | Signal processing device, signal processing method, and range finding device | |
| CN117037535A (en) | Collision early warning system, vehicle, method and device | |
| WO2024001177A1 (en) | Visual field enhancement method, electronic device and storage medium | |
| Lylykangas et al. | Ears Outside the Car: Evaluation of Binaural Vehicular Sounds and Visual Animations as Driver's Blind Spot Indicators | |
| CN117533326A (en) | Vehicle control method, device, storage medium and electronic equipment | |
| WO2020090456A1 (en) | Signal processing device, signal processing method, and program | |
| JP2006182324A (en) | Alarm device for vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |