CN109657593A - A kind of trackside information fusion method and system - Google Patents
A kind of trackside information fusion method and system Download PDFInfo
- Publication number
- CN109657593A CN109657593A CN201811518421.8A CN201811518421A CN109657593A CN 109657593 A CN109657593 A CN 109657593A CN 201811518421 A CN201811518421 A CN 201811518421A CN 109657593 A CN109657593 A CN 109657593A
- Authority
- CN
- China
- Prior art keywords
- ultrasonic sensor
- ultrasonic
- drive test
- sensor data
- machine vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/86—Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/88—Sonar systems specially adapted for specific applications
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention discloses a kind of trackside information fusion method and systems, this method comprises: machine vision sensor obtains the drive test image in preset range；Ultrasonic sensor is continuously sent out and received ultrasonic signal, and ultrasonic signal is converted to Ultrasonic Sensor Data；By drive test image and the Ultrasonic Sensor Data, module is sent to server by wireless communication；The altimetric image that satisfies the need carries out pretreatment and remove to Ultrasonic Sensor Data to make an uproar；According to except after making an uproar Ultrasonic Sensor Data obtain subpoint of the effective target in the drive test image pixel plane；Area-of-interest is established around the subpoint, completes Ultrasonic Sensor Data and the fusion of machine vision information space.This method can establish in the plane of the data projection of ultrasonic sensor to image pixel area-of-interest around subpoint, complete Ultrasonic Sensor Data and the fusion of machine vision information space, realization is accurately identified and detected to drive test information.
Description
Technical field
The present invention relates to the technical fields of data sampling and processing, more particularly to a kind of trackside information fusion side
Method and system.
Background technique
Currently, automobile Internet of Things is increasingly becoming therein important answer as Internet of Things is applied in every profession and trade increasingly deep
One of with.Car networking is by advanced sensor technology, the communication technology, data processing technique, network technology, automatically controls skill
Real-time, accurate, the efficient traffic of one kind that art etc. organically applies to entire traffic management system and sets up is transported
Defeated integrated management and control system, the industry is at present in foreign countries by high attention, it is considered to be radio frequency technology of Internet of things
One major application.
Car networking includes vehicle-car networking and Che-road networking, and the final target of vehicle-car networking is to realize that vehicle is driven automatically
It sails, and vehicle-car networking premise is must to realize Che-road networking first." Che-road " information system is always intelligent transportation hair
The major fields of exhibition.In the world, the systems such as the IVHS in the U.S., Japanese VICS pass through establishes effectively between vehicle and road
Information communication, have been realized in the management and information service of intelligent transportation.In China, most has and realize that car networking is quickly fallen
Answer accessory for the highway field of advanced IT application in ground.
Car networking includes a variety of services towards fields such as public security, traffic, internet, personal users, is needed to collection
Mass data carry out concurrent processing.So data middleware technology is made full use of in car networking trackside information service platform,
Mass data processing and shared efficiency can be effectively improved.
But information processing at present and fusion still need to be further increased the space of computing capability.
Summary of the invention
In view of this, the present invention provides a kind of trackside information fusion methods and system, this method to pass through ultrasonic wave respectively
Sensor and visual sensor perceive drive test information, and the method for proposing data fusion improves accuracy of identification.
To achieve the goals above, the present invention adopts the following technical scheme:
In a first aspect, the present invention provides: a kind of trackside information fusion method, comprising:
From in vehicle driving process, machine vision sensor persistently obtains the drive test image in preset range；Ultrasonic wave passes
Sensor is continuously sent out and received ultrasonic signal, and the ultrasonic signal sent and received is converted to supersonic sensing
Device data；
By the drive test image and the Ultrasonic Sensor Data, module is sent to server by wireless communication；
The drive test image is carried out pretreatment and remove to the Ultrasonic Sensor Data to make an uproar；
Effective target is obtained in the drive test image pixel plane according to except the Ultrasonic Sensor Data described after making an uproar
Subpoint；
Area-of-interest is established around the subpoint, completes Ultrasonic Sensor Data and machine vision information space
Fusion.
In one embodiment, the wireless communication module is 5G mobile communications network module.
In one embodiment, the drive test image is pre-processed, comprising: image gray processing and filtering processing.
In one embodiment, the Ultrasonic Sensor Data remove and be made an uproar, comprising:
Obtain Noise ultrasound echo signal mathematical model f (t)=s (t)+r (t), s (t) be probe receive decline
Ultrasound echo signal after subtracting, r (t) are all noises including structural noise；
It chooses wavelet and calculates the scaling function and filter function of the wavelet；
By the filter function convolution of ultrasound echo signal mathematical model and wavelet, the ultrasonic signal denoised.
In one embodiment, effective target is obtained in the drive test according to except the Ultrasonic Sensor Data described after making an uproar
Subpoint in image pixel plane, comprising:
The first conversion obtained between ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor coordinate system is closed
System；
Obtain the second transformational relation between the machine vision sensor coordinate system and pixel coordinate system；
According to first transformational relation and the second transformational relation, realize the ultrasonic sensor coordinate system with it is described
Conversion between pixel coordinate obtains the effective target of the ultrasonic sensor detection in the drive test image pixel plane
On subpoint.
In one embodiment, area-of-interest is established around the subpoint, completes Ultrasonic Sensor Data and machine
Device visual information Space integration, comprising:
Centered on subpoint, according to inverted pyramid model, establish with effective target distance increases and reduced sense is emerging
Interesting region.
Second aspect, the present invention also provides a kind of trackside information fusion systems, comprising:
Module is obtained, for from vehicle driving process, machine vision sensor persistently to obtain the drive test in preset range
Image；Ultrasonic sensor is continuously sent out and received ultrasonic signal, and the ultrasonic signal sent and received is converted
For Ultrasonic Sensor Data；
Sending module, for module to be sent out by wireless communication by the drive test image and the Ultrasonic Sensor Data
Give server；
Processing module, for carrying out pretreatment to the drive test image and being carried out to the Ultrasonic Sensor Data
Except making an uproar；
Projection module removes the Ultrasonic Sensor Data after making an uproar for foundation and obtains effective target in the drive test figure
As the subpoint on pixel planes；
Fusion Module completes Ultrasonic Sensor Data and machine for establishing area-of-interest around the subpoint
Visual information Space integration.
In one embodiment, in the sending module, the wireless communication module is 5G mobile communications network module.
In one embodiment, the processing module, comprising: pre-process submodule and except submodule of making an uproar；
It is described except submodule of making an uproar includes:
Subelement is obtained, for obtaining ultrasound echo signal mathematical model f (t)=s (t)+r (t) of Noise, s (t)
It is the ultrasound echo signal after the decaying that probe receives, r (t) is all noises including structural noise；
Subelement is chosen, for choosing wavelet and calculating the scaling function and filter function of the wavelet；
Convolution subelement, for being gone the filter function convolution of ultrasound echo signal mathematical model and wavelet
The ultrasonic signal made an uproar.
In one embodiment, the projection module, comprising:
First acquisition submodule is sat for obtaining ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor
The first transformational relation between mark system；
Second acquisition submodule, for obtaining between the machine vision sensor coordinate system and pixel coordinate system
Two transformational relations；
Transform subblock, for realizing the supersonic sensing according to first transformational relation and the second transformational relation
Conversion between device coordinate system and the pixel coordinate obtains the effective target of the ultrasonic sensor detection on the road
Subpoint on altimetric image pixel planes.
In one embodiment, the Fusion Module is specifically used for centered on subpoint, according to inverted pyramid model,
It establishes as effective target distance increases and reduced area-of-interest.
It can be seen via above technical scheme that compared with prior art, the present disclosure provides one kind to be based on ultrasonic wave
With machine vision information fusion method, this method passes through ultrasonic sensor respectively and visual sensor perceives drive test information, mentions
The method of data fusion out, improves accuracy of identification.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
The embodiment of the present invention for those of ordinary skill in the art without creative efforts, can be with
Other attached drawings are obtained according to the attached drawing of offer.
Fig. 1 attached drawing is the flow chart of trackside information fusion method provided by the invention.
Fig. 2 attached drawing is the schematic diagram that ultrasonic sensor provided by the invention and machine vision sensor are installed.
Fig. 3 attached drawing is ultrasonic sensor operation principle schematic diagram provided by the invention.
Fig. 4 attached drawing is ultrasonic sensor detection schematic diagram provided by the invention.
Fig. 5 attached drawing is the structural schematic diagram of piezoelectric-type ultrasonic wave sensor provided by the invention.
Fig. 6 attached drawing is ccd image sensor structural schematic diagram provided by the invention.
Fig. 7 attached drawing is that Ultrasonic Sensor Data provided by the invention is carried out except the flow chart made an uproar.
Fig. 8 attached drawing is that Ultrasonic Sensor Data provided by the invention obtains effective target in drive test image pixel plane
On subpoint flow chart.
Fig. 9 attached drawing is the block diagram of trackside information fuse device provided by the invention.
Figure 10 attached drawing is the block diagram of processing module 93 provided by the invention.
Figure 11 attached drawing is the block diagram provided by the invention based on ultrasonic wave and machine vision information fuse device.
In figure: 1- ultrasonic sensor；2- machine vision sensor.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts it is all its
His embodiment, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a kind of trackside information fusion methods, shown referring to Fig.1, including S101~S105；
S101, from vehicle driving process, machine vision sensor persistently obtains the drive test image in preset range；It is super
Sonic sensor is continuously sent out and received ultrasonic signal, and the ultrasonic signal sent and received is converted to ultrasound
Wave sensor data；
S102, by the drive test image and the Ultrasonic Sensor Data, module is sent to service by wireless communication
Device；
S103, it carries out pretreatment to the drive test image and the Ultrasonic Sensor Data remove to make an uproar；
S104, according to except the Ultrasonic Sensor Data described after making an uproar, to obtain effective target flat in the drive test image pixel
Subpoint on face；
S105, area-of-interest is established around the subpoint, complete Ultrasonic Sensor Data and machine vision information
Space integration.
Referring to shown in Fig. 2, ultrasonic sensor 1 and machine vision sensor 2 are installed on the vehicle.Wave ultrasonic wave is then
Frequency is higher than the mechanical wave of 20kHz, and the characteristic of ultrasonic wave is that frequency is high, wavelength is short, diffraction phenomenon is small.Its most significant characteristic
It is good directionality, penetrating power is big, and apparent reflection and refraction can be generated by encountering dielectric interface, can be applied to detection road
Measurement information, such as detection roadbed, road sign, direction board and abrupt slope, crossings on different level, the parallel road of difference in height, lane, lane
Medium line, lane sideline, road boundary, pavement marker, signal lamp, pavement, road periphery object etc..
Because ultrasound examination angle is too small, vehicle needs to install several in different angle, in addition to this ultrasonic wave
Advantage: it is 1., waterproof, dust-proof, even a small amount of silt blocks；2., have the probe of metal material, can be with carbody shell knot
That closes is fine；3., the smallest monitoring distance can reach 0.1-0.3 meters；Further, for more typical 40KHz supersonic sensing
Device, range accuracy are about 1~3 centimetre or so (depending on back-end circuit and server data processing performance).
Ultrasonic wave usually has longitudinal wave, shear wave and surface wave, when ultrasonic wave is propagated in the medium, with the increasing of propagation distance
Add, energy is gradually decayed.The decaying of energy is decided by the diffusion, scattering and absorption of ultrasonic wave.
Specifically, in the present embodiment, ultrasonic sensor important technological parameters:
Use voltage: DC5V；
Quiescent current: less than 2mA；
Level output: high 5V；
Level output: bottom 0V；
Incude angle: being not more than 15 degree；
Detection range: 50px-11250px, in high precision up to 5px；
Mode of connection port: VCC (power supply), trig (control terminal), echo (receiving end), GND (ground).
The course of work is as follows: as shown in Figure 3-4；
1. ranging is triggered using I/O, at least high level signal of 10us；
2. module sends the square wave of 8 40kHZ automatically, signal return has been detected whether automatically；
3. there is signal return, a high level is exported by I/O, high level duration is exactly ultrasonic wave from being emitted to
The time of return.
Measuring distance=(the high level time * velocity of sound (340M/S))/2.
Specific in the present embodiment, the types such as piezoelectric type, magnetostriction type, electromagnetic type are can be used in ultrasonic sensor
Sensor, the present embodiment does not limit.
By taking piezoelectric-type ultrasonic wave sensor as an example:
It is illustrated in figure 5 the structure of piezoelectric-type ultrasonic wave sensor, piezoelectric-type ultrasonic wave sensor is to utilize piezoelectric material
Piezoelectric effect principle carry out work.Common sensing element material mainly has piezo-electric crystal and piezoelectric ceramics.
According to the difference of forward and inverse piezoelectric effect, piezoelectric-type ultrasonic wave sensor is divided into generator (transmitting probe) and receives
Two kinds of device (receiving transducer), can be divided into according to structure with the wave mode difference used normal probe, surface wave probe, Lamb wave probe,
Variable angle probe, double crystal probe, focusing probe, immersion probe, water spray probe and dedicated probe etc..
Piezoelectric-type ultrasonic wave producer is that high frequency electric oscillation is converted into high frequency machinery using the principle of inverse piezoelectric effect to shake
It is dynamic, to generate ultrasonic wave.Resonance can be generated when the frequency of additional alternating voltage is equal to the intrinsic frequency of piezoelectric material, this
When the ultrasonic wave that generates it is most strong.Piezoelectric-type ultrasonic wave sensor can produce tens kilo hertzs to tens megahertzs of high-frequency ultrasonic,
Its sound intensity is every square centimeter up to tens watts.
Piezoelectric ultrasonic receiver is worked using direct piezoelectric effect principle.When ultrasonic wave is applied to piezoelectricity
Cause chip flexible on chip, just generates opposite polarity charge on two surfaces of chip, these charges are converted into electricity
Press it is amplified be sent to measuring circuit, finally record or show.The structure and ultrasonic wave of piezoelectric ultrasonic receiver are sent out
Raw device is essentially identical, just doubles as two kinds of purposes of generator and receiver with the same sensor sometimes.
Ultrasonic sensor structure is mainly made of piezoelectric chip, absorption block (damping block), protective film etc..Piezoelectric chip
Mostly circular plate type, ultrasonic frequency are inversely proportional with its thickness.The two sides of piezoelectric chip is silver coated, as conductive pole plate, bottom
Face ground connection, is connected to lead-out wire above.In order to avoid sensor is directly contacted with measured piece and wears piezoelectric chip, in piezo crystals
Layer protecting film is bonded under piece.The effect for absorbing block is the mechanical quality for reducing piezoelectric chip, absorbs the energy of ultrasonic wave.
In step S101, the image acquisition units of machine vision sensor mainly by CCD/CMOS camera, optical system,
Lighting system and image pick-up card composition, are converted into digital picture for optical image, pass to image processing unit.Usually make
Imaging sensor mainly has ccd image sensor and two kinds of cmos image sensor.
In the present embodiment, by taking ccd image sensor as an example, referring to shown in Fig. 6, ccd image sensor is by micro lens, filter
Color chips, photosensitive element up of three-layer.Each photosensitive element of ccd image sensor is by a photodiode and control phase
The storage unit composition photoelectric tube of adjacent charge is for capturing photon use, and converting photons are electronics by it, and the light being collected into is got over
By force, the electron amount of generation is more, and the electronic signal the strong, easier to be recorded and not easy to lose, and image detail is then more
Add abundant.
In this embodiment, passing through the acquisition of both machine vision sensor and ultrasonic sensor from vehicle driving process
Data, be sent to server through wireless communication module, server after treatment, by the data projection of ultrasonic sensor
Onto the plane of image pixel, area-of-interest is established around subpoint, completes Ultrasonic Sensor Data and machine vision letter
Space integration is ceased, completion is accurately identified and detected to drive test information, provides sound assurance for subsequent vehicle automatic obstacle-avoiding.
Above-mentioned steps are described in detail separately below.
In one embodiment, in step S102, wireless communication module is preferably 5G mobile communications network, and 5G has more
High rate, broader bandwidth, therefore 5G has higher reliability；Using in the present embodiment, mainly utilize it lower
Time delay feature, can satisfy safe driving of vehicle, such as the particular demands of automatic Pilot.
In one embodiment, in step S103, above-mentioned drive test image is pre-processed, pretreatment is broadly divided into figure
As gray processing and filtering.It is triple channel RGB color image by the image that ccd image sensor obtains, first has to carry out it
It is converted into grayscale image from cromogram by gray processing.For any pixel point I, conversion formula are as follows:
Igray=IR×0.299+IG×0.587+IB×0.114
After obtaining grayscale image, it is filtered to remove the interference of the unrelated lines of the noise in image.Such as
Uniform filtering can be used, give one template of object pixel on the image, which is made of several pixels of its neighbour, modulus
The mean value of all pixels in plate, then current pixel point I is assigned the mean value(x, y), as the ash of image after processing at that point
Spend g(x,y), i.e.,
M is comprising the total number of pixels including current pixel in the template, and S indicates template area, I(m,n)Expression point (m,
N) gray value at place.In order to effectively filter interference information, especially lines interference information, slightly larger template may be selected: such as M=
{ 36,49,64 } etc..
In one embodiment, above-mentioned Ultrasonic Sensor Data is carried out except making an uproar, referring to Fig. 7, comprising:
S701, ultrasound echo signal mathematical model f (t)=s (the t)+r (t) for obtaining Noise, s (t) are that probe receives
Ultrasound echo signal after the decaying arrived, r (t) are all noises including structural noise；
S702, it chooses wavelet and calculates the scaling function and filter function of the wavelet；
S703, by the filter function convolution of ultrasound echo signal mathematical model and wavelet, the ultrasonic wave denoised
Signal.
In the present embodiment, the ultrasound echo signal mathematical model with defect is represented by
F (t)=s (t)+r (t) (1)
S (t) is the ultrasound echo signal after the decaying that probe receives；R (t) refers to the institute including structural noise
There is noise, takes r (t)=br under normal circumstancesn(t), b is noise coefficient, rnIt (t) is structural noise.
Then, using the small wave system of Daubechies as wavelet (also referred to as morther wavelet), filter function is ψ (t)；This
It is used in invention dbN (N is wavelet-order), rule and the size selection of threshold value should be according to ultrasound echo signal attenuations
And noise size elects；Preferably, the level-one in 4~6 grades is selected.
For continuous wavelet transform filter coefficient h0(n) and h1(n) scaling function is solvedWith filter function ψ (t).
It enablesψ(t)、h0(n)、h1(n) corresponding Fourier transformation is Φ (w), Ψ (w), H respectively0(W)、H1(W)。
H0(z)=h0(0)+h0(1)z-1+h0(2)z-2+h0(3)z-3+......+h0(n)z-n (5)
H is obtained by formula (5) formula0(z), H is obtained using transform0(W), by H0(W) and H1(W) Φ (w), Ψ are solved
(w) scaling function and filter function can then be found out.It is worth noting that can be solved by formula (5) only under rare occasion
The solution of analysis form, it is however generally that, analytic solutions can not be in most cases acquired, it can only be to h0(n) make iterative numerical convolution algorithm
It acquires
After acquiring scaling function and filter function, it is modified by formula (7) by filter function of the scale to solution.Ruler
Spend functionIt is lowpass function, passes throughψ (t) can be found out, ψ (t) isWeighted shift:
K value is from 2-2N~1, and N value is different, weight gkValue it is also different.
Finally, after the function ψ (t) of wavelet is made displacement τ, then at different scale α with ultrasound echo signal mathematics
Model x (t) makees inner product:
Equivalent frequency domain representation are as follows:
After formula (7) operation, the ultrasonic signal of denoising can be obtained.
In one embodiment, in above-mentioned steps S104, according to except Ultrasonic Sensor Data above-mentioned after making an uproar obtain it is effective
Subpoint of the target in drive test image pixel plane, referring to shown in Fig. 8, comprising:
First between S801, acquisition ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor coordinate system
Transformational relation；
The second transformational relation between S802, the acquisition machine vision sensor coordinate system and pixel coordinate system；
S803, according to first transformational relation and the second transformational relation, realize the ultrasonic sensor coordinate system with
Conversion between the pixel coordinate obtains the effective target of the ultrasonic sensor detection in the drive test image pixel
Subpoint in plane.
In the present embodiment, by the conversion between ultrasonic sensor and machine vision sensor and various coordinate systems
The Space integration of relationship completion Ultrasonic Sensor Data and Machine Vision Detection information.
In one embodiment, area-of-interest is established around the subpoint, completes Ultrasonic Sensor Data and machine
Device visual information Space integration, comprising:
Centered on subpoint, according to inverted pyramid model, establish with effective target distance increases and reduced sense is emerging
Interesting region.
Data fusion module of the invention is divided into two parts: (1) it for the first time merges: R is extracted based on ultrasonic sensor:
Vehicle detection process carries out in R, wherein R are as follows: defining R is the polygon mask defined based on scene.Image mask is more in R
Side shape returns to 1, and other positions shown in the figure return to 0.By defining R, the processing region of video can be limited, to subtract
The calculating time and memory consumption that few network training needs.
In addition, Zone R domain eliminates the interference from ambient enviroment, and the processing region on road is focused together.
Using image doubling technology, the region outside mask will return to 0, can eliminate the region other than lane in this way.Based on ultrasonic wave
Sensor, drive test image area-of-interest by the classifier trained by convolutional neural networks carry out identification judge in the region
Whether have drive test information (such as detection roadbed, road sign, direction board and abrupt slope, crossings on different level, difference in height it is parallel
Road, lane, lane medium line, lane sideline, road boundary, pavement marker, signal lamp, pavement, road periphery object etc.
Deng).Therefore the definition of area-of-interest will directly influence fusion results.
(2) merge for second: candidate region: the process of classification is to judge whether some detection window has drive test information,
There is drive test information region to detection window and carries out two classification without drive test information region, and positioning will then point out have in image
Imitate the specific location of target.Using being exactly image-region in Fast-RCNN, extracts multiple candidate regions and be trained, then is whole
Close the result that territorial classification obtains.And the R for having merged Ultrasonic Sensor Data reduces original pixels, it can be according to ultrasound
The target range adjust automatically that wave sensor measures divides the size of candidate region.If apart from remote, it is clear that R is big, then amplification region
Domain reduces region if target is close.
Based on the same inventive concept, the present invention also provides a kind of trackside information fusion systems, referring to shown in Fig. 9, comprising:
Module 91 is obtained, for from vehicle driving process, machine vision sensor persistently to obtain the road in preset range
Altimetric image；Ultrasonic sensor is continuously sent out and received ultrasonic signal, and the ultrasonic signal sent and received is turned
It is changed to Ultrasonic Sensor Data；
Sending module 92, for by the drive test image and the Ultrasonic Sensor Data module by wireless communication
It is sent to server；
Processing module 93, for the drive test image carry out pretreatment and to the Ultrasonic Sensor Data into
Row is removed and is made an uproar；
Projection module 94 removes the Ultrasonic Sensor Data after making an uproar for foundation and obtains effective target in the drive test
Subpoint in image pixel plane；
Fusion Module 95 completes Ultrasonic Sensor Data and machine for establishing area-of-interest around the subpoint
Device visual information Space integration.
In one embodiment, in the sending module 92, the wireless communication module is 5G mobile communications network mould
Block.
In one embodiment, the processing module 93, referring to Fig.1 shown in 0, comprising: pre-process submodule 931 and remove
It makes an uproar submodule 932；
It is described except submodule 931 of making an uproar includes:
Subelement 9311 is obtained, for obtaining ultrasound echo signal mathematical model f (t)=s (t)+r (t) of Noise, s
(t) it is ultrasound echo signal after the decaying that receives of probe, r (t) is all noises including structural noise；
Choose subelement 9312, for choose wavelet and calculate the wavelet scaling function and filtering letter
Number；
Convolution subelement 9313, for obtaining the filter function convolution of ultrasound echo signal mathematical model and wavelet
To the ultrasonic signal of denoising.
In one embodiment, the projection module 94, referring to Fig.1 shown in 1, comprising:
First acquisition submodule 941, for obtaining ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor
The first transformational relation between coordinate system；
Second acquisition submodule 942, for obtaining between the machine vision sensor coordinate system and pixel coordinate system
Second transformational relation；
Transform subblock 943, for realizing the ultrasonic wave according to first transformational relation and the second transformational relation
Conversion between sensor coordinate system and the pixel coordinate obtains the effective target of the ultrasonic sensor detection in institute
State the subpoint in drive test image pixel plane.
In one embodiment, the Fusion Module 95 is specifically used for centered on subpoint, according to inverted pyramid mould
Type is established as effective target distance increases and reduced area-of-interest.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with its
The difference of his embodiment, the same or similar parts in each embodiment may refer to each other.For being filled disclosed in embodiment
For setting, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method portion
It defends oneself bright.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, defined herein
General Principle can realize in other embodiments without departing from the spirit or scope of the present invention.Therefore, originally
Invention is not intended to be limited to the embodiments shown herein, and is to fit to special with principles disclosed herein and novelty
The consistent widest scope of point.
Claims (9)
1. a kind of trackside information fusion method characterized by comprising
While the car is driving, machine vision sensor persistently obtains the drive test image in preset range；Ultrasonic sensor
It continuously sends out and received ultrasonic signal, and the ultrasonic signal sent and received is converted into ultrasonic sensor number
According to；
By the drive test image and the Ultrasonic Sensor Data, module is sent to server by wireless communication；
The drive test image is carried out pretreatment and remove to the Ultrasonic Sensor Data to make an uproar；
Throwing of the effective target in the drive test image pixel plane is obtained according to except the Ultrasonic Sensor Data after making an uproar
Shadow point；
Area-of-interest is established around the subpoint, completes Ultrasonic Sensor Data and the fusion of machine vision information space.
2. a kind of trackside information fusion method according to claim 1, which is characterized in that the wireless communication module is 5G
Mobile communications network module.
3. a kind of trackside information fusion method based on ultrasonic wave, machine vision sensor and 5G according to claim 1
And system, which is characterized in that the drive test image is pre-processed, comprising: image gray processing and filtering processing.
4. a kind of trackside information fusion method according to claim 1, which is characterized in that the ultrasonic sensor number
It makes an uproar according to remove, comprising:
Ultrasound echo signal mathematical model f (t)=s (t)+r (t) of Noise is obtained, s (t) is after popping one's head in the decaying received
Ultrasound echo signal, r (t) is all noises including structural noise；
It chooses wavelet and calculates the scaling function and filter function of the wavelet；
By the filter function convolution of ultrasound echo signal mathematical model and wavelet, the ultrasonic signal denoised.
5. a kind of trackside information fusion method according to claim 1, which is characterized in that according to except the ultrasonic wave after making an uproar
Sensing data obtains subpoint of the effective target in the drive test image pixel plane, comprising:
Obtain the first transformational relation between ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor coordinate system；
Obtain the second transformational relation between the machine vision sensor coordinate system and pixel coordinate system；
According to first transformational relation and the second transformational relation, realize that the ultrasonic sensor coordinate system and the pixel are sat
Conversion between mark obtains projection of the effective target of the ultrasonic sensor detection in the drive test image pixel plane
Point.
6. a kind of trackside information fusion method according to claim 1, which is characterized in that establish sense around the subpoint
Complete Ultrasonic Sensor Data and the fusion of machine vision information space in interest region, comprising:
Centered on subpoint, according to inverted pyramid model, establish as effective target distance increases and reduced region of interest
Domain.
7. a kind of trackside information fusion system characterized by comprising
Module is obtained, for from vehicle driving process, machine vision sensor persistently to obtain the drive test image in preset range；
Ultrasonic sensor is continuously sent out and received ultrasonic signal, and the ultrasonic signal sent and received is converted to ultrasound
Wave sensor data；
Sending module, for module to be sent to clothes by wireless communication by the drive test image and the Ultrasonic Sensor Data
Business device；
Processing module, for carrying out pretreatment and remove to the Ultrasonic Sensor Data to make an uproar to the drive test image；
Projection module removes the Ultrasonic Sensor Data after making an uproar for foundation and obtains effective target in the drive test image pixel
Subpoint in plane；
Fusion Module completes Ultrasonic Sensor Data and machine vision for establishing area-of-interest around the subpoint
Information space fusion.
8. a kind of trackside information fusion system according to claim 7, which is characterized in that the processing module, comprising: pre-
Handle submodule and except submodule of making an uproar；
It is described except submodule of making an uproar includes:
Subelement is obtained, for obtaining ultrasound echo signal mathematical model f (t)=s (t)+r (t) of Noise, s (t) is probe
Ultrasound echo signal after the decaying received, r (t) are all noises including structural noise；
Subelement is chosen, for choosing wavelet and calculating the scaling function and filter function of the wavelet；
Convolution subelement, for being denoised the filter function convolution of ultrasound echo signal mathematical model and wavelet
Ultrasonic signal.
9. a kind of trackside information fusion system according to claim 1, which is characterized in that the projection module, comprising:
First acquisition submodule, for obtain ultrasonic sensor two-dimensional plane coordinate system and machine vision sensor coordinate system it
Between the first transformational relation；
Second acquisition submodule, for obtaining the second conversion between the machine vision sensor coordinate system and pixel coordinate system
Relationship；
Transform subblock, for realizing that the ultrasonic sensor is sat according to first transformational relation and the second transformational relation
Conversion between mark system and the pixel coordinate obtains the effective target of the ultrasonic sensor detection in the drive test image
Subpoint on pixel planes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811518421.8A CN109657593B (en) | 2018-12-12 | 2018-12-12 | Road side information fusion method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811518421.8A CN109657593B (en) | 2018-12-12 | 2018-12-12 | Road side information fusion method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109657593A true CN109657593A (en) | 2019-04-19 |
| CN109657593B CN109657593B (en) | 2023-04-28 |
Family
ID=66113836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811518421.8A Active CN109657593B (en) | 2018-12-12 | 2018-12-12 | Road side information fusion method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109657593B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287832A (en) * | 2019-06-13 | 2019-09-27 | 北京百度网讯科技有限公司 | High-Speed Automatic Driving Scene barrier perception evaluating method and device |
| CN112535476A (en) * | 2020-12-01 | 2021-03-23 | 业成科技（成都）有限公司 | Fall detection system and method thereof |
| CN113362395A (en) * | 2021-06-15 | 2021-09-07 | 上海追势科技有限公司 | Sensor fusion-based environment sensing method |
| CN113963060A (en) * | 2021-09-22 | 2022-01-21 | 腾讯科技（深圳）有限公司 | Vehicle information image processing method and device based on artificial intelligence and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170020489A1 (en) * | 2013-10-09 | 2017-01-26 | Shenzhen Mindray Bio-Medical Electronics Co., Ltd. | Ultrasound fusion imaging method and ultrasound fusion imaging navigation system |
| CN107609522A (en) * | 2017-09-19 | 2018-01-19 | 东华大学 | A kind of information fusion vehicle detecting system based on laser radar and machine vision |
-
2018
- 2018-12-12 CN CN201811518421.8A patent/CN109657593B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170020489A1 (en) * | 2013-10-09 | 2017-01-26 | Shenzhen Mindray Bio-Medical Electronics Co., Ltd. | Ultrasound fusion imaging method and ultrasound fusion imaging navigation system |
| CN107609522A (en) * | 2017-09-19 | 2018-01-19 | 东华大学 | A kind of information fusion vehicle detecting system based on laser radar and machine vision |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287832A (en) * | 2019-06-13 | 2019-09-27 | 北京百度网讯科技有限公司 | High-Speed Automatic Driving Scene barrier perception evaluating method and device |
| CN112535476A (en) * | 2020-12-01 | 2021-03-23 | 业成科技（成都）有限公司 | Fall detection system and method thereof |
| CN113362395A (en) * | 2021-06-15 | 2021-09-07 | 上海追势科技有限公司 | Sensor fusion-based environment sensing method |
| CN113963060A (en) * | 2021-09-22 | 2022-01-21 | 腾讯科技（深圳）有限公司 | Vehicle information image processing method and device based on artificial intelligence and electronic equipment |
| CN113963060B (en) * | 2021-09-22 | 2022-03-18 | 腾讯科技（深圳）有限公司 | Vehicle information image processing method and device based on artificial intelligence and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109657593B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109657593A (en) | A kind of trackside information fusion method and system | |
| JP5878345B2 (en) | Large area ultrasonic contact image processing | |
| US8201739B2 (en) | Biometric sensor with delay layer | |
| US10014344B2 (en) | Large area ultrasonic receiver array | |
| Tang et al. | Pulse-echo ultrasonic fingerprint sensor on a chip | |
| Gao et al. | UV-disparity based obstacle detection with 3D camera and steerable filter | |
| CN102785719B (en) | Method for shooting water gage images of ship | |
| CN106915303B (en) | Automobile A-column blind area perspective method based on depth data and fish eye images | |
| CN102928844A (en) | Underwater sub-wavelength resolution ratio three-dimensional imaging method | |
| Shao et al. | 3D ultrasonic object detections with> 1 meter range | |
| CN104408746A (en) | Passenger flow statistical system based on depth information | |
| CN114581817B (en) | Method and system for intelligently detecting wave height and wind speed from offshore wave monitoring video | |
| CN101887579B (en) | Underwater image restoration method based on scattering model | |
| Asmare et al. | Image Enhancement by Fusion in Contourlet Transform. | |
| CN109674490A (en) | A kind of low reflection artifacts photoacoustic microscope imaging method of ultrasonic guidance | |
| CN101726829B (en) | Method for automatically focusing zoom lens | |
| CN105043524A (en) | Laser speckle defocused imaging-based vibration detection method | |
| TWI243903B (en) | Method using blurred image to measure speed | |
| CN108375368A (en) | A kind of embedded onboard distance measuring method and its system based on monocular vision | |
| CN105701811A (en) | Sound coding interaction method based on RGB-IR camera | |
| CN109799493A (en) | Radar and Multisensor video fusion system and method | |
| CN110187142A (en) | Flow monitoring method and system | |
| Earshia et al. | A Wearable Ultrasonic Obstacle Sensor for Aiding Visually Impaired and Blind Individuals | |
| Bi et al. | Infrared sensors and ultrasonic sensors | |
| NL2032646B1 (en) | Method and system for monitoring local changes of underwater topography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |