CN107844746A - Complex road condition sensory perceptual system and device based on deep learning - Google Patents
Complex road condition sensory perceptual system and device based on deep learning Download PDFInfo
- Publication number
- CN107844746A CN107844746A CN201710941720.1A CN201710941720A CN107844746A CN 107844746 A CN107844746 A CN 107844746A CN 201710941720 A CN201710941720 A CN 201710941720A CN 107844746 A CN107844746 A CN 107844746A
- Authority
- CN
- China
- Prior art keywords
- network
- road condition
- deep learning
- complex road
- sensory perceptual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
- Traffic Control Systems (AREA)
Abstract
Complex road condition sensory perceptual system and device based on deep learning.A kind of complex road condition sensory perceptual system based on deep learning, the system include five steps：The first step gathers vision sensor data；Second step pre-processes to view data, including white balance, gamma correction, denoising；3rd step carries out feature extraction using first deep neural network；The feature that 4th step is extracted using second deep neural network carries out target positioning；5th step uses carries out target identification using the 3rd deep neural network to localization region, obtains final testing result.The present invention is used for the complex road condition sensory perceptual system based on deep learning.
Description
Technical field：
The present invention relates to a kind of complex road condition sensory perceptual system and device based on deep learning.
Background technology：
It is a unified depth god by Technology Integrations such as feature extraction, target positioning, target identifications using depth learning technology
Through network, and form a whole set of complete end-to-end solution and take precautions against, and optimize the model of deep neural network on mobile unit
Complexity is based on performance, and visual task can be fast and accurately handled in onboard system.
Traditional method can only carry out single goal detection, if necessary to multi-target detection, need to use different features and
Different graders, the design difficulty of whole system is so added, and different features can not in different graders
It is shared, the repeatability of calculating is result in, the efficiency of detection, and extensive energy of traditional algorithm under complex scene can not be improved
Power is weaker, is unable to reach the value of practicality.
The content of the invention：
Analysis On Multi-scale Features are extracted using special network structure it is an object of the invention to provide a kind of, reuse an extra subnet
It is complexed the complex road condition sensory perceptual system and device based on deep learning of simultaneously Analysis On Multi-scale Features.
Above-mentioned purpose is realized by following technical scheme：
A kind of complex road condition sensory perceptual system based on deep learning, the system include five steps：The first step gathers imaging sensor number
According to；Second step pre-processes to view data, including white balance, gamma correction, denoising；3rd step uses first
Deep neural network carries out feature extraction；The feature that 4th step is extracted using second deep neural network carries out target positioning；
5th step uses carries out target identification using the 3rd deep neural network to localization region, obtains final testing result.
The described complex road condition sensory perceptual system based on deep learning, the 3rd described step is to do feature multiplexing；Described
4th step is to separate foreground and background；The network used in described the 3rd step, the 4th described step, the 5th described step is whole
Be combined into a deeper deep neural network, the network that each of which step uses as the sub-network of whole network at
Reason.
The described complex road condition sensory perceptual system based on deep learning, described sub-network are a kind of multiple dimensioned nerve nets
Network, feature extraction is carried out on different yardsticks, will be carried out finally by a special sub-network without the feature under yardstick
Integrate, obtain final Analysis On Multi-scale Features.
The described complex road condition sensory perceptual system based on deep learning, after Multi resolution feature extraction, it is exactly in next step
Generation positioning candidate frame；Using extra sub-network generation positioning anchor point, each anchor point is in conjunction with 5 dimensional variations and 5
Angular transformation, ultimately generate 25 candidate frames.
The described complex road condition sensory perceptual system based on deep learning, 1000 positioning anchor points are chosen, are ultimately generated
1000x25=25000 candidate frame；Two classification finally are carried out to this 25000 candidate frames using a sub-network, retained final
Target candidate frame.
A kind of complex road condition sensing device based on deep learning, its composition include:Adopted using DMA chips for data
The camera of collection, described camera are inserted in the slot of bottom plate, and the side of described bottom plate is connected by screw one end of fixed strip,
The other end connecting ring of described fixed strip, the opposite side of described bottom plate are provided with screwed hole, described screwed hole connection spiral shell
Bar, described screw rod coupling hook, the described link of described hook mounting fix described camera；The bottom of described bottom plate
Arc buckle is buckled, the both ends of described arc buckle are all connected with fixed plate, lead between described bottom plate and described fixed plate
Cross screw to fix, described arc buckle fixes support bar, and a side end of described support bar connects left screw rod, a described left side
Screw rod is sequentially connected with No.1 swivel nut and No. two swivel nuts, described No.1 swivel nut connection No.1 supporting leg, and described No. two swivel nuts connect
No. two supporting legs are connect, the end side of described support bar connects right screw rod, and described right screw rod is sequentially connected with No. three swivel nuts
With No. four swivel nuts, described No. three swivel nuts connect No. three supporting legs, and described No. four swivel nuts connect No. four supporting legs, and described one
Number supporting leg separately supports described bottom plate, described No. three supporting legs and No. four described supports with No. two described supporting legs
Leg separately supports described bottom plate.
Beneficial effect：
1. the present invention builds environment perception technology using depth learning technology, solves target positioning and identification under complex scene
Problem；The Multi resolution feature extraction of deep learning；Deep learning is accurately positioned；Efficiency and power consumption.
Traditional theory is thought, the generalization ability of network can be lifted using the network of deeper.But lifting network depth
While, its amount of calculation also greatly promotes, while its convergence time is more complicated, more difficult training.The present invention proposes one kind
Multiple dimensioned network, Analysis On Multi-scale Features are extracted using special network structure, reuse an extra sub-network and merge multiple dimensioned spy
Sign；The parameter format of each layer of network can be reduced using this network structure, therefore reduces the overall number of parameters of network, drop
The low network overall calculating time.With the reduction of parameter, its convergence rate also greatly promotes.
The present invention is in order to solve the test problems of Small object,, can using a liter dimension method when final feature is formed
Characteristic pattern is reflected to the characteristic pattern for being transformed to higher resolution, so when Small object is detected, using the teaching of the invention it is possible to provide more information, carry
Risen detection performance, meanwhile, than using higher resolution picture the methods of, reduce the consume of time.
The present invention can more accurately position target, and anchor point is become using 5 change of scale and 5 angular transformations
Change；5 yardsticks are 3.0,6.0,9.0,16.0 and 32.0 respectively, yardstick minimum to 8 pixels, are up to 512 pixels, greatly
The big demand for meeting detection.
The support bottom that No.1 supporting leg, No. two supporting legs, No. three supporting legs, No. four supporting legs of the present invention can either be stablized
The support camera that plate, No.1 supporting leg and No. two supporting legs, No. three supporting legs and No. four supporting legs can separately be stablized, No.1 branch
It is also very convenient that support leg merges carrying with No. two supporting legs, No. three supporting legs with No. four supporting legs.
The fixed strip of the present invention can be good at fixed camera, camera is fixed on bottom plate using very safety.
Brief description of the drawings：
Accompanying drawing 1 is the convolution kernel figure of this product.
Accompanying drawing 2 is the network structure of this product.
Accompanying drawing 3 is the structural representation of this product.
Accompanying drawing 4 is the rearview of accompanying drawing 3.
Embodiment：
Below in conjunction with the accompanying drawing of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described.
Embodiment 1：
A kind of complex road condition sensory perceptual system based on deep learning, the system include five steps：The first step gathers imaging sensor number
According to；Second step pre-processes to view data, including white balance, gamma correction, denoising；3rd step uses first
Deep neural network carries out feature extraction；The feature that 4th step is extracted using second deep neural network carries out target positioning；
5th step uses carries out target identification using the 3rd deep neural network to localization region, obtains final testing result.
Embodiment 2：
The complex road condition sensory perceptual system based on deep learning described in embodiment 1, the 3rd described step is to do feature multiplexing；It is described
The 4th step be to separate foreground and background；The network used in described the 3rd step, the 4th described step, the 5th described step
A deeper deep neural network is integrated into, the network that each of which step uses is carried out as the sub-network of whole network
Processing.
Embodiment 3：
The complex road condition sensory perceptual system based on deep learning described in embodiment 2, described sub-network are a kind of multiple dimensioned nerves
Network, feature extraction is carried out on different yardsticks, will entered finally by a special sub-network without the feature under yardstick
Row is integrated, and obtains final Analysis On Multi-scale Features.
Embodiment 4：
The complex road condition sensory perceptual system based on deep learning described in embodiment 3, after Multi resolution feature extraction, in next step
It is generation positioning candidate frame；Using extra sub-network generation positioning anchor point, each anchor point is in conjunction with 5 dimensional variations and 5
Individual angular transformation, ultimately generate 25 candidate frames.
Embodiment 5：
The complex road condition sensory perceptual system based on deep learning described in embodiment 4,1000 positioning anchor points are chosen, are ultimately generated
1000x25=25000 candidate frame；Two classification finally are carried out to this 25000 candidate frames using a sub-network（Two classification point
For background and target）, retain final target candidate frame.
Embodiment 6：
The complex road condition sensory perceptual system based on deep learning described in above-described embodiment, it is camera data collection first；In data
During collection, using DMA chips, directly by the data transfer of sensor collection into internal memory, it can so greatly reduce data biography
Defeated delay.
, it is necessary to be pre-processed to the data of collection after camera data is obtained, common processing has white balance, gamma
Correction and denoising.Corrected for white balance and gamma, ISP can be used（Image Signal
Processing）Handled.And for denoising, high speed filtering kernel function can be used to be handled, convolution kernel is for example attached
Fig. 1.
It is exactly that feature extraction is carried out to image using deep neural network after to pre-processing image data, in order to
Multiple dimensioned feature is extracted, it is necessary to design special network structure, and is finally merging these Analysis On Multi-scale Features, network structure
Such as accompanying drawing 2.
After feature extraction is complete, it is remaining just as classics faster-RCNN it is the same, positioned and identification at
Reason.
Embodiment 7：
A kind of complex road condition sensing device based on deep learning, its composition include:It is used for data acquisition using DMA chips
Camera 1, described camera are inserted in the slot 2 of bottom plate 21, and the side of described bottom plate is connected by screw the one of fixed strip 3
End, the other end connecting ring 4 of described fixed strip, the opposite side of described bottom plate are provided with screwed hole, and described screwed hole connects
Screw rod 5, described screw rod coupling hook 6 are connect, the described link of described hook mounting fixes described camera；Described bottom plate
Bottom buckle arc buckle 7, the both ends of described arc buckle are all connected with fixed plate 8, described bottom plate and described fixation
Fixed between plate by screw 9, described arc buckle fixes support bar 10, and a side end of described support bar connects left spiral shell
Bar 11, described left screw rod are sequentially connected with No.1 swivel nut 12 and No. two swivel nuts 13, described No.1 swivel nut connection No.1 supporting leg
14, described No. two swivel nuts connect No. two supporting legs 15, and the end side of described support bar connects right screw rod 16, described
Right screw rod is sequentially connected with No. three swivel nuts 17 and No. four swivel nuts 18, and described No. three swivel nuts connect No. three supporting legs 19, and described four
Number swivel nut connects No. four supporting legs 20, and described No.1 supporting leg separately supports described bottom plate with No. two described supporting legs,
No. three described supporting legs separately support described bottom plate with No. four described supporting legs.
Claims (6)
1. a kind of complex road condition sensory perceptual system based on deep learning, it is characterized in that:The system includes five steps：
The first step gathers vision sensor data；Second step pre-processes to view data, including white balance, gamma correction,
Denoising；3rd step carries out feature extraction using first deep neural network；4th step uses second depth nerve net
The feature of network extraction carries out target positioning；5th step uses carries out target knowledge using the 3rd deep neural network to localization region
Not, final testing result is obtained.
2. the complex road condition sensory perceptual system according to claim 1 based on deep learning, it is characterized in that:Described the 3rd
Step is to do feature multiplexing；The 4th described step is to separate foreground and background；It is described the 3rd step, the 4th described step, described
The 5th step in the Network integration that uses be a deeper deep neural network, the network conduct that each of which step uses
The sub-network of whole network is handled.
3. the complex road condition sensory perceptual system according to claim 2 based on deep learning, it is characterized in that:Described subnet
Network is a kind of multiple dimensioned neutral net, and feature extraction is carried out on different yardsticks, will finally by a special sub-network
Integrated without the feature under yardstick, obtain final Analysis On Multi-scale Features.
4. the complex road condition sensory perceptual system according to claim 3 based on deep learning, it is characterized in that:In Analysis On Multi-scale Features
It is exactly generation positioning candidate frame in next step after extraction；Using extra sub-network generation positioning anchor point, each anchor point is tied again
5 dimensional variations and 5 angular transformations are closed, ultimately generate 25 candidate frames.
5. the complex road condition sensory perceptual system according to claim 4 based on deep learning, it is characterized in that:1000 are chosen to determine
Position anchor point, ultimately generates 1000x25=25000 candidate frame；Finally this 25000 candidate frames are carried out using a sub-network
Two classification, retain final target candidate frame.
6. a kind of complex road condition sensing device based on deep learning, its composition includes:It is used for data acquisition using DMA chips
Camera, it is characterized in that:In the slot of described camera insertion bottom plate, the side of described bottom plate is connected by screw fixed strip
One end, the other end connecting ring of described fixed strip, the opposite side of described bottom plate is provided with screwed hole, described screwed hole
Connecting screw, described screw rod coupling hook, the described link of described hook mounting fix described camera；Described bottom plate
Bottom buckle arc buckle, the both ends of described arc buckle are all connected with fixed plate, described bottom plate and described fixed plate
Between fixed by screw, described arc buckle fixes support bar, and a side end of described support bar connects left screw rod, institute
The left screw rod stated is sequentially connected with No.1 swivel nut and No. two swivel nuts, described No.1 swivel nut connection No.1 supporting leg, described No. two
Swivel nut connects No. two supporting legs, and the end side of described support bar connects right screw rod, and described right screw rod is sequentially connected with three
Number swivel nut and No. four swivel nuts, described No. three swivel nuts connect No. three supporting legs, and described No. four swivel nuts connect No. four supporting legs, institute
The No.1 supporting leg stated separately supports described bottom plate, described No. three supporting legs and described four with No. two described supporting legs
Number supporting leg separately supports described bottom plate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710941720.1A CN107844746A (en) | 2017-10-11 | 2017-10-11 | Complex road condition sensory perceptual system and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710941720.1A CN107844746A (en) | 2017-10-11 | 2017-10-11 | Complex road condition sensory perceptual system and device based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107844746A true CN107844746A (en) | 2018-03-27 |
Family
ID=61662152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710941720.1A Pending CN107844746A (en) | 2017-10-11 | 2017-10-11 | Complex road condition sensory perceptual system and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107844746A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705370A (en) * | 2019-09-06 | 2020-01-17 | 中国平安财产保险股份有限公司 | Deep learning-based road condition identification method, device, equipment and storage medium |
| CN111652219A (en) * | 2020-06-03 | 2020-09-11 | 有米科技股份有限公司 | Image-text identification detection and identification method and device, server and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206130475U (en) * | 2016-11-08 | 2017-04-26 | 天津市电力科技发展有限公司 | Camera tripod with range finding function |
| CN107220703A (en) * | 2016-12-29 | 2017-09-29 | 恩泊泰（天津）科技有限公司 | A kind of deep neural network based on multiple scale detecting |
-
2017
- 2017-10-11 CN CN201710941720.1A patent/CN107844746A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206130475U (en) * | 2016-11-08 | 2017-04-26 | 天津市电力科技发展有限公司 | Camera tripod with range finding function |
| CN107220703A (en) * | 2016-12-29 | 2017-09-29 | 恩泊泰（天津）科技有限公司 | A kind of deep neural network based on multiple scale detecting |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705370A (en) * | 2019-09-06 | 2020-01-17 | 中国平安财产保险股份有限公司 | Deep learning-based road condition identification method, device, equipment and storage medium |
| CN110705370B (en) * | 2019-09-06 | 2023-08-18 | 中国平安财产保险股份有限公司 | Road condition identification method, device, equipment and storage medium based on deep learning |
| CN111652219A (en) * | 2020-06-03 | 2020-09-11 | 有米科技股份有限公司 | Image-text identification detection and identification method and device, server and storage medium |
| CN111652219B (en) * | 2020-06-03 | 2023-08-04 | 有米科技股份有限公司 | Image-text identification detection and identification method, device, server and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104036488B (en) | Binocular vision-based human body posture and action research method | |
| CN104034269B (en) | A kind of monocular vision measuring method and device | |
| CN110517306B (en) | Binocular depth vision estimation method and system based on deep learning | |
| CN103471523B (en) | A kind of detection method of arabidopsis profile phenotype | |
| CN104205822B (en) | Use the scene three-dimensional reconstruction method of asynchronous sensor | |
| CN106407875A (en) | Target feature extraction method and apparatus | |
| CN106296825A (en) | A kind of bionic three-dimensional information generating system and method | |
| CN105354856A (en) | Human matching and positioning method and system based on MSER and ORB | |
| CN106705849A (en) | Calibration method of linear-structure optical sensor | |
| CN113192646A (en) | Target detection model construction method and different target distance monitoring method and device | |
| CN108460794B (en) | Binocular three-dimensional infrared salient target detection method and system | |
| CN112880563B (en) | Single-dimensional pixel combination mode equivalent narrow-area-array camera spatial position measuring method | |
| CN113343925B (en) | Face three-dimensional reconstruction method and device, electronic equipment and storage medium | |
| CN104424383A (en) | Infrared image based hardware processing algorithm effectiveness performance evaluation device and method | |
| CN103438834A (en) | Hierarchy-type rapid three-dimensional measuring device and method based on structured light projection | |
| CN107844746A (en) | Complex road condition sensory perceptual system and device based on deep learning | |
| CN110096993A (en) | The object detection apparatus and method of binocular stereo vision | |
| CN103913149A (en) | Binocular range finding system based on STM 32 single chip microcomputer and range finding method thereof | |
| CN109360144A (en) | A kind of image real time correction improved method based on cell phone platform | |
| CN114998447A (en) | Multi-view vision calibration method and system | |
| CN103020997B (en) | Satellite target extraction method | |
| CN116935181B (en) | Three-dimensional measurement method for full binary speckle embedded pulse width modulation mode | |
| CN103278141A (en) | Infant sleep monitoring system and method thereof | |
| CN107767366A (en) | A kind of transmission line of electricity approximating method and device | |
| CN106127147B (en) | A kind of face depth texture restorative procedure based on three-dimensional data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |