CN109326168A - Prediction type drives computer aided simulation system - Google Patents
Prediction type drives computer aided simulation system Download PDFInfo
- Publication number
- CN109326168A CN109326168A CN201811483282.XA CN201811483282A CN109326168A CN 109326168 A CN109326168 A CN 109326168A CN 201811483282 A CN201811483282 A CN 201811483282A CN 109326168 A CN109326168 A CN 109326168A
- Authority
- CN
- China
- Prior art keywords
- image
- prediction
- module
- target
- core
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/04—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a kind of prediction types to drive computer aided simulation system.Real-time road image can accurately be obtained using the present invention and multipoint parallelization is handled.The technical scheme is that: Road quality simulation module acquires real-time road data using the optical camera on vehicle；Acquisition display front end controls camera by configurator in real time, and the Gao Siwei of each color is chosen after acquisition simulation road conditions as output image data, high five are converted in gray level image deposit FPGA memory；Image procossing and prediction module carry out conventional road conditions object matches to the gray level image in deposit FPGA, output is conventional to drive standby signal, by moving-target and background separation, recycle the mode for being embedded in multi-core parallel concurrent in soft core, it carries out curve fitting to multiple positions of target and simulates Future movement track, export superimposed image；Control simulation terminal, which exports speech prompt information and controls steering engine, comes conversion direction or control motor to control speed.
Description
Technical field
The driving computer aided simulation system with forecast function that the present invention relates to a kind of.
Background technique
In recent years, while car ownership is significantly increased, traffic accident also be can not be ignored, and human factor
The main reason for being traffic accident, such as fatigue driving, operation error all bring a series of serious consequences.Simultaneously because driving
That trains is irregular, and the limitation of driver attention amplifies out great number of issues, such as congested traffic, multiple traffic meaning
Outside, the operation of violation also creates the huge crowd of a collection of radix --- possess the new hand driver that driving license but dare not independently drive a vehicle,
Wherein female driver is especially in the majority.Majority causes fault more, timid, needs driving experience abundant old due to lacking driving experience
Driver accompanies guidance for a long time, or thoroughly says goodbye to driving career.Therefore auxiliary driving, safety instruction, traffic safety prediction and warning
The research of technology is extremely urgent, possesses the huge market demand.
Automatic Pilot development is burning hot, and the companies such as tesla, benz are proposed autonomous driving vehicle in foreign countries, and market is closed
Note degree is high.Current pilotless automobile forms comprehensive Internet of Things no also in research and development and test phase, but nearly ten years
It is public relatively low to its ability to accept for safety and habitual consideration before the unmanned environment of control.Especially
Does need to solve there are also how many hang-ups: how algorithm adapt to environment complicated and changeable for Chinese market? appearance accident power and responsibility
Do you how to assert? it is whether perfectly safe in terms of automatic Pilot technical security, accuracy rate? how is the not open automobile control of automobile vendor
Developed? does expensive cost adapt to Chinese market? these problems all imply automatic Pilot popularizing in China
Code was there are also 10 years to vicennial phase buffer, was driven with auxiliary to cut this period how not popularize also in automatic Pilot
Point, develops adaptation status, services new hand crowd, has itself selling point, moderate cost, towards ordinary populace and is suitble to
The system of verification algorithm, so that quickly seizing market is that follow-up developments layout needs to further investigate.
The similar system of traffic safety in the market only has simple prompt facility, such as run-off-road, lane change at present, cannot
It is effectively and rapidly captured after investigation to maneuvering target, and only for the fact that occurred in-situ processing, the reaction time is nervous.Simultaneously
It, only being capable of voice prompting due to the influence of vehicle control power.It is existing in order to increase the reaction time, can effectively simulating vehicle control
There is technology to occur handling the data information from sources such as DV, laser, radar and other inductors, to hold
The row such as automobile-used vision control system of the generic task of route departure prompt, drowsiness induction or parking auxiliary or blind-spot detection system
System.Certain processed information displays of automobile-used vision control system support function, such as parking auxiliary that can show on a display screen
Or it is broadcast out by audible alert signal.But current driver assistance system (ADAS), can be in complicated vehicle performance
Auxiliary and supplement are provided for driver in the process, is not realized finally unmanned.It is very multiple due to that can be faced in driving procedure
Miscellaneous traffic condition needs to carry out dynamic Control to automobile, also to keep the concern to environment surrounding automobile.How one kind is developed
The driving assistance system ADAS for being able to carry out the following automatic Pilot proof of algorithm is still the emphasis of research.Prediction type ADAS also only
The movement that vehicle can partially be controlled is prevented accident generation.Driving assistance system ADAS is to utilize vehicle-mounted all kinds of biographies
Sensor, the environment for incuding surrounding at any time while the car is driving collect data, carry out static, dynamic object identification, detect
It surveys and tracks, and navigation instrument map datum carries out the operation and analysis of system, to allow driver to perceive possibility in advance
The danger of generation effectively increases the comfortableness and security of car steering.Current existing some driving assistance systems both at home and abroad
Based on the fact that image is judged, threatens judgement to be mainly reflected in the aspect at a distance from itself to mobile, be easy by light, weather
It influences.Since current Driving control class is not yet completely open at home, even if wanting to carry out automatic Pilot, auxiliary driving class exploitation,
The not open automobile control of automobile vendor becomes maximum problem.Real vehicles and making for Driving Scene also determine its valuableness
Cost.External such as tesla, benz company are all directly to set about automatic Pilot, but people often have one kind for new things
The psychology of fear.Many beginners cause independently to drive because fearing mood.
Summary of the invention
The purpose of the present invention is for shortcoming existing for existing market, Driving control ginseng can be reduced by providing one kind
With reduce dangerous generation, increase the processing time, strong real-time, processing speed is fast, and the true prediction type of simulation effect drives auxiliary
Simulation system is helped, reduces vehicle enterprise intervention difficulty and research and development test cost in an analog manner.
In order to solve the above-mentioned technical problem, the present invention adopts the following technical scheme: a kind of prediction type drives computer aided simulation system
System, comprising: Road quality simulation module, the front end of acquisition display in real time, image procossing and prediction module and control simulation terminal, feature
Be: Road quality simulation module acquires using the optical camera being mounted in actual vehicle and stores the data of real-time road；It is real
When acquisition display front end by write camera configurator control COMS camera, acquisition Road quality simulation module it is being played on
Road condition data is simulated, the pixel data for being divided to two clocks to return to RGB565 format, the Gao Siwei for choosing each color connects as VGA
The output image data of mouth is converted to the exploitation that gray level image deposit has field programmable gate array FPGA chip for high five
It is further processed in board memory；Image procossing and prediction module pre-process simultaneously the gray level image of deposit in FPGA
Conventional road conditions object matches are carried out, the conventional standby signal that drives is exported according to matching result and carries out conventional road conditions object matches, together
When choose multiple image and carry out difference processing, threshold value is set using distance between target sizes, multiple target and movement velocity, by dynamic mesh
Mark and background separation, recycle the mode of multi-core parallel concurrent in the soft core of the nios of insertion, at the same to multiple behaviors for threatening targets into
Every trade is storehouse matching, exports corresponding standby signal according to matching result and stores dbjective state, to multiple positions of target into
Row curve matching simultaneously simulates Future movement track, and superimposed image is exported on VGA, to meeting same prediction process
The requirement of threshold time is kept, corresponding prediction standby signal is then exported according to matching result；Control simulation terminal is according to it
The priority of preceding all kinds of standby signals exports corresponding voice prompting sound, and the duty of Controlling model vehicle voltage change square wave
Than having the function that drive simulating so that controlling steering engine comes conversion direction or control motor to control speed.
The present invention has the following beneficial effects: compared with the prior art
Reduce Driving control participation, reduces dangerous generation.The Road quality simulation module that the present invention uses utilizes and is mounted on reality
Optical camera on the vehicle of border acquires and stores the data of real-time road, and the front end of acquisition display in real time is matched by writing camera
Process control COMS camera is set, to the road conditions image being acquired is playing, two clocks is divided to return to RGB565 format
Pixel data chooses output of the Gao Siwei as USB interface of each color, is converted to gray level image deposit for high five and has
It is further processed in the exploitation board memory of fpga chip, is then divided according to the alert level of prediction and carry out warning note,
Increase the customer responsiveness time, prompts user to shift to an earlier date note that reducing dangerous generation.The forecast function of introducing can increase processing
Time reduces dangerous generation.In the fact preceding look-ahead can occur for increased trend prediction function, following to deterrent
Direction of travel, curve, speed predicted, using template matching and frame difference technology, prompts current dangerous, using prediction algorithm,
The ability to predict to future risk is improved, voice and Mechanical course drive simulating supporting process are utilized.Driver can be reduced to exist
Participation in vehicle control plays guidance driving, automatic prompt and the effect controlled, can effectively reduce the work load of driver
And improve the convenience of manipulation and the safety of driving.
Strong real-time, processing speed are fast.The image procossing and prediction module that the present invention uses are in FPGA to the ash of deposit
Degree image is pre-processed and is carried out conventional road conditions object matches, exports conventional driving standby signal according to matching result.Simultaneously
It chooses multiple image and carries out difference processing, by moving-target and background separation, utilize distance between target sizes, multiple target and movement speed
Degree setting threshold value, recycle multi-core parallel concurrent mode simultaneously to it is multiple threaten targets behaviors carry out behavior storehouse matchings, according to
Corresponding standby signal is exported with result and is stored dbjective state, carry out curve fitting to multiple positions of target and is simulated not
Carry out motion profile, OpenCL is introduced into the kernel of FPGA insertion and carries out multi-core parallel concurrent acceleration: utilizing FPGA programming language itself
Parallel characteristics, and FPGA insertion the soft core of NIOS mode, using multi-core parallel concurrent processing by the way of, OpenCL standard is drawn
Enter in FPGA, there is stronger real-time.It realizes that a variety of parallel modes cooperate by exploitation, performance can be greatly improved,
Speed up processing.
It is true to simulate effect.Priority of the control simulation terminal that the present invention uses according to preceding all kinds of prompt informations, control
The duty ratio of simulation vehicle voltage change square wave is controlled steering engine in a manner of simulation system and comes conversion direction or control motor to control
Speed processed reduces system cost and experimentation cost, and simulation effect is close true, safer.By utilizing Road quality simulation
Module carries out real-time road simulation, can carry out algorithm checking computations and adjustment indoors, effect is close to really.Whole system can also be with
It is attached in actual vehicle, carries out driving aid prompting, while using the control approximate simulation vehicle control of model car, more pacifying
It is complete and low in cost.It is suitble to auxiliary to drive the early development and test of class product, provides transition for subsequent market.More meet mesh
The psychological needs of preceding masses.Hair, which opens driving computer aided simulation system, can allow user to break the dependence to experience, pass through voice
The mode of prompt allows driver to grasp correct crisis processing capacity, can be applied in the scenes such as driving school's training, allows public first
From auxiliary drive, even simulating vehicle control mode experience in gradually adapt to full-automatic driving mode.
Height emulates, and the following Mechanical course can reduce artificial participation.With reference to manually the mode of driving is accompanied, mentioned with voice
Show, can also experience vehicle control effect using simulator at home, height emulates, and to look forward to after being combined with vehicle, can be realized
Function for Automatic Pilot reduces artificial participate in.
The present invention sentences suitable for interior driving prompt system, driving school's training system, the following automated driving system, traffic violations
The scenes such as fixed.
Detailed description of the invention
For a clearer understanding of the present invention, now by specific embodiment through the invention, referring concurrently to attached drawing, to retouch
State the present invention, in which:
Fig. 1 is the schematic diagram that prediction type of the present invention drives computer aided simulation system.
Fig. 2 is the front-end process schematic diagram of acquisition display in real time in Fig. 1.
Fig. 3 is image procossing and the parallel algorithm flow diagram of prediction module in Fig. 1.
The present invention will be further explained below with reference to the attached drawings.
Specific embodiment
Refering to fig. 1.In the embodiment described below, a kind of driving computer aided simulation system with forecast function, comprising:
Road quality simulation module, the front end of acquisition display in real time, image procossing and prediction module and control simulation terminal.Wherein, Road quality simulation
Module hardware includes the VGA display screen of the singlechip control panel for connecting optical camera and broadcasting, singlechip control panel
Singlechip chip, memory module and USB interface are carried, in singlechip chip, camera shooting is write according to camera control protocol
Head configuration module.The front end of acquisition display in real time contains the input COMS camera of connection camera configuration module, carries 2
The FPGA control panel of memory modules, clock control circuit module and data interactive interface, camera configuration module are in FPGA core
According to the software program module of COMS camera control protocol file edit in piece.Image procossing and prediction module include to carry
Behavior library memory module, multiple memory modules, VGA output interface, standby signal output interface and clock control module and
The FPGA development board of fpga chip, conventional traveling matching algorithm module in fpga chip comprising software programming threaten target to mention
Take algoritic module and the soft core of Nios, in the soft core of Nios in the multicore processing control journey for meeting OpenCL standard comprising software programming
Sequence and behavior prediction algoritic module, the soft core of Nios handle control program using multicore and call behavior prediction algorithm mould come multidiameter delay
Block；Behavior prediction algoritic module carries out curve fitting according to multiple continuous positions of the threat target extracted, the prediction of generation
Image data is exported by VGA after geometric locus and original image pixels, while according to prediction locus curve and external behavior library
In behavior pattern matched, corresponding standby signal is exported after successful match.Conventional traveling matching algorithm module will acquire
Image carry out similarity mode with the object that is previously stored or traveling sign image, after matching with the progress behavior of behavior library
Match, exports corresponding indication signal；After threatening object extraction algorithm module that multiple image is made the difference simultaneously binary conversion treatment, extract
There is the target area of huge difference with conventional environment, impurity point is gone by the way of glide filter to target area, according to thresholding
Value, holds up to 3 threat targets, and final threat target area, coloured frame and primitive frame image are drawn a circle to approve in a manner of coloured frame
Image data is exported after pixel superposition；Control simulation terminal mainly includes to connect with speech chip and audio output interface, SD card
The voice development board of mouth, SD card, the microcontroller development board for Controlling model vehicle, model car and the sound equipment for being stored with audio file
Equipment, SD card connect voice development board by SD card interface, and voice development board will be prompted to information data by built-in speech chip
Send microcontroller development board to, microcontroller development board passes through built-in singlechip chip link model vehicle.
Road quality simulation module carries out real-time road condition acquiring using the optical camera being mounted in actual vehicle and stores, benefit
Configuration control is carried out with common single-chip microcontroller, has selection playing function, image data is shown on VGA screen as road conditions mould
Quasi- platform, simulates real-time road.
Acquisition display front end controls COMS camera by writing camera configurator in fpga chip in real time, to just
It is acquired in the road conditions image of broadcasting, the pixel data for being divided to two clocks to return to RGB565 format chooses the height of each color
Four output as USB interface, high five be converted to gray level image deposit with fpga chip exploitation board memory in carry out
It handles in next step, whole process is convenient faster to access by carrying out ping-pong operation to two panels memory, realizes most fast processing speed
Degree.The data of storage are sent into image procossing by data interaction interface and prediction module carries out subsequent processing.
Image procossing and prediction module mainly use parallel mode to realize multitask, the multiple programming language carried using FPGA
Sentence, and the soft core of NIOS is embedded in FPGA, multi-core parallel concurrent function is realized using OpenCL programming in soft core.When acquisition is shown
After image data is sent into front end, image procossing and prediction module complete multichannel task using the multiple programming sentence that FPGA is carried:
The model library write in the gray level image of the deposit of task all the way and program is subjected to conventional road conditions object matches, such as marks out lane
Line, road surface are advanced mark, traffic lights etc., are then matched with behavior library, export whether the conventional lines such as the direction of travel in inclined road
Into indication signal；Another way task accumulates multiple image, then carries out difference processing to the image of accumulation (conventional to choose 3 frames)
(difference processing method is to make the difference first frame and the second frame image to form a new images A, and the second frame and third frame are made the difference shape
At a new images B, then image A and image B are made the difference to obtain a final difference image C), to the difference after final process
Partial image carries out the processing of binaryzation using empirical value, goes by moving-target and background separation, then by the way of glide filter
After impurity point.Then impend Objective extraction, i.e., threshold value is arranged using distance between target sizes, multiple target and movement velocity,
Merge part moving-target, finally only holds up to 3 and maximum moving-target is threatened to ontology.Then FPGA parallel subqueries are utilized again
It realizes two-way parallel task simultaneously: being to be marked out moving-target to carry out coloured frame and original image using coloured frame all the way
After pixel superposition, final image information is shown on VGA.Another way is the multicore processing control journey called in the soft core of nios
Sequence, in the way of multiple software parallel cores of software programming, i.e. core 1, core 2.. core N it is each to one threaten the behavior of target into
Every trade is storehouse matching.To be each separated into again after corresponding threat target and behavior storehouse matching in each core two cores (such as
Core 11, core 12) parallel work-flow: corresponding standby signal, another way software are exported according to matching result in software parallel core all the way
Dbjective state is stored in Parallel Kernel, carries out state cumulative, if 5 continuous states for meeting time empirical thresholds of accumulation are identical,
Then prediction algorithm (multiple continuous state positions of target is taken to carry out curve fitting and simulate Future movement track) is called,
Then it is again broken down into two cores (such as core 121, core 122) parallel work-flow: the movement rail that will be predicted in software parallel core all the way
Trace curve is superimposed on the image data in a manner of colored line, is exported finally by USB interface to VGA display screen.Another way
Then the path curves status data predicted is stored in software parallel core, track is carried out more according to real-time stream
New adjustment (i.e. when 5 continuous states, when having new state to enter, forms 5 new companies after most start state is abandoned
Continuous state is handled), it 5 continuous states is all carried out to obtain with behavior library threatens behavior prediction as a result, if meeting wherein 3
A threat behavior prediction result is identical, then exports corresponding prediction and threaten matching result (i.e. corresponding prediction standby signal).Often
Core 1 all the way, core 2.. core N treatment process is identical, all exports corresponding prediction standby signal.Then by whole image processing and in advance
The standby signal exported in all treatment processes of module is surveyed preferentially to be determined according to time order and function, threat level progress output signal,
Standby signal is exported according to priority.
Control simulation terminal is then after configuring speech chip, according to the preferential of all kinds of prompt informations of previous module output
Sequentially (sequence mainly determines according to size variation, velocity variations, with the weighted values of the self-settinies such as ontology direction change,
Constantly update is adjusted in test), it calls corresponding mp3 file in SD card and plays out, reach prompt operation, auxiliary is driven
The effect sailed.All kinds of prompt informations are also prioritized to input microcontroller development board by microcontroller development board simultaneously, pass through control
Voltage changes the duty ratio of square wave, then by the voltage signal input model vehicle of variation, is turned by the steering engine of Controlling model vehicle
Direction or control motor are changed to control speed, has the function that drive simulating.
Refering to Fig. 2.Acquisition display front end controls COMS camera by writing camera configurator in real time, to broadcasting
The road conditions image put is acquired, and is stored and is sent after processing and is sent into subsequent image processing and prediction module in real time.System is opened
After beginning work, the front end of acquisition display in real time judges whether there is initializing signal, after pressing initialization key, referring to serial camera shooting
Head control bus SCCB agreement carries out camera register configuration, judges whether configuration successful, after complete configuration successful, FPGA
Chip reads camera data and control signal, and (camera module constantly returns to 8 data DATA, pixel clock PCLK, horizontal
Synchronization signal HSYNC, field sync signal VSYNC etc. signals), it then carries out image real time transfer and (is divided to two clocks will be without adding
The image of work RAW format is converted into RGB RGB565 format, i.e. two clocks amount to 16 data, and low 5 expressions are blue, in
Between 6 expressions green, high 5 expression red).Using the parallel subqueries of FPGA, while realizing two dataway operations: being all the way will be red green
Blue each color only retain the Gao Wuwei of itself digit and be converted to gray level image by the way of ping-pong operation in turn deposit open
In two memories for sending out plate, output is also to be read in a manner of ping-pong operation and currently deposit the picture number in several different memories
According to then by image data feeding subsequent image processing and prediction module progress subsequent processing.Another dataway operation is by RGB
Each color is exported as VGA image data to VGA display screen after only retaining high four.
Refering to Fig. 3.Image procossing and prediction module mainly use the parallel subqueries mode Parallel Implementation multitask of FPGA,
It is embedded in soft-core processor NIOS core in on-site programmable gate array FPGA, parses open operation language using full angle wherein
OpenCL realizes multi-core parallel concurrent function.After getting the image data sent at the front end of acquisition display in real time, simultaneously using FPGA
Line statement realizes two dataway operations simultaneously: task is that the model library write in gray level image and the program by being stored into carries out all the way
Conventional road conditions object matches, such as lane line, road surface traveling mark, traffic lights are marked out, then matched with behavior library, it is defeated
Out direction of travel, whether the conventional traveling indication signal such as inclined road.Another way task accumulates multiple image, then to the image of accumulation
(conventional to choose 3 frames) carries out difference processing, and (difference processing method is to make the difference first frame and the second frame image to form a new figure
As A, the second frame and third frame are made the difference to form a new images B, then by image A and image B make the difference to obtain one it is final
Difference image C), the processing of binaryzation is carried out using empirical value to the difference image after final process, by moving-target and background
Separation, then after removing impurity point by the way of glide filter.Then impend Objective extraction, that is, utilizes target sizes, more mesh
Threshold value is arranged in distance and movement velocity between mark, merges part moving-target, finally only holds up to 3 and threatens maximum move to ontology
Target.Then two-way parallel task is realized simultaneously using FPGA parallel subqueries again: being to be marked moving-target using coloured frame all the way
Out, after coloured frame being superimposed with original image progress pixel, final image information is shown on VGA.Another way is to call
Multicore processing control program in the soft core of nios, in the way of multiple software parallel cores of software programming, i.e. core 1, core 2.. core
N is each to threaten the behavior of target to carry out behavior storehouse matching to one.By corresponding threat target and behavior storehouse matching in each core
It is each separated into two cores (such as core 11, core 12) parallel work-flow again afterwards: being exported in software parallel core according to matching result all the way
Corresponding standby signal stores dbjective state in another way software parallel core, carries out state cumulative, if accumulation 5 meets
Between empirical thresholds continuous state it is identical, then call prediction algorithm (take target multiple continuous state positions carry out curve
It is fitted and simulates Future movement track), then it is again broken down into two cores (such as core 121, core 122) parallel work-flow: soft all the way
The path curves predicted are superimposed on the image data, finally by VGA in a manner of colored line in part Parallel Kernel
Interface is exported to VGA display screen.Then the path curves status data predicted is deposited in another way software parallel core
Storage carries out track according to real-time stream and updates adjustment (i.e. when 5 continuous states, when having new state to enter, by what is most started
One state forms 5 new continuous states after abandoning and is handled), an acquisition is all carried out with behavior library to 5 continuous states
Behavior prediction is threatened as a result, identical if meeting wherein 3 threats behavior prediction results, corresponding prediction is exported and threatens to match and tie
Fruit (i.e. corresponding prediction standby signal).Per core 1 all the way, core 2.. core N treatment process is identical, all exports corresponding prediction prompt
Signal.Then by the standby signal exported in whole image processing and all treatment processes of prediction module according to time order and function, prestige
Side of body grade carries out output signal and preferentially determines, exports standby signal according to priority.
Above in conjunction with attached drawing to the present invention have been described in detail, it is to be noted that being described in examples detailed above
Preferred embodiment only of the invention, is not intended to restrict the invention, and the invention may be variously modified and varied, for example takes the photograph
As head is not limited to CMOS camera, the camera for arbitrarily meeting required precision outside example can be, by writing configuration
File implements control.For in the selection of FPGA, be not limited to this example use FPGA development board, can use example with
The outer FPGA processing board for needing to meet logic gate, the FPGA development board of request memory or designed, designed.Parallel processing is not limited to
The soft core of NIOS is embedded in the FPGA that this example uses, it, can be using other than example in the method for wherein applying OpenCL standard
Other Embedded Soft Cores, multi-core CPU, the modes such as GPU.All within the spirits and principles of the present invention, it is made it is any modification,
Equivalent replacement, improvement etc., should be included within scope of the presently claimed invention.
Claims (10)
1. a kind of prediction type drives computer aided simulation system, comprising: Road quality simulation module, acquisition display in real time front end, image procossing
And terminal is simulated in prediction module and control, it is characterised in that: Road quality simulation module is taken the photograph using the optics being mounted in actual vehicle
As head acquires and stores the data of real-time road；Acquisition display front end is taken the photograph by writing camera configurator control COMS in real time
As head, Road quality simulation module simulation road condition data being played on is acquired, the pixel number for being divided to two clocks to return to RGB565 format
According to, choose output image data of the Gao Siwei as USB interface of each color, by high five be converted to gray level image deposit band
It is further processed in the exploitation board memory for having field programmable gate array FPGA chip；Image procossing and prediction module exist
Conventional road conditions object matches are pre-processed and carried out in FPGA to the gray level image of deposit, are driven according to matching result output is conventional
Standby signal is sailed, while choosing multiple image and carrying out difference processing, is set using distance between target sizes, multiple target and movement velocity
Threshold value is set, by moving-target and background separation, recycles the mode of multi-core parallel concurrent in the soft core of the nios of insertion, while to multiple threats
The behavior of target carries out behavior storehouse matching, exports corresponding standby signal according to matching result and stores dbjective state, to target
Multiple positions carry out curve fitting and simulate Future movement track, and superimposed image is exported on VGA, it is same to meeting
A kind of prediction process keeps the requirement of threshold time, then exports corresponding prediction standby signal according to matching result；Control mould
The priority of quasi- terminal all kinds of standby signals before, exports corresponding voice prompting sound, and Controlling model vehicle voltage becomes
Change the duty ratio of square wave, so that controlling steering engine comes conversion direction or control motor to control speed, has the function that drive simulating.
2. prediction type as claimed in claim drives computer aided simulation system, it is characterised in that: Road quality simulation module hardware includes,
It connects the singlechip control panel of optical camera and the VGA display screen of broadcasting, singlechip control panel carries monolithic machine core
Piece, memory module and USB interface write camera configuration module according to camera control protocol in singlechip chip.
3. prediction type as described in claim 1 drives computer aided simulation system, it is characterised in that: the front end of acquisition display in real time includes
The input COMS camera of connection camera configuration module carries 2 memory modules, clock control circuit module and data
The FPGA control panel of interactive interface, camera configuration module are to be compiled in fpga chip according to COMS camera control protocol file
The software program module write.
4. prediction type as described in claim 1 drives computer aided simulation system, it is characterised in that: image procossing and prediction module packet
Containing carrying behavior library memory module, multiple memory modules, VGA output interface, standby signal output interface and clock control mould
The FPGA development board of block and fpga chip, conventional traveling matching algorithm module in fpga chip comprising software programming threaten
Object extraction algorithm module and the soft core of Nios, the multicore processing control for meeting OpenCL standard in the soft core of Nios comprising software programming
Processing procedure sequence and behavior prediction algoritic module.
5. prediction type as claimed in claim 4 drives computer aided simulation system, it is characterised in that: the soft core of Nios is handled using multicore
Control program carrys out multidiameter delay and calls behavior prediction algoritic module；Behavior prediction algoritic module is according to the threat target extracted
Multiple continuous positions carry out curve fitting, and the prediction locus curve of generation is superimposed with original image pixels exports image by VGA
Data, while being matched according to prediction locus curve with the behavior pattern in external behavior library, it exports and corresponds to after successful match
Standby signal.
6. prediction type as claimed in claim 4 drives computer aided simulation system, it is characterised in that: conventional traveling matching algorithm module
The image that will acquire carries out similarity mode with the object or traveling sign image being previously stored, and after matching and behavior library carries out
Behavior matching, exports corresponding indication signal.
7. prediction type as claimed in claim drives computer aided simulation system, it is characterised in that: threaten object extraction algorithm module
After multiple image is made the difference simultaneously binary conversion treatment, the target area for having huge difference with conventional environment is extracted, to target area
Impurity point is removed by the way of glide filter, according to threshold value, is retained at most 3 threat targets, is drawn a circle to approve in a manner of coloured frame
Final to threaten target area, coloured frame exports image data after being superimposed with primitive frame image pixel.
8. prediction type as described in claim 1 drives computer aided simulation system, it is characterised in that: when figure is sent into acquisition display front end
As after data, image procossing and prediction module complete multichannel task using the multiple programming sentence that FPGA is carried: will task all the way
The model library write in the gray level image and program of deposit carries out conventional road conditions object matches, is then matched with behavior library,
Output whether the conventional traveling indication signal such as the direction of travel in inclined road；Another way task accumulates multiple image, to the image of accumulation
It at least chooses 3 frames and carries out difference processing, make the difference first frame and the second frame image to form a new images A, by the second frame and the
Three frames make the difference to form a new images B, then make the difference image A and image B to obtain a final difference image C, to final
Treated, and difference image carries out the processing of binaryzation using empirical value, by moving-target and background separation, then using sliding filter
After the mode of wave removes impurity point, extracts using distance between target sizes, multiple target and movement velocity setting threshold value and threaten target, close
And only retain at most 3 moving-targets threatened ontology after the moving-target of part；Then it is realized simultaneously using FPGA parallel subqueries again
Two-way parallel task: being to be marked out moving-target coloured frame is carried out pixel with original image and is superimposed using coloured frame all the way
Afterwards, final image information is shown on VGA；Another way calls the multicore processing control program in the soft core of nios, utilizes software
The multiple software parallel cores write threaten the behavior of target to carry out behavior storehouse matching each.
9. prediction type as claimed in claim 8 drives computer aided simulation system, it is characterised in that: will in each core of the soft core of nios
It is each separated into two core parallel work-flows again after corresponding threat target and behavior storehouse matching: basis in software parallel core all the way
Corresponding standby signal is exported with result, dbjective state is stored in another way software parallel core, state cumulative is carried out, calls simultaneously
Prediction algorithm takes multiple continuous state positions of target to carry out curve fitting, and simulates Future movement track, then divides again
At two core parallel work-flows: the path curves that will be predicted in software parallel core all the way are superimposed in a manner of colored line
On the image data, it exports finally by USB interface to VGA display screen；The movement that another way software parallel core will then predict
Geometric locus status data is stored, and is carried out track according to real-time stream and is updated adjustment, will most when having new state to enter
The state started forms 5 new continuous states after abandoning and is handled, and all carries out with behavior library to 5 continuous states
, it obtains and threatens behavior prediction as a result, identical if meeting wherein 3 threats behavior prediction results, export corresponding every core all the way
It handles obtained prediction and threatens matching result；The standby signal exported in whole image processing and all treatment processes of prediction module
Output signal is carried out according to time order and function, threat level preferentially to determine, standby signal is exported according to priority.
10. prediction type as claimed in claim drives computer aided simulation system, it is characterised in that: after system work, acquire in real time
Display front end judges whether there is initializing signal, after initialization, is imaged referring to serial camera control bus SCCB agreement
Head register configuration, judges whether configuration successful, and after complete configuration successful, fpga chip reads camera data and control letter
Number, camera module constantly returns to 8 data DATA, pixel clock PCLK, horizontal synchronizing signal HSYNC and field sync signal
Then VSYNC carries out image real time transfer, be divided to two clocks, converts RGB for the image of not processed RAW format
RGB565 format；It using the parallel subqueries of FPGA, while realizing two dataway operations: RGB each color only being retained itself all the way
The Gao Wuwei of digit is simultaneously converted to gray level image, is stored in turn in two memories of development board by the way of ping-pong operation, defeated
It reads out and in a manner of ping-pong operation and currently deposits the image data in several different memories, be then sent into image data
Subsequent image processing and prediction module carry out subsequent processing, and another dataway operation is after RGB each color only to be retained to high four
It exports as VGA image data to VGA display screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811483282.XA CN109326168B (en) | 2018-12-06 | 2018-12-06 | Predictive driving assistance simulation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811483282.XA CN109326168B (en) | 2018-12-06 | 2018-12-06 | Predictive driving assistance simulation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109326168A true CN109326168A (en) | 2019-02-12 |
| CN109326168B CN109326168B (en) | 2020-12-15 |
Family
ID=65256338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811483282.XA Active CN109326168B (en) | 2018-12-06 | 2018-12-06 | Predictive driving assistance simulation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109326168B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109910790A (en) * | 2019-03-05 | 2019-06-21 | 同济大学 | A kind of ADAS domain controller |
| CN110284426A (en) * | 2019-06-25 | 2019-09-27 | 衡橡科技股份有限公司 | A kind of bridge inspection vehicle frequency-changing control system and its control method |
| CN110780608A (en) * | 2019-11-26 | 2020-02-11 | 北京百度网讯科技有限公司 | Simulation test method and device |
| CN113642372A (en) * | 2020-04-27 | 2021-11-12 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray-scale image in operation of autonomous driving vehicle |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101641248A (en) * | 2007-03-27 | 2010-02-03 | 丰田自动车株式会社 | Collision avoidance device |
| CN101870292A (en) * | 2009-04-22 | 2010-10-27 | 爱信艾达株式会社 | Drive assistance device, driving assistance method and driving auxiliary routine |
| US20110144859A1 (en) * | 2009-12-14 | 2011-06-16 | Electronics And Telecommunications Research Institue | Apparatus and method for preventing collision of vehicle |
| CN102137252A (en) * | 2011-02-28 | 2011-07-27 | 兰州大学 | Vehicle-mounted virtual panoramic display device |
| CN102353954A (en) * | 2011-06-30 | 2012-02-15 | 桂林电子科技大学 | Linear frequency-modulated continuous wave automobile anti-collision radar system and using method thereof |
| CN102663758A (en) * | 2012-04-20 | 2012-09-12 | 北京工业大学 | Image acquiring and processing method based on FPGA (field programmable gate array) serving as control core |
| CN104786933A (en) * | 2015-03-04 | 2015-07-22 | 江苏大学 | Panoramic image driving auxiliary device and panoramic image driving auxiliary method |
| CN106167045A (en) * | 2015-05-21 | 2016-11-30 | Lg电子株式会社 | Human pilot auxiliary device and control method thereof |
| CN206249643U (en) * | 2016-09-09 | 2017-06-13 | 山西省交通科学研究院 | A kind of vehicle blind zone monitoring and warning onboard system |
| CN207924502U (en) * | 2018-03-23 | 2018-09-28 | 南京越博动力***股份有限公司 | A kind of ADAS drive assistance functions using camera test system |
-
2018
- 2018-12-06 CN CN201811483282.XA patent/CN109326168B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101641248A (en) * | 2007-03-27 | 2010-02-03 | 丰田自动车株式会社 | Collision avoidance device |
| CN101870292A (en) * | 2009-04-22 | 2010-10-27 | 爱信艾达株式会社 | Drive assistance device, driving assistance method and driving auxiliary routine |
| US20110144859A1 (en) * | 2009-12-14 | 2011-06-16 | Electronics And Telecommunications Research Institue | Apparatus and method for preventing collision of vehicle |
| CN102137252A (en) * | 2011-02-28 | 2011-07-27 | 兰州大学 | Vehicle-mounted virtual panoramic display device |
| CN102353954A (en) * | 2011-06-30 | 2012-02-15 | 桂林电子科技大学 | Linear frequency-modulated continuous wave automobile anti-collision radar system and using method thereof |
| CN102663758A (en) * | 2012-04-20 | 2012-09-12 | 北京工业大学 | Image acquiring and processing method based on FPGA (field programmable gate array) serving as control core |
| CN104786933A (en) * | 2015-03-04 | 2015-07-22 | 江苏大学 | Panoramic image driving auxiliary device and panoramic image driving auxiliary method |
| CN106167045A (en) * | 2015-05-21 | 2016-11-30 | Lg电子株式会社 | Human pilot auxiliary device and control method thereof |
| CN206249643U (en) * | 2016-09-09 | 2017-06-13 | 山西省交通科学研究院 | A kind of vehicle blind zone monitoring and warning onboard system |
| CN207924502U (en) * | 2018-03-23 | 2018-09-28 | 南京越博动力***股份有限公司 | A kind of ADAS drive assistance functions using camera test system |
Non-Patent Citations (2)
| Title |
|---|
| 毛亮: "基于FPGA的车道偏移告警***", 《信息技术》 * |
| 黄海波: "基于Nios Ⅱ的辅助驾驶视频监控***", 《电视技术》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109910790A (en) * | 2019-03-05 | 2019-06-21 | 同济大学 | A kind of ADAS domain controller |
| CN109910790B (en) * | 2019-03-05 | 2021-11-09 | 同济大学 | ADAS domain controller |
| CN110284426A (en) * | 2019-06-25 | 2019-09-27 | 衡橡科技股份有限公司 | A kind of bridge inspection vehicle frequency-changing control system and its control method |
| CN110284426B (en) * | 2019-06-25 | 2021-07-09 | 衡橡科技股份有限公司 | Bridge inspection vehicle frequency conversion control system and control method thereof |
| CN110780608A (en) * | 2019-11-26 | 2020-02-11 | 北京百度网讯科技有限公司 | Simulation test method and device |
| CN110780608B (en) * | 2019-11-26 | 2023-03-10 | 北京百度网讯科技有限公司 | Simulation test method and device |
| CN113642372A (en) * | 2020-04-27 | 2021-11-12 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray-scale image in operation of autonomous driving vehicle |
| CN113642372B (en) * | 2020-04-27 | 2024-02-20 | 百度（美国）有限责任公司 | Method and system for recognizing object based on gray image in operation of autonomous driving vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109326168B (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109326168A (en) | Prediction type drives computer aided simulation system | |
| Chen et al. | Learning by cheating | |
| CN105632293B (en) | Intelligent coach vehicle assisted teaching system and its application method | |
| Ramanishka et al. | Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning | |
| CN108549366B (en) | Intelligent automobile road driving and virtual test parallel mapping experimental method | |
| Che et al. | D $^ 2$-city: a large-scale dashcam video dataset of diverse traffic scenarios | |
| US10901416B2 (en) | Scene creation system for autonomous vehicles and methods thereof | |
| CN205847406U (en) | A kind of ADAS test of heuristics device based on forward sight photographic head | |
| CN102541061B (en) | Micro intelligent vehicle based on visual and auditory information | |
| CN107506830A (en) | Towards the artificial intelligence training platform of intelligent automobile programmed decision-making module | |
| CN105654808A (en) | Intelligent training system for vehicle driver based on actual vehicle | |
| CN109562763A (en) | The control method and control device of automatic driving vehicle | |
| US20050275717A1 (en) | Method and apparatus for testing stereo vision methods using stereo imagery data | |
| CN113688652A (en) | Method and device for processing abnormal driving behaviors | |
| CN112258841B (en) | Intelligent vehicle risk assessment method based on vehicle track prediction | |
| CN107745710A (en) | A kind of automatic parking method and system based on machine vision and machine learning | |
| CN104875681A (en) | Dynamic vehicle-mounted camera control method based on application scenarios | |
| CN110363093A (en) | A kind of driver's action identification method and device | |
| CN108171134A (en) | A kind of operational motion discrimination method and device | |
| CN105261259A (en) | Intelligent vehicle driving training system | |
| CN109345861A (en) | Unmanned plane dispatching method, unmanned plane and unmanned plane cluster | |
| CN104134350A (en) | Intelligent dome camera system for traffic violation behavior recognition | |
| CN110930811B (en) | System suitable for unmanned decision learning and training | |
| CN109410608B (en) | Picture self-learning traffic signal control method based on convolutional neural network | |
| CN205665919U (en) | Intelligence learner -driven vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |