CN109741456A - 3D based on GPU concurrent operation looks around vehicle assistant drive method and system - Google Patents
3D based on GPU concurrent operation looks around vehicle assistant drive method and system Download PDFInfo
- Publication number
- CN109741456A CN109741456A CN201811542584.XA CN201811542584A CN109741456A CN 109741456 A CN109741456 A CN 109741456A CN 201811542584 A CN201811542584 A CN 201811542584A CN 109741456 A CN109741456 A CN 109741456A
- Authority
- CN
- China
- Prior art keywords
- texture image
- concurrent operation
- texture
- gpu
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013507 mapping Methods 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 13
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 13
- 238000005286 illumination Methods 0.000 claims description 26
- 230000009466 transformation Effects 0.000 claims description 21
- 241000208340 Araliaceae Species 0.000 claims description 20
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims description 20
- 235000003140 Panax quinquefolius Nutrition 0.000 claims description 20
- 235000008434 ginseng Nutrition 0.000 claims description 20
- 238000009877 rendering Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 10
- 238000013461 design Methods 0.000 claims description 8
- 238000000844 transformation Methods 0.000 claims description 6
- 238000009434 installation Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 5
- 239000011159 matrix material Substances 0.000 description 21
- 230000000007 visual effect Effects 0.000 description 14
- 238000003384 imaging method Methods 0.000 description 13
- 238000012937 correction Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000004744 fabric Substances 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 241000251468 Actinopterygii Species 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000002594 fluoroscopy Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000004215 lattice model Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The present invention discloses a kind of 3D based on GPU concurrent operation and looks around vehicle assistant drive method and system, this method is applied to the 3D based on GPU concurrent operation and looks around vehicle DAS (Driver Assistant System), method includes the following steps: the real-time acquired image of each camera is mapped in a manner of texture on pre-generated 3D grid model, corresponding texture image is obtained；According to 3D grid model, pre-generated mask plate, texture image is handled in GPU concurrent operation method, the texture image that obtains that treated, all coordinate informations of treated texture image contains texture image；Correspondence mappings relationship after acquisition processing on the texture coordinate value of texture image and 3D grid model between the coordinate value of grid vertex spatial position；It according to 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.Compared with the existing technology, the present invention improves automobile assistant driving intelligence, promotes drive safety.
Description
Technical field
It is auxiliary the present invention relates to assisting driving technology field more particularly to a kind of 3D based on GPU concurrent operation to look around vehicle
Help drive manner and system.
Background technique
Automobile is convenient as modern society, and fast, the efficient vehicles are used by more and more people, but due to
The use of the complexity of driving environment and the level difference of driver, automobile also brings many problems.
The automobile storage of traditional only sided mirror unit is easy to cause safety problem, is in the inherent shortcoming of limited view
This hidden danger is solved, research staff proposes a series of scheme:
Scheme one installs single rearview camera in automobile tail, returns and show on operator seat related device and clapped
Take the photograph image.This scheme can make driver obtain automobile tail scene, but cannot eliminate the blind area of A column, B column and bonnet.
Scheme two, the two-dimensional display system of four-way CCD camera.This method is taken the photograph by all around respectively installing one in automobile
The scene of vehicle body surrounding is acquired as head, then corrects the image that each road acquires, and after perspective projection, synthesizes one with vapour
Vehicle geometric center is the roof top view of origin.The program enhances perception of the driver to vehicle body surrounding to a certain extent,
But it is small that there are visual fields, distorts serious at field of view edge, does not meet the habit of human vision, compare be suitble to park and run at a low speed with
It and is not very high situation to field range requirement.
Scheme three, four common tunnels acquire three-dimensional display system, such as patent " the vehicle multi-angle panorama for assisting driving
Generation method ", " one kind is based on for patent " a kind of panorama auxiliary parking system, device and panoramic image display method " and patent
The vehicle-mounted of OPENGL looks around multi-angle of view panorama generation method ".These methods are by four-way CCD camera acquired image, by normalizing
It is mapped in a manner of texture on fixed threedimensional model again after changing to the same coordinate system, is rendered into screen finally by OpenGL
It goes up and shows.The program expands the visual field, and driver is made to have certain feeling of immersion, but there are model fixation can not on-line tuning,
Under computational efficiency is relatively low, parallel computation is insufficient, and video is less smooth, and there is no the potential for playing GPU, is not easy to other similar
The defects of system integration of pedestrian detection.
Summary of the invention
It is a primary object of the present invention to propose a kind of 3D based on GPU concurrent operation look around vehicle assistant drive method and
System, it is intended to preferably play the potential of GPU parallel computation, boosting algorithm processing speed reduces model seam crossing and outside
More multi-functional, raising automobile assistant driving intelligence is merged in the distortion of edge.
To achieve the above object, the present invention provides a kind of 3D based on GPU concurrent operation and looks around vehicle assistant drive method,
The method is applied to the 3D based on GPU concurrent operation and looks around vehicle DAS (Driver Assistant System), is equipped on the vehicle for adopting
Several cameras for collecting vehicle-periphery image, the described method comprises the following steps:
The real-time acquired image of each camera is mapped in a manner of texture on pre-generated 3D grid model,
Obtain corresponding texture image；
According to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image into
Row processing, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；
Obtain after the processing grid vertex spatial position coordinate in the texture coordinate value of texture image and 3D grid model
Correspondence mappings relationship between value；
It according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.
Further technical solution of the invention is, described according to the 3D grid model, pre-generated mask plate, with
The step of GPU concurrent operation method handles the texture image, the texture image that obtains that treated include:
According to the 3D grid model, pre-generated mask plate, two neighboring camera collected figure in real time is obtained
The vision area-of-interest of the overlapping region of picture；
According to the vision area-of-interest, using GPU concurrent operation, while the OpenCL kernel function of design, knot are utilized
The exposure compensating coefficient of the overlapping region is calculated in the rgb value for closing the real-time acquired image of the camera；
Illumination compensation is carried out to each texture image according to the exposure compensating coefficient, the texture maps after obtaining illumination compensation
Picture；
Texture image after loading each illumination compensation does the position coordinates of the texture image after each illumination compensation and reflects
Penetrate transformation, the position coordinates after obtaining mapping transformation；
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established, simultaneously using GPU
Row operation handles the texture image, the texture image that obtains that treated.
Further technical solution of the invention is, described according to the 3D grid vertex and include that all coordinates are believed
The step of Texture image synthesis 3D of breath looks around roaming system include:
It by the 3D grid vertex and include that the texture image of all coordinate informations is sent to the GPU rendering of OpenGL
On pipeline, generates 3D and look around roaming system.
Further technical solution of the invention is, it is described by the real-time acquired image of each camera with the side of texture
Include: before the step of formula is mapped on pre-generated 3D grid model, obtains corresponding texture image
Obtain in advance each camera internal reference and outer ginseng；
Mask plate is generated according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
Further technical solution of the invention is, it is described by the real-time acquired image of each camera with the side of texture
Before the step of formula is mapped on pre-generated 3D grid model, obtains corresponding texture image further include:
Pre-generated 3D grid model.
To achieve the above object, the present invention also proposes that a kind of 3D based on GPU concurrent operation looks around vehicle assistant drive system
System, the system comprises several cameras being installed on vehicle, memory, processors, and are stored on the memory
The 3D based on GPU concurrent operation look around vehicle assistant drive program, the 3D based on GPU concurrent operation looks around vehicle auxiliary
Piloting procedure executes following steps when being run by the processor:
The real-time acquired image of each camera is mapped in a manner of texture on pre-generated 3D grid model,
Obtain corresponding texture image；
According to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image into
Row processing, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；
Obtain after the processing grid vertex spatial position coordinate in the texture coordinate value of texture image and 3D grid model
Correspondence mappings relationship between value；
It according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.
Further technical solution of the invention is that the 3D based on GPU concurrent operation looks around vehicle assistant drive journey
Sequence also executes following steps when being run by the processor:
According to the 3D grid model, pre-generated mask plate, two neighboring camera collected figure in real time is obtained
The vision area-of-interest of the overlapping region of picture；
According to the vision area-of-interest, using GPU concurrent operation, while the OpenCL kernel function of design, knot are utilized
The exposure compensating coefficient of the overlapping region is calculated in the rgb value for closing the real-time acquired image of the camera；
Illumination compensation is carried out to each texture image according to the exposure compensating coefficient, the texture maps after obtaining illumination compensation
Picture；
Texture image after loading each illumination compensation does the position coordinates of the texture image after each illumination compensation and reflects
Penetrate transformation, the position coordinates after obtaining mapping transformation；
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established, simultaneously using GPU
Row operation handles the texture image, the texture image that obtains that treated.
Further technical solution of the invention is that the 3D based on GPU concurrent operation looks around vehicle assistant drive journey
Sequence also executes following steps when being run by the processor:
It by the 3D grid vertex and include that the texture image of all coordinate informations is sent to the GPU rendering of OpenGL
On pipeline, generates 3D and look around roaming system.
Further technical solution of the invention is that the 3D based on GPU concurrent operation looks around vehicle assistant drive journey
Sequence also executes following steps when being run by the processor:
Obtain in advance each camera internal reference and outer ginseng；
Mask plate is generated according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
Further technical solution of the invention is that the 3D based on GPU concurrent operation looks around vehicle assistant drive journey
Sequence also executes following steps when being run by the processor:
Pre-generated 3D grid model.
Look around vehicle assistant drive method and system the present invention is based on the 3D of GPU concurrent operation through the above technical solutions,
The real-time acquired image of each camera is mapped on pre-generated 3D grid model in a manner of texture, is corresponded to
Texture image；According to the 3D grid model, pre-generated mask plate, in GPU concurrent operation method to the texture maps
As being handled, the texture image that obtains that treated, all coordinates letter of treated texture image the contains texture image
Breath；It obtains after the processing in the texture coordinate value of texture image and 3D grid model between the coordinate value of grid vertex spatial position
Correspondence mappings relationship；According to the 3D grid vertex and include all coordinate informations Texture image synthesis 3D look around it is unrestrained
Trip system preferably plays the potential of GPU parallel computation compared with the existing technology, and boosting algorithm processing speed reduces model and connects
It at seam and outer peripheral distortion, merges more multi-functional, to improve automobile assistant driving intelligence, promotes driving safety
Property.
Detailed description of the invention
Fig. 1 is to look around the process of vehicle assistant drive method first embodiment the present invention is based on the 3D of GPU concurrent operation to show
It is intended to；
Fig. 2 is the refinement flow diagram of step S20 in first embodiment shown in FIG. 1；
Fig. 3 is to look around the process of vehicle assistant drive method second embodiment the present invention is based on the 3D of GPU concurrent operation to show
It is intended to；
Fig. 4 is the system building flow chart that the 3D based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System)；
Fig. 5 is the system structure figures of outer ginseng measurement.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
In order to solve existing vehicle DAS (Driver Assistant System), there are model fixations cannot be adjusted, and does not play the potential of GPU, counts
The defects of calculation inefficiency, parallel computation is insufficient, and video is not smooth, is not easy to the system integration of other similar pedestrian detection, this
Invention proposes that a kind of 3D based on GPU concurrent operation looks around vehicle assistant drive method and system, parallel preferably to play GPU
The potential of calculating, boosting algorithm processing speed reduce model seam crossing and outer peripheral distortion, merge more multi-functional, raising
Automobile assistant driving is intelligent, promotes drive safety.
Fig. 1 is please referred to, Fig. 1 is that the implementation of vehicle assistant drive method first is looked around the present invention is based on the 3D of GPU concurrent operation
The flow diagram of example.
It is applied to be based on it should be noted that looking around vehicle assistant drive method the present invention is based on the 3D of GPU concurrent operation
The 3D of GPU concurrent operation looks around vehicle DAS (Driver Assistant System), is equipped on the vehicle for acquiring vehicle environmental view picture
Several cameras.
As shown in Figure 1, the present embodiment propose the 3D based on GPU concurrent operation look around vehicle assistant drive method include with
Lower step:
The real-time acquired image of each camera is mapped to pre-generated 3D net by step S10 in a manner of texture
On lattice model, corresponding texture image is obtained.
Step S20, according to the 3D grid model, pre-generated mask plate, in GPU concurrent operation method to the line
Reason image is handled, the texture image that obtains that treated, all seats of treated texture image the contains texture image
Mark information.
Step S30 obtains after the processing grid vertex space in the texture coordinate value of texture image and 3D grid model
Correspondence mappings relationship between position coordinate value.
Step S40 according to the 3D grid vertex and includes that the Texture image synthesis 3D of all coordinate informations is looked around
Roaming system.
When it is implemented, by the 3D grid vertex and can include that the texture images of all coordinate informations is sent to
On the GPU rendering pipeline of OpenGL, generates 3D and look around roaming system.
Referring to figure 2., Fig. 2 is step S20 in the present embodiment, according to the 3D grid model, pre-generated mask plate,
The texture image is handled in GPU concurrent operation method, the refinement flow diagram for the texture image that obtains that treated.
As shown in Fig. 2, above-mentioned steps S20 specifically includes the following steps:
Step S201 obtains two neighboring camera and adopts in real time according to the 3D grid model, pre-generated mask plate
The vision area-of-interest of the overlapping region of the image collected.
Step S202 using GPU concurrent operation, while utilizing the OpenCL of design according to the vision area-of-interest
The exposure compensating system of the overlapping region is calculated in conjunction with the rgb value of the real-time acquired image of the camera in kernel function
Number.
Step S203 carries out illumination compensation to each texture image according to the exposure compensating coefficient, obtains illumination compensation
Texture image afterwards.
Step S204, the texture image after loading each illumination compensation, to the position of the texture image after each illumination compensation
It sets coordinate and does mapping transformation, the position coordinates after obtaining mapping transformation.
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established by step S205,
The texture image is handled using GPU concurrent operation, the texture image that obtains that treated.
The present embodiment is through the above technical solutions, by the real-time acquired image of each camera with the side of texture as a result,
Formula is mapped on pre-generated 3D grid model, obtains corresponding texture image, according to the 3D grid model, is pre-generated
Mask plate, the texture image is handled in GPU concurrent operation method, obtain that treated texture image, the place
Texture image after reason contains all coordinate informations of texture image, obtain after the processing texture coordinate value of texture image with
Correspondence mappings relationship on 3D grid model between the coordinate value of grid vertex spatial position, according to the 3D grid vertex and packet
Texture image synthesis 3D containing all coordinate informations looks around roaming system, the traditional arithmetic speed looked around of effective solution
It is lower, it is unfavorable for integrating other modules, autonomy-oriented is intelligent insufficient, and 3d scene generates fixes, and not can solve traditional mould
It is stretched seriously at type seam and edge of model, vehicle driver and passenger's visual experience is poor, and the disadvantage of feeling of immersion deficiency effectively mentions
High vehicle drive safety, comfort, and it is intelligent.
Referring to figure 3., Fig. 3 is that the implementation of vehicle assistant drive method second is looked around the present invention is based on the 3D of GPU concurrent operation
The flow diagram of example.
The present embodiment and the difference of first embodiment shown in FIG. 1 are that above-mentioned steps S10 is real-time by each camera
The step of acquired image is mapped on pre-generated 3D grid model in a manner of texture, obtains corresponding texture image
Include: before
Step S101, obtain in advance each camera internal reference and outer ginseng.
Step S102 generates mask plate according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
In addition, in the present embodiment, above-mentioned steps S10, by the real-time acquired image of each camera in a manner of texture
Can also include: before the step of being mapped on pre-generated 3D grid model, obtaining corresponding texture image
Step S103 pre-generates 3D grid model.
It should be noted that step S103 can both be completed before step S101, it can also be complete after step s 102
At the present embodiment is not construed as limiting this.
It does into one below with reference to Fig. 4 and Fig. 5 to looking around vehicle assistant drive method the present invention is based on the 3D of GPU concurrent operation
Step elaborates.
About the present invention is based on the brief introductions and process that the 3D of GPU concurrent operation looks around vehicle assistant drive method:
Camera is installed in vehicle's surroundings certain position, for acquiring environment surrounding automobile image in real time, as walked in attached drawing 4
Suddenly 1. shown.
To the imaging device that selected four cameras are constituted, determine camera internal reference and outer ginseng.Wherein, it images
Head internal reference is mainly used for correcting the fault image that camera obtains, and joins outside camera and is mainly used for establishing camera part world seat
Certain in mark system puts coordinate and its mapping relations on camera photosurface between corresponding pixel coordinate, such as step in attached drawing 4
2. 3. shown with step.
The internal reference correcting image obtained using previous step, the part of each camera is determined according to the installation site of camera
Relationship between world coordinate system and normalized unified world coordinate system, is mapped to the normalized world for the image after correction
In coordinate system, the mapping relations of corresponding points coordinate in the pixel coordinate and world coordinate system of direct acquired image are determined, such as
Step is 4. shown in attached drawing 4.
The installation site of camera determines that each adjacent camera acquired image has the scene of coincidence, according to upper one
The relationship for walking determining image and world coordinate system, determines and is overlapped coordinate range, and accordingly generate mask plate, with mark acquisition figure
As in content whether belong to overlapping region, as in attached drawing 4 step 5. shown in.
In order to make driver and passenger have enough feeling of immersion, the present invention relatively independently, reasonably design and generates 3D grid
Model, so that each camera acquired image to be mapped on grid in a manner of texture, such as step 6. institute in attached drawing 4
Show.
It should be noted that above step, i.e. step in respective figure 4 2. all can offline really to the content of step 6.
It is fixed, it once determination, can be loaded, not needed treated in real time by way of importing later.
By above step, the scene information by 360 degree of visual fields that camera acquires is needed to be shown to driver and passenger.Load
The 3D gridding information generated loads mask plate, determines the vision area-of-interest (ROI) of overlapping region, is illumination compensation
Prepare, as in attached drawing 4 step 7., step 8. shown in.
Hereafter, using ROI obtained in the previous step, GPU concurrent operation unit and the kernel function of OpenCL is designed, is utilized
The algorithm of design, the region parallel computation exposure compensating coefficient that four are overlapped, and by obtained coefficient acting in entire texture
Image so that the brightness of each image have uniformity, as in attached drawing 4 step 9. shown in.
Hereafter, relevant information, the correlation of pedestrian in such as lane that other Parallel Units obtain in Same Scene are loaded
Information, and mapping transformation is done to position coordinates therein, these coordinates are all included under unified normalized coordinate and are managed, it is such as attached
In Fig. 4 step 10., stepStepIt is shown；
Hereafter, it generates comprising various coordinate informations (such as characteristic point coordinate, lane information coordinate, pedestrian information coordinate)
Texture, determines the coordinate value of its texture on the 3D grid vertex of generation, and regards its display effect, adjusts the coordinate value of texture,
So that the acquisition changing image as texture has preferable display effect on three-dimensional grid textures, such as 4 step of attached drawingIt is shown.
Finally by obtained 3D grid vertex (three-dimensional space position coordinate information, texture coordinate information including vertex) with
And texture image is pushed on the GPU rendering pipeline of OpenGL, to formed an optimal automotive integrated information 3D look around it is unrestrained
Trip system, such as step in attached drawing 4StepIt is shown.
Above content is described in detail below:
1, about Image Acquisition.Four tunnels big visual field (flake) camera is installed on vehicle all around, when installation needs to protect
There is common scene in the adjacent camera visual field of the road Zheng Ge.The content item correspond to Fig. 4 in step 1..
2, about camera internal reference, the acquisition of outer ginseng.Before how introduction obtains camera internal reference and outer ginseng, first briefly
Introduce the physical model of optical imagery.
Light reaches eyes or camera since object emission, through lens, then reaches retina or Image Acquisition
This imaging process of device is there are many illustrating, here, we are described using classical " pin-hole imaging model ".
It is different from ideal pin-hole imaging model, in imaging process, " focal length " of cam lens itself, " offset ", with
And " distortion " etc. can produce bigger effect the light entered in camera lens, we are referred to as these parameters the internal reference of camera.
The internal reference of camera has determined projection relation of the camera from three-dimensional space to two dimensional image.
Camera " outer ginseng " is influenced by the installation site of camera and towards (such as azimuth, pitch angle and roll angle),
" outer ginseng " determines the relationship between camera coordinate and world coordinates, relatively independent with " internal reference ".
Internal reference can be obtained by way of asking for pick-up lens manufacturer, also can off-line measurement.Outer participation lens distortion
It is relatively independent, change with application scenarios, must generally be obtained in application by demarcating measurement again.
The brief mathematical description of pin-hole imaging physical model is as follows:
In the case where camera lens does not have distortion, i.e., when not considering the influence of distortion factor in camera internal reference, if P=
(X, Y, Z) is a bit in world coordinate system, in pin-hole imaging model, a point p=(μ, ν) in this respective pixel coordinate system,
Then the mapping relations between point p and P can be indicated with following formula (1):
Wherein, Metzler matrix is to join outside camera, characterizes point under world coordinate system passing through rigid body translation (rotation and translation)
The mapping relations of respective point under camera shooting leader system are transformed to, K is camera internal reference matrix, characterizes point under camera coordinate system and arrives
Respective point mapping relations under pixel coordinate system.
Spin matrix can indicate in above formula are as follows: R=RZ(φ)Ry(θ)Rx(ψ)。
Wherein, Rx(ψ) indicates to rotate around x axis the matrix of ψ, Ry(θ) indicates the matrix that θ is rotated around y-axis, RZ(φ) is indicated around z
The matrix of axis rotation.The expression of each spin matrix is by simple analytic geometry knowledge.
Multiplication is done to the above matrix, obtains spin matrix expression formula to the end are as follows:
According to above-mentioned expression formula, the R of confederate matrix00, R10, R20, R21, R22The posture Eulerian angles of camera can be solved.
It is not distort or distort in lesser situation in camera above, pixel coordinate is fastened a little and world coordinate system
Relationship between upper point, for big visual field camera lens, lens distortion has to consider, i.e., the distortion factor in " internal reference " is necessary
It accounts for.Distortion herein is divided into " radial distortion " and " tangential distortion ", the former is from lens form, and the latter is from whole
The assembling process of a camera.
" radial distortion " of cam lens shows themselves in that place of the light far from lens centre than leaning on paracentral place more
Add bending.
This radial distortion is smaller at optical instrument center, more bigger toward edge, therefore can be to arrive optical center position
Distance does Taylor expansion.Coordinate points and the relationship before coordinate points before after distorting are set up, enables p' for pixel coordinate after correction
Point, p are corresponding pixel coordinate point before correcting, then the relationship between them can indicate are as follows:
P'=f (p)=α0+α1r+α2r2+α3r3+...…(6)
Mirror symmetry when in view of imaging, therefore only have the even power of r to be not zero in expression formula, take the first three items of expression formula
Have:
xcorrected=x (1+k1r2+k2r4+k3r6)
ycorrected=y (1+k1r2+k2r4+k3r6)…(7)
(x, y) is home position of the distortion point on imager only generated by radial distortion, (x in above formulacorrected,
ycorrected) be correction after corresponding points new position.
Second is " tangential distortion " to the distortion of internal reference having an important influence on, and this distortion is it is also assumed that be lens
Defect in manufacture make lens itself not parallel with imaging plane and generate." tangential distortion " can use two additional ginsengs
Number p1And p2It describes, as described in the following formula:
x'corrected=x+ [2p1y+p2(r2+2x2)]
y'corrected=y+ [p1(r2+2y2)+2p2x]…(8)
(x, y) is home position of the distortion point on imager only generated by " tangential distortion " in above formula,
(x'corrected, y'corrected) be correction after corresponding points new position.
Finally, due to which two above distortion causes the position obtained on image to be false.If pin-hole model has been
Beauty, (μ on imaging surfacep,νp) point distortion point corresponding to this point position coordinates (μd,νd) there is following relationship:
(9) formula is substituted into (1) formula, and matrix unification will be joined into homography matrix H outside interior participation, pixel after being corrected
The relational expression of corresponding points in coordinate points and world coordinate system:
The acquisition joined outside unicity matrix H and imaging system internal reference.First according to the coordinate system shown in Fig. 4, obtain
The coordinate information (X, Y, Z) for demarcating the characteristic point specified on cloth, then finds the pixel coordinate of individual features point again on image
(μd,vd), 4 pairs or more the corresponding relationships put are found, and set element a in homography matrix H33It is 1.0, solves (10) formula, exhibition
Open the relationship that can be obtained between coordinate.
Here unknown number has 8, four couples of point (μ of subsistence leveld,vd), (X, Y) can solve homography matrix H, accordingly may be used
It, can be in the hope of ginseng outside camera then by expression formula (1) to (5) in the hope of camera internal reference.The content item respective figure 4
In step 2., step 3..
3, it is corrected in real time about big visual field fish eye images concurrent operation.Each frame image collected to every road camera,
Imaging system bring pattern distortion must be eliminated, in favor of determining that the pixel in image is corresponding with world coordinate system midpoint
Relationship, while being also beneficial to the texture mapping of last rendering stage.Big visual field (flake) image is produced in actual operation
To the look-up table LUT of fluoroscopy images, to LUT table, GPU piecemeal is simultaneously in OpenCL equipment in a manner of single-instruction multiple-data SIMD
Row is to improve the speed of service.
4, the point that the corresponding world coordinates of each point is fastened on each camera captured images can be obtained by content item 2,
And texture coordinate of this under the camera posture.Corresponding Softcam, the camera are generated to four cameras
Coordinate it is identical as real camera coordinate, but the viewpoint of Softcam is towards straight down, where vertical chessboard calibration cloth
Plane.Have at this time:
pbirdeye=Hbirdeye*P…(12)
It can be by method shown in similar content item 2, using four pairs of given points (that is, as plane characteristic point and correspondence
World coordinate system characteristic point) mapping relations solve Hbirdeye, field can be further determined that after obtaining this mapping relations
Coincidence content of the scape under the same coordinate system.
5, the foundation about unified normalized coordinate system.It seeks after joining outside camera system, can get each camera shooting
The corresponding relationship of pixel and world coordinate system under head coordinate system and the coordinate system, so after being corrected on image pixel and
World coordinates fastens corresponding relationship a little.We establish unified coordinate system as shown in Fig. 5.In Fig. 5, central large wire frame is
Abstract automobile, automobile, which is located at, to be laid on the gridiron pattern on ground, and automobile wire frame surrounding disk is abstract camera.Definition wires
The geometric center of frame (i.e. automobile) is the geometric center of world coordinate system, i.e., at (x, y, z)=(0,0,0), horizontally to the right for x
Axis, with vertical automobile plane vertically on for y-axis, with vertical x, the vector for the plane that y coordinate system is constituted is z-axis, three coordinates to
Amount constitutes x, y, the right-handed coordinate system of z-order.4. content item 3,4,5 corresponds to the step in attached drawing 4.
6, it is calculated about seam mask.To the image Image after correctionbirdeyeUsing HbirdeyeHomography conversion is done, then
By the unification of transformed image into normalization world coordinate system, the overlapping region of four imaging systems is obtained accordingly, according to
Image Image after the setting correction of overlapping regionbirdeyeMask plate Imagemask, which sets image after correction
The ownership in region, i.e., it is whether exclusive by a certain camera, if to be overlapping region, if be seam region the case where, will weigh
The pixel value for closing region is set as 1, is zero at remaining.5. the content item corresponds to the step in attached drawing 4.
7, about three dimensional network grid generation.Regulatable three-dimensional grid is designed, and is split and projects to step in attached drawing 4
5. in the coordinate system, establish the texture relationship of each projection grid with corresponding correction figure, adjustment grid certain vertex with
Keep it corresponding with corresponding texture characteristic points, obtains correcting image Image accordinglyundistoriedBetween texture corresponding with 3D grid
Part relations.6. the content item corresponds to the step in attached drawing 4.
8, rendering system is established, after user specifies the position of virtual view, direction, visual field, loads 3D type, texture,
Figure is rendered on display screen using GPU concurrent operation.
9, in the load of 3D model data, vertex position coordinate data is put into vertical array, establishes vertical array object
(VAO) and opposite vertexes array is managed, to reduce array switching bring loss in efficiency.Vertex buffer object (VBO) is established,
This is bound with vertical array.Texture coordinate data is loaded, and is put into specific array, is established texture buffer object (TBO), it will
TBO and texture array are bound.Function call can be reduced in this way, avoids data redundancy, improve performance.The content item corresponds to attached
Step in Fig. 4 7. with step 8..
10, about illumination compensation.The mask plate obtained is loaded, due to region (the i.e. each camera being overlapped in mask plate
The part being overlapped in sampled images, and seam is located at the center of mask plate at this time) pixel value is 1, according to the exposure mask of acquisition
Plate set vision region of interest ROI, and the step of ROI is applied to attached drawing 4 8. obtained in go distortion image
ImageundistoriedOn, using GPU general-purpose computations, the pixel that ROI is related to is summed, the OpenCL core of design location of pixels statistics
Function (kernel), and being acted on the ROI image region of piecemeal, with pixel and and pixel Euler's distance for weight pair
The image of seam crossing seamlessly transits, and is adjusted to the brightness of whole image.The content item corresponds to the step in attached drawing 4
⑨。
11, about information integration and coordinate transform.Acquire pedestrian information that other Parallel Units obtain and lane information
Co-ordinate content is mapped as texture coordinate after equally doing normalization world coordinate system transformation to these coordinates, in conjunction with above content item
7, the corresponding texture coordinate of image adjusted is exported, to handle in next step.Step in the content item respective figure 4
10. stepAnd step
12, it is inputted about texture mapping.Combined content item 6 and content item 10 and last display picture further adjust
Texture coordinate, the deficiencies of avoiding the occurrence of distortion and the crack flaw because of the incorrect appearance of texture coordinate map, make every effort to make finally aobvious
Show that scenic picture and prompt information have good visual experience sense.Step in the content item respective figure 4
13, it is inputted about rendering system data.In attached coordinate system shown in fig. 5, by user by touch screen, voice,
Keyboard, mouse, the events such as light pen define the rotation of 3D type, scaling, and translation equiaffine transformation constructs model transformation matrix.By with
Family is specified or viewpoint (Softcam when rendering) information using default, which must include viewpoint position, direction, depending on
Field range, constructs viewpoint matrix and projection matrix accordingly.The model view square that will be made of model transformation matrix and viewpoint matrix
Battle array is applied in 3D vertical array, and apex coordinate is transformed in visual coordinate system by object local coordinate system.
14, projection matrix is acted on into the point in visual coordinate system, and does perspective division to this, it will be under visual coordinate system
Point is mapped in the device coordinate system of standardization, finally does the viewport transform to obtained point, and obtained point is mapped to screen and is sat
Mark is fastened.
15, about GPU data management.By vertex to rendering figure during, by graphics vertex position, associated line
Coordinate and data texturing are managed, after being put into corresponding video memory, bound object such as VAO, VBO, TBO manage data, and will count
According to being pushed to vertex shader (vertex shader), in piece member tinter (fragment shader) pipeline, GPU is transferred to do
Concurrent operation processing.Step in 13,14,15 respective figure 4 of content item
16, about GPU concurrent operation and real-time rendering.Since current some equipment only support the figure API compared with lowest version,
Such as higher tinters such as the calculating tinters (compute shader) of not supporting OpenGL, traditional GPU operation can only
General parallel operation, and the universal parallel operational capability due to calculating tinter are made by way of texture of figure API
Still limited, in order to play the potential of GPU, program operation speed is improved, we devise universal parallel arithmetic element, adopt herein
With the framework of OpenCL, it is certainly not limited to OpenCL, for different equipment, CUDA can also be used, but basic principle is not
Become.
17, during these, OpenCL and OpenGL pass through shared device context, storage object, cache object, line
Manage object, the mode of rendering cache object and event object is interacted and communicated.
In conclusion looking around vehicle assistant drive method the present invention is based on the 3D of GPU concurrent operation passes through above-mentioned technical side
The real-time acquired image of each camera is mapped on pre-generated 3D grid model in a manner of texture, is obtained by case
Corresponding texture image；According to the 3D grid model, pre-generated mask plate, in GPU concurrent operation method to the line
Reason image is handled, the texture image that obtains that treated, all seats of treated texture image the contains texture image
Mark information；Obtain after the processing grid vertex spatial position coordinate value in the texture coordinate value of texture image and 3D grid model
Between correspondence mappings relationship；According to the 3D grid vertex and include all coordinate informations Texture image synthesis 3D ring
Depending on roaming system, compared with the existing technology, the potential of GPU parallel computation is preferably played, boosting algorithm processing speed reduces mould
Type seam crossing and outer peripheral distortion, fusion is more multi-functional, to improve automobile assistant driving intelligence, is promoted and drives peace
Quan Xing.
To achieve the above object, the present invention also proposes that a kind of 3D based on GPU concurrent operation looks around vehicle assistant drive system
System.The system comprises several cameras being installed on vehicle, memory, processors, and are stored on the memory
The 3D based on GPU concurrent operation look around vehicle assistant drive program, the 3D based on GPU concurrent operation looks around vehicle auxiliary
Piloting procedure executes following steps when being run by the processor:
The real-time acquired image of each camera is mapped in a manner of texture on pre-generated 3D grid model,
Obtain corresponding texture image；
According to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image into
Row processing, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；
Obtain after the processing grid vertex spatial position coordinate in the texture coordinate value of texture image and 3D grid model
Correspondence mappings relationship between value；
It according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.
Further, the 3D based on GPU concurrent operation looks around vehicle assistant drive program and is run by the processor
When also execute following steps:
According to the 3D grid model, pre-generated mask plate, two neighboring camera collected figure in real time is obtained
The vision area-of-interest of the overlapping region of picture；
It is real in conjunction with the camera using GPU concurrent operation and OpenCL function according to the vision area-of-interest
When acquired image rgb value, the exposure compensating coefficient of the overlapping region is calculated；
Illumination compensation is carried out to each texture image according to the exposure compensating coefficient, the texture maps after obtaining illumination compensation
Picture；
Texture image after loading each illumination compensation does the position coordinates of the texture image after each illumination compensation and reflects
Penetrate transformation, the position coordinates after obtaining mapping transformation；
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established, simultaneously using GPU
Row operation handles the texture image, the texture image that obtains that treated.
Further, the 3D based on GPU concurrent operation looks around vehicle assistant drive program and is run by the processor
When also execute following steps:
It by the 3D grid vertex and include that the texture image of all coordinate informations is sent to the GPU rendering of OpenGL
On pipeline, generates 3D and look around roaming system.
Further, the 3D based on GPU concurrent operation looks around vehicle assistant drive program and is run by the processor
When also execute following steps:
Obtain in advance each camera internal reference and outer ginseng；
Mask plate is generated according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
Further, the 3D based on GPU concurrent operation looks around vehicle assistant drive program and is run by the processor
When also execute following steps:
Pre-generated 3D grid model.
Vehicle DAS (Driver Assistant System) is looked around the present invention is based on the 3D of GPU concurrent operation through the above technical solutions, by each
The real-time acquired image of camera is mapped on pre-generated 3D grid model in a manner of texture, obtains corresponding texture
Image；According to the 3D grid model, pre-generated mask plate, the texture image is carried out in GPU concurrent operation method
Processing, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；It obtains
The texture coordinate value of texture image is corresponding between the coordinate value of grid vertex spatial position on 3D grid model after the processing
Mapping relations；According to the 3D grid vertex and include all coordinate informations Texture image synthesis 3D look around roaming system
System preferably plays the potential of GPU parallel computation compared with the existing technology, and boosting algorithm processing speed reduces model seam crossing
And outer peripheral distortion, fusion is more multi-functional, to improve automobile assistant driving intelligence, promotes drive safety.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of 3D based on GPU concurrent operation looks around vehicle assistant drive method, which is characterized in that the method is applied to base
Vehicle DAS (Driver Assistant System) is looked around in the 3D of GPU concurrent operation, is equipped on the vehicle for acquiring vehicle environmental view
Several cameras of picture, the described method comprises the following steps:
The real-time acquired image of each camera is mapped on pre-generated 3D grid model in a manner of texture, is obtained
Corresponding texture image；
According to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image at
Reason, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；
Obtain after the processing in the texture coordinate value of texture image and 3D grid model grid vertex spatial position coordinate value it
Between correspondence mappings relationship；
It according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.
2. the 3D according to claim 1 based on GPU concurrent operation looks around vehicle assistant drive method, which is characterized in that
It is described according to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image at
The step of reason, the texture image that obtains that treated includes:
According to the 3D grid model, pre-generated mask plate, the real-time acquired image of two neighboring camera is obtained
The vision area-of-interest of overlapping region；
According to the vision area-of-interest, using GPU concurrent operation, while using the OpenCL kernel function of design, in conjunction with institute
The exposure compensating coefficient of the overlapping region is calculated in the rgb value for stating the real-time acquired image of camera；
Illumination compensation is carried out to each texture image according to the exposure compensating coefficient, the texture image after obtaining illumination compensation；
Texture image after loading each illumination compensation does mapping to the position coordinates of the texture image after each illumination compensation and becomes
It changes, the position coordinates after obtaining mapping transformation；
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established, are transported parallel using GPU
Calculation handles the texture image, the texture image that obtains that treated.
3. the 3D according to claim 2 based on GPU concurrent operation looks around vehicle assistant drive method, which is characterized in that
It is described according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around the step of roaming system
Suddenly include:
It by the 3D grid vertex and include that the texture image of all coordinate informations is sent to the GPU rendering pipeline of OpenGL
On, it generates 3D and looks around roaming system.
4. the 3D according to claim 3 based on GPU concurrent operation looks around vehicle assistant drive method, which is characterized in that
It is described to be mapped to the real-time acquired image of each camera in a manner of texture on pre-generated 3D grid model, it obtains
Include: before the step of corresponding texture image
Obtain in advance each camera internal reference and outer ginseng；
Mask plate is generated according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
5. the 3D according to claim 4 based on GPU concurrent operation looks around vehicle assistant drive method, which is characterized in that
It is described to be mapped to the real-time acquired image of each camera in a manner of texture on pre-generated 3D grid model, it obtains
Before the step of corresponding texture image further include:
Pre-generated 3D grid model.
6. a kind of 3D based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System), which is characterized in that the system comprises installations
In several cameras on vehicle, memory, processor, and be stored on the memory based on GPU concurrent operation
3D looks around vehicle assistant drive program, and the 3D based on GPU concurrent operation looks around vehicle assistant drive program by the processing
Device executes following steps when running:
The real-time acquired image of each camera is mapped on pre-generated 3D grid model in a manner of texture, is obtained
Corresponding texture image；
According to the 3D grid model, pre-generated mask plate, with GPU concurrent operation method to the texture image at
Reason, the texture image that obtains that treated, all coordinate informations of treated texture image the contains texture image；
Obtain after the processing in the texture coordinate value of texture image and 3D grid model grid vertex spatial position coordinate value it
Between correspondence mappings relationship；
It according to the 3D grid vertex and include that the Texture image synthesis 3D of all coordinate informations looks around roaming system.
7. the 3D according to claim 6 based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System), which is characterized in that
The 3D based on GPU concurrent operation is looked around when vehicle assistant drive program is run by the processor and is also executed following steps:
According to the 3D grid model, pre-generated mask plate, the real-time acquired image of two neighboring camera is obtained
The vision area-of-interest of overlapping region；
According to the vision area-of-interest, using GPU concurrent operation, while using the OpenCL kernel function of design, in conjunction with institute
The exposure compensating coefficient of the overlapping region is calculated in the rgb value for stating the real-time acquired image of camera；
Illumination compensation is carried out to each texture image according to the exposure compensating coefficient, the texture image after obtaining illumination compensation；
Texture image after loading each illumination compensation does mapping to the position coordinates of the texture image after each illumination compensation and becomes
It changes, the position coordinates after obtaining mapping transformation；
Position coordinates after all mapping transformations are included under the unified normalized coordinate pre-established, are transported parallel using GPU
Calculation handles the texture image, the texture image that obtains that treated.
8. the 3D according to claim 7 based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System), which is characterized in that
The 3D based on GPU concurrent operation is looked around when vehicle assistant drive program is run by the processor and is also executed following steps:
It by the 3D grid vertex and include that the texture image of all coordinate informations is sent to the GPU rendering pipeline of OpenGL
On, it generates 3D and looks around roaming system.
9. the 3D according to claim 8 based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System), which is characterized in that
The 3D based on GPU concurrent operation is looked around when vehicle assistant drive program is run by the processor and is also executed following steps:
Obtain in advance each camera internal reference and outer ginseng；
Mask plate is generated according to the internal reference of each camera and outer ginseng, and establishes unified normalized coordinate.
10. the 3D according to claim 9 based on GPU concurrent operation looks around vehicle DAS (Driver Assistant System), which is characterized in that
The 3D based on GPU concurrent operation is looked around when vehicle assistant drive program is run by the processor and is also executed following steps:
Pre-generated 3D grid model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811542584.XA CN109741456A (en) | 2018-12-17 | 2018-12-17 | 3D based on GPU concurrent operation looks around vehicle assistant drive method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811542584.XA CN109741456A (en) | 2018-12-17 | 2018-12-17 | 3D based on GPU concurrent operation looks around vehicle assistant drive method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109741456A true CN109741456A (en) | 2019-05-10 |
Family
ID=66360394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811542584.XA Pending CN109741456A (en) | 2018-12-17 | 2018-12-17 | 3D based on GPU concurrent operation looks around vehicle assistant drive method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109741456A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969672A (en) * | 2019-11-14 | 2020-04-07 | 杭州飞步科技有限公司 | Image compression method and device |
| CN111064947A (en) * | 2019-12-04 | 2020-04-24 | 广东康云科技有限公司 | Panoramic-based video fusion method, system, device and storage medium |
| CN111757057A (en) * | 2020-06-17 | 2020-10-09 | 广州市泰睿科技有限公司 | Panoramic all-around display method, device, equipment and storage medium |
| CN112218067A (en) * | 2020-10-16 | 2021-01-12 | 南京云滨信息科技有限公司 | Interactive engine system for complex 3D scene and operation method thereof |
| CN112437287A (en) * | 2020-11-23 | 2021-03-02 | 成都易瞳科技有限公司 | Panoramic image scanning and splicing method |
| CN113066158A (en) * | 2019-12-16 | 2021-07-02 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN114119923A (en) * | 2021-11-29 | 2022-03-01 | 浙江大学 | Three-dimensional face reconstruction method and device and electronic equipment |
| WO2022134442A1 (en) * | 2020-12-25 | 2022-06-30 | 合众新能源汽车有限公司 | Image processing method, device and system, and computer-readable medium |
| CN115086575A (en) * | 2022-08-16 | 2022-09-20 | 之江实验室 | Video picture splicing method and device based on unmanned vehicle remote driving |
| CN117931120A (en) * | 2024-03-22 | 2024-04-26 | 南京达道电子科技有限公司 | Camera image visual angle adjusting method based on GPU |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106683119A (en) * | 2017-01-09 | 2017-05-17 | 河北工业大学 | Moving vehicle detecting method based on aerially photographed video images |
| CN107204018A (en) * | 2017-04-24 | 2017-09-26 | 东北大学 | A kind of color compensation method based on light differential |
| CN108269235A (en) * | 2018-02-26 | 2018-07-10 | 江苏裕兰信息科技有限公司 | A kind of vehicle-mounted based on OPENGL looks around various visual angles panorama generation method |
| US20180300556A1 (en) * | 2017-04-17 | 2018-10-18 | Intel Corporation | Person tracking and privacy and acceleration of data using autonomous machines |
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
-
2018
- 2018-12-17 CN CN201811542584.XA patent/CN109741456A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106683119A (en) * | 2017-01-09 | 2017-05-17 | 河北工业大学 | Moving vehicle detecting method based on aerially photographed video images |
| US20180300556A1 (en) * | 2017-04-17 | 2018-10-18 | Intel Corporation | Person tracking and privacy and acceleration of data using autonomous machines |
| CN107204018A (en) * | 2017-04-24 | 2017-09-26 | 东北大学 | A kind of color compensation method based on light differential |
| CN108269235A (en) * | 2018-02-26 | 2018-07-10 | 江苏裕兰信息科技有限公司 | A kind of vehicle-mounted based on OPENGL looks around various visual angles panorama generation method |
| CN108765496A (en) * | 2018-05-24 | 2018-11-06 | 河海大学常州校区 | A kind of multiple views automobile looks around DAS (Driver Assistant System) and method |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969672A (en) * | 2019-11-14 | 2020-04-07 | 杭州飞步科技有限公司 | Image compression method and device |
| CN111064947A (en) * | 2019-12-04 | 2020-04-24 | 广东康云科技有限公司 | Panoramic-based video fusion method, system, device and storage medium |
| CN113066158B (en) * | 2019-12-16 | 2023-03-10 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN113066158A (en) * | 2019-12-16 | 2021-07-02 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN111757057A (en) * | 2020-06-17 | 2020-10-09 | 广州市泰睿科技有限公司 | Panoramic all-around display method, device, equipment and storage medium |
| CN111757057B (en) * | 2020-06-17 | 2022-06-17 | 广州市泰睿科技有限公司 | Panoramic all-around display method, device, equipment and storage medium |
| CN112218067A (en) * | 2020-10-16 | 2021-01-12 | 南京云滨信息科技有限公司 | Interactive engine system for complex 3D scene and operation method thereof |
| CN112437287A (en) * | 2020-11-23 | 2021-03-02 | 成都易瞳科技有限公司 | Panoramic image scanning and splicing method |
| WO2022134442A1 (en) * | 2020-12-25 | 2022-06-30 | 合众新能源汽车有限公司 | Image processing method, device and system, and computer-readable medium |
| CN114119923A (en) * | 2021-11-29 | 2022-03-01 | 浙江大学 | Three-dimensional face reconstruction method and device and electronic equipment |
| CN114119923B (en) * | 2021-11-29 | 2022-07-19 | 浙江大学 | Three-dimensional face reconstruction method and device and electronic equipment |
| CN115086575A (en) * | 2022-08-16 | 2022-09-20 | 之江实验室 | Video picture splicing method and device based on unmanned vehicle remote driving |
| CN115086575B (en) * | 2022-08-16 | 2022-11-29 | 之江实验室 | Video picture splicing method and device based on unmanned vehicle remote driving |
| CN117931120A (en) * | 2024-03-22 | 2024-04-26 | 南京达道电子科技有限公司 | Camera image visual angle adjusting method based on GPU |
| CN117931120B (en) * | 2024-03-22 | 2024-05-24 | 南京达道电子科技有限公司 | Camera image visual angle adjusting method based on GPU |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109741456A (en) | 3D based on GPU concurrent operation looks around vehicle assistant drive method and system | |
| JP7403700B2 (en) | Fully convolutional point of interest detection and description via homography fitting | |
| KR101956149B1 (en) | Efficient Determination of Optical Flow Between Images | |
| US10104288B2 (en) | Method and apparatus for generating panoramic image with stitching process | |
| CN104246795B (en) | The method and system of adaptive perspective correction for extrawide angle lens image | |
| JP7035112B2 (en) | Computer vision methods and systems | |
| CN103763479B (en) | The splicing apparatus and its method of real time high-speed high definition panorama video | |
| US20170263046A1 (en) | Perceptually-based foveated rendering using a contrast-enhancing filter | |
| CN101363962B (en) | Method for providing output image in perspective mode | |
| CN105354796B (en) | Image processing method and system for auxiliary of driving a vehicle | |
| CN107527327B (en) | A kind of flake bearing calibration based on GPU | |
| US10186067B2 (en) | Method and apparatus for generating panoramic image with rotation, translation and warping process | |
| CN105144245A (en) | Apparatus and method for radiance transfer sampling for augmented reality | |
| US20200211257A1 (en) | Photometric image processing | |
| US20130141547A1 (en) | Image processing apparatus and computer-readable recording medium | |
| CN110930489A (en) | Real-time system and method for rendering stereoscopic panoramic images | |
| CN106558017B (en) | Spherical display image processing method and system | |
| CN109064409A (en) | A kind of the visual pattern splicing system and method for mobile robot | |
| US10719920B2 (en) | Environment map generation and hole filling | |
| CN107145224B (en) | Human eye sight tracking and device based on three-dimensional sphere Taylor expansion | |
| CN106023072A (en) | Image splicing display method for curved-surface large screen | |
| CN109388455B (en) | Fish-eye image unfolding monitoring method based on Opengles supporting multiple platforms | |
| WO2022100419A1 (en) | Image processing method and related device | |
| CN110908510A (en) | Application method of oblique photography modeling data in immersive display equipment | |
| EP3098777B1 (en) | Drawing apparatus, drawing method, and drawing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190510 |
|
| RJ01 | Rejection of invention patent application after publication |