CN103530907A - Complicated three-dimensional model drawing method based on images - Google Patents

Abstract

The invention provides a complicated three-dimensional model drawing method based on images. Vertexes are uniformity selected from a spherical surface surrounding a model as camera coordinate positions, and a color image and a depth image of the model under a sampling visual angle are obtained by taking a sphere center as a target point of a camera. The method comprises the steps of triangularly dividing the spherical surface according to sampling point coordinates to determine a triangle where a virtual visual angle is located, taking corresponding visual angles as reference visual angles when taking vertexes of the triangle as sampling points, and drawing the model under the virtual visual angle with depth images and color images under the reference visual angles: calculating mapping relationships between the virtual visual angle and pixels in the three reference visual angles respectively with parameters of the reference visual angles, and drawing an image of the virtual visual angle with the appropriate pixels of the reference visual angles or a background pixel by taking the depth images as references, and finally, optimizing the drawn color image. The method can meet real-time requirements and obtains a very true-to-life drawing effect.

Description

Complex three-dimensional modeling rendering method based on image

Technical field

The present invention relates to a kind of Realistic Rendering method based on image, be mainly used in the Realistic Rendering of complex model under virtual visual angle.

Background technology

In the middle of traditional graphics, the general process of Realistic Rendering is: user inputs the geometrical property of object and carries out Geometric Modeling, then the illumination information at environment according to model, the physical attributes such as the smoothness of model, transparency, reflectivity refractive index, and superficial makings etc., by spatial alternation, perspective transform etc., calculate the color value of each pixel of object under certain viewing angles.Yet the modeling process of this method is complicated, calculating and the expense showing are large, and time complexity and model complexity coupling are high, are not suitable for the drafting of complex model, and are difficult to obtain drawing result true to nature.

Method for drafting based on image (Image-Based Rendering.IBR) is usingd image as basic input, image on the basis of not carrying out geometric model reconstruction under synthetic virtual visual angle, in fields such as video-game, fictitious tour, ecommerce, industrial detection, having boundless application prospect, is therefore also a study hotspot of three-dimensional picture field of drawing true to nature.Complex three-dimensional modeling rendering method based on image proposed by the invention is a kind of IBR method.

Main method based on Image Rendering is as follows:

1. how much and image blend modeling and method for drafting (Hybrid Geometry and Image-based Approach)

PaulE.Debevec (list of references 1-PaulE.Debevec, camilloj.Taylor, Jitendra make. " Modeling and rendering architecture from photographs:a hybrid geometry and imaged-based approach " .In SIGGRAPH96Processing of the23 ^rdannual conference on computer graphics and interactive techniques.Page11-20) geometry proposing and the key step of image blend modeling and method for drafting are as follows:

A. photographed scene photo, the edge of mutual designated model;

B. the rough model of generation model;

C. utilize the stereoscopic vision algorithm refined model based on model;

D. utilize the synthetic new view of texture based on viewpoint.

The example of this process as shown in Figure 4.The advantage of this method is simple and fast, the new view that can obtain by taking a small amount of photo; This process of shortcoming also needs the profile of artificial designated model, can only be applicable to the scenery of the neat appearances such as common building thing, is not suitable for complex model.

2. the method for view interpolation, conversion (View Interpolation, Transaction)

View interpolation, transform method directly utilize the image under the photo generating virtual visual angle of reference point.View interpolation, transform method (list of references 2-Geetha Ramachandran, Markus Rupp. " MultiView Synthesis From Stereo Views " .In IWSSIP2012, 11-13April2012, PP.341-PP.345. list of references 3-Ankit K.Jain, Lam C.Tran, RamsinKhoshabeh, Truong Q.Nguyen, Efficient Stereo-to-Multiview Synthesis, ICASSP2011.PP.889-PP.892. list of references 4-S.C.Chan, A.C.Bovik, H.R.Sheikh, and E.P.SimomCelli, " Image-Based Rendering and synthesis ", IEEE Signal Process, Mag, vol.24, no.6, PP.22-PP.33) input is coloured image and the depth image of two Regularizations under visual angle, output be to be positioned at (baseline on the straight line that two reference point determine, Baseline) image under virtual visual angle, concrete process is as follows:

A. Stereo matching, generates initial synthetic visual angle image;

B. optimization process, finds possible empty point according to rim detection;

C. fill and generate the corresponding depth map in synthetic visual angle;

D. image reconstruction, empty according to depth image filling color image.

The example of the method as shown in Figure 5.The advantage of this method is that process is simple, noise peakedness ratio high (noise peakedness ratio (PSNR), the image after characterization process and the similarity of processing front image of synthetic image.Signal peak, than higher, illustrates that the validity of composograph is stronger), and filling cavity that can be outstanding.But the method can only produce the image that is positioned at the virtual visual angle on baseline, and picture is carried out to Regularization operation and can introduce projection error, can only generate approximate intermediate image.

Summary of the invention

There is following shortcoming in existing three-dimensional model Realistic Rendering method: it is complicated that the method based on how much exists model to obtain with process of reconstruction, drawing process is subject to model complexity, model illumination properties affect large, draw the effect sense of reality not strong, be not suitable for the drafting of complex model; Method for drafting based on image, synthetic visual angle is limited on two baselines between reference viewing angle, cannot generate the image under object meaning in office visual angle.

For the shortcoming of prior art, the present invention proposes the complex three-dimensional modeling rendering method based on image, and it comprises following process:

(1) demarcation at virtual visual angle: carry out triangle division to surrounding the sphere of model according to sampled point camera position coordinate, determine the tri patch at place, virtual visual angle, the corresponding visual angle, three summits of getting this tri patch is reference viewing angle, and virtual visual angle can be expressed as the linear combination of reference viewing angle;

(2) calculate and draw: according to the position of virtual visual angle, reference viewing angle and camera parameter, calculating the coordinate of each pixel in three reference viewing angle images and the mapping relations between the pixel coordinate under virtual visual angle; According to mapping relations, with reference to each pixel in coloured image under visual angle, be mapped in the image under virtual visual angle, calculate coordinate and the depth value of this pixel in the hypograph of virtual visual angle, for the pixel that has a plurality of reference viewing angle, be mapped to the situation of same position, get the pixel value that depth value is little; With the pixel of all referenced visual angles pixel filling in the hypograph of the virtual visual angle of tense marker, construct the gray-scale map of a width reflection from reference viewing angle to virtual visual angle mapping situation;

(3) optimization of image: for the cavity in the hypograph of virtual visual angle, passing through (2) calculating does not have reference picture to be mapped to the pixel of this position, gray-scale map from reflection reference viewing angle to virtual visual angle mapping situation, extract edge contour information, along edge contour, the coloured image generating is carried out to medium filtering, with the value filling cavity of neighbor pixel; Meanwhile, by medium filtering, filter out noise pixel.

Wherein, by step (1), realize the demarcation to virtual visual angle, be identified for the reference viewing angle that draw at virtual visual angle, the pixel of setting up under reference viewing angle by step (2) arrives the mapping relations between pixel under virtual visual angle, realizes the complex model under virtual visual angle is drawn.

Wherein, in step (2) and step (3), use CUDA (Compute Unified Device Architecture, unified calculation equipment framework) parallel computation, accelerates drafting and optimal speed under virtual visual angle, reaches the requirement of real-time, interactive.

Principle of the present invention is:

The invention provides a kind of complex three-dimensional modeling rendering method based on image, on the sphere that surrounds model, uniform design summit is as camera coordinates position, and the impact point that the centre of sphere of take is camera, obtains coloured image and the depth image of model under this sampling visual angle.According to sample point coordinate, sphere is carried out to triangle division, determine the triangle at place, virtual visual angle, get that to take vertex of a triangle be reference viewing angle as the corresponding visual angle of sampled point, use depth image and coloured image under reference viewing angle to draw the model under virtual visual angle: first, to utilize the parameter of reference viewing angle to calculate respectively the mapping relations between pixel in virtual visual angle and three reference viewing angle; Secondly, take depth image as reference, the image of selecting suitable reference viewing angle pixel or background pixel to draw virtual visual angle; Finally, to drawing the coloured image obtaining, be optimized.In whole drawing process, use CUDA acceleration, realized the parallel fast processing to image.The present invention can meet the requirement of real-time, obtains the drafting effect being really true to life simultaneously.

Compared with prior art, advantage is as follows in the present invention:

(1) virtual visual angle of the present invention can be moved arbitrarily in the middle of space.Utilize three models under the virtual visual angle of reference point to draw, so both can guarantee that virtual visual angle can, in level and vertical two cofree movements of dimension, reduce again input quantity and the storage overhead of algorithm to greatest extent;

(2) method for drafting that the present invention proposes is highly stable, and the complexity coupling of the time complexity of algorithm and scene is low, is particularly useful for the drafting of complex model.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is the principle schematic of this algorithm;

Fig. 3 when the inside of virtual visual angle at certain tri patch, the position relationship at virtual visual angle and reference point visual angle;

Fig. 4 direct transform matrix and inverse-transform matrix;

Fig. 5 is how much and image blend modeling process, wherein: (a) be the profile of determining rebuilt object by mutual, (b) be the rough model that obtains object, (c) being the stereoscopic vision algorithm refined model of using based on model, is (d) to obtain final scene by the texture based on viewpoint;

Fig. 6 is the example of disparity map method, initial input picture is the coloured image (being the schematic diagram of the gray-scale map of this coloured image in figure) of Regularization, initial generation figure has obtained cromogram (being the schematic diagram of the gray-scale map of this cromogram in figure) after pixel mapping and filling process, note the cavity in square frame, the final output of algorithm is hole-filling cromogram (being the schematic diagram of the gray-scale map of this cromogram in figure) later, notices that the cavity in square frame is eliminated;

The cromogram (being the schematic diagram of the gray-scale map of this cromogram in figure) at the virtual visual angle of Fig. 7 after preliminary mapping is filled, the cavity of the inside of the noise of attention model periphery;

The gray-scale map at the virtual visual angle of Fig. 8 after preliminary mapping is filled, the noise of attention model periphery and inner cavity;

Fig. 9 enters the outline map after overexpansion, difference operation;

Figure 10 simply carries out medium filtering result afterwards to all pixels;

Figure 11 edge pixel carried out the result after medium filtering, compares with Figure 10, and Figure 11 has better preserved detailed information, truly stronger;

Figure 12 is the part input picture of invention example, and top three pictures are input picture schematic diagram, and input picture can be coloured image, in figure, does not show, below three pictures are difference depth maps of correspondence with it;

Figure 13 is the part output image of invention example, and the picture of output is synthetic by method for drafting proposed by the invention.

Embodiment

Below in conjunction with drawings and the specific embodiments, further illustrate the present invention.

1. the Realistic Rendering method based on depth image:

For by being evenly distributed on the sphere that surrounds model, the three-dimensional model that the sampled point that comprises depth map under camera perspective parameter, camera perspective, cromogram represents, according to sample point coordinate, sphere is carried out to triangle division, determine the triangle at place, virtual visual angle, get that to take vertex of a triangle be reference viewing angle as the corresponding visual angle of sampled point, use depth image and coloured image under reference viewing angle to draw the model under virtual visual angle: first, to utilize the parameter of reference viewing angle to calculate respectively the mapping relations between pixel in virtual visual angle and three reference viewing angle; Secondly, take depth image as reference, the image of selecting suitable reference viewing angle pixel or background pixel to draw virtual visual angle; Finally, to drawing the coloured image obtaining, be optimized.In whole drawing process, use CUDA acceleration, realized the parallel fast processing to image.Specific implementation process is as follows:

1. the demarcating module at virtual visual angle

In the present invention, the three-dimensional model that represents object with depth image, coloured image and the sampled point camera parameter of different sampled points.A two tuples<K for three-dimensional model M, V>represent, wherein K is a simplicial complex, has represented the annexation of sampled point; V represents the set of sampled point, V=(v<sub TranNum="115">i</sub>| i=1,2,3...|V||), | V| represents the number of sampled point; V<sub TranNum="116">i</sub>=(c<sub TranNum="117">i</sub>, d<sub TranNum="118">i</sub>, p<sub TranNum="119">i</sub>) represent i sampled point, c<sub TranNum="120">i</sub>and d<sub TranNum="121">i</sub>represent respectively i sampled point coloured image and depth image, p<sub TranNum="122">i</sub>the camera parameter that has represented i sampled point, p<sub TranNum="123">i</sub>=(pc<sub TranNum="124">i</sub>, po<sub TranNum="125">i</sub>, asp<sub TranNum="126">i</sub>, fov<sub TranNum="127">i</sub>, zn<sub TranNum="128">i</sub>, zf<sub TranNum="129">i</sub>), pc<sub TranNum="130">i</sub>represent camera position, po<sub TranNum="131">i</sub>represent camera target, asp<sub TranNum="132">i</sub>the aspect ratio that represents the camera visual field, fov<sub TranNum="133">i</sub>the range that represents the visual field of camera, zn<sub TranNum="134">i</sub>, zf<sub TranNum="135">i</sub>the minimum value and the maximal value that represent respectively camera significant depth.

Before the model of drawing under virtual visual angle, need to obtain three sampled points nearest with virtual visual angle, carry out the demarcation at virtual visual angle.Because all sampled points are equally distributed on the sphere that surrounds object, at sphere, according to sample point coordinate, carry out after triangle division, only need to determine the tri patch at place, virtual visual angle, with three summits of this tri patch, be exactly required nearest sampled point, these three nearest sampled points are called as the reference point at virtual visual angle.

＜v ₁,v ₂,v ₃＞＝f(v) (1)

Wherein, v is virtual visual angle, < v ₁, v ₂, v ₃> is the tri patch at place, virtual visual angle, v ₁, v ₂, v ₃it is the reference point of v.In the present invention, by solving the vector and the polyhedral intersection point that approaches encirclement object sphere that points to the centre of sphere from virtual visual angle camera position, according to the tri patch at intersection point place, determine three nearest reference point.

Virtual visual angle and reference point have position relationship as shown in Figure 3.From the knowledge of cartesian geometry, under the position relationship shown in Fig. 3, virtual visual angle can be linear synthetic by reference point, and meet:

$\left\{\begin{array}{c}\stackrel{\&RightArrow;}{v}=\mathrm{\&alpha;}{\stackrel{\&RightArrow;}{v}}_1+\mathrm{\&beta;}{\stackrel{\&RightArrow;}{v}}_2+(1-\mathrm{\&alpha;}-\mathrm{\&beta;}){\stackrel{\&RightArrow;}{v}}_3\\ 0\&le;\mathrm{\&alpha;}\&le;1\\ 0\&le;\mathrm{\&beta;}\&le;1\\ 0\&le;1-\mathrm{\&alpha;}-\mathrm{\&beta;}\&le;1\end{array}\right.---\left(2\right)$

Wherein,

represent respectively the vector by virtual visual angle coordinate points and three reference point coordinate points centre ofs sphere.

As shown in Figure 3, the coordinate of three reference point and the centre of sphere form tetrahedral structure, and the coordinate at virtual visual angle is positioned at the tri patch that reference point coordinate surrounds.Order vector

tetrahedron volume can be expressed as the not form of the parallelopipedal product on three limits in same plane:

$u=\frac{1}{6}\stackrel{\&RightArrow;}{v}\&times;\stackrel{\&RightArrow;}{e_1}\&CenterDot;\stackrel{\&RightArrow;}{e_2}=-\frac{1}{6}\stackrel{\&RightArrow;}{v_1}\&CenterDot;\stackrel{\&RightArrow;}{v_2}\&CenterDot;\stackrel{\&RightArrow;}{e_2}---\left(3\right)$

Formula 2 is out of shape:

$\stackrel{\&RightArrow;}{v_1}=\frac{\stackrel{\&RightArrow;}{v}-\mathrm{\&beta;}\stackrel{\&RightArrow;}{v_2}-(1-\mathrm{\&alpha;}-\mathrm{\&beta;})\stackrel{\&RightArrow;}{v_3}}{\mathrm{\&alpha;}}---\left(4\right)$

The latter half of formula (4) being brought into formula (3) obtains following formula:

$\mathrm{\&alpha;}=\frac{-\stackrel{\&RightArrow;}{v}\&CenterDot;\stackrel{\&RightArrow;}{v_2}\&times;\stackrel{\&RightArrow;}{e_2}}{\stackrel{\&RightArrow;}{v}\&times;\stackrel{\&RightArrow;}{e_1}\&CenterDot;\stackrel{\&RightArrow;}{e_2}}---\left(5\right)$

In like manner.Can obtain:

$\mathrm{\&beta;}=\frac{-\stackrel{\&RightArrow;}{v}\&CenterDot;\stackrel{\&RightArrow;}{v_1}\&times;\stackrel{\&RightArrow;}{e_2}}{\stackrel{\&RightArrow;}{v}\&times;\stackrel{\&RightArrow;}{e_1}\&CenterDot;\stackrel{\&RightArrow;}{e_2}}---\left(6\right)$

In sum, only need in the middle of all dough sheets, travel through, utilize formula (5), (6) to obtain α, β in each tri patch, if α, β meet the constraint condition in formula (2), virtual visual angle is just among the encirclement of this dough sheet, and this virtual visual angle can be according to formula (2) by reference point linear expression.

2. calculate and drafting module

This module comprises two subprocess, computation process and drawing process.Computation process determines that the pixel under each reference point visual angle arrives the mapping relations of pixel under virtual visual angle; Drawing process, for each pixel under virtual visual angle, selects pixel or background pixel under 1 to 3 reference point visual angle to draw according to the mapping relations of having tried to achieve.

2.1 computation processes:

Pixel coordinate and the pixel value of the depth image under given a certain reference point visual angle, in conjunction with the camera parameter of reference point, can obtain the corresponding coordinate in three-dimensional world coordinate system of this pixel, and this process is reversible.Be that under any reference point visual angle, the pixel coordinate of depth image and the coordinate in world coordinate system of pixel value and three-dimensional body exist dijection relation:

$\begin{array}{c}\stackrel{\&RightArrow;}{\mathrm{pixel}}=M\&CenterDot;\stackrel{\&RightArrow;}{\mathrm{object}}\\ \stackrel{\&RightArrow;}{\mathrm{object}}=M^{-1}\&CenterDot;\stackrel{\&RightArrow;}{\mathrm{pixel}}\\ \stackrel{\&RightArrow;}{\mathrm{pixel}}=(i,j,l,\mathrm{depth})\\ \stackrel{\&RightArrow;}{\mathrm{object}}=(x,y,z,\mathrm{depth})\end{array}---\left(7\right)$

I wherein, j is pixel coordinate, x, y, z is coordinate in world coordinate system, depth is the pixel value of depth image.M is invertible matrix, by sampled point camera parameter, is determined.Being defined in the present invention coordinate in world coordinate system is forward conversion matrix to the transition matrix M of pixel coordinate, M ^-1for reverse transformation matrix.As shown in Figure 4, the forward conversion matrix of take illustrates its solution procedure as example to the solution procedure of forward conversion matrix and reverse transformation matrix.

First through camera view, conversion is converted to camera volume coordinate to the coordinate of object in world coordinate system, then is converted into pixel coordinate through perspective projection transformation, that is:

M＝mProject·mLookAt (8)

Formula (7), (8) simultaneous are obtained:

$\stackrel{\&RightArrow;}{\mathrm{pixel}}=\mathrm{mProject}\&CenterDot;\mathrm{mLookAt}\&CenterDot;\stackrel{\&RightArrow;}{\mathrm{object}}---\left(9\right)$

$\stackrel{\&RightArrow;}{\mathrm{object}}={(\mathrm{mProject}\&CenterDot;\mathrm{mLookAt})}^{-1}\&CenterDot;\stackrel{\&RightArrow;}{\mathrm{pixel}}$

Wherein, mLookAt is the transformation matrix from world coordinates to camera volume coordinate, and this matrix determines by position coordinates pc, coordinate of ground point po and the positive dirction coordinate up of camera, and concrete form is as follows:

$\mathrm{mLookAt}=\left(\begin{array}{cccc}\stackrel{\&RightArrow;}{\mathrm{xaxis}}.x& \stackrel{\&RightArrow;}{\mathrm{yaxis}}.x& \stackrel{\&RightArrow;}{\mathrm{zaxis}}.x& 0\\ \stackrel{\&RightArrow;}{\mathrm{xaxis}}.y& \stackrel{\&RightArrow;}{\mathrm{yaxis}}.y& \stackrel{\&RightArrow;}{\mathrm{zaxis}}.y& 0\\ \stackrel{\&RightArrow;}{\mathrm{xaxis}}.z& \stackrel{\&RightArrow;}{\mathrm{yaxis}}.y& \stackrel{\&RightArrow;}{\mathrm{zaxis}}.z& 0\\ -\stackrel{\&RightArrow;}{\mathrm{xaxis}}.\mathrm{po}& -\stackrel{\&RightArrow;}{\mathrm{yaxis}}.\mathrm{po}& -\stackrel{\&RightArrow;}{\mathrm{zaxis}}.\mathrm{po}& 1\end{array}\right)$

$\left\{\begin{array}{c}\stackrel{\&RightArrow;}{\mathrm{zaxis}}=\frac{\mathrm{pc}-\mathrm{po}}{|\mathrm{pc}-\mathrm{po}|}\\ \stackrel{\&RightArrow;}{\mathrm{xaxis}}=\frac{\mathrm{up}\&times;\stackrel{\&RightArrow;}{\mathrm{zaxis}}}{|\mathrm{up}\&times;\stackrel{\&RightArrow;}{\mathrm{zaxis}}|}\\ \stackrel{\&RightArrow;}{\mathrm{yaxis}}=\stackrel{\&RightArrow;}{\mathrm{zaxis}}\&times;\stackrel{\&RightArrow;}{\mathrm{xaxis}}\end{array}\right.---\left(10\right)$

MProject is perspective projection transformation matrix, and this matrix is by visual angle range (fov) aspect ratio (asp) degree of depth (zn) and the degree of depth (zf) decision farthest recently of camera, and concrete form is as follows:

$\mathrm{mproject}=\left(\begin{array}{cccc}\mathrm{xScale}& 0& 0& 0\\ 0& \mathrm{yScale}& 0& 0\\ 0& 0& \frac{\mathrm{zf}}{\mathrm{zf}-\mathrm{zn}}& 0\\ 0& 0& -\mathrm{zn}*\frac{\mathrm{zf}}{\mathrm{zf}-\mathrm{zn}}& 1\end{array}\right)---\left(11\right)$

$\left\{\begin{array}{c}\mathrm{yScale}=\cot \left(\frac{\mathrm{fov}}{2}\right)\\ \mathrm{xScale}=\frac{\mathrm{yScale}}{\mathrm{asp}}\end{array}\right.$

Therefore, the pixel under reference point visual angle is as follows to the mapping relations of pixel under virtual visual angle:

$\stackrel{\&RightArrow;}{\mathrm{pixel}}=\mathrm{mProject}\&CenterDot;\mathrm{mLookAt}\&CenterDot;{(\mathrm{mP}{\mathrm{roject}}_{v_i}\&CenterDot;\mathrm{mLook}{\mathrm{At}}_{v_i})}^{-1}\&CenterDot;\stackrel{\&RightArrow;}{{\mathrm{pixel}}_{v_i}}---\left(12\right)$

Wherein,

pixel coordinate and the depth value under virtual visual angle,

reference point v _ipixel coordinate and depth value, mLookAt _v, mProject _vtransformation matrix and the perspective projection matrix of the camera coordinates from world coordinates to virtual visual angle,

from world coordinates to reference point v _itransformation matrix and the perspective projection matrix of camera coordinates.

2.2 drawing process:

Image under virtual visual angle is stored in the middle of coloured image.By formula (12), set out, under the pixel under virtual visual angle and reference point visual angle, pixel may have multiple corresponding relation:

Situation 1: the pixel of virtual visual angle hypograph does not have the pixel under reference point visual angle corresponding with it, and this pixel is an empty part;

Situation 2: the pixel of virtual visual angle hypograph only has 1 pixel under reference point visual angle corresponding with it, directly utilizes the pixel under reference point visual angle to fill the image under virtual visual angle;

Situation 3: some pixels of virtual visual angle hypograph have 2 to 3 pixels under reference point visual angle corresponding with it, carry out the Image Rendering under virtual visual angle according to formula (13).Wherein, the pixel under virtual visual angle is p, pixel p under three reference viewing angle ₁, p ₂, p ₃corresponding with it, under the reference point visual angle of selected depth value minimum, pixel is drawn the image at virtual visual angle in principle.α wherein _iit is the weight at the reference point visual angle of trying to achieve in formula 2.

$\left\{\begin{array}{c}p=\underset{i=1}{\overset{Q}{\mathrm{\&Sigma;}}}\frac{{\mathrm{\&alpha;}}_i}{\mathrm{\&alpha;}}{p}_i,p_i\&Element;Q\\ Q=\left\{q\right|q-p^{\text{'}}<\mathrm{threash},|p^{\text{'}}=\min (p_1\&CenterDot;\&CenterDot;\&CenterDot;p_n)\}\\ \mathrm{\&alpha;}=\underset{i}{\overset{Q}{\mathrm{\&Sigma;}}}{\mathrm{\&alpha;}}_i\end{array},n=\mathrm{2,3}\right.---\left(13\right)$

The all pixels that belong to situation 2 and situation 3 of mark in the process of drawing, are stored in label information in gray-scale map simultaneously.In the process of drawing, use CUDA parallel computation, a plurality of pixels under synthetic virtual visual angle simultaneously, speed that can greatly acceleration drawing.

3. the optimization module of image

The coloured image of above-mentioned drawing process output contains noise and cavity, the reason in cavity illustrates in the calculating above and drafting module, the reason that produces noise is: owing to having error in extraction of depth information process, and what formula (10) obtained might not be the mapping between integer pixel, therefore in drawing process, there will be noise, as shown in Figure 7, Figure 8.Comparatively simple solution is that the image at virtual visual angle is carried out to medium filtering, does like this and can eliminate most noises and cavity.Although it is image blurring and too level and smooth that this method simply, can cause, as shown in figure 10.First the present invention expands and difference operation to gray-scale map, extracts the edge contour of image, then along edge contour, carries out medium filtering, utilizes empty surrounding pixel filling cavity, simultaneously the noise signal at removal of images edge.The module of image optimization comprises following subprocess:

3.1 extract edge pixel

Expansion is to ask local maximum operation.Expansion is that image and core are carried out to convolution, calculates the maximal value of pixel in the middle of the region that core covers, and by this maximal value assignment the pixel to reference point appointment; As calculated with drafting module after the coloured image and the gray level image that generate all contain cavity, gray level image is carried out to expansive working and can eliminate cavity.Gray-scale map after expansion and original gray-scale map are done difference computing, and what obtain is exactly outline map, and what in figure, store is exactly edge contour information, and cavity is with regard in the middle of existence and edge contour, as shown in Figure 9.

3.2 medium filtering

Adopt in the present invention the method for medium filtering to carry out smoothing processing to empty pixel, medium filtering is replaced each pixel in the square neighborhood of Filtering Template center pixel by the pixel in the middle of template.Because medium filtering can cause image fault, the pixel of therefore carrying out intermediate value replacement is limited in the middle of edge contour by strict, can reduce to greatest extent like this number of times of filtering operation, keeps detailed information.Meanwhile, because mostly noise signal is isolated point in the middle of space, when carrying out medium filtering, most edge noise point is covered by background pixel, has reached the object of eliminating noise.The result that final filtering obtains as shown in figure 11.

Similar to drawing process, by using CUDA parallel computation to accelerate image manipulation, improve the travelling speed of method for drafting.

2. implementation process

Next the drawing process of beautiful Buddhist of take is example explanation specific embodiment of the invention process.

(1) on the spherical outside surface that surrounds beautiful Buddhist, select uniformly 162 sampled points, take the centre of sphere as true origin, r=430, the coordinate of i sampled point is v _i=(x _i, y _i, z _i), coordinate meets following formula:

$\left\{\begin{array}{c}{x}_i=r(1-\frac{2i-1}{n})\cos \left(\arcsin (1-\frac{2i-1}{n})\sqrt{\mathrm{n\&pi;}}\right)\\ y_i=r(1-\frac{2i-1}{n})\sin \left(\arcsin (1-\frac{2i-1}{n})\sqrt{\mathrm{n\&pi;}}\right)\\ z_i=r\cos \left(\arcsin (1-\frac{2i-1}{n})\right)\end{array}\right.---\left(14\right)$

Wherein, n=162.

(2) according to sample point coordinate, spherical outside surface is carried out to triangle division, according to the result of triangle division, model data is divided into groups.The present invention adopts one of method construct that triangle approaches to take the ball inpolyhedron that triangle is basic model.After division, on sphere, have 162 sampled points, 320 tri patchs.

(3) sampled point parameter is set, takes depth image and coloured image.The parameter p of sampled point _i=(pc _i, po _i, asp _i, fov _i, zn _i, zf _i), pc _ibe the coordinate of sampled point, by formula (14), calculated, arranging of all the other parameters is as shown in the table:

Table 1

po i	fov i	asp i	zn i	zf i
					(0，0，0)	47′	1.333	350	550

Set after parameter first color image shot, then take depth image.Select wherein three sampled point v ₁, v ₂, v ₃, wherein:

${\mathrm{pc}}_{v_1}=(-\mathrm{182.8899,365.7798,132.8773})$

${\mathrm{pc}}_{v_2}=(-\mathrm{220.4835,217.4600},-298.3256)$

${\mathrm{pc}}_{v_3}=\left(\mathrm{295.9221,226.0644,215.0000}\right)$

The coloured image collecting and depth image are as shown in figure 12.

(4) demarcate virtual visual angle, calculate mapping relations.In example, user uses keyboard and the virtual visual angle of the mutual control of mouse to roam in three-dimensional scenic, and initial virtual visual angle is (0,430,0), and the changing value between adjacent virtual visual angle is specified by user interactions.Known current virtual visual angle, the tri patch obtaining in traversal step (2), determines the tri patch at place, virtual visual angle, and demarcates virtual visual angle with the vector quantization of the reference point sensing centre of sphere according to formula (2), (5), (6).According to the mapping relations of the parameter calculation formula of the parameter at virtual visual angle and sampled point (12).

(5) synthetic virtual visual angle hypograph, and carry out medium filtering.

A. the pixel in virtual visual angle does not have the pixel of reference point with it at once, uses background pixel to fill virtual visual angle hypograph;

B. in the virtual visual angle in pixel only with a reference point pixel at once, use this respective pixel to fill virtual visual angle hypograph;

When c. the pixel of the pixel in virtual visual angle and 2 to 3 reference point is corresponding, set thresh=15, according to formula (13), select the synthetic virtual visual angle of one or more reference point pixel hypograph.

The all pixels that belong to situation b and situation c of mark in the process of drawing, are stored in label information in gray-scale map simultaneously.Gray-scale map is carried out to expansive working, and subtract each other with original gray-scale map, obtain the profile information of model, along the profile of model, carry out medium filtering, obtain the coloured image under final virtual visual angle.

The output of this example is the cromogram under the virtual visual angle under being controlled by user interactions, and Figure 13 is the part Output rusults of this example.

3. implementation result

Mainly from real-time and two aspect explanation implementation results of the sense of reality.The main configuration of testing computing machine used is as shown in the table:

Table 2

Operating system	64bit windows7 Ultimate
		Processor	Intel Core TM2Q9400(4CPUs)2.66GHz
Video card	NVIDIA GeForce GTX470
		Internal memory	4G

2.1 real-time

On testing computer, take beautiful Buddhist as tested object, use 3DMAX software to carry out beautiful Buddhist modeling, optical signature due to jade material, there is scattering after injecting material inside in light, in the middle of finally penetrating object and coming into view, this phenomenon is called as Subsurface Scattering, calculating Subsurface Scattering need to expend the plenty of time, and the drafting used time of using 3DMAX to carry out a visual angle is approximately 40 minutes.The three-dimensional model drawing method of employing based on image, during modeling, do not need to consider the material of model, during drafting, according to pixel, fill, the real time rate of drawing on testing computer was about for 25 frame/seconds, can respond fast user's input, control the roaming arbitrarily in space of virtual visual angle, reach the requirement of real-time, interactive completely.

2.2 the sense of reality

In the middle of this application, 162 sampled points have been selected, and three sampled points that selection and virtual visual angle approach are the most reference point, and the pixel of take is drawn as unit, carries out scene optimization after tentatively completing, repair the noise in cavity removal of images, the effect finally obtaining as shown in figure 13.Make comparisons with original input Figure 12, in Figure 13, there is no cavity and noise pixel, need not calculate and also can obtain drafting effect true to nature through complicated Subsurface Scattering.

The part that the present invention does not elaborate belongs to technology as well known to those skilled in the art.

Claims (3)

1. the complex three-dimensional modeling rendering method based on image, its feature comprises following process:

(1) demarcation at virtual visual angle: carry out triangle division to surrounding the sphere of model according to sampled point camera position coordinate, determine the tri patch at place, virtual visual angle, the corresponding visual angle, three summits of getting this tri patch is reference viewing angle, and virtual visual angle can be expressed as the linear combination of reference viewing angle;

(2) calculate and draw: according to the position of virtual visual angle, reference viewing angle and camera parameter, calculating the coordinate of each pixel in three reference viewing angle images and the mapping relations between the pixel coordinate under virtual visual angle; According to mapping relations, with reference to each pixel in coloured image under visual angle, be mapped in the image under virtual visual angle, calculate coordinate and the depth value of this pixel in the hypograph of virtual visual angle, for the pixel that has a plurality of reference viewing angle, be mapped to the situation of same position, get the pixel value that depth value is little; With the pixel of all referenced visual angles pixel filling in the hypograph of the virtual visual angle of tense marker, construct the gray-scale map of a width reflection from reference viewing angle to virtual visual angle mapping situation;

(3) optimization of image: for the cavity in the hypograph of virtual visual angle, passing through (2) calculating does not have reference picture to be mapped to the pixel of this position, gray-scale map from reflection reference viewing angle to virtual visual angle mapping situation, extract edge contour information, along edge contour, the coloured image generating is carried out to medium filtering, with the value filling cavity of neighbor pixel; Meanwhile, by medium filtering, filter out noise pixel.

2. according to the complex three-dimensional modeling rendering method based on image described in claim 1, it is characterized in that: by step (1), realize the demarcation to virtual visual angle, be identified for the reference viewing angle that draw at virtual visual angle, the pixel of setting up under reference viewing angle by step (2) arrives the mapping relations between pixel under virtual visual angle, realizes the complex model under virtual visual angle is drawn.

3. according to the complex three-dimensional modeling rendering method based on image described in claim 1 or 2, it is characterized in that: in step (2) and step (3), use CUDA parallel computation, accelerate drafting and optimal speed under virtual visual angle, reach the requirement of real-time, interactive.

Images