CN101872468B - Image scaling method for keeping visual quality of sensitive target - Google Patents

Abstract

The invention discloses an image scaling method for keeping visual quality of a sensitive target, which comprises the following steps of: calculating the sensitivity of a pixel point in an image by using four characteristics comprising color, gradient, brightness and center distance of the pixel point in the image to obtain an image sensitivity graph in accordance with the primary image, then generating a triangle mesh covering the primary image by adopting Delaunay triangulation according to the image sensitivity graph, establishing a visual loss function of the image on the basis, obtaining a target mesh with the same size as a target image by solving the optimization problem of the minimum value of the visual loss function under integrity constraint of the image, and finally generating a final result image in a texture mapping mode.

Description

A kind of image-scaling method that keeps visual quality of sensitive target

Technical field

The present invention relates to a kind of image-scaling method that keeps visual quality of sensitive target, the susceptibility that relates to pixel in image calculates and the image distortion method based on triangle gridding, high-quality zoomed image keeping obtaining under the substantially indeformable prerequisite of sensitive target the best visual effect, belongs to technical field of computer vision.

Background technology

At present be all to adopt the mode of direct linear scale to carry out convergent-divergent to original image in actual applications, the two class problems that directly image-scaling method of linear scale exists: 1) directly may cause important goal undersized after linear scale, be not easy to observe.2), if when the length breadth ratio of the length breadth ratio of original image and mobile terminal screen differs larger, can cause the gross distortion distortion of target in image.The image-scaling method that keeps visual quality of sensitive target is exactly the image scaling technology being widely studied in order to solve the problem of direct Zoom method existence.The pungent university of University of Wisconsin-Madison, Microsoft Research, Israel Tel Aviv university, Mitsubishi electrical equipment research institute, Israel Wei Ciman Science Institute and domestic Tsing-Hua University, Institute of Automation, CAS, Hong Kong Chinese University, Taiwan success university etc. have all carried out a large amount of research work, and have obtained correlative study achievement.Different according to the principle of these achievements in research, they can be divided into following four classes: the image-scaling method based on window clipping, the image-scaling method based on distortion of the mesh, the image-scaling method based on resampling and the image-scaling method merging based on several different methods.

The basic ideas of the image-scaling method based on window clipping are: by certain Selection Strategy, search out a window identical with target size size in original image, then the picture material in window is cut out and be used as last scaled results output.Because the content of just having chosen in original image in a window identical with target size is as a result of exported, if Fig. 1 (a) is original image, Fig. 1 (b) is image after convergent-divergent, after image scaling, only retained original image center position image information, all the other original image informations are all lost.So, Zoom method based on window clipping is not suitable for the unconcentrated situation of sensitive target distribution, for example sensitive target is distributed in the edge, two of left and right of image, and a crop window cannot comprise all sensitive targets wherein, at this moment must cause sensitive information to lose.

Image-scaling method based on distortion of the mesh has mainly adopted the thought of distortion of the mesh.First image is covered and grid is out of shape with the grid of an equal size, wherein the sensitive target region in grid is adopted similarity transformation or remained unchanged, the distortion of larger coefficient is carried out in other region, obtain the grid after the distortion measure-alike with target image; Then the coordinate in original mesh according to each pixel of target gridding, adopts certain interpolation method to generate the pixel value in target gridding the pixel value of the pixel in original image, thereby generates the target image after convergent-divergent.Method based on distortion of the mesh, owing to having adopted morphing, may introduce deformation distortion.

Image-scaling method based on resampling is mainly, according to certain strategy, the pixel of original image is carried out to resampling, generates class methods of final target image.Method based on resampling easily causes the distortion of structural stronger sensitive target in image.If Fig. 2 (a) is original image, Fig. 2 (b) is result images after convergent-divergent.Circular sensitive target in Fig. 2 (a) is deformed into the ellipse that serious deformation occurs in Fig. 2 (b).

The image-scaling method merging based on several different methods is mainly that the Zoom method that multiple sensitive target is kept becomes a kind of new Zoom method according to certain strategy combination.The technique computes amount merging based on several different methods is larger, is unfavorable for applying on the limited mobile device of processing power.

Summary of the invention

The object of this invention is to provide a kind of image-scaling method that keeps visual quality of sensitive target.The method is utilized the sensitivity of a plurality of low-level image feature computed image pixels of image pixel, then according to the sensitivity of pixel, adopt the triangle gridding of different densities to cover original image, by meeting the optimization procedure of the vision loss minimum under image integrity constraint, original image triangle gridding is deformed to target image grid again, finally by the mode productive target image of texture.The method has reduced sensitive target because convergent-divergent produces the probability of deformation.Realized high-quality sensitive target zooming effect.

For achieving the above object, the present invention adopts following technical scheme.It is characterized in that comprising the following steps:

Step 1: calculate the sensitivity of original image pixels point by extracting the low-level image feature of image slices vegetarian refreshments, obtain comprising the sensitivity figure of all pixel sensitivitys in original image;

Step 2: according to the sensitivity figure obtaining in step 1, in original image, choose the coordinate points of some as the summit of triangle gridding, original image is carried out to triangulation, obtain the triangle gridding identical with original image size, the triangle gridding number of vertex that wherein the higher region of sensitivity is chosen is more;

Step 3: the triangle gridding obtaining according to step 2, calculates the sensitivity sum of pixel of all delta-shaped regions in triangle gridding as the sensitivity in this region;

Step 4: utilize the triangle gridding area sensitive degree obtaining in step 3, by the visual effect loss S in formula (1) is minimized, calculate the target triangle gridding identical with target image size,

$S=\frac{1}{2}\underset{(i,j\∈\mathrm{edges})}{\mathrm{\Σ}}{\mathrm{\λ}}_{\mathrm{ij}}{(P_i^{*}-P_j^{*})}^2---\left(1\right)$

The wherein visual effect of S presentation video loss, i, j represents the summit on any limit in triangle gridding; represent respectively summit i, the position after j convergent-divergent; λ _ijfor vision loss energy coefficient,

${\mathrm{\&lambda;}}_{\mathrm{ij}}=\mathrm{Saliency}(i,j,j-1)*(L_{i,j-1}^2+L_{j,j-1}^2-L_{i,j}^2)/<P_i,P_j,P_{j-1}>,$

$+\mathrm{Saliency}(i,j,j+1)*(L_{i,j+1}^2+L_{j,j+1}^2-L_{i,j}^2)/<P_i,P_j,P_{j+1}>$

J-1, j+1 is respectively and summit i, and j forms vertex of a triangle, and Saliency (i, j, j-1) represents the sensitivity of triangle (i, j, j-1), and Saliency (i, j, j+1) represents the sensitivity of triangle (i, j, j+1); the length that represents limit (i, j); <P _i, P _j, P _j-1> represents the area of triangle (i, j, j-1); <P _i, P _j, P _j+1> represents the area of triangle (i, j, j+1);

Step 5: according to the target triangle gridding obtaining in step 4, the method by texture obtains target image.

Preferably, the image low-level image feature in described step 1 comprises: the color of pixel, gradient, brightness and with the distance of central pixel point.

Preferably, in described step 1 the calculating of image slices vegetarian refreshments sensitivity by each low-level image feature is weighted to average acquisition.

Preferably, choosing of described step 2 intermediate cam grid vertex is by original image being divided into the square area that size is identical, count the sensitivity sum of all pixels in each square area, above-mentioned sensitivity sum and predetermined threshold value are compared to the quantity of determining each square area intermediate cam grid vertex.

The image-scaling method of maintenance visual quality of sensitive target provided by the present invention can reduce the distortion of sensitive target and the information dropout of sensitive target that image causes due to convergent-divergent effectively.Relevant test result shows, this method all has good zooming effect for all kinds of images, and the situation effect of the especially large percentage in image for sensitive target, and image scaling large percentage is more obvious.

Accompanying drawing explanation

Fig. 1 window clipping class keeps visual quality of sensitive target method design sketch.

Fig. 2 class that resamples keeps visual quality of sensitive target method design sketch.

The image-scaling method process flow diagram that Fig. 3 sensitive target of the present invention keeps.

Embodiment

Before address, the present invention obtains the sensitivity figure corresponding with original image by analyzing the sensitivity of original image, adopt on this basis triangulation method to generate the triangle gridding identical with original image size, then by solving, make the optimization procedure of image vision Minimal energy loss obtain the target gridding consistent with target image size.Finally the mode by texture generates result images.

Below in conjunction with accompanying drawing, implementation of the present invention is described, in Fig. 3, has clearly represented process of the present invention.First, the sensitivity of computed image; Secondly, according to sensitivity, generate original image triangle gridding; Then, triangle gridding is carried out to optimization distortion; Finally, adopt the mode of texture to generate final target image.

It should be noted that following is only the exemplary one embodiment of the present invention of having enumerated:

Step 1: the calculating of image sensitivity

Existing image sensitivity computing method can be divided into two classes: the sensitivity that adopts the low-level image feature computed image of image; Adopt the mode of pattern-recognition to identify the sensitive target in image.Adopt at present the method major part of the low-level image feature of image to concentrate on to adopt single low-level image feature such as gradient, be used as the sensitivity of image.The feature of these class methods is simple, quick.Shortcoming is that the robustness of these class methods is bad, not accurate enough.And adopt the advantage of the method for the sensitive target in the method recognition image of pattern-recognition to be: accurately.Shortcoming is: can only identify a few type objects, Generalization Capability is poor.In the present invention, by adopting average weighted mode to calculate the sensitivity of image to the color of image slices vegetarian refreshments, gradient, brightness, four kinds of features of centre distance, the accuracy have simply, feature having improved the judgement of image susceptibility simultaneously fast, and there is good robustness and Generalization Capability.

A kind of exemplary implementation step of step 1 is as follows:

(1) calculating of color of image susceptibility

The calculating of color of image susceptibility has showed the effect of color aspect image sensitivity.It is generally acknowledged that the less color that distributes in image has higher susceptibility, and general color distributed more widely has lower susceptibility.The computing method of color of image susceptibility can be by adding up this row pixel at R for the pixel of every row, G, color histogram in B triple channel (size of bin is 9), then adopts formula (2) to calculate the color sensitivity Sc of this pixel for each pixel.

$S_{\mathrm{Ci}}=1-\frac{f_i-f_{\min }^i}{f_{\max }^i-f_{\min }^i},i=r,g,b---\left(2\right)$

Sc＝0.34S _Cr+0.33S _Cg+0.33S _Cb

S in formula (2) _cirepresent that pixel is at the sensitivity of i Color Channel, f _ifor the frequency of occurrences of pixel in color value correspondence in color histogram of i Color Channel, the minimum value that represents color histogram medium frequency under i Color Channel, similarly the maximal value that represents color histogram medium frequency under i Color Channel, Sc represents the color sensitivity of pixel.

Because the color of noise pixel point often distributes less in image, the impact of image sensitivity being calculated in order to reduce noise in image point, if the frequency of occurrences of pixel color under Color Channel i is less than threshold value T in the process of computed image sensitivity, can be directly by the sensitivity value S of pixel under this passage _cibe made as 0.

(2) calculating of image gradient susceptibility

Because gradient has reflected the information such as structure of image, for structural stronger sensitive target, such as: straight line, circular etc.The Grad of this class sensitive target is higher.Canny operator has good noise resisting ability, and gradient detectability, therefore can adopt the gradient of canny operator computed image pixel, and finally the Grad after normalization is as the gradient sensing value of image.

(3) calculating of image irradiation susceptibility

Generally, in image the higher region of brightness often the sensitizing range in image because L component in LAB space has represented the illumination of image, therefore, can by RGB color space conversion, be LAB color space by image, directly the value of L component is normalized, and the photoperiod sensitivity value using end value as image.

(4) picture centre is apart from the calculating of susceptibility

It is larger that a large amount of experiments shows that the sensitive target in image appears at the probability of center of image.Therefore this method adopts formula (3) computed image centre distance susceptibility.

$S_p=1-\frac{l}{l_{\max }}---\left(3\right)$

$l=\sqrt{{(x-x_0)}^2+{(y-y_0)}^2},(x,y)\&Element;S$

S in formula (3) _pthe centre distance susceptibility of presentation video, l is pixel (x, y) and picture centre pixel (x ₀, y ₀) between distance.

(5) calculating of image susceptibility

Finally can be by average weighted mode, the method shown in formula (4) for example, the susceptibility of computed image.

S _I＝α ₁×S _C+α ₂×S _L+α ₃×S _G+α ₄×S _P (4)

α ₁+α ₂+α ₃+α ₄＝1

S in formula (4) _ifor the susceptibility of image, S _cfor the color sensitivity of image, S _lfor the photoperiod sensitivity of image, S _gfor the gradient sensing of image, S _pfor the centre distance susceptibility of image, α ₁, α ₂, α ₃, α ₄be respectively color sensitivity, photoperiod sensitivity, the weighting factor of gradient sensing and centre distance susceptibility.

Step 2: the generation of triangle gridding

Triangulation refers to a several picture is divided into mutually disjoint leg-of-mutton process one by one, and the set that the triangle of these divisions forms is called triangle gridding.In the image-scaling method of the maintenance visual quality of sensitive target based on grid in the past, it is all the rectangular node that adopts the rectangle identical with original image size to be divided into.The problem of rectangular node is accurately to express whole sensitive target.In the present invention, problem that can not accurate expression sensitive target for rectangular node, has proposed a kind of triangle gridding based on Delaunay triangulation.This triangle gridding can determine according to the sensitivity of image the distribution density of grid intermediate cam shape, thereby adopt more triangle to be described to sensitive target, and adopt less triangle to be described for non-sensitive target, with respect to rectangular node, sensitive target has more accurately been described.

An exemplary implementation step of step 2 is as follows:

(1) Delaunay triangulation vertex determines

In order to guarantee the existence of Delaunay subdivision, must guarantee that the convex closure of the set of all triangulation points exists.Therefore, for the edge pixel o'clock of image, according to the density of 5 pixels, it is sampled, and sampled point is added to the set of triangulation point, inner at image, image is divided into the set of the patch of 25*25 pixel size.For each patch, calculate pixel wherein susceptibility and, the patch that surpasses certain threshold value for sensitivity, in this grid elements, choose at random 8 triangle gridding summits, otherwise choose at random 1 triangle gridding summit of quantity for the patch that does not surpass threshold value.

(2) Delaunay triangulation

For Delaunay triangulation, if triangulation vertex is definite, the result of Delaunay triangulation is unique so.Method obtains the triangle set of Delaunay triangulation, the i.e. triangle gridding of original image using the triangulation vertex of upper step generation as input.

Step 3: the optimization deformation of triangle gridding

The triangle gridding generating in step 2 is out of shape and is obtained and the identical triangle gridding of target zoomed image size, for the zoomed image keeping obtaining in the substantially indeformable situation of sensitive target visual effect the best.In the present invention, first defined the visual effect loss function of image in convergent-divergent process, thus by solve that optimization problem at the lower vision loss function minimum of image integrity constraint obtains having minimum vision loss with the identical triangle gridding of target zoomed image size.

An exemplary implementation step of step 3 is as follows:

(1) visual effect loss function

In order to realize sensitive target, in the convergent-divergent process of image, do not produce deformation, the convergent-divergent that requires sensitive target is a linear scale, and the angle at the leg-of-mutton mapping angle in the triangle in the triangle gridding after distortion and original triangle gridding remains unchanged.Also just require original triangle gridding to carry out conformal transformation.Formula in the present invention (1) has defined the visual effect loss function of image in convergent-divergent process.

(2) generation of optimum vision loss triangle gridding

In order to obtain the target zoomed image of optimum visual effect, namely require image in convergent-divergent process, to there is minimum vision loss, therefore the present invention is converted into image lattice problem on deformation the optimization problem solving at the lower vision loss function minimum of image integrity constraint, in order to guarantee that final result images is rectangle, the present invention adopts formula (5) as the constraint condition of optimization vision loss problem simultaneously.Because this problem is a quadratic programming problem, by solving this optimization problem, obtained having the target triangle gridding of minimum vision loss.

(5)

In formula (5) be respectively pixel P _icoordinate after convergent-divergent.M ', n ' is width and the height of image after convergent-divergent.

Step 4: texture

In step 3, generated the target triangle gridding with minimum vision loss, in step 2, the present invention has generated the triangle gridding of original image.Because target triangle gridding is to have original triangle gridding distortion to obtain, so the summit in the summit of target triangle gridding and original triangle gridding is relation one to one, therefore to target triangle gridding using corresponding vertex position in original triangle gridding as texture coordinate, in conjunction with original image by the final target zoomed image with optimum vision loss that generates of mode of texture.

Disclosed is above only instantiation of the present invention, according to thought provided by the invention, those skilled in the art can think and variation, all should fall within the scope of protection of the present invention.

Claims (4)

1. an image-scaling method that keeps visual quality of sensitive target, is characterized in that comprising the following steps:

Step 1: calculate the sensitivity of original image pixels point by extracting the low-level image feature of image slices vegetarian refreshments, obtain comprising the sensitivity figure of all pixel sensitivitys in original image;

Step 2: according to the sensitivity figure obtaining in step 1, in original image, choose the coordinate points of some as the summit of triangle gridding, original image is carried out to triangulation, obtain the triangle gridding identical with original image size, the triangle gridding number of vertex that wherein the higher region of sensitivity is chosen is more;

Step 3: the triangle gridding obtaining according to step 2, calculates the sensitivity sum of pixel of all delta-shaped regions in triangle gridding as the sensitivity in this region;

Step 4: utilize the triangle gridding area sensitive degree obtaining in step 3, by the visual effect loss S in formula (1) is minimized, calculate the target triangle gridding identical with target image size,

$S=\frac{1}{2}\underset{(i,j\&Element;\mathrm{edges})}{\mathrm{\&Sigma;}}{\mathrm{\&lambda;}}_{\mathrm{ij}}{(P_i^{*}-P_j^{*})}^2---\left(1\right)$

The wherein visual effect of S presentation video loss, i, j represents the summit on any limit in triangle gridding; represent respectively summit i, the position after j convergent-divergent; λ _ijfor vision loss energy coefficient,

$\begin{array}{c}{\mathrm{\&lambda;}}_{\mathrm{ij}}=\mathrm{Saliency}(i,j,j-1)*(L_{i,j-1}^2+L_{j,j-1}^2-L_{i,j}^2)/{<P}_i,P_j,P_{j-1}>\\ +\mathrm{Saliency}(i,j,j+1)*(L_{i,j+1}^2+L_{j,j+1}^2-L_{i,j}^2)/<P_i,P_j,P_{j+1}>\end{array},$

J-1, j+1 is respectively and summit i, and j forms vertex of a triangle, and Saliency (i, j, j-1) represents the sensitivity of triangle (i, j, j-1), and Saliency (i, j, j+1) represents the sensitivity of triangle (i, j, j+1); the length that represents limit (i, j-1); the length that represents limit (j, j-1); the length that represents limit (i, j); the length that represents limit (i, j+1); the length that represents limit (j, j+1); < P _i, P _j, P _j-1> represents the area of triangle (i, j, j-1); < P _i, P _j, P _j+1> represents the area of triangle (i, j, j+1);

Step 5: according to the target triangle gridding obtaining in step 4, the method by texture obtains target image.

2. the method for claim 1, is characterized in that: the low-level image feature of the image slices vegetarian refreshments in described step 1 comprises: the color of pixel, gradient, brightness and with the distance of central pixel point.

3. the method for claim 1, is characterized in that: in described step 1, the calculating of image slices vegetarian refreshments sensitivity is by being weighted average acquisition to each low-level image feature.

4. the method for claim 1, it is characterized in that: choosing of described step 2 intermediate cam grid vertex is by original image being divided into the square area that size is identical, count the sensitivity sum of all pixels in each square area, above-mentioned sensitivity sum and predetermined threshold value are compared to the quantity of determining each square area intermediate cam grid vertex.