CN109525838A - Stereo image quality evaluation method based on binocular competition - Google Patents
Stereo image quality evaluation method based on binocular competition Download PDFInfo
- Publication number
- CN109525838A CN109525838A CN201811469588.XA CN201811469588A CN109525838A CN 109525838 A CN109525838 A CN 109525838A CN 201811469588 A CN201811469588 A CN 201811469588A CN 109525838 A CN109525838 A CN 109525838A
- Authority
- CN
- China
- Prior art keywords
- monocular
- stereo
- image
- picture
- reference picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The present invention provides a kind of stereo image quality evaluation methods based on binocular competition, are related to image quality evaluation technical field.The present invention is directed to the deficiency of the stereo image quality evaluation method based on flat image quality evaluation strategy, the statistical distribution characteristic of stereo-picture is portrayed using gauss hybrid models, vision content weight based on mutual information is designed, to improve the robustness and stability of stereo image quality evaluation method.Simultaneously, the present invention is based on the binocular Competition Characteristics of human visual system, using the parallax information of stereo-picture or so view, and structural information, Gradient Features and the phase equalization feature of comprehensive stereo-picture or so monocular view devise a kind of stereo image quality evaluation method competed under guidance based on binocular.It is considered the present invention is based on the fusion of multiple characteristic informations and for the inherent perception characteristics of human visual system, can effectively promote the subjective and objective consistency of stereo image quality evaluation method.
Description
Technical field
The present invention relates to technical field of data processing, in particular to a kind of perspective view image quality based on binocular competition
Measure evaluation method.
Background technique
Stereo-picture processing has increasingly wider in fields such as 3D image, compression of images, augmented reality, stereo-picture communications
General application.However since image is inevitably subjected to a degree of distortion in transmission, storage or issuing process, image will go out
The now quality problems as caused by additive noise, data compression, geometry deformation, motion blur etc..Therefore, to stereo image quality into
Row is objective effectively to be evaluated, and is optimized to the algorithm parameter in three-dimensional image processing system, and then obtains high quality
Stereo-picture, this will analyze for subsequent stereo-picture and establishes important basis with processing.
Stereo image quality objectively evaluate with flat image quality to objectively evaluate thought similar, be all to establish and can feel
The mathematics appraisal for knowing picture quality is core, stereo image quality is accurately predicted and be assessed.Common is vertical
Body image quality evaluating method includes complete evaluates with reference to stereo image quality evaluation, partially with reference to stereo image quality and without ginseng
Examine stereo image quality evaluation method three classes.
It can use original perfect image as benchmark, calculated distortion image with reference to stereo image quality evaluation method entirely
Difference or similarity between original image perceive the predicted value of stereo image quality as it.The mean square error of early stage
(Mean Squared Error, MSE) and Y-PSNR (Peak Signal-to-Noise Ratio, PSNR) are to pass through the most
The full reference image quality appraisement method of allusion quotation.Both methods due to the simple and mathematical meaning of calculating simplicity, at image
Reason field is widely used.But mean square error and the evaluation result of Y-PSNR cannot embody human eye well
Perception of the vision system to picture quality.Equally, application also more difficult and people of the MSE and PSNR in terms of stereo image quality evaluation
The subjective vision perception characteristics of eye are consistent.
The existing full stereo image quality evaluation method that refers to usually uses for reference existing flat image quality evaluating method.Such as
Believed using classical structural similarity evaluation method (Structural Similarity Index, SSIM) plus three-dimensional feature
For breath for assessing stereo image quality, although contrast, brightness, structure and the three-dimensional feature of stereo-picture is utilized in this method
Etc. information, but the viewpoint that has ignored stereo-picture or so it is non-to true distortion when influence to picture quality.Yasakethu et al. is mentioned
A kind of stereo image quality evaluation method is gone out, they are directly based upon the evaluation method of flat image quality to left and right visual point image
It assesses respectively, using the mean value of two quality evaluation results of left and right visual point image as the quality evaluation value of entire stereo-picture.Though
Right this method is capable of the quality of effective evaluation part stereo-picture, but this method has ignored between stereo-picture or so view
Parallax information, cause the Generalization Capability of this method poor.
In summary, the basic process of such stereo image quality evaluation method based on flat image quality evaluation strategy
It is the statistical nature or structure feature for first obtaining left view and right view image, then according to the characteristic information of acquisition, using existing
Some flat image quality evaluating methods carry out quality evaluation to left view and right view image respectively, will finally refer to perspective view
The picture left view image quality estimation value corresponding with distortion stereo-picture and reference stereo-picture are corresponding with distortion stereo-picture
Right view image quality estimation value is weighted fusion, obtains the evaluation quality of entire stereo-picture.Such method does not account for
To complicated human visual perception process and characteristic, also without the inherence interaction between stereo-picture or so view of investigating and mutually
It influences.Therefore, the evaluation performance of such method is lower.
Summary of the invention
In view of this, the present invention provides a kind of stereo image quality evaluation methods based on binocular competition.
Technical solution provided by the invention is as follows:
A kind of stereo image quality evaluation method based on binocular competition, comprising:
Obtain with reference to stereo-picture and distortion stereo pairs, wherein it is described with reference to stereo-picture include with reference to left view and
With reference to right view, the distortion stereo-picture includes distortion left view and distortion right view；
Monocular reference picture is synthesized by the reference left view and with reference to left view；
The distortion left view and distortion right view are synthesized into monocular distorted image；
It is built using wavelet coefficient of the Gauss scale mixed model to the monocular reference picture and monocular distorted image
Mould；
The characteristic information of the monocular reference picture and monocular distorted image is extracted, the characteristic information includes monocular reference
The structural information of image and the monocular distorted image, gradient amplitude information and phase equalization information；
According to the characteristic information of the monocular reference picture and monocular distorted image, the monocular reference picture and list are calculated
The characteristic similarity of mesh distorted image；
According to the characteristic similarity of the monocular reference picture and monocular distorted image, in conjunction with vision content weight, building
Stereo-picture evaluation method establishes the subjective and objective consistency nonlinear function of image quality evaluation；
According to default stereoscopic image data library data, stereo image quality is objectively evaluated.
Further, monocular reference picture is synthesized by described with reference to left view and with reference to left view, and by the distortion
Left view and distortion right view the step of synthesizing monocular distorted image include:
Using the multiple Gabor filter function of two dimension to the reference left view, with reference to right view, distortion left view and distortion
Right view is filtered respectively, wherein the multiple Gabor filter function of two dimension are as follows:
Wherein, R1=xcos θ+ysin θ, R2=-sin θ+ycos θ, σx,σyRespectively indicate standard deviation, ζx,ζyIt respectively indicates
Spatial frequency, θ indicate filtering direction；
Be calculated using the following equation it is described with reference to left view, with reference to right view, distortion left view and be distorted right view power
Weight；
Wherein, described with reference to left view, energy response value of the distortion left view on all scales and direction is GEL, institute
Stating with reference to the energy response value of right view, distortion right view on all scales and direction is GER, and d is parallax；
According to it is described with reference to left view, with reference to right view, distortion left view and be distorted right view weight, using following public affairs
Formula synthesizes monocular reference picture and monocular distorted image:
C (x, y)=WL(x,y)IL(x,y)+WR((x+d),y)IR((x+d),y)。
Further, using Gauss scale mixed model to the wavelet systems of the monocular reference picture and monocular distorted image
Counting the step of being modeled includes:
Calculate the mutual information I (R between the monocular reference picture and monocular reference picture after perceiving；E), wherein
Monocular reference picture after perceiving and the monocular distorted image after perceiving are respectively E and F；
Calculate the mutual information I (D between the monocular distorted image and monocular distorted image after perceiving；F)；
Calculate the monocular reference picture after perceiving and the mutual information I (E between the monocular distorted image after perceiving；F)；
According to I (R；E),I(D；) and I (E F；F), vision content weight based on mutual information is calculated.
Further, according to I (R；E),I(D；) and I (E F；F), the step of calculating vision content weight based on mutual information
Include:
It calculates described with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion after perceiving
The covariance of stereo-picture
Calculate the mutual information I (R between the monocular reference picture and monocular reference picture after perceiving；E), the list
Mutual information I (D between mesh distorted image and monocular distorted image after perceiving；F the monocular) and after perceiving is with reference to figure
Picture and the mutual information I (E between the monocular distorted image after perceiving；F)；
According to described with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion after perceiving
Mutual information I (R between the covariance of stereo-picture and the monocular reference picture and monocular reference picture after perceiving；
E), the mutual information I (D between the monocular distorted image and monocular distorted image after perceiving；F the list) and after perceiving
Mutual information I (E between mesh reference picture and monocular distorted image after perceiving；F), the vision content weight is calculated.
Further, it is calculated using the following equation described vertical with reference to stereo-picture, distortion perspective view, the reference after perceiving
Body image, the covariance for being distorted stereo-picture after perceiving:
Wherein, s indicates that multiplication factor, g indicate gain factor,Indicate perception visual noise variance,Indicate that Gauss makes an uproar
Sound variance, I indicate identity matrix, CUFor the covariance matrix of zero-mean gaussian vector U,
Further, according to described with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, through feeling
The covariance of distortion stereo-picture after knowing is calculated using the following equation the monocular reference picture and joins with the monocular after perceiving
Examine the mutual information I (R between image；E), the mutual information between the monocular distorted image and monocular distorted image after perceiving
I(D；F the mutual information I (E between the monocular reference picture) and after perceiving and the monocular distorted image after perceiving；F):
Wherein, the reference image R, E are calculated using the following equation；D, F；Covariance between E, F:
Further, according to described with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, through feeling
Between the covariance and the monocular reference picture and monocular reference picture after perceiving of distortion stereo-picture after knowing
Mutual information I (R；E), the mutual information I (D between the monocular distorted image and monocular distorted image after perceiving；F it) and passes through
Mutual information I (E between monocular reference picture after perception and the monocular distorted image after perceiving；F), using following formula meter
Calculate the vision content weight:
Wherein,Indicate distortion noise variance,Indicate random distortion variance, g indicate gain factor, s indicate multiplication because
Son, λkIndicate k-th of characteristic value, wherein k=1,2 ..., K.
Further, according to the characteristic information of the monocular reference picture and monocular distorted image, the monocular ginseng is calculated
The step of examining the characteristic similarity of image and monocular distorted image include:
Calculate brightness, contrast and the knot of the image block of same position in the monocular reference picture and monocular distorted image
Structure information；
Calculate the structural similarity of the monocular reference picture and monocular distorted image；
Calculate the gradient similarity of the monocular reference picture and monocular distorted image；
Calculate the phase equalization similarity of the monocular reference picture and monocular distorted image.
Further, it is calculated using the following equation the figure of same position in the monocular reference picture and monocular distorted image
As the brightness of block, contrast and structural information:
Wherein, C1=(K1L)2,C2=(K2L)2,C3=C2/ 2, K1<<1,K2< < 1, L ∈ (0,255], μx,μyIt respectively indicates
The mean value of image block x, y；σx,σyRespectively indicate image block x, the variance of y；σxyFor x, the similarity factor of y,
It is calculated using the following equation the structural similarity of the monocular reference picture and monocular distorted image:
SSIM=[l (x, y)]α·[c(x,y)]β·[s(x,y)]γ
Wherein α > 0, β > 0, γ > 0, C1,C2For preset constant；
It is calculated using the following equation the gradient similarity of the monocular reference picture and monocular distorted image:
gx,gyRespectively indicate image block x, the gradient of y,Indicate the variation of gradient, max (gx,gy) indicate
Mask gradient；
It is calculated using the following equation the phase equalization similarity of the monocular reference picture and monocular distorted image:
Wherein, SPCIndicate phase equalization similarity, C4Preset constant greater than 0, PC (x) indicate the monocular with reference to figure
The phase equalization value of image block x as in, PC (y) indicate the phase equalization value of image block y in the monocular reference picture.
Further, it is weighed according to the characteristic similarity and vision content of the monocular reference picture and monocular distorted image
Weight constructs stereo-picture evaluation method, and the step of establishing the subjective and objective consistency nonlinear function of image quality evaluation includes:
It is calculated using the following equation the image quality evaluating value S with reference between stereo-picture and distortion stereo-pictureQA:
Wherein, M indicates scale parameter of the image through wavelet decomposition, wiIndicate that the monocular is with reference to figure on i-th of decomposition scale
Vision content weight between picture and monocular distorted image, S (R, D)jIndicate the monocular reference picture and monocular distorted image
Local similarity between j-th of Image Sub-Band；
It carries out curve fitting to image quality evaluating method, establishes the nonlinear function of subjective assessment:
Wherein μ1,μ2,μ3,μ4,μ5Indicate nonlinear fitting parameter.
The present invention is directed to the deficiency of the stereo image quality evaluation method based on flat image quality evaluation strategy, utilizes height
This mixed model portrays the statistical distribution characteristic of stereo-picture, vision content weight based on mutual information is designed, to improve solid
The robustness and stability of image quality evaluating method.Meanwhile the present invention is based on the binocular Competition Characteristics of human visual system, answer
With the parallax information of stereo-picture or so view, and the structural information of comprehensive stereo-picture or so monocular view, Gradient Features
And phase equalization feature, devise a kind of stereo image quality evaluation method competed under guidance based on binocular.The present invention
Fusion based on multiple characteristic informations and the inherent perception characteristics of human visual system are considered, can effectively promote solid
The subjective and objective consistency of image quality evaluating method.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 is that a kind of process of stereo image quality evaluation method based on binocular competition provided in an embodiment of the present invention is shown
It is intended to.
Fig. 2 is step in a kind of stereo image quality evaluation method based on binocular competition provided in an embodiment of the present invention
The flow diagram of the sub-step of S102.
Fig. 3 is step in a kind of stereo image quality evaluation method based on binocular competition provided in an embodiment of the present invention
The flow diagram of the sub-step of S104.
Fig. 4 is step in a kind of stereo image quality evaluation method based on binocular competition provided in an embodiment of the present invention
Another flow diagram of the sub-step of S106.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause
This, is not intended to limit claimed invention to the detailed description of the embodiment of the present invention provided in the accompanying drawings below
Range, but it is merely representative of selected embodiment of the invention.Based on the embodiment of the present invention, those skilled in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall within the protection scope of the present invention.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.Meanwhile of the invention
In description, term " first ", " second " etc. are only used for distinguishing description, are not understood to indicate or imply relative importance.
Stereo image quality evaluation method in the prior art based on flat image quality evaluation strategy, can be summarized as following
Several key steps.
1) input respectively includes left view and right view image with reference to stereo-picture and distortion stereo pairs.
2) the feature letter of stereo-picture left view and right view image is extracted using manual features or the method for machine learning
Breath, such as contrast, brightness, Texture complication, gradient characteristic information.
3) based on the characteristic information of acquisition, using flat image quality evaluation strategy, such as common structural similarity method,
Characteristic similarity method and the image matter for being based on natural scene statistical property (Natural Scene Statistics, NSS)
Amount method etc. is calculated with reference to left view and the characteristic similarity being distorted between left view two images.Meanwhile it calculating and referring to right view
Characteristic similarity between figure and distortion right view two images.
4) according to the feature phase of the reference stereo-picture left view image and right view image corresponding with distortion stereo-picture
Like degree information, the left view image quality estimation value and right view corresponding with distortion stereo-picture with reference to stereo-picture are calculated separately
Then image quality estimation value is weighted fusion to the quality predictions of left view and right view image, obtain entire three-dimensional
The quality evaluation value of image.
5) evaluation function for designing stereo image quality, according to disclosed stereoscopic image data library data, to stereo-picture
Quality is objectively evaluated.
Existing flat image quality evaluating method is used for reference directly to comment the left and right view image of stereo-picture respectively
Valence, the quality evaluation for then integrating left and right view again obtain the quality of entire stereo-picture, and the main advantage of this technology is benefit
With existing mature flat image quality evaluating method, but the technology has ignored the parallax information of left and right view, and three-dimensional
Inherence between image or so view is interactive and influences each other.
To solve the above-mentioned problems, the embodiment of the present application provides a kind of stereo image quality evaluation based on binocular competition
Method, as shown in Figure 1, including the following steps.
Step S101 is obtained with reference to stereo-picture and distortion stereo-picture, wherein the reference stereo-picture includes reference
Left view and refer to right view, the distortion stereo-picture include distortion left view and be distorted right view.
Step S102 synthesizes monocular reference picture by the reference left view and with reference to left view.
The distortion left view and distortion right view are synthesized monocular distorted image by step S103.
It is detailed, after obtaining with reference to stereo-picture and distortion stereo-picture, competed in human visual system's binocular special
Property guidance under, so that it may original and distortion stereo-picture left and right view is respectively synthesized as a width monocular image, and is applied
The vision energy of stereo-picture or so view carrys out the response of analog vision system, with the energy ratio of left and right view come analog vision system
The binocular Competition Characteristics of system.
The synthesis of monocular reference picture and monocular distorted image can be using the multiple Gabor filter function of two dimension to described
It is filtered respectively with reference to left view, with reference to right view, distortion left view and distortion right view.As shown in Fig. 2, synthesizing
In the process, left view can be set, right view is respectively ILAnd IR, the multiple Gabor filter function of two dimension are as follows:
Wherein, R1=xcos θ+ysin θ, R2=-sin θ+ycos θ, σx,σyRespectively indicate standard deviation, ζx,ζyIt respectively indicates
Spatial frequency, θ indicate filtering direction；
Be calculated using the following equation it is described with reference to left view, with reference to right view, distortion left view and be distorted right view power
Weight；
Wherein, described with reference to left view, energy response value of the distortion left view on all scales and direction is GEL, institute
Stating with reference to the energy response value of right view, distortion right view on all scales and direction is GER, and d is parallax.
According to it is described with reference to left view, with reference to right view, distortion left view and be distorted right view weight, using following public affairs
Formula synthesizes monocular reference picture and monocular distorted image:
C (x, y)=WL(x,y)IL(x,y)+WR((x+d),y)IR((x+d),y)。
Monocular reference picture and monocular distorted image after synthesis are denoted as R and D respectively.
Step S104, using Gauss scale mixed model to the wavelet systems of the monocular reference picture and monocular distorted image
Number is modeled.
Detailed, vision content weight calculation can unified representation are as follows:
W=I (R；E)+I(D；F)-I(E；F)
Wherein, w expression vision weight, I (；) indicate two variables mutual information.
Based on the unified representation formula of above-mentioned vision content weight, as shown in figure 3, the monocular can first be calculated with reference to figure
Picture and the mutual information I (R between the monocular reference picture after perceiving；E), wherein monocular reference picture and warp after perceiving
Monocular distorted image after perception is respectively E and F.Meanwhile it calculating the monocular distorted image and being distorted with the monocular after perceiving
Mutual information I (D between image；F monocular reference picture after perceiving and the monocular distorted image after perceiving are calculated) and
Between mutual information I (E；F).
When calculating each mutual information, can first calculate described with reference to stereo-picture, distortion perspective view, the ginseng after perceiving
Examine stereo-picture, the covariance for being distorted stereo-picture after perceiving.
Wherein, s indicates that multiplication factor, g indicate gain factor,Indicate perception visual noise variance,Indicate that Gauss makes an uproar
Sound variance, I indicate identity matrix.CUFor the covariance matrix of zero-mean gaussian vector U, using gauss hybrid models to reference to figure
As the wavelet coefficient of R is uniformly modeled, R is further represented by R=sU.Covariance matrix CUIt is expressed as
It is calculated using the following equation above-mentioned each mutual information.
In above formula, R, E；D, F；Covariance between E, F calculates as follows:
In summary the calculating of covariance and mutual information, can be further according to I (R；E),I(D；) and I (E F；F), count
Calculate vision content weight based on mutual information.
Vision content weight can further indicate that are as follows:
WhereinIndicate distortion noise variance,Indicate random distortion variance, g indicate gain factor, s indicate multiplication because
Son, λkIndicate k-th of characteristic value, wherein k=1,2 ..., K, this K characteristic value is by covariance matrix CUThrough Eigenvalues Decomposition institute
?.The calculating of covariance matrix is that the result of gauss hybrid models modeling is used to image.
Step S105 extracts the characteristic information of the monocular reference picture and monocular distorted image, the characteristic information packet
Include the structural information, gradient amplitude information and phase equalization information of monocular reference picture and the monocular distorted image.
Step S106, as shown in figure 4, being calculated according to the characteristic information of the monocular reference picture and monocular distorted image
The characteristic similarity of the monocular reference picture and monocular distorted image.
There are stronger correlation between the pixel of image, this correlation reflects the structure feature of picture altitude, people
It is more sensitive to features such as the brightness of image, contrast, structural informations during observing piece image, therefore, the present invention
Using brightness, contrast and structural information as the characteristic information of stereo-picture structural similarity, image is extracted separately below
Brightness, contrast and structural information.
It is bright by the image block x, y of same position in monocular reference picture and monocular distorted image as two input signals
Spend the calculating process of l (x, y), contrast c (x, y) and structural information s (x, y) are as follows:
Wherein, C1=(K1L)2,C2=(K2L)2,C3=C2/ 2, K1<<1,K2< < 1, L ∈ (0,255], μx, μyIt respectively indicates
The mean value of image block x, y, reflect the brightness case of image.σx,σyImage block x is respectively indicated, the variance of y reflects image
Contrast.σxyFor x, the similarity factor of y is represented byImage is reflected in structure
Correlation.
According to the brightness of image, contrast and structural information, structural similarity be may be defined as:
SSIM=[l (x, y)]α·[c(x,y)]β·[s(x,y)]γ
Wherein, α > 0, β > 0, γ > 0.α=1 can be set in the present invention, β=1, γ=1, structural similarity can be further
It indicates are as follows:
Wherein, C1,C2The constant for indicating a very little, is not 0 to control denominator.
When calculating gradient similarity, by the image block x, y of same position in monocular reference picture and monocular distorted image
As two input signals, the gradient value of two image blocks is calculated separately, value is the maximum weighted average value of image block.Image
The gradient value of block x are as follows:The gradient of image block y is calculated as*
Indicate convolution operation.Wherein Mk, k=1,2,3,4 indicate k-th of filter, and number of filter can be set to 4 in the present invention.
The mean function of mean2 () representing matrix.In view of visibility threshold (Visibility threshold) and image pair
Than the variation of degree, the gradient-structure similarity between image block x and image block y be may be expressed as:
Wherein, C1For the constant of a very little, it is not 0 and is arranged to control the value of denominator.gx,gyRespectively indicate image
The gradient of block x, y.Indicate gradient variation (masked gradient change), value range be [0,
1]。max(gx,gy) indicate mask gradient (masking gradient).
Phase equalization is able to maintain preferable consistency for the variation of brightness of image or contrast, will not be with image
Brightness and contrast variation and change.The phase equalization feature of image can measure the journey of image consistency well
Degree.The important feature that the present invention evaluates this characteristic as stereo image quality.A two dimensional logarithmic can be applied
Gabor filter carries out convolution algorithm with monocular reference picture and monocular distorted image respectively, obtains the phase of monocular reference picture
Bit integrity value is
PC (x) indicates the phase equalization value of image block x in monocular reference picture,It indicates in direction θjOn part
Energy, θjIndicate the deflection of two dimensional logarithmic Gabor filter, θj=j π/J, j={ 0,1,2 ..., J-1 }, J indicate direction number.Expression image block x is n, direction θ in scalejOn range value.ε can be a lesser constant greater than 0.
Similarly, the phase equalization value of monocular distorted image is
PC (y) indicates the phase equalization value of image block y in monocular reference picture,It indicates in direction θjOn part
Energy, θjIndicate the deflection of two dimensional logarithmic Gabor filter, θj=j π/J, j={ 0,1,2 ..., J-1 }, J indicate direction number.Expression image block y is n, direction θ in scalejOn range value.ε is a small constant, and is greater than 0.More than being based on
Process, phase equalization similarity is defined as:
Wherein SPCIndicate phase equalization similarity, C4Indicate the constant for being greater than 0 an of very little.
Step S107, according to the characteristic similarity of the monocular reference picture and monocular distorted image, in conjunction with vision content
Weight constructs stereo-picture evaluation method, establishes the subjective and objective consistency nonlinear function of image quality evaluation.
Step S108 objectively evaluates stereo image quality according to default stereoscopic image data library data.
Local similarity between monocular reference picture and monocular distorted image may be expressed as: S (R, D)=[Sssim]a·
[SG]b·[SPC]c, a, b, c be partial structurtes similarity, gradient similarity and phase equalization constant factor, the present invention in
A is set, and the value of b, c are 1.Next, in conjunction with vision content weight and local similarity, it is three-dimensional with reference to stereo-picture and distortion
Image quality evaluating value between image may be expressed as:
Wherein, M indicates scale parameter of the image through wavelet decomposition, wiIndicate on i-th of decomposition scale monocular reference picture and
Vision content weight between monocular distorted image.S(R,D)jIndicate j-th of image of monocular reference picture and monocular distorted image
Local similarity between subband.
It finally carries out curve fitting to image quality evaluating method, to measure the subjective and objective consistency of quality evaluation, will scheme
As the quality assessment process objectively evaluated regards the nonlinear function of subjective assessment, the function as is defined as:
Wherein, μ1,μ2,μ3,μ4,μ5Indicate nonlinear fitting parameter.
The present invention is directed to the deficiency of the stereo image quality evaluation method based on flat image quality evaluation strategy, utilizes height
This mixed model portrays the statistical distribution characteristic of stereo-picture, vision content weight based on mutual information is designed, to improve solid
The robustness and stability of image quality evaluating method.Meanwhile the present invention is based on the binocular Competition Characteristics of human visual system, answer
With the parallax information of stereo-picture or so view, and the structural information of comprehensive stereo-picture or so monocular view, Gradient Features
And phase equalization feature, devise a kind of stereo image quality evaluation method competed under guidance based on binocular.The present invention
Fusion based on multiple characteristic informations and the inherent perception characteristics of human visual system are considered, can effectively promote solid
The subjective and objective consistency of image quality evaluating method.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.It should also be noted that similar label and letter exist
Similar terms are indicated in following attached drawing, therefore, once being defined in a certain Xiang Yi attached drawing, are then not required in subsequent attached drawing
It is further defined and explained.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. a kind of stereo image quality evaluation method based on binocular competition characterized by comprising
It obtains with reference to stereo-picture and distortion stereo pairs, wherein the reference stereo-picture includes referring to left view and reference
Right view, the distortion stereo-picture include distortion left view and distortion right view；
Monocular reference picture is synthesized by the reference left view and with reference to left view；
The distortion left view and distortion right view are synthesized into monocular distorted image；
It is modeled using wavelet coefficient of the Gauss scale mixed model to the monocular reference picture and monocular distorted image；
The characteristic information of the monocular reference picture and monocular distorted image is extracted, the characteristic information includes monocular reference picture
With the structural information, gradient amplitude information and phase equalization information of the monocular distorted image；
According to the characteristic information of the monocular reference picture and monocular distorted image, calculates the monocular reference picture and monocular loses
The characteristic similarity of true image；
According to the characteristic similarity of the monocular reference picture and monocular distorted image, in conjunction with vision content weight, building is three-dimensional
Image evaluation method establishes the subjective and objective consistency nonlinear function of image quality evaluation, and according to default stereoscopic image data
Library data, objectively evaluate stereo image quality.
2. the stereo image quality evaluation method according to claim 1 based on binocular competition, which is characterized in that will be described
Monocular reference picture is synthesized with reference to left view and with reference to left view, and the distortion left view and distortion right view are synthesized
The step of monocular distorted image includes:
Using the multiple Gabor filter function of two dimension to the reference left view, with reference to right view, distortion left view and the right view of distortion
Figure is filtered respectively, wherein the multiple Gabor filter function of two dimension are as follows:
Wherein, R1=xcos θ+ysin θ, R2=-sin θ+ycos θ, σx,σyRespectively indicate standard deviation, ζx,ζyRespectively indicate space
Frequency, θ indicate filtering direction；
Be calculated using the following equation it is described with reference to left view, with reference to right view, distortion left view and be distorted right view weight；
Wherein, described with reference to left view, energy response value of the distortion left view on all scales and direction is GEL, the ginseng
Examining the energy response value of right view, distortion right view on all scales and direction is GER, and d is parallax；
According to it is described with reference to left view, with reference to right view, distortion left view and be distorted right view weight, closed using following formula
At monocular reference picture and monocular distorted image:
C (x, y)=WL(x,y)IL(x,y)+WR((x+d),y)IR((x+d),y)。
3. the stereo image quality evaluation method according to claim 1 based on binocular competition, which is characterized in that using high
The step of this scale mixed model models the wavelet coefficient of the monocular reference picture and monocular distorted image include:
Calculate the mutual information I (R between the monocular reference picture and monocular reference picture after perceiving；E), wherein through feeling
Monocular reference picture after knowing and the monocular distorted image after perceiving are respectively E and F；
Calculate the mutual information I (D between the monocular distorted image and monocular distorted image after perceiving；F)；
Calculate the monocular reference picture after perceiving and the mutual information I (E between the monocular distorted image after perceiving；F)；
According to I (R；E),I(D；) and I (E F；F), vision content weight based on mutual information is calculated.
4. the stereo image quality evaluation method according to claim 3 based on binocular competition, which is characterized in that according to I
(R；E),I(D；) and I (E F；F), the step of calculating vision content weight based on mutual information further include:
It calculates described three-dimensional with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion after perceiving
The covariance of image.
5. it is according to claim 4 based on binocular competition stereo image quality evaluation method, which is characterized in that use with
Lower formula calculates described vertical with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion after perceiving
The covariance of body image:
Wherein, s indicates that multiplication factor, g indicate gain factor,Indicate perception visual noise variance,Indicate Gaussian noise side
Difference, I indicate identity matrix, CUFor the covariance matrix of zero-mean gaussian vector U,
6. the stereo image quality evaluation method according to claim 4 based on binocular competition, which is characterized in that according to institute
State the association side with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion stereo-picture after perceiving
Difference, the mutual information I (R being calculated using the following equation between the monocular reference picture and monocular reference picture after perceiving；
E), the mutual information I (D between the monocular distorted image and monocular distorted image after perceiving；F the list) and after perceiving
Mutual information I (E between mesh reference picture and monocular distorted image after perceiving；F):
Wherein, the monocular reference image R, E are calculated using the following equation；D, F；Covariance between E, F:
7. the stereo image quality evaluation method according to claim 4 based on binocular competition, which is characterized in that according to institute
State the association side with reference to stereo-picture, distortion perspective view, the reference stereo-picture after perceiving, the distortion stereo-picture after perceiving
Mutual information I (R between difference and the monocular reference picture and monocular reference picture after perceiving；E), the monocular loses
Mutual information I (D between true image and monocular distorted image after perceiving；F monocular reference picture) and after perceiving with
Mutual information I (E between monocular distorted image after perceiving；F), it is calculated using the following equation the vision content weight:
Wherein,Indicate distortion noise variance,Indicate random distortion variance, g indicates that gain factor, s indicate multiplication factor, λk
Indicate k-th of characteristic value, wherein k=1,2 ..., K.
8. the stereo image quality evaluation method according to claim 1 based on binocular competition, which is characterized in that according to institute
The characteristic information for stating monocular reference picture and monocular distorted image calculates the spy of the monocular reference picture and monocular distorted image
Levy similarity the step of include:
Calculate the brightness of the image block of same position in the monocular reference picture and monocular distorted image, contrast and structure letter
Breath；
Calculate the structural similarity of the monocular reference picture and monocular distorted image；
Calculate the gradient similarity of the monocular reference picture and monocular distorted image；
Calculate the phase equalization similarity of the monocular reference picture and monocular distorted image.
9. it is according to claim 8 based on binocular competition stereo image quality evaluation method, which is characterized in that use with
Lower formula calculates brightness, contrast and the structure of the image block of same position in the monocular reference picture and monocular distorted image
Information:
Wherein, C1=(K1L)2,C2=(K2L)2,C3=C2/ 2, K1<<1,K2< < 1, L ∈ (0,255], μx,μyRespectively indicate image
The mean value of block x, y；σx,σyRespectively indicate image block x, the variance of y；σxyFor x, the similarity factor of y,
It is calculated using the following equation the structural similarity of the monocular reference picture and monocular distorted image:
SSIM=[l (x, y)]α·[c(x,y)]β·[s(x,y)]γ
Wherein α > 0, β > 0, γ > 0, C1,C2For preset constant；
It is calculated using the following equation the gradient similarity of the monocular reference picture and monocular distorted image:
gx,gyRespectively indicate image block x, the gradient of y,Indicate the variation of gradient, max (gx,gy) indicate mask
Gradient；
It is calculated using the following equation the phase equalization similarity of the monocular reference picture and monocular distorted image:
Wherein, SPCIndicate phase equalization similarity, C4Preset constant greater than 0, PC (x) are indicated in the monocular reference picture
The phase equalization value of image block x, PC (y) indicate the phase equalization value of image block y in the monocular reference picture.
10. the stereo image quality evaluation method according to claim 1 based on binocular competition, which is characterized in that according to
The characteristic similarity and vision content weight of the monocular reference picture and monocular distorted image construct stereo-picture evaluation side
Method, the step of establishing the subjective and objective consistency nonlinear function of image quality evaluation include:
It is calculated using the following equation the image quality evaluating value S with reference between stereo-picture and distortion stereo-pictureQA:
Wherein, M indicates scale parameter of the image through wavelet decomposition, wiIndicate the monocular reference picture and list on i-th of decomposition scale
Vision content weight between mesh distorted image, S (R, D)jIndicate j-th of figure of the monocular reference picture and monocular distorted image
As the local similarity between subband；
It carries out curve fitting to image quality evaluating method, establishes the nonlinear function of subjective assessment:
Wherein μ1,μ2,μ3,μ4,μ5Indicate nonlinear fitting parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811469588.XA CN109525838A (en) | 2018-11-28 | 2018-11-28 | Stereo image quality evaluation method based on binocular competition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811469588.XA CN109525838A (en) | 2018-11-28 | 2018-11-28 | Stereo image quality evaluation method based on binocular competition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109525838A true CN109525838A (en) | 2019-03-26 |
Family
ID=65794162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811469588.XA Pending CN109525838A (en) | 2018-11-28 | 2018-11-28 | Stereo image quality evaluation method based on binocular competition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109525838A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140168388A1 (en) * | 2012-12-19 | 2014-06-19 | Nvidia Corporation | System and method for displaying a three-dimensional image on a video monitor |
| CN104853182A (en) * | 2015-05-21 | 2015-08-19 | 天津大学 | Amplitude and phase based stereo image quality objective evaluation method |
| CN107578404A (en) * | 2017-08-22 | 2018-01-12 | 浙江大学 | The complete of view-based access control model notable feature extraction refers to objective evaluation method for quality of stereo images |
| CN108898600A (en) * | 2018-07-11 | 2018-11-27 | 上饶师范学院 | Image quality evaluating method and device |
-
2018
- 2018-11-28 CN CN201811469588.XA patent/CN109525838A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140168388A1 (en) * | 2012-12-19 | 2014-06-19 | Nvidia Corporation | System and method for displaying a three-dimensional image on a video monitor |
| CN104853182A (en) * | 2015-05-21 | 2015-08-19 | 天津大学 | Amplitude and phase based stereo image quality objective evaluation method |
| CN107578404A (en) * | 2017-08-22 | 2018-01-12 | 浙江大学 | The complete of view-based access control model notable feature extraction refers to objective evaluation method for quality of stereo images |
| CN108898600A (en) * | 2018-07-11 | 2018-11-27 | 上饶师范学院 | Image quality evaluating method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105959684B (en) | Stereo image quality evaluation method based on binocular fusion | |
| CN103763552B (en) | Stereoscopic image non-reference quality evaluation method based on visual perception characteristics | |
| CN105338343B (en) | It is a kind of based on binocular perceive without refer to stereo image quality evaluation method | |
| CN106504222A (en) | A kind of Polarization Image Fusion system under water based on bionic visual mechanism | |
| CN101610425B (en) | Method for evaluating stereo image quality and device | |
| Su et al. | Color and depth priors in natural images | |
| Liu et al. | Statistical modeling of 3-D natural scenes with application to Bayesian stereopsis | |
| CN107635136B (en) | View-based access control model perception and binocular competition are without reference stereo image quality evaluation method | |
| CN108389192A (en) | Stereo-picture Comfort Evaluation method based on convolutional neural networks | |
| CN104318545B (en) | A kind of quality evaluating method for greasy weather polarization image | |
| CN109831664B (en) | Rapid compressed stereo video quality evaluation method based on deep learning | |
| Ma et al. | Reduced-reference stereoscopic image quality assessment using natural scene statistics and structural degradation | |
| CN108765414A (en) | Based on wavelet decomposition and natural scene count without referring to stereo image quality evaluation method | |
| CN109242812A (en) | Image interfusion method and device based on conspicuousness detection and singular value decomposition | |
| CN109523513A (en) | Based on the sparse stereo image quality evaluation method for rebuilding color fusion image | |
| Geng et al. | A stereoscopic image quality assessment model based on independent component analysis and binocular fusion property | |
| CN108520510B (en) | No-reference stereo image quality evaluation method based on overall and local analysis | |
| CN104954778A (en) | Objective stereo image quality assessment method based on perception feature set | |
| CN109788275A (en) | Naturality, structure and binocular asymmetry are without reference stereo image quality evaluation method | |
| CN105894507B (en) | Image quality evaluating method based on amount of image information natural scene statistical nature | |
| CN102821301B (en) | HVS (Human Visual System)-based objective quality evaluation method for three-dimensional (3D) information | |
| Yang et al. | No-reference stereoimage quality assessment for multimedia analysis towards Internet-of-Things | |
| CN104918039B (en) | image quality evaluating method and system | |
| Kim et al. | Quality assessment of perceptual crosstalk on two-view auto-stereoscopic displays | |
| Chamaret et al. | No-reference harmony-guided quality assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190326 |
|
| RJ01 | Rejection of invention patent application after publication |