CN110930398A - Log-Gabor similarity-based full-reference video quality evaluation method - Google Patents

Log-Gabor similarity-based full-reference video quality evaluation method Download PDF

Info

Publication number
CN110930398A
CN110930398A CN201911250288.7A CN201911250288A CN110930398A CN 110930398 A CN110930398 A CN 110930398A CN 201911250288 A CN201911250288 A CN 201911250288A CN 110930398 A CN110930398 A CN 110930398A
Authority
CN
China
Prior art keywords
component
similarity
video
amplitude
total
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911250288.7A
Other languages
Chinese (zh)
Other versions
CN110930398B (en
Inventor
汪斌
陈淑聪
姜飞龙
朱海滨
毛凌航
徐翘楚
张奥
李兴隆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiaxing University
Original Assignee
Jiaxing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiaxing University filed Critical Jiaxing University
Priority to CN201911250288.7A priority Critical patent/CN110930398B/en
Publication of CN110930398A publication Critical patent/CN110930398A/en
Application granted granted Critical
Publication of CN110930398B publication Critical patent/CN110930398B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

The invention discloses a full-reference video quality evaluation method based on Log-Gabor similarity, which comprises the steps of firstly carrying out Log-Gabor transformation on U and V components of a video frame to obtain the amplitude and phase of a transformation coefficient, calculating the amplitude similarity and the phase similarity, and comprehensively obtaining the Log-Gabor similarity; then constructing stereo LBP characteristics for Y video components of three adjacent frames, extracting a stereo LBP characteristic histogram, and calculating stereo LBP characteristic similarity of a distorted video and a reference video; and finally, combining the log-Gabor similarity and the stereo LBP characteristic similarity to obtain the total similarity which is used as an objective video quality evaluation result. The invention fully considers the Y, U of the video and the transformation domain and space domain characteristics of the V component, and adopts the stereo LBP characteristics to extract the time domain characteristics, thereby improving the video quality evaluation precision.

Description

Log-Gabor similarity-based full-reference video quality evaluation method
Technical Field
The invention belongs to the field of video processing, and particularly relates to a full-reference video quality evaluation method based on Log-Gabor similarity.
Background
Video quality evaluation is a key problem in the field of video processing, and video quality evaluation methods can be divided into subjective video quality evaluation methods and objective video quality evaluation methods according to whether people participate in the video quality evaluation methods. The subjective video quality evaluation method is characterized in that people score images, the evaluation result is accurate, but the evaluation process is complex, the time consumption is long, and real-time application is difficult to achieve. The objective video quality evaluation method does not need human participation, automatically predicts the image quality through a specific computer algorithm, and can be divided into a full-reference video quality evaluation method, a half-reference video quality evaluation method and a no-reference video quality evaluation method according to whether an original distortion-free video is used as a reference. The full-reference video quality evaluation method predicts the video quality by using all information of the reference video, the semi-reference video quality evaluation method predicts the video quality by using partial information of the reference video, and the no-reference video quality evaluation method does not evaluate the video quality by using any information of the reference video. The traditional full-reference video quality evaluation method adopts MSE (mean square error) or PSNR (peak signal to noise ratio) to evaluate the video quality, has definite physical significance and simple algorithm, has the defects of mismatching with subjective visual characteristics of human eyes and the like, and cannot be widely applied in practical occasions. For Video Quality evaluation, many scholars propose improved methods, Zhang [ y.zhang, x. -b.gao, l.he, w.lu, r.he.objective Video Quality Assessment association Combining Transfer Learning with cnn.ieee Transactions on Neural Networks and Learning Systems (IEEE TNNLS),2019] adopt convolutional Neural network merging migration Learning to perform Video image Quality evaluation; lu [ w.lu, r.he, j.yang, c.jia, x. -b.gao.a spatiot Model of Video Quality Assessment via 3DGradient difference information Science, vol.478, pp.141-151,2019 ] proposes a Video Quality Assessment method based on three-dimensional gradient difference, and although these methods improve the accuracy of non-reference image Quality Assessment, the results thereof are still different from the subjective image Quality Assessment results of human eyes.
Disclosure of Invention
The invention aims to provide a full-reference video quality evaluation method based on Log-Gabor similarity aiming at the defects of the prior art.
The purpose of the invention is realized by the following technical scheme: a full reference video quality evaluation method based on Log-Gabor similarity comprises the following steps:
(1) respectively inputting a reference video and a distortion video of a training video set and a video set to be tested;
(2) respectively extracting a Y component, a U component and a V component of a frame of video in a reference video and a distorted video of a video set to be detected, and filtering by using Log-Gabor filters with different scales and different directions, wherein the method comprises the following substeps:
(2.1) constructing a frequency domain Log-Gabor filter G (omega) with S scales and O directions, wherein the frequency domain expression is as follows:
Figure BDA0002308827390000021
where ω is an angular frequency variable, ω0Is the center frequency, σ, of the filterωIs the filter variance;
(2.2) filtering the extracted Y component, U component and V component by using the filter G (omega) constructed in the step (2.1) to obtain filter coefficients of the Y component, the U component and the V component, which are recorded as GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n); wherein s is a scale index of the filter coefficient, o is a direction index of the filter coefficient, m is a row index of the Y-component filter coefficient, and n is a column index of the Y-component filter coefficient; the filter coefficients GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n) are complex numbers;
(2.3) respectively calculating the amplitude and the phase of the filter coefficients of the Y component, the U component and the V component obtained in the step (2.2), wherein the calculation formula is as follows:
Figure BDA0002308827390000022
Figure BDA0002308827390000023
wherein, MY (s, o, m, n) is the amplitude of the Y component filter coefficient, AY (s, o, m, n) is the phase of the Y component filter coefficient, and R _ GY (s, o, m, n) and I _ GY (s, o, m, n) are the real part and imaginary part of the Y component filter coefficient GY (s, o, m, n); similarly, the amplitude MU (s, o, m, n), the phase AU (s, o, m, n) of the U component filter coefficient, the amplitude MV (s, o, m, n) of the V component filter coefficient, and the phase AV (s, o, m, n) can be obtained;
(3) respectively calculating the amplitude similarity and the phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure BDA0002308827390000024
Figure BDA0002308827390000025
wherein S _ MY (S, o, m, n) is the amplitude similarity of the Y component, and S _ AY (S, o, m, n) is the phase similarity of the Y component; c1And C2A constant set to avoid the denominator being zero; MY _ D (S, o, m, n) and AY _ D (S, o, m, n) are amplitude and phase of Y component filter coefficients of the distorted video, and MY _ S (S, o, m, n) and AY _ S (S, o, m, n) are amplitude and phase of Y component filter coefficients of the reference video; similarly, the amplitude similarity S _ MU (S, o, m, n), the angle similarity S _ AU (S, o, m, n) of the U component, the amplitude similarity S _ MV (S, o, m, n), the angle similarity S _ AV (S, o, m, n), C of the V component can be obtained1And C2A constant set to avoid the denominator being 0;
(4) and calculating the total amplitude similarity and the total phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure BDA0002308827390000031
Figure BDA0002308827390000032
wherein, S _ MY is the total amplitude similarity of the Y component filter coefficients, S _ AY is the total angle similarity of the Y component filter coefficients, and M, N is the total row number and the total column number of the certain scale and direction filter coefficients of the Y component; similarly, the total amplitude similarity S _ MU, the total angle similarity S _ AU and the total amplitude similarity S _ MV and the total angle similarity S _ AV of the U component filter coefficient and the V component filter coefficient can be obtained;
(5) calculating filter coefficient similarity S1The calculation formula is as follows:
SY＝ω1×S_MY+ω2×S_AY
SU＝ω1×S_MU+ω2×S_AU
SV＝ω1×S_MV+ω2×S_AV
S1＝ω3×SY+ω4×SU+ω5×SV
wherein S isYIs the total similarity of the Y components, SUIs the total similarity of the U components, SVIs the total similarity of the V components, ω1、ω2、ω3、ω4And ω5Are all weighting coefficients;
(6) respectively applying cube local binary pattern operators to the training video set and the video set to be tested to obtain local binary pattern feature vectors:
(6.1) applying a stereo local binary pattern operator to the Y components of the reference video and the distorted video in the training video set to obtain a vector L1And opposite vector L1Obtaining E clustering centers by using a K nearest neighbor clustering method;
(6.2) applying a stereo local binary pattern operator to the Y component of the concentrated distorted video of the video to be detected to obtain a vector L2And using K nearest neighbor clustering method to cluster vector L2E clustering centers obtained in the step (6.1) are classified; according to the vectors L divided in the E clustering centers2Constructing a local binary pattern histogram according to the number of the feature vectors P, and obtaining corresponding feature vectors P; similarly, applying a three-dimensional local binary pattern operator to the Y component of the reference video in the video set to be detected and constructing a local binary pattern histogram to obtain a feature vector Q;
(7) calculating stereo local binary pattern feature similarity S of reference video and distorted video in to-be-detected video set2The calculation formula is as follows:
Figure BDA0002308827390000041
wherein, C3Is a constant, is a vector inner product operator, | | | | · | | | is a modular operation of solving the vector;
(8) according to the filter coefficient similarity S obtained in the step (5)1And the stereo local binary pattern feature similarity S obtained in the step (7)2Calculating the visual similarity theta of the reference video in the video set to be detected and the current frame of the distorted video according to the following formula;
θ＝ω6S1+ω7S2
wherein, ω is6、ω7Is a weighting coefficient;
(9) obtaining the visual similarity of other frames of the video set to be detected through the steps (2) to (8), and recording the visual similarity of the t-th frame as thetat(ii) a And calculating a video image quality evaluation score Z, wherein the calculation formula is as follows:
Figure BDA0002308827390000042
and T is the total frame number of the reference video or the distorted video in the video set to be detected.
Further, the amplitude MY _ D (S, o, m, n), the phase AY _ D (S, o, m, n) of the Y component filter coefficient of the distorted video in the step (3), and the amplitude MY _ S (S, o, m, n) and the phase AY _ S (S, o, m, n) of the Y component filter coefficient of the reference video are calculated by the formula in the step (2.3).
Further, the applying a stereo local binary pattern operator in step (6) includes the following sub-steps:
(61) acquiring a Y component of the YUV video, and extracting a pixel point Y of the Y component at an (i, j) position in a t-th framet(i, j) and 8 pixel points of neighborhood thereof are respectively Yt(i-1,j-1)、Yt(i-1,j)、Yt(i-1,j+1)、Yt(i,j-1)、Yt(i,j+1)、Yt(i+1,j-1)、Yt(i +1, j) and Yt(i+1,j+1)；
(62) Extracting pixel point Y of (i, j) position of Y component in t-1 framet-1(i, j) and 8 pixel points of neighborhood thereof are respectively Yt-1(i-1,j-1)、Yt-1(i-1,j)、Yt-1(i-1,j+1)、Yt-1(i,j-1)、Yt-1(i,j+1)、Yt-1(i+1,j-1)、Yt-1(i +1, j) and Yt-1(i+1,j+1)；
(63) Extracting pixel point Y of (i, j) position of Y component in t +1 th framet+1(i, j) and 8 pixel points of neighborhood thereof are respectively Yt+1(i-1,j-1)、Yt+1(i-1,j)、Yt+1(i-1,j+1)、Yt+1(i,j-1)、Yt+1(i,j+1)、Yt+1(i+1,j-1)、Yt+1(i +1, j) and Yt+1(i+1,j+1)；
(64) With Y in the t-th framet(i, j) as the center pixel point, and Y is compared by the following formulat(i, j) and the size of the 3 x 3 neighborhood pixels in the space domain and the time domain extracted in the steps (61) to (63) to obtain a comparison result sk(a,b)：
Figure BDA0002308827390000051
Wherein, Yk(a, b) is Yt(i, j) the neighborhood pixel value, k belongs to { t-1, t, t +1}, a belongs to { i-1, i, i +1}, and b belongs to { j-1, j, j +1 };
(65) comparing the result s obtained in step (64)k(a, b) according to st-1(i-1,j-1)、st-1(i-1,j)、st-1(i-1,j+1)、st-1(i,j-1)、st-1(i,j)、st-1(i,j+1)、st-1(i+1,j-1)、st-1(i+1,j)、st-1(i +1, j +1) and st(i-1,j-1)、st(i-1,j)、st(i-1,j+1)、st(i,j-1)、st(i,j+1)、st(i+1,j-1)、st(i+1,j)、st(i +1, j +1) and st+1(i-1,j-1)、st+1(i-1,j)、st+1(i-1,j+1)、st+1(i,j-1)、st+1(i,j)、st+1(i,j+1)、st+1(i+1,j-1)、st+1(i+1,j)、st+1The sequential arrangement of (i +1, j +1) constitutes a vector L.
Further, the vector L ∈ R in the step (65)26×1。
Further, the feature vector P, Q ∈ R in the step (6.2)E×1。
Further, the total frame number of the reference video and the distortion video is the same; the reference video and the distortion video are YUV videos.
The invention has the beneficial effects that: firstly, carrying out Log-Gabor transformation on U and V components of a video frame to obtain the amplitude and phase of a transformation coefficient, calculating the amplitude similarity and the phase similarity, and synthesizing to obtain Log-Gabor similarity; then, constructing stereo LBP (Local Binary Patterns) characteristics for Y video components of three adjacent frames, extracting a stereo LBP characteristic histogram, and calculating stereo LBP similarity of a distorted video and a reference video; and finally, obtaining the total similarity by combining the log-Gabor similarity and the stereo LBP similarity, and taking the total similarity as an objective video quality evaluation result. The method fully considers the Y, U of the video and the transformation domain and spatial domain characteristics of the V component, and adopts the stereo LBP characteristics to extract the time domain characteristics, thereby improving the video quality evaluation precision.
Drawings
Fig. 1 is a flow chart of a full-reference video quality evaluation method based on Log-Gabor similarity.
Detailed Description
The invention is described in detail below with reference to the accompanying drawings and examples. In specific implementation, a LIVE video database is used as an experiment database; the database contains 160 videos, the 160 videos are divided into 10 groups, each group contains 1 reference video and 15 distorted videos, and the 15 distorted videos in each group contain four types of wireless distortion, IP distortion, h.264 compression distortion and MPEG-2 compression distortion.
The specific steps adopted by the invention are shown in fig. 1, and comprise:
step (1): inputting a reference YUV video and a distorted YUV video, randomly selecting 20% of the reference YUV video and the distorted YUV video as a training video set, and selecting 80% of the reference YUV video and the distorted YUV video as a test video set;
step (2): respectively extracting a Y component, a U component and a V component of a frame of video in a YUV video sequence from a reference YUV video and a distorted YUV video of the test video set; filtering the extracted Y component, U component and V component by using Log-Gabor filters with different scales and different directions respectively, and comprising the following substeps:
step (2.1): constructing a frequency domain Log-Gabor filter with S scales and O directions, wherein the calculation formula is as follows:
Figure BDA0002308827390000061
where ω is an angular frequency variable, ω0Is the center frequency, σ, of the filterωFor the variance of the filter, exp (-) is an exponential operation, ln (-) is a logarithm operation, G (omega) is a frequency domain expression of the Log-Gabor filter, S is 3, and O is 4;
step (2.2): filtering the Y component, the U component and the V component extracted in the step (1) by using the filter constructed in the step (2.1) to respectively obtain filter coefficients of the Y component, the U component and the V component, and recording the filter coefficients as GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n); wherein GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n) are complex numbers, s is a scale index of the filter coefficients, o is a direction index of the filter coefficients, m is a row index of the Y component filter coefficients, and n is a column index of the Y component filter coefficients;
step (2.3): calculating the amplitudes MY (s, o, m, n), MU (s, o, m, n), MV (s, o, m, n) and phases AY (s, o, m, n), AU (s, o, m, n), AV (s, o, m, n) of the filter coefficients GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n) of the Y component, U component and V component in the step (2.2), wherein the calculation formula is as follows:
Figure BDA0002308827390000062
Figure BDA0002308827390000071
Figure BDA0002308827390000072
Figure BDA0002308827390000073
Figure BDA0002308827390000074
Figure BDA0002308827390000075
wherein, R _ GY (s, o, m, n) and I _ GY (s, o, m, n) are the real part and the imaginary part of the Y component filter coefficient GY (s, o, m, n); r _ GU (s, o, m, n) and I _ GU (s, o, m, n) are the real part and the imaginary part of the U component filter coefficient GU (s, o, m, n); r _ GV (s, o, m, n), I _ GV (s, o, m, n) are the real part and imaginary part of the V component filter coefficient GV (s, o, m, n);
and (3): respectively calculating the amplitude similarity and the phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure BDA0002308827390000076
Figure BDA0002308827390000077
wherein S _ MY (S, o, m, n) is the amplitude similarity of the Y component, and S _ AY (S, o, m, n) is the phase similarity of the Y component; c1And C2A constant set to avoid the denominator being zero; MY _ D (S, o, m, n) is the amplitude of the Y component filter coefficient of the distorted video, MY _ S (S, o, m, n) is the amplitude of the Y component filter coefficient of the reference video, AY _ D (S, o, m, n) is the phase of the Y component filter coefficient of the distorted video, and AY _ S (S, o, m, n) is the amplitude of the Y component filter coefficient of the reference video; similarly, the amplitude similarity S _ MU (S, o, m, n), the angle similarity S _ AU (S, o, m, n) of the U component, the amplitude similarity S _ MV (S, o, m, n), the angle similarity S _ AV (S, o, m, n), C of the V component can be obtained1And C2Constants set to avoid denominator being 0, C1And C2The value is 0.01;
and (4): and calculating the total amplitude similarity and the total phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure BDA0002308827390000082
Figure BDA0002308827390000081
wherein S _ MY is the total amplitude similarity of the Y component filter coefficients, S _ AY is the total angle similarity of the Y component filter coefficients, M is a certain scale and the total row number of the direction filter coefficients of the Y component, N is a certain scale and the total column number of the direction filter coefficients of the Y component, S is the total scale number of the filter coefficients, and O is the total direction number of the filter coefficients; similarly, the total amplitude similarity S _ MU, the total angle similarity S _ AU and the total amplitude similarity S _ MV and the total angle similarity S _ AV of the U component filter coefficient and the V component filter coefficient can be obtained;
and (5): and calculating to obtain the total filter coefficient similarity S by combining the total amplitude similarity and the total phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions1The calculation formula is as follows:
SY＝ω1×S_MY+ω2×S_AY
SU＝ω1×S_MU+ω2×S_AU
SV＝ω1×S_MV+ω2×S_AV
S1＝ω3×SY+ω4×SU+ω5×SV
wherein S isYIs the total similarity of the Y components, SUIs the total similarity of the U components, SVIs the total similarity of the V components, ω1、ω2、ω3、ω4And ω5For weighting coefficients, set by human beings, omega1、ω2Value of 0.5, omega3Value of 0.75, omega4And ω5A value of 0.125;
and (6): applying stereo Local Binary Pattern (LBP) operators to YUV reference video and YUV distorted video components in the test video set and the training video set, respectively, comprising the following sub-steps:
step (6.1): extracting a pixel point Y at the (i, j) position of the Y video in the t framet(i, j) and obtaining 8 pixel points in the neighborhood of the pixel position of (i, j), wherein the pixel positions are Yt(i-1,j-1)、Yt(i-1,j)、Yt(i-1,j+1)、Yt(i,j-1)、Yt(i,j+1)、Yt(i+1,j-1)、Yt(i +1, j) and Yt(i +1, j +1), wherein t is the frame number of the current frame, t-1 is the frame number of the previous frame, and t +1 is the frame number of the next frame;
step (6.2): extracting pixel points Y of the Y video at the t-1 th frame and the (i, j) positiont-1(i, j) and obtaining 8 pixel points of t-1 th frame in the neighborhood of the (i, j) position, wherein the pixel points are Y respectivelyt-1(i-1,j-1)、Yt-1(i-1,j)、Yt-1(i-1,j+1)、Yt-1(i,j-1)、Yt-1(i,j+1)、Yt-1(i+1,j-1)、Yt-1(i +1, j) and Yt-1(i+1,j+1)；
Step (6.3): extracting pixel points Y of the Y video at the t +1 th frame and the (i, j) positiont+1(i, j) and obtaining 8 pixel points of t +1 th frame in the neighborhood of the (i, j) position, wherein the pixel points are Yt+1(i-1,j-1)、Yt+1(i-1,j)、Yt+1(i-1,j+1)、Yt+1(i,j-1)、Yt+1(i,j+1)、Yt+1(i+1,j-1)、Yt+1(i +1, j) and Yt+1(i+1,j+1)；
Step (6.4): with Y in the t-th framet(i, j) as the center pixel point, and comparing Yt(i, j) the size of the 3 × 3 neighborhood pixels in the space domain and the time domain, if the neighborhood pixel value is greater than or equal to the central pixel value, the comparison result is marked as 1, otherwise, the comparison result is 0; the comparative formula is as follows:
Figure BDA0002308827390000091
wherein, Yk(m, n) is Yt(i, j) neighborhood pixel value, k belongs to { t-1, t, t +1}, m belongs to { i-1, i, i +1}, n belongs to { j-1, j, j +1}, sk(m, n) is the comparison result of the (m, n) pixel positions;
step (6.5): will sk(m, n) as st-1(i-1,j-1)、st-1(i-1,j)、st-1(i-1,j+1)、st-1(i,j-1)、st-1(i,j)、st-1(i,j+1)、st-1(i+1,j-1)、st-1(i+1,j)、st-1(i +1, j +1) and st(i-1,j-1)、st(i-1,j)、st(i-1,j+1)、st(i,j-1)、st(i,j+1)、st(i+1,j-1)、st(i+1,j)、st(i +1, j +1) and st+1(i-1,j-1)、st+1(i-1,j)、st+1(i-1,j+1)、st+1(i,j-1)、st+1(i,j)、st+1(i,j+1)、st+1(i+1,j-1)、st+1(i+1,j)、st+1The sequential arrangement of (i +1, j +1) constitutes a vector L, where L ∈ R26×1；
Step (6.6): extracting a vector L from the reference video and the distorted video in the training video set according to the steps (6.1) - (6.5)1Obtaining E clustering centers by adopting a K-nearest neighbor (KNN) clustering method, wherein the value of E is 32;
step (6.7): extracting a vector L from the distorted video in the test video set to each pixel point according to the steps (6.1) - (6.5)2And applying KNN methodClassifying it into E clusters;
step (6.8): vector L classified into E cluster centers according to distorted videos in test video set2Constructing a local binary pattern histogram by the number to obtain a corresponding local binary pattern feature vector P, wherein P belongs to RE×1；
Step (6.9): processing the Y video component of the reference video in the test video set in the same way through the steps (6.1) - (6.8), applying a cube local binary pattern operator to obtain a local binary pattern feature vector Q, wherein Q belongs to RE×1；
And (7): calculating stereo local binary pattern feature similarity S of reference video and distorted video in test video set2The calculation formula is as follows:
Figure BDA0002308827390000101
wherein, C3Constants set to avoid denominator being 0, C3The value is 0.01, the operator is the inner product of the vector, | | | · | | is the modular operation of solving the vector;
and (8): combining the Log-Gabor filter coefficient similarity S obtained in the step (5)1And (7) calculating the similarity S of the characteristics of the three-dimensional local binary pattern2To obtain the visual similarity theta of the current framet；
θt＝ω6S1+ω7S2
Wherein, ω is6、ω7For weighting coefficients, set by human beings, omega6And ω7The values are 0.8 and 0.2 respectively;
and (9): and calculating to obtain a final video image quality evaluation score Z, wherein the calculation formula is as follows:
Figure BDA0002308827390000102
where T is the total number of frames of the reference video or the distorted video.
The Log-Gabor filter is more consistent with the nonlinear logarithmic characteristic of human eye vision, is consistent with the visual perception process of human eyes, and obtains the similarity of the filter coefficients in different scales and different directions by extracting the Log-Gabor filter coefficients in multiple scales and multiple directions and comparing the Log-Gabor filter coefficients of the reference video and the distorted video, and synthesizes the obtained total Log-Gabor similarity, wherein the similarity is more consistent with the visual characteristic of human eyes; meanwhile, the stereo LBP feature fuses texture features of a space domain and a time domain of the video, and the obtained stereo LBP feature similarity fully considers the texture distortion degree of the distorted video in an adjacent space domain and the time domain, so that the video image quality evaluation score obtained by fusing the Log-Gabor similarity and the stereo LBP feature similarity can fully reflect the distortion degree of the video. Wherein, the larger the video image quality evaluation score Z is, the better the video quality is.

Claims (6)

1. A full reference video quality evaluation method based on Log-Gabor similarity is characterized by comprising the following steps:
(1) respectively inputting a training video set and a video set to be tested, wherein the input training video set and the video set to be tested both comprise a reference video and a distortion video.
(2) Respectively extracting a Y component, a U component and a V component of a frame of video in a reference video and a distorted video of a video set to be detected, and filtering by using Log-Gabor filters with different scales and different directions, wherein the method comprises the following substeps:
(2.1) constructing a frequency domain Log-Gabor filter G (omega) with S scales and O directions, wherein the frequency domain expression is as follows:
Figure FDA0002308827380000011
where ω is an angular frequency variable, ω0Is the center frequency, σ, of the filterωIs the filter variance.
(2.2) filtering the extracted Y component, U component and V component by using the filter G (omega) constructed in the step (2.1) to obtain filter coefficients of the Y component, the U component and the V component, which are recorded as GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n); wherein s is a scale index of the filter coefficient, o is a direction index of the filter coefficient, m is a row index of the Y-component filter coefficient, and n is a column index of the Y-component filter coefficient; the filter coefficients GY (s, o, m, n), GU (s, o, m, n) and GV (s, o, m, n) are complex numbers.
(2.3) respectively calculating the amplitude and the phase of the filter coefficients of the Y component, the U component and the V component obtained in the step (2.2), wherein the calculation formula is as follows:
Figure FDA0002308827380000012
Figure FDA0002308827380000013
wherein, MY (s, o, m, n) is the amplitude of the Y component filter coefficient, AY (s, o, m, n) is the phase of the Y component filter coefficient, and R _ GY (s, o, m, n) and I _ GY (s, o, m, n) are the real part and imaginary part of the Y component filter coefficient GY (s, o, m, n); similarly, the amplitude MU (s, o, m, n), the phase AU (s, o, m, n) of the U component filter coefficient, the amplitude MV (s, o, m, n) of the V component filter coefficient, and the phase AV (s, o, m, n) can be obtained;
(3) respectively calculating the amplitude similarity and the phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure FDA0002308827380000021
Figure FDA0002308827380000022
wherein S _ MY (S, o, m, n) is the amplitude similarity of the Y component, and S _ AY (S, o, m, n) is the phase similarity of the Y component; c1And C2A constant set to avoid the denominator being zero; MY _ D (S, o, m, n) and AY _ D (S, o, m, n) are amplitude and phase of Y component filter coefficients of the distorted video, and MY _ S (S, o, m, n) and AY _ S (S, o, m, n) are amplitude and phase of Y component filter coefficients of the reference video; similarly, the amplitude similarity S _ MU (S, o, m, n), the angle similarity S _ AU (S, o, m, n) of the U component, and the amplitude of the V component can be obtainedDegree similarity S _ MV (S, o, m, n), angle similarity S _ AV (S, o, m, n), C1And C2A constant set to avoid the denominator being 0;
(4) and calculating the total amplitude similarity and the total phase similarity of the filter coefficients of the Y component, the U component and the V component in different scales and directions, wherein the calculation formula is as follows:
Figure FDA0002308827380000023
Figure FDA0002308827380000024
wherein, S _ MY is the total amplitude similarity of the Y component filter coefficients, S _ AY is the total angle similarity of the Y component filter coefficients, and M, N is the total row number and the total column number of the certain scale and direction filter coefficients of the Y component; similarly, the total amplitude similarity S _ MU, the total angle similarity S _ AU and the total amplitude similarity S _ MV and the total angle similarity S _ AV of the U component filter coefficient and the V component filter coefficient can be obtained;
(5) calculating filter coefficient similarity S1The calculation formula is as follows:
SY＝ω1×S_MY+ω2×S_AY
SU＝ω1×S_MU+ω2×S_AU
SV＝ω1×S_MV+ω2×S_AV
S1＝ω3×SY+ω4×SU+ω5×SV
wherein S isYIs the total similarity of the Y components, SUIs the total similarity of the U components, SVIs the total similarity of the V components, ω1、ω2、ω3、ω4And ω5Are all weighting coefficients;
(6) respectively applying cube local binary pattern operators to the training video set and the video set to be tested to obtain local binary pattern feature vectors:
(6.1) for reference video and distorted video in training video setThe Y component obtains a vector L by applying a stereo local binary pattern operator1And opposite vector L1Obtaining E clustering centers by using a K nearest neighbor clustering method;
(6.2) applying a stereo local binary pattern operator to the Y component of the concentrated distorted video of the video to be detected to obtain a vector L2And using K nearest neighbor clustering method to cluster vector L2E clustering centers obtained in the step (6.1) are classified; according to the vectors L divided in the E clustering centers2Constructing a local binary pattern histogram according to the number of the feature vectors P, and obtaining corresponding feature vectors P; similarly, applying a three-dimensional local binary pattern operator to the Y component of the reference video in the video set to be detected and constructing a local binary pattern histogram to obtain a feature vector Q;
(7) calculating stereo local binary pattern feature similarity S of reference video and distorted video in to-be-detected video set2The calculation formula is as follows:
Figure FDA0002308827380000031
wherein, C3Is a constant, is a vector inner product operator, | | | | · | | | is a modular operation of solving the vector;
(8) according to the filter coefficient similarity S obtained in the step (5)1And the stereo local binary pattern feature similarity S obtained in the step (7)2Calculating the visual similarity theta of the reference video in the video set to be detected and the current frame of the distorted video according to the following formula;
θ＝ω6S1+ω7S2
wherein, ω is6、ω7Is a weighting coefficient;
(9) obtaining the visual similarity of other frames of the video set to be detected through the steps (2) to (8), and recording the visual similarity of the t-th frame as thetat(ii) a And calculating a video image quality evaluation score Z, wherein the calculation formula is as follows:
Figure FDA0002308827380000032
and T is the total frame number of the reference video or the distorted video in the video set to be detected.
2. The Log-Gabor similarity-based full-reference video quality evaluation method according to claim 1, wherein the amplitude MY _ D (S, o, m, n), the phase AY _ D (S, o, m, n) of the Y component filter coefficient of the distorted video and the amplitude MY _ S (S, o, m, n), the phase AY _ S (S, o, m, n) of the Y component filter coefficient of the reference video in step (3) are calculated by the formula in step (2.3).
3. The method for evaluating quality of full-reference video based on Log-Gabor similarity according to claim 1, wherein the step (6) of applying a stereo local binary pattern operator comprises the following sub-steps:
(61) acquiring a Y component of the YUV video, and extracting a pixel point Y of the Y component at an (i, j) position in a t-th framet(i, j) and 8 pixel points of neighborhood thereof are respectively Yt(i-1,j-1)、Yt(i-1,j)、Yt(i-1,j+1)、Yt(i,j-1)、Yt(i,j+1)、Yt(i+1,j-1)、Yt(i +1, j) and Yt(i+1,j+1)；
(62) Extracting pixel point Y of (i, j) position of Y component in t-1 framet-1(i, j) and 8 pixel points of neighborhood thereof are respectively Yt-1(i-1,j-1)、Yt-1(i-1,j)、Yt-1(i-1,j+1)、Yt-1(i,j-1)、Yt-1(i,j+1)、Yt-1(i+1,j-1)、Yt-1(i +1, j) and Yt-1(i+1,j+1)；
(63) Extracting pixel point Y of (i, j) position of Y component in t +1 th framet+1(i, j) and 8 pixel points of neighborhood thereof are respectively Yt+1(i-1,j-1)、Yt+1(i-1,j)、Yt+1(i-1,j+1)、Yt+1(i,j-1)、Yt+1(i,j+1)、Yt+1(i+1,j-1)、Yt+1(i +1, j) and Yt+1(i+1,j+1)；
(64) With Y in the t-th framet(i, j) as the center pixel point, and Y is compared by the following formulat(i, j) and the size of the 3 x 3 neighborhood pixels in the space domain and the time domain extracted in the steps (61) to (63) to obtain a comparison result sk(a,b)：
Figure FDA0002308827380000041
Wherein, Yk(a, b) is Yt(i, j) the neighborhood pixel value, k belongs to { t-1, t, t +1}, a belongs to { i-1, i, i +1}, and b belongs to { j-1, j, j +1 };
(65) comparing the result s obtained in step (64)k(a, b) according to st-1(i-1,j-1)、st-1(i-1,j)、st-1(i-1,j+1)、st-1(i,j-1)、st-1(i,j)、st-1(i,j+1)、st-1(i+1,j-1)、st-1(i+1,j)、st-1(i +1, j +1) and st(i-1,j-1)、st(i-1,j)、st(i-1,j+1)、st(i,j-1)、st(i,j+1)、st(i+1,j-1)、st(i+1,j)、st(i +1, j +1) and st+1(i-1,j-1)、st+1(i-1,j)、st+1(i-1,j+1)、st+1(i,j-1)、st+1(i,j)、st+1(i,j+1)、st+1(i+1,j-1)、st+1(i+1,j)、st+1The sequential arrangement of (i +1, j +1) constitutes a vector L.
4. The method for full-reference video quality evaluation based on Log-Gabor similarity according to claim 3, wherein the vector L e R in the step (65)26×1。
5. The method for full-reference video quality evaluation based on Log-Gabor similarity according to claim 1, wherein the feature vector P, Qe R in the step (6.2)E×1。
6. The method for evaluating the quality of the full reference video based on the Log-Gabor similarity according to claim 1, wherein the total number of frames of the reference video and the distorted video is the same; the reference video and the distortion video are YUV videos.
CN201911250288.7A 2019-12-09 2019-12-09 Total reference video quality evaluation method based on Log-Gabor similarity Active CN110930398B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911250288.7A CN110930398B (en) 2019-12-09 2019-12-09 Total reference video quality evaluation method based on Log-Gabor similarity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911250288.7A CN110930398B (en) 2019-12-09 2019-12-09 Total reference video quality evaluation method based on Log-Gabor similarity

Publications (2)

Publication Number Publication Date
CN110930398A true CN110930398A (en) 2020-03-27
CN110930398B CN110930398B (en) 2023-05-09

Family

ID=69858413

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911250288.7A Active CN110930398B (en) 2019-12-09 2019-12-09 Total reference video quality evaluation method based on Log-Gabor similarity

Country Status (1)

Country Link
CN (1) CN110930398B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111680549A (en) * 2020-04-28 2020-09-18 肯维捷斯（武汉）科技有限公司 Paper pattern recognition method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB201417807D0 (en) * 2014-10-08 2014-11-19 Univ Lancaster Data structuring and searching methods and apparatus
CN104918039A (en) * 2015-05-05 2015-09-16 四川九洲电器集团有限责任公司 Image quality evaluation method and image quality evaluation system
CN107274379A (en) * 2017-05-09 2017-10-20 武汉大学 A kind of image quality evaluating method and system
CN109523506A (en) * 2018-09-21 2019-03-26 浙江大学 The complete of view-based access control model specific image feature enhancing refers to objective evaluation method for quality of stereo images
CN109598707A (en) * 2018-11-26 2019-04-09 浙江科技学院 A kind of full reference picture assessment method for encoding quality based on feature information processing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB201417807D0 (en) * 2014-10-08 2014-11-19 Univ Lancaster Data structuring and searching methods and apparatus
CN104918039A (en) * 2015-05-05 2015-09-16 四川九洲电器集团有限责任公司 Image quality evaluation method and image quality evaluation system
CN107274379A (en) * 2017-05-09 2017-10-20 武汉大学 A kind of image quality evaluating method and system
CN109523506A (en) * 2018-09-21 2019-03-26 浙江大学 The complete of view-based access control model specific image feature enhancing refers to objective evaluation method for quality of stereo images
CN109598707A (en) * 2018-11-26 2019-04-09 浙江科技学院 A kind of full reference picture assessment method for encoding quality based on feature information processing

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ZHANGKAI NI,ET AL: "A Gabor Feature-Based Quality Assessment Model for the Screen Content Images", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 *
汪斌等: "基于单片机和加速度传感器的跌倒检测设备设计", 《科技视界》 *
符颖;曾焕强;倪张凯;陈婧;蔡灿辉;: "采用边缘信息的屏幕图像质量评价", 信号处理 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111680549A (en) * 2020-04-28 2020-09-18 肯维捷斯（武汉）科技有限公司 Paper pattern recognition method
CN111680549B (en) * 2020-04-28 2023-12-05 肯维捷斯（武汉）科技有限公司 Paper grain identification method

Also Published As

Publication number Publication date
CN110930398B (en) 2023-05-09

Similar Documents

Publication Publication Date Title
CN106920224B (en) A method of assessment stitching image clarity
CN104008538B (en) Based on single image super-resolution method
CN108428227B (en) No-reference image quality evaluation method based on full convolution neural network
CN106127741B (en) Non-reference picture quality appraisement method based on improvement natural scene statistical model
CN106709472A (en) Video target detecting and tracking method based on optical flow features
CN111028292B (en) Sub-pixel level image matching navigation positioning method
CN109255358B (en) 3D image quality evaluation method based on visual saliency and depth map
CN108416753B (en) Image denoising algorithm based on non-parametric alternating direction multiplier method
Jia et al. Image denoising via sparse representation over grouped dictionaries with adaptive atom size
CN109886945B (en) No-reference contrast distortion image quality evaluation method based on contrast enhancement
CN109410248B (en) Flotation froth motion characteristic extraction method based on r-K algorithm
CN108234819B (en) Video synchronization method based on homograph
CN105243385B (en) A kind of image quality evaluating method based on unsupervised learning
CN109685772B (en) No-reference stereo image quality evaluation method based on registration distortion representation
CN111292336B (en) Omnidirectional image non-reference quality evaluation method based on segmented spherical projection format
CN113269682A (en) Non-uniform motion blur video restoration method combined with interframe information
CN110910365A (en) Quality evaluation method for multi-exposure fusion image of dynamic scene and static scene simultaneously
CN111127353B (en) High-dynamic image ghost-removing method based on block registration and matching
Peña et al. Burst ranking for blind multi-image deblurring
CN110930398B (en) Total reference video quality evaluation method based on Log-Gabor similarity
CN108492275B (en) No-reference stereo image quality evaluation method based on deep neural network
CN108629771B (en) A kind of blind evaluation method of picture quality with scale robustness
Xiong et al. ψ-net: Point structural information network for no-reference point cloud quality assessment
Sun et al. A hybrid demosaicking algorithm for area scan industrial camera based on fuzzy edge strength and residual interpolation
CN111311657B (en) Infrared image homologous registration method based on improved corner principal direction distribution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information

Inventor after: Mao Linghang

Inventor after: Wang Bin

Inventor after: Chen Shucong

Inventor after: Jiang Feilong

Inventor after: Zhu Haibin

Inventor after: Xu Qiaochu

Inventor after: Zhang Ao

Inventor after: Li Xinglong

Inventor before: Wang Bin

Inventor before: Chen Shucong

Inventor before: Jiang Feilong

Inventor before: Zhu Haibin

Inventor before: Mao Linghang

Inventor before: Xu Qiaochu

Inventor before: Zhang Ao

Inventor before: Li Xinglong

CB03 Change of inventor or designer information
GR01 Patent grant
GR01 Patent grant