CN108449596B - 3D stereoscopic image quality evaluation method integrating aesthetics and comfort - Google Patents

3D stereoscopic image quality evaluation method integrating aesthetics and comfort Download PDF

Info

Publication number
CN108449596B
CN108449596B CN201810342793.3A CN201810342793A CN108449596B CN 108449596 B CN108449596 B CN 108449596B CN 201810342793 A CN201810342793 A CN 201810342793A CN 108449596 B CN108449596 B CN 108449596B
Authority
CN
China
Prior art keywords
image
parallax
representing
aesthetic
comfort
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810342793.3A
Other languages
Chinese (zh)
Other versions
CN108449596A (en
Inventor
牛玉贞
钟伊妮
柯逍
施逸青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fuzhou University
Original Assignee
Fuzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuzhou University filed Critical Fuzhou University
Priority to CN201810342793.3A priority Critical patent/CN108449596B/en
Publication of CN108449596A publication Critical patent/CN108449596A/en
Application granted granted Critical
Publication of CN108449596B publication Critical patent/CN108449596B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a 3D stereo image quality evaluation method fusing aesthetics and comfort, which comprises the following steps: step S1: extracting aesthetic characteristics of left and right views and aesthetic consistency characteristics of the left and right views from each stereo image in the training image set and the image set to be predicted to obtain an aesthetic characteristic set F1; step S2: extracting comfort level features from each stereo image in the training image set and the image set to be predicted to obtain a comfort level feature set F2; step S3: combining all images in the training image set with an aesthetic feature set F1 and a comfort feature set F2, taking the images as a machine learning feature set T1, and training to obtain a stereo image quality evaluation model; step S4: and evaluating each image to be predicted by using the trained quality evaluation model to obtain the final quality evaluation score of all the images to be predicted. The method is beneficial to improving the consistency of the evaluation result and the user subjective score.

Description

3D stereoscopic image quality evaluation method integrating aesthetics and comfort
Technical Field
The invention relates to the field of image and video processing and computer vision, in particular to a 3D (three-dimensional) image quality evaluation method integrating aesthetics and comfort.
Background
The image quality evaluation algorithm is divided into reference, semi-reference and non-reference quality evaluation. The quality evaluation without reference is to perform quality evaluation without using a corresponding reference image by using the characteristics of the image itself and the like.
A 3D stereoscopic image is composed of left and right views, which makes the 3D stereoscopic image have both the depth feature of a binocular image and the feature of a monocular image. In real life, the most intuitive experience for the user is the aesthetic feature of the monocular image. Niu et al, in evaluating the frame pictures of professional video, mention that the requirements of professional photographers for pictures are a small number of keytones, moderate color saturation, and smooth luminance variation range, etc.
Compared with the common image, the stereoscopic impression of the binocular image is derived from left and right views to generate parallax, so that the imaging on the retina of human eyes has difference, and the spatial difference is just like the stereoscopic landscape seen by human eyes in the real world. Lamb oij et al suggest that these visual discomforts are caused by excessive disparity values, blurring in some unnatural states, color space mismatches, and the like. Among the parallaxes, there are again a negative parallax imaged in front of the screen and a positive parallax imaged behind the screen, and a zero parallax plane. Among them, negative parallax is an important influencing factor leading to visual comfort experience. In the study of the visual comfort experience of stereoscopic images by Shao et al, in order to specifically analyze the effects of large disparity values in stereoscopic images, the top 10% of the disparity data at the maximum and minimum were analyzed separately. According to the knowledge, a visual comfort area exists in a certain range from a zero parallax plane, the parallax value in the area accords with the natural regulation of human eyeballs, and various problems such as visual imaging conflict and the like do not exist. However, the disparity value is inevitably outside the range of the comfort zone in the processes of shooting and post-production, so that the disparity range of the stereoscopic image has uncertainty.
Disclosure of Invention
The invention aims to provide a 3D image quality evaluation method integrating aesthetics and comfort, which is beneficial to improving the consistency of an evaluation result and a user subjective score.
In order to achieve the purpose, the technical scheme of the invention is as follows: A3D stereoscopic image quality evaluation method integrating aesthetics and comfort comprises the following steps:
step S1: inputting a training image set, a set of images to be predicted and subjective quality evaluation scores of a user on each three-dimensional image in the two sets; extracting aesthetic characteristics of left and right views and aesthetic consistency characteristics of the left and right views from each stereo image in the training image set and the image set to be predicted to obtain an aesthetic characteristic set F1;
step S2: extracting comfort level features from each stereo image in the training image set and the image set to be predicted to obtain a comfort level feature set F2;
step S3: combining all images in the training image set with an aesthetic feature set F1 and a comfort feature set F2, taking the images as a machine learning feature set T1, and training to obtain a stereo image quality evaluation model;
step S4: and evaluating each image to be predicted by using the trained quality evaluation model to obtain the final quality evaluation score of all the images to be predicted.
Further, in the step S1, the method extracts the aesthetic features of the left and right views and the aesthetic consistency features of the left and right views from the stereo image to obtain an aesthetic feature set F1, and includes the following steps:
step S11: the aesthetic characteristics of the left and right views in three aspects of dominant hue number, color saturation and brightness range are extracted from the stereo image, and the calculation formula is as follows:
Figure BDA0001630945470000021
Figure BDA0001630945470000022
Lt＝L0(β*W*H)-L0((1-β)*W*H+1)
wherein t denotes a left view or a right view of the stereoscopic image, t-L denotes a left view, and t-R denotes a right view; htThe number of dominant tones representing the t view; converting the t view from RGB color space to HSV color space, representing the H channel value by using a color histogram which is equally divided by n,
Figure BDA0001630945470000023
representing the j-th equally divided color histogram value in the t view, m is the maximum equally divided value in the color histogram, α is a set parameter;
Figure BDA0001630945470000024
representing solving for a dominant hue set, when a certain aliquot of color histogram values is greater than α m, i.e., α times m, it is considered thatThe color corresponding to the equal division belongs to a dominant hue set; count () represents the number of elements in which to count; stRepresenting the color saturation mean of the t view; w, H, respectively representing the width and height of the image, S (i, j) represents the value of the S channel of the pixel point (i, j) in the HSV color space; l istDenotes the value of the L channel, L, in the CIE LUV color space for the t view0Representing a sequence obtained by sequencing L channel values of all pixel points in the t view from small to large, wherein the brightness range of the image is the part of the value range which accounts for β times of the middle of the sequence, β is a set parameter, β∈ [0,1 ]]；
Step S12: the consistency characteristic of the left view and the right view of the stereo image on the aesthetic characteristic is calculated, and the calculation formula is as follows:
Hc＝|HL-HR|,Sc＝|SL-SR|,Lc＝|LL-LR|
wherein, Hc, Sc, Lc represent the consistency characteristic of the left and right views of the stereoscopic image in the aesthetic characteristic respectively, and are as follows in sequence: dominant hue number consistency, color saturation consistency, and brightness range consistency; in addition, the color consistency F between the left view and the right view of the stereo image is calculated by using an image quality evaluation method CSVD based on the color contrast similarity and the color value differenceCSVD；
Combining the steps S11-S12 to obtain the aesthetic feature set F1 ═ Ht,St,Lt,Hc,Sc,Lc,FCSVD}。
Further, in the step S2, extracting a comfort level feature from each stereo image in the training image set and the image set to be predicted to obtain a comfort level feature set F2, including the following steps:
step S21: calculating horizontal and vertical disparity maps of the stereo image by using an SIFT Flow dense matching algorithm; on the basis of the obtained disparity map, calculating comfort features from multiple aspects of positive disparity, negative disparity, disparity mean and disparity variance; the calculation formula is as follows:
Figure BDA0001630945470000031
Figure BDA0001630945470000032
Figure BDA0001630945470000033
wherein the content of the first and second substances,
Figure BDA0001630945470000034
respectively representing horizontal positive parallax, horizontal negative parallax, vertical positive parallax and vertical negative parallax; w and H represent the width and height of the image, respectively; vx(i, j) and Vy(i, j) respectively represent horizontal and vertical viewing difference values of the stereoscopic image at (i, j); n (omega +) and N (omega-) respectively represent the number of pixel points in the positive parallax set omega + and the negative parallax set omega-; dd denotes the parallax range, drelativeA relative depth representing a disparity; on the basis of calculating the parallax mean value, calculating the variance corresponding to each mean value, wherein the calculation formula is as follows:
Figure BDA0001630945470000035
Figure BDA0001630945470000036
wherein the content of the first and second substances,
Figure BDA0001630945470000037
respectively representing the variance of horizontal positive parallax, the variance of horizontal negative parallax, the variance of vertical positive parallax and the variance of vertical negative parallax; std (z) represents the variance of all elements in solution set z;
step S22: calculating edge parallax features; and calculating the parallax value at the first t% in the horizontal positive parallax and the horizontal negative parallax by the following calculation formula:
Figure BDA0001630945470000041
wherein d ismaxMeans of positive parallax with absolute value greater than the first percent t, dminRepresenting a negative disparity average with an absolute value greater than the top percent t,
Figure BDA0001630945470000042
respectively representing the positive disparity set with the absolute value of the first t%
Figure BDA0001630945470000043
And absolute value preceding t% negative disparity set
Figure BDA0001630945470000044
The number of the pixel points in (1);
step S23: calculating spatial frequency correlation characteristics; respectively calculating the spatial frequency characteristics of the left view and the right view, then taking the mean value of the spatial frequency characteristics and the left view and the right view to represent the spatial frequency of the stereo image, wherein the calculation formula is as follows:
Figure BDA0001630945470000045
Figure BDA0001630945470000046
wherein fl and fr respectively represent the spatial frequency characteristics of the left and right views, SBl(i,j)、SBr(i, j) respectively representing the numerical values of all pixel points of the left view and the right view calculated by using a sobel edge detection operator at the position (i, j); sigma1、σ2、σ3Respectively representing the connection characteristics established between the f and the parallax characteristics;
step S24: calculating relevant characteristics of the visual comfort zone; the calculation formula is as follows:
Figure BDA0001630945470000047
wherein, γ+Threshold, γ, representing the visual comfort zone in front of the screen where the retina can be adjustably imaged-A threshold value representing a visual comfort zone behind the screen, behind which the retina can be adjustably imaged, p representing the pupil diameter;s represents an eyeball length, and v represents a viewing distance; when gamma is+And gamma-When the control range of the human eyes is exceeded, the stereoscopic image can generate blur, and the fatigue feeling of a viewer is increased;
integrating the steps S21-S24, a comfort feature set is obtained as follows:
Figure BDA0001630945470000048
further, in the step S3, the aesthetic feature set F1 and the comfort feature set F2 are combined with all images in the training image set to serve as a machine learning feature set T1, and a stereo image quality evaluation model is obtained through training, which specifically includes:
fusing feature sets F1 and F2 of all images in the training data set and a label set L1 obtained by subjective quality assessment scores of all images in the training image set by a user, and forming a feature set T1 of the training image set, namely { F1, F2} and a label set L1; and training by using a random forest regression method through the feature set T1 and the label set L1 to obtain a three-dimensional image quality evaluation model M.
Further, in step S4, the features of all the images in the data set to be predicted are fused, the feature set T2 forming the image set to be predicted is { F1, F2}, and the final quality assessment score of all the images to be predicted is calculated by using the stereo image quality assessment model trained in step S3.
Compared with the prior art, the invention has the beneficial effects that: the 3D stereo image quality evaluation method based on the fusion of the aesthetic characteristics and the comfort characteristics influencing the stereo image quality solves the problem that the 3D stereo image evaluates the quality from a single angle, can keep higher consistency with the subjective score of a user according to the 3D stereo image quality score obtained by evaluation, and can be used in the fields of image quality evaluation, image or video classification and the like.
Drawings
FIG. 1 is a block flow diagram of the method of the present invention.
Fig. 2 is a flow chart of the overall method implementation in the embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the embodiments.
The invention provides a 3D stereoscopic image quality evaluation method integrating aesthetics and comfort, which comprises the following steps as shown in figures 1 and 2:
step S1: inputting a training image set, a set of images to be predicted and subjective quality assessment scores of users for each stereo image in the two sets. And extracting the aesthetic characteristics of the left and right views and the aesthetic consistency characteristics of the left and right views from each stereo image in the training image set and the image set to be predicted to obtain an aesthetic characteristic set F1. The method specifically comprises the following steps:
step S11: the aesthetic characteristics of the left and right views in three aspects of dominant hue number, color saturation and brightness range are extracted from the stereo image, and the calculation formula is as follows:
Figure BDA0001630945470000051
Figure BDA0001630945470000052
Lt＝L0(β*W*H)-L0((1-β)*W*H+1)
wherein t denotes a left view or a right view of the stereoscopic image, t-L denotes a left view, and t-R denotes a right view; htThe number of dominant tones representing the t view; converting the t view from the RGB color space to the HSV color space, representing the H channel value by a color histogram with n being equally divided (in the embodiment, n is 20),
Figure BDA0001630945470000053
representing the j-th equally divided color histogram value in the t view, m is the maximum equally divided value in the color histogram, α is a set parameter;
Figure BDA0001630945470000061
representing solving a dominant hue set, when the value of a color histogram of a certain equal part is more than α m, namely α times of m, the color corresponding to the equal part is considered to belong to the dominant hue set, count () represents calculating the number of elements in the equal part, StRepresenting the color saturation mean of the t view; w, H, respectively representing the width and height of the image, S (i, j) represents the value of the S channel of the pixel point (i, j) in the HSV color space; luminance is a factor affecting image quality, LtDenotes the value of the L channel, L, in the CIE LUV color space for the t view0Representing a sequence obtained by sequencing L channel values of all pixel points in the t view from small to large, wherein the brightness range of the image is the part of the value range which accounts for β times of the middle of the sequence, β is a set parameter, β∈ [0,1 ]]In the embodiment, β is 90%;
step S12: the consistency characteristic of the left view and the right view of the stereo image on the aesthetic characteristic is calculated, and the calculation formula is as follows:
Hc＝|HL-HR|,Sc＝|SL-SR|,Lc＝|LL-LR|
wherein, Hc, Sc, Lc represent the consistency characteristic of the left and right views of the stereoscopic image in the aesthetic characteristic respectively, and are as follows in sequence: dominant hue number consistency, color saturation consistency, and brightness range consistency; when the difference of the aesthetic features between the left view and the right view is too large, the quality of the stereoscopic image is influenced, so the invention considers the consistency feature of the aesthetic features of the left view and the right view; in addition, the color consistency F between the left view and the right view of the stereo image is calculated by using an image quality evaluation method CSVD based on the color contrast similarity and the color value differenceCSVD；
Combining the steps S11-S12 to obtain the aesthetic feature set F1 ═ Ht,St,Lt,Hc,Sc,Lc,FCSVD}。
Step S2: and (3) extracting comfort level characteristics from each stereo image in the training image set and the image set to be predicted to obtain a comfort level characteristic set F2. The method specifically comprises the following steps:
step S21: calculating the moving conditions of pixel points between left and right views in the stereo image in the horizontal direction and the vertical direction by using an SIFT Flow dense matching algorithm, and obtaining the horizontal and vertical disparity maps of the stereo image according to the obtained result; on the basis of the obtained disparity map, comfort features are calculated from multiple aspects such as positive disparity, negative disparity, disparity mean, disparity variance and the like; the calculation formula is as follows:
Figure BDA0001630945470000062
Figure BDA0001630945470000063
Figure BDA0001630945470000064
wherein the content of the first and second substances,
Figure BDA0001630945470000065
respectively representing horizontal positive parallax, horizontal negative parallax, vertical positive parallax and vertical negative parallax; w and H represent the width and height of the image, respectively; vx(i, j) and Vy(i, j) respectively represent horizontal and vertical viewing difference values of the stereoscopic image at (i, j); n (omega)+) And N (omega)-) Respectively representing a positive disparity set omega+And negative disparity set Ω-The number of the pixel points in (1); dd denotes the parallax range, drelativeA relative depth representing a disparity; on the basis of calculating the parallax mean value, calculating the variance corresponding to each mean value, wherein the calculation formula is as follows:
Figure BDA0001630945470000071
Figure BDA0001630945470000072
wherein the content of the first and second substances,
Figure BDA0001630945470000073
respectively represent the positive horizontalA variance of parallax, a variance of horizontal negative parallax, a variance of vertical positive parallax, and a variance of vertical negative parallax; std (z) represents the variance of all elements in solution set z;
step S22: calculating edge parallax features; because too large parallax value can cause discomfort to human eyes, the number of pixel points with larger parallax is generally not large, but the influence of the pixel points on the viewing experience cannot be ignored, the parallax value at the first t% in horizontal positive parallax and horizontal negative parallax is calculated by the invention, and the calculation formula is as follows:
Figure BDA0001630945470000074
wherein d ismaxMeans of positive parallax with absolute value greater than the first percent t, dminRepresenting a negative disparity average with an absolute value greater than the top percent t,
Figure BDA0001630945470000075
respectively representing the positive disparity set with the absolute value of the first t%
Figure BDA0001630945470000076
And absolute value preceding t% negative disparity set
Figure BDA0001630945470000077
The number of the pixel points in (1);
step S23: calculating spatial frequency correlation characteristics; in the stereo image, the spatial frequency has an important influence on the fusion of the binocular images; research shows that the higher the spatial frequency, the smaller the corresponding limit value of binocular fusion, so the higher spatial frequency can limit the binocular fusion of the stereo image in the human eye system; binocular fusion is a key factor for evaluating the quality of stereo images; the spatial frequency is an important characteristic influencing binocular fusion, so that the spatial frequency characteristics of the left view and the right view are respectively calculated, then the average value of the spatial frequency characteristics is taken to represent the spatial frequency of the stereo image, and the calculation formula is as follows:
Figure BDA0001630945470000078
Figure BDA0001630945470000079
wherein fl and fr respectively represent the spatial frequency characteristics of the left and right views, SBl(i,j)、SBr(i, j) respectively representing the numerical values of all pixel points of the left view and the right view calculated by using a sobel edge detection operator at the position (i, j); sigma1、σ2、σ3Respectively representing the connection characteristics established between the f and the parallax characteristics;
step S24: calculating relevant characteristics of the visual comfort zone; judging whether the stereo image is in a visual comfort area after being imaged by the retina is an important factor for evaluating whether the stereo image is a high-quality image; by utilizing various parameters such as the viewing distance, the radius of the retina and the like, the calculation formula of the relevant characteristics of the visual comfort zone is as follows:
Figure BDA0001630945470000081
wherein, γ+Threshold, γ, representing the visual comfort zone in front of the screen where the retina can be adjustably imaged-A threshold value representing a visual comfort zone behind the screen, behind which the retina can be adjustably imaged, p representing the pupil diameter; s represents an eyeball length, and v represents a viewing distance; when the value exceeds the control range of human eyes, the stereoscopic image can generate blur, and the fatigue feeling of a viewer is increased;
integrating the steps S21-S24, a comfort feature set is obtained as follows:
Figure BDA0001630945470000082
step S3: and (3) combining all images in the training image set with the aesthetic feature set F1 and the comfort feature set F2 to serve as a machine learning feature set T1, and training to obtain a stereo image quality evaluation model. The specific method comprises the following steps:
fusing feature sets F1 and F2 of all images in the training data set and a label set L1 obtained by subjective quality assessment scores of all images in the training image set by a user, and forming a feature set T1 of the training image set, namely { F1, F2} and a label set L1; and training by using a random forest regression method through the feature set T1 and the label set L1 to obtain a three-dimensional image quality evaluation model M.
Step S4: and fusing the characteristics of all images in the data set to be predicted to form a characteristic set T2 of the image set to be predicted { F1, F2}, and evaluating each image to be predicted by using a trained stereo image quality evaluation model to obtain the final quality evaluation score of all images to be predicted.
The above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.

Claims (3)

1. A3D stereoscopic image quality evaluation method integrating aesthetics and comfort is characterized by comprising the following steps:
step S1: inputting a training image set, a set of images to be predicted and subjective quality evaluation scores of a user on each three-dimensional image in the two sets; extracting aesthetic characteristics of left and right views and aesthetic consistency characteristics of the left and right views from each stereo image in the training image set and the image set to be predicted to obtain an aesthetic characteristic set F1;
step S2: extracting comfort level features from each stereo image in the training image set and the image set to be predicted to obtain a comfort level feature set F2;
step S3: combining all images in the training image set with an aesthetic feature set F1 and a comfort feature set F2, taking the images as a machine learning feature set T1, and training to obtain a stereo image quality evaluation model;
step S4: evaluating each image to be predicted by using the trained quality evaluation model to obtain the final quality evaluation score of all the images to be predicted;
in the step S1, the method extracts the aesthetic features of the left and right views and the aesthetic consistency features of the left and right views from the stereo image to obtain an aesthetic feature set F1, and includes the following steps:
step S11: the aesthetic characteristics of the left and right views in three aspects of dominant hue number, color saturation and brightness range are extracted from the stereo image, and the calculation formula is as follows:
Figure FDA0002540467800000011
Figure FDA0002540467800000012
Lt＝L0(β*W*H)-L0((1-β)*W*H+1)
wherein t denotes a left view or a right view of the stereoscopic image, t-L denotes a left view, and t-R denotes a right view; htThe number of dominant tones representing the t view; converting the t view from RGB color space to HSV color space, representing the H channel value by using a color histogram which is equally divided by n,
Figure FDA0002540467800000013
representing the j-th equally divided color histogram value in the t view, m is the maximum equally divided value in the color histogram, α is a set parameter;
Figure FDA0002540467800000014
representing solving a dominant hue set, when the value of a color histogram of a certain equal part is more than α m, namely α times of m, the color corresponding to the equal part is considered to belong to the dominant hue set, count () represents calculating the number of elements in the equal part, StRepresenting the color saturation mean of the t view; w, H, respectively representing the width and height of the image, S (i, j) represents the value of the S channel of the pixel point (i, j) in the HSV color space; l istDenotes the value of the L channel, L, in the CIE LUV color space for the t view0Representing a sequence obtained by sequencing L channel values of all pixel points in the t view from small to large, wherein the brightness range of the image is the part of the value range which accounts for β times of the middle of the sequence, β is set parameters, β∈[0,1]；
Step S12: the consistency characteristic of the left view and the right view of the stereo image on the aesthetic characteristic is calculated, and the calculation formula is as follows:
Hc＝|HL-HR|,Sc＝|SL-SR|,Lc＝|LL-LR|
wherein, Hc, Sc, Lc represent the consistency characteristic of the left and right views of the stereoscopic image in the aesthetic characteristic respectively, and are as follows in sequence: dominant hue number consistency, color saturation consistency, and brightness range consistency; in addition, the color consistency F between the left view and the right view of the stereo image is calculated by using an image quality evaluation method CSVD based on the color contrast similarity and the color value differenceCSVD；
Combining the steps S11-S12 to obtain the aesthetic feature set F1 ═ Ht,St,Lt,Hc,Sc,Lc,FCSVD}；
In the step S2, extracting a comfort level feature from each stereo image in the training image set and the image set to be predicted to obtain a comfort level feature set F2, including the following steps:
step S21: calculating horizontal and vertical disparity maps of the stereo image by using an SIFT Flow dense matching algorithm; on the basis of the obtained disparity map, calculating comfort features from multiple aspects of positive disparity, negative disparity, disparity mean and disparity variance; the calculation formula is as follows:
Figure FDA0002540467800000021
Figure FDA0002540467800000022
dd＝max{Vx(i,j)}-min{Vx(i,j)},
Figure FDA0002540467800000023
wherein the content of the first and second substances,
Figure FDA0002540467800000024
respectively representing horizontal positive parallax, horizontal negative parallax, vertical positive parallax and vertical negative parallax; w and H represent the width and height of the image, respectively; vx(i, j) and Vy(i, j) respectively represent horizontal and vertical viewing difference values of the stereoscopic image at (i, j); n (omega)+) And N (omega)-) Respectively representing a positive disparity set omega+And negative disparity set Ω-The number of the pixel points in (1); dd denotes the parallax range, drelativeA relative depth representing a disparity; on the basis of calculating the parallax mean value, calculating the variance corresponding to each mean value, wherein the calculation formula is as follows:
Figure FDA0002540467800000025
Figure FDA0002540467800000026
wherein the content of the first and second substances,
Figure FDA0002540467800000027
respectively representing the variance of horizontal positive parallax, the variance of horizontal negative parallax, the variance of vertical positive parallax and the variance of vertical negative parallax; std (z) represents the variance of all elements in solution set z;
step S22: calculating edge parallax features; and calculating the parallax value at the first t% in the horizontal positive parallax and the horizontal negative parallax by the following calculation formula:
Figure FDA0002540467800000031
wherein d ismaxMeans of positive parallax with absolute value greater than the first percent t, dminRepresenting a negative disparity average with an absolute value greater than the top percent t,
Figure FDA0002540467800000032
respectively representing the positive disparity set with the absolute value of the first t%
Figure FDA0002540467800000033
And absolute value preceding t% negative disparity set
Figure FDA0002540467800000034
The number of the pixel points in (1);
step S23: calculating spatial frequency correlation characteristics; respectively calculating the spatial frequency characteristics of the left view and the right view, then taking the mean value of the spatial frequency characteristics and the left view and the right view to represent the spatial frequency of the stereo image, wherein the calculation formula is as follows:
Figure FDA0002540467800000035
Figure FDA0002540467800000036
wherein fl and fr respectively represent the spatial frequency characteristics of the left and right views, SBl(i,j)、SBr(i, j) respectively representing the numerical values of all pixel points of the left view and the right view calculated by using a sobel edge detection operator at the position (i, j); sigma2、σ3Respectively representing the connection characteristics established between the f and the parallax characteristics;
step S24: calculating relevant characteristics of the visual comfort zone; the calculation formula is as follows:
Figure FDA0002540467800000037
wherein, γ+Threshold, γ, representing the visual comfort zone in front of the screen where the retina can be adjustably imaged-A threshold value representing a visual comfort zone behind the screen, behind which the retina can be adjustably imaged, p representing the pupil diameter; s represents an eyeball length, and v represents a viewing distance; when gamma is+And gamma-When the control range of the human eyes is exceeded, the stereoscopic image can generate blur, and the fatigue feeling of a viewer is increased;
integrating the steps S21-S24, a comfort feature set is obtained as follows:
Figure FDA0002540467800000038
2. the 3D stereoscopic image quality assessment method integrating aesthetics and comfort as claimed in claim 1, wherein in the step S3, for all images in the training image set, the aesthetic feature set F1 and the comfort feature set F2 are combined and used as a machine learning feature set T1, and a stereoscopic image quality assessment model is obtained by training, specifically, the method comprises:
fusing feature sets F1 and F2 of all images in the training data set and a label set L1 obtained by subjective quality assessment scores of all images in the training image set by a user, and forming a feature set T1 of the training image set, namely { F1, F2} and a label set L1; and training by using a random forest regression method through the feature set T1 and the label set L1 to obtain a three-dimensional image quality evaluation model M.
3. The 3D stereoscopic image quality assessment method with integration of aesthetics and comfort as claimed in claim 1, wherein in step S4, the features of all images in the data set to be predicted are integrated to form a feature set T2 ═ F1, F2} of the image set to be predicted, and the final quality assessment scores of all images to be predicted are calculated by using the stereoscopic image quality assessment model trained in step S3.
CN201810342793.3A 2018-04-17 2018-04-17 3D stereoscopic image quality evaluation method integrating aesthetics and comfort Expired - Fee Related CN108449596B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810342793.3A CN108449596B (en) 2018-04-17 2018-04-17 3D stereoscopic image quality evaluation method integrating aesthetics and comfort

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810342793.3A CN108449596B (en) 2018-04-17 2018-04-17 3D stereoscopic image quality evaluation method integrating aesthetics and comfort

Publications (2)

Publication Number Publication Date
CN108449596A CN108449596A (en) 2018-08-24
CN108449596B true CN108449596B (en) 2020-09-01

Family

ID=63200047

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810342793.3A Expired - Fee Related CN108449596B (en) 2018-04-17 2018-04-17 3D stereoscopic image quality evaluation method integrating aesthetics and comfort

Country Status (1)

Country Link
CN (1) CN108449596B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523590B (en) * 2018-10-22 2021-05-18 福州大学 3D image depth information visual comfort evaluation method based on sample
CN109584326A (en) * 2018-11-07 2019-04-05 厦门欢乐逛科技股份有限公司 Poster Automatic color matching method, medium, computer equipment and device
CN109801256B (en) * 2018-12-15 2023-05-26 华南理工大学 Image aesthetic quality assessment method based on region of interest and global features
CN109815965B (en) * 2019-02-13 2021-07-06 腾讯科技（深圳）有限公司 Image filtering method and device and storage medium
CN110533097B (en) * 2019-08-27 2023-01-06 腾讯科技（深圳）有限公司 Image definition recognition method and device, electronic equipment and storage medium
CN111046953B (en) * 2019-12-12 2022-06-21 电子科技大学 Image evaluation method based on similarity comparison
CN112738501B (en) * 2020-12-29 2022-05-17 杭州电子科技大学 Three-dimensional image comfort level testing method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105574885B (en) * 2016-01-13 2017-03-08 福州大学 Based on machine learning and merge the full reference picture method for evaluating quality of visual signature
CN105678775B (en) * 2016-01-13 2017-03-08 福州大学 A kind of color correction appraisal procedure based on machine learning
CN107578404B (en) * 2017-08-22 2019-11-15 浙江大学 View-based access control model notable feature is extracted complete with reference to objective evaluation method for quality of stereo images
CN107396095B (en) * 2017-08-28 2019-01-15 方玉明 A kind of no reference three-dimensional image quality evaluation method

Also Published As

Publication number Publication date
CN108449596A (en) 2018-08-24

Similar Documents

Publication Publication Date Title
CN108449596B (en) 3D stereoscopic image quality evaluation method integrating aesthetics and comfort
CN105139438B (en) video human face cartoon generation method
Dong et al. Human visual system-based saliency detection for high dynamic range content
TWI457853B (en) Image processing method for providing depth information and image processing system using the same
US10277877B2 (en) 3D system including a neural network
CN106874830A (en) A kind of visually impaired people's householder method based on RGB D cameras and recognition of face
KR102009130B1 (en) The System Providing Diagnosis of Makeup and Question and Answer Service
CA2994514A1 (en) 2d-to-3d video frame conversion
CN103384340A (en) Method for obtaining 3D imaging image from single 2D image
Lin et al. Stereoscopic 3D experience optimization using cropping and warping
CN109859138B (en) Infrared image enhancement method based on human visual characteristics
JP5210416B2 (en) Stereoscopic image generating apparatus, stereoscopic image generating method, program, and recording medium
TWI436636B (en) Method and apparatus for adjusting three dimension video image
CN106683072B (en) 3D image comfort level quality evaluation method and system based on PUP image
TW201733351A (en) Three-dimensional auto-focusing method and the system thereof
Guo et al. Adaptive estimation of depth map for two-dimensional to three-dimensional stereoscopic conversion
CN111405264A (en) 3D video comfort level improving method based on depth adjustment
CN116567191A (en) Stereoscopic vision content parallax adjustment method for comfort level improvement
KR101513931B1 (en) Auto-correction method of composition and image apparatus with the same technique
KR101581586B1 (en) Compensation method for noise of depth image
Bharathi et al. 2D-to-3D Conversion of Images Using Edge Information
Chen et al. Virtual view quality assessment based on shift compensation and visual masking effect
CN105959679A (en) Combined optimization method of visual comfort and deep sense of a stereoscopic image
Niu et al. Stereoscopic Image Quality Assessment Based on both Distortion and Disparity
CN116850012B (en) Visual training method and system based on binocular vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200901

CF01 Termination of patent right due to non-payment of annual fee