CN112766419A - Image quality evaluation method and device based on multitask learning - Google Patents
Image quality evaluation method and device based on multitask learning Download PDFInfo
- Publication number
- CN112766419A CN112766419A CN202110253923.8A CN202110253923A CN112766419A CN 112766419 A CN112766419 A CN 112766419A CN 202110253923 A CN202110253923 A CN 202110253923A CN 112766419 A CN112766419 A CN 112766419A
- Authority
- CN
- China
- Prior art keywords
- image
- quality evaluation
- image quality
- natural attribute
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention is suitable for the technical field of computers, and provides an image quality evaluation method and device based on multitask learning, wherein the method comprises the following steps: processing the image to be quality-evaluated according to an image quality evaluation model based on a convolutional neural network generated according to multi-task learning training, and determining a quality evaluation result of the image; wherein the image quality evaluation model is generated by training in advance based on natural attribute feature task learning; the natural attribute feature task learning is to enable a natural attribute feature response result obtained after the image is processed by the image quality evaluation model and a natural attribute feature real result obtained by feature extraction of the image according to a preset natural attribute feature extraction rule to meet a preset requirement, and compared with an image quality evaluation model generated by training by using a no-reference image quality evaluation algorithm, the accuracy of model analysis is remarkably improved.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to an image quality evaluation method and device.
Background
The image is an important carrier for information transmission, can help people to better know the objective world and plays a very important role in daily work and life, but the image is easily influenced by a plurality of factors in the processes of acquisition, storage, transmission and the like, so that the quality of the image is damaged, and the perception of human beings on the image content is influenced. Therefore, the image quality evaluation method can help people to quickly identify the quality of the target image, screen out the image with seriously damaged quality and avoid influencing the perception of human beings on the image content.
However, the existing image quality assessment method generally utilizes an image quality assessment model trained by a non-reference image quality assessment algorithm to perform feature screening, and in fact, because the image quality assessment algorithm does not refer to the relevant features of the original image, the screened image features cannot be accurately matched with the real features of the original image, that is, the image quality assessment model trained by the image quality assessment method cannot efficiently and accurately assess the image quality. That is to say, the accuracy of an analysis model for image feature screening by using an image quality evaluation model trained by a non-reference image quality evaluation algorithm is poor, and the practical application capability is low.
Therefore, the image quality evaluation result obtained by the image quality evaluation model trained by the existing image quality evaluation method has the technical problems of poor accuracy and low practical application capability.
Disclosure of Invention
The embodiment of the invention aims to provide an image quality evaluation method, and aims to solve the technical problems of poor accuracy and low practical application capability of an image quality evaluation result obtained by an image quality evaluation model obtained by training with the conventional image quality evaluation method.
The embodiment of the invention is realized in such a way that an image quality evaluation method comprises the following steps:
acquiring an image to be subjected to quality evaluation;
processing the image to be quality-evaluated according to an image quality evaluation model based on a convolutional neural network generated according to multi-task learning training, and determining a quality evaluation result of the image; wherein
Processing the image to be quality-evaluated according to an image quality evaluation model which is constructed and trained based on a convolutional neural network, and determining the quality evaluation result of the image; wherein
The image quality evaluation model constructed based on the convolutional neural network is generated by training with image natural attribute characteristics and quality scores as a multi-task learning target; the natural attribute feature task learning is a subtask taking natural attribute features obtained by feature extraction of images according to a preset natural attribute feature extraction rule as model learning, and the image quality score learning is another subtask taking an image subjective quality score as model learning.
Another object of an embodiment of the present invention is to provide an image quality evaluation apparatus, including:
the sample image acquisition unit is used for acquiring an image to be subjected to quality evaluation;
and the image processing unit for quality evaluation is used for processing the image to be quality evaluated according to the image quality evaluation model which is generated by multi-task learning training and is based on the convolutional neural network, and determining the quality evaluation result of the image.
The image quality evaluation method provided in the embodiment of the invention processes the image to be quality evaluated according to the image quality evaluation model based on the convolutional neural network generated by the multitask learning training, determines and outputs the quality evaluation result of the image, because the image quality evaluation model based on the convolutional neural network generated according to the multitask learning training is the image quality evaluation model obtained by taking the natural attribute characteristics and the quality evaluation score of the sample image as the loss value calculation standard and adjusting and optimizing the corresponding variable parameters through the convolutional neural network, the accuracy of the evaluation result obtained by analyzing the quality of the image to be evaluated according to the image quality evaluation model can be kept at a higher level, compared with the image quality evaluation model generated by the existing non-reference image quality evaluation algorithm, the accuracy of model analysis is remarkably improved, namely, the image quality evaluation method with high accuracy and good application effect is provided.
Drawings
FIG. 1 is a flowchart illustrating steps of an image quality evaluation method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a procedure for training a generated image quality assessment model according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a rule for extracting natural attribute features of a sample image in an image quality evaluation method according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a step of determining a mean contrast normalization coefficient MSCN of a sample image in an image quality evaluation method according to an embodiment of the present invention;
fig. 5 is a flowchart of a step of determining a total loss value between a natural attribute feature response result and a natural attribute feature true result, and a loss value between a quality evaluation true score and a quality evaluation corresponding score in an image quality evaluation method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image quality evaluation apparatus.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In consideration of the defects of image quality evaluation and identification in the prior art, the embodiment of the invention provides a method for determining the quality evaluation result of an image by processing the image to be subjected to quality evaluation according to an image quality evaluation model which is generated by multi-task learning training and is based on a convolutional neural network. Firstly, establishing an image quality evaluation model based on a convolutional neural network generated according to multi-task learning training, namely extracting the real results of the natural attribute characteristics and the quality evaluation scores of a sample pattern and performing loss value judgment on the response data corresponding to the image quality evaluation model, and performing optimization adjustment iteration on variable parameters in the image quality evaluation model through a gradient descent algorithm and the loss values to obtain a mathematical relation that the response scores of the natural attribute characteristics and the quality evaluation scores of the sample image in the model are infinitely close to the real results. And then, carrying out quality evaluation on the image to be evaluated of the obtained image quality evaluation model to obtain a corresponding evaluation result.
As shown in fig. 1, a flowchart of steps of an image quality evaluation method provided in an embodiment of the present invention specifically includes the following steps:
step S102, obtaining an image to be subjected to quality evaluation.
In the embodiment of the invention, the image to be subjected to quality evaluation is a two-dimensional digital image and is used for evaluating whether the image quality of the two-dimensional digital image is damaged in the processes of acquisition, storage, transmission and the like.
And step S104, processing the image to be quality-evaluated according to the image quality evaluation model which is generated according to the multi-task learning training and is constructed and trained based on the convolutional neural network, and determining the quality evaluation result of the image.
In the embodiment of the invention, the convolutional neural network model algorithm is sequentially connected with 3 modules consisting of convolutional layers and residual blocks, wherein the residual block consists of two convolutional layers which are sequentially connected, a maximum pooling layer is connected behind each residual block, downsampling is carried out through the maximum pooling layer, two full-connection layers FC1 and FC2 are connected behind the last maximum pooling layer, then the maximum pooling layer is connected to a natural attribute feature regression layer FC3 for executing subtask learning, and then a full-connection layer FC4 and a regression layer FC5 containing a neuron for executing a quality score learning task are connected.
In the embodiment of the present invention, the convolution kernel size and padding of the convolutional layer are respectively: the first convolutional layer convolution kernel size is 5 × 5, padding is 2; the sizes of convolution kernels of other convolutional layers are all 3 multiplied by 3, and padding is 1; the number of the channels of the convolutional layers is as follows in sequence: 32, 32, 64, 64, 64; the window size of the maximum pooling layer is 2 multiplied by 2, and the step length is 2; the number of the neurons of the full connection layer is as follows in sequence: 1024, 512, 36, 512, 1; and the activation function of the convolutional neural network model algorithm is a Relu function.
As shown in fig. 2, a flowchart of the steps of training and generating the image quality assessment model provided in the embodiment of the present invention specifically includes the following steps:
step S202 is to acquire a plurality of training sample images from the image quality evaluation database.
In the embodiment of the present invention, the source of the sample image is the image quality evaluation data set, and for each image, the normalization preprocessing is performed after the grayscale image is read, and the image is divided into 32 × 32 sample images according to the step length 24.
And step S204, extracting a natural attribute feature real result of the sample image according to a preset natural attribute feature extraction rule, and constructing a training sample and a training label by combining the subjective quality score of the image.
In the embodiment of the present invention, the relevant extraction rule of the natural attribute features of the image will be described in detail in fig. 3, and will not be described herein again.
In step S206, an initialized multitask learning image quality evaluation model containing variable parameters is constructed.
And S208, inputting the training samples and the training labels into an image quality evaluation model to obtain natural attribute characteristic pre-estimated values and quality score pre-estimated values of all the training samples by the image evaluation model.
In the embodiment of the present invention, the natural attribute features of the image include 36 natural attribute features extracted from the original scale and the down-sampled one-half scale of the sample image, and a specific extraction rule of the natural attribute features of the image will be described in detail in fig. 3, which is not described herein again.
Step S210, calculating the training loss value between the natural attribute characteristic estimated value and the quality score estimated value and the real result and the subjective quality score of the natural attribute characteristic.
In the embodiment of the present invention, the specific steps of the loss value between the response result of the natural attribute feature and the actual result of the natural attribute feature are shown in fig. 5, and are not described herein again.
Step S212, judging whether the training of the image quality evaluation model is finished according to whether the loss value is converged. When no, go to step S214; when judged yes, step S216 is executed.
In the embodiment of the present invention, the preset loss value condition is usually determined by whether the number of iterations or the maximum fitness in the population exceeds a preset threshold, when the preset loss value condition is satisfied, the algorithm is ended, the optimal individual is determined as the optimal solution, and when the preset loss value condition is not satisfied, further loss value optimization is required.
In the embodiment of the present invention, it is obvious that the image quality evaluation model with the highest fitness is the optimal image quality evaluation model.
In the embodiment of the present invention, the image objective quality score calculation formula of the image to be evaluated is as follows:
wherein N is the number of image blocks of the segmentation of the image to be evaluated,
and obtaining a corresponding result of quality score evaluation for the image to be evaluated through the image quality evaluation model.
Step S214, adjusting variable parameters in the image quality evaluation model according to the gradient descent algorithm and the loss value, determining an updated image quality evaluation model, returning to step S208 to predict the sample image of the current image quality evaluation model, and obtaining natural attribute characteristic pre-estimated values and quality score pre-estimated values of all training samples of the image evaluation model
And adjusting the variable parameters in the image quality evaluation model according to a gradient descent algorithm and the loss value, determining an updated image quality evaluation model, and returning to the step S208.
In the embodiment of the invention, the image quality evaluation parameters are determined according to the fitness, the current image quality evaluation model is recombined and updated for simulating the vicious and the vicious, the optimal solution is gradually approached, and then the step S208 is returned to repeat the processing iteration.
Step S216, determining the current image quality evaluation model as an image quality evaluation model constructed and trained on the basis of the convolutional neural network.
Step S218, inputting the image to be evaluated into an image quality evaluation model which is trained by a multitask learning method and is based on a convolutional neural network, and obtaining a prediction quality score evaluation result of the image sample.
The extraction rule of the natural attribute feature of the sample image in the image quality evaluation method in fig. 2 is shown in fig. 3 and is described in detail as follows.
Step S302, determining a mean contrast normalization coefficient MSCN of each pixel point in the sample image under the original scale.
In the embodiment of the present invention, the mean contrast normalization coefficient MSCN of each pixel point in the sample image at the original scale includes a local mean, a variance, and a neighborhood coefficient of the sample image at the pixel point at the original scale.
In the embodiment of the invention, the pixel value I (I, j) of the sample image at the pixel point coordinate (I, j) under the original scale is obtained, wherein I belongs to 1,2 … M, j belongs to 1,2 … N, M and N respectively represent the height and width of the image. Determining a local mean value mu (I, j) and a local variance sigma (I, j) of the sample image at the pixel point coordinate (I, j) according to the pixel value I (I, j), wherein the calculation formulas are respectively as follows:
wherein K and L are positive integers, and w = { w = { (w) (k,l)Taking | K = -K, …, K, L = -L, …, L } as a two-dimensional Gaussian weighting function with central symmetry, and taking K = L = 7, w k,l And
and (4) the pixel value of the sample image at the position of K or L of the displacement at the pixel point coordinate (i, j) under the original scale is obtained.
Calculating the pixel value I (I, j), the local mean value mu (I, j) and the local variance sigma (I, j) of the pixel point coordinate (I, j) of the obtained sample image under the original scale to obtain the neighborhood coefficient
Namely:
wherein, in order to avoid the denominator being 0 in the calculation process, a constant C =1 is taken.
And step S304, fitting the MSCN coefficient according to a generalized Gaussian distribution algorithm, and determining a first shape parameter and a variance.
In the embodiment of the invention, the extracted MSCN coefficient is fitted according to the generalized Gaussian distribution GGD, and the first shape parameter alpha and the variance sigma in the GGD are estimated by adopting a quick matching mode2And the two natural attribute characteristics are taken as the two natural attribute characteristics of the image. Wherein, the expression of the generalized Gaussian distribution is as follows:
in the formula, β and gamma function Γ () are:
wherein X is a mean contrast normalization coefficient MSCN of the sample image, i.e.
。
And step S306, respectively carrying out neighborhood product calculation on the MSCN coefficient of the sample image according to a plurality of preset directions.
In the embodiment of the invention, the product of neighborhood coefficients of the image is statistically modeled from the horizontal direction H, the vertical direction V, the main diagonal direction D1 and the negative diagonal direction D2 of the sample image at the pixel point coordinates (i, j), namely
And obtaining the adjacent domain coefficient product values of the sample image in four directions at the pixel point coordinates (i, j) under the original scale.
Step S308, fitting the neighborhood product calculation results of the MSCN coefficients of the sample image in the multiple directions under the original scale according to an asymmetric generalized Gaussian distribution algorithm, and determining the distribution mean eta, the second shape variance v and the left difference sigma of the sample image in the multiple directions under the original scalel 2The difference sigma in right directionr 2。
In the embodiment of the present invention, an asymmetric generalized gaussian distribution model is used to fit the neighborhood coefficient products in the four directions, that is:
in the formula (I), the compound is shown in the specification,
according to the fitting result of the asymmetric generalized Gaussian distribution model aiming at the neighborhood coefficient product, four natural attribute features, namely a distribution mean eta, a second shape variance v and a left difference sigma, are respectively extracted in each directionl 2The difference sigma in right directionr 2。
And step S310, constructing a natural attribute feature real result of the sample image under the original scale according to the first shape parameter, the variance, the distribution mean, the second shape variance, the left difference and the right difference.
In the embodiment of the invention, a plurality of images are collected from an image quality evaluation data set as sample images, and natural attribute features of the sample images are extracted through a specific extraction rule, wherein the natural attribute features of the sample images comprise a first shape parameter alpha and a variance sigma2Distribution mean η, second shape variance v, left difference σl 2The difference sigma in right directionr 2. The natural attribute extraction rule can extract 18 natural attribute features of the sample image under the original scale.
In step S312, an image of the sample image at the down-sampling one-half scale is obtained.
In the embodiment of the invention, the sample image is normalized after being read to obtain the image of the sample image under the down-sampling one-half scale.
Step S314, natural attribute feature extraction operation is carried out on the down-sampling image, and a real result of the natural attribute feature of the sample image under a half scale is determined.
In the embodiment of the present invention, the rule for extracting the real result of the natural attribute feature of the sample image at one-half scale is the same as the step flowchart in fig. 4, and is not described herein again.
And step S316, constructing a natural attribute feature response result of the sample image according to the natural attribute feature real result of the sample image under the half scale and the natural attribute feature real result of the sample image under the original scale.
In the embodiment of the invention, the natural attribute feature extraction operation is respectively carried out under the original scale and the down-sampling half scale of the sample image, the natural attribute features extracted under the original scale and the down-sampling half scale of the sample image are combined to construct the natural attribute feature response result of the sample image, and 36 natural attribute features are calculated in total.
And step S318, constructing a training sample and a training label according to the sample pattern, the image natural attribute feature true result and the image subjective quality score.
As shown in fig. 4, a flowchart of the steps for determining the mean contrast normalization coefficient MSCN of the sample image in the image quality assessment method according to the embodiment of the present invention is described in detail as follows.
Step S402, a pixel value at a pixel point of the sample image is acquired.
In the embodiment of the present invention, the pixel value of the sample image at the pixel point is the pixel value I (I, j) of the sample image at the pixel point coordinate (I, j) at the original scale and the down-sampling half scale.
Step S404, determining a local mean and a variance of the sample image at the pixel point according to the pixel value of the sample image at the pixel point.
In the embodiment of the present invention, the method for determining the local mean and the variance of the sample image at the pixel point is as described in step S302, and is not repeated herein.
S406, determining the neighborhood coefficient of the sample image at the pixel point according to the pixel value, the local mean and the variance of the sample image at the pixel point.
In the embodiment of the present invention, the method for determining the neighborhood coefficients of the sample image at the pixel point is as described in step S302, and is not repeated herein.
As shown in fig. 5, a flowchart of the steps of determining the loss value between the response result of the natural attribute feature and the true result of the natural attribute feature in the image quality evaluation method according to the embodiment of the present invention is described in detail as follows.
Step S502, inputting the training samples and the training labels into an image quality evaluation model to obtain natural attribute characteristic pre-estimated values and quality score pre-estimated values of all the training samples by the image evaluation model.
And step S504, inputting a preset loss function according to the obtained natural attribute characteristic estimated value and the quality score estimated value, and the real value of the natural attribute characteristic and the subjective quality score of the sample to obtain a training loss value.
In the embodiment of the present invention, the calculation formula of the total loss value is as follows:
where n is the number of input sample images, FiAnd SiRespectively representing the natural attribute characteristics of the ith sample image and the real results of the quality evaluation scores,
and
respectively representing the image response result of the model for the ith input sample.
Step S506, determining whether the neural network model based on the convolutional neural network is converged according to the obtained loss value, if not, executing step S608, returning to step S602, predicting the sample image by the current image quality evaluation model, obtaining natural attribute feature pre-evaluation values and quality score pre-evaluation values of all training samples by the image evaluation model, and if so, executing step S610.
Step S508, adjusting the variable parameters in the image quality evaluation model, and determining an updated image quality evaluation model.
Step S510, determining the current image quality evaluation model as an image quality evaluation model constructed and trained based on the convolutional neural network.
As shown in fig. 6, an image quality evaluation apparatus provided in an embodiment of the present invention specifically includes the following units.
A sample image acquisition unit 610 for acquiring an image to be quality evaluated.
In the embodiment of the invention, the image to be subjected to quality evaluation is a two-dimensional digital image and is used for evaluating whether the image quality of the two-dimensional digital image is damaged in the processes of acquisition, storage, transmission and the like.
And the image to be quality-evaluated processing unit 620 is configured to process the image to be quality-evaluated according to an image quality evaluation model based on a convolutional neural network generated according to the multitask learning training, and determine a quality evaluation result of the image.
In the embodiment of the invention, the convolutional neural network model algorithm is sequentially connected with 3 modules consisting of convolutional layers and residual blocks, wherein the residual block consists of two convolutional layers which are sequentially connected, a maximum pooling layer is connected behind each residual block, downsampling is carried out through the maximum pooling layer, two full-connection layers FC1 and FC2 are connected behind the last maximum pooling layer, then the maximum pooling layer is connected to a natural attribute feature regression layer FC3 for executing subtask learning, and then a full-connection layer FC4 and a regression layer FC5 containing a neuron for executing a quality score learning task are connected.
In the embodiment of the present invention, the convolution kernel size and padding of the convolutional layer are respectively: the first convolutional layer convolution kernel size is 5 × 5, padding is 2; the sizes of convolution kernels of other convolutional layers are all 3 multiplied by 3, and padding is 1; the number of the channels of the convolutional layers is as follows in sequence: 32, 32, 64, 64, 64; the window size of the maximum pooling layer is 2 multiplied by 2, and the step length is 2; the number of the neurons of the full connection layer is as follows in sequence: 1024, 512, 36, 512, 1; and the activation function of the convolutional neural network model algorithm is a Relu function.
In the embodiment of the invention, the image quality evaluation result is determined and output according to the extracted data in the image to be evaluated by the image quality evaluation model. The image quality evaluation comprises two parameter factors, namely a natural attribute characteristic and a quality evaluation score, and the parameter characteristic in the image to be evaluated is processed and identified, so that the quality evaluation result of the image is accurately identified and output.
The image quality evaluation device provided by the embodiment of the invention trains the initialized image quality evaluation model according to a convolutional neural network model algorithm, so that the optimized and adjusted image quality evaluation model is infinitely close to the natural attribute characteristics of the sample image and the real result and the response result of the quality evaluation score, then the image to be evaluated is evaluated by the image quality evaluation model to obtain the quality prediction score of the image, the quality prediction score of the image is processed to obtain the objective quality score of the image to be evaluated, the accuracy of the model parameters is improved, and the extension capability and the prediction capability of the finally established image quality evaluation model are effectively improved.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (6)
1. An image quality evaluation method, characterized in that the method comprises:
acquiring an image to be subjected to quality evaluation;
processing the image to be quality-evaluated according to an image quality evaluation model which is generated according to multi-task learning training and is constructed and trained on the basis of a convolutional neural network, and determining the quality evaluation result of the image; wherein
The image quality evaluation model constructed based on the convolutional neural network is generated by training with image natural attribute characteristics and quality scores as a multi-task learning target; the natural attribute feature task learning is a subtask taking natural attribute features obtained by feature extraction of images according to a preset natural attribute feature extraction rule as model learning, and the image quality score learning is another subtask taking an image subjective quality score as model learning.
2. The image quality assessment method according to claim 1, wherein the step of training and generating the image quality assessment model specifically comprises:
acquiring a plurality of training sample images from an image quality evaluation database;
extracting a natural attribute feature real result of the sample image according to a preset natural attribute feature extraction rule, and constructing a training sample and a training label by combining image subjective quality scores;
constructing an initialized multi-task learning image quality evaluation model containing variable parameters;
inputting the training samples and the training labels into an image quality evaluation model to obtain natural attribute characteristic pre-evaluation values and quality score pre-evaluation values of the image evaluation model to all the training samples;
calculating training loss values between the natural attribute characteristic estimated value and the quality score estimated value and between the natural attribute characteristic real result and the subjective quality score;
judging whether the image quality evaluation model is trained according to whether the loss value is converged;
when the judgment is negative, adjusting variable parameters in the image quality evaluation model according to a gradient descent algorithm and the loss value, determining an updated image quality evaluation model, and returning to the step of predicting the sample image according to the current image quality evaluation model to obtain natural attribute feature pre-evaluation values and quality score pre-evaluation values of all training samples by the image evaluation model;
when the judgment is yes, determining the current image quality evaluation model as an image quality evaluation model constructed and trained on the basis of the convolutional neural network;
and inputting the image to be evaluated into an image quality evaluation model which is trained by using a multi-task learning method and is based on a convolutional neural network, and obtaining an evaluation result of the image sample.
3. The image quality evaluation method according to claim 2, wherein the image quality evaluation model is generated by training image natural attribute features and quality scores as a multitask learning target;
extracting a natural attribute feature real result of the sample image according to a preset natural attribute feature extraction rule, and constructing a training sample and a training label by combining image subjective quality scores, wherein the method specifically comprises the following steps:
determining a mean contrast normalization coefficient MSCN of each pixel point in the sample image;
fitting MSCN coefficients according to a generalized Gaussian distribution algorithm, and determining a first shape parameter and a variance;
performing neighborhood product calculation on MSCN coefficients of the sample image according to a plurality of preset directions;
fitting neighborhood product calculation results of MSCN coefficients of the sample image in the multiple directions according to an asymmetric generalized Gaussian distribution algorithm, and determining a distribution mean, a second shape variance, a left difference and a right variance of the sample image in the multiple directions;
constructing a natural attribute feature real result of the sample image under the original scale according to the first shape parameter, the variance, the distribution mean, the second shape variance, the left difference and the right difference;
acquiring a down-sampling image of the sample image under a half scale;
performing natural attribute feature extraction operation on the down-sampling image, and determining a real result of the natural attribute feature of the sample image under a half scale;
constructing a natural attribute feature real result of the sample image according to the natural attribute feature real result of the sample image under the half scale and the natural attribute feature real result of the sample image under the original scale;
and constructing a training sample and a training label according to the sample image, the image natural attribute characteristic real result and the image subjective quality score.
4. The image quality evaluation method according to claim 3, wherein the step of determining the mean contrast normalization coefficient MSCN of each pixel in the sample image specifically comprises:
obtaining pixel values at pixel points of the sample image;
determining a local mean and a variance of the sample image at the pixel point according to the pixel value of the sample image at the pixel point;
and determining the neighborhood coefficient of the sample image at the pixel point according to the pixel value, the local mean and the variance of the sample image at the pixel point.
5. The image quality evaluation method according to claim 1, wherein the image quality model constructed based on the convolutional neural network is obtained by training through the following steps:
inputting the training samples and the training labels into an image quality evaluation model to obtain natural attribute characteristic pre-evaluation values and quality score pre-evaluation values of the image evaluation model to all the training samples;
inputting a preset loss function according to the obtained natural attribute characteristic estimated value and the quality score estimated value, and the real value of the natural attribute characteristic and the subjective quality score of the sample to obtain a training loss value;
determining whether the neural network model based on the convolutional neural network converges according to the obtained loss value; if not, adjusting variable parameters in the image quality evaluation model, determining an updated image quality evaluation model, and returning to the step of predicting the sample image according to the current image quality evaluation model to obtain natural attribute feature estimated values and quality score estimated values of all training samples by the image evaluation model;
and if so, determining the current image quality evaluation model as an image quality evaluation model constructed and trained on the basis of the convolutional neural network.
6. An image quality evaluation apparatus characterized by comprising:
the sample image acquisition unit is used for acquiring an image to be subjected to quality evaluation;
and the image processing unit for quality evaluation is used for processing the image to be quality evaluated according to the image quality evaluation model which is generated by multi-task learning training and is based on the convolutional neural network, and determining the quality evaluation result of the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110253923.8A CN112766419A (en) | 2021-03-09 | 2021-03-09 | Image quality evaluation method and device based on multitask learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110253923.8A CN112766419A (en) | 2021-03-09 | 2021-03-09 | Image quality evaluation method and device based on multitask learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112766419A true CN112766419A (en) | 2021-05-07 |
Family
ID=75690912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110253923.8A Pending CN112766419A (en) | 2021-03-09 | 2021-03-09 | Image quality evaluation method and device based on multitask learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112766419A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113222032A (en) * | 2021-05-19 | 2021-08-06 | 西安电子科技大学 | No-reference image quality evaluation method based on self-attention image coding |
| CN113407774A (en) * | 2021-06-30 | 2021-09-17 | 广州酷狗计算机科技有限公司 | Cover determining method and device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510485A (en) * | 2018-03-27 | 2018-09-07 | 福州大学 | It is a kind of based on convolutional neural networks without reference image method for evaluating quality |
| CN109215028A (en) * | 2018-11-06 | 2019-01-15 | 福州大学 | A kind of multiple-objection optimization image quality measure method based on convolutional neural networks |
| CN110189291A (en) * | 2019-04-09 | 2019-08-30 | 浙江大学 | A kind of general non-reference picture quality appraisement method based on multitask convolutional neural networks |
| CN111127435A (en) * | 2019-12-25 | 2020-05-08 | 福州大学 | No-reference image quality evaluation method based on double-current convolutional neural network |
| CN112184672A (en) * | 2020-09-30 | 2021-01-05 | 上海大学 | No-reference image quality evaluation method and system |
| CN112348809A (en) * | 2020-11-30 | 2021-02-09 | 天津大学 | No-reference screen content image quality evaluation method based on multitask deep learning |
-
2021
- 2021-03-09 CN CN202110253923.8A patent/CN112766419A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510485A (en) * | 2018-03-27 | 2018-09-07 | 福州大学 | It is a kind of based on convolutional neural networks without reference image method for evaluating quality |
| CN109215028A (en) * | 2018-11-06 | 2019-01-15 | 福州大学 | A kind of multiple-objection optimization image quality measure method based on convolutional neural networks |
| CN110189291A (en) * | 2019-04-09 | 2019-08-30 | 浙江大学 | A kind of general non-reference picture quality appraisement method based on multitask convolutional neural networks |
| CN111127435A (en) * | 2019-12-25 | 2020-05-08 | 福州大学 | No-reference image quality evaluation method based on double-current convolutional neural network |
| CN112184672A (en) * | 2020-09-30 | 2021-01-05 | 上海大学 | No-reference image quality evaluation method and system |
| CN112348809A (en) * | 2020-11-30 | 2021-02-09 | 天津大学 | No-reference screen content image quality evaluation method based on multitask deep learning |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113222032A (en) * | 2021-05-19 | 2021-08-06 | 西安电子科技大学 | No-reference image quality evaluation method based on self-attention image coding |
| CN113222032B (en) * | 2021-05-19 | 2023-03-10 | 西安电子科技大学 | No-reference image quality evaluation method based on self-attention image coding |
| CN113407774A (en) * | 2021-06-30 | 2021-09-17 | 广州酷狗计算机科技有限公司 | Cover determining method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110110799B (en) | Cell sorting method, cell sorting device, computer equipment and storage medium | |
| CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
| CN111915572B (en) | Adaptive gear pitting quantitative detection system and method based on deep learning | |
| CN110728656A (en) | Meta-learning-based no-reference image quality data processing method and intelligent terminal | |
| CN111709914B (en) | Non-reference image quality evaluation method based on HVS characteristics | |
| CN110929687B (en) | Multi-user behavior recognition system based on key point detection and working method | |
| CN112766419A (en) | Image quality evaluation method and device based on multitask learning | |
| CN117355038B (en) | X-shaped hole processing method and system for circuit board soft board | |
| CN109918542B (en) | Convolution classification method and system for relational graph data | |
| CN112861718A (en) | Lightweight feature fusion crowd counting method and system | |
| CN111160389A (en) | Lithology identification method based on fusion of VGG | |
| CN112950570A (en) | Crack detection method combining deep learning and dense continuous central point | |
| CN112070720A (en) | Transformer substation equipment defect identification method based on deep learning model | |
| CN111860407A (en) | Method, device, equipment and storage medium for recognizing expressions of characters in video | |
| CN111915595A (en) | Image quality evaluation method, and training method and device of image quality evaluation model | |
| CN113421223B (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN107633527B (en) | Target tracking method and device based on full convolution neural network | |
| CN110874835A (en) | Crop leaf disease resistance identification method and system, electronic equipment and storage medium | |
| CN117934463A (en) | Beef cattle carcass quality grading method based on optical test | |
| Pang et al. | Convolutional neural network-based sub-pixel line-edged angle detection with applications in measurement | |
| CN113255543B (en) | Facial expression recognition method based on graph convolution network | |
| CN117078670B (en) | Production control system of cloud photo frame | |
| CN111860601B (en) | Method and device for predicting type of large fungi | |
| CN113743443A (en) | Image evidence classification and identification method and device | |
| CN117315380A (en) | Deep learning-based pneumonia CT image classification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210507 |