CN109118470B - Image quality evaluation method and device, terminal and server - Google Patents
Image quality evaluation method and device, terminal and server Download PDFInfo
- Publication number
- CN109118470B CN109118470B CN201810669989.3A CN201810669989A CN109118470B CN 109118470 B CN109118470 B CN 109118470B CN 201810669989 A CN201810669989 A CN 201810669989A CN 109118470 B CN109118470 B CN 109118470B
- Authority
- CN
- China
- Prior art keywords
- image
- quality
- standard
- feature map
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 84
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 52
- 238000011156 evaluation Methods 0.000 claims abstract description 19
- 238000010586 diagram Methods 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 6
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 239000011521 glass Substances 0.000 description 4
- 210000003128 head Anatomy 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 240000001114 Aniseia martinicensis Species 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013077 scoring method Methods 0.000 description 2
- 208000009119 Giant Axonal Neuropathy Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 201000003382 giant axonal neuropathy 1 Diseases 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004379 similarity theory Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image quality evaluation method, an image quality evaluation device, a terminal and a server, wherein the method comprises the steps of obtaining a suspected standard image and a reference image set; taking the suspected standard image as an evaluation standard, obtaining the quality score of each reference image in the reference image set, and obtaining the corresponding relation between the reference image and the quality score; and constructing a training set according to the reference image and the quality score, training a quality evaluation model, and evaluating the quality of the image to be evaluated according to the quality evaluation model. According to the invention, through training the quality evaluation model, objective scores can be given to any image on the premise that a standard image is not available, so that whether the image meets the specific requirements of industry or security can be accurately judged.
Description
Technical Field
The invention relates to the field of computers, in particular to an image quality evaluation method, an image quality evaluation device, a terminal and a server.
Background
In the technical field of image evaluation, the prior art mainly includes an evaluation method based on PSNR and an evaluation method based on SSIM, but these two evaluation methods cannot reflect real actual image quality, are difficult to apply to complex environments, and are difficult to meet security requirements. In addition, in the prior art, the image quality evaluation depends on the standard image, and the standard image has high requirement, so that the standard image cannot be obtained in many application scenes, and the application range of the image quality evaluation method is limited.
Disclosure of Invention
In order to solve the technical problem, the invention provides an image quality evaluation method, an image quality evaluation device, a terminal and a server. The invention is realized by the following technical scheme:
in a first aspect, an image quality evaluation method includes:
acquiring an original image, wherein the original image meets the basic condition of generating a standard image;
generating a suspected standard image according to the original image;
obtaining a reference image set corresponding to the original image;
taking the suspected standard image as an evaluation standard, obtaining the quality score of each reference image in the reference image set, and obtaining the corresponding relation between the reference image and the quality score;
constructing a training set according to the reference images and the quality scores, and training a quality evaluation model, wherein the quality evaluation model takes the images as input and takes the quality scores corresponding to the images as output;
acquiring an image to be evaluated;
and inputting the image to be evaluated into the quality evaluation model, and obtaining the quality score of the image to be evaluated. .
In a second aspect, an image quality evaluation apparatus includes:
the original image acquisition module is used for acquiring an original image, and the original image meets the basic condition of generating a standard image;
the suspected standard image generating module is used for generating a suspected standard image according to the original image;
a reference image set obtaining module, configured to obtain a reference image set corresponding to the original image;
the reference image quality score acquisition module is used for acquiring the quality scores of all reference images in the reference image set by taking the suspected standard images as evaluation standards to obtain the corresponding relation between the reference images and the quality scores;
the training module is used for constructing a training set according to the reference images and the quality scores, training a quality evaluation model, and outputting the quality evaluation model by taking the images as input and the quality scores corresponding to the images as output;
the image to be evaluated acquiring module is used for acquiring an image to be evaluated;
and the quality score output module is used for inputting the image to be evaluated into the quality evaluation model and obtaining the quality score of the image to be evaluated.
In a third aspect, a computer-readable storage medium stores a program for implementing the image quality evaluation method described above.
In a fourth aspect, a server is used for operating the image quality evaluation device.
In a fifth aspect, a terminal is provided for operating an image quality evaluation apparatus as described above.
The invention provides an image quality evaluation method, an image quality evaluation device, a terminal and a server, which have the following beneficial effects:
(1) by training the quality evaluation model, objective scores can be given to any image on the premise that a standard image is not available, and whether the image meets the specific requirements of industry or security or not can be accurately judged.
(2) By the method for generating the suspected standard image, the high-resolution image can be obtained by using the low-resolution image, and the high-resolution image quality can meet the requirement of the standard image. Therefore, when the quality evaluation model is trained, the excessive dependence on the resolution of the input image can be avoided, and the standard image is not a necessary condition for image evaluation any more.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of an image quality evaluation method according to an embodiment of the present invention;
FIG. 2 is a flowchart of a suspected standard image generation method provided in an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating an implementation principle of GAN according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a basic principle of a residual error network provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram of a countermeasure generation network according to an embodiment of the present invention;
fig. 6 is a flowchart of a method for obtaining a reference image quality score according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a training logic of a quality assessment model provided by an embodiment of the present invention;
fig. 8 is a block diagram of an image quality evaluation apparatus according to an embodiment of the present invention;
FIG. 9 is a block diagram of a reference image quality score obtaining module according to an embodiment of the present invention;
fig. 10 is a block diagram of a reference image quality score obtaining unit according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of a server according to an embodiment of the present invention;
fig. 12 is a block diagram of a terminal according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the prior art, the PSNR-based evaluation method uses the PSNR value to evaluate the quality of an image, and a larger PANR value indicates a higher quality of the image. PSNR (peak signal to noise ratio), the most common and most widely used objective measurement method for evaluating image quality, but many experimental results show that the PSNR score cannot be completely consistent with the visual quality seen by human eyes, and there is a possibility that a person with a high PSNR looks worse than a person with a low PSNR score. This is because the sensitivity of human vision to errors is not absolute, and the perception result is affected by many factors (e.g., human eyes are more sensitive to contrast differences with low spatial frequency, human eyes are more sensitive to luminance contrast differences, and human eyes are affected by neighboring areas around the neighboring areas).
In the SSIM-based evaluation method, the SSIM value is an index for evaluating structural similarity information of an image, and a larger SSIM value indicates a higher structural similarity between the processed image and an original image. Structural similarity ranges from 0 to 1. When the two images are identical, the value of SSIM is equal to 1. As an implementation of the structural similarity theory, the structural similarity index defines structural information from the perspective of image composition as being independent of brightness and contrast, reflects attributes of object structures in a scene, and models distortion as a combination of three different factors of brightness, contrast, and structure. The mean is used as an estimate of the luminance, the standard deviation as an estimate of the contrast, and the covariance as a measure of the degree of structural similarity. In image quality evaluation, local SSIM index is better than global, but the local size and weight require a priori knowledge.
In conclusion, the two evaluation methods cannot reflect the real actual image quality or depend on the prior knowledge excessively, so that the practical value is limited.
Further, in the prior art, there is a high requirement for the resolution of the standard image used in the image quality evaluation, which makes it impossible to obtain the standard image and further to evaluate the quality of the image to be evaluated in many application scenarios, which significantly limits the application range of the image quality evaluation method. Therefore, in order to evaluate the quality in any scene, the technical problem of standard image acquisition must be solved, which needs to perform super-resolution reconstruction on the low-resolution image to obtain a suspected standard image capable of replacing the standard image. However, the super-resolution reconstruction technique in the prior art is not mature. The image reconstruction in the prior art mainly uses the following methods:
(1) the core idea of the bilinear interpolation algorithm, which is a bilinear interpolation difference value, is to perform linear interpolation in two directions respectively. The method can calculate interpolation values through weighted average of four pixel values in a 2x2 area, so that obvious 'mosaic' phenomenon cannot be generated when the generated pixels are interpolated into an image; on the other hand, when bilinear interpolation is adopted for four adjacent pixel points, the surfaces of the obtained images are consistent in the field, when the slopes are not consistent, the smoothing effect of the surfaces of the obtained images may be that the details of the images are degraded, that is, after the bilinear interpolation processing is carried out, the high-frequency information of the images may be degraded, and the phenomenon is particularly obvious in the super-resolution reconstruction process of the images.
(2) bicubic interpolation, i.e., bicubic interpolation. In this method, the new pixel value is obtained by a weighted average of 16 points in its vicinity, and an interpolation operation is also required once in each of two directions. At present, bicubic interpolation can generally interpolate the most effective and accurate interpolation image, but the speed is almost slowest due to complex calculation.
As can be seen, a technical solution capable of obtaining a super-resolution image with high quality at a high speed is not available in the prior art.
In order to achieve the purpose of objective picture quality evaluation in any scene, the embodiment of the invention mainly solves three technical problems:
(1) obtaining a fine and vivid super-resolution image based on the low-resolution image, and taking the fine and vivid super-resolution image as a quality evaluation standard, thereby solving the technical problem of standard image acquisition;
(2) an objective and scientific scoring method is adopted, so that the technical problem of quality evaluation under the premise of standard images is solved;
(3) and (3) training the neural network according to the scoring method in the step (2) to obtain a model capable of performing quality evaluation in a scene without a standard image, so that the technical problem that objective picture quality evaluation can be performed in any scene is solved.
An embodiment of the present invention provides an image quality evaluation method, as shown in fig. 1, including:
s101, obtaining an original image, wherein the original image meets the basic condition of generating a standard image.
And S102, generating a suspected standard image according to the original image.
In the prior art, a standard image needs to be acquired for image quality evaluation, and other images are compared with the standard image to obtain objective quality evaluation of the other images. The common image is limited by hardware storage space and format, generally has low resolution, and is not enough to be used as a standard image in some occasions, so that the difficulty in acquiring the standard image in the prior art is increased. The steps S101-S102 can carry out super-resolution reconstruction on the original image with low resolution meeting the basic condition to obtain the suspected standard image with high resolution.
In the embodiment of the present invention, a standard image is not required to be acquired, but a normal image satisfying the basic conditions of the standard image is used, and a standard for evaluating image quality (suspected standard image) is obtained by processing the normal image, because the suspected standard image is not acquired directly by an image acquirer, but is generated by the normal image in the embodiment of the present invention, the suspected standard image is referred to as a suspected standard image in the embodiment of the present invention, and the suspected standard image is also used as the standard for evaluating image quality in the embodiment of the present invention.
In particular, the basic condition is related to the actual image acquisition scenario and is independent of the performance of the acquisition device. For example, in an identification scene, the following requirements exist for a standard image of a certificate photo: the citizen himself, the front, the crown-free and the color head portrait, the head occupies 2/3 of the size of the photo, the citizen who usually wears glasses should wear the glasses without making clothes or a white jacket, the white background has no frame, the portrait is clear, the layers are rich, no obvious distortion exists, and the resolution ratio reaches 350 dpi. In the requirements, the citizen himself, the front, the crown-free, the color head portrait, the head of which occupies 2/3 of the size of a photo, no standard clothing or white jacket, the resident who wears glasses frequently should wear the glasses, and the white background has no frame, which is related to the image acquisition scene and is unrelated to the performance of the acquisition equipment, belongs to the basic conditions described in the embodiment of the invention, and the citizen portrait is clear, rich in layers and free from obvious distortion, and the resolution reaches 350 dpi' which is related to the performance of the acquisition equipment, and does not belong to the basic conditions described in the embodiment of the invention.
And S103, obtaining a reference image set corresponding to the original image.
And the reference images in the reference image set and the original image have corresponding relations, and the corresponding relations are embodied in the way that the reference images and the original image describe the same things. For example, the original image is a low-resolution Zhang's identification photograph, and the reference image may be Zhang's facial photograph based on different angles of spatial variation, and so on.
And S104, taking the suspected standard image as an evaluation standard, obtaining the quality score of each reference image in the reference image set, and obtaining the corresponding relation between the reference image and the quality score.
The reference image in step S104 may be an image at different angles and under different environments, and an evaluation result of a score of the reference image with respect to the super-resolution reconstructed image (suspected standard image) is obtained.
It is difficult and subjective to artificially calibrate the quality score of an image relative to a standard image, and an unqualified calibration may have a great influence on quality evaluation, so a standard objective quality evaluation method should be used to evaluate the reference image in S104, and the specific evaluation method is described in detail in the following section.
And S105, constructing a training set according to the reference images and the quality scores, and training a quality evaluation model, wherein the quality evaluation model takes the images as input and the quality scores corresponding to the images as output.
For different original images, different corresponding relations between the reference images and the quality scores exist, a plurality of training sets also exist, and a better quality evaluation result can be obtained by continuously increasing the training sets and continuously optimizing the quality evaluation model.
Specifically, the quality evaluation model in the embodiment of the present invention may fit the scores of the images based on the space (different deflection angles) in the training set, and further may evaluate the quality of the image to be evaluated without a standard illumination.
For each training set, it can be obtained using the method of steps S101-S104.
And S106, obtaining an image to be evaluated.
And S107, inputting the image to be evaluated into the quality evaluation model, and obtaining the quality score of the image to be evaluated.
In the embodiment of the invention, the suspected standard image is generated by processing the common image, so that the excessive dependence on the image quality of the original image in the image quality evaluation in the prior art is avoided, namely, the processing result can be used as the standard of the image quality evaluation in a mode of processing the image which has poor image quality and meets the basic condition. Further, in the embodiment of the invention, the objective quality evaluation standard of the image can be obtained by training the quality evaluation model, so that after the quality evaluation model is obtained, the quality evaluation can be performed on any image without providing a standard image corresponding to the any image. The characteristic enables the embodiment of the invention to be widely applied to scenes which cannot provide standard images, and can more scientifically evaluate the quality of the images.
As can be seen from the above, in the embodiment of the present invention, the original image in step S101 needs to be subjected to image processing to obtain the pseudo standard image. Although the original image meets the basic conditions of the standard image in the embodiment of the invention, the image quality of the original image cannot be used as the standard image or directly used for quality evaluation, for example, the original image has low resolution and poor definition. The suspected standard image generation method is shown in fig. 2, and includes:
s201, constructing a countermeasure generation network, wherein the countermeasure generation network comprises a generator network and a discriminator network, and a residual error network is embedded in the generator network.
The confrontation generation network (GAN) is a network constructed based on a two-player zero-sum game idea (two-player), the sum of benefits of two game parties is a constant, and two game players exist in the GAN and are a generator network and a discriminator network respectively. Both the generator network and the arbiter network can be seen as a black box, resembling an input-output map. The function of the generator network is to take noisy samples and pack them into a realistic sample for output. The function of the discriminator network is to determine whether the input sample is true or false.
The principle of execution of GAN is shown in fig. 3, and the task of GAN in the embodiment of the present invention is to simulate an image by inputting a noise sample, and the image can be very vivid so as to be spurious. The generator of GAN for this purpose can be used to generate the pseudo standard image from the original image in step S102.
From the above analysis, the execution purpose of the generator network is just opposite to that of the discriminator network, and the generator network is in order to judge the real image and to be in the false. And (4) training GANs of both game parties, wherein the purpose of the training is to obtain samples which are falsified and mistrued, so that the discriminator network cannot distinguish the true samples from the false samples.
S202, obtaining a generator training set, wherein the generator training set comprises low-resolution original images and high-resolution standard images corresponding to the low-resolution original images.
And S203, training the confrontation generating network according to the generator training set, and obtaining a generator network capable of meeting preset requirements.
And S204, inputting the original image into the generator network to obtain a suspected standard image.
The purpose of the embodiment of the invention is that the generator network generates a corresponding image with high resolution according to a low-resolution original image, and the discriminator network cannot distinguish whether the corresponding image with high resolution and the standard image with high resolution are true or false, so that the training of resisting the generation network can be completed, and the generator network meeting the requirements of the embodiment of the invention is obtained. The generator network may be used to generate a suspected standard image.
The GAN network in the embodiment of the present invention can generate an image very similar to the input image, and generate a pseudo standard image corresponding to the original image in step S101 using GAN based on this property. The GAN is based on the principle that a generator network is used to generate a high-resolution image, and a discriminator network is used to judge the authenticity of the high-resolution image generated by the generator network, and the process is repeated until the discriminator network cannot judge the authenticity of the input image, which is particularly embodied in that for any input high-resolution image, the probability of authenticity given by the discriminator network is 0.5, the generator network is required by the embodiment of the invention, and the discriminator can be discarded. Namely, the embodiment of the invention uses the trained generator network to generate the suspected standard image.
However, in the GAN training process, after the number of network layers reaches a certain number, the performance of the network is saturated, the performance of the network begins to degrade when the network is increased, the training precision and the testing precision are both reduced, and in order to ensure the training precision and prevent the time and the computation complexity from rapidly increasing when the network depth is increased, thereby ensuring the rapid convergence and avoiding the gradient extinction and gradient dispersion phenomena, the embodiment of the present invention embeds the residual network in the generator network.
As shown in fig. 4, which illustrates the basic principle of the residual network, the residual network uses a hopping structure as the basic structure of the network, and converts the optimization target from h (x) to h (x) -x through the hopping structure, where h (x) ═ f (x) + x, so that the deep network can achieve the same effect as the shallow network by performing an equivalent mapping on the upper layers on the basis of the shallow network, thereby significantly reducing the training difficulty.
Specifically, as shown in fig. 5, the residual network in the embodiment of the present invention designs 8 residual blocks (residual blocks), each of which includes a convolutional layer (Conv) and a normalization layer (batchnorm). The number of the residual blocks can be automatically adjusted according to the complexity of the task before network training, and the higher the complexity of the task is, the more the number of the residual blocks can be designed.
In fig. 5, not only the training precision and the training speed can be improved by the embedded residual error network in the generator network, but also the training precision and the training speed can be improved by the embedded residual error network in the discriminator network.
The method includes inputting a Low resolution image (Low resolution image) and outputting a High resolution image (High resolution image) in a generator network, the generator network including a convolution layer (Conv), a normalization layer (Batchnorm), an active layer (PReLU), and a residual error network (N residual block), and inputting a High fraction image and a real image output by the generator network and outputting a discrimination result by a discriminator network, the discriminator network and the generator network having a similar structure.
The basic principle of the generator network for generating high resolution images is as follows:
high-dimensional information (feature map) of the low-resolution image is extracted using a residual network, and a high-resolution image is reconstructed based on the high-dimensional information. Taking a 100 × 100 image as an example, assuming that the size of the feature map formed after the image passes through the residual network is (1, 100, 100, 256), and the meaning of (1, 100, 100, 256) is that 256 100 × 100 images are obtained from a 100 × 100 image, if a subwev 2d layer is designed in the generator network, the feature map is convolved to (1, 200, 200, 64) size, and then subwev 2d is performed again, so that the new feature map size is (1, 400, 400, 16) size, and finally the new feature map is rolled to (1, 400, 400, 3), where the dimension 3 is RGB three channels, and thus the reconstruction of an input image of 100 × 100 into a super-resolution image of 400 is completed, and the total of the input image becomes 4 times of the original image in two directions and 16 times. It can be seen that if different resolutions are to be reconstructed, the number of the supbpixelconv 2d is adjusted. Assuming that the input image is 256 × 256 in size, the processed output size is 512 × 512, which is sufficient in most training of neural networks.
The suspected standard image generation method further provided by the embodiment of the invention is mainly used for reconstructing a high-resolution image by using a countermeasure generation network (GAN) and a residual error network (Resnet), so that the defects of too little information carried by the low-resolution image and low dimensionality are avoided, and excessive data concentration can be effectively avoided.
Further, after disclosing the method for generating the suspected standard image, the embodiment of the present invention further discloses a method for acquiring the reference image quality score, as shown in fig. 6, including:
s301, a first feature map corresponding to the suspected standard image is obtained.
S302, a second characteristic diagram corresponding to the reference image is obtained.
And S303, evaluating the similarity of the first feature map and the second feature map, and outputting the quality score of the reference image according to the similarity.
The first characteristic diagram and the second characteristic diagram can be obtained by using a VGG19 network in the embodiment of the invention. The VGG19 network is a currently published trained neural network model, the training data of which is millions of life images, and the suspected standard images are input into the VGG19 network to obtain the feature map of relu5-4, namely the feature map after the 5 th convolution level and the 4 th activation level, and the feature map is taken as the first feature map.
And inputting the reference image into a VGG19 network, obtaining a relu5-4 feature map corresponding to the reference image, and taking the feature map as a second feature map.
Specifically, in the step of obtaining the similarity, the euclidean distance between the first feature map and the second feature map may be calculated, and the euclidean distance may be used as a measure of the similarity. In order to obtain a uniform quality score, the euclidean distance may be mapped to the (0, 1) interval to obtain the quality score.
Further, fine tuning (fine tuning) may also be performed on the VGG19 network to achieve better results in order to obtain a more scientific quality score. The fine tune is to train a new model by using a model trained by others and adding existing data. The benefit of fine tune is that the model is not completely retrained, thereby improving efficiency.
The embodiment of the invention provides a method for evaluating the quality of a reference image by extracting features through a VGG19 network, which discards PSNR and SSIM used in the prior art and evaluates the quality of the image by using a method based on feature map similarity. Mainly, the feature map represents high-dimensional and global information of the image, so that the quality evaluation method provided by the embodiment of the invention is more objective and can reflect the difference between the image and the standard image.
Referring to fig. 7, which shows a schematic diagram of a training logic of a quality evaluation model, in an implementation scenario of the embodiment of the present invention, face shots from different angles based on spatial variation may be used as reference images, and quality scores corresponding to the reference images are obtained, at this time, a simple deep neural network may be used to train an image-quality score pair, and finally, the trained deep neural network model may be used to regress any image and obtain a quality score thereof.
In the implementation process of the embodiment of the present invention, the obtained deep neural network may be a network with a very simple structure, and therefore, the computational complexity of the image quality evaluation performed based on the method of the embodiment of the present invention is also low, and therefore, no limitation is imposed on the hardware for operation thereof, and any portable device or fixed device may operate the quality evaluation method of the embodiment of the present invention.
The image quality evaluation method provided by the embodiment of the invention can be used for solving the problem of image quality evaluation in the application scene without standard images, thereby providing powerful support for image safety in various scenes and greatly reducing the manual marking cost.
The method can still perform relatively accurate quality evaluation on the input image in the scene without the standard image, so that the method can be applied to the monitoring scene needing to ensure the image quality, such as an identity card image comparison system, a face gate system, a suspect detection system and the like, and provides warehousing image evaluation service. When the method of the embodiment of the invention is specifically realized, a high-definition camera or a common monitoring camera can be adopted in front-end hardware to collect the original image, wherein the camera can be used in each corner of various scenes, and various expanding functions are provided by each large product manufacturer or directly imported into the image; the post-correlation data processing may be performed on the server or some terminal.
An embodiment of the present invention provides an image quality evaluation apparatus, as shown in fig. 8, including:
an original image obtaining module 100, configured to obtain an original image, where the original image meets a basic condition for generating a standard image;
a suspected standard image generating module 200, configured to generate a suspected standard image according to the original image;
a reference image set obtaining module 300, configured to obtain a reference image set corresponding to the original image;
a reference image quality score obtaining module 400, configured to obtain a quality score of each reference image in the reference image set by using the suspected standard image as an evaluation standard, and obtain a corresponding relationship between the reference image and the quality score;
the training module 500 is configured to construct a training set according to the reference image and the quality score, train a quality evaluation model, and take an image as an input and a quality score corresponding to the image as an output;
an image to be evaluated obtaining module 600, configured to obtain an image to be evaluated;
and the quality score output module 700 is configured to input the image to be evaluated into the quality evaluation model, and obtain a quality score of the image to be evaluated.
As shown in fig. 9, the reference image quality score obtaining module 400 includes:
the first feature map obtaining unit 401 is configured to obtain a first feature map corresponding to the suspected standard image.
A second feature map obtaining unit 402, configured to obtain a second feature map corresponding to the reference image.
A reference image quality score obtaining unit 403, configured to evaluate similarity between the first feature map and the second feature map, and output a quality score of the reference image according to the similarity.
As shown in fig. 10, the reference image quality score obtaining unit 403 includes:
a euclidean distance calculating subunit 4031 configured to calculate euclidean distances of the first feature map and the second feature map.
And a mapping subunit 4032, configured to map the euclidean distance to the interval (0, 1) to obtain a quality score.
The image quality evaluation device and the method embodiment described in the device embodiment of the invention are based on the same inventive concept.
Embodiments of the present invention also provide a storage medium, which can be used to store program codes used in implementing the embodiments. Optionally, in this embodiment, the storage medium may be located in at least one network device of a plurality of network devices of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Specifically, fig. 11 is a schematic diagram of a server structure provided in an embodiment of the present invention, where the server structure may be used to operate an image quality evaluation apparatus. The server 800, which may vary significantly depending on configuration or performance, may include one or more Central Processing Units (CPUs) 822 (e.g., one or more processors) and memory 832, one or more storage media 830 (e.g., one or more mass storage devices) storing applications 842 or data 844. Memory 832 and storage medium 830 may be, among other things, transient or persistent storage. The program stored in the storage medium 830 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, a central processor 822 may be provided in communication with the storage medium 830 for executing a series of instruction operations in the storage medium 830 on the server 800. The server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input-output interfaces 858, and/or one or more operating systems 841, such as Windows Server, MacOSXTM, UnixTM, LinuxTM, FreeBSDTM, and so forth. The steps performed by the above-described method embodiment may be based on the server structure shown in fig. 11.
The embodiment of the invention also provides a terminal, which comprises an image quality evaluation terminal. The terminal may be a mobile terminal or the like. Optionally, in this embodiment, the terminal may also be a computer terminal, and may also be replaced by any one computer terminal device in a computer terminal group.
Optionally, in this embodiment, the computer terminal or the mobile terminal may be located in at least one of a plurality of network devices of a computer network.
Alternatively, fig. 12 is a block diagram of a terminal according to an embodiment of the present invention. As shown in fig. 12, the terminal may include: one or more processors (only one of which is shown), memory, and transmission means.
The memory may be used for storing software programs and modules, and the processor may execute various functional applications and data processing by operating the software programs and modules stored in the memory. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory located remotely from the processor, which may be connected to a computer terminal or a mobile terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The above-mentioned transmission device is used for receiving or transmitting data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device includes a network adapter that is connectable to the router via a network cable to communicate with the internet or a local area network. In one example, the transmission device is a radio frequency module, which is used for communicating with the internet in a wireless manner.
Wherein the memory stores, in particular, a program for performing image quality evaluation.
The processor can call the information and application program stored in the memory through the transmission device to execute the following steps:
optionally, the processor may further execute the program code of the following steps:
acquiring an original image, wherein the original image meets the basic condition of generating a standard image;
generating a suspected standard image according to the original image;
obtaining a reference image set corresponding to the original image;
taking the suspected standard image as an evaluation standard, obtaining the quality score of each reference image in the reference image set, and obtaining the corresponding relation between the reference image and the quality score;
constructing a training set according to the reference images and the quality scores, and training a quality evaluation model, wherein the quality evaluation model takes the images as input and takes the quality scores corresponding to the images as output;
acquiring an image to be evaluated;
and inputting the image to be evaluated into the quality evaluation model, and obtaining the quality score of the image to be evaluated.
The integrated unit in the above embodiments may be stored in a readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more mobile terminals or computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed terminal can be implemented in other manners. The above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
It should be noted that: the sequence of the above embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (12)
1. An image quality evaluation method is characterized by comprising:
acquiring an original image, wherein the original image meets the basic condition of generating a standard image; the base condition is related to the actual image acquisition scenario and is independent of the performance of the acquisition device;
generating a suspected standard image according to the original image;
obtaining a reference image set corresponding to the original image;
taking the suspected standard image as an evaluation standard, obtaining the quality score of each reference image in the reference image set, and obtaining the corresponding relation between the reference image and the quality score;
constructing a training set according to the reference images and the quality scores, training a quality evaluation model, wherein the quality evaluation model is used for taking the images to be evaluated as input, taking the quality scores corresponding to the images to be evaluated as output, and evaluating the quality of the images to be evaluated under the condition of no standard images by fitting the scores of the images based on different deflection angles in the training set;
acquiring an image to be evaluated;
and inputting the image to be evaluated into the quality evaluation model, and obtaining the quality score of the image to be evaluated.
2. The method of claim 1, wherein the suspected standard image generation method comprises:
constructing a countermeasure generation network, wherein the countermeasure generation network comprises a generator network and a discriminator network, and a residual error network is embedded in the generator network;
acquiring a generator training set, wherein the generator training set comprises a low-resolution original image and a high-resolution standard image corresponding to the low-resolution original image;
training the confrontation generating network according to the generator training set, and obtaining a generator network capable of meeting preset requirements;
and inputting the original image into the generator network to obtain a suspected standard image.
3. The method of claim 1, wherein the reference image quality score is obtained by a method comprising:
acquiring a first feature map corresponding to the suspected standard image;
acquiring a second characteristic diagram corresponding to the reference image;
and evaluating the similarity of the first feature map and the second feature map, and outputting a quality score of the reference image according to the similarity.
4. The method of claim 3, wherein:
the first feature map and the second feature map are obtained using a VGG19 network.
5. The method of claim 4, wherein:
the obtaining of the first feature map corresponding to the suspected standard image includes:
inputting the suspected standard image into a VGG19 network, obtaining a feature map of relu5-4 as a first feature map;
the second feature map corresponding to the reference image comprises:
and inputting the reference image into a VGG19 network, obtaining a relu5-4 feature map corresponding to the reference image, and taking the feature map as a second feature map.
6. The method of claim 3, wherein the evaluating the similarity of the first feature map and the second feature map and outputting a quality score of the reference image according to the similarity comprises:
calculating Euclidean distances of the first feature map and the second feature map;
and mapping the Euclidean distance to the (0, 1) interval to obtain the quality score.
7. An image quality evaluation apparatus, comprising:
the original image acquisition module is used for acquiring an original image, and the original image meets the basic condition of generating a standard image; the base condition is related to the actual image acquisition scenario and is independent of the performance of the acquisition device;
the suspected standard image generating module is used for generating a suspected standard image according to the original image;
a reference image set obtaining module, configured to obtain a reference image set corresponding to the original image;
the reference image quality score acquisition module is used for acquiring the quality scores of all reference images in the reference image set by taking the suspected standard images as evaluation standards to obtain the corresponding relation between the reference images and the quality scores;
the training module is used for constructing a training set according to the reference images and the quality scores, training a quality evaluation model, wherein the quality evaluation model is used for taking the images to be evaluated as input, taking the quality scores corresponding to the images to be evaluated as output, and evaluating the quality of the images to be evaluated under the condition of no standard images by fitting the scores of the images based on different deflection angles in the training set;
the image to be evaluated acquiring module is used for acquiring an image to be evaluated;
and the quality score output module is used for inputting the image to be evaluated into the quality evaluation model and obtaining the quality score of the image to be evaluated.
8. The apparatus of claim 7, wherein the reference image quality score obtaining module comprises:
the first feature map acquisition unit is used for acquiring a first feature map corresponding to the suspected standard image;
the second characteristic image acquisition unit is used for acquiring a second characteristic image corresponding to the reference image;
and the reference image quality score acquisition unit is used for evaluating the similarity of the first characteristic diagram and the second characteristic diagram and outputting the quality score of the reference image according to the similarity.
9. The apparatus according to claim 8, wherein the reference image quality score obtaining unit includes:
the Euclidean distance calculating subunit is used for calculating Euclidean distances of the first feature map and the second feature map;
and the mapping subunit is used for mapping the Euclidean distance into the (0, 1) interval to obtain the quality score.
10. A computer-readable storage medium storing a program for implementing the image quality evaluation method according to claim 1.
11. A server for operating an image quality evaluation apparatus according to claim 7.
12. A terminal characterized in that it is adapted to run an image quality evaluation apparatus according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810669989.3A CN109118470B (en) | 2018-06-26 | 2018-06-26 | Image quality evaluation method and device, terminal and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810669989.3A CN109118470B (en) | 2018-06-26 | 2018-06-26 | Image quality evaluation method and device, terminal and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109118470A CN109118470A (en) | 2019-01-01 |
| CN109118470B true CN109118470B (en) | 2020-12-15 |
Family
ID=64822481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810669989.3A Active CN109118470B (en) | 2018-06-26 | 2018-06-26 | Image quality evaluation method and device, terminal and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109118470B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919830B (en) * | 2019-01-23 | 2023-02-10 | 复旦大学 | Method for restoring image with reference eye based on aesthetic evaluation |
| CN110335237B (en) * | 2019-05-06 | 2022-08-09 | 北京字节跳动网络技术有限公司 | Method and device for generating model and method and device for recognizing image |
| CN112084825B (en) * | 2019-06-14 | 2023-03-24 | 佛山市顺德区美的电热电器制造有限公司 | Cooking evaluation method, cooking recommendation method, computer device and storage medium |
| CN110737795A (en) * | 2019-10-16 | 2020-01-31 | 北京字节跳动网络技术有限公司 | Photo album cover determining method, device, equipment and storage medium |
| CN110782445A (en) * | 2019-10-25 | 2020-02-11 | 北京华捷艾米科技有限公司 | No-reference image quality evaluation method and system |
| CN112861589A (en) * | 2019-11-28 | 2021-05-28 | 马上消费金融股份有限公司 | Portrait extraction, quality evaluation, identity verification and model training method and device |
| CN111179257A (en) * | 2019-12-31 | 2020-05-19 | 上海联影医疗科技有限公司 | Evaluation method and device, electronic equipment and storage medium |
| CN111260623B (en) * | 2020-01-14 | 2023-07-25 | 广东小天才科技有限公司 | Picture evaluation method, device, equipment and storage medium |
| CN111784693A (en) * | 2020-08-12 | 2020-10-16 | 成都佳华物链云科技有限公司 | Image quality evaluation method and device, electronic equipment and storage medium |
| CN112001200A (en) * | 2020-09-01 | 2020-11-27 | 杭州海康威视数字技术股份有限公司 | Identification code identification method, device, equipment, storage medium and system |
| CN112561879B (en) * | 2020-12-15 | 2024-01-09 | 北京百度网讯科技有限公司 | Ambiguity evaluation model training method, image ambiguity evaluation method and image ambiguity evaluation device |
| CN112446879B (en) * | 2021-01-06 | 2022-09-23 | 天津科技大学 | Contrast distortion image quality evaluation method based on image entropy |
| CN114926468B (en) * | 2022-07-22 | 2022-12-06 | 深圳华声医疗技术股份有限公司 | Ultrasonic image quality control method, ultrasonic device, and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123705A (en) * | 2014-07-22 | 2014-10-29 | 北华大学 | Super-resolution reconstructed image quality Contourlet domain evaluation method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102509269B (en) * | 2011-11-10 | 2014-04-02 | 重庆工业职业技术学院 | Image denoising method combined with curvelet and based on image sub-block similarity |
| CN105184819B (en) * | 2015-09-14 | 2018-01-12 | 浙江大学 | Objective image quality evaluation method for medical image reconstruction parameter optimizing |
| US10402967B2 (en) * | 2015-12-21 | 2019-09-03 | Koninklijke Philips N.V. | Device, system and method for quality assessment of medical images |
| CN105869161B (en) * | 2016-03-28 | 2018-11-20 | 西安电子科技大学 | Hyperspectral image band selection method based on image quality evaluation |
| CN106570504A (en) * | 2016-10-12 | 2017-04-19 | 成都西纬科技有限公司 | Image quality evaluation system and method |
| CN106683048B (en) * | 2016-11-30 | 2020-09-01 | 浙江宇视科技有限公司 | Image super-resolution method and device |
| CN108090902B (en) * | 2017-12-30 | 2021-12-31 | 中国传媒大学 | Non-reference image quality objective evaluation method based on multi-scale generation countermeasure network |
-
2018
- 2018-06-26 CN CN201810669989.3A patent/CN109118470B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123705A (en) * | 2014-07-22 | 2014-10-29 | 北华大学 | Super-resolution reconstructed image quality Contourlet domain evaluation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109118470A (en) | 2019-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109118470B (en) | Image quality evaluation method and device, terminal and server | |
| Liu et al. | PQA-Net: Deep no reference point cloud quality assessment via multi-view projection | |
| US11830230B2 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| Gu et al. | Multiscale natural scene statistical analysis for no-reference quality evaluation of DIBR-synthesized views | |
| Su et al. | Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation | |
| Panetta et al. | Human-visual-system-inspired underwater image quality measures | |
| Liu et al. | Image retargeting quality assessment | |
| Wang et al. | Reduced-reference image quality assessment using a wavelet-domain natural image statistic model | |
| Zheng et al. | No-reference quality assessment for screen content images based on hybrid region features fusion | |
| Wang et al. | Novel spatio-temporal structural information based video quality metric | |
| CN101562675B (en) | No-reference image quality evaluation method based on Contourlet transform | |
| Zheng et al. | UIF: An objective quality assessment for underwater image enhancement | |
| US20140126808A1 (en) | Recursive conditional means image denoising | |
| CN112365418B (en) | Image distortion evaluation method and device and computer equipment | |
| Gupta et al. | A novel full reference image quality index for color images | |
| Li et al. | Boosting paired comparison methodology in measuring visual discomfort of 3DTV: performances of three different designs | |
| Xu et al. | Quality assessment of stereoscopic 360-degree images from multi-viewports | |
| Yang et al. | Full reference image quality assessment by considering intra-block structure and inter-block texture | |
| Lu et al. | Point cloud quality assessment via 3D edge similarity measurement | |
| CN115131229A (en) | Image noise reduction and filtering data processing method and device and computer equipment | |
| CN113569713A (en) | Stripe detection method and device for video image and computer readable storage medium | |
| CN107578406A (en) | Based on grid with Wei pool statistical property without with reference to stereo image quality evaluation method | |
| Yang et al. | Subjective quality evaluation of compressed digital compound images | |
| Fan et al. | Stereoscopic image quality assessment based on the binocular properties of the human visual system | |
| Farah et al. | Full-reference and reduced-reference quality metrics based on SIFT |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |