CN116934618A - Image halftone method, system and medium based on improved residual error network - Google Patents

Image halftone method, system and medium based on improved residual error network Download PDF

Info

Publication number
CN116934618A
CN116934618A CN202310857834.3A CN202310857834A CN116934618A CN 116934618 A CN116934618 A CN 116934618A CN 202310857834 A CN202310857834 A CN 202310857834A CN 116934618 A CN116934618 A CN 116934618A
Authority
CN
China
Prior art keywords
image
network
halfresnet
halftone
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310857834.3A
Other languages
Chinese (zh)
Other versions
CN116934618B (en
Inventor
刘登峰
陈世海
郭文静
郭虓赫
付玄辉
柴志雷
吴秦
陈璟
王宁
周浩杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangnan University
Original Assignee
Jiangnan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangnan University filed Critical Jiangnan University
Priority to CN202310857834.3A priority Critical patent/CN116934618B/en
Publication of CN116934618A publication Critical patent/CN116934618A/en
Application granted granted Critical
Publication of CN116934618B publication Critical patent/CN116934618B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Processing (AREA)

Abstract

The application discloses an image halftone method, system and medium based on an improved residual error network, which comprises the following steps: step S1: acquiring an image data set, and dividing the image data set into a training set and a testing set; step S2: inputting the training set into a halfResNet network model for training, wherein the halfResNet network model is improved based on the ResNet network model; step S3: and after training is finished, obtaining weight parameters, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image outputs a high-quality halftone image through screening processing. The application combines the deep learning model with the halftone image, and the generated halftone image can maintain better structural details and greatly improve the processing efficiency.

Description

Image halftone method, system and medium based on improved residual error network
Technical Field
The application relates to the technical field of image halftoning, in particular to an image halftoning method, an image halftoning system and an image halftoning medium based on an improved residual error network.
Background
Digital halftoning, also known as spatial dithering, is a method of presenting successive image illusions on a device that can only produce binary image elements, for converting a successive image into an image with limited tonal values, while preserving image quality as much as possible. Due to the low-pass filtering characteristics of the human visual system, halftone images are considered to be continuous images when observed over a sufficient distance. Halftone images are commonly used in the printing industry, mainly in print engines such as laser and inkjet printers. According to the different modes of dot clustering, the method can be divided into clustering dots and discrete dots, and in addition, the method can be divided into periodic textures and aperiodic textures according to halftone texture classification. The main feature of a halftone pattern is to fit the texture features of a continuous image based on the distribution of black and white dots, which may correspond to many different modes of halftone images for a given input continuous image, where learning and generating the best visual effect halftone images is very challenging. The best-looking halftone image belongs to a blue noise distribution, typically a dispersed pattern of uniformly distributed black and white dots, due to the low-pass nature of human perception. The halftone mode conforming to the blue noise characteristics is a key to generate a high quality halftone image.
In halftone research, the quality and processing efficiency of a generated image are the most important problems. In the prior art, the method based on ordered dithering has higher parallelism and higher processing speed, but the quality of the generated halftone image is poorer; the error diffusion-based method keeps better balance between the image quality and the processing effect, the sequential processing of the algorithm is difficult to perform parallel calculation, the generated halftone image can introduce related artifacts, but the image quality generated by comparing with the ordered dithering is better; search-based methods can produce the best quality halftone image among these three types of algorithms, but their processing speed is very slow; halftone images produced by Convolutional Neural Network (CNN) based methods have good recoverability but do not achieve strong blue noise characteristics; the halftone image generated by the depth generation type network method does not contain artifacts related to the error diffusion calculation method, but takes a long time and cannot be processed in real time.
Disclosure of Invention
The application aims to solve the problem that the processing speed and the image quality of a halftone image in the prior art cannot be simultaneously considered, and provides an image halftone method, an image halftone system and an image halftone medium based on an improved residual error network, which aim to combine a deep learning model with the halftone image, and the generated halftone image can keep better structural details and greatly improve the processing efficiency.
In order to solve the above technical problems, an embodiment of the present application provides an image halftone method based on an improved residual network, the method including the steps of:
step S1: acquiring an image data set, and dividing the image data set into a training set and a testing set;
step S2: inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on the ResNet network model, modifying an original residual block to form light-weight residual blocks, adding a noise compensation block before each light-weight residual block, and adding blue noise loss in a loss function to form;
step S3: and after training is finished, obtaining weight parameters, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image outputs a high-quality halftone image through screening processing.
In one embodiment of the present application, the original residual block uses the feature mapping of the output of the previous layer network and the mapping of the convolution layer, the batch normalization layer, the ReLU activation layer, the convolution layer and the batch normalization layer sequentially, and obtains the input of the next layer network through the additive change and the ReLU activation layer.
In one embodiment of the present application, the lightweight residual block uses the feature mapping of the output of the upper layer network and the mapping sequentially passing through the convolution layer, the ReLU activation layer, and the convolution layer, and obtains the input of the lower layer network through additive variation.
In one embodiment of the present application, the noise compensation block uses two cooperative convolutions as the feature map and gaussian noise map of the output of the upper network, respectively, and then obtains the input of the lower network through additive variation.
In one embodiment of the application, the blue noise loss is used only to train a batch of dithered constant gray images after each training iteration.
In one embodiment of the application, the loss function further comprises: hue consistency loss, binary loss, and perceived consistency loss.
In one embodiment of the present application, the halfrenet network model includes a shallow feature extraction module, a deep feature extraction module and a high quality image halftone module, wherein the shallow feature extraction module adopts a convolution layer to extract shallow features, the deep feature extraction module adopts n lightweight residual blocks and n noise compensation blocks to further extract deep features from the shallow features, and the high quality image halftone module aggregates the shallow features and deep features to reconstruct a halftone image.
In one embodiment of the present application, the training mode in step S2 is a self-supervision mode.
The present application also provides an image halftoning system based on an improved residual network, the system comprising:
the data acquisition module is used for acquiring an image data set and dividing the image data set into a training set and a testing set;
the light model building module is used for inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on a ResNet network model, an original residual block of the HalfResNet network model is modified to form a light residual block, a noise compensation block is added before each light residual block, and blue noise loss is added in a loss function to form;
the halftone image generation module is used for obtaining weight parameters after training is completed, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image outputs a high-quality halftone image through screening processing. The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of an improved residual network based image halftoning method described above.
From the above technical scheme, the application has the following advantages:
1. the original residual block of the ResNet network model is improved, and a lightweight residual block is generated, so that the processing effect and the processing speed of the newly constructed HalfResNet network model are greatly improved.
2. The noise compensation module is added to enable the HalfResNet network model to learn a better jitter mode, so that the problem of flatness degradation of the original CNN is solved, and the performance of the CNN is further improved.
3. An explicit blue noise loss is added in the loss function, the blue noise characteristic of the model is enhanced, the injection of additional modes is avoided, and the quality of a halftone image generated after the blue noise loss is added is better.
Drawings
FIG. 1 is a flow chart of an implementation of an image halftoning method based on an improved residual network;
FIGS. 2 (a) -2 (c) are residual block structure comparisons of a conventional ResNet model, an SRResNet model, and a HalfResNet model, respectively;
FIG. 3 is a block diagram of noise compensation;
FIG. 4 is a block diagram of the HalResNet model;
FIGS. 5 (a) -5 (f) are halftone image visualizations generated by a continuous tone image and a Bayer ordered dithering algorithm, an OVED error diffusion algorithm, a DBS algorithm, an RVH algorithm, and a HalResNet algorithm, respectively;
FIGS. 6 (a) -6 (c) are halftone spectral analysis diagrams of the OVED error diffusion algorithm, RVH algorithm and HalResNet algorithm, respectively, under constant gray scale;
fig. 7 (a) to 7 (d) are respectively a continuous gray-scale image, a halftone image whose model contains only a lightweight residual block output, a halftone image obtained by introducing a noise compensation block output on the basis of the lightweight residual block, and a halftone image obtained by adding a blue noise loss output to a loss function.
Detailed Description
The present application will be further described with reference to the accompanying drawings and specific examples, which are not intended to be limiting, so that those skilled in the art will better understand the application and practice it.
Example 1
Referring to fig. 1, the specific operation steps of the image halftone method based on the improved residual error network provided by the present application are as follows:
step S1: acquiring an image data set, and dividing the image data set into a training set and a testing set;
step S2: inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on the ResNet network model, modifying an original residual block to form light-weight residual blocks, adding a noise compensation block before each light-weight residual block, and adding blue noise loss in a loss function to form;
step S3: and after training is finished, obtaining weight parameters, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image outputs a high-quality halftone image through screening processing.
The application is based on the dithering mode of blue noise, and utilizes the ordered dithering process to construct a halftone image, and provides an image halftone model based on a lightweight residual convolution neural network, which has fewer parameters and less calculated amount, and a multilayer noise compensation block is introduced to ensure that the model keeps a better dithering mode, so that the phenomenon of flatness degradation caused by CNN is avoided; in order to further enhance the quality of the model generated halftone image, an explicit blue noise penalty is designed in the penalty function, which facilitates the model to generate a halftone pattern that meets the blue noise characteristics by generating a high quality halftone image.
Referring to fig. 2, an original residual block of a conventional res net model adopts a feature mapping of an output of a previous layer network and a mapping sequentially passing through a convolutional layer (Conv), a batch normalization layer (BN), a ReLU activation layer, a convolutional layer, and a batch normalization layer, and obtains an input of a next layer network through an additive change (Addition) and a ReLU activation layer; the residual block of the SRResNet model is that the last ReLU activation layer is removed on the basis of the traditional ResNet model; the residual block of the halfResNet model, namely the lightweight residual block mentioned in the application, is obtained by removing two layers of batch normalization layers and a final ReLU activation layer on the basis of the traditional ResNet model.
Binding experiments prove that: the lightweight residual block can greatly improve the performance of the model through modification. In addition, as the batch normalization is equivalent to the memory consumed by the convolution layer, the memory consumption of the GPU is sufficiently reduced, and therefore, a model with better performance than the traditional ResNet model can be established under the environment of limited computing resources.
Referring to fig. 3, the noise compensation block uses two cooperative convolutions as the feature map of the output of the upper layer network and the gaussian noise map, and obtains the input y of the lower layer network through additive variation:
y＝f 1 (I o )+f 2 (N) (1)
wherein f 1 And f 2 Are all single-layer convolutional networks, I o Representing the output characteristics of the upper network, N represents a dynamically sampled gaussian noise map.
A typical CNN consists of a convolution layer with a bias term and an activation function, which cannot introduce spatial variation of the output when the input is a planar input, a phenomenon known as flatness degradation, which is caused by the convolution paradigm of the spatial sharing kernel. Given a planar input X, the operation of CNN is degenerated to scaling operation y=αx (α is the scaling factor), and degradation of flatness can prevent CNN from dithering a constant gray scale. In order to avoid the problem of flatness degradation of CNN, the application is provided with a noise compensation block before each light residual block, and can generate good halftone jitter in a flat area without adding extra parameter quantity to a model by the noise compensation block.
The loss function includes: hue consistency loss, binary loss, blue noise loss, and perceived consistency loss, expressed as follows:
wherein,,for tone consistency loss, ++>For binary loss, ++>Blue noise loss, < >>To perceive a consistency loss, the super parameter ω 1 ＝0.6，ω 2 ＝0.3，ω 3 ＝0.1，ω 4 =1 is empirically set.
Tone consistency loss: to ensure that the generated halftone image remains visually similar to the input, the Gaussian filtered gray scale input I is measured separately c And halftone image O h The expression of the MSE between the two is as follows:
wherein G (·) is a Gaussian filter with a convolution kernel size of 11x11, E Ic E I {. Cndot. } represents all input images I in the training set c Is a mean operator of (c).
Binary loss: the half tone map generated contains only 0 or 1 values, in order to ensure the stability of the model, we excite the output value of the model to be as close to 0 or 1 as possible, so the output binarization loss of the constraint model by adopting the binarization loss is expressed as:
wherein the method comprises the steps ofIs a pseudo halftone image before binary gate, C d Is in combination with->And a constant matrix with the same size, wherein the values of all the elements are d= {0,1}.
Blue noise loss: conventional halftone algorithms typically require blue noise characteristics to avoid injecting additional modes. The loss function of the application combines blue noise characteristics, penalizes low frequency components on dithering constant gray scale, and optimizes anisotropy of a high frequency region.
Wherein the method comprises the steps ofFor low frequency loss, σ is the weight coefficient, preferably set to 0.2, DCT (. Cndot.) represents the discrete cosine transform,. C.f. represents the element product, M is a constant binary mask, where the low frequency component is set to 1 and the other components are set to 0. The input I of the halftone process, a continuous tone map with gray values between 0 and 1, is output as a binary discrete image. For a fixed gray scale image, the aperiodic halftone algorithm can be modeled as a Bernoulli distribution and thus a smooth stochastic process. Whereas the fourier transform of the autocorrelation function of the stationary random process is its power spectrum P (f). The autocorrelation function of the non-periodic half-tone process is unknown and the power spectrum can be obtained by means of spectral estimation:
where f represents the sampling frequency, N represents the number of sampling frequencies, and DFT (. Cndot.) represents the discrete Fourier transform.
Radial Average Power Spectral Density (RAPSD) is defined as:
wherein n (r (f) ρ ) Is represented at f ρ The nearby bandwidth Δρ=1 the number of all sampling points in the annulus,representing samples in a frequency bandObservation of a dot, while anisotropy is defined as:
the anisotropic loss function can thus be defined as:
i.e. optimizing the molecules in formula (9). Because the spectral analysis is only meaningful for a dithered constant gray image, training a batch of constant gray images after each training iteration uses blue noise loss, while training other training sets without using the loss, where the constant gray image is a pseudo-halftone image generated by the halfrenet model.
Perceptual consistency loss: to capture the perceived consistency of halftone images, measuring the difference in halftone patterns within a continuous feature domain is employed. Briefly, a pre-trained inverse halftone model is used to recover the halftone image into a continuous tone image, and the pixel-level mean square error between the halftone image and the input continuous tone image is calculated, where the perceived consistency loss is defined as:
wherein F (-) represents the inverse half-tone mode.
As shown in fig. 4, the halfrenetnetwork model includes a shallow feature extraction module, a deep feature extraction module, and a high quality image halftone module, each of which is described in detail below:
shallow layer feature extraction module: given a continuous color Image (IC) input(H, W, cin is image height, width and number of input channels, respectively) we use 3X 3 convolution layer H SF (. About.) extraction of shallow features>
F 0 ＝H SF (I c ) (12)
Wherein C is the number of characteristic channels, the convolution layer is good at early visual processing, the optimization is more stable, and the result is better.
Deep feature extraction module: from F 0 Deep features of Chinese medicine
F DF ＝H DF (F 0 ) (13)
Wherein H is DF (·) is a deep feature extraction module comprising n lightweight residual blocks and n noise compensation blocks, preferably consisting of 16 lightweight residual blocks of channel number 64 and 16 noise compensation blocks.
High quality image halftoning module: reconstructing a high quality halftone image by polymerizing shallow features and deep features:
I RH ＝H RH (F 0 +F DF ) (14)
wherein H is RH (. Cndot.) is a halftone reconstruction module. The shallow features mainly contain low-frequency information, and the deep features contain more high-frequency information. The HalfResNet model directly transmits low-frequency information to the reconstruction module through long jump connection, so that the deep feature extraction module can be helped to concentrate on extracting more high-frequency information and stabilize training. For the halftone reconstruction module, the features were reconstructed using a 3 x 3 convolution. In addition, the pseudo halftone image generated by the HalfResNet model is continuously participated in the training link of the model, and the high-quality halftone image is screened out in the testing link, particularly, the screening is carried out through a binary gate structure, the binary gate strictly quantizes the output of the model to a discrete value of 0 or 1, and the correct halftone image can be output, because the binary gate operation is not tiny, the step only occurs in the testing stage and the training stageThe binary gate is removed.
The superiority of halfResNet model generation halftone image in image quality and processing efficiency proposed by the present application is verified by experiments as follows.
The dataset used for the experiment was from the published VOC2012 dataset, which had 17125 color images, of which 13758 images were randomly selected for training, and 3367 images were retained as the test dataset for quantitative evaluation. By cropping and resizing, the size of all these images is 256×256. The GPU used in the experiment was NVIDIA GeForce RTX 2080Ti, the deep learning framework was PyTorch 1.10.1, the acceleration library was CUDA 11.3 and cuDNN 8.2.0, the network was trained by minimizing the loss function, the ADAM optimizer was used for training, the batch size was set to 16, the learning rate was adjusted from 3e-4 to 1e-5, the cosine annealing program was used for a total of 30 runs, and the training was self-supervising, so that more unlabeled continuous tone images could be easily collected.
Experiment 1 compares the halfResNet algorithm with a different class of typical algorithms: the Bayer order dithering algorithm, the ostomoukhov error diffusion algorithm, the DBS algorithm, and the RVH model are compared. The difference between the generated halftone image and the continuous gray scale image is compared using two indicators of peak signal to noise ratio (PSNR) and Structural Similarity (SSIM).
Tone consistency is measured by PSNR between hvs-filtered halftone and continuous tone, and SSIM is used to measure structural similarity between halftone and continuous tone. Test results as shown in table 1, were tested on test data sets, on "lenna" test pictures of size 512 x 512, "x" indicates that the method was accelerated on the GPU. The halfResNet algorithm achieves a competitive PSNR score and the best SSIM score, and the reference quantity is reduced by 98.38% compared with the RVH algorithm, so that the processing speed is improved.
Table 1 quantitative evaluation of semi-tuning algorithm
Figure 5 shows the halftone effect of various algorithms on a Snail Shaped Organ picture, the traditional algorithm has poor edge detail processing effect, and the appearance of artifacts is restrained at the cost of excessively blurring fine texture details, while the RVH algorithm and the HalfResNet algorithm of the application can well keep the edge texture details of an image, and the halftone visual effect is better, and compared with the RVH algorithm, the HalfResNet algorithm contains less noise in the edge detail part.
To further illustrate the generalization ability of the halftonet algorithm (halftanet algorithm) proposed by the present application, halftone tests were performed on a plurality of public data sets, the quantitative results of the various methods on the test data sets are shown in table 2, and SSIM index and PSNR index of each algorithm on the data sets are recorded, and black bolded indicates the index value achieving the best performance. As can be seen from the experimental data in Table 2, the HalfResNet algorithm can obtain the best SSIM index and better PSNR index on each public data set, and has better generalization capability.
Table 2 discloses the results of the test on the dataset (SSIM/PSNR)
Referring to fig. 6, given a constant gray scale of 15/255, fig. 6 (a) -6 (c) are halftone results, fourier amplitude spectra, and radial average power spectral densities in that order in the horizontal direction. Specifically, the fourier amplitude spectrum represents the amplitude of the frequency components, with the low frequency components expected to have lower amplitudes in the blue noise profile. The radial average power spectrum shows one-dimensional blue noise characteristics. The results of these 3 classes of algorithms all have different degrees of blue noise characteristics, contain fewer low-frequency components, and from the radial average power spectral density, the halres net algorithm is more consistent with the blue noise characteristics, so that the halftone visual effect is better.
To verify the effectiveness of the different components of the HalfResNet algorithm (halfrennet) proposed by the present application, several experiments were performed on the same dataset to show performance. The effectiveness of each method is analyzed through the integral ablation experiment, and a noise compensation block and blue noise loss are gradually added into the ResNet network. The global ablation experiment is shown in table 3 ("v" indicates that the component is referenced), and fig. 7 shows halftone images generated by different models. According to the integral ablation experimental result, a noise compensation block is not added, the generated half tone map generates a degradation phenomenon in a flat area, the noise compensation block overcomes the defect of a native CNN, the randomness of noise is beneficial to a dithering mode, and the integral distribution of the mode concerned by the noise compensation block is not the value of a single pixel; the addition of anisotropic blue noise loss contributes to further improvement of model performance, and the halftone image generated in the edge area has better visual effect.
TABLE 3 influence of each component
The embodiment provides a high-efficiency halftone method based on a lightweight residual neural network, which uses a noise compensation block to solve the problem of flatness degradation of CNN, and can generate good dithering effect in a flat area; the blue noise characteristics of an explicit blue noise loss enhancement model are employed. And compared with RVH model, the parameter of HalfResNet model is reduced by 98.26%, and the processing rate is increased by 57.28%. A large number of experiments prove that the HalfResNet model obtains the best SSIM index and the competitive PSNR index on a plurality of data sets, and can retain more texture details.
Example two
The application also discloses an image halftone system based on the improved residual error network, which comprises:
the data acquisition module is used for acquiring an image data set and dividing the image data set into a training set and a testing set;
the light model building module is used for inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on a ResNet network model, an original residual block of the HalfResNet network model is modified to form a light residual block, a noise compensation block is added before each light residual block, and blue noise loss is added in a loss function to form;
the halftone image generation module is used for obtaining weight parameters after training, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image is subjected to screening processing to output a high-quality halftone image.
Example III
The application also discloses a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the image halftoning method based on the improved residual network described in embodiment one.
The application effectively combines the deep learning model with the halftone image, provides a high-efficiency and high-practicability halftone model, and selects residual connection because the model has a simple structure, contains fewer parameter amounts, has higher processing speed and can be deployed in real-time equipment; in order to solve the problem of flatness degradation of the original CNN, a noise compensation block is provided, so that jitter dependence can be provided for a model, and the performance of the model is improved; in order to further improve the model performance, blue noise loss is increased in a loss function, the low-frequency component of the halftone constant gray level image is restrained and the anisotropy of the high-frequency region of the halftone constant gray level image is optimized, and the halftone image generated by the method is more in line with blue noise characteristics and has satisfactory visual quality.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations and modifications of the present application will be apparent to those of ordinary skill in the art in light of the foregoing description. It is not necessary here nor is it exhaustive of all embodiments. And obvious variations or modifications thereof are contemplated as falling within the scope of the present application.

Claims (10)

1. An image halftoning method based on an improved residual network, comprising the steps of:
step S1: acquiring an image data set, and dividing the image data set into a training set and a testing set;
step S2: inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on the ResNet network model, modifying an original residual block to form light-weight residual blocks, adding a noise compensation block before each light-weight residual block, and adding blue noise loss in a loss function to form;
step S3: and after training is finished, obtaining weight parameters, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image outputs a high-quality halftone image through screening processing.
2. The improved residual network based image halftoning method of claim 1, wherein: the original residual block adopts the characteristic mapping of the output of the upper network and the mapping of the convolution layer, the batch normalization layer, the ReLU activation layer, the convolution layer and the batch normalization layer in sequence, and obtains the input of the lower network through the additive change and the ReLU activation layer.
3. The improved residual network-based image halftoning method of claim 1 or 2, wherein: the lightweight residual block adopts the characteristic mapping of the output of the upper layer network and the mapping of the convolution layer, the ReLU activation layer and the convolution layer in sequence, and obtains the input of the lower layer network through additive change.
4. The improved residual network based image halftoning method of claim 1, wherein: the noise compensation block adopts two cooperative convolutions to respectively serve as the characteristic mapping of the output of the upper layer network and the mapping of Gaussian noise, and then the input of the lower layer network is obtained through additive change.
5. The improved residual network based image halftoning method of claim 1, wherein: the blue noise loss is only used for training a batch of gray scale images with constant jitter after each training iteration.
6. The improved residual network based image halftoning method of claim 1, wherein: the loss function further includes: hue consistency loss, binary loss, and perceived consistency loss.
7. The improved residual network based image halftoning method of claim 1, wherein: the HalfResNet network model comprises a shallow feature extraction module, a deep feature extraction module and a high-quality image halftone module, wherein the shallow feature extraction module adopts a convolution layer to extract shallow features, the deep feature extraction module adopts n lightweight residual blocks and n noise compensation blocks to further extract deep features from the shallow features, and the high-quality image halftone module aggregates the shallow features and the deep features to reconstruct a halftone image.
8. The improved residual network based image halftoning method of claim 1, wherein: the training mode in step S2 is a self-monitoring mode.
9. An image halftoning system based on an improved residual network, comprising:
the data acquisition module is used for acquiring an image data set and dividing the image data set into a training set and a testing set;
the light model building module is used for inputting the training set into a HalfResNet network model for training, wherein the HalfResNet network model is based on a ResNet network model, an original residual block of the HalfResNet network model is modified to form a light residual block, a noise compensation block is added before each light residual block, and blue noise loss is added in a loss function to form;
the halftone image generation module is used for obtaining weight parameters after training, loading the weight parameters into the HalfResNet network model, and aiming at an image to be processed, operating the HalfResNet network model to obtain a pseudo halftone image corresponding to the image to be processed, wherein the pseudo halftone image is subjected to screening processing to output a high-quality halftone image.
10. A computer-readable storage medium, characterized by: the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of an improved residual network based image halftoning method according to any one of claims 1 to 8.
CN202310857834.3A 2023-07-13 2023-07-13 Image halftone method, system and medium based on improved residual error network Active CN116934618B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310857834.3A CN116934618B (en) 2023-07-13 2023-07-13 Image halftone method, system and medium based on improved residual error network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310857834.3A CN116934618B (en) 2023-07-13 2023-07-13 Image halftone method, system and medium based on improved residual error network

Publications (2)

Publication Number Publication Date
CN116934618A true CN116934618A (en) 2023-10-24
CN116934618B CN116934618B (en) 2024-06-11

Family

ID=88388861

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310857834.3A Active CN116934618B (en) 2023-07-13 2023-07-13 Image halftone method, system and medium based on improved residual error network

Country Status (1)

Country Link
CN (1) CN116934618B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002158879A (en) * 2000-09-13 2002-05-31 Sharp Corp Method for compressing/uncompressing image with gradation and compressing/uncompressing device
CN106709875A (en) * 2016-12-30 2017-05-24 北京工业大学 Compressed low-resolution image restoration method based on combined deep network
CN108921886A (en) * 2018-06-11 2018-11-30 昆明理工大学 A kind of texture information fusion Multi-scale model forest digital picture halftoning method
CN111754403A (en) * 2020-06-15 2020-10-09 南京邮电大学 Image super-resolution reconstruction method based on residual learning
CN111986275A (en) * 2020-07-31 2020-11-24 西安理工大学 Inverse halftoning method for multi-modal halftone image
CN112070668A (en) * 2020-08-18 2020-12-11 西安理工大学 Image super-resolution method based on deep learning and edge enhancement
CN114841872A (en) * 2022-04-12 2022-08-02 浙江大学 Digital halftone processing method based on multi-agent deep reinforcement learning
WO2022242029A1 (en) * 2021-05-18 2022-11-24 广东奥普特科技股份有限公司 Generation method, system and apparatus capable of visual resolution enhancement, and storage medium
WO2022257408A1 (en) * 2021-06-10 2022-12-15 南京邮电大学 Medical image segmentation method based on u-shaped network

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002158879A (en) * 2000-09-13 2002-05-31 Sharp Corp Method for compressing/uncompressing image with gradation and compressing/uncompressing device
CN106709875A (en) * 2016-12-30 2017-05-24 北京工业大学 Compressed low-resolution image restoration method based on combined deep network
CN108921886A (en) * 2018-06-11 2018-11-30 昆明理工大学 A kind of texture information fusion Multi-scale model forest digital picture halftoning method
CN111754403A (en) * 2020-06-15 2020-10-09 南京邮电大学 Image super-resolution reconstruction method based on residual learning
CN111986275A (en) * 2020-07-31 2020-11-24 西安理工大学 Inverse halftoning method for multi-modal halftone image
CN112070668A (en) * 2020-08-18 2020-12-11 西安理工大学 Image super-resolution method based on deep learning and edge enhancement
WO2022242029A1 (en) * 2021-05-18 2022-11-24 广东奥普特科技股份有限公司 Generation method, system and apparatus capable of visual resolution enhancement, and storage medium
WO2022257408A1 (en) * 2021-06-10 2022-12-15 南京邮电大学 Medical image segmentation method based on u-shaped network
CN114841872A (en) * 2022-04-12 2022-08-02 浙江大学 Digital halftone processing method based on multi-agent deep reinforcement learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
GUO, J.-M 等: "Error Diffusion Halftone Classification using Contrastive Learning", 2022 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN, 31 December 2022 (2022-12-31) *
张燕;张二虎;: "图像逆半色调技术研究", 西安理工大学学报, no. 03, 30 September 2017 (2017-09-30) *
赵媛, 曾平: "误差分散类逆半调图象的质量评价方法", 计算机工程与应用, no. 34, 1 October 2006 (2006-10-01) *
高净植;刘;张权;桂志国;: "改进深度残差卷积神经网络的LDCT图像估计", 计算机工程与应用, no. 16, 15 August 2018 (2018-08-15) *

Also Published As

Publication number Publication date
CN116934618B (en) 2024-06-11

Similar Documents

Publication Publication Date Title
Da Wang et al. Enhancing resolution of digital rock images with super resolution convolutional neural networks
Wu et al. Reduced-reference image quality assessment with visual information fidelity
Wang et al. Optimized feature extraction for learning-based image steganalysis
CN110443768B (en) Single-frame image super-resolution reconstruction method based on multiple consistency constraints
CN108074239B (en) No-reference image quality objective evaluation method based on prior perception quality characteristic diagram
Liu et al. Pre-attention and spatial dependency driven no-reference image quality assessment
CN114266957B (en) Hyperspectral image super-resolution restoration method based on multi-degradation mode data augmentation
Thakur et al. Agsdnet: Attention and gradient-based sar denoising network
Lahoulou et al. Knowledge-based taxonomic scheme for full-reference objective image quality measurement models
Xu et al. Remote-sensing image usability assessment based on ResNet by combining edge and texture maps
Yang et al. Full reference image quality assessment by considering intra-block structure and inter-block texture
Zhang et al. Linking visual saliency deviation to image quality degradation: A saliency deviation-based image quality index
CN116934618B (en) Image halftone method, system and medium based on improved residual error network
Ponomarenko et al. Sharpness metric for no-reference image visual quality assessment
Da Wang et al. Super resolution convolutional neural network models for enhancing resolution of rock micro-ct images
CN113379641B (en) Single image rain removing method and system based on self-coding convolutional neural network
CN104182935B (en) Image denoising method for selecting wavelet basis on the basis of analytic hierarchy process
CN118429457A (en) Reversible halftone method and system based on residual neural network
Oroumchian ECEI Spectrogram Denoising and Feature Extraction Using a Multi-Wavelet Convolutional Neural Network Model
Plutino et al. Fast ACE (FACE): an error-bounded approximation of Automatic Color Equalization
Qian et al. Dense connected residual generative adversarial network for single image deblurring
CN111402173A (en) Hybrid noise removing method and device, electronic equipment and storage medium
Zhu et al. Reduced-reference image quality assessment based on free-energy principle with multi-channel decomposition
Zheng et al. Image contrast enhancement by contourlet transform and pcnn
Paris Wavelet-based euclidean distance for image quality assessment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant