CN107274378B - Image fuzzy type identification and parameter setting method based on fusion memory CNN - Google Patents
Image fuzzy type identification and parameter setting method based on fusion memory CNN Download PDFInfo
- Publication number
- CN107274378B CN107274378B CN201710609501.3A CN201710609501A CN107274378B CN 107274378 B CN107274378 B CN 107274378B CN 201710609501 A CN201710609501 A CN 201710609501A CN 107274378 B CN107274378 B CN 107274378B
- Authority
- CN
- China
- Prior art keywords
- convolution
- image
- network
- memory
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20088—Trinocular vision calculations; trifocal tensor
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Medicines Containing Antibodies Or Antigens For Use As Internal Diagnostic Agents (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of fuzzy image type identification and parameter calculation, in particular to an image fuzzy type identification and parameter setting method integrating memory CNN (convolutional neural network). The invention comprises the following steps: constructing a converged memory network architecture; setting algorithms of each layer of a fusion memory network; obtaining network parameters through network training; and identifying unknown image fuzzy types and setting parameters. The invention overcomes the defect that the network does not have an independent memory function in the existing fuzzy recognition, and can improve the image fuzzy type and the efficiency of parameter calculation.
Description
Technical Field
The invention relates to the field of fuzzy image type identification and parameter calculation, in particular to an image fuzzy type identification and parameter setting method integrating memory CNN (convolutional neural network).
Background
Today, with the rapid development of network and information technology, CCD and CMOS have become mainstream core sensors following pressure, current and other sensors, and are applied in various fields, no matter in aerial photography or unmanned vehicles, popular face recognition, character recognition, and various industrial detection cameras, and mobile phone camera applications. However, due to the influence of various factors such as environment and the like, the acquired image is not very clear, and especially, the image acquisition in the high-speed motion environment of the camera contains serious fuzzy information, which brings great difficulty to the next application. Therefore, the study of the image blur type and the solution of the blur parameters have important significance for image processing.
Rudin et al use a total variation regularization method to identify fuzzy point extension functions (Rudin L I, Osher S, Fatemi E.nonlinear total variation based fuzzy elimination algorithms [ J ]. Physicd: Nonlinear Phenomena,1992,60(1-4):259-268.), Schmidt et al use a Bayesian model to perform fuzzy kernel estimation (Schmidt U, Schelet K, Roth S.Bayesian approximation with integrated noise estimation [ C ]// Computer Vision and Pattern Recognition (CVPR),2011IEEEConference on.IEEE 2011: 2625-32.) all of which are mathematical models of problems established first to solve parameters. Model methods and parameter methods are successfully applied in many occasions, but various algorithms are mostly suitable for solving specific blurring, the problem of image degradation caused by comprehensive blurring cannot be solved, and the model universality is poor because the algorithms are mostly not considered to be applied to actual image processing.
The emergence of Alphago enables people to know artificial intelligence again, and a large number of scholars apply deep learning to image fuzzy recognition. The reference (YAN R, SHAO L. B. The reference (XUL, REN J S J, LIU C, et al.. Deep connected Neural network for image decoding [ C ]. Advances in Neural Information Processing Systems,2014: 1790-. The reference (SUN J, CAO W, XU Z, et al.. Learing a spatial neural network for non-uniform motion blur removal [ C ]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2015:769-777.) discloses a method using sparse priors, which first predicts the motion blur probability distribution of a partial block in a CNN image, then estimates the blur kernel of the whole image using an MRF model, and then performs image restoration, which cannot estimate the general non-uniform motion blur kernel. Reference (SCHULER C J, HIRSCH M, HAMELLING S, et al. learning to deflur [ J ]. IEEE transactions on pattern analysis and machine estimation, 2016,38(7): 1439-. The image deblurring method based on deep learning focuses on solving certain parameters of a model by applying a neural network, the structure of the neural network is not improved, and the existing network does not have an independent memory function.
Disclosure of Invention
The invention aims to provide an image blur type identification and parameter setting method fusing a memory CNN, which is used for overcoming the defect that a network does not have an independent memory function in the existing blur identification and can improve the efficiency of image blur type and parameter calculation.
The technical scheme of the invention is as follows: an image fuzzy type identification and parameter setting method fusing memory CNN comprises the following steps:
the first step is to construct a converged memory network architecture:
dividing the CNN model into a serial architecture of 5 layers of convolution layers, 1 deep memory network and 1 BP network;
selecting N from the first convolution layer1Convolution operators, each convolution operator being of a size s1×t1A convolution kernel of (2), wherein s1Number of rows of convolution kernels, t1Is the number of columns of the convolution kernel; the convolution kernel is composed of a plurality of different straight lines, a plurality of different discs and a plurality of different circular rings, and each convolution kernel extracts the first-level shape characteristics of the image sub-graph unit;
second convolutional layer selection of N2Convolution operators, each convolution operator being of a size s2×t2A convolution kernel of (2), wherein s2Number of rows of convolution kernels, t2Is the number of columns of the convolution kernel; extracting a secondary shape feature of the image subgraph by each convolution kernel;
third convolutional layer selection of N3Convolution operators, each convolution operator being of a size s3×t3A convolution kernel of (2), wherein s3Number of rows of convolution kernels, t3Is the number of columns of the convolution kernel; extracting a three-level shape feature of the image subgraph by each convolution kernel;
fourth convolutional layer selection of N4Convolution operators, each convolution operator being of a size s4×t4A convolution kernel of (2), wherein s4Number of rows of convolution kernels, t4Is the number of columns of the convolution kernel; extracting a four-level shape feature of the image subgraph by each convolution kernel;
fifth convolutional layer selection of N5Convolution operators, each convolution operator being of a size s5×t5A convolution kernel of (2), wherein s5Number of rows of convolution kernels, t5Is the number of columns of the convolution kernel; extracting a five-level shape feature of the image subgraph by each convolution kernel;
the deep memory network adopts a memory model with the depth of D units and adopts a selective updating rule;
the BP network adopts a 4-layer structure, an input layer, two hidden layers and an output layer;
secondly, setting algorithms of each layer of the fusion memory network:
1) carrying out gray processing on an input image, converting the original input image into a corresponding gray image by a vector mapping method when the original input image is a three-dimensional color image, and skipping the step if the input image is the gray image;
2) two-dimensional Fourier transform is carried out on the detected image, the two-dimensional Fourier transform is converted into a spectrogram, and the spectrogram is marked as an image P0;
3) Image P0N with the first winding layer1The convolution operators respectively carry out convolution operation, and the calculation expression is as follows:
whereinIs an image P0At pixel [ (i-1) Δ1+1+x,(j-1)Δ1+1+y]The gray-scale value of (a) is,denotes the n-th1At position [ x, y ] of convolution operator]The weight of the (c) position(s),for the convolved image P1At pixel [ i, j]Gray value of(s)1Number of rows, t, of convolution operators1Is the number of columns, Δ, of the convolution operator1For convolution shift step size, n1Is the serial number of the convolution operator, and the range is n is more than or equal to 11≤N1;
4) Regularizing the convolution image, wherein the processing process comprises the following steps:
5) will be provided withPerforming maximum pooling calculation on the image, wherein the calculation method comprises the following steps:
6) calculating a result of the second layer convolution operation according to the second layer convolution kernel by referring to the step 3), the step 4) and the step 5);
7) calculating the result of the third layer of convolution operation according to the third layer of convolution kernel by referring to the step 3), the step 4) and the step 5);
8) performing similarity clustering analysis on the matrix obtained in the step 7), and keeping the number of the third-level features of the image as M1A plurality of;
9) on the basis of the step 8), calculating a result of the fourth layer convolution operation according to the fourth layer convolution kernel by referring to the step 3), the step 4) and the step 5);
10) carrying out similarity clustering analysis on the matrix obtained in the step 9), and keeping the fourth-level feature quantity of the image as M2A plurality of;
11) calculating a result of fifth-layer convolution operation according to a fifth-layer convolution kernel in the step 10) by referring to the step 3), the step 4) and the step 5);
12) carrying out similarity clustering analysis on the matrix obtained in the step 11), and obtaining the matrix by taking the sum of all elements of the matrix as comprehensive characteristicsTo M3A plurality of different feature points;
13) inputting the characteristic values output in the step 12) into corresponding memory models, and generating corresponding output information through the memory models; the concrete memory model is as follows:
a network structure having D independent memory cells, wherein the network input x (t) is compared with the memory values of the memory cells, and the error at the cell k closest to the input is:
δk(t)=Min{|Ci(t)-x(t)|,i=1,2,…,D}, (4),
when deltak(t) is less than or equal to the network identification threshold epsilon, indicating that the network successfully identifies the k-th class of information, and the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
when deltak(t) when the memory coefficient is greater than the network recognition threshold epsilon, the network recognition process is not input, the memory network updates the information with the worst memory according to the forgetting rule, namely the unit k with the lowest memory coefficient is replaced by the current input information, and the memory coefficient β of each memory unit at the momenti(t) and memory information Ci(t) the selective memory update rule is:
βk(t)=Min{βi(t),i=1,2,…,D}, (7),
the network output is:
h(t+1)=Ck(t+1), (10),
14) taking the elements output in the step 13) as the input of the full-connection BP network, designing the number of nodes of the intermediate hidden layer in a decreasing mode, and outputting 5 nodes on the layer; the meaning of each node of the output layer is respectively: the 1 st output node is of a fuzzy type, the defocus fuzzy is 1, the motion fuzzy is 2, and the Gaussian fuzzy is 3; the 2 nd output node is the radius r of the defocus blur, when the 1 st output node is 1, the output is the calculated radius value, otherwise, the output is 0; the 3 rd output node is the length of the motion blur; the 4 th output node is the direction angle of the motion model lake; the 5 th output node is the noise variance of Gaussian blur;
15) the network provides a manual judgment feedback function in the parameter training process, when a user finds that the network identification is wrong, error information is input into a network correction interface, the network automatically learns again, and the weight matrix information of the system is updated;
thirdly, obtaining network parameters through network training:
after the network construction and the algorithm setting are finished, learning training is carried out by using the known images of 1-100 ten thousand frames and the characteristics of the known images to obtain network parameters;
fourthly, unknown image fuzzy type identification and parameter setting:
image information acquired in the actual production process or fuzzy image information to be identified is input into the network, and the fuzzy type, the out-of-focus fuzzy radius, the motion fuzzy length, the motion model lake direction angle and Gaussian fuzzy noise variance network output are obtained through network calculation, so that the image fuzzy type identification and parameter setting are realized.
On the basis of memory CNN, the invention adopts 5 layers of variable step length convolution operation aiming at the image with high resolution, and when the resolution is higher than 200 x 200 pixels, the invention adopts step length control to accelerate convolution convergence; when the resolution is lower than 200 × 200 pixels, the number of features is guaranteed using a single-step convolution control. In order to avoid the phenomenon of back-layer scale surge in the convolutional network, the number of layer output feature matrixes is reduced by adopting clustering analysis, and the difference and the calculation scale of network features are ensured. And simulating a human memory process in the obtained detail feature processing process, and memorizing, shaping and outputting the features through a deep memory network model. And finally, identifying and calculating the fuzzy type and the parameter size through a BP network. The invention can improve the image blur type and the efficiency of parameter calculation.
Detailed Description
The invention provides an image fuzzy type identification and parameter setting method integrating memory CNN, which is used for overcoming the defect that a network does not have a memory function in the existing fuzzy identification.
The present invention will be described in detail with reference to examples (track surface defect detection).
The first step is to construct a converged memory network architecture:
dividing the CNN model into a serial architecture of 5 layers of convolution layers, 1 deep memory network and 1 BP network;
the first convolution layer selects 96 convolution operators, each convolution operator is a convolution kernel of 16 multiplied by 16, each convolution kernel comprises 72 straight lines with different forms, 8 discs with different sizes and 16 circular rings with different forms, and each convolution kernel extracts the primary shape characteristics of the image sub-graph unit;
the second convolution layer selects 256 convolution operators, each convolution operator is an 8 x 8 convolution kernel, and each convolution kernel extracts a two-level shape feature of the image sub-graph;
the third convolution layer selects 256 convolution operators, each convolution operator is a convolution kernel of 5 multiplied by 5, and each convolution kernel extracts a three-level shape feature of the image subgraph;
the fourth convolution layer selects 384 convolution operators, each convolution operator is a convolution kernel of 3 multiplied by 3, and each convolution kernel extracts a four-level shape feature of the image subgraph;
the fifth convolution layer selects 384 convolution operators, each convolution operator is a convolution kernel of 3 multiplied by 3, and each convolution kernel extracts a five-level shape feature of the image subgraph;
the deep memory network adopts a memory model with the depth of 10 units and adopts a selective updating rule;
the BP network adopts a 4-layer structure, an input layer, two hidden layers and an output layer;
secondly, setting algorithms of each layer of the fusion memory network:
1) carrying out gray processing on an input image, converting the original input image into a corresponding gray image by a vector mapping method when the original input image is a three-dimensional color image, and skipping the step if the input image is the gray image;
2) two-dimensional Fourier transform is carried out on the detected image, the two-dimensional Fourier transform is converted into a spectrogram, and the spectrogram is marked as an image P0Picture P0The size is 1024 × 1024;
3) image P0And respectively carrying out convolution operation with 96 convolution operators of the first convolution layer of the network, wherein the calculation expression is as follows:
whereinIs an image P0At pixel [2i-1+ x,2j-1+ y]The gray-scale value of (a) is,indicating the convolution operator at position x, y]The value of (a) is (b),for the convolved image P1At pixel [ i, j]The gray value of (d); after the first layer of convolution operation, 96 convolution images with the size of 504 multiplied by 504 are output;
4) regularizing the convolution image, wherein the processing process comprises the following steps:
5) will be provided withPerforming maximum pooling calculation on the image, wherein the calculation method comprises the following steps:
outputting 96 feature matrixes with the size of 252 multiplied by 252 through the maximum pooling effect;
6) repeating the step 3), the step 4) and the step 5) to carry out second-layer convolution operation, and selecting the parameter s2And t2Are all 8, Δ2Obtaining 24576 feature matrices with the size of 61 × 61;
7) repeating the step 3), the step 4) and the step 5) to carry out a third layer of convolution operation, and selecting a parameter s3And t3Are all 5, Δ3For 1, 629 ten thousand feature matrices of 28 × 28 size are obtained;
8) carrying out similarity clustering analysis on the matrix obtained in the step 7), and keeping 10 ten thousand image feature matrices;
9) repeating the step 3), the step 4) and the step 5) to carry out a fourth layer of convolution operation, and selecting a parameter s4And t4Are all 3, Δ4For 1, 3840 ten thousand feature matrices with the size of 12 × 12 are obtained;
10) performing similarity clustering analysis on the matrix obtained in the step 9), and keeping 1 ten thousand image feature matrices;
11) repeating the steps 3), 4) and 5) to carry out fifth-layer convolution operation, and selecting the parameter s5And t5Are all 3, Δ5Obtaining 384 ten thousand feature matrices with the size of 4 multiplied by 4 as 1;
12) carrying out similarity clustering analysis on the matrix obtained in the step 11), and taking the sum of all elements of the matrix as comprehensive characteristics to obtain 1000 different characteristic points;
13) inputting the characteristic values output in the step 12) into corresponding memory models, wherein the network is provided with 1000 independent memory units with the memory depth of 10, and corresponding output information is generated through the memory models; the concrete memory model is as follows:
a network structure having 10 independent memory cells, wherein the network input x (t) is compared with the memory values of the memory cells, and the error at the cell k closest to the input is:
δk(t)=Min{|Ci(t)-x(t)|,i=1,2,…,10}, (13),
when deltak(t) is less than or equal to the network identification threshold epsilon, indicating that the network successfully identifies the k-th class of information, and the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
when deltak(t) when the value is larger than the network identification threshold value epsilon, the network identification process is not input, the memory network updates the worst memorized information according to the forgetting rule, βk(t) replacing the lowest memory coefficient k with the current input information, wherein the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
βk(t)=Min{βi(t),i=1,2,…,10}, (15),
the network output h (t +1) is:
h(t+1)=Ck(t+1), (10),
14) taking the elements output in the step 13) as the input of a full-connection BP network, wherein 500 nodes are designed in a first hidden layer, 50 nodes are designed in a second hidden layer, and 5 nodes are output in the output layer; the meaning of each node of the output layer is respectively: the 1 st output node is of a fuzzy type, the defocus fuzzy is 1, the motion fuzzy is 2, and the Gaussian fuzzy is 3; the 2 nd output node is the radius r of the defocus blur, when the 1 st output node is 1, the output is the calculated radius value, otherwise, the output is 0; the 3 rd output node is the length of the motion blur; the 4 th output node is the direction angle of the motion model lake; the 5 th output node is the noise variance of Gaussian blur;
15) the network provides a manual judgment feedback function in the parameter training process, when a user finds that the network identification is wrong, error information is input into a network correction interface, the network automatically learns again, and the weight matrix information of the system is updated;
thirdly, obtaining network parameters through network training:
constructing 50 ten thousand fuzzy images with different fuzzy types and different parameters based on an image database, namely, a cultech 101dataset and an acquired track surface video image database, taking the fuzzy image set as input and the corresponding parameters as output, and performing network training to obtain network parameters;
fourthly, unknown image fuzzy type identification and parameter setting:
taking 1000 images in a caltech 101dataset database as tested images, testing, and calculating the fuzzy type, the out-of-focus fuzzy radius, the motion fuzzy length, the direction angle of a motion model lake and the noise variance network output of Gaussian blur corresponding to each image; comparing the network output with a known result, wherein the accuracy reaches 99.7 percent, and the image fuzzy type identification and parameter setting are realized;
meanwhile, applying the fusion memory CNN to an image deblurring processing process in the track surface defect detection system, and calculating the blur type, the defocus blur radius, the motion blur length, the motion model lake direction angle and the Gaussian blur noise variance of the detected image; and applying the setting parameters to image deblurring, wherein the deblurring effect obtained from result analysis meets the requirements of a detection system.
The described embodiments are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (2)
1. An image fuzzy type identification and parameter setting method fusing memory CNN is characterized by comprising the following steps:
the first step is to construct a converged memory network architecture:
dividing the CNN model into a serial architecture of 5 layers of convolution layers, 1 deep memory network and 1 BP network;
selecting N from the first convolution layer1Convolution operators, each convolution operator being of a size s1×t1A convolution kernel of (2), wherein s1Number of rows of convolution kernels, t1Is the number of columns of the convolution kernel; the convolution kernel is composed of a plurality of different straight lines, a plurality of different discs and a plurality of different circular rings, and each convolution kernel extracts the first-level shape characteristics of the image sub-graph unit;
second convolutional layer selection of N2Convolution operators, each convolution operator being of a size s2×t2A convolution kernel of (2), wherein s2Number of rows of convolution kernels, t2Is the number of columns of the convolution kernel; extracting a secondary shape feature of the image subgraph by each convolution kernel;
third convolutional layer selection of N3Convolution operators, each convolution operator being of a size s3×t3A convolution kernel of (2), wherein s3Number of rows of convolution kernels, t3Is the number of columns of the convolution kernel; extracting a three-level shape feature of the image subgraph by each convolution kernel;
fourth convolutional layer selection of N4Convolution operators, each convolution operator being of a size s4×t4A convolution kernel of (2), wherein s4Number of rows of convolution kernels, t4Is the number of columns of the convolution kernel; extracting a four-level shape feature of the image subgraph by each convolution kernel;
fifth convolutional layer selection of N5A convolution operator, per volumeThe product operator is of a size s5×t5A convolution kernel of (2), wherein s5Number of rows of convolution kernels, t5Is the number of columns of the convolution kernel; extracting a five-level shape feature of the image subgraph by each convolution kernel;
the deep memory network adopts a memory model with the depth of D units and adopts a selective updating rule;
the BP network adopts a 4-layer structure, an input layer, two hidden layers and an output layer;
secondly, setting algorithms of each layer of the fusion memory network:
1) carrying out gray processing on an input image, converting the original input image into a corresponding gray image by a vector mapping method when the original input image is a three-dimensional color image, and skipping the step if the input image is the gray image;
2) two-dimensional Fourier transform is carried out on the detected image, the two-dimensional Fourier transform is converted into a spectrogram, and the spectrogram is marked as an image P0;
3) Image P0N with the first winding layer1The convolution operators respectively carry out convolution operation, and the calculation expression is as follows:
whereinIs an image P0At pixel [ (i-1) Δ1+1+x,(j-1)Δ1+1+y]The gray-scale value of (a) is,denotes the n-th1At position [ x, y ] of convolution operator]The weight of the (c) position(s),for the convolved image P1At pixel [ i, j]Gray value of(s)1Number of rows, t, of convolution operators1Is the number of columns, Δ, of the convolution operator1For convolution shift step size, n1Is the serial number of the convolution operator, and the range is n is more than or equal to 11≤N1;
4) Regularizing the convolution image, wherein the processing process comprises the following steps:
5) will be provided withPerforming maximum pooling calculation on the image, wherein the calculation method comprises the following steps:
6) calculating a result of the second layer convolution operation according to the second layer convolution kernel by referring to the step 3), the step 4) and the step 5);
7) calculating the result of the third layer of convolution operation according to the third layer of convolution kernel by referring to the step 3), the step 4) and the step 5);
8) performing similarity clustering analysis on the matrix obtained in the step 7), and keeping the number of the third-level features of the image as M1A plurality of;
9) on the basis of the step 8), calculating a result of the fourth layer convolution operation according to the fourth layer convolution kernel by referring to the step 3), the step 4) and the step 5);
10) carrying out similarity clustering analysis on the matrix obtained in the step 9), and keeping the fourth-level feature quantity of the image as M2A plurality of;
11) calculating a result of fifth-layer convolution operation according to a fifth-layer convolution kernel in the step 10) by referring to the step 3), the step 4) and the step 5);
12) carrying out similarity clustering analysis on the matrix obtained in the step 11), and obtaining M by taking the sum of all elements of the matrix as comprehensive characteristics3A plurality of different feature points;
13) inputting the characteristic values output in the step 12) into corresponding memory models, and generating corresponding output information through the memory models; the concrete memory model is as follows:
a network structure having D independent memory cells, wherein the network input x (t) is compared with the memory values of the memory cells, and the error at the cell k closest to the input is:
δk(t)=Min{|Ci(t)-x(t)| i=1,2,…,D}, (4),
when deltak(t) is less than or equal to the network identification threshold epsilon, indicating that the network successfully identifies the k-th class of information, and the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
when deltak(t) when the memory coefficient is greater than the network recognition threshold epsilon, the network recognition process is not input, the memory network updates the information with the worst memory according to the forgetting rule, namely the unit k with the lowest memory coefficient is replaced by the current input information, and the memory coefficient β of each memory unit at the momenti(t) and memory information Ci(t) the selective memory update rule is:
βk(t)=Min{βi(t),i=1,2,…,D}, (7),
the network output is:
h(t+1)=Ck(t+1), (10),
14) taking the elements output in the step 13) as the input of the full-connection BP network, designing the number of nodes of the intermediate hidden layer in a decreasing mode, and outputting 5 nodes on the layer; the meaning of each node of the output layer is respectively: the 1 st output node is of a fuzzy type, the defocus fuzzy is 1, the motion fuzzy is 2, and the Gaussian fuzzy is 3; the 2 nd output node is the radius r of the defocus blur, when the 1 st output node is 1, the output is the calculated radius value, otherwise, the output is 0; the 3 rd output node is the length of the motion blur; the 4 th output node is the direction angle of the motion model lake; the 5 th output node is the noise variance of Gaussian blur;
15) the network provides a manual judgment feedback function in the parameter training process, when a user finds that the network identification is wrong, error information is input into a network correction interface, the network automatically learns again, and the weight matrix information of the system is updated;
thirdly, obtaining network parameters through network training:
after the network construction and the algorithm setting are finished, learning training is carried out by using the known images of 1-100 ten thousand frames and the characteristics of the known images to obtain network parameters;
fourthly, unknown image fuzzy type identification and parameter setting:
image information acquired in the actual production process or fuzzy image information to be identified is input into the network, and the fuzzy type, the out-of-focus fuzzy radius, the motion fuzzy length, the motion model lake direction angle and Gaussian fuzzy noise variance network output are obtained through network calculation, so that the image fuzzy type identification and parameter setting are realized.
2. The image blur type identification and parameter setting method of the fusion memory CNN as claimed in claim 1, which is implemented as follows:
the first step is to construct a converged memory network architecture:
dividing the CNN model into a serial architecture of 5 layers of convolution layers, 1 deep memory network and 1 BP network;
the first convolution layer selects 96 convolution operators, each convolution operator is a convolution kernel of 16 multiplied by 16, each convolution kernel comprises 72 straight lines with different forms, 8 discs with different sizes and 16 circular rings with different forms, and each convolution kernel extracts the primary shape characteristics of the image sub-graph unit;
the second convolution layer selects 256 convolution operators, each convolution operator is an 8 x 8 convolution kernel, and each convolution kernel extracts a two-level shape feature of the image sub-graph;
the third convolution layer selects 256 convolution operators, each convolution operator is a convolution kernel of 5 multiplied by 5, and each convolution kernel extracts a three-level shape feature of the image subgraph;
the fourth convolution layer selects 384 convolution operators, each convolution operator is a convolution kernel of 3 multiplied by 3, and each convolution kernel extracts a four-level shape feature of the image subgraph;
the fifth convolution layer selects 384 convolution operators, each convolution operator is a convolution kernel of 3 multiplied by 3, and each convolution kernel extracts a five-level shape feature of the image subgraph;
the deep memory network adopts a memory model with the depth of 10 units and adopts a selective updating rule;
the BP network adopts a 4-layer structure, an input layer, two hidden layers and an output layer;
secondly, setting algorithms of each layer of the fusion memory network:
1) carrying out gray processing on an input image, converting the original input image into a corresponding gray image by a vector mapping method when the original input image is a three-dimensional color image, and skipping the step if the input image is the gray image;
2) two-dimensional Fourier transform is carried out on the detected image, the two-dimensional Fourier transform is converted into a spectrogram, and the spectrogram is marked as an image P0Picture P0The size is 1024 × 1024;
3) image P0And respectively carrying out convolution operation with 96 convolution operators of the first convolution layer of the network, wherein the calculation expression is as follows:
whereinIs an image P0At pixel [2i-1+ x,2j-1+ y]The gray-scale value of (a) is,indicating the convolution operator at position x, y]The value of (a) is (b),for the convolved image P1At pixel [ i, j]The gray value of (d); after the first layer of convolution operation, 96 convolution images with the size of 504 multiplied by 504 are output;
4) regularizing the convolution image, wherein the processing process comprises the following steps:
5) will be provided withPerforming maximum pooling calculation on the image, wherein the calculation method comprises the following steps:
outputting 96 feature matrixes with the size of 252 multiplied by 252 through the maximum pooling effect;
6) repeating the step 3), the step 4) and the step 5) to carry out second-layer convolution operation, and selecting the parameter s2And t2Are all 8, Δ2Obtaining 24576 feature matrices with the size of 61 × 61;
7) repeating the step 3), the step 4) and the step 5) to carry out a third layer of convolution operation, and selecting parameterss3And t3Are all 5, Δ3For 1, 629 ten thousand feature matrices of 28 × 28 size are obtained;
8) carrying out similarity clustering analysis on the matrix obtained in the step 7), and keeping 10 ten thousand image feature matrices;
9) repeating the step 3), the step 4) and the step 5) to carry out a fourth layer of convolution operation, and selecting a parameter s4And t4Are all 3, Δ4For 1, 3840 ten thousand feature matrices with the size of 12 × 12 are obtained;
10) performing similarity clustering analysis on the matrix obtained in the step 9), and keeping 1 ten thousand image feature matrices;
11) repeating the steps 3), 4) and 5) to carry out fifth-layer convolution operation, and selecting the parameter s5And t5Are all 3, Δ5Obtaining 384 ten thousand feature matrices with the size of 4 multiplied by 4 as 1;
12) carrying out similarity clustering analysis on the matrix obtained in the step 11), and taking the sum of all elements of the matrix as comprehensive characteristics to obtain 1000 different characteristic points;
13) inputting the characteristic values output in the step 12) into corresponding memory models, wherein the network is provided with 1000 independent memory units with the memory depth of 10, and corresponding output information is generated through the memory models; the concrete memory model is as follows:
a network structure having 10 independent memory cells, wherein the network input x (t) is compared with the memory values of the memory cells, and the error at the cell k closest to the input is:
δk(t)=Min{|Ci(t)-x(t)|,i=1,2,…,10}, (13),
when deltak(t) is less than or equal to the network identification threshold epsilon, indicating that the network successfully identifies the k-th class of information, and the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
when deltak(t) when the value is larger than the network identification threshold value epsilon, the network identification process is not input, the memory network updates the worst memorized information according to the forgetting rule, βk(t) replacing the lowest memory coefficient k with the current input information, wherein the memory coefficient β of each memory celli(t) and memory information Ci(t) the selective memory update rule is:
βk(t)=Min{βi(t),i=1,2,…,10}, (15),
the network output h (t +1) is:
h(t+1)=Ck(t+1), (10),
14) taking the elements output in the step 13) as the input of a full-connection BP network, wherein 500 nodes are designed in a first hidden layer, 50 nodes are designed in a second hidden layer, and 5 nodes are output in the output layer; the meaning of each node of the output layer is respectively: the 1 st output node is of a fuzzy type, the defocus fuzzy is 1, the motion fuzzy is 2, and the Gaussian fuzzy is 3; the 2 nd output node is the radius r of the defocus blur, when the 1 st output node is 1, the output is the calculated radius value, otherwise, the output is 0; the 3 rd output node is the length of the motion blur; the 4 th output node is the direction angle of the motion model lake; the 5 th output node is the noise variance of Gaussian blur;
15) the network provides a manual judgment feedback function in the parameter training process, when a user finds that the network identification is wrong, error information is input into a network correction interface, the network automatically learns again, and the weight matrix information of the system is updated;
thirdly, obtaining network parameters through network training:
constructing 50 ten thousand fuzzy images with different fuzzy types and different parameters based on an image database, namely, a cultech 101dataset and an acquired track surface video image database, taking the fuzzy image set as input and the corresponding parameters as output, and performing network training to obtain network parameters;
fourthly, unknown image fuzzy type identification and parameter setting:
taking 1000 images in a caltech 101dataset database as tested images, testing, and calculating the fuzzy type, the out-of-focus fuzzy radius, the motion fuzzy length, the direction angle of a motion model lake and the noise variance network output of Gaussian blur corresponding to each image; comparing the network output with a known result, wherein the accuracy reaches 99.7 percent, and the image fuzzy type identification and parameter setting are realized;
meanwhile, applying the fusion memory CNN to an image deblurring processing process in the track surface defect detection system, and calculating the blur type, the defocus blur radius, the motion blur length, the motion model lake direction angle and the Gaussian blur noise variance of the detected image; and applying the setting parameters to image deblurring, wherein the deblurring effect obtained from result analysis meets the requirements of a detection system.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710609501.3A CN107274378B (en) | 2017-07-25 | 2017-07-25 | Image fuzzy type identification and parameter setting method based on fusion memory CNN |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710609501.3A CN107274378B (en) | 2017-07-25 | 2017-07-25 | Image fuzzy type identification and parameter setting method based on fusion memory CNN |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107274378A CN107274378A (en) | 2017-10-20 |
CN107274378B true CN107274378B (en) | 2020-04-03 |
Family
ID=60079524
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710609501.3A Active CN107274378B (en) | 2017-07-25 | 2017-07-25 | Image fuzzy type identification and parameter setting method based on fusion memory CNN |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107274378B (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108446631B (en) * | 2018-03-20 | 2020-07-31 | 北京邮电大学 | Deep learning intelligent spectrogram analysis method based on convolutional neural network |
CN108537746B (en) * | 2018-03-21 | 2021-09-21 | 华南理工大学 | Fuzzy variable image blind restoration method based on deep convolutional network |
CN108830801A (en) * | 2018-05-10 | 2018-11-16 | 湖南丹尼尔智能科技有限公司 | A kind of deep learning image recovery method of automatic identification vague category identifier |
CN108898563B (en) * | 2018-07-02 | 2021-01-22 | 京东方科技集团股份有限公司 | Processing method of optical detection image of display panel and computer readable medium |
CN109345449B (en) * | 2018-07-17 | 2020-11-10 | 西安交通大学 | Image super-resolution and non-uniform blur removing method based on fusion network |
CN109445457B (en) * | 2018-10-18 | 2021-05-14 | 广州极飞科技股份有限公司 | Method for determining distribution information, and method and device for controlling unmanned aerial vehicle |
CN111339995B (en) * | 2020-03-16 | 2024-02-20 | 合肥闪捷信息科技有限公司 | Sensitive image recognition method based on neural network |
CN112183650B (en) * | 2020-10-09 | 2023-09-22 | 青岛中瑞车云工业互联网科技有限公司 | Digital detection and identification method under camera defocus condition |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104392205A (en) * | 2014-10-24 | 2015-03-04 | 浙江力石科技股份有限公司 | Abnormal vehicle license plate recognition method and system |
CN105844239A (en) * | 2016-03-23 | 2016-08-10 | 北京邮电大学 | Method for detecting riot and terror videos based on CNN and LSTM |
CN106782602A (en) * | 2016-12-01 | 2017-05-31 | 南京邮电大学 | Speech-emotion recognition method based on length time memory network and convolutional neural networks |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10095950B2 (en) * | 2015-06-03 | 2018-10-09 | Hyperverge Inc. | Systems and methods for image processing |
-
2017
- 2017-07-25 CN CN201710609501.3A patent/CN107274378B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104392205A (en) * | 2014-10-24 | 2015-03-04 | 浙江力石科技股份有限公司 | Abnormal vehicle license plate recognition method and system |
CN105844239A (en) * | 2016-03-23 | 2016-08-10 | 北京邮电大学 | Method for detecting riot and terror videos based on CNN and LSTM |
CN106782602A (en) * | 2016-12-01 | 2017-05-31 | 南京邮电大学 | Speech-emotion recognition method based on length time memory network and convolutional neural networks |
Non-Patent Citations (1)
Title |
---|
基于深度卷积神经网络的快速图像分类算法;王华利;《计算机工程与应用》;20160510;第53卷(第13期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN107274378A (en) | 2017-10-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107274378B (en) | Image fuzzy type identification and parameter setting method based on fusion memory CNN | |
CN110188685B (en) | Target counting method and system based on double-attention multi-scale cascade network | |
Suganuma et al. | Attention-based adaptive selection of operations for image restoration in the presence of unknown combined distortions | |
CN109360171B (en) | Real-time deblurring method for video image based on neural network | |
CN108764292B (en) | Deep learning image target mapping and positioning method based on weak supervision information | |
Guo et al. | Multiview high dynamic range image synthesis using fuzzy broad learning system | |
CN107229914B (en) | Handwritten digit recognition method based on deep Q learning strategy | |
CN105701508B (en) | Global local optimum model and conspicuousness detection algorithm based on multistage convolutional neural networks | |
CN111126412B (en) | Image key point detection method based on characteristic pyramid network | |
CN109598704B (en) | Fecal microscopic image definition evaluation method based on BP neural network | |
CN108399435B (en) | Video classification method based on dynamic and static characteristics | |
CN107368887B (en) | Deep memory convolutional neural network device and construction method thereof | |
CN108171249B (en) | RGBD data-based local descriptor learning method | |
CN108985443B (en) | Action recognition method and neural network generation method and device thereof, and electronic equipment | |
CN104268524A (en) | Convolutional neural network image recognition method based on dynamic adjustment of training targets | |
CN113128424A (en) | Attention mechanism-based graph convolution neural network action identification method | |
KR101563569B1 (en) | Learnable Dynamic Visual Image Pattern Recognition System and Method | |
CN112561807B (en) | End-to-end radial distortion correction method based on convolutional neural network | |
CN110827304A (en) | Traditional Chinese medicine tongue image positioning method and system based on deep convolutional network and level set method | |
CN115376024A (en) | Semantic segmentation method for power accessory of power transmission line | |
CN111428664A (en) | Real-time multi-person posture estimation method based on artificial intelligence deep learning technology for computer vision | |
CN111553296B (en) | Two-value neural network stereo vision matching method based on FPGA | |
CN112819096A (en) | Method for constructing fossil image classification model based on composite convolutional neural network | |
CN113160179A (en) | Image deblurring method based on dynamic region convolution | |
CN116342536A (en) | Aluminum strip surface defect detection method, system and equipment based on lightweight model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |