CN110378044B - Multi-time scale convolution neural network soft measurement method based on attention mechanism - Google Patents

Multi-time scale convolution neural network soft measurement method based on attention mechanism Download PDF

Info

Publication number
CN110378044B
CN110378044B CN201910667918.4A CN201910667918A CN110378044B CN 110378044 B CN110378044 B CN 110378044B CN 201910667918 A CN201910667918 A CN 201910667918A CN 110378044 B CN110378044 B CN 110378044B
Authority
CN
China
Prior art keywords
time
time sequence
neural network
variable
auxiliary variable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910667918.4A
Other languages
Chinese (zh)
Other versions
CN110378044A (en
Inventor
赵彦涛
丁伯川
杨黎明
张玉玲
郝晓辰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yanshan University
Original Assignee
Yanshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yanshan University filed Critical Yanshan University
Priority to CN201910667918.4A priority Critical patent/CN110378044B/en
Publication of CN110378044A publication Critical patent/CN110378044A/en
Application granted granted Critical
Publication of CN110378044B publication Critical patent/CN110378044B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Abstract

The invention relates to a multi-time scale convolution neural network soft measurement method based on an attention mechanism, and belongs to the technical field of soft measurement. The method comprises the following steps: 1. determining an auxiliary variable and processing data, selecting an easily-measured variable related to a difficultly-measured parameter as the auxiliary variable of the soft measurement model and collecting a time sequence of the auxiliary variable and the difficultly-measured parameter; then removing abnormal values of the acquired time sequence; 2. an attention mechanism and an attention area are selected, and the attention area is divided according to the time delay and the effective time scale of each auxiliary variable relative to the difficultly-measured parameter; 3. establishing input of a soft measurement model, forming a matrix by the time sequence of each auxiliary variable, and determining the input of the soft measurement model by combining an attention area of an attention mechanism; 4. establishing a time sequence convolution neural network soft measurement model; 5. training a time sequence convolution neural network soft measurement model; 6. and 5, estimating the difficultly-measured parameters in real time by using the trained time sequence convolution neural network model in the step 5.

Description

Multi-time scale convolution neural network soft measurement method based on attention mechanism
Technical Field
The invention relates to a multi-time scale convolution neural network soft measurement method based on an attention mechanism, and belongs to the technical field of soft measurement.
Background
In the modern industrial production process, in order to realize energy conservation and benefit maximization, the monitoring and the control of important parameters in the production process are of great significance. Generally, for important parameters in industrial production processes, the measurement method mainly comprises on-line measurement and off-line measurement. On-line measurement refers to the measurement of parameters directly by using an instrument, but the equipment is expensive and difficult to maintain, and the accuracy of the measurement result is easily influenced by the field working condition. The off-line measurement refers to measuring parameters by using an off-line inspection method, but the off-line inspection usually requires a long time, so that the measurement result obtained off-line has a large time delay for guiding the production process. Therefore, how to estimate the undetected parameters in real time becomes a key problem to be solved first in process control.
Since the 90 s of the 20 th century, the rapid development of soft measurement technology is becoming one of the effective ways to solve the above problems. The soft measurement technology is a method for realizing online real-time estimation of difficultly-measured parameters by establishing a prediction model by using easily-obtained auxiliary variables, and can provide required important real-time information for process monitoring, optimization and control so as to realize the aims of saving energy and maximizing benefits.
Disclosure of Invention
The invention aims to provide a multi-time scale convolutional neural network soft measurement method based on an attention mechanism, so that real-time online estimation of difficultly-measured parameters is realized.
In order to achieve the purpose, the invention adopts the technical scheme that:
the method for soft measurement of the multi-time scale convolutional neural network based on the attention mechanism comprises the following steps:
step 1, determining auxiliary variables and processing data
Through the analysis of the industrial process flow, primarily selecting an easily-measured variable related to a difficultly-measured parameter as an auxiliary variable of a soft measurement model and collecting a time sequence of the auxiliary variable and the difficultly-measured parameter;
then, removing abnormal values in the acquired data by adopting a 3 sigma criterion, and carrying out normalization processing on the data before training;
step 2, attention mechanism and attention area selection
A hard attention mechanism is adopted, and an attention area is divided according to the time delay and the effective time scale of each auxiliary variable relative to the difficultly-measured parameter in the process flow;
step 3, input for constructing soft measurement model
Forming a matrix by the time sequence of each auxiliary variable, and determining the input of a soft measurement model by combining the attention area of an attention mechanism;
step 4, establishing a soft measurement model of the time sequence convolution neural network
Determining initial parameters of a time sequence convolution neural network model, and carrying out forward training on the network; the initial parameters comprise the number of the convolution layers and the number of the pooling layers of the time sequence convolution neural network, the learning rate, the weight w and the bias b of each hidden layer, all-connected layer and output layer, and the number and the size of convolution kernels and pooling kernels;
step 5, training time sequence convolution neural network soft measurement model
Carrying out supervised training by utilizing error reverse fine tuning, and optimizing weight w and bias b in the time sequence convolutional neural network by correcting errors;
step 6, estimating the difficultly-measured parameters in real time by utilizing the time sequence convolution neural network model trained in the step 5
The technical scheme of the invention is further improved as follows: in the step 1, when a 3 sigma criterion is adopted to process data abnormal values, the abnormal values of the parameters which are difficult to be measured are removed; in the process of eliminating abnormal values of the parameters difficult to measure, the time sequence of the auxiliary variables corresponding to the abnormal values is eliminated;
the specific method comprises the following steps:
setting the sampling sequence of the difficultly-detected parameters at different moments as y (k) ═ y (0), …, y (N)), judging each point y (i) in y (k), if the formula (1) is satisfied, indicating that the point is an abnormal point and needs to be removed, and simultaneously removing each auxiliary variable data corresponding to the point;
Figure BDA0002140712450000031
wherein y is the mean of the variables y (k); σ is the standard deviation of the variable y (k);
carrying out Min-Max standardization treatment on the data of each variable according to the formula (2), converting the data into a non-dimensionalized index mapping evaluation value, enabling the data of each variable to be in the same quantity grade, and carrying out comprehensive evaluation analysis;
Figure BDA0002140712450000032
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000033
normalized time series for variable i, ximinIs the minimum value of variable i, ximaxThe maximum value of the variable i.
The technical scheme of the invention is further improved as follows: in step 2, judging the time delay parameters of the auxiliary variables relative to the difficultly-measured parameters by analyzing the specific industrial process flow and combining with expert experience; determining the time scale of each auxiliary variable according to the acting time of each auxiliary variable in the process flow, and constructing a region of interest in each auxiliary variable time sequence;
the specific method comprises the following steps:
let a certain sampling frequency be fiThe total time sequence of some auxiliary variable sample data in the difficult-to-measure parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1))，NiIs the length of the auxiliary variable time series; the approximate time delay range of the auxiliary variable relative to the undetected parameter obtained by expert experience is Tdimin～TdimaxDuration of action Tsimin～TsimaxThen the time range in the region of interest of the auxiliary variable
Figure BDA0002140712450000037
Comprises the following steps:
Figure BDA0002140712450000034
the time series within the secondary variable time region of interest
Figure BDA0002140712450000035
Comprises the following steps:
Figure BDA0002140712450000036
Figure BDA0002140712450000041
the technical scheme of the invention is further improved as follows: in step 3, feature compression is carried out on the original time sequence of each auxiliary variable and the time sequence in the concerned time period, and then a two-dimensional input matrix is formed by each auxiliary variable time sequence after feature compression and is used as the input of a soft measurement model;
the specific treatment method comprises the following steps:
3-1), feature compression process:
(1) and (3) compression process of all time series characteristics of auxiliary variables:
the total time sequence of the auxiliary variable within the refractory parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1)) and the number of values in the time series after feature compression is m and the time series after feature compression is x'i(k)＝(x′i(0),……x′i(m-1))。
Degree of characteristic compression liComprises the following steps:
Figure BDA0002140712450000042
the characteristic compression process formula is as follows:
Figure BDA0002140712450000043
(2) and (3) compression process of time series features in the auxiliary variable attention area:
the time sequence in the region of interest of an auxiliary variable is
Figure BDA0002140712450000044
The number of values in the time series after the feature compression is n, and the time series after the feature compression is
Figure BDA0002140712450000045
Degree of feature compression
Figure BDA0002140712450000046
Comprises the following steps:
Figure BDA0002140712450000047
the characteristic compression process formula is as follows:
Figure BDA0002140712450000051
3-2), constructing an input matrix of the soft measurement model:
(1) the two-dimensional input matrix formed by the whole time series of the auxiliary variables is:
Figure BDA0002140712450000052
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000053
are each composed of the entire time series x'0And x'r-1Transposing the formed vector, wherein m is the number of numerical values contained in each auxiliary variable time sequence after feature compression, and r is the number of auxiliary variables;
(2) the two-dimensional input matrix formed by the time sequence in the auxiliary variable attention area is as follows:
Figure BDA0002140712450000054
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000055
respectively, time series from the region of interest
Figure BDA0002140712450000056
And
Figure BDA0002140712450000057
and (3) transposing the formed vector, wherein n is the number of numerical values of the time sequence in the attention area of each auxiliary variable after feature compression, and r is the number of the auxiliary variables.
The technical scheme of the invention is further improved as follows: in step 4, the soft measurement model is a multi-channel convolution neural network, the number of convolution layers and pooling layers of each channel, the weight w and bias b of each hidden layer and all-connected layer, and the number and size of convolution kernels and pooling kernels can be respectively set according to the input data characteristics of each channel; extracting the characteristics of each row of each channel in a one-dimensional convolution pooling mode, and finally inputting the extracted characteristics of each channel into a full-connection layer after performing characteristic fusion;
the specific method for feature fusion comprises the following steps:
the feature fusion process of the multi-time scale convolutional neural network model based on the attention mechanism on the features of each channel is completed in a full connection layer, and the feature fusion formula is as follows:
Figure BDA0002140712450000061
in the formula, yk-1Is a fully-connected layer after being fused,
Figure BDA0002140712450000062
the full connection layers are respectively a channel 0, a channel i and a channel n, a is the corresponding position of the feature vector of the full connection layer, and MAX () is the maximum value of the obtained feature.
The technical scheme of the invention is further improved as follows: in step 5, the reverse error correction algorithm in the supervised reverse fine tuning reference BP neural network realizes the optimization of the weight w and the bias b layer by layer, and the reverse training in the time sequence convolution neural network is supervised training.
Due to the adoption of the technical scheme, the invention has the following technical effects:
1. the multi-time scale convolutional neural network soft measurement method based on the attention mechanism can well solve the problem of estimating difficultly-measured parameters in real time. The established soft measurement model has good generalization capability, and can not only provide guidance for operators, but also provide prerequisites for the intelligent control of subsequent industrial production.
2. According to the process characteristics of the process industry, the time delay and the time length of each auxiliary variable relative to the difficultly-measured parameters are roughly determined by combining with the expert experience, the calculation amount required by time sequence matching is reduced, and the dilemma that the time delay and the time length are difficult to accurately measure due to strong nonlinearity, strong coupling and time lag among the variables and the intervention of a control system is avoided.
3. The introduction of the attention mechanism in the invention can well consider the important remarkable characteristics contained in the short subsequences containing more information in the long-time sequences of each auxiliary variable. Meanwhile, the soft measurement model can refer to the global features and the local features of each auxiliary variable time sequence.
4. According to the invention, the time sequence is subjected to feature compression, so that the problems that the lengths of the time sequence are inconsistent and the convolutional neural network model input is difficult to construct due to different sampling frequencies among auxiliary variables are effectively solved, and the data redundancy caused by the same data of adjacent sampling points of the same auxiliary variable is also avoided.
5. The feature fusion method can well fuse the features of all channels, the fused features refer to the difference among the features of all channels, and meanwhile, the redundancy among the features of all channels is reduced.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram of a soft measurement scenario after application of the method of the present invention to cement clinker fCao;
FIG. 3 is a model structure diagram of a multivariate multi-time scale time series convolution neural network based on an attention mechanism designed by the present invention;
FIG. 4 is a timing convolution process for a channel;
FIG. 5 is a two-channel feature fusion process;
FIG. 6 is a diagram of the prediction results after the method of the present invention is applied to the training of a clinker fCao soft measurement model in the cement production process.
Detailed Description
The invention is described in further detail below with reference to the following figures and specific embodiments:
the invention discloses a multi-time scale convolutional neural network soft measurement method based on an attention mechanism, and the method is characterized in that fig. 1 is a flow chart of the measurement method.
The method comprises the following steps:
step 1, determining auxiliary variables and processing data
Through the analysis of the industrial process flow, primarily selecting an easily-measured variable related to a difficultly-measured parameter as an auxiliary variable of a soft measurement model and collecting a time sequence of the auxiliary variable and the difficultly-measured parameter;
then data acquisition is carried out, abnormal values of the data are removed by adopting a 3 sigma criterion, and normalization processing is carried out on the data before training; eliminating abnormal values of the difficultly-measured parameters when the abnormal values of the data are processed by adopting a 3 sigma criterion; in the process of eliminating abnormal values of the parameters difficult to measure, the time sequence of the auxiliary variables corresponding to the abnormal values is eliminated;
the method is characterized in that:
setting the sampling sequence of the difficultly-detected parameters at different moments as y (k) ═ y (0), …, y (N)), judging each point y (i) in y (k), if the formula (1) is satisfied, indicating that the point is an abnormal point and needs to be removed, and simultaneously removing each auxiliary variable data corresponding to the point;
Figure BDA0002140712450000081
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000082
is the mean of the variable y (k); σ is the standard deviation of the variable y (k);
carrying out Min-Max standardization treatment on the data of each variable according to the formula (2), converting the data into a non-dimensionalized index mapping evaluation value, enabling the data of each variable to be in the same quantity grade, and carrying out comprehensive evaluation analysis;
Figure BDA0002140712450000083
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000084
normalized time series for variable i, ximinIs the minimum value of variable i, ximaxThe maximum value of the variable i.
Step 2, attention mechanism and attention area selection
A hard attention mechanism is adopted, and an attention area is divided according to the time delay and the effective time scale of each auxiliary variable relative to the difficultly-measured parameter in the process flow;
judging the time delay parameters of the auxiliary variables relative to the difficultly-measured parameters by analyzing the specific industrial process flow and combining with expert experience; determining the time scale of each auxiliary variable according to the acting time of each auxiliary variable in the process flow, and constructing a region of interest in each auxiliary variable time sequence;
the specific method comprises the following steps:
let a certain sampling frequency be fiThe total time sequence of some auxiliary variable sample data in the difficult-to-measure parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1))，NiIs the length of the auxiliary variable time series; the approximate time delay range of the auxiliary variable relative to the difficult parameter is Tdimin～TdimaxDuration of action is Tsimin～TsimaxThen the time range in the region of interest of the auxiliary variable
Figure BDA0002140712450000085
Comprises the following steps:
Figure BDA0002140712450000091
the time series within the secondary variable time region of interest
Figure BDA0002140712450000095
Comprises the following steps:
Figure BDA0002140712450000092
step 3, input for constructing soft measurement model
Forming a matrix by the time sequence of each auxiliary variable, and determining the input of a soft measurement model by combining the attention area of an attention mechanism;
performing feature compression on the original time sequence of each auxiliary variable and the time sequence in the concerned time period, and then forming a two-dimensional input matrix by the time sequence of each auxiliary variable after feature compression to be used as the input of a soft measurement model;
the specific treatment method comprises the following steps:
3-1), feature compression process:
(1) and (3) compression process of all time series characteristics of auxiliary variables:
the total time sequence of the auxiliary variable within the refractory parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1)) and the number of values in the time series after feature compression is m and the time series after feature compression is x'i(k)＝(x′i(0),……x′i(m-1))。
Degree of characteristic compression liComprises the following steps:
Figure BDA0002140712450000093
the characteristic compression process formula is as follows:
Figure BDA0002140712450000094
(2) and (3) compression process of time series features in the auxiliary variable attention area:
the time sequence in the region of interest of an auxiliary variable is
Figure BDA0002140712450000101
The number of values in the time series after the feature compression is n, and the time series after the feature compression is
Figure BDA0002140712450000102
Degree of feature compression
Figure BDA0002140712450000103
Comprises the following steps:
Figure BDA0002140712450000104
the characteristic compression process formula is as follows:
Figure BDA0002140712450000105
3-2), constructing an input matrix of the soft measurement model:
(1) the two-dimensional input matrix formed by the whole time series of the auxiliary variables is:
Figure BDA0002140712450000106
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000107
are each composed of the entire time series x'0And x'r-1Transposing the formed vector, wherein m is the number of numerical values contained in each auxiliary variable time sequence after feature compression, and r is the number of auxiliary variables;
(3) the two-dimensional input matrix formed by the time sequence in the auxiliary variable attention area is as follows:
Figure BDA0002140712450000108
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000109
respectively, time series from the region of interest
Figure BDA00021407124500001010
And
Figure BDA00021407124500001011
and (3) transposing the formed vector, wherein n is the number of numerical values of the time sequence in the attention area of each auxiliary variable after feature compression, and r is the number of the auxiliary variables.
Step 4, establishing a soft measurement model of the time sequence convolution neural network
Determining initial parameters of a time sequence convolution neural network model, and carrying out forward training on the network; the initial parameters comprise the number of the convolution layers and the number of the pooling layers of the time sequence convolution neural network, the learning rate, the weight w and the bias b of each hidden layer, all-connected layer and output layer, and the number and the size of convolution kernels and pooling kernels;
the soft measurement model is a multi-channel convolution neural network, the number of convolution layers and pooling layers of each channel, the weight w and bias b of each hidden layer and all-connected layer, and the number and size of convolution kernels and pooling kernels can be respectively set according to the input data characteristics of each channel; because the data of each variable has the characteristics of time sequence, coupling, time lag and the like, each channel adopts a one-dimensional convolution pooling mode to extract the characteristics of each column, and the extracted characteristics of each channel are input into a full-connection layer after being subjected to characteristic fusion;
the specific method for feature fusion comprises the following steps:
the feature fusion process of the multi-time scale convolutional neural network model based on the attention mechanism on the features of each channel is completed in a full connection layer, and the feature fusion formula is as follows:
Figure BDA0002140712450000111
in the formula, yk-1Is a fully-connected layer after being fused,
Figure BDA0002140712450000112
the full connection layers are respectively a channel 0, a channel i and a channel n, a is the corresponding position of the feature vector of the full connection layer, and MAX () is the maximum value of the obtained feature.
Step 5, training time sequence convolution neural network soft measurement model
And carrying out supervised training by utilizing error reverse fine tuning, and optimizing the weight w and the bias b in the time sequence convolutional neural network by correcting the error. And optimizing the weight w and the bias b layer by using a reverse error correction algorithm in the supervised reverse fine tuning reference BP neural network, wherein the reverse training in the time sequence convolution neural network is supervised training.
And 6, estimating the difficultly-measured parameters in real time by using the time sequence convolution neural network model trained in the step 5.
The invention provides a multi-time scale convolutional neural network soft measurement method based on an attention mechanism, which is used for estimating difficultly-measured parameters in real time. The method incorporates expert experience to introduce an attention mechanism to segment out regions of interest in the time series of input variables. The problem that the lengths of time sequences of all auxiliary variables are not matched due to different sampling frequencies is solved by compressing all the time sequences of the input variables and the time sequences in the attention area. And finally, performing feature fusion on the output of the attention mechanism module and the output of the global module, and retaining the difference information of the channels to remove redundant information. The introduction of multiple time scales can enable the soft measurement model to learn the coarse and fine granularity characteristics of each auxiliary variable time sequence, and the problem of time-varying delay among the variables is solved. In conclusion, the soft measurement method well solves the influence of strong nonlinearity, coupling and large time lag on the prediction result in the industrial process, is directly based on actual data, and has the advantages of strong applicability, low cost, simple algorithm and the like.
The following is the process of real-time estimation of the cement clinker fCaO when the measuring method is actually applied to the actual production of a certain cement plant. FIG. 2 is a soft measurement scenario after application of the method of the present invention to cement clinker fCao.
The general idea is that variable selection is firstly carried out, the related variable of clinker fCaO is obtained according to the cement process analysis, and the time sequence of soft measurement modeling is determined. Determining time delay and duration of each auxiliary variable by combining expert experience, determining an attention area, performing feature compression on all time sequences containing feature information of each variable and time sequences in the attention area, constructing an input matrix as modeling data, inputting the modeling data into a dual-channel soft measurement model, extracting features of each channel in a one-dimensional convolution pooling kernel mode according to the characteristics of the time sequences, performing feature fusion on the extracted features of each channel, and transmitting the feature information obtained by synthesizing all connecting layers and fusing all channels to an output layer. The constructed multi-time scale convolutional neural network soft measurement model based on the attention mechanism is shown in FIG. 3. And finally, carrying out supervised parameter fine adjustment by using an error reverse fine adjustment principle in the BP neural network, and completing the construction of a soft measurement model.
The specific measurement contents and steps are as follows:
step 1: determining auxiliary variables and processing the data
And (3) comprehensively analyzing the cement process, selecting 13 variables related to the clinker fCaO, and taking a data sequence of each variable in a certain time period as the input of a soft measurement model.
It is known from cement technology that cement raw materials are calcined in a rotary kiln at high temperature to form a sintering zone, a sintering reaction is carried out, solid particle materials obtained by cooling are called cement clinker, and a solidified body contains a small amount of uncombined calcium oxide called free calcium (fCaO). Too high a free calcium content will decrease the stability of the cement and too low a free calcium content will increase the energy consumption for cement firing, so that the fCaO needs to be controlled within a reasonable range. In the cement calcination process, all parameters of a burning zone play a crucial role in the content of the clinker fCaO, so that the parameters of a burning system are main factors for realizing soft measurement of the clinker fCaO. The heat source of the burning zone is the coal feeding amount of the decomposing furnace, the coal feeding amount of the kiln head and secondary air recycled into the kiln from the grate cooler, the temperature of the burning zone influences the content of calcium oxide generated in the decomposition process of raw materials, and the generated calcium oxide is converted into other calcium oxideCompound (SiO)2、Al2O3、Fe2O3) Absorption conditions. The high-temperature fan and the kiln head negative pressure generate huge air pressure difference in the kiln, so that the air passage of the cement firing system is ensured to be smooth, and the pressure in the kiln is kept stable. When the rotary kiln rotates, a kiln motor is required to provide power, the uniformity of chemical reaction of materials in the rotary kiln is ensured, and the higher the current of a kiln main machine is, the higher the viscosity of the materials in the kiln is, and the higher the temperature in the kiln is. The grate pressure reflects the thickness of the material on the grate cooler to a certain extent.
From the above analysis, 13 variables closely related to the fcoa content of the cement clinker were selected: the coal feeding amount of the decomposing furnace, the rotating speed of a high-temperature fan, the outlet temperature of the decomposing furnace, the feeding amount, the temperature of a kiln tail, the negative pressure of a kiln head, the temperature of secondary air, the pressure under a two-chamber grate, the kiln current, the coal feeding amount of the kiln head and three values (HM, IM and SM).
And then data acquisition and preprocessing are carried out.
Outliers of the collected data are removed by using a 3 sigma criterion, and the data are normalized before training.
Assuming that the sampling sequence of the cement clinker fCaO at different times in the clinker firing process is y (k) ═ y (0), …, y (n)), each point y (i) in y (k) is judged, if the formula (1) is satisfied, the point is an abnormal point and needs to be removed, and the time sequence of each auxiliary variable corresponding to the point is also required to be removed.
Figure BDA0002140712450000131
In the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000132
is the mean of the variable y (k); σ is the standard deviation of the variable y (k);
since the 13 auxiliary variable data have different dimensions, the evaluation criteria are also different, and in order to unify the evaluation criteria, the data of the 13 auxiliary variables need to be subjected to Min-Max standardization according to the formula (2) to be converted into a dimensionless index mapping evaluation value, so that the comparability between the data is satisfied. When the variable data are in the same quantity level, comprehensive evaluation analysis can be carried out.
Figure BDA0002140712450000141
In the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000142
normalized time series for variable i, ximinIs the minimum value of variable i, ximaxThe maximum value of the variable i.
Step 2: attention mechanism and selection of region of interest
And according to the expert experience and the hard attention mechanism, roughly dividing an attention area according to the time delay and the action duration of each auxiliary variable relative to the difficultly-measured parameter.
Let f be a sampling frequency in the clinker sintering processiHas a total time sequence of x auxiliary variables within the clinker fCaO sampling interval Ti(k)＝(xi(0),…,xi(Ni-1))，NiIs the length of the auxiliary variable time series. The approximate time-delay range of this auxiliary variable with respect to the clinker fCaO, which is derived from the experience of experts in the cement plant, is Tdimin～TdimaxDuration of action is Tsimin～TsimaxThen the time range in the region of interest of the auxiliary variable
Figure BDA0002140712450000143
Comprises the following steps:
Figure BDA0002140712450000144
the time series within the secondary variable time region of interest
Figure BDA0002140712450000145
Comprises the following steps:
Figure BDA0002140712450000146
if the total time sequence of the outlet temperature of the decomposing furnace with the sampling frequency of 12 times/min within 60min of the sampling interval of the cement clinker fCaO is x (k) ═ (x (0), … … and x (719)), the expert experiences that the approximate time delay range of the outlet temperature of the decomposing furnace relative to the cement clinker fCaO is 57-59 min, the effective action time is about 10-20 min, and the time attention area of the auxiliary variable is
Figure BDA0002140712450000151
Comprises the following steps:
Figure BDA0002140712450000152
time series within a region of interest
Figure BDA0002140712450000153
Comprises the following steps:
Figure BDA0002140712450000154
according to this method, the remaining 12 auxiliary variables are processed.
And step 3: input to build a soft measurement model
Because the sampling frequency may be different between the auxiliary variables during the production process, the original time series of the auxiliary variables and the time series in the time period of interest need to be feature-compressed, so that the feature-compressed time series of the auxiliary variables can form a two-dimensional input matrix.
Comprehensively considering the sampling frequency of each auxiliary variable in the cement clinker production process, determining that each auxiliary variable time sequence contains the same number of values after characteristic compression, wherein the number of values in the original time sequence is integral multiple of the number of values in the time sequence after the characteristic compression.
The step mainly comprises two processing procedures, wherein the characteristic compression procedure is referred to as step 3-1 for convenience of description, and the procedure of constructing the input matrix of the soft measurement model is referred to as step 3-2. The following is a detailed description.
Step 3-1, the feature compression process comprises the following two processing processes:
(1) and (3) compression process of all time series characteristics of auxiliary variables:
an auxiliary variable x within a cement clinker fCaO sampling interval TiIs xi(k)＝(xi(0),…,xi(Ni-1)) and the number of values in the time series after feature compression is m and the time series after feature compression is x'i(k)＝(x′i(0),……x′i(m-1))。
Degree of characteristic compression liComprises the following steps:
Figure BDA0002140712450000155
the characteristic compression process formula is as follows:
Figure BDA0002140712450000161
(2) and (3) compression process of time series features in the auxiliary variable attention area:
some auxiliary variable xiThe time sequence in the region of interest is
Figure BDA0002140712450000162
The number of values in the time series after the feature compression is n, and the time series after the feature compression is
Figure BDA0002140712450000163
Degree of feature compression
Figure BDA0002140712450000164
Comprises the following steps:
Figure BDA0002140712450000165
the characteristic compression process formula is as follows:
Figure BDA0002140712450000166
if the total time series of the outlet temperature of the decomposing furnace and the time series in the attention area in step 3 are subjected to feature compression, the number of numerical values in the total time series after the feature compression is 60, and the number of numerical values in the time series in the attention area is also 60, then the time series in the attention area is equivalent to the local amplification in the total time series, and the attention area may contain more information.
The whole time sequence of the outlet temperature of the decomposing furnace after the characteristic compression is as follows:
x′(k)＝(x′(0),……x′(59))
the time sequence in the attention area of the outlet temperature of the decomposing furnace after characteristic compression is as follows:
Figure BDA0002140712450000167
according to this method, the remaining 12 auxiliary variables are processed.
Step 3-2, the process of constructing the input matrix of the soft measurement model is as follows:
because the input requirement of the time series convolutional neural network model is a two-dimensional tensor, i.e., the input is a two-dimensional matrix. All time series of the characteristic compressed cement clinker fCaO auxiliary variable and the time series in the concerned area are required to be constructed into a two-dimensional matrix as the input of the soft measurement model.
(1) The two-dimensional input matrix formed by the whole time series of the auxiliary variables is:
Figure BDA0002140712450000171
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000172
respectively from the total timeSequence x'0And x'12In 60 × 13, 60 is the number of values contained in each auxiliary variable time sequence after feature compression, and 13 is the number of auxiliary variables;
(4) the two-dimensional input matrix formed by the time sequence in the auxiliary variable attention area is as follows:
Figure BDA0002140712450000173
in the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000174
respectively, time series from the region of interest
Figure BDA0002140712450000175
And
Figure BDA0002140712450000176
in the transpose of the constructed vector, 60 is the number of time-series values in the region of interest of each auxiliary variable after feature compression, and 13 is the number of auxiliary variables, in 60 × 13.
And 4, step 4: establishing a soft measurement model of a time sequence convolution neural network
The structure diagram of the multi-time scale convolutional neural network model based on the attention mechanism is shown in fig. 3, the soft measurement model is a dual-channel convolutional neural network, the number of convolutional layers and pooling layers of each channel, the weight w and the offset b of each hidden layer and all-connected layer, and the number and the size of convolutional kernels and pooling kernels can be respectively set according to the input data characteristics of each channel. And each channel adopts a single-dimensional convolution pooling mode to extract the characteristics of each column, the extracted characteristics of each channel are input into a full-connection layer, and meanwhile, the dual-channel characteristics are fused at the full-connection layer.
The time delay among the variables is difficult to determine due to the strong coupling among the variables in the production process of the cement clinker, data of different variables at the same moment may not have relevance, and the influence degrees of the variables on difficultly-measured parameters are different. Convolution pool using single dimensionThe time lag uncertainty problem among the auxiliary variables and between the auxiliary variables and the cement clinker fCaO is avoided, the calculation amount required by time sequence matching is reduced to a great extent, and the loss of characteristic information possibly caused during the time sequence matching is avoided. The convolution pooling for a channel is shown in FIG. 4, where the k-1 layer of the channel is the feature vector obtained after convolution and pooling for multiple times, and is used as the input of the fully-connected layer. Input of the full connection layer
Figure BDA0002140712450000181
And output
Figure BDA0002140712450000182
The relationship between them is as follows.
Figure BDA0002140712450000183
In the formula (I), the compound is shown in the specification,
Figure BDA0002140712450000184
respectively, the weight and the offset of the fully connected layer in the channel i.
Because the extracted information of each channel has difference and redundancy, the extracted information of each channel needs to be subjected to feature fusion, and the difference information is retained to remove the redundancy information. The feature fusion process of the multi-time scale convolutional neural network model based on the attention mechanism on the features of each channel is completed in a full connection layer.
The dual-channel feature fusion mode adopted by the invention is shown in FIG. 5, and the feature fusion formula is as follows:
Figure BDA0002140712450000185
in the formula, yk-1Is a fully-connected layer after being fused,
Figure BDA0002140712450000186
is full connection of a channel 0 and a channel 1 respectivelyNext, a is the feature vector corresponding position of the full connection layer, and MAX () is the maximum value for finding the feature.
In order to avoid overfitting, a regularization method, namely a data loss (Dropout) technology is adopted before the output layer of the network model, so that the purpose of improving the generalization capability of the network model is achieved. As shown in the k-th layer of fig. 5, the output layer of the time-series convolutional neural network directly calculates the value of the undetected parameter by using linear weighted summation. Then the layer enters xkThe calculation formula with the output value y' is:
y'＝wkxk+bk (12)
in the formula wkAnd bkAs are the weights and offsets of the output layer.
And 5: training time sequence convolution neural network soft measurement model
And (4) inputting the input matrix constructed in the step (4) into a soft measurement model. And (3) carrying out supervised training by utilizing an error reverse fine adjustment principle in the BP neural network, and optimizing the weight w and the bias b in the time sequence convolution neural network by correcting the error.
Step 6: and 5, estimating the difficultly-measured parameters in real time by using the trained time sequence convolution neural network model in the step 5.
The prediction result of the method applied to the clinker fCaO soft measurement model in the cement production process after training is shown in FIG. 6.
The above-mentioned embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solution of the present invention by those skilled in the art should fall within the protection scope defined by the claims of the present invention without departing from the spirit of the present invention.

Claims (5)

1. A multi-time scale convolution neural network soft measurement method based on an attention mechanism is characterized in that: the method comprises the following steps:
step 1, determining auxiliary variables and processing data
Through the analysis of the industrial process flow, primarily selecting an easily-measured variable related to a difficultly-measured parameter as an auxiliary variable of a soft measurement model and collecting a time sequence of the auxiliary variable and the difficultly-measured parameter;
then, removing abnormal values in the acquired time sequence by adopting a 3 sigma criterion, and carrying out normalization processing on data before training;
step 2, attention mechanism and attention area selection
A hard attention mechanism is adopted, and an attention area is divided according to the time delay and the effective time scale of each auxiliary variable relative to the difficultly-measured parameter in the process flow;
judging the time delay parameters of the auxiliary variables relative to the difficultly-measured parameters by analyzing the specific industrial process flow and combining with expert experience; determining the time scale of each auxiliary variable according to the acting time of each auxiliary variable in the process flow, and constructing a region of interest in each auxiliary variable time sequence;
the specific method comprises the following steps:
let a certain sampling frequency be fiThe total time sequence of some auxiliary variable sample data in the difficult-to-measure parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1))，NiIs the length of the auxiliary variable time series; the time delay range of the auxiliary variable relative to the difficult parameter is Tdimin～TdimaxDuration of action is Tsimin～TsimaxThen the time range in the region of interest of the auxiliary variable
Figure FDA0002842081700000011
Comprises the following steps:
Figure FDA0002842081700000012
the time series within the secondary variable time region of interest
Figure FDA0002842081700000013
Comprises the following steps:
Figure FDA0002842081700000014
step 3, input for constructing soft measurement model
Forming a matrix by the time sequence of each auxiliary variable, and determining the input of a soft measurement model by combining the attention area of an attention mechanism;
step 4, establishing a soft measurement model of the time sequence convolution neural network
Determining initial parameters of a time sequence convolution neural network model, and carrying out forward training on the network; the initial parameters comprise the number of the convolution layers and the number of the pooling layers of the time sequence convolution neural network, the learning rate, the weight w and the bias b of each hidden layer, all-connected layer and output layer, and the number and the size of convolution kernels and pooling kernels;
step 5, training time sequence convolution neural network soft measurement model
Carrying out supervised training by utilizing error reverse fine tuning, and optimizing weight w and bias b in the time sequence convolutional neural network by correcting errors;
and 6, estimating the difficultly-measured parameters in real time by using the time sequence convolution neural network model trained in the step 5.
2. The attention mechanism-based multi-time scale convolutional neural network soft measurement method of claim 1, wherein: in the step 1, when a 3 sigma criterion is adopted to process data abnormal values, the abnormal values of the parameters which are difficult to be measured are removed; in the process of eliminating abnormal values of the parameters difficult to measure, the time sequence of the auxiliary variables corresponding to the abnormal values is eliminated;
the specific method comprises the following steps:
setting the sampling sequence of the difficultly-detected parameters at different moments as y (k) ═ y (0), …, y (N)), judging each point y (i) in y (k), if the formula (1) is satisfied, indicating that the point is an abnormal point and needs to be removed, and simultaneously removing each auxiliary variable data corresponding to the point;
Figure FDA0002842081700000021
in the formula (I), the compound is shown in the specification,
Figure FDA0002842081700000022
is the mean of the variable y (k); σ is the standard deviation of the variable y (k);
carrying out Min-Max standardization treatment on the data of each variable according to the formula (2), converting the data into a non-dimensionalized index mapping evaluation value, enabling the data of each variable to be in the same quantity grade, and carrying out comprehensive evaluation analysis;
Figure FDA0002842081700000031
in the formula (I), the compound is shown in the specification,
Figure FDA0002842081700000032
normalized time series for variable i, ximinIs a variable xiMinimum value of (1), ximaxIs a variable xiIs measured.
3. The attention mechanism-based multi-time scale convolutional neural network soft measurement method of claim 1, wherein: in step 3, feature compression is carried out on the original time sequence of each auxiliary variable and the time sequence in the concerned time period, and then a two-dimensional input matrix is formed by each auxiliary variable time sequence after feature compression and is used as the input of a soft measurement model;
the specific treatment method comprises the following steps:
3-1), feature compression process:
(1) and (3) compression process of all time series characteristics of auxiliary variables:
the total time sequence of the auxiliary variable within the refractory parameter sampling interval T is xi(k)＝(xi(0),…,xi(Ni-1)) and the number of values in the time series after feature compression is m and the time series after feature compression is x'i(k)＝(x′i(0),……x′i(m-1))；
Degree of characteristic compression liComprises the following steps:
Figure FDA0002842081700000033
the characteristic compression process formula is as follows:
Figure FDA0002842081700000034
(2) and (3) compression process of time series features in the auxiliary variable attention area:
the time sequence in the region of interest of an auxiliary variable is
Figure FDA0002842081700000035
The number of values in the time series after the feature compression is n, and the time series after the feature compression is
Figure FDA0002842081700000041
Degree of feature compression
Figure FDA0002842081700000042
Comprises the following steps:
Figure FDA0002842081700000043
the characteristic compression process formula is as follows:
Figure FDA0002842081700000044
3-2), constructing an input matrix of the soft measurement model:
(1) the two-dimensional input matrix formed by the whole time series of the auxiliary variables is:
Figure FDA0002842081700000045
in the formula (I), the compound is shown in the specification,
Figure FDA0002842081700000046
are each composed of the entire time series x'0And x'r-1Transposing the formed vector, wherein m is the number of numerical values contained in each auxiliary variable time sequence after feature compression, and r is the number of auxiliary variables;
(2) the two-dimensional input matrix formed by the time sequence in the auxiliary variable attention area is as follows:
Figure FDA0002842081700000047
in the formula (I), the compound is shown in the specification,
Figure FDA0002842081700000048
respectively, time series from the region of interest
Figure FDA0002842081700000049
And
Figure FDA00028420817000000410
and (3) transposing the formed vector, wherein n is the number of numerical values of the time sequence in the attention area of each auxiliary variable after feature compression, and r is the number of the auxiliary variables.
4. The attention mechanism-based multi-time scale convolutional neural network soft measurement method of claim 1, wherein: in step 4, the soft measurement model is a multi-channel convolution neural network, the number of convolution layers and pooling layers of each channel, the weight w and bias b of each hidden layer and all-connected layer, and the number and size of convolution kernels and pooling kernels can be respectively set according to the input data characteristics of each channel; extracting the characteristics of each row of each channel in a one-dimensional convolution pooling mode, and finally inputting the extracted characteristics of each channel into a full-connection layer after performing characteristic fusion;
the specific method for feature fusion comprises the following steps:
the feature fusion process of the multi-time scale convolutional neural network model based on the attention mechanism on the features of each channel is completed in a full connection layer, and the feature fusion formula is as follows:
Figure FDA0002842081700000051
in the formula, yk-1Is a fully-connected layer after being fused,
Figure FDA0002842081700000052
the full connection layers are respectively a channel 0, a channel i and a channel n, a is the corresponding position of the feature vector of the full connection layer, and MAX () is the maximum value of the obtained feature.
5. The attention mechanism-based multi-time scale convolutional neural network soft measurement method of claim 1, wherein: in step 5, the reverse error correction algorithm in the supervised reverse fine tuning reference BP neural network realizes the optimization of the weight w and the bias b layer by layer, and the reverse training in the time sequence convolution neural network is supervised training.
CN201910667918.4A 2019-07-23 2019-07-23 Multi-time scale convolution neural network soft measurement method based on attention mechanism Active CN110378044B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910667918.4A CN110378044B (en) 2019-07-23 2019-07-23 Multi-time scale convolution neural network soft measurement method based on attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910667918.4A CN110378044B (en) 2019-07-23 2019-07-23 Multi-time scale convolution neural network soft measurement method based on attention mechanism

Publications (2)

Publication Number Publication Date
CN110378044A CN110378044A (en) 2019-10-25
CN110378044B true CN110378044B (en) 2021-06-11

Family

ID=68255212

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910667918.4A Active CN110378044B (en) 2019-07-23 2019-07-23 Multi-time scale convolution neural network soft measurement method based on attention mechanism

Country Status (1)

Country Link
CN (1) CN110378044B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110807428B (en) * 2019-11-05 2023-05-30 南方科技大学 Coal sample identification method, device, server and storage medium
CN111222798B (en) * 2020-01-13 2023-04-07 湖南师范大学 Complex industrial process key index soft measurement method
CN111242068B (en) * 2020-01-17 2021-03-02 科大讯飞(苏州)科技有限公司 Behavior recognition method and device based on video, electronic equipment and storage medium
CN111506835B (en) * 2020-04-17 2022-12-23 北京理工大学 Data feature extraction method fusing user time features and individual features
US11681914B2 (en) 2020-05-08 2023-06-20 International Business Machines Corporation Determining multivariate time series data dependencies
CN112215351B (en) * 2020-09-21 2022-05-03 浙江大学 Enhanced multi-scale convolution neural network soft measurement method
CN112270996B (en) * 2020-11-13 2023-04-25 南京信息工程大学 Classification method for multi-variable medical sensing data flow
CN112488392B (en) * 2020-12-01 2022-10-21 重庆邮电大学 Intelligent water affair daily water consumption prediction method based on machine learning
CN112686898B (en) * 2021-03-15 2021-08-13 四川大学 Automatic radiotherapy target area segmentation method based on self-supervision learning
CN113850931B (en) * 2021-11-29 2022-02-15 武汉大学 Flight feature extraction method for flight abnormity
CN114994294B (en) * 2022-05-20 2023-04-18 燕山大学 Soft measurement method for free calcium of cement clinker based on attention and window gating mechanism

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107015541A (en) * 2017-04-26 2017-08-04 燕山大学 The flexible measurement method being combined based on mutual information and least square method supporting vector machine
CN108197743A (en) * 2017-12-31 2018-06-22 北京化工大学 A kind of prediction model flexible measurement method based on deep learning
CN108845072A (en) * 2018-07-06 2018-11-20 南京邮电大学 A kind of dynamic soft-measuring method of the 4-CBA content based on convolutional neural networks
CN109147878A (en) * 2018-10-08 2019-01-04 燕山大学 A kind of clinker free calcium flexible measurement method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160337702A1 (en) * 2015-05-12 2016-11-17 Rovi Guides, Inc. Methods and systems for recommending supplemental media assets based on recently mentioned media references
CN109919188A (en) * 2019-01-29 2019-06-21 华南理工大学 Timing classification method based on sparse local attention mechanism and convolution echo state network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107015541A (en) * 2017-04-26 2017-08-04 燕山大学 The flexible measurement method being combined based on mutual information and least square method supporting vector machine
CN108197743A (en) * 2017-12-31 2018-06-22 北京化工大学 A kind of prediction model flexible measurement method based on deep learning
CN108845072A (en) * 2018-07-06 2018-11-20 南京邮电大学 A kind of dynamic soft-measuring method of the 4-CBA content based on convolutional neural networks
CN109147878A (en) * 2018-10-08 2019-01-04 燕山大学 A kind of clinker free calcium flexible measurement method

Also Published As

Publication number Publication date
CN110378044A (en) 2019-10-25

Similar Documents

Publication Publication Date Title
CN110378044B (en) Multi-time scale convolution neural network soft measurement method based on attention mechanism
CN109147878B (en) Soft measurement method for free calcium of cement clinker
Zhao et al. Online cement clinker quality monitoring: A soft sensor model based on multivariate time series analysis and CNN
Zhang et al. Local parameter optimization of LSSVM for industrial soft sensing with big data and cloud implementation
CN103336107B (en) Soft measurement method for f-CaO content of cement clinker
CN109508818B (en) Online NOx prediction method based on LSSVM
CN106202946A (en) Clinker free calcium levels Forecasting Methodology based on degree of depth belief network model
CN110444257B (en) Cement free calcium soft measurement method based on unsupervised and supervised learning
CN110245380A (en) Soft instrument training and sample compensation process
CN102601881B (en) Method for monitoring on-line quality and updating prediction model of rubber hardness
CN110322014A (en) A kind of finished cement specific surface area prediction technique based on BP neural network
CN109342703B (en) Method and system for measuring content of free calcium in cement clinker
CN109685283B (en) Method for predicting submerged arc furnace working condition based on confidence rule base reasoning
CN113177358A (en) Soft measurement method for cement quality based on fuzzy fine-grained feature extraction
CN111833970A (en) Construction method and application of cement clinker quality characterization parameter prediction model
CN103279030B (en) Dynamic soft measuring modeling method and device based on Bayesian frame
CN101446828A (en) Nonlinear process quality prediction method
Hao et al. R-WGAN-based multitimescale enhancement method for predicting f-CaO cement clinker
CN117076936A (en) Time sequence data anomaly detection method based on multi-head attention model
CN110763830B (en) Method for predicting content of free calcium oxide in cement clinker
CN112365935A (en) Cement free calcium soft measurement method based on multi-scale depth network
Pani et al. Neural network soft sensor application in cement industry: Prediction of clinker quality parameters
CN116757078A (en) Method and system for measuring flow velocity of pulverized coal based on acting force
Zheng et al. Just-in-time learning for cement free lime prediction with empirical mode decomposition and database monitoring index
CN111178627B (en) Neural network hybrid optimization prediction method based on SPCA

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant