CN109214592B - Multi-model-fused deep learning air quality prediction method - Google Patents
Multi-model-fused deep learning air quality prediction method Download PDFInfo
- Publication number
- CN109214592B CN109214592B CN201811210072.3A CN201811210072A CN109214592B CN 109214592 B CN109214592 B CN 109214592B CN 201811210072 A CN201811210072 A CN 201811210072A CN 109214592 B CN109214592 B CN 109214592B
- Authority
- CN
- China
- Prior art keywords
- data
- air quality
- model
- prediction
- factor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000013135 deep learning Methods 0.000 title claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims abstract description 8
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000013136 deep learning model Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000010606 normalization Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 31
- 238000013528 artificial neural network Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 15
- 230000007246 mechanism Effects 0.000 claims description 12
- 230000007787 long-term memory Effects 0.000 claims description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 6
- 230000001537 neural effect Effects 0.000 description 6
- 230000000306 recurrent effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 238000003915 air pollution Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000006872 improvement Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 239000002245 particle Substances 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000006403 short-term memory Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 239000000809 air pollutant Substances 0.000 description 2
- 231100001243 air pollutant Toxicity 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910052664 nepheline Inorganic materials 0.000 description 1
- 239000010434 nepheline Substances 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000012887 quadratic function Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- General Health & Medical Sciences (AREA)
- General Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Entrepreneurship & Innovation (AREA)
- Biomedical Technology (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Educational Administration (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an air quality prediction method for deep learning of multi-model fusion, which comprises the following steps of firstly, acquiring historical air quality data and meteorological data; secondly, missing value interpolation and normalization processing are carried out on historical air quality data; thirdly, a deep learning model based on seq2seq is constructed by using historical air quality data and serves as a single-factor prediction model; fourthly, constructing a seq2seq deep learning model based on a double-attention machine system by utilizing historical air quality data and meteorological data to serve as a multi-factor prediction model; and fifthly, fusing the single-factor prediction model prediction result, the multi-factor prediction model prediction result and the current meteorological data of the air quality data into an xgboost tree for regression calculation to obtain a final predicted value of the air quality data.
Description
Technical Field
The invention belongs to the crossing field of computer disciplines and environmental disciplines, and particularly relates to an air quality prediction method for deep learning with multi-model fusion.
Background
Due to the fact that social progress and rapid industrial development in recent years lead to a large number of serious environmental pollution problems, especially air pollution is always intensified, the quantity of inhalable particles in the air is increased, and the quality of the air which people live increasingly is gradually reduced. Among the inhalable particles in the air, the particles PM2.5 and PM10 are the most serious, which not only have great influence on the air quality, but also cause great harm to human bodies. In order to reduce the harm of air pollutants to human bodies, the method has important practical significance for analyzing and predicting the air quality.
In response to this problem of air quality prediction, a number of researchers have made a great deal of contribution in recent years. They have combined the latest ideas and new technologies to carry out quantitative research, enabling to find out the causes of air pollution and the fundamental trends of air quality. Where air quality is affected by many factors and has a lot of uncertainty and complexity. Conventional artificial neural networks, Back Propagation (BP) neural networks and regression prediction methods are not accurate enough to predict, and it is difficult to learn internal rules.
Wangxin et al uses LSTM (long short term memory network) recurrent neural network to predict the failure time sequence; machine learning is used by dalikey and the like to dynamically forecast the short-term concentration of PM 2.5; nepheline et al use a genetic algorithm and a Back Propagation (BP) neural network for air quality prediction; establishing an air pollution space-time forecasting model by using a Recurrent Neural Network (RNN) in Vancouxiang and the like; zhengyi and the like establish a PM2.5 prediction model of a deep belief network; zhangtian et al used a BP neural network for air quality prediction.
When the monitoring station acquires the air quality data, the air quality data generally has more missing values due to the problems of equipment failure or network delay blocking and the like. In the conventional data preprocessing, especially missing value processing, most methods for filling missing values are methods such as deletion, averaging, proximity and the like, the filling precision is poor, and the air quality data has time sequence information, so that the precision of sequence data during training is poor. The prediction model uses a traditional single model in selection, problems of overfitting or poor precision and the like can occur, and compared with machine learning models such as an artificial neural network and a traditional support vector machine, the prediction model based on deep learning can be better in the aspects of data capturing characteristics and prediction precision. The conventional machine learning method is difficult to extract the time sequence information from the sequence data aiming at the time sequence information in the sequence data, and the technology of deep learning can be used for extracting the time sequence characteristics, so that the prediction accuracy is improved.
Disclosure of Invention
In view of this, the invention provides a multi-model fusion deep learning air quality prediction method, which can improve the prediction accuracy of air quality data.
The technical scheme for realizing the invention is as follows:
a multi-model fusion deep learning air quality prediction method comprises the following steps:
acquiring historical air quality data and meteorological data;
step two, carrying out missing value interpolation and normalization processing on historical air quality data;
thirdly, constructing a deep learning model based on seq2seq (Sequence to Sequence) as a single-factor prediction model by using historical air quality data;
step four, constructing a seq2seq deep learning model based on a double attention (attention) mechanism by utilizing historical air quality data and meteorological data to serve as a multi-factor prediction model;
inputting the historical air quality data into a single-factor prediction model to obtain a prediction result of the single-factor prediction model, inputting the historical air quality data and the meteorological data into a multi-factor prediction model to obtain a prediction result of the multi-factor prediction model, and fusing the prediction result of the single-factor prediction model, the prediction result of the multi-factor prediction model and the current meteorological data into an xgboost tree to perform regression calculation to obtain a final prediction value of the air quality data.
Further, the air quality data includes PM2.5, PM10, NO2、CO、O3And SO2Concentration data of one or more than two types.
Further, when the air quality data is more than two types of concentration data, in the fifth step, one type of the air quality data is predicted in sequence, and the prediction result of the single-factor prediction model and the prediction result of the multi-factor prediction model of the air quality data and the current meteorological data are merged into the xgboost tree to perform regression calculation to obtain the prediction value of the air quality data.
Further, meteorological data includes temperature, barometric pressure, humidity, wind direction, wind speed, and weather indicators.
Further, the missing value interpolation is performed by using an expectation maximization method.
Furthermore, an input layer of the single-factor prediction model is historical air quality data, a hidden layer is of a seq2seq model, an encoding and decoding structure is arranged inside the hidden layer, an encoding and decoding unit is an LSTM long-time memory neural network, and an output layer is an air quality prediction value.
Furthermore, the input layer of the multi-factor prediction model is historical air quality data and meteorological data, the hidden layer structure is a seq2seq model of a double-attention-machine system, an encoding-decoding (encoder-decoder) structure is arranged inside the hidden layer, a layer of attention machine system is added before encoding (encoder) and decoding (decoder), and the output layer is an air quality data prediction value.
Has the advantages that:
1. according to the method, on the construction of a single-factor prediction model and a multi-factor prediction model, relevant factors influencing air quality are analyzed, and then important features are extracted and used as the input of the models.
2. In the invention, a seq2seq model of a double-attention mechanism is used for selecting the model, the cyclic neural network (RNN) is expanded, the sequence data can be well processed, and the future trend can be predicted by the self-learning rule of the neural network.
3. Aiming at the complexity of air quality prediction, the method integrates the prediction results of multiple models, and cooperates with each other to solve the optimal solution, so that the optimal predicted value is obtained, and the accurate prediction of the air quality is completed.
4. The method combines the special self attribute of the air quality, uses the EM maximum expectation algorithm to fill the missing value of the air quality data, and can well fit the distribution of the data.
Drawings
FIG. 1 is a schematic diagram of the model of the present invention.
FIG. 2 is a diagram of a single-factor prediction model according to the present invention.
FIG. 3 is a diagram of a multi-factor predictive model of the present invention.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The invention needs to model the air data, and the selection of the characteristic data before modeling is very important, especially the air quality data, which relates to the influence on the space and the time, such as the daily working and working hours, the weekend of each week, the days before and after each month, and each season (the air pollution degree is different in summer and winter). Meanwhile, the spatial influence, such as the influence of weather factors, the influence of wind direction and the influence of humidity of surrounding areas, can be received by each monitoring station. Therefore, models are respectively established in the time dimension and the space dimension, firstly, a single-factor model is established in the time dimension, a single factor (such as PM2.5) is selected as the input of the single-factor model, the trend in time is found out, and a local rule is obtained; then establishing a multi-factor model in the spatial dimension, and selecting the multi-factor (surrounding area weather conditions and the like) as the input of the multi-factor model to obtain a global rule; and finally, using a fusion model to take the output of the single-factor model and the multi-factor model as input, and calculating the weight by combining the weather condition of the current time to obtain the optimal predicted value. As shown in fig. 1, the specific steps are as follows:
step one, acquiring historical weather data comprising PM2.5, PM10 and NO per hour2，CO，O3，SO2Temperature, air pressure, humidity, wind direction, wind speed, weather index, wherein, the temperature unit: DEG C, air pressure unit: hectopa (hPa), humidity unit: percent (%), wind direction unit: north begins clockwise angle definition, wind speed unit: m/s, weather index: sunny days, rains, cloudy, etc.
Step two, preprocessing historical data
2.1, because the air quality data are obtained through outdoor sensors, in special or artificial situations, the sensors cannot acquire data accurately or the sensors stop collecting data, and the air quality data have missing values. Therefore, when air quality prediction is performed, missing value interpolation or completion needs to be performed on historical data.
The missing value completion is performed using the Expectation Maximization (EM) method. The EM algorithm is used for estimating the maximum likelihood and the maximum posterior probability of a probability parameter model containing hidden variables, wherein air quality data are used as the hidden variables, maximization is performed according to a likelihood formula to obtain a parameter expression of the model, then iteration is performed on real data to solve parameters, and finally a missing value is calculated.
2.2, normalizing the historical data
Wherein X is the original data, XminRepresenting the minimum value, X, in the raw datamaxThe maximum value in the raw data is indicated.
And 2.3, dividing the processed historical data into 80% training sets and 20% testing sets according to the proportion.
The invention uses the maximum expectation EM algorithm to carry out interpolation processing on the missing value, and the EM algorithm has better fitting capability on time sequence data and is very close to the distribution of original data.
Step three, constructing a single-factor prediction model: the model structure is divided into an input layer, a hidden layer and an output layer, wherein the data of the input layer is PM2.5, PM10 and NO2、CO、O3，SO2Six concentration values are used as training data, and the dimension of an input layer is K multiplied by 6. The output layers are PM2.5, PM10 and NO2、CO、O3，SO2Predicted value of concentration, output layer dimension is 6 × T.
Wherein K represents the length of the selected time series, i.e. K time frames of air quality data (PM2.5, PM10, NO)2、CO、O3，SO2) For example, K is 72h (hours). T denotes air quality data at a future time, that is, air quality data at a predicted future time T, for example, T48 h (hours).
Hidden layer structure of single factor model: the hidden layer adopts a Sequence to Sequence (seq) model and uses a coding-decoding (encoder-decoder) structure, wherein the coding encoder structure is a multi-layer Long and Short-Term Memory (LSTM) neural unit, and after coding (encoder), a context vector is calculated to express important characteristics of input data. The decoding (decoder) part is mainly used for decoding and is structured as a multi-layered LSTM unit.
Step four, constructing a multi-factor prediction model: the model structure is divided into an input layer, a hidden layer and an output layer, wherein the data of the input layer is the whole historical weather data, including PM2.5, PM10 and NO2、CO、O3、SO2Temperature, pressure, humidity, windThe dimension of the input layer is K × 12 for 12 data such as direction, wind speed, weather index, etc., where K represents the length of the selected time series, for example, K is 72h (hours). The output layer is an air quality data prediction value with the dimension of 6 multiplied by T, wherein T represents the air quality data at the future moment and comprises PM2.5, PM10 and NO2、CO、O3、SO2Six concentration data, e.g., T ═ 48h (hours).
Hidden layer structure of multi-factor model: the model is a Sequence-to-Sequence (seq 2 seq) model of a double-attention mechanism, an encoding-decoding (encoder-decoder) structure is arranged in the model, a layer of attention (attention) mechanism is added before encoding to acquire context information before encoding, the encoding encoder part is a multi-layer LSTM (Long Short-Term Memory) neural unit, and after encoding (encoder), a context vector is calculated to express important characteristics of input data. A layer of attention (attention) mechanism is also added at this point prior to decoding. The decoding (decoder) part is mainly used to output the final result state, and is structured as multiple layers of LSTM units.
Step five, fusing the prediction model: sequentially training PM2.5, PM10 and NO by using xgboost (eXtreme Gradient boosting) lifting tree as a training model2、CO、O3And SO2One of the six air mass concentration data is input into the prediction result of the single-factor prediction model and the multi-factor model prediction of a certain air mass concentration data during each training, the weather conditions at the latest moment including temperature, air pressure, humidity, wind direction, wind speed, weather indexes and the like are also input, and the corresponding labeled data during the training are the current air mass data (PM2.5, PM10, NO)2、CO、O3、SO2) For example, when the input data during training is eight data features such as PM2.5 (single-factor model result), PM2.5 (multi-factor model result), temperature, air pressure, humidity, wind direction, wind speed, weather index (six current weather conditions), etc., the labeled data is the current PM2.5 concentration data, and so on, the remaining five types (PM10, NO) can be obtained2、CO、O3、SO2) Input data and label number ofAnd inputting the input data and the marking data of each type into the xgboost tree. The xgboost model establishes n regression subtrees in the training process, namely n tree nodes, each tree node has a plurality of leaf nodes, wherein the leaf nodes are the predicted values of the single-factor and multi-factor prediction models and eight input characteristic values of meteorological data at the latest moment, the corresponding leaf nodes of each tree node are determined according to the weights of the eight input characteristic values, the node splitting principle is to select the characteristic with the maximum information gain as a splitting point, so that the leaf nodes of each subtree have importance (weight value), each round of training adjusts the weight values of the leaf nodes according to a target function to form a new subtree, the optimal subtree is reached after multiple iterations, each subtree node is a predicted value, and all the predicted values of the subtree nodes are accumulated to obtain the air quality predicted value. After input data and marking data are input into the xgboost training, six air quality predicted values can be obtained respectively, wherein the six air quality predicted values comprise PM2.5, PM10 and NO2、CO、O3And SO2Six kinds of concentration data.
Prediction result evaluation
In order to evaluate and analyze the final prediction result, the obtained prediction model is detected and evaluated by using test set data, a Root Mean Square Error (RMSE) is used as an evaluation index, and a formula of the root mean square error is shown as a formula:
wherein,
expressed is the predicted value, yiThe original values are represented and n represents the total number of original values.
Meanwhile, in order to prevent the singularity of the evaluation index, R-square is used as the evaluation index of regression analysis, wherein the formula of the R-square is shown as the formula:
wherein,
expressed is the predicted value, yiIt is shown that the original value is,
the average values are shown and n represents the total number of original values.
Example 1
In order to better explain the technical scheme of the invention, the invention selects the data of 36 air quality monitoring stations and the data of 18 meteorological stations in Beijing City and obtains sufficient sample data, and the specific implementation steps of the invention are as follows:
step 1: obtaining data
The air quality data used by the method is the hourly data collected from 2017, 1 month to 2018, 1 month in Beijing, wherein the air quality data mainly comprises the following important air pollutants: PM2.5(μ g/m3), PM10(μ g/m3), NO2(μg/m3)，CO(mg/m3)，O3(. mu.g/m 3) and SO2(μ g/m 3). And weather and meteorological data which mainly comprise weather, temperature, air pressure, humidity, wind speed and wind direction. Wherein the weather mainly comprises sunny days, snowy days, cloudy days, light rain, heavy rain, sand raising and the like; the temperature is a temperature value monitored by a meteorological station, and the unit is centigrade; barometric pressure refers to atmospheric pressure in units of hectopascal (hPa); humidity refers to the amount of water vapor in the air, in percent (%); wind speed refers to the speed of wind monitored by a weather station, and is measured in meters per second (m/s); wind direction refers to the origin of the wind, for example, north wind refers to wind blowing from north to south, and wind direction is defined by an angle clockwise from north. For example, the wind direction is 180 degrees from the south, and 90 degrees from the east.
Step 2: data pre-processing
For historical air quality data, problems of monitoring instruments are inevitable, andand finally, the data is lost, interpolation processing of the lost value is carried out before the data is input into the prediction model, and the interpolation method adopted by the invention is a maximum Expectation (EM) method, which comprises the steps of E, obtaining maximum likelihood estimation on the sample and M, obtaining the maximum likelihood result and maximizing the maximum likelihood result. The given training sample is { x }1，…，xmIndependent samples, the goal is to find the implicit class z of each sample, maximizing the maximum likelihood function p (x, z).
And E, calculating the log-likelihood function expectation, and performing maximum likelihood estimation on the hidden variable:
Qi(z(i)):＝p(z(i)|x(i)；θ)
wherein Q isiGiven x(i)With respect to z by the parameter theta(i)Solves the problem of how to select Q (z) to make the likelihood function equal to its lower bound, namely, let Qi(z(i)) Is z(i)The posterior probability of (2).
And for the M step, the expected result of the likelihood function of the E step is maximized, so that a new expected value is obtained, and the latest expected value can be applied to the missing value to fill the missing value. In step E, Q is selectediMaximum lower bound of log-likelihood function:
and E and M are continuously and circularly repeated until convergence.
The data after interpolation is normalized by a min-max normalization method, and the normalization can well reduce the data within a certain range, and can carry out non-dimensionalization and accelerate the convergence rate of model solution. The invention reduces the data between [0, 1], and the normalized formula is as follows:
wherein X is originalWeather data, XminRepresenting the minimum value, X, in the raw weather datamaxThe maximum value in the raw weather data is indicated.
Dividing the preprocessed data into a training data set and a testing data set according to the proportion of 8: 2.
and step 3: constructing a single factor prediction model
The input data of the single-factor prediction model are PM2.5, PM10 and NO2、CO、O3And SO2Six concentrations are used as training data, the dimension of an input layer is K multiplied by 6, and K represents the length of the selected time sequence. .
The single-factor model adopts a seq2seq model, the internal structure of the model is encoding-decoding (encoder-decoder), and a given sequence x is { x ═ x1,x2,…,xTThe formula of the coding part is specifically as follows:
ht＝f(ht-1,xt)
wherein t is the current time, htFor the hidden state at time t, f is the LSTM encoder.
After the encoder finishes encoding, all htThe vector c is combined and used as the input of a decoder, and the formula of the decoding part is specifically as follows:
h(t)＝f(h(t-1),y(t-1),c)
p(yt|y(t-1),y(t-2),...y(1),c)＝g(h(t),y(t-1),c)
wherein h is(t)And c is the output of the encoder, c is the last state in the encoder, and f and g are nonlinear activation functions and are set as LSTM neural network units.
The model diagram is shown in fig. 2, in which the encoding (encoder) part is composed of 2 layers of Recurrent Neural Networks (RNN), and each layer is provided with 64 LSTM neural units. And a decoding (decoder) part is composed of 2 layers of Recurrent Neural Networks (RNN), each layer also provided with 64 LSTM neural units.
The training parameter settings for the model are shown in table 1:
TABLE 1 model training parameters
And 4, step 4: constructing a multi-factor prediction model
The input to the multi-factor predictive model is the entire historical weather data, including PM2.5, PM10, NO2、CO、O3、SO2Temperature, air pressure, humidity, wind direction, wind speed, weather index and other 12 kinds of data, and the dimension of the input layer is Kx 12, wherein K represents the length of the selected time sequence.
The multi-factor model adopts a seq2seq model with a double attention machine mechanism, is an improvement on the seq2seq model, can better learn the influence of other meteorological factors on the air quality after improvement, and can capture key information, thereby better learning the time sequence relation among data and having stronger robustness.
Its encoder part, given K sequence data
Constructing a feedforward neural network and adding an attention mechanism, wherein the specific formula is as follows:
Use of softmax function to ensure all
The sum of the weights of (a) is equal to 1,
wherein,
attention weights of k sequences at t moment, and with the attention weights, we can adaptively extract
Followed by encoding using a multi-layer LSTM neural network
Where f is the LSTM neural network element,
is a new sequence with attention weight, htIs a hidden state at time t.
For the state h obtained after the encoder has performed codingtAn attention mechanism is added as follows:
use of softmax function to ensure all
The sum of the weights of (a) is equal to 1, as follows:
wherein
Is the weight of the encoder output state at time i.
Attention weight and encoder hidden state h1,h2,…hTThe weighted sum of } as the context vector ct：
Its decoder part, using the multi-layered LSTM as decoder, can match the weights and context vectors c to a given target sequence (y) once they are obtained1,y2,…,yT-1) Combining:
wherein
And
new in relation to the size of the decoder input
I.e., the new decoder state, the state value is input into the decoder as follows:
wherein g is an LSTM neural network element, dtThe final output state, as part of the decoder.
The model diagram is shown in fig. 3, in which an attention mechanism (attention) is added before the coding (encoder), and the coding part is composed of 2 layers of Recurrent Neural Networks (RNN), and each layer is provided with 64 LSTM neural units. An attention mechanism (attention) is added before decoding, and a decoding (decoder) part consists of 2 layers of Recurrent Neural Networks (RNN), and 64 layers are also arranged in each layer
LSTM neural units.
And 5: integrated predictive model
Using an xgboost (extreme Gradient boosting) lifting tree to carry out regression prediction, wherein the input of the lifting tree is the result of a single-factor model, the result of a multi-factor model and the meteorological condition at the current moment, and specifically, the result of the single-factor model prediction is p ═ { p { (p) }1,p2…p6The result of the multifactor model is z ═ z { (z)1,z2…z6P and z represent PM2.5, PM10, NO2，CO，O3And SO2Six kinds of concentration data, the meteorological condition g ═ { g at the current moment1，g2，…，g6G represents six data of temperature, air pressure, humidity, wind direction, wind speed and weather index at the latest moment, and during training or prediction, the data input in the xgboost tree is xi＝{pi,ziG, i is any one of 1 to 6, y is during trainingiIs labeled current air quality data including PM2.5, PM10, NO2，CO，O3，SO2And y is measured or predictediI.e. predicted air quality data, e.g. x1(ii) pm2.5 (single factor model result), pm2.5 (multi-factor model result), temperature, barometric pressure, humidity, wind direction, wind speed, weather indicator (weather conditions at six most recent times) }, y1Pm2.5 (current value in training set) }, xiAnd yiAnd so on. The xgboost tree is trained by adding the current meteorological condition into the input data, and the weight of the air quality data is flexibly adjusted, so that an accurate predicted value is obtained.
The method for regression calculation of Xgboost is as follows:
if there are k trees in xgboost,
in the above formula F represents all the functional space in the regression forest,
as model predicted value, fkThe predicted value of the ith sample in the kth tree is shown.
The parameters now required are the structure of each tree and the weight of each leaf, and simply the calculation of the sub-trees fkWherein, it is provided with:
Θ＝{f1,f2,...fk}
however, the objective function obj () is to find a good parameter Θ, and the formula is as follows:
obj(Θ)＝L(Θ)+Ω(Θ)
wherein, L (Θ) is a fitting condition of an error function used for expressing data, and Ω (Θ) is a regularization term used for penalizing a complex model.
The step of training the model is that each tree is accumulated until k trees stop, the process is as follows:
wherein,
representsAfter the i-th cycle, xiT represents the model training of the t-th round.
The error in training is as follows:
wherein y isiI.e. the corresponding annotation data,
the time-predicted data, l, represents a loss function, and loss calculation is carried out, wherein square loss and Logistic loss are common.
The penalty term during training is as follows:
finally, the formula for simplifying the target function by using the Taylor formula is obtained as follows:
therein is provided with
C is a constant value, and C is a constant value,
ft(x)＝wq(x),w∈RT,q:Rd→ {1,2, …, T }, f (x) denotes the node prediction value of each subtree, w denotes the weight of leaves, q denotes the structure of the tree, T denotes the number of leaves of the tree, RdRepresenting a data set with a number of features d.
Further, the method is converted into the minimum value of the quadratic function of the above formula, namely:
wherein the formula for calculating the information Gain (Gain) is as follows:
wherein L represents the left sub-tree, R represents the right sub-tree, HLRepresents the left sub-tree weight, HRRepresenting the right subtree weight and gamma representing the complexity cost introduced by adding a new leaf node.
Thus trained, the xgboost tree is split by information Gain (Gain) to create each sub-tree ft(x)＝wq(x)Each data feature, i.e. xi＝{pi，zi，g1G2, g3, g4, g5, g6} includes air quality data p derived from one-factor and multi-factor modelsiAnd ziMapping current meteorological data g to leaf nodes of each tree, adjusting the leaf nodes of each subtree according to weight w according to multi-round training, updating all subtrees at the moment, wherein each subtree node is a predicted value, adjusting the weight according to an objective function obj to obtain an optimal value w, and finally mapping all subtrees fkAnd accumulating the predicted values to obtain the finally predicted air quality data.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. A multi-model fusion deep learning air quality prediction method is characterized by comprising the following steps:
acquiring historical air quality data and meteorological data;
step two, carrying out missing value interpolation and normalization processing on historical air quality data;
thirdly, establishing a seq2 seq-based deep learning model as a single-factor prediction model by using historical air quality data;
step four, establishing a seq2seq deep learning model based on a double-attention machine system as a multi-factor prediction model by utilizing historical air quality data and meteorological data;
inputting historical air quality data into a single-factor prediction model to obtain a prediction result of the single-factor prediction model, inputting the historical air quality data and meteorological data into a multi-factor prediction model to obtain a prediction result of the multi-factor prediction model, and fusing the prediction result of the single-factor prediction model, the prediction result of the multi-factor prediction model and the current meteorological data into an xgboost tree to perform regression calculation to obtain a final prediction value of the air quality data;
when the air quality data is more than two types of concentration data, one type of the air quality data is predicted in sequence, and the prediction result of the single-factor prediction model and the prediction result of the multi-factor prediction model of the air quality data and the current meteorological data are merged into an xgboost tree to perform regression calculation to obtain the prediction value of the air quality data;
the method comprises the following specific steps:
s1, training PM2.5, PM10 and NO in sequence by using the xgboost tree as a training model2、CO、O3And SO2One of the six air mass concentration data is input into the prediction result of the single-factor prediction model and the multi-factor model prediction of a certain air mass concentration data during each training, the weather conditions at the latest moment including temperature, air pressure, humidity, wind direction, wind speed and weather indexes are also input, and the corresponding labeled data during the training are the current air mass data, namely PM2.5, PM10 and NO2、CO、O3、SO2Inputting the input data and the labeled data of each type into the xgboost tree;
s2, establishing n regression subtrees inside the xgboost model during training, that is, there are n tree nodes, and each tree node has multiple leavesThe method comprises the following steps that nodes are provided, wherein leaf nodes are predicted values of a single-factor and multi-factor prediction model and eight characteristic values input by meteorological data at the latest moment, which leaf nodes correspond to each tree node are determined according to weights of the eight input characteristic values, the principle of node splitting is to select a characteristic with the largest information gain as a splitting point, so that the leaf nodes of each sub-tree have weight values, the weight values of the leaf nodes are adjusted according to a target function in each training round to form a new sub-tree, the optimal sub-tree is achieved after multiple iterations, each sub-tree node is a predicted value, and all the predicted values of the sub-tree nodes are accumulated to obtain an air quality predicted value; after input data and labeled data are input into the xgboost training, six air quality predicted values including PM2.5, PM10 and NO can be obtained respectively2、CO、O3And SO2Six kinds of concentration data.
2. The method of claim 1, wherein the air quality data comprises PM2.5, PM10, NO2、CO、O3And SO2Concentration data of one or more than two types.
3. The method of claim 1, wherein the meteorological data includes temperature, barometric pressure, humidity, wind direction, wind speed, and weather indicators.
4. The method of claim 1, wherein the missing value interpolation is performed by using an expectation maximization method.
5. The method as claimed in claim 1, wherein the input layer of the single-factor prediction model is historical air quality data, the structure of the hidden layer is seq2seq model, the inside is coding and decoding structure, the coding and decoding unit is LSTM long-term memory neural network, and the output layer is air quality prediction value.
6. The method as claimed in claim 1, wherein the input layers of the multi-model fusion deep learning air quality prediction method are historical air quality data and meteorological data, the hidden layer structure is a seq2seq model with a double attention mechanism, the interior of the model is an encoding-decoding structure, a layer of attention mechanism is added before encoding and decoding, and the output layer is an air quality data prediction value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811210072.3A CN109214592B (en) | 2018-10-17 | 2018-10-17 | Multi-model-fused deep learning air quality prediction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811210072.3A CN109214592B (en) | 2018-10-17 | 2018-10-17 | Multi-model-fused deep learning air quality prediction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109214592A CN109214592A (en) | 2019-01-15 |
| CN109214592B true CN109214592B (en) | 2022-03-08 |
Family
ID=64980570
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811210072.3A Active CN109214592B (en) | 2018-10-17 | 2018-10-17 | Multi-model-fused deep learning air quality prediction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109214592B (en) |
Families Citing this family (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109685290B (en) * | 2019-02-11 | 2023-06-16 | 南方电网科学研究院有限责任公司 | Power consumption prediction method, device and equipment based on deep learning |
| CN110009134B (en) * | 2019-03-08 | 2020-12-18 | 浙江大学 | Pulping energy consumption prediction method based on seq2seq dynamic feature extraction model |
| CN109991685A (en) * | 2019-04-03 | 2019-07-09 | 北京市天元网络技术股份有限公司 | A kind of precipitation prediction technique and device based on more LSTM Model Fusions |
| CN110209131B (en) * | 2019-05-07 | 2020-11-17 | 西安交通大学 | Quality prediction method based on error transfer network and lifting tree algorithm |
| CN110333556A (en) * | 2019-06-03 | 2019-10-15 | 深圳中兴网信科技有限公司 | Air Quality Forecast method, apparatus, computer equipment and readable storage medium storing program for executing |
| CN110428106A (en) * | 2019-08-05 | 2019-11-08 | 山东农业大学 | A kind of crop water requirement prediction technique based on machine learning |
| CN110738349B (en) * | 2019-09-05 | 2023-07-11 | 国网浙江省电力有限公司杭州供电公司 | Power grid fault rush-repair duration prediction method based on multi-model fusion |
| CN110851796B (en) * | 2019-11-12 | 2021-09-24 | 北京工商大学 | Music copyright protection system based on block chain intelligent contract |
| CN111144625A (en) * | 2019-12-10 | 2020-05-12 | 北京蛙鸣信息科技发展有限公司 | Air quality prediction method and system based on adjacent space data principal component elements |
| CN111160628A (en) * | 2019-12-13 | 2020-05-15 | 重庆邮电大学 | Air pollutant concentration prediction method based on CNN and double-attention seq2seq |
| CN113052353B (en) * | 2019-12-27 | 2022-10-18 | 中移雄安信息通信科技有限公司 | Air quality prediction and prediction model training method and device and storage medium |
| CN111209968B (en) * | 2020-01-08 | 2023-05-12 | 浙江师范大学 | Multi-meteorological-factor mode prediction temperature correction method and system based on deep learning |
| CN111243752A (en) * | 2020-01-16 | 2020-06-05 | 四川大学华西医院 | Prediction model for acute pancreatitis induced organ failure |
| CN111141879B (en) * | 2020-02-21 | 2023-02-03 | 防灾科技学院 | Deep learning air quality monitoring method, device and equipment |
| CN111582551B (en) * | 2020-04-15 | 2023-12-08 | 中南大学 | Wind power plant short-term wind speed prediction method and system and electronic equipment |
| CN111639787B (en) * | 2020-04-28 | 2024-03-15 | 北京工商大学 | Spatio-temporal data prediction method based on graph convolution network |
| CN111598156B (en) * | 2020-05-14 | 2024-07-19 | 北京工业大学 | PM based on multi-source heterogeneous data fusion2.5Prediction method |
| CN111553543B (en) * | 2020-05-18 | 2022-11-18 | 润联软件***(深圳)有限公司 | TPA-Seq2 Seq-based power load prediction method and related components |
| CN112580859B (en) * | 2020-06-01 | 2022-10-18 | 北京理工大学 | Haze prediction method based on global attention mechanism |
| CN111798051B (en) * | 2020-07-02 | 2023-11-10 | 杭州电子科技大学 | Air quality space-time prediction method based on long-term and short-term memory neural network |
| CN111784073A (en) * | 2020-07-16 | 2020-10-16 | 武汉空心科技有限公司 | Deep learning-based work platform task workload prediction method |
| CN111814964A (en) * | 2020-07-20 | 2020-10-23 | 江西省环境监测中心站 | Air pollution treatment method based on air quality condition prediction and storage medium |
| CN112163527B (en) * | 2020-09-29 | 2022-06-14 | 华中科技大学 | Fusion model-based tobacco leaf baking state identification method, device and system |
| CN114429800B (en) * | 2020-10-15 | 2024-06-21 | 中国石油化工股份有限公司 | Methane hydrate generation rate prediction method and system based on model fusion |
| CN112862168A (en) * | 2021-01-28 | 2021-05-28 | 中山大学 | Neural network multi-model combination-based population density prediction method and system |
| US11512864B2 (en) * | 2021-04-14 | 2022-11-29 | Jiangnan University | Deep spatial-temporal similarity method for air quality prediction |
| CN113960925A (en) * | 2021-08-30 | 2022-01-21 | 中科苏州微电子产业技术研究院 | Building energy consumption control method and device based on artificial intelligence |
| CN113688822A (en) * | 2021-09-07 | 2021-11-23 | 河南工业大学 | Time sequence attention mechanism scene image identification method |
| CN113837487A (en) * | 2021-10-13 | 2021-12-24 | 国网湖南省电力有限公司 | Power system load prediction method based on combined model |
| CN114518611B (en) * | 2021-12-24 | 2023-06-06 | 山东省青岛生态环境监测中心(中国环境监测总站黄海近岸海域环境监测分站) | Ozone forecasting method based on similar case discriminant analysis |
| CN114510850B (en) * | 2022-04-20 | 2022-06-21 | 四川国蓝中天环境科技集团有限公司 | Multi-model fusion calibration method and system for atmospheric six-parameter differentiation |
| CN114662791A (en) * | 2022-04-22 | 2022-06-24 | 重庆邮电大学 | Long time sequence pm2.5 prediction method and system based on space-time attention |
| CN114547017B (en) * | 2022-04-27 | 2022-08-05 | 南京信息工程大学 | Meteorological big data fusion method based on deep learning |
| CN115079308B (en) * | 2022-07-04 | 2023-10-24 | 湖南省生态环境监测中心 | Air quality set forecasting system and method thereof |
| CN115616978B (en) * | 2022-10-20 | 2024-06-28 | 重庆大学 | SQ-LSTMA-based thermal error prediction model, prediction method and control system |
| CN115732041B (en) * | 2022-12-07 | 2023-10-13 | 中国石油大学(北京) | Carbon dioxide capture quantity prediction model construction method, intelligent prediction method and device |
| CN115794981B (en) * | 2022-12-14 | 2023-09-26 | 广西电网有限责任公司 | Method and system for counting meteorological data by using model |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10169656B2 (en) * | 2016-08-29 | 2019-01-01 | Nec Corporation | Video system using dual stage attention based recurrent neural network for future event prediction |
| CN107909206B (en) * | 2017-11-15 | 2021-06-04 | 电子科技大学 | PM2.5 prediction method based on deep structure recurrent neural network |
| CN108197736B (en) * | 2017-12-29 | 2021-08-13 | 北京工业大学 | Air quality prediction method based on variational self-encoder and extreme learning machine |
| CN108510113A (en) * | 2018-03-21 | 2018-09-07 | 中南大学 | A kind of application of XGBoost in short-term load forecasting |
| CN108537383A (en) * | 2018-04-09 | 2018-09-14 | 山东建筑大学 | A kind of room air prediction technique based on Model Fusion |
-
2018
- 2018-10-17 CN CN201811210072.3A patent/CN109214592B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109214592A (en) | 2019-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109214592B (en) | Multi-model-fused deep learning air quality prediction method | |
| CN109802430B (en) | Wind power grid control method based on LSTM-Attention network | |
| CN110135630B (en) | Short-term load demand prediction method based on random forest regression and multi-step optimization | |
| CN111210082B (en) | Precipitation prediction method based on optimized BP neural network algorithm | |
| CN113205226B (en) | Photovoltaic power prediction method combining attention mechanism and error correction | |
| CN111160620B (en) | Short-term wind power prediction method based on end-to-end memory network | |
| CN111260126B (en) | Short-term photovoltaic power generation prediction method considering correlation degree of weather and meteorological factors | |
| CN113554466A (en) | Short-term power consumption prediction model construction method, prediction method and device | |
| CN112232554A (en) | Construction method of local short-term rainfall forecast model based on BP neural network | |
| CN113255972B (en) | Short-term rainfall prediction method based on Attention mechanism | |
| CN114970362B (en) | Power grid load scheduling prediction method and system under multi-energy structure | |
| CN108399470B (en) | Indoor PM2.5 prediction method based on multi-example genetic neural network | |
| CN106600060B (en) | Solar irradiation amount prediction method based on similar day sunny coefficient correction | |
| CN106845080B (en) | Based on the modified Scene Tourist meteorological disaster intelligent Forecasting of difference | |
| CN112733996A (en) | GA-PSO (genetic Algorithm-particle swarm optimization) based hydrological time sequence prediction method for optimizing XGboost | |
| CN113505923A (en) | Regional power grid short-term load prediction method and system | |
| CN114462718A (en) | CNN-GRU wind power prediction method based on time sliding window | |
| CN114357670A (en) | Power distribution network power consumption data abnormity early warning method based on BLS and self-encoder | |
| CN114091647A (en) | Solar 10.7 cm radio flow forecasting method based on BP neural network | |
| CN115186923A (en) | Photovoltaic power generation power prediction method and device and electronic equipment | |
| CN114021830A (en) | Multi-time-range wind speed prediction method based on CNN-LSTM | |
| CN110046756B (en) | Short-term weather forecasting method based on wavelet denoising and Catboost | |
| CN116187540A (en) | Wind power station ultra-short-term power prediction method based on space-time deviation correction | |
| CN114692981A (en) | Medium-and-long-term runoff forecasting method and system based on Seq2Seq model | |
| CN114564513A (en) | Sea fog prediction method, device, equipment and storage medium based on neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240116 Address after: 230000 floor 1, building 2, phase I, e-commerce Park, Jinggang Road, Shushan Economic Development Zone, Hefei City, Anhui Province Patentee after: Dragon totem Technology (Hefei) Co.,Ltd. Address before: 100048, Fu Cheng Road, Beijing, Haidian District, No. 33 Patentee before: BEIJING TECHNOLOGY AND BUSINESS University |
|
| TR01 | Transfer of patent right |