CN116703644A - Attention-RNN-based short-term power load prediction method - Google Patents
Attention-RNN-based short-term power load prediction method Download PDFInfo
- Publication number
- CN116703644A CN116703644A CN202310388456.9A CN202310388456A CN116703644A CN 116703644 A CN116703644 A CN 116703644A CN 202310388456 A CN202310388456 A CN 202310388456A CN 116703644 A CN116703644 A CN 116703644A
- Authority
- CN
- China
- Prior art keywords
- rnn
- prediction
- power load
- short
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000007246 mechanism Effects 0.000 claims abstract description 35
- 230000007774 longterm Effects 0.000 claims abstract description 14
- 238000005065 mining Methods 0.000 claims abstract description 10
- 239000013598 vector Substances 0.000 claims description 49
- 239000011159 matrix material Substances 0.000 claims description 37
- 230000010354 integration Effects 0.000 claims description 23
- 238000013507 mapping Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 12
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000012512 characterization method Methods 0.000 claims description 4
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000001932 seasonal effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Development Economics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Game Theory and Decision Science (AREA)
- Artificial Intelligence (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Educational Administration (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Primary Health Care (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Water Supply & Treatment (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a short-term power load prediction method based on an Attention-RNN, and relates to the technical field of power system load prediction. The current power load prediction has the problem that nonlinear and non-stationary data cannot be processed; the method comprises the following steps: based on an Attention mechanism, quantifying the implicit time sequence correlation among time nodes in a load time sequence, and extracting cross-correlation characteristics; then, the memory characteristics of the RNN network are used to extract trend features and cycle features implicit in the load long-term sequence based on the RNN, and the time-series dependence of the load sequence is mined. And mining the time correlation and long-term dependence characteristics of the load time sequence from the historical data by using an attribute mechanism and RNN network characteristics to form a short-term power load prediction model based on the attribute-RNN, and carrying out short-term power load prediction according to the short-term power load prediction model. According to the method, the load time sequence characteristics and the external multidimensional influence factors are comprehensively considered, and compared with a traditional prediction method, the accuracy of short-term power load prediction is effectively improved.
Description
Technical Field
The invention relates to the technical field of power system load prediction, in particular to a short-term power load prediction method based on Attention-RNN.
Background
Short-term power load prediction refers to predicting the power load of a power grid within hours or days in the future, so that a power company can reasonably plan power production and supply, thereby ensuring the stable operation of the power grid and meeting the power demand of users. Short-term power load prediction generally adopts a statistical and machine learning method based on historical data, and the technology related behind the method mainly comprises data acquisition, preprocessing, feature extraction, model construction, evaluation and the like.
First, predicting the power load requires collecting a large amount of historical data, including various factors such as weather, holidays, weekdays/holidays, and the like. The data are collected by means of sensors, weather forecast, electric meters and the like to form a large data set. Next, the data is preprocessed, including data cleaning, normalization, denoising, etc., to ensure data quality and reliability.
Feature extraction is one of the key steps in constructing predictive models, and generally requires that useful features be extracted from data by methods such as time series analysis, frequency domain analysis, wavelet analysis, and the like. For example, characteristics of average load, highest load, lowest load, load fluctuation, and the like per day of the week can be extracted from the history data.
Model construction is the core of short-term power load prediction, requiring the selection and training of an appropriate model. Common models include regression models, time series models, neural network models, and the like. The neural network model is excellent in short-term power load prediction, particularly a deep learning model such as a long-short-term memory network (RNN) and the like, can automatically learn characteristics and has good prediction capability.
Finally, to evaluate the accuracy and performance of the prediction model, some metrics are required to measure the prediction error. For example, the Root Mean Square Error (RMSE) may be used to evaluate the magnitude of the error of the predicted result, or the mean absolute error (MAPE) may be used to evaluate the degree of deviation of the predicted result.
Current power load prediction algorithms, while well established, suffer from drawbacks such as inability to handle non-linearity problems, inability to handle non-stationary data, and the like. More advanced algorithms are therefore needed to improve prediction accuracy.
Disclosure of Invention
The invention aims to solve the technical problems and provide the technical task of perfecting and improving the prior art scheme, and provides a short-term power load prediction method based on the Attention-RNN so as to improve the short-term power load prediction precision. For this purpose, the present invention adopts the following technical scheme.
A short-term power load prediction method based on an Attention-RNN comprises the following steps:
1) Determining a characterization representation method of multi-element heterogeneous data comprising air temperature and holiday information, defining a data set format of short-term load prediction considering various load influence factors, and constructing a single-step prediction and multi-step prediction basic framework of daily electric load prediction;
2) According to the daily power load prediction basic framework formed in the step 1), quantifying time sequence correlation implied among time nodes in a load time sequence based on an Attention mechanism, and forming a daily power load prediction Attention module;
3) According to the daily power load prediction basic framework formed in the step 1), extracting trend features and period features implied in a load long-term sequence based on the RNN, and mining time sequence dependence of the load sequence to form a daily power load prediction RNN module;
4) Based on the Attention module and the RNN module formed in the step 2) and the step 3), constructing a serial and parallel model integration framework, and mining time correlation and long-term dependence characteristics of a load time sequence from historical data by using an Attention mechanism and RNN network characteristics to form a short-term power load prediction model based on the Attention-RNN, and carrying out short-term power load prediction according to the short-term power load prediction model.
According to the technical scheme, on the basis of RNN, the Attention mechanism is introduced, so that key information can be adaptively learned in long sequence data, and the change and trend in the sequence can be more accurately captured. Compared with the traditional RNN model, the RNN model utilizing the Attention mechanism can better process long-term dependence problems, and can adaptively adjust weights when modeling historical data, so that the prediction accuracy of the model is improved. With the introduction of the attention mechanism, key information in time series data can be adaptively captured. The method generally combines a gating circulation unit and an attention mechanism, can improve the modeling capacity of long-sequence data, and has stronger generalization capacity. Long-term sequence data can be better processed and perform better in cases where input characteristics are uncertain or data noise is large.
As a preferable technical means: in step 1), the original dataset is represented as:
wherein: x is an input matrix; x is x i For the input eigenvector of the ith day, the input eigenvector is composed of 96-point load eigenvector l i And an external factor feature vector f i Constructing; l (L) i ∈R 96 For the load vector on day i, the number of points depends on the sampling frequency; f (f) i ∈R 26 For the feature vector of the external factor on the ith day, d is coded by the workday type i ∈R 7 Seasonal type code s i ∈R 4 Moon type code m i ∈R 12 Holiday type code h i ∈R 2 Weather feature vector n i F is formed of i Expressed as:
f i ＝[d i s i m i h i n i ] (16)
wherein, the time scale characteristic coding mode adopts one-hot coding; after determining the data structure of the short-term power load prediction, the single-step prediction and multi-step prediction base frame of the day-ahead power load prediction is expressed as:
X i+1:i+T ＝[x i+1 x i+2 … x i+T ] T (19)
F i+T+K:i+T+K+τ ＝[f i+T+K f i+T+K+1 … f i+T+K+τ ] T (20)
in the method, in the process of the invention,a power load prediction matrix; x is X i+1:i+T Input a matrix for history; f (F) i+T+K:i+T+K+τ Is a synchronous feature matrix; t and τ are the historical window width and the predicted window width, respectively; k is the number of days of advance prediction; Φ represents a short-term power load prediction model; θ * The resulting parameters are trained for the model.
The original data set adopts days, seasons, months, holidays and weather to predict short-term power load, is favorable for improving prediction precision, generalization capability and reliability, and has important significance for practical application. The method comprises the following steps:
1. factors of various aspects are considered: by introducing multiple features, the complexity and uncertainty of the power load change can be reflected more comprehensively, including factors of natural environment, social culture, population flow and the like. These features may provide for more accurate input of information, thereby improving the accuracy of the prediction.
2. Data distribution non-uniformity is improved: in consideration of various features, data distribution unevenness can be improved, and prediction errors can be reduced. For example, when important events such as holidays or weather abrupt changes are encountered, the actual change situation of the power load can be reflected better.
3. The generalization capability of the model is enhanced: by considering a plurality of time scales and different types of characteristics, the adaptability of the model to unknown conditions can be improved, and the generalization capability of the model is enhanced, so that the model has reliability and robustness.
In addition, in the technical scheme, the time scale characteristics of one-hot coding and the basic frames of single-step prediction and multi-step prediction are adopted, so that the flexibility, the expandability and the accuracy of the model can be improved, and the method is suitable for various short-term power load prediction scenes.
The time scale features are encoded by adopting one-hot codes, so that the model processing is convenient: in neural networks, the use of numerical features is easily misunderstood, and one-hot encoding can convert the classification variables into vector form for neural network processing. In this way, the model can better understand the relationship between different time points in the time series data. In addition, the numerical value of the misleading model can be avoided: the time scale features in numerical form may mislead the model such that the model erroneously considers certain points in time to be more important than others. By using one-hot coding, this misdirection can be eliminated, ensuring that each time point is treated equally.
The flexibility can be improved by adopting a basic framework of single-step prediction and multi-step prediction: the single-step prediction and multi-step prediction frameworks can be flexibly applied to different data sets and tasks to meet different application requirements. For example, for prediction tasks requiring finer granularity, a multi-step prediction framework can be adopted to improve prediction accuracy and stability; for simple prediction tasks, a single-step prediction framework may be selected to reduce the amount of computation. And the expandability is improved: the framework based on single-step prediction and multi-step prediction can be combined with other model structures to realize more complex prediction tasks. For example, these frameworks can be combined with deep learning methods, statistical models, traditional time series prediction methods to improve prediction accuracy and robustness.
As a preferable technical means: in step 2), quantifying the time sequence correlation implied between each time node in the load time sequence based on an Attention mechanism, and extracting cross-correlation characteristics; in order to quantify the time correlation between each time step of the load time sequence, a key-value mechanism is adopted; the input feature vector is converted into a query vector q through linear mapping i Key vector k i Value vector v i The method comprises the steps of carrying out a first treatment on the surface of the The relevance score a between different time steps is obtained by calculating the dot product of the query vector and the key vector:
a＝q·k (21)
the time correlation between different time steps is obtained by calculating dot products of different time step query vectors and key vectors, and the overall process for quantifying the load time sequence correlation based on an Attention mechanism is expressed as follows:
q i ＝W q x i ,k i ＝W k x i ,v i ＝W v x i ,a i,j ＝q i ·k j (22)
A＝softmax(K T Q),H＝VA (24)
in which W is q ，W k ，W v For the query mapping matrix, a key mapping matrix and a value mapping matrix; a, a i,j For the input vector x i ，x j A correlation score between; matrix A is composed of a i,j Constructing; k and Q are key matrix and query matrix, respectively defined by K i And q i Constructing; h is the Attention output.
The key-value mechanism is adopted to quantify the time correlation of each time step of the load time sequence, and the method has the advantages of definite time sequence relationship, strong flexibility, convenience in data processing and the like. Definite time sequence relation: the relationship between time steps can be clearly described by using a key-value mechanism, i.e. each key represents a characteristic of a time step, and a value represents a specific numerical value or vector of the time step. This mechanism enables the model to better grasp the dependency between different time steps in the time series data, thereby predicting future trends more accurately. The flexibility is strong: when the key-value mechanism is used, keys of each time step can be customized according to actual needs, for example, different time scales such as seasons, months and the like are adopted, and meanwhile, a plurality of keys can be customized for different application scenes, so that the prediction precision and generalization capability are further improved. And the data processing is convenient: different information can be stored and processed in the same format by using a key-value mechanism, batch reading and processing of data are facilitated, and the expressive power of a model is further improved by combining the key-value mechanism with an attention mechanism.
As a preferable technical means: in step 3), the process formula for forming the pre-day power load prediction RNN module based on the load timing dependency characteristics of the RNN is as shown in (11) - (14);
in the method, in the process of the invention,a weight matrix of the first layer of the RNN; />Bias for RNN layer i; sigma is a sigmoid activation function; w (W) o Is a linear mapping matrix; />Extracting the t-moment hidden state characteristics for the first layer RNN network; />The hidden state characteristics of the RNN output layer and the nearest moment of the moment to be predicted are obtained; h is a RNN Feature vectors extracted for RNNs; l is the number of RNN network layers; t is the RNN input window width.
As a preferable technical means: in step 4): generating a history window and a prediction window by the original data set through sliding window processing;
in the serial integration framework: firstly, inputting data in a history window into an RNN module, and processing the data by the RNN module to extractTaking the time sequence dependency characteristic of the load sequence; then, extracting the feature vector group obtained by the RNN moduleInputting an Attention module, extracting the correlation of each time step, and finally inputting H and the synchronous characteristics in a prediction window into a full-connection layer together, and outputting to obtain a short-term power load predicted value;
in a parallel integration framework: firstly, inputting data in a history window into an RNN module and an attribute module in parallel, and mining time correlation and long-term dependence characteristics of a load time sequence; then, the RNN extracted feature vector setThe Attention output H and the synchronization characteristic in the prediction window are input into the full connection layer together, and the short-term power load predicted value is obtained through output.
The history window is used as input, and multi-model fusion is carried out in the serial integration framework or the parallel integration framework, so that complementarity among different models can be fully mined, and the prediction accuracy is improved.
The beneficial effects are that: according to the technical scheme, the Attention module and the RNN module are adopted, the serial and parallel model integration frames are adopted by the two modules, the serial integration frame can improve the mapping capability of the prediction model, the load prediction error is reduced by reducing the model prediction variance, the capacity of the prediction model can be improved by the parallel integration frame, and the load prediction error is reduced by reducing the model deviation. Based on the attribute mechanism and the RNN network characteristics, the load time sequence characteristics and external multidimensional influence factors are considered, the RNN model utilizing the attribute mechanism can better process long-term dependence problems, and the weight can be adaptively adjusted when historical data is modeled, so that the prediction precision of the model is improved. With the introduction of the attention mechanism, key information in time series data can be adaptively captured. Compared with the traditional prediction scheme, such as a short-term load prediction method based on models such as a feedforward neural network, a convolution neural network and a cyclic neural network, the technical scheme effectively improves the accuracy of short-term power load prediction.
Drawings
FIG. 1 is a schematic overall flow chart of the present invention.
Fig. 2 is a schematic diagram of a sliding window process of the present invention.
FIG. 3 is a schematic diagram of the Attention mechanism of the present invention.
FIG. 4 is a schematic representation of the RNN according to the present invention.
Fig. 5 is a schematic diagram of a serial integration framework of the present invention.
Fig. 6 is a schematic diagram of a parallel integration framework of the present invention.
Fig. 7 is a graph of the short-term load prediction of the present invention.
Detailed Description
For better understanding of the objects, technical solutions and technical effects of the present invention, the present invention will be further explained below with reference to the accompanying drawings.
The embodiment provides a short-term power load prediction method based on an attribute-RNN, and an implementation process of the short-term power load prediction method comprises the following detailed steps:
step 1: providing a characterization representation method of multi-element heterogeneous data such as air temperature, holiday information and the like, defining a data set format of short-term load prediction considering various load influence factors, and constructing a single-step prediction and multi-step prediction basic framework of daily electric load prediction;
the raw data set of short-term power load predictions may be represented as
Wherein: x is an input matrix; x is x i For the input eigenvector of the ith day, the input eigenvector is composed of 96-point load eigenvector l i And an external factor feature vector f i Constructing; l (L) i ∈R 96 For the load vector on day i, the number of points depends on the sampling frequency; f (f) i ∈R 26 For the feature vector of the external factor on the ith day, d is coded by the workday type i ∈R 7 Seasonal type code s i ∈R 4 Moon type code m i ∈R 12 Holiday type code h i ∈R 2 Weather feature vector n i F is formed of i Can be expressed as
f i ＝[d i s i m i h i n i ] (2)
The time scale feature coding (workday type coding, season type coding and the like) adopts one-hot coding, namely, 1 is set in the dimension of the type on the same day, and 0 is set in the other dimension. After determining the data structure of the short-term power load prediction, the single-step prediction and multi-step prediction base frame of the day-ahead power load prediction may be expressed as
In the method, in the process of the invention,a power load prediction matrix; x is X i+1:i+T Input a matrix for history; f (F) i+T+K:i+T+K+τ Is a synchronous feature matrix; t and τ are the historical window width and the predicted window width, respectively; k is the number of days of advance prediction; Φ represents a short-term power load prediction model; θ * The resulting parameters are trained for the model.
The key to short-term power load predictive modeling is to build an appropriate model and determine parameters of the model. The parameters of the model in this embodiment are determined by minimizing the desired risk R exp And (3) determining:
in the method, in the process of the invention,representing a loss function. The desired parameters of the final model satisfy:
in practical applications, we approximate the expected risk by calculating the empirical risk, which will approach the expected risk when the sample is sufficiently large, according to the law of large numbers:
to implement rolling prediction of short-term power load, this embodiment proposes a sliding window processing method, as shown in fig. 2. Historical load data and synchronization related data within the sliding window are used as inputs to the short-term power load prediction model to account for the auto-correlation characteristics of load changes and the cross-correlation characteristics of the load and its influencing factors. As the sliding window slides, the predictive model gradually outputs the load predictive value.
Step 2: according to the daily power load prediction basic framework formed in the step 1, quantifying the time sequence correlation implied among all time nodes in the load time sequence based on an Attention mechanism, and forming a daily power load prediction Attention module;
and (3) quantifying the time sequence correlation implicit among all time nodes in the load time sequence based on an Attention mechanism according to the daily power load prediction basic framework formed in the step (1), extracting cross-correlation characteristics, and calculating the Attention according to the figure 3. To quantify the time dependence of the load sequence between time steps, a "key-value" mechanism is introduced. The input feature vector is converted into a query vector q through linear mapping i Key vector k i Value vector v i . The correlation score a between different time steps is obtained by calculating the dot product between key value vectors:
a＝q·k (10)
the time correlation between different time steps is obtained by calculating dot products among key value vectors of different time steps, and the whole flow for quantifying the load time sequence correlation based on the Attention mechanism can be expressed as follows
q i ＝W q x i ,k i ＝W k x i ,v i ＝W v x i ,a i,j ＝q i ·k j (11)
A＝softmax(K T Q),H＝VA (13)
In which W is q ，W k ，W v For the query mapping matrix, a key mapping matrix and a value mapping matrix; a, a i,j For the input vector x i ，x j A correlation score between; matrix A is composed of a i,j Constructing; k and Q are key matrix and query matrix, respectively defined by K i And q i Constructing; and H is the output obtained by weighting and summing the time step value vectors by the attribute score weight.
Step 3: according to the daily power load prediction basic framework formed in the step 1, extracting trend characteristics and period characteristics implied in a load long-term sequence based on the RNN, and mining time sequence dependence of the load sequence to form a daily power load prediction RNN module;
the processing flow of the RNN-based load timing dependent feature extraction module is as shown in (14) - (17).
In the method, in the process of the invention,a weight matrix of the first layer of the RNN; />Bias for RNN layer i; sigma is a sigmoid activation function; w (W) o Is a linear mapping matrix; />Extracting the t-moment hidden state characteristics for the first layer RNN network; />And the hidden state characteristics of the most adjacent time between the RNN output layer and the time to be predicted are obtained. h is a RNN Feature vectors extracted for RNNs; l is the number of RNN network layers; t is the RNN input window width.
The multidimensional RNN network layer is formed by connecting a plurality of units in series, and load state characteristic information at different moments passes through hidden statesAnd the RNN can extract and memorize the dependency characteristics among the time nodes of the continuous load sequence and couple the load information of different time nodes. In order to enhance the capability of the RNN network to extract and represent load time sequence characteristic information, the embodiment stacks multiple layers of RNNs to learn the complex nonlinear relationship between industry load and its influencing factors. The RNN-based daily electrical load prediction RNN module is shown in fig. 4.
Step 4: based on the Attention module and the RNN module formed in the step 2 and the step 3, constructing a serial model integration framework and a parallel model integration framework, and utilizing an Attention mechanism and RNN network characteristics to mine time correlation and long-term dependence characteristics of a load time sequence from historical data so as to form an Attention-RNN-based short-term power load prediction method;
based on the Attention module and the RNN module, a serial model integration framework and a parallel model integration framework are constructed. In the serial integration framework: firstly, inputting data in a history window into an RNN module, and extracting time sequence dependency characteristics of a load sequence through processing of the RNN module; then, extracting the feature vector group obtained by the RNN moduleInputting an Attention module, extracting the correlation of each time step to obtain an output matrix H considering the correlation of each time step; finally, H and the synchronization characteristic in the prediction window are input into a full connection layer together, short-term power load predicted values are obtained through output, and a serial integration framework schematic diagram is shown in fig. 5. In a parallel integration framework: firstly, inputting data in a history window into an RNN module and an attribute module in parallel, and mining time correlation and long-term dependence characteristics of a load time sequence; then, the RNN extracted feature vector group +.>The content output H and the synchronization characteristic in the prediction window are input into the full connection layer together, the short-term power load predicted value is obtained through output, and a parallel integration framework schematic diagram is shown in fig. 6.
For evaluating the model prediction effect, the root mean square error and the average absolute percentage error are used as prediction evaluation indexes, as shown in the formula (18) and the formula (19), respectively.
In order to verify the effectiveness and accuracy of the proposed short-term power load prediction method based on the Attention-RNN, an example simulation was performed with 96-point daily load data of 2021 in a city.
Firstly, preprocessing the original 96-point daily load data according to the characterization representation method of the multi-component heterogeneous data and the single-step prediction and multi-step prediction basic framework of daily power load prediction, wherein the preprocessing comprises normalization, feature coding, data shaping and sliding window processing of the data.
Then, based on the sliding window data, forming an RNN module and an Attention module according to the RNN feature extraction method and the Attention time sequence correlation quantization method mentioned in the step 2 and the step 3.
Then, according to the two frameworks proposed in step 3, the RNN module and the Attention module are integrated, and meanwhile, the synchronization related characteristic influence in the prediction window is considered, and the synchronization related characteristic influence is input into the full connection layer. In order to improve the training efficiency of the model while ensuring certain mapping capability of the model, the number of hidden layers in the GRU module is set to 2, the number of layers of attention is set to 1, the number of layers of synchronous characteristic nonlinear mapping is set to 2, and the number of layers of full connection is set to 1, so that modeling work of the short-term daily load prediction model is completed. Dividing the data set subjected to sliding windowing into a training set and a testing set on the basis of model structure determination, wherein the training set is used for training a constructed model and determining model parameters; the test set is used for verifying the effect of the constructed model.
In this embodiment, an Attention module and an RNN module are combined with a serial and parallel model integration framework, and load time sequence characteristics and external multidimensional influence factors are considered by using an Attention mechanism and RNN network characteristics, so that compared with the traditional prediction method, the method has the following advantages:
consider the external multidimensional influencing factor: the traditional prediction method only considers the historical load data, and the technical scheme can effectively consider external multidimensional influence factors such as temperature, humidity, holidays and the like by introducing an Attention mechanism, so that the prediction precision is improved.
Fully mining time sequence characteristics: by adopting the RNN network, the time series characteristics of the load data can be modeled, so that the information such as periodicity, trend and the like in the load data can be better captured, and the prediction accuracy is further improved.
Model fusion improves robustness: the attribute module and the RNN module are fused through a serial and parallel model integration framework, so that complementarity among different models is fully excavated, and the robustness and stability of prediction are improved.
The interpretability is strong: due to the adoption of the Attention mechanism, the importance of each external influence factor in prediction can be displayed in a visual mode, so that the interpretation of the model is improved.
In order to verify the effectiveness and superiority of the method provided by the invention, other typical prediction models, such as ANN, CNN, GRU, CNN-GRU and the like, are constructed, and compared with the prediction results of the invention, the evaluation indexes of the prediction models are compared as shown in the following table.
TABLE 1 comparison of short-term load prediction model evaluation index
A short-term load prediction curve based on the above model is shown in fig. 7. As can be seen from fig. 7 and table 1, compared with other short-term load prediction models, the short-term load prediction model based on the Attention-RNN proposed by the present invention has relatively higher prediction accuracy, wherein the Attention-RNN short-term load prediction model of the serial architecture has the highest accuracy, the smallest error and the best generalization performance.
The short-term power load prediction method based on the Attention-RNN is a specific embodiment of the invention, has shown the substantial characteristics and the progress of the invention, can be subjected to equivalent modification in terms of shape, structure and the like according to actual use requirements under the teaching of the invention, and is within the scope of protection of the scheme.
Claims (5)
1. A short-term power load prediction method based on an Attention-RNN is characterized by comprising the following steps:
1) Determining a characterization representation method of multi-element heterogeneous data comprising air temperature and holiday information, defining a data set format of short-term load prediction considering various load influence factors, and constructing a single-step prediction and multi-step prediction basic framework of daily electric load prediction;
2) According to the daily power load prediction basic framework formed in the step 1), quantifying time sequence correlation implied among time nodes in a load time sequence based on an Attention mechanism, and forming a daily power load prediction Attention module;
3) According to the daily power load prediction basic framework formed in the step 1), extracting trend features and period features implied in a load long-term sequence based on the RNN, and mining time sequence dependence of the load sequence to form a daily power load prediction RNN module;
4) Based on the Attention module and the RNN module formed in the step 2) and the step 3), constructing a serial and parallel model integration framework, and mining time correlation and long-term dependence characteristics of a load time sequence from historical data by using an Attention mechanism and RNN network characteristics to form a short-term power load prediction model based on the Attention-RNN, and carrying out short-term power load prediction according to the short-term power load prediction model.
2. The short-term power load prediction method based on Attention-RNN according to claim 1, wherein: in step 1), the original dataset is represented as:
wherein: x is an input matrix; x is x i For the input eigenvector of the ith day, the input eigenvector is composed of 96-point load eigenvector l i And an external factor feature vector f i Constructing; l (L) i ∈R 96 For the load vector on day i, the number of points depends on the sampling frequency; f (f) i ∈R 26 For the feature vector of the external factor on the ith day, d is coded by the workday type i ∈R 7 Seasonal type code s i ∈R 4 Moon type code m i ∈R 12 Holiday type code h i ∈R 2 Weather feature vector n i F is formed of i Expressed as:
f i ＝[d i s i m i h i n i ] (2)
wherein, the time scale characteristic coding mode adopts one-hot coding; after determining the data structure of the short-term power load prediction, the single-step prediction and multi-step prediction base frame of the day-ahead power load prediction is expressed as:
X i+1:i+T ＝[x i+1 x i+2 … x i+T ] T (5)
F i+T+K:i+T+K+τ ＝[f i+T+K f i+T+K+1 … f i+T+K+τ ] T (6)
in the method, in the process of the invention,a power load prediction matrix; x is X i+1:i+T Input a matrix for history; f (F) i+T+K:i+T+K+τ Is a synchronous feature matrix; t and τ are the historical window width and the predicted window width, respectively; k is the number of days of advance prediction; Φ represents a short-term power load prediction model; θ * The resulting parameters are trained for the model.
3. The short-term power load prediction method based on Attention-RNN according to claim 1, wherein: in step 2), quantifying the time sequence correlation implied between each time node in the load time sequence based on an Attention mechanism, and extracting cross-correlation characteristics; in order to quantify the time correlation between each time step of the load time sequence, a key-value mechanism is adopted; input feature vector passingConversion of linear mapping into query vector q i Key vector k i Value vector v i The method comprises the steps of carrying out a first treatment on the surface of the The relevance score a between different time steps is obtained by calculating the dot product of the query vector and the key vector:
a＝q·k (7)
the time correlation between different time steps is obtained by calculating dot products of different time step query vectors and key vectors, and the overall process for quantifying the load time sequence correlation based on an Attention mechanism is expressed as follows:
q i ＝W q x i ,k i ＝W k x i ,v i ＝W v x i ,a i,j ＝q i ·k j (8)
A＝softmax(K T Q),H＝VA (10)
in which W is q ，W k ，W v For the query mapping matrix, a key mapping matrix and a value mapping matrix; a, a i,j For the input vector x i ，x j A correlation score between; matrix A is composed of a i,j Constructing; k and Q are key matrix and query matrix, respectively defined by K i And q i Constructing; h is the Attention output.
4. The short-term power load prediction method based on Attention-RNN according to claim 1, wherein: in step 3), the process formula for forming the pre-day power load prediction RNN module based on the load timing dependency characteristics of the RNN is as shown in (11) - (14);
in the method, in the process of the invention,a weight matrix of the first layer of the RNN; />Bias for RNN layer i; sigma is a sigmoid activation function; w (W) o Is a linear mapping matrix; />Extracting the t-moment hidden state characteristics for the first layer RNN network; />The hidden state characteristics of the RNN output layer and the nearest moment of the moment to be predicted are obtained; h is a RNN Feature vectors extracted for RNNs; l is the number of RNN network layers; t is the RNN input window width.
5. The short-term power load prediction method based on Attention-RNN according to claim 1, wherein: in step 4): generating a history window and a prediction window by the original data set through sliding window processing;
in the serial integration framework: firstly, inputting data in a history window into an RNN module, and extracting time sequence dependency characteristics of a load sequence through processing of the RNN module; then, extracting the feature vector group obtained by the RNN moduleInputting an Attention module, extracting the correlation of each time step, and finally inputting H and the synchronous characteristics in a prediction window into a full-connection layer together, and outputting to obtain a short-term power load predicted value;
in a parallel integration framework: firstly, inputting data in a history window into an RNN module and an attribute module in parallel, and mining time correlation and long-term dependence characteristics of a load time sequence; then, the RNN extracted feature vector setThe Attention output H and the synchronization characteristic in the prediction window are input into the full connection layer together, and the short-term power load predicted value is obtained through output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310388456.9A CN116703644A (en) | 2023-04-12 | 2023-04-12 | Attention-RNN-based short-term power load prediction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310388456.9A CN116703644A (en) | 2023-04-12 | 2023-04-12 | Attention-RNN-based short-term power load prediction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116703644A true CN116703644A (en) | 2023-09-05 |
Family
ID=87839916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310388456.9A Pending CN116703644A (en) | 2023-04-12 | 2023-04-12 | Attention-RNN-based short-term power load prediction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116703644A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117113159A (en) * | 2023-10-23 | 2023-11-24 | 国网山西省电力公司营销服务中心 | Deep learning-based power consumer side load classification method and system |
-
2023
- 2023-04-12 CN CN202310388456.9A patent/CN116703644A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117113159A (en) * | 2023-10-23 | 2023-11-24 | 国网山西省电力公司营销服务中心 | Deep learning-based power consumer side load classification method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113962364B (en) | Multi-factor power load prediction method based on deep learning | |
| CN111260136A (en) | Building short-term load prediction method based on ARIMA-LSTM combined model | |
| CN112633604B (en) | Short-term power consumption prediction method based on I-LSTM | |
| CN109635245A (en) | A kind of robust width learning system | |
| CN111985719B (en) | Power load prediction method based on improved long-term and short-term memory network | |
| CN114239718B (en) | High-precision long-term time sequence prediction method based on multi-element time sequence data analysis | |
| CN115660161A (en) | Medium-term and small-term load probability prediction method based on time sequence fusion Transformer model | |
| CN111861013A (en) | Power load prediction method and device | |
| CN113537469B (en) | Urban water demand prediction method based on LSTM network and Attention mechanism | |
| CN110852496A (en) | Natural gas load prediction method based on LSTM recurrent neural network | |
| CN112508286A (en) | Short-term load prediction method based on Kmeans-BilSTM-DMD model | |
| CN114781744A (en) | Deep learning multi-step long radiance prediction method based on codec | |
| CN115860177A (en) | Photovoltaic power generation power prediction method based on combined machine learning model and application thereof | |
| CN116703644A (en) | Attention-RNN-based short-term power load prediction method | |
| CN114595861A (en) | MSTL (modeling, transformation, simulation and maintenance) and LSTM (least Square TM) model-based medium-and-long-term power load prediction method | |
| CN114819395A (en) | Industry medium and long term load prediction method based on long and short term memory neural network and support vector regression combination model | |
| CN115456245A (en) | Prediction method for dissolved oxygen in tidal river network area | |
| CN115238854A (en) | Short-term load prediction method based on TCN-LSTM-AM | |
| CN114091776A (en) | K-means-based multi-branch AGCNN short-term power load prediction method | |
| CN113762591A (en) | Short-term electric quantity prediction method and system based on GRU and multi-core SVM counterstudy | |
| CN116739168A (en) | Runoff prediction method based on gray theory and codec | |
| CN112529268B (en) | Medium-short term load prediction method and device based on manifold learning | |
| Kang et al. | Research on forecasting method for effluent ammonia nitrogen concentration based on GRA-TCN | |
| Li et al. | A Novel Short-term Load Forecasting Model by TCN-LSTM Structure with Attention Mechanism | |
| CN115222113B (en) | CNN-LSTM-based railway area freight volume short-term prediction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |