CN116050595A - Attention mechanism and decomposition mechanism coupled runoff amount prediction method - Google Patents
Attention mechanism and decomposition mechanism coupled runoff amount prediction method Download PDFInfo
- Publication number
- CN116050595A CN116050595A CN202211710368.8A CN202211710368A CN116050595A CN 116050595 A CN116050595 A CN 116050595A CN 202211710368 A CN202211710368 A CN 202211710368A CN 116050595 A CN116050595 A CN 116050595A
- Authority
- CN
- China
- Prior art keywords
- runoff
- component
- prediction
- sequence
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000000354 decomposition reaction Methods 0.000 title claims abstract description 25
- 230000007246 mechanism Effects 0.000 title claims abstract description 20
- 230000000737 periodic effect Effects 0.000 claims abstract description 26
- 230000008878 coupling Effects 0.000 claims abstract description 18
- 238000010168 coupling process Methods 0.000 claims abstract description 18
- 238000005859 coupling reaction Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 21
- 238000010606 normalization Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 17
- 230000000295 complement effect Effects 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000002779 inactivation Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000010008 shearing Methods 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 abstract description 2
- 230000009467 reduction Effects 0.000 abstract description 2
- 208000037004 Myoclonic-astatic epilepsy Diseases 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000000874 microwave-assisted extraction Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003651 drinking water Substances 0.000 description 1
- 235000020188 drinking water Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Tourism & Hospitality (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- General Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Educational Administration (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention belongs to the field of machine learning, and particularly discloses a method for predicting runoff quantity by coupling an attention mechanism and a decomposition mechanism, which comprises the following steps: normalizing the historically observed runoff data, and inputting the normalized runoff data into a trained runoff prediction model to obtain a runoff prediction sequence; the runoff quantity prediction model comprises a positive standardization module, a time sequence decomposition module, a multi-head self-attention module, a time convolution network module and an inverse standardization module; the invention combines time sequence decomposition, a multi-head self-attention mechanism and a time convolution network, couples trend characteristics, periodic characteristics and self-attention characteristics on the basis of initial prediction of the time convolution network, comprehensively and efficiently digs strong trend characteristics and strong periodic characteristics of runoff data, adopts inverse standardized reduction, enhances the distribution consistency of the data, and realizes high-accuracy prediction of the runoff.
Description
Technical Field
The invention belongs to the field of machine learning, and particularly relates to a runoff prediction method with a coupling of an attention mechanism and a decomposition mechanism.
Background
The runoff is the total flow flowing through the section of the river channel within a specific time range (such as three days, one week, half month, one quarter and the like). In real life, the runoff amount prediction technology has wide application scenes. For example, by predicting the runoff of each main flow and each tributary in the flow domain, important data support can be provided for overall command of flood control and drought control, irrigation and reasonable dispatching of drinking water resources.
From the observation data over the years, the nature of natural river channel runoff is a time series with strong periodicity. The time sequence is a data sequence sequenced according to time sequence, and reflects the trend of the continuous evolution change of the observed parameters along with time. The current runoff prediction method generally adopts a time sequence prediction thought, and can be divided into a traditional runoff prediction method, a runoff prediction method based on pattern recognition and a runoff prediction method based on deep learning. The traditional runoff quantity prediction method utilizes the statistical characteristics of the historical time sequence to establish a statistical model and solve the problems of high parameter sensitivity and the like; the runoff amount prediction method based on pattern recognition needs to select characteristics by using professional knowledge, and the migration capability of a usage scene is poor; the runoff amount prediction method based on deep learning has the problems of high calculation complexity, insufficient utilization of historical data and the like.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a runoff quantity prediction method with a coupling attention mechanism and a decomposition mechanism, which aims to excavate strong trend characteristics and strong periodic characteristics of runoff quantity observation data in month and quarter scales, integrate a multi-head self-attention mechanism and a decomposition mechanism and realize high-accuracy prediction of runoff quantity.
In order to achieve the above purpose, the specific technical scheme is as follows:
a method of traffic prediction with a coupling of an attention mechanism and a resolution mechanism, comprising: normalizing the historically observed runoff data, and inputting the normalized runoff data into a trained runoff prediction model to obtain a runoff prediction sequence; the runoff quantity prediction model comprises a positive standardization module, a time sequence decomposition module, a multi-head self-attention module, a time convolution network module and an inverse standardization module;
the positive standardization module is used for carrying out positive standardization processing on the runoff sequence samples in the training set to obtain a standardized runoff sequence;
the time sequence decomposition module is used for decomposing the standardized runoff sequence to obtain a trend component and a periodic component;
the multi-head self-attention module is used for obtaining self-attention prediction components;
the time convolution network module is used for obtaining an initial prediction component;
the inverse normalization module is used for performing inverse normalization processing on the coupling prediction sequence obtained by the multi-component feature fusion to obtain a runoff prediction sequence.
Further, the process of training the runoff quantity prediction model comprises the following steps:
s1: all data in the runoff data set are normalized;
s2: dividing the runoff data set into a training set, a verification set and a test set according to the proportion;
s3: inputting the runoff sequence samples in the training set into a positive standardization module for positive standardization processing to obtain a standardized runoff sequence;
s4: inputting the standardized runoff sequence into a time sequence decomposition module for time sequence decomposition, and inputting the decomposition result into a linear layer to obtain a trend component and a periodic component;
s5: inputting the standardized runoff sequence into a multi-head self-attention module to obtain a self-attention prediction component;
s6: inputting the standardized runoff sequence into a time convolution network, and obtaining initial predicted components through a plurality of time convolution modules connected in series;
s7: weighting and adding the trend component, the periodic component, the self-attention prediction component and the initial prediction component to obtain a coupling prediction sequence;
s8: inputting the coupling prediction sequence into an inverse standardization module for inverse standardization processing to obtain a runoff prediction sequence;
s9: calculating a loss function of the runoff prediction model according to the runoff prediction sequence and the runoff observation sequence;
s10: continuously adjusting the learning rate, and dynamically optimizing model parameters according to the learning rate;
s11: and verifying the model obtained by training on a verification set, and completing model training when the loss function is minimum.
Wherein the loss function in step S9 is preferably a mean square error (Mean Square Error, MSE); the strategy for adjusting the learning rate in step S10 is preferably piecewise constant decay, and the model optimization is preferably Adam algorithm.
In step S1, the normalization process calculates the mean value and standard deviation of all the data, and assigns values to each sample in the runoff data set again, so as to make it conform to gaussian distribution.
Further, in step S3, the positive normalization process is performed on the traffic sequence samples in the training set, which may be expressed as the following formula:
wherein ,
n represents the length of the runoff amount prediction period;
the observation value of the i-time runoff amount after the positive normalization processing is shown.
Further, in step S4, the process of obtaining the trend component and the periodic component includes the following steps:
s4.1: the data complement at two ends of the data are adjusted according to the average kernel size, and one-dimensional average pooling is carried out on the data after complement to obtain an initial trend component;
s4.2: subtracting the initial trend component from the complement data to obtain an initial periodic component;
s4.3: and respectively inputting the initial trend component and the initial periodic component into a linear layer, and unifying the output dimensions to be the same as the target sequence to obtain the trend component and the periodic component.
Further, in step S5, the process of obtaining the self-attention prediction component includes the steps of:
s5.1: inputting the normalized runoff sequence into a linear layer to obtain an initial time sequence component;
s5.2: respectively inputting the initial time sequence components into three different linear layers to obtain a query sub-component, a key value sub-component and a numerical value sub-component;
s5.3: performing self-attention operation on the query sub-component, the key value sub-component and the numerical sub-component to obtain a self-attention prediction initial component;
s5.4: the self-attention prediction initial component is input into a linear layer, and the output dimension is adjusted to obtain the self-attention prediction component.
Still further, the process of deriving the self-attention prediction component may be expressed as the following formula:
wherein ,
linear represents a Linear layer;
W Q 、W K and WV Respectively representing a weight matrix corresponding to each of the query sub-component, the key value sub-component and the numerical value sub-component;
q, K and V represent query and value sub-components, respectively;
K T a transposed component representing a key-value subcomponent;
d k representing model dimensions;
softmax represents the normalized exponential function;
F A representing the self-attention prediction component.
Further, in step S6, the process of obtaining the initial predicted component includes the following steps:
s6.1: constructing a time convolution network, wherein the time convolution network comprises six time convolution modules connected in series, and each time convolution module comprises two sub-modules consisting of a one-dimensional expansion convolution layer and a shear layer;
s6.2: in the time convolution module, firstly, one-dimensional expansion convolution is carried out on input sequence data, then redundant data of a header is sheared to ensure one-way transmission of predicted information flow, and then one-dimensional expansion convolution and shearing are carried out again;
s6.3: the output of the time convolution module will be the input of the next stage of time convolution module until six serially connected time convolution blocks are passed.
Still further, the process of obtaining the initial predicted component may be expressed as the following formula:
wherein ,
conv represents a one-dimensional dilated convolution layer;
chomp represents a shear layer;
ReLU represents a nonlinear activation function;
dropout represents a random inactivation function;
conv1D represents one-dimensional convolution;
F t representing the data sequence processed by a basic dilated convolution layer;
F c representing the data sequence processed by a time convolution module;
F T representing the initial predicted component extracted over time by the convolutional network.
Further, in step S8, the process of performing inverse normalization processing on the coupling prediction sequence to obtain the runout prediction value may be expressed as the following formula:
wherein ,
y represents a coupled predicted sequence;
sigma represents the standard deviation of the normalized runoff observation;
The invention has the following remarkable effects:
the invention combines time sequence decomposition, a multi-head self-attention mechanism and a time convolution network, couples trend characteristics, periodic characteristics and self-attention characteristics on the basis of initial prediction of the time convolution network, comprehensively and efficiently digs strong trend characteristics and strong periodic characteristics of runoff data, adopts inverse standardized reduction, enhances the distribution consistency of the data, and realizes high-accuracy prediction of the runoff.
Drawings
FIG. 1 is a flow chart of the runoff prediction of a preferred embodiment of the present invention.
Detailed Description
The principles and features of the present invention are described below in connection with examples, which are set forth only to illustrate the present invention and not to limit the scope of the invention.
A method of traffic prediction with a coupling of an attention mechanism and a resolution mechanism, comprising: normalizing the historically observed runoff data, and inputting the normalized runoff data into a trained runoff prediction model to obtain a runoff prediction sequence (a flow chart is shown in figure 1); the runoff quantity prediction model comprises a positive standardization module, a time sequence decomposition module, a multi-head self-attention module, a time convolution network module and an inverse standardization module;
the positive standardization module is used for carrying out positive standardization processing on the runoff sequence samples in the training set to obtain a standardized runoff sequence;
the time sequence decomposition module is used for decomposing the standardized runoff flow sequence to obtain a trend component and a periodic component;
the multi-head self-attention module is used for obtaining self-attention prediction components;
the time convolution network module is used for obtaining an initial prediction component;
and the inverse normalization module is used for performing inverse normalization processing on the coupling prediction sequence obtained by the multi-component feature fusion to obtain a runoff prediction sequence.
The process for training the runoff quantity prediction model comprises the following steps:
s1: and (5) normalizing all data in the runoff data set. The runoff data set is a runoff data set of hydrological stations along the Yangtze river basin, the normalization processing is to calculate the mean value and standard deviation of all data, and each sample in the runoff data set is assigned again to enable the samples to accord with Gaussian distribution;
s2: dividing the runoff data set into a training set, a verification set and a test set according to the ratio of 7:1:2, wherein the training set is used for training the runoff prediction model, the verification set is used for checking and iteratively optimizing the effect of the model in the training process, and the test set is used for evaluating the prediction effect.
S3: inputting the runoff sequence samples in the training set into a positive standardization module for positive standardization processing to obtain a standardized runoff sequence;
s4: inputting the standardized runoff sequence into a time sequence decomposition module for time sequence decomposition, and inputting the decomposition result into a linear layer to obtain a trend component and a periodic component;
s5: inputting the standardized runoff sequence into a multi-head self-attention module to obtain a self-attention prediction component;
s6: inputting the standardized runoff sequence into a time convolution network, and obtaining initial predicted components through a plurality of time convolution modules connected in series;
s7: weighting and adding the trend component, the periodic component, the self-attention prediction component and the initial prediction component to obtain a coupling prediction sequence, and setting weights according to a ratio of 1:1:1:1 in order to simplify calculation;
s8: inputting the coupling prediction sequence into an inverse standardization module for inverse standardization processing to obtain a runoff prediction sequence;
s9: calculating a mean square error (Mean Square Error, MSE) loss function of the traffic prediction model according to the traffic prediction sequence and the traffic observation sequence;
s10: continuously adjusting the learning rate by adopting a piecewise constant attenuation strategy, and dynamically optimizing model parameters by adopting an Adam algorithm according to the learning rate;
s11: and verifying the model obtained by training on a verification set, and completing model training when the loss function is minimum.
In step S1, normalization processing is to calculate the mean value and standard deviation of all data, and assign values to each sample in the runoff data set again, so as to make it conform to gaussian distribution.
In step S3, the positive normalization process is performed on the runoff sequence samples in the training set, which can be expressed as the following formula:
wherein ,
n represents the length of the runoff amount prediction period;
the observation value of the i-time runoff amount after the positive normalization processing is shown.
In step S4, the process of obtaining the trend component and the periodic component includes the following steps:
s4.1: the data complement at two ends of the data are adjusted according to the average kernel size, and one-dimensional average pooling is carried out on the data after complement to obtain an initial trend component;
s4.2: subtracting the initial trend component from the complement data to obtain an initial periodic component;
s4.3: and respectively inputting the initial trend component and the initial periodic component into a linear layer, and unifying the output dimensions to be the same as the target sequence to obtain the trend component and the periodic component.
In step S5, the process of obtaining the self-attention prediction component includes the steps of:
s5.1: inputting the normalized runoff sequence into a linear layer to obtain an initial time sequence component;
s5.2: respectively inputting the initial time sequence components into three different linear layers to obtain a query sub-component, a key value sub-component and a numerical value sub-component;
s5.3: performing self-attention operation on the query sub-component, the key value sub-component and the numerical sub-component to obtain a self-attention prediction initial component;
s5.4: the self-attention prediction initial component is input into a linear layer, and the output dimension is adjusted to obtain the self-attention prediction component.
The process of deriving the self-attention prediction component S5.1-S5.4 can be expressed as follows:
wherein ,
linear represents a Linear layer;
W Q 、W K and WV Respectively representing a weight matrix corresponding to each of the query sub-component, the key value sub-component and the numerical value sub-component;
q, K and V represent query and value sub-components, respectively;
K T a transposed component representing a key-value subcomponent;
d k representing model dimensions;
softmax represents the normalized exponential function;
F A representing the self-attention prediction component.
In step S6, the process of obtaining the initial predicted component includes the steps of:
s6.1: constructing a time convolution network, wherein the time convolution network comprises six time convolution modules connected in series, and each time convolution module comprises two sub-modules consisting of a one-dimensional expansion convolution layer and a shear layer;
s6.2: in the time convolution module, firstly, one-dimensional expansion convolution is carried out on input sequence data, then redundant data of a header is sheared to ensure one-way transmission of predicted information flow, and then one-dimensional expansion convolution and shearing are carried out again;
s6.3: the output of the time convolution module is used as the input of the next time convolution module until the time convolution expansion rates in the six time convolution blocks are sequentially 1, 2, 4, 8, 16 and 32, and the number of middle convolution layer channels is sequentially 32, 64, 128, 64, 32 and N V Number of tail convolutional layer channels N V Should be consistent with the number of variables predicted at the same time.
The process of obtaining the initial predicted component in S6.1-S6.3 can be expressed as follows:
wherein ,
conv represents a one-dimensional dilated convolution layer;
chomp represents a shear layer;
ReLU represents a nonlinear activation function;
dropout represents a random inactivation function;
conv1D represents one-dimensional convolution;
F t representing the data sequence processed by a basic dilated convolution layer;
F c representing the data sequence processed by a time convolution module;
F T representing the initial predicted component extracted over time by the convolutional network.
In step S8, the process of performing inverse normalization processing on the coupling prediction sequence to obtain the runout prediction value may be expressed as the following formula:
wherein ,
y represents a coupled predicted sequence;
sigma represents the standard deviation of the normalized runoff observation;
In step S9, a Mean Square Error (MSE) loss function of the traffic prediction model is calculated according to the traffic prediction sequence and the traffic observation sequence, where the expression is:
wherein ,
n represents the length of the runoff amount prediction period;
In order to verify the effectiveness of the method, the method is compared with a classical time sequence prediction method Long-Short Term Memory (LSTM) and a transform to predict the effect in a Yangtze river basin along a hydrographic station runoff data set, and an average absolute error (Mean Absolute Error, MAE) and a Nash-Sutcliffe Efficiency, NSE are adopted as evaluation indexes.
MAE is the average value of absolute values of all single predicted values and observed value differences, and can better reflect the magnitude of the prediction error, and the calculation expression is as follows:
wherein ,
n represents the length of the runoff amount prediction period;
NSE is an index commonly used in hydrology for evaluating the prediction result of a hydrologic model, and the range of the NSE is minus infinity to 1. The NSE value is close to 1, indicating high reliability of prediction; the value is close to 0, which means that the predicted result is close to the average level of the observed value, namely the overall result is reliable, but a certain error exists in a single predicted value; a value much smaller than 0 indicates that the prediction result is not trusted.
wherein ,
n represents the length of the runoff amount prediction period;
Table 1 shows MAEs at different prediction periods (three days, one week, half month, one quarter) for a Yangtze river basin standing along a hydrologic station runoff data set to a home dam hydrologic station in cubic meters per second. From the data in the table, the MAE values of the method of the present invention were lower than those of the comparative method in all test cycles, indicating that the overall prediction error was minimal and that greater advantage was exhibited in long-cycle prediction.
TABLE 1 mean absolute error contrast at different prediction periods
Table 2 shows NSE at different prediction periods (three days, one week, half month, one quarter) for the Yangtze river basin along the hydrographic station runoff data set to the home dam hydrographic station. As can be seen from the data in the table, compared with the comparison method, the NSE value of the method is closer to 1 in all test periods, so that the reliability of the diameter flow prediction result is highest, and the method also shows greater advantages in long period prediction.
TABLE 2 Nash correlation coefficient comparison at different prediction periods
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (9)
1. A method for predicting a runoff amount by coupling an attention mechanism with a decomposition mechanism, comprising: normalizing the historically observed runoff data, and inputting the normalized runoff data into a trained runoff prediction model to obtain a runoff prediction sequence; the runoff quantity prediction model comprises a positive standardization module, a time sequence decomposition module, a multi-head self-attention module, a time convolution network module and an inverse standardization module;
the positive standardization module is used for carrying out positive standardization processing on the runoff sequence samples in the training set to obtain a standardized runoff sequence;
the time sequence decomposition module is used for decomposing the standardized runoff sequence to obtain a trend component and a periodic component;
the multi-head self-attention module is used for obtaining self-attention prediction components;
the time convolution network module is used for obtaining an initial prediction component;
the inverse normalization module is used for performing inverse normalization processing on the coupling prediction sequence obtained by the multi-component feature fusion to obtain a runoff prediction sequence.
2. The method of traffic prediction according to claim 1, characterized in that the training of the traffic prediction model comprises the steps of:
s1: all data in the runoff data set are normalized;
s2: dividing the runoff data set into a training set, a verification set and a test set according to the proportion;
s3: inputting the runoff sequence samples in the training set into a positive standardization module for positive standardization processing to obtain a standardized runoff sequence;
s4: inputting the standardized runoff sequence into a time sequence decomposition module for time sequence decomposition, and inputting the decomposition result into a linear layer to obtain a trend component and a periodic component;
s5: inputting the standardized runoff sequence into a multi-head self-attention module to obtain a self-attention prediction component;
s6: inputting the standardized runoff sequence into a time convolution network, and obtaining initial predicted components through a plurality of time convolution modules connected in series;
s7: weighting and adding the trend component, the periodic component, the self-attention prediction component and the initial prediction component to obtain a coupling prediction sequence;
s8: inputting the coupling prediction sequence into an inverse standardization module for inverse standardization processing to obtain a runoff prediction sequence;
s9: calculating a loss function of the runoff prediction model according to the runoff prediction sequence and the runoff observation sequence;
s10: continuously adjusting the learning rate, and dynamically optimizing model parameters according to the learning rate;
s11: and verifying the model obtained by training on a verification set, and completing model training when the loss function is minimum.
3. The method for predicting the runoff amount according to claim 2, wherein in step S3, the forward normalization processing is performed on the runoff amount sequence samples in the training set, and may be expressed as the following formula:
wherein ,
n represents the length of the runoff amount prediction period;
4. The runoff amount prediction method according to claim 2, wherein in step S4, the process of obtaining the trend component and the periodic component includes the steps of:
s4.1: the data complement at two ends of the data are adjusted according to the average kernel size, and one-dimensional average pooling is carried out on the data after complement to obtain an initial trend component;
s4.2: subtracting the initial trend component from the complement data to obtain an initial periodic component;
s4.3: and respectively inputting the initial trend component and the initial periodic component into a linear layer, and unifying the output dimensions to be the same as the target sequence to obtain the trend component and the periodic component.
5. The runoff amount prediction method according to claim 2, wherein in step S5, the process of obtaining the self-attention prediction component includes the steps of:
s5.1: inputting the normalized runoff sequence into a linear layer to obtain an initial time sequence component;
s5.2: respectively inputting the initial time sequence components into three different linear layers to obtain a query sub-component, a key value sub-component and a numerical value sub-component;
s5.3: performing self-attention operation on the query sub-component, the key value sub-component and the numerical sub-component to obtain a self-attention prediction initial component;
s5.4: the self-attention prediction initial component is input into a linear layer, and the output dimension is adjusted to obtain the self-attention prediction component.
6. The runoff amount prediction method of claim 5, wherein the process of obtaining the self-attention prediction component may be expressed as the following formula:
wherein ,
linear represents a Linear layer;
W Q 、W K and WV Respectively representing a weight matrix corresponding to each of the query sub-component, the key value sub-component and the numerical value sub-component;
q, K and V represent query and value sub-components, respectively;
K T a transposed component representing a key-value subcomponent;
d k representing model dimensions;
softmax represents the normalized exponential function;
F A representing the self-attention prediction component.
7. The method for predicting runout according to claim 2, wherein in step S6, the process of obtaining the initial predicted component includes the steps of:
s6.1: constructing a time convolution network, wherein the time convolution network comprises six time convolution modules connected in series, and each time convolution module comprises two sub-modules consisting of a one-dimensional expansion convolution layer and a shear layer;
s6.2: in the time convolution module, firstly, one-dimensional expansion convolution is carried out on input sequence data, then redundant data of a header is sheared, and then one-dimensional expansion convolution and shearing are carried out again;
s6.3: the output of the time convolution module will be the input of the next stage of time convolution module until six serially connected time convolution blocks are passed.
8. The method of traffic prediction according to claim 7, wherein the process of obtaining the initial predicted component can be expressed as the following formula:
wherein ,
conv represents a one-dimensional dilated convolution layer;
chomp represents a shear layer;
ReLU represents a nonlinear activation function;
dropout represents a random inactivation function;
conv1D represents one-dimensional convolution;
F t representing the data sequence processed by a basic dilated convolution layer;
F c representing the data sequence processed by a time convolution module;
F T representing the initial predicted component extracted over time by the convolutional network.
9. The method for predicting the runout according to claim 2, wherein in step S8, the process of performing inverse normalization processing on the coupling prediction sequence to obtain the predicted value of the runout may be expressed as the following formula:
wherein ,
y represents a coupled predicted sequence;
sigma represents the standard deviation of the normalized runoff observation;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211710368.8A CN116050595A (en) | 2022-12-29 | 2022-12-29 | Attention mechanism and decomposition mechanism coupled runoff amount prediction method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211710368.8A CN116050595A (en) | 2022-12-29 | 2022-12-29 | Attention mechanism and decomposition mechanism coupled runoff amount prediction method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116050595A true CN116050595A (en) | 2023-05-02 |
Family
ID=86119270
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211710368.8A Withdrawn CN116050595A (en) | 2022-12-29 | 2022-12-29 | Attention mechanism and decomposition mechanism coupled runoff amount prediction method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116050595A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117273241A (en) * | 2023-11-17 | 2023-12-22 | 北京京东乾石科技有限公司 | Method and device for processing data |
CN117522416A (en) * | 2023-12-28 | 2024-02-06 | 北京芯盾时代科技有限公司 | Transaction account identification method and device |
-
2022
- 2022-12-29 CN CN202211710368.8A patent/CN116050595A/en not_active Withdrawn
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117273241A (en) * | 2023-11-17 | 2023-12-22 | 北京京东乾石科技有限公司 | Method and device for processing data |
CN117273241B (en) * | 2023-11-17 | 2024-04-05 | 北京京东乾石科技有限公司 | Method and device for processing data |
CN117522416A (en) * | 2023-12-28 | 2024-02-06 | 北京芯盾时代科技有限公司 | Transaction account identification method and device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN116050595A (en) | Attention mechanism and decomposition mechanism coupled runoff amount prediction method | |
Shiri et al. | Estimation of daily suspended sediment load by using wavelet conjunction models | |
CN107463993A (en) | Medium-and Long-Term Runoff Forecasting method based on mutual information core principle component analysis Elman networks | |
CN113554466A (en) | Short-term power consumption prediction model construction method, prediction method and device | |
Xu et al. | A water level prediction model based on ARIMA-RNN | |
CN112330065A (en) | Runoff forecasting method based on basic flow segmentation and artificial neural network model | |
CN108021773A (en) | The more play flood parameters rating methods of hydrological distribution model based on DSS data base read-writes | |
CN112348290B (en) | River water quality prediction method, river water quality prediction device, storage medium and storage device | |
Golob et al. | Neural-network-based water inflow forecasting | |
CN109919356A (en) | One kind being based on BP neural network section water demand prediction method | |
CN112653198A (en) | Wind power output scene generation method and system based on prediction box | |
CN116702627A (en) | Urban storm waterlogging rapid simulation method based on deep convolutional neural network | |
CN114357670A (en) | Power distribution network power consumption data abnormity early warning method based on BLS and self-encoder | |
CN114154716B (en) | Enterprise energy consumption prediction method and device based on graph neural network | |
CN111680712A (en) | Transformer oil temperature prediction method, device and system based on similar moments in the day | |
CN116579447A (en) | Time sequence prediction method based on decomposition mechanism and attention mechanism | |
CN114692981A (en) | Medium-and-long-term runoff forecasting method and system based on Seq2Seq model | |
CN107330538A (en) | A kind of method of climate lower storage reservoir adaptability scheduling rule establishment | |
CN116227738B (en) | Method and system for predicting traffic interval of power grid customer service | |
Wang et al. | Prediction of water quality in South to North Water Transfer Project of China based on GA-optimized general regression neural network | |
CN115221731A (en) | Transformer life evaluation method based on data fusion and Wiener model | |
Lyu et al. | Water level prediction model based on GCN and LSTM | |
CN116050604A (en) | Water-wind-light power combined forecasting method, device and equipment considering space-time complementarity | |
CN116247658A (en) | Photovoltaic power generation amount prediction method based on DLRM deep learning model | |
CN115829150A (en) | Accumulated water prediction system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
WW01 | Invention patent application withdrawn after publication | ||
WW01 | Invention patent application withdrawn after publication |
Application publication date: 20230502 |