CN115544889A - Numerical mode precipitation deviation correction method based on deep learning - Google Patents

Abstract

The invention discloses a method for correcting deviation of numerical mode precipitation products based on deep learning, which comprises the following specific steps: acquiring ERA5 land reanalysis data, ECMWF daily rainfall prediction data and ground station daily rainfall observation data; selecting effective characteristics by utilizing the correlation between meteorological factors and ground observation precipitation values through Pearson correlation coefficients; obtaining a relatively accurate daily rainfall label; establishing a model precipitation product deviation correction model based on deep learning, and designing a weight loss function to update model parameters; and training to obtain an optimal model. The method has the advantages that the historical meteorological sequence is added into the deep learning model, the model design double-branch structure not only pays attention to local precipitation but also gives consideration to overall precipitation, the problems of large precipitation span and extreme unbalance of the weight loss function environment are solved, the model can well learn precipitation errors in a numerical mode, and the precipitation deviation correction effect is achieved.

Description

Numerical mode precipitation deviation correction method based on deep learning

Technical Field

The invention relates to the field of deep learning rainfall forecast deviation correction, in particular to a numerical mode rainfall deviation correction method based on deep learning.

Background

Inferring and predicting weather has been a long-standing challenge throughout human history. In modern weather forecasting, a numerical weather forecasting (NWP) model based on physical equations and using a high-performance computer as a calculation tool is in a leading position. The European middle-term Weather forecast center (ECMWF) is the leading numerical Weather forecast center in the world. In addition, the ECMWF forecast has a better reference meaning for the forecast accuracy of China regions, and errors inevitably occur in numerical mode forecast due to the existence of accumulated errors caused by a plurality of uncertain factors and integrals and the averaging effect caused by the aggregation of a plurality of aggregation members in the numerical simulation process. Therefore, reasonable, objective and quantitative correction is a bridge for connecting a numerical mode and accurate prediction, and is an indispensable link for deeply excavating numerical prediction potential.

At present, a great deal of research work is done by a great number of weather researchers on the aspect of weather forecast correction tasks. Mode error correction is mainly divided into two categories: the first type is that the quality of initial field data is improved and the performance of numerical prediction is improved from the mode prediction itself; another category is post-processing error correction of the forecast based on statistical models. Vannitsem and Tian use a Model Output Statistical (MOS) method to prove that the model has better effect on a high-resolution model. Krishhnamurti et al propose a multi-ensemble prediction idea and correct precipitation in tropical, united states, etc. of the world, and Cui et al correct 2m temperature ensemble average prediction by using an adaptive Kalman filtering method.

It is worth noting that: the weather evolution process is random and discontinuous, and the traditional statistical method has very limited effect on correcting the deviation.

In recent years, deep learning provides a new idea for solving uncertainty of numerical weather forecast, and different from a numerical mode, the deep learning belongs to a class of data-driven models, and a new rule is found from a large amount of space-time data. One possible way to improve the bias correction is to continuously learn the weather dynamics from the EC mode data and improve the correction capability through a deep learning algorithm. The method takes the information of the underlying surfaces such as the terrain, ERA5 historical reanalysis data and an EC mode 24h forecast precipitation product as the input of the network, takes the corresponding ground observation value as the network target, uses a plurality of station data correction numerical modes of the research area to forecast the precipitation product, and does not change the output and input shapes.

The new framework is divided into three modules, namely a spatial coding module, a temporal coding module and a spatial correction module. The spatial coding module codes context information to a research area and considers the influence of the surrounding environment on a target area, the time coding module predicts the spatial variation information of the image factors in the future time for inputting the historical reanalysis data of the ERA5 in the past week, and the spatial correction module adopts a multi-branch network structure and uses a global branch and a local branch to extract the spatial correlation of precipitation and pay attention to the details of local precipitation. And finally, obtaining accurate precipitation in the area.

Disclosure of Invention

Technical problem

Aiming at the defects and the feasible improvement points in the prior art, the invention provides a numerical mode rainfall forecast deviation correction method based on deep learning, and the potential relation of the rainfall deviation is mined by utilizing the learning capacity of the deep learning in big data to achieve the effect of rainfall correction.

Technical scheme

In order to achieve the purpose, the invention adopts the following technical scheme:

the invention relates to a numerical mode precipitation deviation correction method based on deep learning, which comprises the following steps of:

1) Firstly, analyzing the correlation between meteorological data and ground observation precipitation by utilizing ERA5, and selecting meteorological characteristics which have a relation to precipitation as effective characteristics of a model;

2) Cutting the effective meteorological features selected in the step 1) into a research area range to form an area feature area, and carrying out normalization processing on the feature space to form a training sample set;

3) Inputting the training sample set in the step 2), zooming the feature range to the size of a research area in the extraction of the input spatial features, inputting the meteorological features into a space-time prediction network, extracting and predicting the variation trend of the meteorological features in the future time, and keeping the resolution of the feature space consistent with the resolution of numerical prediction through deconvolution;

4) And (3) fusing the prediction result obtained in the step 3) with prior knowledge such as numerical prediction precipitation and DEM (digital elevation model), inputting the result into a space correction module to define a weight loss function, and selecting an optimal model through model training to obtain a final precipitation prediction value.

5) And (5) carrying out inspection and evaluation on the precipitation result after deviation correction.

Further, as a preferred technical solution of the present invention: selecting effective meteorological feature eliminating interference features in the step 1), calculating the correlation between each meteorological feature and ground observation precipitation by using a Pearson correlation coefficient, setting a threshold value, and regarding all features larger than the threshold value as effective feature calculation formulas as follows.

In the formula

Is a characteristic value

The covariance of (a) is determined,

representing the variance of the features X, Y, respectively.

Further, as a preferred technical solution of the present invention: the process of constructing the data set in the step 2) is to select the data of the target area as the training sample characteristics after selecting the effective characteristics, and to normalize the characteristics, because the precipitation distribution is discrete distribution and the span is large, log normalization is performed on the precipitation data, and the Z-SCORE normalization formula is performed on the other meteorological characteristics.

Further, as a preferred technical solution of the present invention: the step 3) of predicting the change trend of the meteorological environment in the future time is to input the constructed historical sequence of ERA5, scale the sample to the size of a research area through spatial feature extraction, input the sample into the MotionRNN to predict the change trend of the cloud cluster in the future research area, and keep the spatial resolution of the predicted trend consistent with the numerical mode data through deconvolution.

Further, as a preferred technical solution of the present invention: the spatial correction module in the step 4) is used for more accurately performing deviation correction on the mode prediction data, the module uses a classical U-Net structure and is divided into two branches, one branch is responsible for global correction, the other branch is dedicated to a smaller area for deviation correction of details, and finally the result is fused and passes through an output layer to obtain a final mode prediction result.

When the network parameters are updated to enable the model to be optimal, the fact that precipitation is long tail distribution data and balance data is needed is considered, so that a weight loss function is designed, the loss function is composed of MSE and MAE, different weights are set for different precipitation strengths, and the weight of strong precipitation is larger than that of weak precipitation. The Loss function is as follows:

and finally, running the optimal model on a verification set, and verifying the performance of the model by using the relevant indexes.

Advantageous effects

The invention provides a numerical model rainfall product deviation correction method based on deep learning, which fully considers the element correlation, the time dependency and the space correlation in the rainfall process, constructs a data structure with time sequence characteristics by selecting factors having potential influence on rainfall, constructs a rainfall deviation correction model with time-space fusion, takes ground observation as real rainfall as a label, and continuously trains and optimizes the model to obtain optimal network parameters.

The model has a good correction effect on the precipitation deviation correction of the ECMWF mode, and is simple and convenient to operate; after the optimal model parameters are obtained, the precipitation deviation correction time of the target area only needs one to five seconds, and the method has high reliability and good application prospect.

On the basis of a model precipitation product, a historical meteorological database is constructed by analyzing weather changes in the precipitation process, and the precipitation intensity is corrected by combining a deep learning method; the original spatial resolution of historical data is reserved, sub-pixel convolution is added into a model to improve the resolution, and new errors caused by artificial interpolation are avoided; when parameters are updated, considering that precipitation is distributed in a discrete mode and precipitation data are extremely unbalanced, and adopting weight loss to relieve unbalanced data; the precipitation bisection and the rain belt position after correction by using the model provided by the invention are obviously improved compared with a numerical mode.

Drawings

FIG. 1 is a study area presentation;

FIG. 2 is a schematic diagram of the basic flow of numerical model precipitation deviation correction in the practice of the present invention;

FIG. 3 is a schematic view of the model structure of the present invention;

FIG. 4 is a schematic diagram of the offset correction model parameter update of the present invention;

FIG. 5 is a TS score in an embodiment of the present invention;

FIG. 6 is a schematic representation of a comparison of precipitation errors in the vicinity of certain observation stations in a target area in accordance with an embodiment of the present invention.

Detailed Description

The method for correcting deviation of numerical model precipitation forecast based on deep learning according to the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

The invention carries out deviation correction on 24-hour accumulated precipitation forecast results in a numerical model, the study area is shown in figure 1 (Shanghai region, jiang and Zhe), the data set mainly selects an ECMWF European middle-term numerical forecast center service product as numerical model forecast data, the data are reported twice every day, the time is 00 hours and 12 hours (UTC time), the spatial resolution of the data in the model is 10km, 24-hour forecast precipitation is selected for the study, the historical reanalysis data of a plurality of meteorological elements on the near-land surface of ERA5 are used for carrying out spatial resolution of 25km and the time resolution of 1 hour, relevant Chinese terrain information is added into the model, ASTER GDEM V2 data is used in the text and comes from a geographic space data cloud, the data are jointly developed by METI and NASA, the spatial resolution of DEM data is 30m, other terrain factors including gradient and slope direction, the relevant tools are used for deriving from the DEM data, and 230 national day data of precipitation stations in 2017-2019 are selected as real ground label data for training.

As shown in fig. 2, before the deviation correction of the numerical model precipitation, the data is selected, first, the correlation between each meteorological feature in the ERA5 data and the ground observed precipitation is calculated by using the pearson correlation coefficient, a threshold value of 0.2 is set, all features larger than the threshold value are considered as valid features, and then, the data is intercepted to the research area range, and the range contains the context information of the spatial features of the research area, namely: the method comprises the steps of including information of the surrounding environment of a research area, namely utilizing context information through describing local information interaction of adjacent positions when feature extraction is carried out, improving spatial correlation, and extracting precipitation spatial distribution information better, wherein time resolution of ERA5 reanalysis data collected by people is 1 hour is inconsistent with time resolution of EC precipitation forecast data, and reanalysis data need to be consistent with EC in time resolution.

The method specifically comprises the steps of adding precipitation characteristics to obtain 24h cumulative daily precipitation, reducing the scales of other variables into 24h time resolution according to the characteristics of the variables, processing missing data and abnormal values of collected EC forecast data, rain gauge observation data and ERA5 historical reanalysis data in order to eliminate problems of data loss, errors and the like caused by system faults or data storage and the like, supplementing the missing values by average values of areas around or near the position, normalizing the data along a time dimension because UTC time is used for the EC precipitation forecast data and Beijing time is used for the rain gauge observation data, and transforming the precipitation characteristics according to a Z-Score algorithm to relieve imbalance because the discrete distribution span of precipitation is large in consideration of different weather characteristics in DEM, EC precipitation forecast data and historical reanalysis data, wherein the normalization processing is required to be carried out on the data along the time dimension, and the precipitation characteristics are required to be transformed to relieve the imbalance because the discrete distribution span of precipitation is large:

wherein

Is the overall average of each of the elements,

the method is characterized in that the standard deviation of the total data is obtained, z is a normalized result, x is daily precipitation data, data used in the method is data of three years 2017-2019, 24-hour predicted precipitation in 2017-2018 is divided into a training set and a verification set, in order to guarantee the diversity of the verification data, a sample is extracted every 7 days to serve as the verification set, and all data in 2019 serve as a test set to evaluate the performance of a model.

The precipitation deviation correction network is shown in fig. 3, the first part is that a spatial coding module inputs historical reanalysis data of effective features of a past week to construct a historical database, the spatial coding module is composed of a pyramid convolution and two common convolutions, the pyramid convolution has four convolution kernels of 3 × 3,5 × 5,7 × 7 and 9 × 9 with different scales, extraction of multi-scale features is guaranteed, the multi-scale features are extracted and fused, the step length is 2 in the common convolution to reduce a feature map to a research area, the spatial correlation is improved by using context information, and a feature map sequence is further input to a MotionRNN to predict the future change trend of a plurality of meteorological element environments by learning the past meteorological environment sequence.

The Motion RNN overcomes the defects that the existing RNN-based space-time prediction model cannot make quick response to complex transient changes and cannot model Motion trends, the Motion GRU is embedded into the model to enhance the modeling of complex motions, and meanwhile, motion Highway connection is added for balancing moving and non-moving parts. This is obviously very practical for transform prediction of complex meteorological environments, the first one

The operating principle of the layer, at time t, is expressed as follows:

in the formula

It is shown that for the modeling of the motion of a feature,

representing the learned movement trend of the whole.

Denotes the first

The input of the layer(s) is (are),

representing the prediction unit hidden variable. And after the future change trend of each meteorological feature is obtained through the motionRNN, the feature graph is expanded to the spatial resolution which is the same as that of the EC precipitation prediction data through the transposed convolution. And secondly, fusing the mode prediction data subjected to the smoothing treatment of the 1 × 1 convolution operation once with the DEM and the slope static data and the prediction characteristics to wait for entering a space precipitation correction module.

The correction module uses the network structure of U-Net, which has the advantage that different sizes of objects can be captured in the input by using features of different proportions, which is particularly important for the precipitation correction task, because some precipitation details are often ignored in the mode data, and the accurate precipitation value of each grid must be corrected by our network.

Most of traditional deep learning networks adopt a single-branch structure, only single characteristics can be sensed, overall attention to global and local characteristics is lacked, and data characteristics cannot be well extracted.

In order to improve the overall understanding of the characteristics, the invention uses two branches in a precipitation correction network, namely a global branch for processing the whole area and a local branch for processing the regional characteristics comprise two branches, one branch is a global branch, the other branch is a local branch, the two branches are divided into different parts and are mutually coordinated, the details of regional overall precipitation and local precipitation are considered, and finally the spatial characteristics of the precipitation in a target region can be obtained by splicing and fusing the characteristics; and finally, entering an output layer to obtain a final rainfall forecast.

For the various levels of 24h precipitation (from 10mm for light rain to 100mm for heavy rain, and even more), which are highly unbalanced, the distribution of these values is a long tail distribution, which if used would degrade the model correction performance, since most samples belong to negative samples, which do not give any significant signal, the invention proposes to use a weighted loss function to help solve this problem, and the specific precipitation magnitude distribution ratios are shown in the following table, for example:

in particular, we assign a weight w (x) to each precipitation intensity x.

The loss function is composed of a weighted Mean Square Error (MSE) and a weighted Mean Absolute Error (MAE) and has the following formula:

wherein N represents the number of rain gauges in the area,

and

respectively representing the actual longitude and latitude and the precipitation after correction,

a specific process of updating the parameters in the reverse direction for the weight corresponding to the rainfall intensity of the ith station is shown in fig. 4.

And applying the trained optimal model to a verification set, and counting the training result of the model to calculate the TS score to evaluate the performance of the model.

Specifically, as shown in fig. 5 and 6, the TS score, i.e., risk score formula shown in fig. 5 is as follows:

wherein

Defining boundaries of positive and negative samples, in particular, with a mark greater than

Is considered as a positive sample, marked less than

Is considered as a negative sample at a precipitation threshold

Under the condition of (1), NA is the number of predicted correct grid points after correction, and NB is the number of missed grid points after correction; NC is the number of the blank newspaper lattice points after correction; ND is that the forecast and observed precipitation is less than the precipitation threshold

The number of grid points.

Claims (5)

1. The numerical model precipitation product deviation correction method based on deep learning is characterized by comprising the following steps of:

1) Firstly, utilizing correlation between ERA5 re-analysis meteorological data and ground observation precipitation amount, and selecting an influence factor having a potential relation to precipitation as an effective characteristic of a model;

2) Cutting the influence factors which have potential relation to precipitation and are selected in the step 1) into a research area range (containing context information) to form a characteristic area, and carrying out normalization processing on the characteristic space to form a training sample set;

3) Inputting the training sample set in the step 2), zooming the characteristic range to the size of a research area in the extraction of the input spatial characteristics, inputting the meteorological characteristics into a space-time prediction network, extracting and predicting the variation trend of the meteorological characteristics in future time, and keeping the characteristic spatial resolution consistent with the numerical prediction resolution through deconvolution;

4) Superposing the prediction result obtained in the step 3) with prior knowledge such as numerical forecast precipitation products, DEMs and the like, inputting the superposition result into a space correction module, updating model parameters by using a self-defined weight loss function, and selecting an optimal model to verify the effect of model deviation correction in a test set;

5) And (5) carrying out inspection and evaluation on the precipitation result after deviation correction.

2. The method of claim 1, wherein the step 1) of selecting valid meteorological feature information, calculating correlation between each meteorological feature and the precipitation observed at the ground station by using a pearson correlation coefficient, setting a threshold value to determine all features of the meteorological features larger than the threshold value as valid features, and calculating the pearson correlation coefficient by:

in the formula

Is a characteristic value

The covariance of (a) of (b),

respectively represent the characteristics

The variance of (c).

3. The method for correcting the deviation of numerical mode precipitation based on deep learning of claim 1, wherein the steps of preparing the data set and normalizing the features in the step 2) are as follows:

(a) The selected active features are segmented into the study area and spread around to take into account the effect of the surrounding environment on precipitation in the target area.

4. (b) Carrying out normalization operation in the segmented characteristics, wherein a Z-SCORE normalization method is used except for precipitation characteristics, and the calculation formula is as follows:

the data features are normalized to the same dimension, wherein

Is the overall average of each of the elements,

is the standard deviation of the overall data;

and because the precipitation is distributed in a discrete type, and the magnitude of the precipitation is greatly expanded, a log normalization mode is adopted, and a calculation formula for relieving the condition that the precipitation has a large difference is as follows:

the method for correcting the deviation of numerical mode precipitation based on deep learning of claim 1, wherein the spatiotemporal prediction network in step 3) considers the influence of ambient weather environment information on a target area, uses MotionRNN to extract the change of the weather characteristics in the past period of time and infer the change trend of the target area in the future for the constructed weather historical sequence information, and uses deconvolution to obtain the spatial resolution consistent with the numerical mode precipitation.

5. The deep learning-based numerical model precipitation deviation correction method according to claim 1, wherein step 4) said future meteorological change trend is fused with prior knowledge such as numerical forecast precipitation and DEM and is input to a value space correction module, which is divided into two branches:

global branch and local branch compromise local precipitation and global precipitation, and the training of whole model uses the weight loss function to optimize the parameter, alleviates precipitation distribution extremely unbalanced problem, and the expression is:

wherein N represents the number of rain gauges in the area,

and

respectively representing the actual longitude and latitude and the precipitation after correction,

the weight corresponding to the rainfall intensity of the ith station is that the weight of heavy rain is greater than that of light rain, which is specifically as follows:

and finally, evaluating the performance of the model by using the optimal model and a verification set.

Images