CN115859797A - Satellite quantitative precipitation estimation method based on deep learning - Google Patents
Satellite quantitative precipitation estimation method based on deep learning Download PDFInfo
- Publication number
- CN115859797A CN115859797A CN202211486452.6A CN202211486452A CN115859797A CN 115859797 A CN115859797 A CN 115859797A CN 202211486452 A CN202211486452 A CN 202211486452A CN 115859797 A CN115859797 A CN 115859797A
- Authority
- CN
- China
- Prior art keywords
- data
- satellite
- precipitation
- training
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a satellite quantitative precipitation estimation method based on deep learning, and belongs to the technical field of meteorological satellites. Firstly, historical satellite observation data of a region to be estimated for precipitation and historical regional precipitation data measured by GPM-IMERG half hour by half hour are collected, and a training data set is constructed. And then constructing a deep learning network model, and training the deep learning network model by using a training data set to obtain a satellite quantitative rainfall estimation model. And finally, processing new satellite observation data at a certain moment to obtain a part of regional data, taking the regional data as input data of a satellite quantitative precipitation estimation model, outputting a regional quantitative precipitation estimation result through the satellite quantitative precipitation estimation model, and visualizing the result into a rainfall intensity distribution graph. The method improves the accuracy of precipitation estimation by utilizing the nonlinear mapping capability of deep learning.
Description
Technical Field
The invention belongs to the technical field of meteorological satellites, and particularly relates to a satellite quantitative precipitation estimation method based on deep learning.
Background
Precipitation is a fundamental component of global water circulation and is a key hydrologic variable of water circulation in meteorology, climate and hydrology. Accurate estimation of precipitation and its regional and global distribution has long been a challenging scientific goal. The rainfall is used as a part of water circulation, and water is continuously conveyed from the sea to the land through the life process of the rainfall, so that water resources are continuously updated, the material energy exchange among regions is promoted, and the rainfall plays an important role in shaping the surface morphology diversity and the biological diversity. Meanwhile, the generation of precipitation is accompanied by the absorption and release of latent heat, and the energy transfer among regions and among atmospheric junctions is also influenced critically. In addition, accurate acquisition of regional rainfall distribution has important significance for watershed hydrological analysis, water resource planning and management, hydraulic engineering design and scheduling, flood drought monitoring, geological disaster early warning and the like.
With the successful emission of a large number of satellites and the rapid development of remote sensing technology, people are increasingly simple to acquire remote sensing data, and the satellite multi-channel combined observation of cloud and precipitation becomes a research hotspot in the field of current atmospheric remote sensing and climate change. Due to the obvious advantages of wide range, short period, large information amount, low cost and the like, the satellite observation is applied to the aspects of life and plays an important role in monitoring weather and early warning of disastrous weather.
The satellite remote sensing is the only means which can realize global precipitation observation at present, the meteorological satellite observes the earth above the earth, the method is not limited by geographical and natural conditions, the development and evolution of a wide-range whole-process monitoring cloud system can be realized, the defect that an observation station network observes precipitation can be effectively overcome, a detector carried by the satellite is used for monitoring and inverting a cloud cluster, and a satellite inversion precipitation result can be obtained.
All weather departments actively research and develop own quantitative precipitation products, such as representative global precipitation products PERSIANN, PERSIANN-CCS, CHIRPS, TRMM, CMORPH GPM-IMERG based on satellite observation and quantitative precipitation estimation products of China weather satellites.
The existing satellite quantitative precipitation estimation product has more phenomena of error estimation and omission estimation, resulting in misalignment of the obtained precipitation area, poor precipitation hit rate performance, low reliability, low precision when the specific precipitation is estimated, and inaccurate estimation result.
Disclosure of Invention
The invention aims to solve the problem of inaccurate regional precipitation estimation and provides a satellite quantitative precipitation estimation method based on deep learning.
The satellite quantitative precipitation estimation method based on deep learning comprises the following specific technical scheme:
collecting historical satellite observation data of a precipitation amount area to be estimated and historical precipitation data measured by GPM-IMERG half hour by half hour to construct a training data set;
the detailed steps for constructing the training data set are as follows:
step 101, according to a conversion relation given by a national satellite meteorological center, mutual conversion of a column number and longitude and latitude is carried out on historical satellite observation data to obtain satellite observation data based on the longitude and latitude.
The static orbit nominal projection of the historical satellite observation data is defined by CGMSRIT/HRIT global specification, the geographic coordinates are calculated based on a WGS84 reference ellipsoid to obtain a formula for converting the row number into the longitude and the latitude, and the row number is converted into the longitude and the latitude through the formula;
the longitude and latitude coordinate (lat, lon) conversion formula is as follows:
wherein λ is D The longitude of the satellite subsatellite point is shown, ea represents the distance of the earth major semi-axis, and eb represents the distance of the earth minor semi-axis; r 1 、R 2 And R 3 Are all intermediate variables, expressed as:
R 1 ＝d-R n ×cos(x)×cos(y)
R 2 ＝R n ×sin(x)×cos(y)
R 3 ＝-R n ×sin(y)
wherein d is the distance from the geocenter to the satellite; x, y and R n Is an intermediate variable, expressed as:
where CFAC is the column scale factor, COFF is the column offset, LFAC is the row scale factor, LOFF is the row offset, l is the nominal column number, and c is the nominal row number. R d For intermediate variables, the formula is as follows:
102, selecting a satellite viewing range in the longitude and latitude range of the area to be estimated from the satellite viewing data based on the longitude and latitudeMeasuring all channels of the data to obtain a full-channel data set, wherein F = { F = { (F) 1 ，F 2 ，F 3 ，…，F n In which F n Representing the data of the nth channel.
The full-channel data comprises data of visible light, short wave infrared, medium wave infrared, long wave infrared and other wave bands;
step 103, performing calibration operation on each channel data in the full channel data set F to obtain a calibrated satellite observation channel data set Y = { Y = Y = 1 ，Y 2 ，Y 3 ，…，Y n In which Y is n Representing the scaled data of the nth channel;
step 104, for the satellite observation channel data set Y = { Y = { (Y) 1 ，Y 2 ，Y 3 ，…，Y n Projection conversion is carried out, and a satellite observation data set A = { A } in an equidistant cylindrical projection format can be obtained 1 ，A 2 ，A 3 ，…，A n In which A is n Data representing the projection-converted nth channel;
105, screening out whole-point and half-point observation data from the satellite observation data set A because the time resolution of regional precipitation data measured by the GPM-IMERG is half an hour, repeatedly carrying out calibration and projection conversion operations, and obtaining a grid satellite observation data file with equal longitude and latitude through regional cutting;
step 106, carrying out time relation matching on regional precipitation data measured by the GPM-IMERG and grid satellite observation data with equal longitude and latitude to obtain GPM precipitation data corresponding to satellite observation time;
step 107, carrying out interpolation and turning processing on the GPM precipitation data obtained in the step 106 to obtain precipitation data with the same longitude and latitude as those of the grid satellite observation data file with the same longitude and latitude, and storing the precipitation data in a label directory, namely GPM label data in the training data set;
step 108, flattening the GPM label data file into one-dimensional data, and splicing all the one-dimensional data; meanwhile, splicing all data files of the satellite observation data set A together according to different observation channels; and then, performing file splicing on the two spliced files to obtain an n +1 row of data files containing the time period required by the training data set, wherein the first n rows are satellite channel data and the n +1 th row is GPM precipitation data.
Step 109, transposing the data files of the n +1 rows to obtain n +1 rows of data, converting the data into a pandas format, and obtaining correlation coefficients between different channel data of the satellite and precipitation label data through the pandas to obtain a thermodynamic diagram reflecting the data correlation relationship;
the correlation coefficient calculation formula is as follows:
the formula can be simplified as:
wherein P represents n rows of satellite channel data, Q represents precipitation label data, and cov represents covariance; sigma P Representing the standard deviation, σ, of the satellite channel data Q A standard deviation representing precipitation signature data; k represents the total number of a column of data; p is a radical of ni I data, q, referring to the nth column of satellite channel data i Indicating ith data in the precipitation label data;
represents the mean of n columns of satellite channel data,
represents the mean of the precipitation signature data.
And the corr command of the pandas can obtain the correlation between the two rows of data to generate a thermodynamic diagram.
Step 110, screening m columns of satellite channel data with highest correlation with the column of the precipitation data according to the thermodynamic diagram to form a data set B = { B = 1 ,B 2 ,B 3 ,...,B m In which B is 1 ,B 2 ,B 3 ,...,B m And respectively representing the screened m rows of satellite channel data, and storing the data set B into a training data file directory of a training data set. At this point, the stored training file directory and the stored label directory respectively correspond to the training data and the label data and jointly form a training data set.
Secondly, constructing a deep learning network model of the satellite quantitative precipitation estimation method;
the constructed deep learning network model is an Encoder-Decoder network structure, wherein the Encoder is a module consisting of a convolution layer, a residual error unit and a down-sampling operation, and the Decoder is a module consisting of an anti-convolution layer and an up-sampling operation.
The convolution layer comprises convolution, batch standardization and three operation contents of an activation function, the down-sampling operation comprises the convolution, batch standardization and three operation contents of the activation function, the up-sampling comprises deconvolution, batch standardization and three contents of the activation function, and the residual error unit comprises two convolution layers and a short jump connection to sum the input content and the output content;
the Encode-Decoder network structure in the deep learning network model enables the network to fuse the feature maps of the corresponding positions of the Encoder on the channels in the up-sampling process of each level of the Decoder through skip-connection, and parameter sharing is achieved.
Training the training data set by using a deep learning network model to obtain a satellite quantitative precipitation estimation model;
the method comprises the following specific steps:
training parameters of the deep learning network in a variable step learning rate mode by taking training data and label data in a training data set as input of a deep learning network model and a mean square loss function as a loss function of the model, setting an initial learning rate by an optimizer in the training process by adopting an Adam gradient descent method, and setting parameters to represent the degree of learning rate attenuation. And obtaining an optimal parameter model after loop iteration. And storing the deep learning network model applying the optimal parameters as a satellite quantitative precipitation estimation model.
The mean square loss function is determined according to the difference between data obtained after training data pass through a deep training network model and label data, and the calculation formula is as follows:
wherein f is output obtained after the training data passes through the deep learning network, g is label data, h is the number of the training data, and f j ，g j For the jth data in f and g, respectively.
Processing new satellite observation data at a certain moment to obtain data with the same shape and longitude and latitude range as the training data, and performing calibration and projection processing to obtain a region data;
and step five, taking the region data as input data of the satellite quantitative precipitation estimation model, outputting the satellite quantitative precipitation estimation model as a region quantitative precipitation estimation result, and visualizing the result into a rainfall intensity distribution diagram.
The invention has the advantages that:
(1) The method uses satellite observation data, utilizes the nonlinear mapping capability of deep learning, takes the GPM-IMERG precipitation product which is most widely used in the world at present as a label, constructs a model reflecting the internal relation between the satellite observation data and precipitation, and realizes that the precipitation information of a required area can be obtained only by the satellite observation data at a required moment.
(2) The method expands the data use of meteorological satellites, improves the accuracy of precipitation estimation, and has the advantages of rapidness and less occupied computing resources compared with other similar products.
Drawings
FIG. 1 is a flow chart of a method for estimating quantitative precipitation of a satellite based on deep learning according to the present invention;
FIG. 2 is a thermodynamic diagram obtained in an embodiment of the present invention;
fig. 3 is a schematic diagram of a deep learning network structure according to an embodiment of the present invention.
Detailed Description
In order to clearly and completely present the advantages, technical features and objects of the present invention, the technical details of the present invention will be described in detail and completely with reference to the accompanying drawings and embodiments. It should be noted that the illustrated embodiments are included in part of the present disclosure, and not in all aspects of the present disclosure, and other embodiments obtained by others skilled in the art without making any inventive breakthrough are included in the scope of the present disclosure.
The satellite applied in the embodiment is a Fengyun No. 4A satellite which is marked as FY-4A.
The satellite quantitative precipitation estimation method based on deep learning comprises the following steps:
in step 101, because the historical satellite observation data does not include longitude and latitude data, the data row number and the longitude and latitude need to be converted mutually according to the conversion relation given by the national satellite meteorological center, so as to obtain the longitude and latitude information.
The static orbit nominal projection of the historical satellite observation data is defined by CGMSRIT/HRIT global specification, a formula for converting the row and column numbers into the longitudes and latitudes can be obtained by calculating the geographic coordinates based on a WGS84 reference ellipsoid, and the row and column numbers are converted into the longitudes and latitudes through the formula;
102, selecting all channels of satellite observation data in a required longitude and latitude range aiming at any satellite observation data based on the longitude and latitude information obtained by conversion to obtain a full-channel data set, wherein F = { F = { (F) } 1 ，F 2 ，F 3 ，…，F n In which F n The data of the nth channel is represented, and the F set comprises data of a visible light channel, a near infrared channel, a short wave infrared channel, a medium wave infrared channel, a water vapor channel and a long wave infrared channel;
103, scaling the full-channel data set F, wherein the visible light is obtained by the lookup tableReflectivity and infrared channel brightness temperature data are obtained, and a calibrated satellite observation channel data set Y = { Y = is obtained 1 ，Y 2 ，Y 3 ，…，Y n In which Y is n Representing the scaled data of the nth channel;
step 104, for the satellite observation channel data set Y = { Y = { (Y) 1 ，Y 2 ，Y 3 ，…，Y n Projection conversion is carried out, and a satellite observation data set A = { A } in an equidistant cylindrical projection format can be obtained 1 ，A 2 ，A 3 ，…，A n In which A is n Data representing the projection-converted nth channel;
105, screening FY-4A satellite observation data of the whole point and the half point from a satellite observation data set A, repeatedly carrying out calibration and projection conversion operations, and obtaining an equal-longitude-latitude grid satellite observation data file through regional cutting because the time resolution of the precipitation data in the GPM-IMERG region is half an hour;
step 106, carrying out time relation matching on the GPM-IMERG precipitation product data and the equal longitude and latitude grid satellite observation data to obtain GPM precipitation data at corresponding time;
step 107, because the spatial resolution of the satellite observation data is higher, and the spatial resolution of the GPM data is lower, in order to ensure the high resolution of the result, the GPM precipitation data needs to be interpolated on the same longitude and latitude grid as the satellite observation data by a bilinear interpolation method to obtain data of the same longitude and latitude area as the satellite observation data of the longitude and latitude grid in the step 105, namely the tag data in the training data set, and the tag data is stored in a tag directory;
step 108, flattening the label data obtained in the step 107 into a one-dimensional label, and then splicing all the files; meanwhile, the data set a = { a ] obtained in step 104 1 ，A 2 ，A 3 ，…，A n Splicing data in all files according to different observation channels; and finally, splicing the two spliced files to obtain an n + 1-dimensional data file containing all required time periods of the training data set.
And step 109, transposing the n + 1-dimensional data obtained in the step 108 to obtain n +1 column data, converting the column data into a pandas format, and obtaining correlation coefficients between different channel data and label precipitation data through a pandas command to obtain a thermodynamic diagram showing the data correlation relationship.
The correlation coefficient calculation formula is as follows:
the formula can be simplified as:
wherein P represents n rows of satellite channel data, and Q represents precipitation label data; k represents the total number of a column of data; p is a radical of ni I data, q, referring to the nth column of satellite channel data i Indicating ith data in the precipitation label data;
represents the mean value of n columns of satellite channel data, <' > or>
Represents the mean of the precipitation signature data.
The corr command of pandas can obtain the correlation relationship between every two of the n +1 lines of data, and a thermodynamic diagram is generated.
Step 110, screening out channels with high correlation with the row of the precipitation data according to the thermodynamic diagram, and extracting the screened channel data to obtain a data set B = { B = { B 1 ,B 2 ,B 3 ,...,B m In which B is 1 ,B 2 ,B 3 ,...,B m And respectively representing the screened m channel data instead of the previous m satellite observation channel data, and storing the file extracted at each moment into a training data file directory of a training data set. The stored training file directory and label directory respectively correspond to the training data and label data in the training data set, and jointly formForming a training data set.
The thermodynamic diagram displays the correlation between each channel column and the precipitation column, and channels with correlation coefficients larger than about 0.3 are selected to form a data set B.
the constructed deep learning network model comprises an Encoder-Decoder network structure and convolution, up-sampling, down-sampling operation and residual error units contained in the network structure;
the Encoder is a module consisting of a convolutional layer, a residual error unit and a downsampling operation; the Decoder is a module consisting of an deconvolution layer and an upsampling operation. The convolution layer comprises convolution, batch standardization and three operation contents of an activation function, the down-sampling operation comprises the convolution, batch standardization and three operation contents of the activation function, the up-sampling comprises the contents of deconvolution, batch standardization and three operation contents of the activation function, and the residual error unit comprises two convolution layers and a short jump connection to sum the input content and the output content;
an Encoder-Decoder network structure in the deep learning network model enables a network to fuse feature maps of positions corresponding to an Encoder on a channel in the up-sampling process of each level of a Decoder through skip-connection, and parameter sharing is achieved;
according to the training data set in step 110, a loss function is determined according to a difference between data obtained by passing training data through a network and tag data, and the final loss function is determined as a mean square loss function, and a calculation formula is as follows:
wherein f is output obtained after the training data passes through the deep learning network, g is label data, h is the number of the training data, and f j ，g j For the jth data in f and g, respectively.
Training data and label data in a training data set are used as input of a deep learning network model, a mean square loss function is used as a loss function of the model, an optimizer in the model training process adopts an Adam gradient descent method, parameter training in the deep learning network uses a variable step learning rate mode, an initial learning rate is set, and parameters are set to represent the degree of learning rate attenuation. And (4) obtaining an optimal parameter model after cyclic iteration, and storing the deep learning model of the parameter as a satellite quantitative precipitation estimation model. By trying different initial learning rates and attenuation parameters, a proper learning rate and attenuation parameter scheme is finally obtained, so that the convergence rate of the model is guaranteed, the model is prevented from falling into local optimization, and an optimal solution cannot be obtained.
Step 4, processing new satellite observation data at a certain moment to obtain data with the same shape and longitude and latitude range as the training data, and obtaining a region data after calibration and projection processing;
and 5, taking the region data obtained in the step 4 as input data of the satellite quantitative precipitation estimation model, and outputting a quantitative precipitation estimation result of the region by the satellite quantitative precipitation estimation model.
Examples
The satellite observation data adopted in this embodiment is, for example, a stationary orbit radiation imager of wind cloud number four a star, and L1-level full-disc data of a china area observed by using the wind cloud number four a star includes 2 visible light channels, 1 near-infrared channel, 3 short-wave infrared channels, 2 medium-wave infrared channels, 2 water vapor channels, and 4 long-wave infrared channels. The longitude and latitude of the satellite sub-satellite point center of the data are 104.7 degrees, the resolution is 4km, and nominal projection is adopted. The label data adopts a GPM-IMERG (general purpose computing-inertial measurement System) major thrust satellite precipitation product of NASA (national aeronautics administration) to gradually reduce precipitation for half an hour, and regional precipitation data with the same longitude and latitude range and data format as the region to be estimated are obtained after calibration projection. The latitude and longitude range to be estimated adopted by the embodiment is 110-122.75 degrees of east longitude and 22-34.75 degrees of north latitude.
The flowchart of the steps of the algorithm for quantitatively estimating the precipitation in the region to be estimated in this embodiment is shown in fig. 1, and the specific implementation steps are as follows:
step 101, performing interconversion of column numbers and longitude and latitude on historical satellite observation data of an area to be estimated to obtain satellite observation data based on the longitude and latitude.
Because the observation data of the wind and cloud satellite stationary orbit radiation imager does not include longitude and latitude data, the historical satellite observation data needs to be converted into data row numbers and longitude and latitude according to a conversion relation given by a national satellite meteorological center, and longitude and latitude information is obtained. The static orbit nominal projection of the historical satellite observation data is defined by adopting CGMSRIT/HRIT global specification, a formula for converting row and column numbers into longitude and latitude can be obtained by calculating the geographic coordinates based on a WGS84 reference ellipsoid, and the row and column numbers are converted into the longitude and latitude through the formula, and the specific calculation is as follows:
wherein λ is D Is the longitude, lambda, of the satellite's subsatellite point D =104.7.ea represents the earth half-length distance, ea =6378.137km; eb denotes the earth minor semi-axis distance, eb =6356.7523km; r 1 、R 2 And R 3 Are all intermediate variables, expressed as:
R 1 ＝d-R n ×cos(x)×cos(y)
R 2 ＝R n ×sin(x)×cos(y)
R 3 ＝-R n ×sin(y)
wherein d is the distance from the earth's center to the satelliteD =42164km. x, y and R n Is an intermediate variable, expressed as:
wherein, for the data of 4km resolution in the present embodiment, CFAC is a column scale factor, CFAC =10233137; COFF is column offset, COFF =1373.5; LFAC is the row scale factor, LFAC =10233137; LOFF is row offset, LOFF =1373.5.l is the nominal column number and c is the nominal row number. Intermediate variable R d The formula is as follows:
102, selecting data of all channels of a grade-A star L1 of Fengyun four in an east longitude 110-122.75 degree and north latitude 22-34.75 degree region from the satellite observation data based on longitude and latitude obtained in the step 101 to obtain all 14 channel data sets, wherein F = { F = { (F) } 1 ，F 2 ，F 3 ，…，F 14 }，F 1 ，F 2 ，F 3 ，…，F 14 Respectively representing observation data of 14 channels of a satellite, specifically 2 visible light channels, 1 near infrared channel, 3 short wave infrared channels, 2 medium wave infrared channels, 2 water vapor channels and 4 long wave infrared channels;
the space resolution of the latitude and longitude grid of the Fengyun IV A star is 0.05 degrees multiplied by 0.05 degrees, and the data size is 14 multiplied by 256 pixel points.
103, carrying out calibration operation on the full channel data set F to obtain calibrated satellite observation channel dataSet Y = { Y 1 ，Y 2 ，Y 3 ，…，Y 14 }，Y 1 ，Y 2 ，Y 3 ，…，Y 14 And respectively representing the data after calibration of 14 channels, wherein the reflectivity of the visible light is obtained by looking up a calibration table, and the brightness temperature data is obtained by the infrared channel.
Step 104, for the satellite observation channel data set Y = { Y = { (Y) 1 ，Y 2 ，Y 3 ，…，Y 14 Projection conversion is carried out, and a satellite observation data set A = { A } in an equidistant cylindrical projection format can be obtained 1 ，A 2 ，A 3 ，…，A 14 }，A 1 ，A 2 ，A 3 ，…，A 14 Respectively representing data corresponding to 14 channels after projection conversion;
105, screening satellite observation data of a whole point and a half point from a satellite observation data set A, repeatedly carrying out calibration and projection conversion operations, and obtaining an equal-longitude-latitude grid satellite observation data file through region cutting because the time resolution of the GPM-IMERG data is half an hour;
step 106, matching the GPM-IMERG precipitation product data with the grid satellite observation data with equal longitude and latitude in a time relation to obtain GPM precipitation data corresponding to satellite observation time;
step 107, carrying out interpolation and turnover processing on the GPM precipitation data obtained in the step 106 to obtain precipitation data with the same longitude and latitude as the satellite observation data, and storing the precipitation data in a label directory, namely GPM label data in the training data set;
the specific process of interpolation and inversion processing is as follows:
firstly, because the longitude and latitude in the GPM-IMERG data storage format are different from the satellite observation data storage mode, the GPM precipitation data needs to be transposed, so that the longitude and latitude formats of the two data are kept consistent;
then, cutting the converted GPM precipitation data to obtain an area with a range larger than the latitude and longitude range of the satellite observation data, eliminating the marginal problem of an interpolation result, and interpolating the GPM precipitation data with the larger range to a latitude and longitude grid of the satellite observation data;
and finally, flip overturning the interpolated GMP precipitation data, eliminating a file missing value and a value smaller than 0, and cutting again to obtain GMP precipitation data in longitude and latitude areas equal to satellite observation data.
Step 108, flattening the GMP label data into one-dimensional data, and then splicing all the one-dimensional data; meanwhile, the satellite observation data set a = { a ] obtained in step 104 1 ，A 2 ，A 3 ，…，A 14 Splicing all the data files according to different observation channels; finally, the two spliced files are spliced together again to obtain an n + 1-dimensional data file containing all the required time periods of the training data set, which is a 15-dimensional data set in this embodiment.
Step 109, transposing the 15-dimensional data obtained in step 108 to obtain 15 rows of data, converting the data into a pandas format, and obtaining correlation coefficients between different channel data and label precipitation data through a pandas command to obtain a thermodynamic diagram reflecting the data correlation;
the correlation coefficient calculation formula is as follows:
the formula can be simplified as:
wherein P represents n rows of satellite channel data, and Q represents precipitation label data; k represents the total number of a column of data; p is a radical of ni I data, q, referring to the nth column of satellite channel data i Indicating ith data in the precipitation label data;
represents the mean value of n columns of satellite channel data, <' > or>
Represents the mean of the precipitation signature data.
The corr command of the pandas can obtain the correlation between every two of the 15 rows of data to generate a thermodynamic diagram, as shown in fig. 2, the numbers 0-13 represent 14 satellite channel data, and the number 14 represents precipitation data.
Step 110, screening out channels with high correlation with the row where the precipitation data are located according to the thermodynamic diagram, and extracting the screened channel data to obtain a data set B = { B = { (B) 1 ,B 2 ,B 3 ,...,B 12 }，B 1 ,B 2 ,B 3 ,...,B 12 Respectively representing 12 channel data after screening, but not referring to the first 12 satellite observation channel data, and the corresponding channel codes are respectively 1,2,3,4,7,8,9, 10, 11, 12, 13 and 14. The files extracted at each moment corresponding to step 105 are stored in a training data file directory of the training data set. The stored training file directory and the stored label directory respectively correspond to training data and label data to jointly form a training data set.
the deep learning network model is an Encoder-Decoder network structure, as shown in FIG. 3. The model structure is divided into an Encoder and a Decoder section having four downsamplings and four upsamplings, respectively, corresponding to the left and right sides in fig. 3.
In the Encoder part, input data are processed wind cloud four-star A star observation images, and training data B = { B = { B = 1 ,B 2 ,B 3 ,...,B 12 Contains 12 × 256 × 256 pixels. The input data is adjusted according to the number of data channels through the convolutional layer, so that information loss caused in the input layer is avoided. The Encoder portion contains one residual unit in each layer, allowing the network to increase the depth of the network while avoiding information loss. During downsampling, the deep learning network uses convolution with the step size of 2 to replace the pooling operation for downsampling, so that information loss caused in the pooling process is avoided. Meanwhile, a mode of combining a large convolution kernel and a small convolution kernel is used, and in the last two down-sampling processesAnd extracting features by using a convolution kernel with the size of 11 to obtain a larger receptive field and extract more feature information.
In the Decoder section, the deep learning network sends the original information to the Decoder section using skip-connection, and connects the original information with the up-sampled information. The residual module and the upsampling layer restore the data size to the original size, and finally, the number of data channels is changed to 1 layer (1 × 256 × 256 pixels) by the 1 × 1 convolutional layer. Each pixel point corresponds to the precipitation intensity information representing the current position, and finally, pixel-by-pixel precipitation intensity estimation is achieved, and the change of the data size in the Encoder and Dncoder processes is shown as a digital mark in FIG. 3.
inputting training data and label data into a deep learning neural network, determining a loss function according to a difference value between the data obtained after the training data passes through the network and the label data, and finally determining the loss function as a mean square loss function, wherein a calculation formula is as follows:
training data and label data in a training data set are used as input of a deep learning network model, a mean square loss function is used as a loss function of the model, an optimizer in the model training process adopts an Adam gradient descent method, parameter training in the deep learning network uses a variable step learning rate mode, an initial learning rate is set, and parameters are set to represent the degree of learning rate attenuation. Obtaining an optimal parameter model after cyclic iteration, and storing a deep learning model of the parameter as a satellite quantitative precipitation estimation model;
and 4, selecting any new moment from the wind-cloud-IV-star-A observation data, wherein the moment is not included in the training data moment in the training data set, obtaining processing data with the same format as the training data through steps 102-104, inputting the processing data serving as a model into a saved satellite quantitative precipitation estimation model with optimal parameters, outputting grid data which is required regional precipitation estimation, wherein the grid size is consistent with the size of an input region and is still 256 multiplied by 256, and visualizing the output grid data to obtain a regional precipitation estimation diagram.
The satellite quantitative precipitation estimation method based on deep learning achieves the following technical effects:
1. the GPM-IMERG data is subjected to interpolation on the same spatial resolution as the satellite observation data, the high-resolution data of the observation satellite is fully utilized, and the fineness of quantitative precipitation estimation is improved.
2. Training the integrated satellite observation data and rainfall label data, improving the reliability of rainfall estimation and saving computing resources on the basis of ensuring rapidity.
3. For the multichannel characteristic of satellite observation data, a deep neural network capable of being flexibly butted according to the number of input channels is designed, the nonlinear relation between the satellite observation data and precipitation can be effectively excavated, and the accuracy of precipitation estimation is improved.
The embodiments disclosed above are illustrative of the ways to make and use the invention by those skilled in the art. Modifications and simulation of the embodiments of the invention will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention.
Claims (6)
1. A satellite quantitative precipitation estimation method based on deep learning is characterized by comprising the following specific technical scheme:
collecting historical satellite observation data of a precipitation amount area to be estimated and historical precipitation data measured by using a GPM-IMERG half hour by half hour to construct a training data set;
the detailed steps for constructing the training data set are as follows:
step 101, performing interconversion of a column number and longitude and latitude on historical satellite observation data according to a conversion relation given by a national satellite meteorological center to obtain satellite observation data based on the longitude and latitude;
102, selecting all channels of satellite observation data in a latitude and longitude range of an area to be estimated from the satellite observation data based on the latitude and longitude to obtain a full-channel data set, wherein F = { F = { 1 ，F 2 ，F 3 ，…，F n }；
Wherein F n Data representing an nth channel;
step 103, performing calibration operation on each channel data in the full channel data set F to obtain a calibrated satellite observation channel data set Y = { Y = Y = 1 ，Y 2 ，Y 3 ，…，Y n }；
Wherein Y is n Representing the scaled data of the nth channel;
step 104, for the satellite observation channel data set Y = { Y = { (Y) 1 ，Y 2 ，Y 3 ，…，Y n Projection conversion is carried out, and a satellite observation data set A = { A } in an equidistant cylindrical projection format can be obtained 1 ，A 2 ，A 3 ，…，A n }；
Wherein A is n Data representing the projection-converted nth channel;
105, screening integral point and half point observation data from the satellite observation data set A, repeatedly carrying out calibration and projection conversion operations, and obtaining an equal longitude and latitude grid satellite observation data file through regional cutting;
step 106, matching the historical precipitation data measured by the GPM-IMERG with the grid satellite observation data with equal latitude and longitude in a time relationship to obtain the GPM precipitation data corresponding to the satellite observation time;
step 107, carrying out interpolation and turning processing on the GPM precipitation data obtained in the step 106 to obtain precipitation data with the same longitude and latitude as those of the grid satellite observation data file with the same longitude and latitude, and storing the precipitation data in a label directory, namely GPM label data in the training data set;
step 108, flattening the GPM tag data file into one-dimensional data, and splicing all the one-dimensional data; meanwhile, splicing all data files of the satellite observation data set A together according to different observation channels; then, performing file splicing on the two spliced files to obtain a data file containing the time period required by the training data set in n +1 rows;
the first n rows are satellite channel data and the (n + 1) th row is GPM precipitation data;
step 109, transposing the data files of the n +1 rows to obtain n +1 rows of data, converting the data into a pandas format, and obtaining correlation coefficients between different channel data of the satellite and precipitation label data through the pandas to obtain a thermodynamic diagram reflecting the data correlation relationship;
the correlation coefficient calculation formula is as follows:
the formula can be simplified as:
wherein P represents n rows of satellite channel data, Q represents precipitation label data, and cov represents covariance; sigma P Representing the standard deviation, σ, of the satellite channel data Q A standard deviation representing precipitation signature data; k represents the total number of a column of data; p is a radical of ni I data, q, referring to the nth column of satellite channel data i Indicating ith data in the precipitation label data;
represents the mean value of n columns of satellite channel data, <' > or>
A mean value representing precipitation signature data;
the correlation relationship between every two rows of data can be obtained through corr commands of the pandas, and a thermodynamic diagram is generated;
step 110, screening m rows of satellite channel data with the highest correlation with the row of the precipitation data according to the thermodynamic diagram to form a data set B = { B = { (B) 1 ,B 2 ,B 3 ,...,B m }; storing the data set B into a training data file directory; at this point, the stored training file directory and the stored label directory respectively correspond to training data and label data and jointly form a training data set;
wherein B is 1 ,B 2 ,B 3 ,...,B m Respectively representing the screened m rows of satellite channel data;
secondly, constructing a deep learning network model of the satellite quantitative precipitation estimation method;
the constructed deep learning network model is an Encoder-Decoder network structure, wherein the Encoder is a module consisting of a convolution layer, a residual error unit and a down-sampling operation, and the Decoder is a module consisting of an anti-convolution layer and an up-sampling operation;
training the training data set by using a deep learning network model to obtain a satellite quantitative precipitation estimation model;
the method specifically comprises the following steps:
training parameters of the deep learning network in a variable step learning rate mode by taking training data and label data in a training data set as input of a deep learning network model and a mean square loss function as a loss function of the model, setting an initial learning rate by an optimizer in the training process by adopting an Adam gradient descent method, and setting parameters to represent the degree of learning rate attenuation; obtaining an optimal parameter model after loop iteration; saving the deep learning network model applying the optimal parameters as a satellite quantitative precipitation estimation model;
processing new satellite observation data at a certain moment to obtain data with the same shape and longitude and latitude range as the training data, and performing calibration and projection processing to obtain a region data;
and fifthly, taking the region data as input data of the satellite quantitative precipitation estimation model, outputting the satellite quantitative precipitation estimation model as a region quantitative precipitation estimation result, and visualizing the result into a precipitation intensity distribution diagram.
2. The method for estimating quantitative precipitation of satellite based on deep learning of claim 1, wherein the historical satellite observation data is interconverted by column number and longitude and latitude according to the following formula:
wherein λ is D The longitude of the satellite subsatellite point is shown, ea represents the distance of the earth major semi-axis, and eb represents the distance of the earth minor semi-axis; r 1 、R 2 And R 3 Are all intermediate variables, expressed as:
R 1 ＝d-R n ×cos(x)×cos(y)
R 2 ＝R n ×sin(x)×cos(y)
R 3 ＝-R n ×sin(y)
wherein d is the distance from the geocenter to the satellite; x, y and R n Is an intermediate variable, expressed as:
where CFAC is the column scale factor, COFF is the column offset, LFAC is the row scale factor, and LOFF is the row scale factorOffset, l is the nominal column number, c is the nominal row number; r d For intermediate variables, the formula is as follows:
3. the method for satellite quantitative precipitation estimation based on deep learning of claim 1, wherein the full channel data comprises data of visible light, short wave infrared, medium wave infrared and long wave infrared bands.
4. The method of claim 1, wherein the convolutional layer comprises three operation contents of convolution, batch normalization and activation function, the down-sampling operation comprises three operation contents of convolution, batch normalization and activation function, the up-sampling comprises three operation contents of deconvolution, batch normalization and activation function, and the residual unit comprises two convolutional layers and a short-jump connection to sum the input content and the output content.
5. The deep learning-based satellite quantitative precipitation estimation method according to claim 1, wherein the Encoder-Decoder network structure enables a network to fuse feature maps of a position corresponding to an Encoder on a channel in an up-sampling process of each stage of a Decoder through skip-connection, so as to realize parameter sharing.
6. The deep learning-based satellite quantitative precipitation estimation method according to claim 1, wherein the mean square loss function is determined according to a difference between data obtained by training data through a deep training network model and tag data, and a calculation formula is as follows:
wherein f is output obtained after the training data passes through the deep learning network, g is label data, h is the number of the training data, and f j ，g j For the jth data in f and g, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211486452.6A CN115859797A (en) | 2022-11-24 | 2022-11-24 | Satellite quantitative precipitation estimation method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211486452.6A CN115859797A (en) | 2022-11-24 | 2022-11-24 | Satellite quantitative precipitation estimation method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115859797A true CN115859797A (en) | 2023-03-28 |
Family
ID=85666181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211486452.6A Pending CN115859797A (en) | 2022-11-24 | 2022-11-24 | Satellite quantitative precipitation estimation method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115859797A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383701A (en) * | 2023-04-13 | 2023-07-04 | 中国人民解放军国防科技大学 | Microwave link rainy period detection method based on learning reconstruction |
-
2022
- 2022-11-24 CN CN202211486452.6A patent/CN115859797A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383701A (en) * | 2023-04-13 | 2023-07-04 | 中国人民解放军国防科技大学 | Microwave link rainy period detection method based on learning reconstruction |
| CN116383701B (en) * | 2023-04-13 | 2024-01-26 | 中国人民解放军国防科技大学 | Microwave link rainy period detection method based on learning reconstruction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chin et al. | A multi-scale high-resolution analysis of global sea surface temperature | |
| MacCallum et al. | Surface water temperature observations of large lakes by optimal estimation | |
| Weng et al. | Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data | |
| Wan | MODIS land surface temperature products users’ guide | |
| Ma et al. | An updated moving window algorithm for hourly-scale satellite precipitation downscaling: A case study in the Southeast Coast of China | |
| Song et al. | An improved surface soil moisture downscaling approach over cloudy areas based on geographically weighted regression | |
| Hossain et al. | Improving flood forecasting in international river basins | |
| Yoo et al. | Spatial downscaling of MODIS land surface temperature: Recent research trends, challenges, and future directions | |
| Kilibarda et al. | Global geographic and feature space coverage of temperature data in the context of spatio-temporal interpolation | |
| Meddens et al. | Developing 5 m resolution canopy height and digital terrain models from WorldView and ArcticDEM data | |
| CN103728609A (en) | Intersected radiometric calibration method for satellite-borne multispectral infrared sensor | |
| CN116011342B (en) | All-weather reconstruction method for high-resolution thermal infrared surface temperature | |
| Li et al. | The alternative of CubeSat-based advanced infrared and microwave sounders for high impact weather forecasting | |
| Hou et al. | On the value of available MODIS and Landsat8 OLI image pairs for MODIS fractional snow cover mapping based on an artificial neural network | |
| Liu et al. | Estimation of surface and near-surface air temperatures in arid northwest china using landsat satellite images | |
| CN117008217A (en) | Short-time strong precipitation prediction method integrating self-attention module and Unet model | |
| Zhou et al. | Reconstructing spatial–temporal continuous MODIS land surface temperature using the DINEOF method | |
| CN107576399B (en) | MODIS forest fire detection-oriented brightness and temperature prediction method and system | |
| CN115859797A (en) | Satellite quantitative precipitation estimation method based on deep learning | |
| Cao et al. | A new global gridded sea surface temperature data product based on multisource data | |
| Pour et al. | Impact of satellite-based lake surface observations on the initial state of HIRLAM. Part I: evaluation of remotely-sensed lake surface water temperature observations | |
| Gao et al. | A two-step integrated MLP-GTWR method to estimate 1 Km land surface temperature with complete spatial coverage in humid, cloudy regions | |
| He et al. | Deep learning approaches to spatial downscaling of GRACE terrestrial water storage products using EALCO model over Canada | |
| CN113237554B (en) | Method and device for generating surface temperature image under cloud and terminal equipment | |
| Xu et al. | Generating 5 km resolution 1981–2018 daily global land surface longwave radiation products from AVHRR shortwave and longwave observations using densely connected convolutional neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |