CN111859054B - Meteorological satellite data processing method and device - Google Patents
Meteorological satellite data processing method and device Download PDFInfo
- Publication number
- CN111859054B CN111859054B CN202010717906.0A CN202010717906A CN111859054B CN 111859054 B CN111859054 B CN 111859054B CN 202010717906 A CN202010717906 A CN 202010717906A CN 111859054 B CN111859054 B CN 111859054B
- Authority
- CN
- China
- Prior art keywords
- satellite data
- meteorological satellite
- meteorological
- data
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 9
- 238000012549 training Methods 0.000 claims abstract description 73
- 101100455978 Arabidopsis thaliana MAM1 gene Proteins 0.000 claims abstract description 63
- 238000000034 method Methods 0.000 claims abstract description 61
- 238000009826 distribution Methods 0.000 claims abstract description 54
- 238000010801 machine learning Methods 0.000 claims abstract description 10
- 230000008859 change Effects 0.000 claims description 38
- 238000012545 processing Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 230000007547 defect Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000013528 artificial neural network Methods 0.000 description 18
- 238000004422 calculation algorithm Methods 0.000 description 17
- 238000004458 analytical method Methods 0.000 description 15
- 230000000694 effects Effects 0.000 description 11
- 238000001514 detection method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000012800 visualization Methods 0.000 description 9
- 230000000737 periodic effect Effects 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 6
- 230000007774 longterm Effects 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 230000002060 circadian Effects 0.000 description 5
- 238000004088 simulation Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000001994 activation Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000000611 regression analysis Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000013079 data visualisation Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000011897 real-time detection Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 1
- CBENFWSGALASAD-UHFFFAOYSA-N Ozone Chemical compound [O-][O+]=O CBENFWSGALASAD-UHFFFAOYSA-N 0.000 description 1
- 125000002015 acyclic group Chemical group 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000001311 chemical methods and process Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000005433 ionosphere Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/904—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/909—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Library & Information Science (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a meteorological satellite data processing method and device. Wherein the method comprises the following steps: acquiring first meteorological satellite data in a historical time period of a ground meteorological station; training a model-independent element learning MAML model by using first meteorological satellite data to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; acquiring second meteorological satellite data in a global area; and obtaining a time sequence of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through a preset regression model. The invention solves the technical problem that the global distribution of the meteorological satellite data can not be accurately known because the global distribution mode of the meteorological satellite data is easy to have the defects of time and space sparsity by the meteorological satellite through the carried detector in the related technology.
Description
Technical Field
The invention relates to the field of data processing methods, in particular to a meteorological satellite data processing method and device.
Background
Satellites can detect the distribution of meteorological satellite data worldwide however, due to orbiting the satellites can reach the same area again after one week of travel. The time between two detections in the same region is long, and the change of meteorological satellite data between the two detections cannot be known. In addition, the meteorological satellite data is concentrated near a single orbit line in one period, and the distribution of the global meteorological satellite data cannot be known.
The current weather satellite data mainly is global distribution of the weather satellite data measured by the weather satellite through a carried detector. The data recorded by the meteorological satellites is a series of space-time data distributed along the track. After one detection, the meteorological satellite needs to travel one circle along the track to reach the same position again. The time interval between two detections of the same area is long, and the change process of the meteorological satellite data between the two detections cannot be known. The domain expert encounters two challenges when fine-grained analysis of meteorological satellite data: 1. the prior study mainly simulates the physical and chemical process of the atmosphere through a super computer: the numerical weather model takes meteorological satellite data as input, and predicts time change of the meteorological satellite data in the two detection intervals; the numerical weather model succeeds in partial meteorological satellite data, however, the numerical simulation method is not applicable to meteorological parameters with clear and unclear physical and chemical change mechanisms or influence of various factors; 2. the existing research estimates meteorological satellite data at any global position by an interpolation method, such as kriging interpolation, IDW (inverse distance weight interpolation) and the like, however, the coverage range of a meteorological satellite orbit in one period is limited, the space range required to be interpolated is larger, the data distribution in the direction perpendicular to the orbit plane is sparse, and the interpolation difficulty is larger. In addition, the interpolation method generates a smoother result, a fine structure of spatial distribution cannot be obtained, and abnormal data is covered. The abnormal data is helpful for understanding the change rule of the meteorological parameters and improving the existing theory.
Aiming at the problem that the global distribution mode of the meteorological satellite data measured by the meteorological satellite through the carried detector in the related technology is easy to have the defects of time and space sparsity, so that the global meteorological satellite data distribution cannot be accurately known, no effective solution is proposed at present.
Disclosure of Invention
The embodiment of the invention provides a processing method and a processing device for meteorological satellite data, which at least solve the technical problem that the global distribution mode of meteorological satellite data is easy to have time and space sparsity by a meteorological satellite through a carried detector in the related art, so that the global meteorological satellite data distribution cannot be accurately known.
According to an aspect of the embodiment of the present invention, there is provided a method for processing meteorological satellite data, including: acquiring first meteorological satellite data in a historical time period of a ground meteorological station; training a model by using the first meteorological satellite data, and learning an MAML model by using an independent element to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; acquiring second meteorological satellite data in a global area; and obtaining a time sequence of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through the preset regression model.
Optionally, acquiring first meteorological satellite data over a ground meteorological station history period of time includes: acquiring a similar task with the similarity to a target task being greater than a preset threshold, wherein the target task is a meteorological satellite data change of a determined preset area; acquiring relevant meteorological satellite data of the similar task; the associated meteorological satellite data is determined to be the first meteorological satellite data.
Optionally, acquiring relevant meteorological satellite data of the similar task includes: transmitting a meteorological satellite data request message to a plurality of ground stations in the global area, wherein the meteorological satellite data request message carries area identification information of the preset area; acquiring feedback information sent by the ground stations based on the meteorological satellite data request message; and determining the relevant meteorological satellite data based on the feedback information.
Optionally, training a model-independent element learning MAML model using the first meteorological satellite data to obtain initial network parameters of a predetermined regression model, including: dividing the first meteorological satellite data based on a preset dividing unit to obtain a time sequence curve which corresponds to the preset dividing unit and represents the change of the first meteorological satellite data; taking the time series curve as an input of the MAML model; obtaining the output of the MAML model; the initial network parameters are derived based on the output of the MAML model.
Optionally, before dividing the first meteorological satellite data based on a predetermined dividing unit to obtain a time sequence curve corresponding to the predetermined dividing unit and representing the change of the first meteorological satellite data, the processing method of the meteorological satellite data further includes: performing time conversion on the acquisition time of the first meteorological satellite data; wherein time converting the acquisition time of the first meteorological satellite data includes: determining a first acquisition time of the first meteorological satellite data, wherein the first acquisition time is a universal time UTC time; converting the UTC time into a local time.
Optionally, acquiring second meteorological satellite data in the global area includes: dividing the global area into a plurality of area grids based on meteorological features of different areas in the global area; determining regional weather satellite data corresponding to the plurality of regional grids; and taking regional meteorological satellite data corresponding to the regional grids as the second meteorological satellite data.
Optionally, obtaining the time sequence of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through the predetermined regression model includes: loading the initial network parameters into the preset regression model to obtain a preset regression model taking the initial network parameters as network parameters; inputting the second meteorological satellite data to the predetermined regression model; obtaining the output of the preset regression model, and obtaining a time sequence regression curve corresponding to each region in the multiple region grids; sampling the time sequence regression curve corresponding to each region in the multiple region grids according to a preset period to obtain a time sequence of the second meteorological satellite data distribution in each period.
According to another aspect of the embodiment of the present invention, there is also provided a processing device for meteorological satellite data, including: the first acquisition unit is used for acquiring first meteorological satellite data in a historical time period of the ground meteorological station; the training unit is used for training a model non-meta learning MAML model by using the first meteorological satellite data to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; the second acquisition unit is used for acquiring second meteorological satellite data in the global area; and the third acquisition unit is used for obtaining a time sequence of the second meteorological satellite data distribution by utilizing the initial network parameters and the second meteorological satellite data through the preset regression model.
Optionally, the first acquisition unit includes: the first acquisition module is used for acquiring similar tasks with similarity larger than a preset threshold value with the target tasks, wherein the target tasks are weather satellite data changes of a preset area; the second acquisition module is used for acquiring the related meteorological satellite data of the similar task; and the first determining module is used for determining the relevant meteorological satellite data as the first meteorological satellite data.
Optionally, the second acquisition module includes: a transmitting sub-module, configured to transmit a meteorological satellite data request message to a plurality of ground stations in the global area, where the meteorological satellite data request message carries area identification information of the predetermined area; the acquisition sub-module is used for acquiring feedback information sent by the ground stations based on the meteorological satellite data request message; and the determining submodule is used for determining the relevant meteorological satellite data based on the feedback information.
Optionally, the training unit includes: the dividing module is used for dividing the first meteorological satellite data based on a preset dividing unit to obtain a time sequence curve which corresponds to the preset dividing unit and represents the change of the first meteorological satellite data; a determining sub-module for taking the time series curve as an input to the MAML model; the third acquisition module is used for acquiring the output of the MAML model; and a fourth obtaining module, configured to obtain the initial network parameter based on an output of the MAML model.
Optionally, the meteorological satellite data processing device further includes: a conversion unit, configured to perform time conversion on acquisition time of the first meteorological satellite data before dividing the first meteorological satellite data based on a predetermined dividing unit to obtain a time series curve corresponding to the predetermined dividing unit and representing a change of the first meteorological satellite data; wherein the conversion unit includes: the second determining module is used for determining a first acquisition time of the first meteorological satellite data, wherein the first acquisition time is a universal standard time UTC time; and the conversion module is used for converting the UTC time into local time.
Optionally, the second obtaining unit includes: the dividing module is used for dividing the global area into a plurality of area grids based on meteorological features of different areas in the global area; the third determining module is used for determining regional weather satellite data corresponding to the regional grids; and the fourth determining module is used for taking regional weather satellite data corresponding to the regional grids as the second weather satellite data.
Optionally, the third obtaining unit includes: the loading module is used for loading the initial network parameters into the preset regression model to obtain a preset regression model taking the initial network parameters as network parameters; the input module is used for inputting the second meteorological satellite data into the preset regression model; a fifth obtaining module, configured to obtain an output of the predetermined regression model, to obtain a time-series regression curve corresponding to each region in the plurality of region meshes; and a sixth acquisition module, configured to sample the time sequence regression curve corresponding to each region in the multiple region grids according to a predetermined period, so as to obtain a time sequence of the second meteorological satellite data distribution in each period.
According to another aspect of the embodiment of the present invention, there is provided a computer readable storage medium, including a stored computer program, where the computer program, when executed by a processor, controls a device in which the computer storage medium is located to perform the method for processing meteorological satellite data according to any one of the above.
According to another aspect of the embodiment of the present invention, there is provided a processor, configured to execute a computer program, where the computer program executes the method for processing meteorological satellite data according to any one of the foregoing methods.
In the embodiment of the invention, first meteorological satellite data in a historical time period of a ground meteorological station are acquired; training a model-independent element learning MAML model by using first meteorological satellite data to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; acquiring second meteorological satellite data in a global area; the method for processing the meteorological satellite data provided by the embodiment of the invention realizes that the initial network parameters of the preset regression model are obtained by training the MAML model before the time sequence of the global meteorological satellite data distribution is obtained by the preset regression model by utilizing the initial network parameters and the second meteorological satellite data through the preset regression model, so that the purpose of carrying out regression by utilizing less sample time sequence data is achieved, the technical effect of improving the accuracy of the global meteorological satellite data distribution is achieved, and the technical problem that the global distribution mode of measuring the meteorological satellite data through a carried detector in the related technology is easy to have time and space sparsity is solved, so that the global meteorological satellite data distribution cannot be accurately known is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
FIG. 1 is a flow chart of a method of processing meteorological satellite data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a method of processing meteorological satellite data according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Aiming at the problems, a visual analysis framework based on MAML is provided in the embodiment of the invention, so that domain experts can be assisted to explore the space-time change rule of meteorological satellite data in a fine granularity. The training method of MAML solves the regression task of meteorological satellite data under the conditions of less data quantity, longer time interval and uneven distribution. Specifically, by designing three visual interfaces, a domain expert lacking in the experience of neural network use is helped to train a regression model by using MAML; for example, case studies can be performed using long-term data from 6 ground stations and ionosphere total electron content data acquired by a cloud three-D satellite occultation detector. In addition, the influence of different super parameters on the regression result can be evaluated, and guidance is provided for the configuration of the super parameters. In summary, the main contributions of the present invention are: 1) The method provides a framework for exploring the space-time change rule of meteorological satellite data, and solves the problems of small data quantity, long data interval and uneven distribution by combining MAML regression algorithm and space division; 2) The interactive visual interface is provided to help the domain expert lacking the experience of using the neural network to diagnose the MAML model, adjust the super parameters and training data, and improve the model performance.
The visualization of the neural network training process in the embodiment of the invention is beneficial to understanding, diagnosing and improving the neural network. Recent visualizations of neural network training processes are largely divided into two parts: visualization of the sample and visualization of the model. The samples directly affect the neural network parameters and researchers want to know the various behaviors of the model on the input data. For example, actiVis is an interactive visualization system for interpreting large-scale deep learning models and results. ActiVis compares activations from different samples by tightly integrating multiple coordinated views, helping users explore the activation of different samples by complex deep neural network models to analyze potential causes of misclassification. Visualization of the model is mainly used to analyze the links between the structure and components of the model. For example, CNNVis uses Directed Acyclic Graphs (DAGs) to analyze the behavior of each neuron. The method helps the user understand the behavior of the model by aggregating neurons with similar activations. Gan lab integrates a model overview map summarizing Gan structures and a hierarchical distribution view that helps the user explain interactions between sub-models. Gan lab interactively trains the generative model and visualizes intermediate results of the dynamic training process. Furthermore, there have been recent studies combining the two methods. For example, GANViz analyzes the behavior of each sample of the GAN model at various layers of the neural network and compares the differences between the actual image and the generated image. DQNViz combines data and models to reveal details of the training process from four levels in order to improve the performance of the reinforcement learning agent.
The satellite data visualization is realized in such a way that the satellite measures different meteorological data through the onboard detector, so that the satellite data visualization is applied to the analysis of the meteorological data. The satellite can only record the data of the orbit, and the missing data needs to be complemented before visualization. The existing research mainly complements the missing data by interpolation and simulation algorithms. Hong et al, obtained global data by a kriging interpolation method, realized a query-driven visual analysis system. Through the two views, a user selects a proper seismic event in the global scope, extracts different disturbance models, and explores the correlation between the seismic event and the ionospheric disturbance mode. Wang et al utilize Nested Parallel Coordinates Plot to conduct parametric analysis in multi-resolution numerical simulation analysis of meteorological data. The numerical simulation method iteratively simulates the state of the next time step according to satellite data. The spatial region of interest is discretized into a grid of blocks and iteratively computed over these grids to compute the state of the next time period. Cheng et al complements satellite data using the LSTM method, clusters the data and explores long-term changes in the data.
The meteorological satellite is one of the most widely used detection methods for global meteorological observation, and has the advantages of all weather, large observation area, no influence of atmospheric cloud layers and the like. Hundreds of meteorological satellites each day acquire a large amount of meteorological data via onboard sensors. Satellite observation is widely used for researching weather phenomenon formation mechanisms, global weather transformation and other large-scale problems due to the fact that weather changes are global. The physicochemical changes of the weather data (such as ozone content, electron content, etc.) in one region are affected by the changes of the weather data in an adjacent region, so that it is necessary to know the evolution law of the weather data by analyzing the space-time changes of the global weather data distribution.
The satellite moves along the orbit and does not spin with the earth, so the projected position of the satellite on the earth is constantly changing. Meteorological satellites are mostly low-orbit satellites, short in period and high in speed, and are wound around the earth for 14 circles (periods) a day. The meteorological satellite data is a series of space-time data distributed along the track, and the data detected by the meteorological satellite all day are distributed in most parts of the world, as shown in the figure. Analysis of meteorological satellite data requires consideration of the effects of time and space, respectively. Weather data in different areas (such as low latitude, medium latitude, high latitude or sea and land) are inconsistent in time variation law. For example, time series analysis of weather satellite data requires that an area be specified to exclude the effects of spatial position variations.
The method and the device for processing meteorological satellite data provided by the embodiment of the invention are described below.
Example 1
According to an embodiment of the present invention, there is provided a method embodiment of a method for processing meteorological satellite data, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different from that herein.
FIG. 1 is a flowchart of a method for processing meteorological satellite data according to an embodiment of the present invention, as shown in FIG. 1, the method for processing meteorological satellite data includes the following steps:
step S102, first meteorological satellite data in a historical time period of a ground meteorological station is acquired.
Alternatively, the first weather satellite data may be weather satellite data detected by the ground detection station during a certain period of time.
Step S104, training a model by using the first meteorological satellite data to learn the MAML model without an element, and obtaining initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data.
The meta learning is a hot research direction in the field of deep learning, and the research content of the meta learning is how to learn, and is mainly applied to a few-sample learning task. By learning a large number of tasks similar to the target task, the meta-learning method can significantly improve the performance of the neural network with less training data. The training method of the traditional neural network needs to search network parameters in a huge parameter space, and when training data are less, the fitting phenomenon is very easy to occur. MAML is the most advanced method widely used in the meta-learning field; MAML comprises two phases: firstly, finding out an optimal initial network parameter by training a large number of similar learning tasks; and loading the trained network initial parameters by the neural network model, and training with a small amount of training data to obtain a final model. Because the meteorological data has continuity in time and space, the dependence relationship and the periodic change between the data can be learned from the ground station historical observation data through the MAML algorithm, and the regression model can be trained, so that the defects of time and space sparsity are overcome.
MAML is a meta-learning algorithm used for less-sample learning. The meta-learning study learns how to learn, and by improving the learning capacity of the model, the model can achieve good effect by using only a small amount of training data. Existing neural network algorithms require training a model with a large amount of data to find network parameters in a vast parameter space. However, for small sample learning with a small amount of data, it is difficult for the model to find appropriate network parameters with a small amount of data.
In addition, MAML is a powerful method of learning with few samples, and can avoid overfitting with few data. MAML heavily trains tasks similar to the target task to find an initial network parameter. The trained network parameters are loaded, and the neural network model can achieve higher accuracy rate by using a small amount of training data. The MAML algorithm generally comprises two loops, an inner loop and an outer loop, as shown. The outer layer circulation aims at updating network parameters, and the inner layer circulation aims at training specific tasks, which are equivalent to the traditional neural network algorithm. In particular, T is a task that we need to solve, such as regression of meteorological satellite data. p (T) is a distribution of tasks similar to the target task T, such as regression of ground station weather data. fθ is a model of θ for one network parameter trained for task T. At the outer loop, the algorithm samples different tasks Ti-p (T) randomly. For each task Ti, a loss function LTi (fθ) is calculated and the network parameters are updated as: wherein α is Inner Step Size Hyperparameters; after the training of the inner layer circulation all tasks Ti is completed, the outer layer circulation updates the network parameters of MAML as follows:in this process, because the weather data has continuity over time, a neural network can be used to predict the time variation of the weather data. For the defects, a large amount of ground station history data can be used, and The MAML algorithm learns the circadian variation of the meteorological data and trains a regression model. In addition, although the meteorological satellite data in one period is only distributed near the orbit, the regression model can be trained through the data in other periods, the periodicity law can be learned, and the value of the missing data far from the orbit area can be predicted.
That is, in the embodiment of the invention, a new meteorological satellite data analysis framework is provided to assist the domain expert to explore the space-time variation rule of meteorological data in a fine granularity.
From the above, in the embodiment of the present invention, in order to learn the time series change rule of the weather satellite data at the full day-hour level, the MAML regression model is trained using a large number of historical data of the ground stations, so as to obtain the initial network parameters of the regression model.
Step S106, second meteorological satellite data in the global area is acquired.
Optionally, the second weather satellite data is weather satellite data detected by a plurality of ground stations in the global area, that is, the second weather satellite data converges weather satellite data in each global area.
And S108, obtaining a time sequence of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through a preset regression model.
In the embodiment of the invention, the first meteorological satellite data in the historical time period of the ground meteorological station can be obtained; training a model-independent element learning MAML model by using first meteorological satellite data to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; acquiring second meteorological satellite data in a global area; the initial network parameters of the preset regression model are obtained by utilizing the initial network parameters and the second meteorological satellite data through the preset regression model, so that the initial network parameters of the preset regression model are obtained by training the MAML model before the time sequence of the global meteorological satellite data distribution is obtained through the preset regression model, the purpose of carrying out regression by utilizing less sample time sequence data is achieved, and the technical effect of improving the accuracy of the global meteorological satellite data distribution is achieved.
Notably, in the embodiment of the invention, the regression method using the MAML model fuses the circadian periodic variation law shown by the long-term ground station meteorological satellite data and the real-time detection capability of the meteorological satellite, so that the defects of large interval time span and less data volume in the whole day and incapability of accurately regressing in the regional analysis are overcome, the problem of uneven data distribution of the global meteorological satellite is solved through regional division and time sequence sampling, and the time sequence variation of the whole-day global meteorological satellite data is more accurately obtained.
The processing method of the meteorological satellite data solves the technical problem that in the related art, the global distribution mode of the meteorological satellite data is easy to have time and space sparsity through the meteorological satellite through the carried detector, so that the global meteorological satellite data distribution cannot be accurately known.
In an alternative embodiment, acquiring first weather satellite data over a ground weather station history period includes: acquiring a similar task with the similarity to a target task being greater than a preset threshold, wherein the target task is a meteorological satellite data change of a determined area; acquiring related meteorological satellite data of similar tasks; the associated meteorological satellite data is determined to be first meteorological satellite data.
In this embodiment, after the target task is determined, a similar task similar to the target task may be searched, and then the relevant weather satellite data of the similar task may be acquired as the first weather satellite data.
In an alternative embodiment, acquiring relevant weather satellite data for similar tasks includes: transmitting a meteorological satellite data request message to a plurality of ground stations in a global area, wherein the meteorological satellite data request message carries area identification information of a preset area; acquiring feedback information sent by a plurality of ground stations based on meteorological satellite data request messages; relevant weather satellite data is determined based on the feedback information.
Because the time change of the meteorological satellite data is related to the position, the global meteorological satellite data can be divided according to grids, and the quantity and the quality of the meteorological satellite data in the grids need to be considered when the global meteorological satellite data are divided.
From the above, in an alternative embodiment, training the model using the first meteorological satellite data to learn the MAML model without an element, obtaining initial network parameters of a predetermined regression model includes: dividing the first meteorological satellite data based on a preset dividing unit to obtain a time sequence curve which corresponds to the preset dividing unit and represents the change of the first meteorological satellite data; taking the time sequence curve as the input of the MAML model; obtaining the output of the MAML model; initial network parameters are derived based on the output of the MAML model.
In the embodiment of the invention, when exploring day-night changes of meteorological satellite data in one day, domain experts can encounter the problem of small data volume. Due to the orbital motion, the meteorological satellite needs to travel one turn to reach the same position again, and the time interval between two detections is long. And because of the earth rotation, the satellite orbit changes continuously relative to the earth, and the time interval between meteorological satellite data is different.
For example, a field expert wishes to explore the weather data changes over japan. However, the meteorological satellite flies around the earth only 14 times a day, and only 5 times of 14 times pass over the sky in japan. Compared with the 24-hour variation range of the whole day, the 5 data have small data quantity, the time interval between the two data can be longer than 4 hours, and the intervals between the two data are inconsistent. These all result in difficulty in analyzing meteorological satellite data, and require experienced field specialists to combine with a variety of situations that may occur in real data analysis. Neural network methods can be used for periodic changes in regression data over time, but training of the neural network requires a large amount of training data. The model cannot find the best network parameters in a large amount of parameter space with only 5 data, and over-fitting often occurs.
MAML is often used to solve the training problem of such neural networks with small amounts of data. MAML algorithm first requires collecting data of a large number of similar tasks; ground weather stations record a large number of long-term historical observations. Compared with meteorological satellites, the ground station can only continuously detect one area, and lacks data diversity. By combining ground stations in different regions of the world, data diversity can be increased. The world contains many ground weather stations, each containing a variety of detectors, providing historical data for a large number of different weather parameters.
And the weather data change rules of different areas are inconsistent. In order to increase the diversity of data, multiple weather stations need to be selected as data sources. To learn the circadian variation law of the weather data on a daily hour-by-day level, the ground station data may be segmented into 24-hour time series curves, which contain periodic patterns of variation of the weather parameters. The interval between time series needs to be set according to the characteristics of the data. For weather data that is highly variable and requires fine analysis, the time series interval may be set to 5-15 minutes.
Specifically, in the outer loop of the MAML algorithm, the algorithm randomly selects a ground station time series D, and the regression of this curve is used as a task T of the algorithm. The outer loop includes a number of subtasks as the inner loop. In the inner loop, the subtask selects K data in the sequence D as the input of the neural network, and calculates the loss value L and the gradient. After all inner layer loops are over, the algorithm updates the model according to the gradients of all subtasks. After all outer layer loops are finished, the algorithm obtains the network parameters of the regression model.
Notably, the meteorological data is a series of spatiotemporal data distributed along the track. The location of the weather data varies with the track and the data is spread throughout the world throughout the day. Analysis of meteorological satellite data requires balancing the effects of time and space. When analyzing the effect of time variations on meteorological parameters to explore the periodic law of time variations, domain experts want to control the spatial variations within a small range to exclude the effect of the spatial variations. However, data recorded by weather satellites over short periods of time spans multiple time zones, which data exhibits significantly different periodic variations. The time of the data varies greatly as the satellites pass through 24 different time zones in one period. Therefore, the time transformation method can be used to transform the UTC time recorded by the satellite into local time (specifically, 0 point in local time is early morning of the time zone, 12 points are noon, and the sun is directly incident to the region. Local solar irradiation intensity is more exhibited when studying time variation in a small area. And the solar irradiation intensity is one of the most important influencing factors of the day-night change of the meteorological data.
Thus, in an alternative embodiment, before dividing the first meteorological satellite data based on the predetermined dividing unit to obtain a time series curve representing a change in the first meteorological satellite data corresponding to the predetermined dividing unit, the method for processing meteorological satellite data may further include: performing time conversion on the acquisition time of the first meteorological satellite data; wherein time converting the acquisition time of the first meteorological satellite data comprises: determining a first acquisition time of first meteorological satellite data, wherein the first acquisition time is a universal standard time UTC time; the UTC time is converted into a local time.
The day and night change of the meteorological parameters has strong correlation with the position. Because the weather conditions such as sunlight, monsoon and ocean currents at different positions are different, the weather data can show different change rules. However, the position of the meteorological satellite is changed continuously, and continuous and uninterrupted detection of the same position cannot be performed. In addition, meteorological satellite data is unevenly distributed, and global distribution changes are difficult to explore from the meteorological satellite data. To exclude the effect of spatial variations on the time variations of the weather satellites, the data may be grid-partitioned spatially. The world is divided into many small areas by meshing. When the area of the grid area is small, the day-night variation of the data in the grids is similar, and one grid can be analyzed as a whole. The unstructured scattered data is converted into grid data by grid division, so that the data format is unified, and the analysis and the processing are convenient. Expert can study day and night changes in small areas through gridding data. The grid data lacking in one period is complemented by a regression method, and the method can be used for analyzing the distribution change of global meteorological data.
The data quantity and the data quality in the grid need to be comprehensively considered in the space division. Data quality refers to the time range covered. When there is little in-grid data (e.g., 1-4 data), subsequent regression of the in-grid data becomes difficult. When the data only covers a small time range, the time range required for regression is too long, and the accuracy is low. However, if the mesh area is too large, the data amount may be increased, but the quality of the data in the mesh may be poor, the diurnal variation may be inconsistent, and the regression analysis of the mesh as a whole may not be performed. Meshing therefore requires a balance between data volume and data quality.
Thus, in an alternative embodiment, acquiring second meteorological satellite data in a global area includes: dividing the global area into a plurality of area grids based on meteorological features of different areas in the global area; determining regional weather satellite data corresponding to the plurality of regional grids; and taking regional meteorological satellite data corresponding to the regional grids as second meteorological satellite data.
In this embodiment, the global data is meshed, and a spatial matrix divided into 18 in the horizontal direction, 9 in the vertical direction, and 9×18 in the global direction may be set. The rectangular areas within the grid view were observed and more than half of the rectangular areas within the grid were found to occupy less than 50%, indicating that there was less data within the grids. In mid-latitude areas, the color of the rectangle in many grids is purple, which indicates that the data quality in the grids is poor and the time coverage of the data is small. When the number of training data is small and the quality is poor, the accuracy of the regression result of MAML is low. For example, MAML cannot infer a regression curve for one day using only two data of 12 and 19 points. The spatial matrix divided horizontally 15, divided vertically 7, and divided globally 7 x 15 may be modified. Again looking at the rectangular areas and rectangular colors within the grid, it can be found that most of the grid meets the minimum requirements for MAML training.
In an alternative embodiment, the obtaining the time series of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through a predetermined regression model includes: loading the initial network parameters into a preset regression model to obtain a preset regression model taking the initial network parameters as network parameters; inputting the second meteorological satellite data into a predetermined regression model; obtaining the output of a preset regression model to obtain a time sequence regression curve corresponding to each region in a plurality of region grids; sampling the time sequence regression curve corresponding to each region in the multiple region grids according to a preset period to obtain a time sequence of the second meteorological satellite data distribution in each period.
In the above embodiment, after space division has been performed, if the day and night change of the meteorological satellite data in the grid is to be explored, the analysis precision of the time change is improved; the existing numerical weather simulation method takes weather satellite data as input of a model to predict day and night changes of the weather satellite data, and the weather satellite data with unclear physical and chemical mechanisms and change rules has differences between the results of the numerical simulation method and real changes. The regression model may also predict changes in data over time; however, the day of weather satellite data in a grid is typically less than 10, for which the regression task belongs to small sample learning, and conventional regression methods often suffer from overfitting.
In the embodiment of the invention, the effect of a preset regression model can be improved by using the initial network parameters trained by MAML by means of the learning capability of the MAML algorithm; the researcher can select an interested grid area from all area grids for further research, load MAML trained initial network parameters and regional weather satellite data, train a preset regression model again, and obtain a time sequence regression curve of the whole day hour level in the region after training the preset regression model so as to be used for analyzing day and night changes.
In an alternative embodiment, if the process of how the distribution of meteorological data changes around the world is explored, the data acquired by satellites is distributed only near the orbit and the data distribution is uneven in one period. The existing research estimates meteorological data of an area which is not covered by a track through an interpolation method, but the result of the interpolation method is inaccurate, and a fine structure of spatial distribution cannot be obtained. The method comprises the steps of obtaining a time sequence regression curve of one day in a grid area by a preset regression model, and respectively training the regression model for all the grid areas to obtain the time sequence regression curve of all the areas of the world. Meteorological satellite data is mostly low orbit satellites, and the period of low orbit satellite data is about 85 minutes, and about 14 turns (cycles) around the earth a day. And (3) sampling the time sequence curves of all grid areas according to the period to obtain the global meteorological data distribution of each period. The 14 global weather data distributions were analyzed to explore the global changes in weather data.
It should be noted that, in the embodiment of the present invention, in order to better combine the visualization with the framework, to assist the domain expert without experience of using the neural network to explore the meteorological satellite data, the following requirements are met: 1) Visual MAML training; 2) Interactive regression analysis: because the data volume is too small, abnormal data can cause larger regression errors, abnormal data in a grid area can be removed according to experience, the data volume and the data volume in the grid area can be displayed through an interactive partition space, the grid area of interest and the data in the grid area can be selected to train a regression model, and the results of different regression models can be compared; 3) The global change exploration can be compared, for example, the time sequence change of all-day meteorological satellite data can be displayed, the actual value of the meteorological satellite data and the predicted value of the regression model can be compared, and meanwhile, the cause of the abnormal phenomenon can be traced back.
Since meteorological satellites are mostly low orbit satellites, satellites contain 14 cycles a day. Thus, the global view contains 14 sub-views, each of which reveals the true and predicted values of the global weather data over a period. The gray map is arranged below each sub-view, so that the expert can conveniently and quickly position the map. The global view reveals the time sequence variation of the meteorological data distribution within 24 hours from left to right. Each sub-view comprises an upper graph and a lower graph, wherein the upper graph represents the original data acquired by the meteorological satellite in the period, and the color represents the height of the meteorological value. The lower graph shows the prediction result of the regression model, and the color shows the level of the prediction value. Diurnal variation is related to solar illumination, and yellow highlighting encodes the range of direct solar radiation. By observing the upper view and the lower view, the difference between the actual value of the meteorological satellite and the predicted value of the regression model can be compared. Building a global hour-level meteorological model requires real values based on meteorological satellites. Abnormal changes in training data can lead to anomalies in regression model results. When an anomaly is found, the cause and basis of the anomaly need to be traced back. By comparing the true value with the predicted value, an anomaly is found which meteorological satellite data caused.
It should be noted that, in the embodiment of the present invention, in order to verify the validity of the MAML regression algorithm, the regression analysis may be performed by selecting the historical data of 6 ground stations. Meteorological satellite data recorded between 1.1.2018 and 31.12.2018 were used for these 6 ground stations. The long-term data of the ground station is divided by day. Because the change of the meteorological satellite data is smooth, the data of one day is sampled every 30 minutes, and a time sequence is obtained. The time series of time series changes represent diurnal variations in the weather satellite data of the area in which the ground station is located.
Specifically, the iteration number of MAML may be set to 15000, the display interval to 1000, the task number to 5, the sample number to 5, the outer loop step number to 0.3, and the inner loop step number to 0.05. And selecting one piece of the meteorological satellite data curve data set as a true value in each iteration. Each iteration contains 5 tasks, each randomly sampling the true value by 5 points as training data, and calculating the loss value and gradient. After updating network parameters, the regression model samples the preset regression model for 30 minutes to obtain a full-day time sequence curve as a predicted value of the task. The regression capability of the current neural network model to the circadian variation of the real data can be known through comparison.
The difference between the predicted value and the true value at the 1000 th and 10000 th iterations is compared with the regression model. The loss value is encoded as the height of the histogram, with the abscissa being the time of sampling the data. By looking at the histogram for each task, one can learn the training state of the neural network and find out what range this task is with poor data processing capabilities. At the 1000 th iteration, the model learning ability is insufficient, and the day-night variation of the data cannot be accurately estimated by only 5 sampling points. The task performs significantly better near the sampling point than in other areas, indicating that the model does not learn the intrinsic rules of the data. At iteration 10000, the model is not much different from the real value in terms of diurnal variation, although the loss value of a few positions is slightly higher. We can infer the TEC change over 24 hours from the regression model results. Furthermore, since the sampling points are different for each task, we cannot judge the model's ability by the performance of only one task. At 8000 th iteration, the loss value of one task is high around 15 points. While other tasks result in low loss values because they are not sampled around 15 points. Thus, the performance of tasks on different areas needs to be considered to find out which data has not been sufficiently trained.
The meteorological satellite data processing method solves the analysis framework of the sparsity of the meteorological satellite data and a meteorological data space-time change visualization analysis prototype system for training, tuning and anomaly discovery; the regression method based on MAML fuses the circadian periodic variation law shown by the long-term ground station data and the real-time detection capability of the meteorological satellite, and solves the problems that the interval time span of two times is large and the data quantity of the whole day is small and the accurate regression cannot be realized during the regional analysis. Through regional division and time sequence sampling, the problem of uneven global data distribution is solved, and the time sequence change of global meteorological parameters throughout the day is obtained.
Example 2
According to another aspect of the embodiment of the present invention, there is provided a processing device for meteorological satellite data, fig. 2 is a schematic diagram of the processing device for meteorological satellite data according to an embodiment of the present invention, and as shown in fig. 2, the processing device for meteorological satellite data includes: a first acquisition unit 21, a training unit 23, a second acquisition unit 25 and a third acquisition unit 27. The processing device of the meteorological satellite data will be described below.
The first acquisition unit 21 is configured to acquire first meteorological satellite data during a historical period of a ground meteorological station.
The training unit 23 is configured to train the model-independent learning MAML model by using the first meteorological satellite data, and obtain initial network parameters of a predetermined regression model, where the predetermined regression model is obtained by performing machine learning training by using multiple sets of training data.
The second acquiring unit 25 is configured to acquire second meteorological satellite data in the global area.
The third obtaining unit 27 is configured to obtain a time sequence of the second meteorological satellite data distribution by using the initial network parameter and the second meteorological satellite data through a predetermined regression model.
Here, the first acquiring unit 21, the training unit 23, the second acquiring unit 25, and the third acquiring unit 27 correspond to steps S102 to S108 in embodiment 1, and the above-described units are the same as examples and application scenarios achieved by the corresponding steps, but are not limited to those disclosed in embodiment 1. It should be noted that the above-described elements may be implemented as part of an apparatus in a computer system such as a set of computer-executable instructions.
As can be seen from the above, in the above embodiments of the present application, the first acquisition unit may be used to acquire the first meteorological satellite data in the historical period of the ground meteorological station; then training a model independent element learning MAML model by using first meteorological satellite data by using a training unit to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data; then, acquiring second meteorological satellite data in the global area by using a second acquisition unit; and obtaining a time sequence of the second meteorological satellite data distribution by using the third acquisition unit through a preset regression model by using the initial network parameters and the second meteorological satellite data. According to the processing device for the meteorological satellite data, disclosed by the embodiment of the invention, the initial network parameters of the preset regression model are obtained by training the MAML model before the time sequence of the global meteorological satellite data distribution is obtained by the preset regression model, so that the purpose of regression is achieved by using less sample time sequence data, the technical effect of improving the accuracy of the global meteorological satellite data distribution is achieved, and the technical problem that the global meteorological satellite data distribution cannot be accurately known due to the fact that the global distribution mode of the meteorological satellite data is easily provided with time and space sparsity by a detector carried by the meteorological satellite in the related art is solved.
In an alternative embodiment, the first acquisition unit comprises: the first acquisition module is used for acquiring similar tasks with the similarity to the target tasks being greater than a preset threshold, wherein the target tasks are weather satellite data changes of a preset area; the second acquisition module is used for acquiring related meteorological satellite data of similar tasks; the first determining module is used for determining the relevant meteorological satellite data as first meteorological satellite data.
In an alternative embodiment, the second acquisition module includes: the transmitting sub-module is used for transmitting a meteorological satellite data request message to a plurality of ground stations in a global area, wherein the meteorological satellite data request message carries area identification information of a preset area; the acquisition sub-module is used for acquiring feedback information sent by the ground stations based on the meteorological satellite data request message; and the determining submodule is used for determining relevant meteorological satellite data based on the feedback information.
In an alternative embodiment, the training unit comprises: the division module is used for dividing the first meteorological satellite data based on a preset division unit to obtain a time sequence curve which corresponds to the preset division unit and represents the change of the first meteorological satellite data; the determining submodule is used for taking the time sequence curve as input of the MAML model; the third acquisition module is used for acquiring the output of the MAML model; and a fourth acquisition module, configured to obtain an initial network parameter based on an output of the MAML model.
In an alternative embodiment, the meteorological satellite data processing apparatus further includes: a conversion unit for performing time conversion on the acquisition time of the first meteorological satellite data before dividing the first meteorological satellite data based on the predetermined dividing unit to obtain a time sequence curve corresponding to the predetermined dividing unit and representing the change of the first meteorological satellite data; wherein the conversion unit includes: the second determining module is used for determining a first acquisition time of the first meteorological satellite data, wherein the first acquisition time is the universal standard time UTC time; and the conversion module is used for converting the UTC time into local time.
In an alternative embodiment, the second acquisition unit comprises: the dividing module is used for dividing the global area into a plurality of area grids based on meteorological features of different areas in the global area; the third determining module is used for determining regional weather satellite data corresponding to the regional grids; and the fourth determining module is used for taking regional meteorological satellite data corresponding to the regional grids as second meteorological satellite data.
In an alternative embodiment, the third acquisition unit comprises: the loading module is used for loading the initial network parameters into a preset regression model to obtain the preset regression model taking the initial network parameters as network parameters; the input module is used for inputting the second meteorological satellite data into a preset regression model; a fifth obtaining module, configured to obtain an output of the predetermined regression model, to obtain a time sequence regression curve corresponding to each region in the plurality of region grids; and the sixth acquisition module is used for sampling the time sequence regression curve corresponding to each region in the multiple region grids according to a preset period to obtain a time sequence of the second meteorological satellite data distribution in each period.
Example 3
According to another aspect of the embodiment of the present invention, there is provided a computer readable storage medium, including a stored computer program, where the computer program when executed by a processor controls a device in which the computer storage medium is located to perform a method for processing meteorological satellite data according to any one of the above.
Example 4
According to another aspect of the embodiment of the present invention, there is provided a processor, configured to execute a computer program, where the computer program executes the method for processing meteorological satellite data according to any one of the above.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology content may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (9)
1. A method for processing meteorological satellite data, comprising:
acquiring first meteorological satellite data in a historical time period of a ground meteorological station;
training a model by using the first meteorological satellite data, and learning an MAML model by using an independent element to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data;
acquiring second meteorological satellite data in a global area;
obtaining a time sequence of the second meteorological satellite data distribution by using the initial network parameters and the second meteorological satellite data through the preset regression model;
wherein, training a model independent element learning MAML model by using the first meteorological satellite data to obtain initial network parameters of a preset regression model, comprising: dividing the first meteorological satellite data based on a preset dividing unit to obtain a time sequence curve which corresponds to the preset dividing unit and represents the change of the first meteorological satellite data; taking the time series curve as an input of the MAML model; obtaining the output of the MAML model; the initial network parameters are derived based on the output of the MAML model.
2. The method of claim 1, wherein acquiring first meteorological satellite data for a ground meteorological station historical period of time comprises:
acquiring a similar task with the similarity to a target task being greater than a preset threshold, wherein the target task is a meteorological satellite data change of a determined preset area;
acquiring relevant meteorological satellite data of the similar task;
the associated meteorological satellite data is determined to be the first meteorological satellite data.
3. The method of claim 2, wherein acquiring relevant weather satellite data for the similar mission comprises:
transmitting a meteorological satellite data request message to a plurality of ground stations in the global area, wherein the meteorological satellite data request message carries area identification information of the preset area;
acquiring feedback information sent by the ground stations based on the meteorological satellite data request message;
and determining the relevant meteorological satellite data based on the feedback information.
4. The method of claim 1, further comprising, prior to dividing the first meteorological satellite data based on a predetermined division unit, obtaining a time series curve representing a change in the first meteorological satellite data corresponding to the predetermined division unit: performing time conversion on the acquisition time of the first meteorological satellite data;
Wherein time converting the acquisition time of the first meteorological satellite data includes:
determining a first acquisition time of the first meteorological satellite data, wherein the first acquisition time is a universal time UTC time;
converting the UTC time into a local time.
5. The method of claim 1, wherein acquiring second meteorological satellite data in a global area comprises:
dividing the global area into a plurality of area grids based on meteorological features of different areas in the global area;
determining regional weather satellite data corresponding to the plurality of regional grids;
and taking regional meteorological satellite data corresponding to the regional grids as the second meteorological satellite data.
6. The method of claim 5, wherein using the initial network parameters and the second meteorological satellite data to obtain a time series of the second meteorological satellite data distribution by the predetermined regression model comprises:
loading the initial network parameters into the preset regression model to obtain a preset regression model taking the initial network parameters as network parameters;
inputting the second meteorological satellite data to the predetermined regression model;
Obtaining the output of the preset regression model, and obtaining a time sequence regression curve corresponding to each region in the multiple region grids;
sampling the time sequence regression curve corresponding to each region in the multiple region grids according to a preset period to obtain a time sequence of the second meteorological satellite data distribution in each period.
7. A meteorological satellite data processing apparatus, comprising:
the first acquisition unit is used for acquiring first meteorological satellite data in a historical time period of the ground meteorological station;
the training unit is used for training a model non-meta learning MAML model by using the first meteorological satellite data to obtain initial network parameters of a preset regression model, wherein the preset regression model is obtained by performing machine learning training by using a plurality of groups of training data;
the second acquisition unit is used for acquiring second meteorological satellite data in the global area;
the third acquisition unit is used for obtaining a time sequence of the second meteorological satellite data distribution by utilizing the initial network parameters and the second meteorological satellite data through the preset regression model;
wherein, training unit includes: the dividing module is used for dividing the first meteorological satellite data based on a preset dividing unit to obtain a time sequence curve which corresponds to the preset dividing unit and represents the change of the first meteorological satellite data; a determining sub-module for taking the time series curve as an input to the MAML model; the third acquisition module is used for acquiring the output of the MAML model; and a fourth obtaining module, configured to obtain the initial network parameter based on an output of the MAML model.
8. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run by a processor, controls a device in which the computer storage medium is located to perform the method of processing meteorological satellite data according to any one of claims 1 to 6.
9. A processor for executing a computer program, wherein the computer program when executed performs the method of processing meteorological satellite data according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010717906.0A CN111859054B (en) | 2020-07-23 | 2020-07-23 | Meteorological satellite data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010717906.0A CN111859054B (en) | 2020-07-23 | 2020-07-23 | Meteorological satellite data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111859054A CN111859054A (en) | 2020-10-30 |
| CN111859054B true CN111859054B (en) | 2023-12-26 |
Family
ID=72950861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010717906.0A Active CN111859054B (en) | 2020-07-23 | 2020-07-23 | Meteorological satellite data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111859054B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705363A (en) * | 2021-08-06 | 2021-11-26 | 成都德辰博睿科技有限公司 | Method and system for identifying uplink signal of specific satellite |
| CN114372212B (en) * | 2022-03-21 | 2022-07-08 | 国家超级计算天津中心 | Numerical weather forecasting method and device based on supercomputer |
| CN116521875B (en) * | 2023-05-09 | 2023-10-31 | 江南大学 | Prototype enhanced small sample dialogue emotion recognition method for introducing group emotion infection |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106404620A (en) * | 2015-07-30 | 2017-02-15 | 中国科学院遥感与数字地球研究所 | Method and system for inverting ground PM 2.5 through combination of geostatistical interpolation and satellite remote sensing |
| CN107133686A (en) * | 2017-03-30 | 2017-09-05 | 大连理工大学 | City-level PM2.5 concentration prediction methods based on Spatio-Temporal Data Model for Spatial |
| CN109615108A (en) * | 2018-10-30 | 2019-04-12 | 中国石油大学(华东) | Multitask oil field Production development model of fit based on meta learning |
| CN109700434A (en) * | 2018-12-28 | 2019-05-03 | 武汉中旗生物医疗电子有限公司 | Meta-learning model training method, system and equipment based on electrocardiographic diagnosis |
| CN110084406A (en) * | 2019-04-03 | 2019-08-02 | 新奥数能科技有限公司 | Load forecasting method and device based on self-encoding encoder and meta learning strategy |
| CN110427881A (en) * | 2019-08-01 | 2019-11-08 | 东南大学 | The micro- expression recognition method of integration across database and device based on the study of face local features |
| CN111426633A (en) * | 2020-06-15 | 2020-07-17 | 航天宏图信息技术股份有限公司 | PM at night2.5Mass concentration estimation method and device |
-
2020
- 2020-07-23 CN CN202010717906.0A patent/CN111859054B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106404620A (en) * | 2015-07-30 | 2017-02-15 | 中国科学院遥感与数字地球研究所 | Method and system for inverting ground PM 2.5 through combination of geostatistical interpolation and satellite remote sensing |
| CN107133686A (en) * | 2017-03-30 | 2017-09-05 | 大连理工大学 | City-level PM2.5 concentration prediction methods based on Spatio-Temporal Data Model for Spatial |
| CN109615108A (en) * | 2018-10-30 | 2019-04-12 | 中国石油大学(华东) | Multitask oil field Production development model of fit based on meta learning |
| CN109700434A (en) * | 2018-12-28 | 2019-05-03 | 武汉中旗生物医疗电子有限公司 | Meta-learning model training method, system and equipment based on electrocardiographic diagnosis |
| CN110084406A (en) * | 2019-04-03 | 2019-08-02 | 新奥数能科技有限公司 | Load forecasting method and device based on self-encoding encoder and meta learning strategy |
| CN110427881A (en) * | 2019-08-01 | 2019-11-08 | 东南大学 | The micro- expression recognition method of integration across database and device based on the study of face local features |
| CN111426633A (en) * | 2020-06-15 | 2020-07-17 | 航天宏图信息技术股份有限公司 | PM at night2.5Mass concentration estimation method and device |
Non-Patent Citations (3)
| Title |
|---|
| Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks;Chelsea Finn等;《the 34 th International Conference on Machine Learning》;1126-1135 * |
| Visual analysis of meteorological satellite data via model-agnostic meta-learning;Shiyu Cheng等;《Journal of Visualization》;301-315 * |
| 基于MAML方法的佤语孤立词分类;王翠等;《云南民族大学学报(自然科学版)》;第29卷(第4期);376-381+395 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111859054A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111859800B (en) | Space-time estimation and prediction method for PM2.5 concentration distribution | |
| CN111859054B (en) | Meteorological satellite data processing method and device | |
| CN112099110B (en) | Ocean internal wave forecasting method based on machine learning and remote sensing data | |
| Meng et al. | Forecasting tropical cyclones wave height using bidirectional gated recurrent unit | |
| CN112699959B (en) | Multi-source multi-scale precipitation data fusion method and device based on energy functional model | |
| CN115222163A (en) | Multi-factor medium-long term real-time forecasting method and system for harbor basin inlet waves and application | |
| CN114936691A (en) | Temperature forecasting method integrating relevance weighting and space-time attention | |
| CN108764527B (en) | Screening method for soil organic carbon library time-space dynamic prediction optimal environment variables | |
| Li et al. | A spatiotemporal directed graph convolution network for ultra-short-term wind power prediction | |
| Xu et al. | A deep learning approach to predict sea surface temperature based on multiple modes | |
| Jia et al. | Review on the change trend, attribution analysis, retrieval, simulation, and prediction of lake surface water temperature | |
| González‐Abad et al. | Using explainability to inform statistical downscaling based on deep learning beyond standard validation approaches | |
| Zheng et al. | Weather image-based short-term dense wind speed forecast with a ConvLSTM-LSTM deep learning model | |
| Mu et al. | ENSO‐GTC: ENSO Deep Learning Forecast Model With a Global Spatial‐Temporal Teleconnection Coupler | |
| Zhang et al. | SolarGAN: Synthetic annual solar irradiance time series on urban building facades via Deep Generative Networks | |
| Li et al. | PV resource evaluation based on Xception and VGG19 two-layer network algorithm | |
| CN117194926A (en) | Method and system for predicting hoisting window period of land wind power base | |
| Joshi et al. | Rainfall prediction using data visualisation techniques | |
| Rajabi-Kiasari et al. | Forecasting of absolute dynamic topography using deep learning algorithm with application to the Baltic Sea | |
| Porto et al. | Machine learning approaches to extreme weather events forecast in urban areas: Challenges and initial results | |
| Toms et al. | Testing the reliability of interpretable neural networks in geoscience using the madden-julian oscillation | |
| Jiang et al. | Application and evaluation of an improved LSTM model in the soil moisture prediction of southeast chinese tobacco-producing areas | |
| Kaselimi | Machine learning methods for modelling and analysis of time series signals in geoinformatics | |
| Saxena et al. | Time Series Prediction of Sea Surface Temperature Using LSTM | |
| CN117634325B (en) | Method and system for identifying extremum event of data-limited estuary area and analyzing composite flood disasters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |