CN114580561A - Machine learning fusion method and model for multisource sea surface physical elements - Google Patents
Machine learning fusion method and model for multisource sea surface physical elements Download PDFInfo
- Publication number
- CN114580561A CN114580561A CN202210253149.5A CN202210253149A CN114580561A CN 114580561 A CN114580561 A CN 114580561A CN 202210253149 A CN202210253149 A CN 202210253149A CN 114580561 A CN114580561 A CN 114580561A
- Authority
- CN
- China
- Prior art keywords
- data
- fusion
- interpolation
- multisource
- physical elements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 39
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 27
- 230000004927 fusion Effects 0.000 claims abstract description 78
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000012950 reanalysis Methods 0.000 claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000007405 data analysis Methods 0.000 claims abstract description 29
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 12
- 238000012545 processing Methods 0.000 description 8
- 238000011160 research Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 108091092919 Minisatellite Proteins 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/02—Instruments for indicating weather conditions by measuring two or more variables, e.g. humidity, pressure, temperature, cloud cover or wind speed
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/10—Devices for predicting weather conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Environmental & Geological Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Atmospheric Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Mathematical Physics (AREA)
- Biodiversity & Conservation Biology (AREA)
- Ecology (AREA)
- Environmental Sciences (AREA)
- Remote Sensing (AREA)
- Astronomy & Astrophysics (AREA)
- Computational Linguistics (AREA)
- Image Processing (AREA)
Abstract
The application discloses a machine learning fusion method and a model of multi-source sea surface physical elements, which comprise the following steps: acquiring multi-source data, and preprocessing the acquired multi-source data; performing mixed interpolation on the preprocessed multi-source data to obtain standard grid data with the same resolution as NWP mode data/reanalysis data; establishing a characteristic project according to the standard grid data and the NWP mode data/reanalysis data to obtain a training sample required by the target fusion model; inputting a training sample into a target fusion model for training to obtain a data fusion model; and reasoning the NWP mode data/reanalysis data according to the data fusion model to obtain a fusion field. The method relies on a machine learning method, data fusion can be rapidly carried out after a fusion model is obtained through training, the operation is simple, the hardware requirement is low, meanwhile, the data fusion method weakens the influence of a physical mechanism on data to a certain extent, and the method is suitable for almost all sea surface physical elements.
Description
Technical Field
The application relates to the technical field of marine data processing, in particular to a data fusion method and a model of multisource sea surface physical elements.
Background
In the research in the marine field, data fusion is indispensable, and powerful data support is provided for further analyzing marine development and rules. There are currently many data fusion algorithms proposed and utilized by researchers. Taking the fusion of the data of the sea surface wind field as an example, the current main fusion methods include an interpolation fusion algorithm and an assimilation variant fusion algorithm. The interpolation algorithm comprises a Cressman interpolation, a Kriging interpolation and a space-time weighting analysis, and the assimilation variation algorithm comprises an optimal interpolation, a three-dimensional variation and the like. Traing and the like fuse offshore satellite wind field and coastal meteorological station wind field data in China through Cressman interpolation. Zhang et al interpolated spatio-temporal weights for multiple satellite sea-surface wind speed data including SSM/I, TMI, QuikSCAT, AMSR-E, etc., resulting in a global range of wind speeds from 1987 to 2006 with a temporal resolution of 12h, daily and monthly 0.25 grid. Zilein and the like fuse sea surface wind field data of the ocean No. two satellite and NCEP numerical value wind field data, and a space-time weight interpolation is also adopted in a fusion algorithm. Yan and the like perform fusion research on wind fields and modes of a multi-source scatterometer and a radiometer in analyzing the wind fields, and establish a global wind field product with the time resolution of 6h and the spatial resolution of 0.25 degrees from 2000 to 2015 by utilizing an optimal interpolation method.
Chinese patent CN105975763A 'a fusion method and device of multi-source sea surface wind field' provides a wind field fusion method and device relating to the field of sea surface wind field, and the patent performs data fusion on sea surface wind field data collected by a plurality of satellite-borne microwave remote sensors and/or a plurality of reanalyzed meteorological sea surface wind field data by utilizing methods such as a space-time interpolation algorithm, a linear interpolation algorithm and the like. The method can exert the advantages of cooperative observation of the multisource satellite, and can effectively improve the coverage range and the space-time resolution of the sea surface wind field by constructing and fusing the sea surface wind field through the fusion of the satellite remote sensing wind field data and the re-analysis meteorological wind field data.
The technology in the above patent uses a conventional interpolation method, that is, interpolation operation is required to be performed each time a wind field is fused, gridding processing is performed on each wind field data, a large amount of time is wasted, and a large amount of space is wasted in intermediate grid data obtained by processing the wind field data.
For the fusion algorithm, the problem of sea surface wind field fusion can be basically solved by adopting an interpolation type fusion algorithm or an assimilation variant type fusion algorithm. However, in practical application, the algorithm is limited by the current computing capability, and due to the complex computing process, the algorithms often need to use a computer cluster, and real-time fusion is difficult to realize.
Disclosure of Invention
Therefore, the data fusion method and the data fusion model for the multi-source sea surface physical elements are provided to solve the problem that in the prior art, the fusion algorithm is limited by computing capacity and real-time fusion is difficult to realize.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, a machine learning fusion method for multi-source sea surface physical elements includes:
acquiring multi-source data and NWP mode data/reanalysis data, and preprocessing the acquired multi-source data;
performing mixed interpolation on the preprocessed multi-source data to obtain standard grid data with the same resolution as the NWP mode data/reanalysis data;
establishing a characteristic project according to the standard grid data and the NWP mode data/reanalysis data, and obtaining training samples required by a target fusion model;
inputting the training sample into the target fusion model for training to obtain a data fusion model; and reasoning the NWP mode data/reanalysis data according to the data fusion model to obtain a fusion field.
Preferably, when the multi-source data is subjected to hybrid interpolation, the method specifically comprises the following steps:
judging whether the data points are overlapped;
if the data points are not overlapped, directly performing interpolation;
and if the data points are overlapped, selecting the data closest to the interpolation time point for interpolation.
Preferably, when the multi-source data is subjected to mixed interpolation, an inverse distance weighted interpolation algorithm is adopted for interpolation.
Preferably, the standard mesh data is 0.25 ° × 0.25 ° standard mesh data.
Preferably, the NWP mode data/reanalysis data are ERA-5 reanalysis data
Preferably, when the feature engineering is established according to the standard grid data and the NWP mode data/reanalysis data, the method specifically comprises the following steps:
and (3) using the satellite interpolation data as a learning target, and selecting a satellite interpolation grid point and an ERA-5 value of 5 multiplied by 5 around the satellite interpolation grid point as training characteristics for training.
Preferably, the target fusion model is an XGBoost learning method.
In a second aspect, the data fusion model of the multisource sea surface physical elements is obtained by training according to the machine learning fusion method of the multisource sea surface physical elements.
In a third aspect, a computer device includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the machine learning fusion method for the multisource sea surface physical elements when executing the computer program.
In a fourth aspect, a computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of a machine learning fusion method of multisource sea surface physical elements.
Compared with the prior art, the method has the following beneficial effects:
1. compared with the existing sea surface data fusion technology, the technical scheme provided by the invention relies on a machine learning method, and the data fusion can be rapidly carried out after a fusion model is obtained through training, so that the operation is simple, the hardware requirement is reduced, and meanwhile, the data fusion method weakens the influence of a physical mechanism on data to a certain extent and is suitable for almost all sea surface physical elements;
2. the method of the invention can be operated on a CPU platform.
Drawings
To more intuitively illustrate the prior art and the present application, several exemplary drawings are given below. It should be understood that the specific shapes, configurations and illustrations in the drawings are not to be construed as limiting, in general, the practice of the present application; for example, it is within the ability of those skilled in the art to make routine adjustments or further optimizations based on the technical concepts disclosed in the present application and the exemplary drawings, for the increase/decrease/attribution of certain units (components), specific shapes, positional relationships, connection manners, dimensional ratios, and the like.
Fig. 1 is a flowchart of a machine learning fusion method of multisource sea surface physical elements provided in an embodiment of the present application;
fig. 2 is a schematic structural diagram of a time selection method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of establishment of feature engineering provided in an embodiment of the present application;
FIG. 4 is a machine learning model training diagram provided in an embodiment of the present application;
FIG. 5 provides a block diagram of a model training and reasoning process according to an embodiment of the present application;
FIG. 6 provides a buoy wind speed verification for embodiments of the present application.
Detailed Description
The present application will be described in further detail below with reference to specific embodiments thereof, with reference to the accompanying drawings.
In the description of the present application: "plurality" means two or more unless otherwise specified. The terms "first", "second", "third", and the like in this application are intended to distinguish one referenced item from another without having a special meaning in technical connotation (e.g., should not be construed as emphasizing a degree or order of importance, etc.). The terms "comprising," "including," "having," and the like, are intended to be inclusive and mean "not limited to" (some elements, components, materials, steps, etc.).
In the present application, terms such as "upper", "lower", "left", "right", "middle", and the like are generally used for easy visual understanding with reference to the drawings, and are not intended to absolutely limit the positional relationship in an actual product. Changes in these relative positional relationships are also considered to be within the scope of the present disclosure without departing from the technical concepts disclosed in the present disclosure.
Numerical model Prediction (NWP) is a method for predicting future Weather by solving a fluid mechanics and thermodynamic equation set describing a Weather evolution process through Numerical calculation under certain initial and side values according to actual conditions of the sea and the atmosphere, such as model Prediction products issued by the European center for Medium-Range Weather Forecasts (ECMWF, EC for short) and the like. The re-analysis Data (Reanalysis Data) is a comprehensive Data obtained by fusing numerical prediction products and various observation Data by using a Data assimilation method, and has the characteristics of rich content, long Data time, summary of wide observation Data and the like, such as ERA-5 re-analysis Data issued by ECMWF.
Referring to fig. 1, an embodiment of the present invention provides a machine learning fusion method for multi-source sea surface physical elements, including:
s1: acquiring multi-source data, and preprocessing the acquired multi-source data;
specifically, the multi-source data in the invention can be satellite remote sensing data, ship data or other observation data, for example, research wind field fusion is performed on a sea surface wind field, three satellites are selected to provide data support in the research, namely an HY-2B satellite, a China sea satellite (CFOSAT) and a MetOp-B satellite, but in practical application, any available polar orbit satellite or geostationary satellite can be selected to provide data support, and the method is not limited to the three satellites.
For an HY-2B satellite, HY-2B scatterometer L2B-level data with time span of 12 hours and spatial resolution of 25km × 25km is selected. The HY-2B scatterometer has about 16 tracks of data per day and can cover 90% of the sea area of the whole world. In the prior art, ECMWF is used for analyzing wind field data and TAO and NDBC buoy actual measurement data, and HY-2B wind field is subjected to overall quality analysis. Analysis shows that the HY-2B wind speed and the wind direction RMSE are respectively superior to 2m/s and 20 degrees in a wind speed range of 4-24 m/s, and the accuracy requirement of the HY-2B scatterometer for business application can be better met.
For the China-French sea satellite, a mature CAST2000 minisatellite platform is adopted, the sun synchronous orbit with the service life of 3 years and the orbit height of 521km and the descending intersection point of 07:00AM is designed, detection data are respectively transmitted to the ground stations of China and French two countries, and the ground application systems of the two countries receive and process the detection data. The satellite plays an important role in marine dynamic environment service monitoring, marine disaster monitoring, forecasting and early warning and marine scientific research. The time span of the L2B-level data of the CFOSAT satellite selected by the invention is 12 hours, the spatial resolution is 12.5km multiplied by 12.5km, the wind speed precision is 1.5m/s, and the wind direction precision is 20 degrees.
For the MetOp-B satellite, the MetOp-B jointly transmitted by the European space bureau and the European meteorological satellite development organization replaces the MetOp-A to serve as a main service observation satellite, the wind speed precision of a sea surface wind field data product provided by the MetOp-B satellite is 2m/s, and the wind speed range is 0-50 m/s. The spatial resolution of the MetOp-B wind field data selected by the invention is 12.5km multiplied by 12.5 km.
For the inspection method of the converged wind field, the buoy data is selected from Tropical atmospheric Ocean observation plan (TAO) buoy data which is more than 50km offshore and has continuous wind vector observation capacity. The buoy has high observation frequency, and the wind speed and the wind direction are observed every 10 minutes.
Referring to fig. 2, for different artificial earth satellites orbiting the earth, the time of flight through the same area may be different. In order to ensure the scientificity and the reasonability of research, the invention processes three different satellite data. Statistical analysis of the time of passage of the different satellites through the study area shows that the target area is passed at approximately time 00UTC and 12UTC, so that time 0 and time 12 (all times are universal time) are used as the target time for the fused data. Meanwhile, in order to ensure the sufficient data volume, a time window is selected to be 3 hours in the experiment, namely satellite data in three hours before and after the target moment are equally selected as the satellite data of the target moment.
S2: performing mixed interpolation on the preprocessed multi-source data to obtain standard grid data with the same resolution as NWP mode data/reanalysis data;
specifically, the remote sensing data such as the multisource satellite and the like are subjected to mixed interpolation to obtain standard grid data with the same resolution as NWP mode data/reanalysis data and the like. The step is only used for training the fusion model, and interpolation is not needed when data fusion is carried out after the fusion model is trained.
The NWP mode data/reanalysis data selected in the invention is ERA-5 reanalysis data, but in practical application, the data is not limited to be reanalyzed by ERA-5. ERA-5 is the fifth generation reanalysis data of global climate and weather for the past 40-70 years by the European mid-term weather forecasting center. The current data was divided into 1950-. ERA-5 provides hourly estimates of the number of large atmospheric, ocean and land surfaces. The ERA-5 reanalysis wind field selected by the invention has the spatial resolution of 0.25 degree multiplied by 0.25 degree.
By taking the research on wind field fusion as an example, the selected HY-2B, CFOSAT and MetOp-B satellite data are uniformly mixed, and the mixed satellite data are uniformly processed into standard grid data of 0.25 degrees multiplied by 0.25 degrees. In the interpolation processing process, because the resolution ratios of the satellite data are different, namely 12.5km multiplied by 12.5km and 25km multiplied by 25km, in order to facilitate uniform interpolation, the invention selects an inverse distance weighting interpolation algorithm to interpolate the mixed satellite data to obtain standard grid data of 0.25 degrees multiplied by 0.25 degrees, namely grid data which is consistent with the selected ERA-5 data.
S3: establishing a characteristic project according to the standard grid data and the NWP mode data/reanalysis data, and obtaining training samples required by a target fusion model;
specifically, taking research wind field fusion as an example, the satellite data and ERA-5 data after interpolation processing are both 0.25 degrees by 0.25 degrees grid data, the invention uses the satellite interpolation wind field data as a learning target, and selects the satellite interpolation grid points and the ERA-5 values around the satellite interpolation grid points (5 times 5 windows) as training characteristics for training. As shown in fig. 3, the left-hand gate green point in the graph represents the satellite interpolation data, the peripheral 5 × 5 grid points are ERA-5 data, the peripheral 5 × 5 ERA-5 grid point data are sequentially acquired as the input of the model, and the satellite interpolation data are trained as the target of the model.
S4: inputting the training sample into the target fusion model for training to obtain a data fusion model; and reasoning the NWP mode data/reanalysis data according to the data fusion model to obtain a fusion field.
Referring to fig. 4, specifically, for example, a machine learning method such as XGBoost is used as a target fusion model, and after a sufficient amount of training samples are obtained, the XGBoost machine learning method is selected to train the model, and finally the fusion model is obtained.
And performing machine reasoning by using the trained data fusion model, and performing data fusion quickly and efficiently.
For example, after a fusion model is generated by using an XGboost machine learning method, the fusion model is used for reasoning NWP mode data/reanalysis data so as to acquire a fusion wind field.
The machine learning fusion method of the multisource sea surface physical elements is characterized in that for the physical elements such as sea surface wind, sea waves and sea mark temperature, a machine learning method is adopted, numerical prediction/reanalysis data of the physical elements are trained, and the numerical prediction/reanalysis data are mapped to a model of satellite remote sensing data of the physical elements. The method mainly aims to reason out the fusion data of the whole region directly from the reanalysis data for the region which is not covered by the satellite remote sensing observation and only has the numerical prediction/reanalysis data, and even possibly obtains the fusion data with high time resolution through model reasoning when the learning capacity of the model reaches a certain level, namely, the fusion data is obtained through reasoning in the time period of the region which is not covered by the satellite.
The data fusion method and the model of the multisource sea surface physical elements can quickly perform data fusion, have low requirements on hardware or computing capacity, obtain standard grid data as a learning target by performing mixed interpolation processing on remote sensing data of multisource satellites and the like, train NWP mode data/reanalysis data as input by using a machine learning method or a deep learning method to obtain a fusion model, and infer satellite remote sensing data from the NWP mode data/reanalysis data according to a model (a trained machine learning model) of the corresponding relation between the NWP mode data/reanalysis data and the satellite remote sensing data. In the data fusion process, only a CPU is needed, interpolation operation is not needed, the requirement on computing capacity is not high, and efficient and rapid fusion can be realized.
The method obtains a final data fusion model through machine learning method training, and then uses the data fusion model to perform data fusion. Compared with an interpolation method, the machine learning method has the characteristics that: and (3) accuracy. Machine learning can find linear or nonlinear rules in objects through data, and an accurate scheme for solving problems is obtained. Along with the increase of data quantity, the precision of the method is improved; and (4) automation. The machine learning does not need manual intervention in the training process and can be automatically carried out; the speed is high. Machine learning can produce results in a few seconds or even milliseconds during the reasoning process, facilitating real-time application. The method only needs 1.5s for single-time data fusion on a platform with a CPU (Central processing Unit) of Intel (R) Xeon (R) CPU E5-2690 [email protected], and for an interpolation method, equipment such as a computer cluster and the like may be needed to ensure the speed.
The method is not limited to a certain machine learning method of the XGboost and the existing deep learning method, is not limited to remote sensing data of a certain satellite and the like and data/reanalysis data of a certain specific NWP mode, and has universal applicability to general satellite remote sensing data and general NWP mode data/reanalysis data.
Referring to fig. 5, the present invention uses a machine learning method to perform data fusion, including two operations of model training and model inference, where the upper half of the diagram is a model training process, and in the process of model training, due to the problems of resolution of remote sensing data such as satellites, interpolation processing needs to be performed on the remote sensing data such as satellites, so as to keep resolution synchronization with NWP mode data/reanalysis data, which facilitates model training, and after the completion of model training, the model inference process is performed, as shown in the lower half of the diagram, as can be seen from the diagram, the interpolation problem is not involved in the model inference process, that is, NWP mode data/reanalysis data is directly used as the input of the model, and the output of the model is the final data fusion result.
Referring to fig. 6, the data fusion method for multi-source sea surface physical elements provided by the present invention can be operated on a CPU platform, and for example, wind field fusion, single-time data fusion in northwest pacific regions (0 ° -45 °, 100 ° -160 °) can be completed in only 1.5 s. And the fused wind field data is compared with the buoy data, and the result shows that the wind speed of the fused wind field data is closer to the buoy data compared with the original wind field, the specific result is shown in fig. 6, the abscissa in the figure represents the matching sample, and the ordinate represents the wind speed, and as can be seen from the figure, the fused wind field is obviously close to the buoy data in the part marked by the black frame, and the result shows that the method for fusing the sea surface physical elements by using machine learning is effective in the aspect of accuracy.
In summary, the method for fusing sea surface physical element data by using machine learning provided by the invention is faster in speed than the traditional method, better in accuracy than the original NWP mode data/reanalysis data, and lower in hardware requirement.
Another embodiment of the invention provides a data fusion model of multisource sea surface physical elements, which is obtained by training according to the machine learning fusion method of the multisource sea surface physical elements.
Another embodiment of the present invention provides a computer apparatus, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the machine learning fusion method of multisource sea surface physical elements when executing the computer program.
Another embodiment of the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of a method of machine learning fusion of multisource surface physical elements.
All the technical features of the above embodiments can be arbitrarily combined (as long as there is no contradiction between the combinations of the technical features), and for brevity of description, all the possible combinations of the technical features in the above embodiments are not described; these examples, which are not explicitly described, should be considered to be within the scope of the present description.
The present application has been described in considerable detail with reference to certain embodiments and examples thereof. It should be understood that several conventional adaptations or further innovations of these specific embodiments may also be made based on the technical idea of the present application; however, such conventional modifications and further innovations may also fall within the scope of the claims of the present application as long as they do not depart from the technical idea of the present application.
Claims (10)
1. A machine learning fusion method for multi-source sea surface physical elements is characterized by comprising the following steps:
acquiring multi-source data and NWP mode data/reanalysis data, and preprocessing the acquired multi-source data;
performing mixed interpolation on the preprocessed multi-source data to obtain standard grid data with the same resolution as the NWP mode data/reanalysis data;
establishing a characteristic project according to the standard grid data and the NWP mode data/reanalysis data, and obtaining training samples required by a target fusion model;
inputting the training sample into the target fusion model for training to obtain a data fusion model; and reasoning the NWP mode data/reanalysis data according to the data fusion model to obtain a fusion field.
2. The machine learning fusion method for multisource sea surface physical elements according to claim 1, wherein when the multisource data is subjected to mixed interpolation, the method specifically comprises the following steps:
judging whether the data points are overlapped;
if the data points are not overlapped, directly carrying out interpolation;
and if the data points are overlapped, selecting the data closest to the interpolation time point for interpolation.
3. The machine learning fusion method of multisource sea surface physical elements of claim 1, wherein the multisource data is interpolated by an inverse distance weighted interpolation algorithm when performing the hybrid interpolation.
4. The method of machine-learning fusion of multisource sea surface physical elements of claim 1, wherein the standard grid data is 0.25 ° x 0.25 ° standard grid data.
5. The method of machine learning fusion of multisource sea surface physical elements of claim 1, in which the NWP mode data/reanalysis data is ERA-5 reanalysis data.
6. The machine learning fusion method for multisource sea surface physical elements according to claim 5, wherein when establishing feature engineering according to the standard grid data and the NWP mode data/reanalysis data, specifically:
and (3) using the satellite interpolation data as a learning target, and selecting a satellite interpolation grid point and an ERA-5 value of 5 multiplied by 5 around the satellite interpolation grid point as training characteristics for training.
7. The machine-learning fusion method of multisource sea surface physical elements of claim 1, wherein the target fusion model is an XGBoost learning method.
8. A data fusion model of multisource surface physical elements, characterized in that the data fusion model is trained according to the machine learning fusion method of multisource surface physical elements of any of claims 1-7.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210253149.5A CN114580561A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method and model for multisource sea surface physical elements |
| CN202310626576.8A CN116644379A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method, equipment and medium for multisource sea surface physical elements |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210253149.5A CN114580561A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method and model for multisource sea surface physical elements |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310626576.8A Division CN116644379A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method, equipment and medium for multisource sea surface physical elements |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114580561A true CN114580561A (en) | 2022-06-03 |
Family
ID=81774520
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210253149.5A Pending CN114580561A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method and model for multisource sea surface physical elements |
| CN202310626576.8A Pending CN116644379A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method, equipment and medium for multisource sea surface physical elements |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310626576.8A Pending CN116644379A (en) | 2022-03-15 | 2022-03-15 | Machine learning fusion method, equipment and medium for multisource sea surface physical elements |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN114580561A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235658A (en) * | 2023-07-13 | 2023-12-15 | 国家卫星海洋应用中心 | Method, device and equipment for fusing wind field data and readable storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117609932B (en) * | 2023-09-22 | 2024-05-24 | 宁波市气象网络与装备保障中心 | Sea surface visibility multi-source data fusion method |
| CN117150437B (en) * | 2023-10-31 | 2024-01-30 | 航天宏图信息技术股份有限公司 | Multi-source satellite sea surface wind field data fusion method, device, equipment and medium |
| CN117633712B (en) * | 2024-01-24 | 2024-04-19 | 国家卫星海洋应用中心 | Sea level height data fusion method, device and equipment based on multi-source data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975763A (en) * | 2016-04-29 | 2016-09-28 | 国家卫星海洋应用中心 | Fusion method and device of multisource sea surface wind field |
| CN112308973A (en) * | 2020-10-30 | 2021-02-02 | 无锡九方科技有限公司 | Three-dimensional ocean element field intelligent construction method and system based on machine learning |
| CN112800634A (en) * | 2021-04-07 | 2021-05-14 | 水利部交通运输部国家能源局南京水利科学研究院 | Rainfall estimation method and system coupling dry-wet state identification and multi-source information fusion |
| CN113720310A (en) * | 2021-08-31 | 2021-11-30 | 中煤科工集团重庆研究院有限公司 | Roadway photographic geology compiling and recording method and system |
| CN113988351A (en) * | 2020-07-27 | 2022-01-28 | 海南省气象科学研究所 | Numerical mode air temperature forecast product release method based on space-time local model |
-
2022
- 2022-03-15 CN CN202210253149.5A patent/CN114580561A/en active Pending
- 2022-03-15 CN CN202310626576.8A patent/CN116644379A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975763A (en) * | 2016-04-29 | 2016-09-28 | 国家卫星海洋应用中心 | Fusion method and device of multisource sea surface wind field |
| CN113988351A (en) * | 2020-07-27 | 2022-01-28 | 海南省气象科学研究所 | Numerical mode air temperature forecast product release method based on space-time local model |
| CN112308973A (en) * | 2020-10-30 | 2021-02-02 | 无锡九方科技有限公司 | Three-dimensional ocean element field intelligent construction method and system based on machine learning |
| CN112800634A (en) * | 2021-04-07 | 2021-05-14 | 水利部交通运输部国家能源局南京水利科学研究院 | Rainfall estimation method and system coupling dry-wet state identification and multi-source information fusion |
| CN113720310A (en) * | 2021-08-31 | 2021-11-30 | 中煤科工集团重庆研究院有限公司 | Roadway photographic geology compiling and recording method and system |
Non-Patent Citations (1)
| Title |
|---|
| S. SALCEDO-SANZ ETAL.: ""Machine learning information fusion in Earth observation: A comprehensive review of methods, applications and data sources"", 《ELSEVIER》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235658A (en) * | 2023-07-13 | 2023-12-15 | 国家卫星海洋应用中心 | Method, device and equipment for fusing wind field data and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116644379A (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114580561A (en) | Machine learning fusion method and model for multisource sea surface physical elements | |
| CN112905560B (en) | Air pollution prediction method based on multi-source time-space big data deep fusion | |
| Ruwali et al. | Implementation of hybrid deep learning model (LSTM-CNN) for ionospheric TEC forecasting using GPS data | |
| Bentamy et al. | Gridded surface wind fields from Metop/ASCAT measurements | |
| CN105975763A (en) | Fusion method and device of multisource sea surface wind field | |
| CN112070286A (en) | Rainfall forecast early warning system for complex terrain watershed | |
| CN104636608A (en) | Direct assimilation method of MODIS satellite data | |
| CN112949178A (en) | Sea surface wind field wind speed intelligent prediction forecasting system based on deep learning, computer equipment and storage medium | |
| CN111859054B (en) | Meteorological satellite data processing method and device | |
| CN113901384A (en) | Ground PM2.5 concentration modeling method considering global spatial autocorrelation and local heterogeneity | |
| CN111210483A (en) | Simulated satellite cloud picture generation method based on generation of countermeasure network and numerical mode product | |
| Fan et al. | A comparative study of four merging approaches for regional precipitation estimation | |
| CN114898138A (en) | Method for predicting forest fire based on landform and remote sensing data | |
| CN112668615B (en) | Satellite cloud picture prediction method based on depth cross-scale extrapolation fusion | |
| Mazorra-Aguiar et al. | Solar radiation forecasting with statistical models | |
| CN113984198A (en) | Short wave radiation prediction method and system based on convolutional neural network | |
| CN117371316A (en) | Deep learning-based stationary satellite solar short wave radiation inversion method and readable storage medium | |
| KR101870743B1 (en) | Tidal flat experience index forecasting apparatus and the method thereof | |
| CN113486515B (en) | Sub-season-annual scale integrated climate mode set prediction system | |
| Ho et al. | Multivariate convolutional LSTMs for relative humidity forecasting | |
| CN115511192A (en) | Rainfall forecasting method and system based on lightning data assimilation | |
| Bhushan et al. | Comparison of NYHOPS hydrodynamic model SST predictions with satellite observations in the Hudson River tidal, estuarine, and coastal plume region | |
| CN112162336A (en) | Visibility prediction method and device based on two-dimensional meteorological element field | |
| Champenois et al. | Real-time reconstruction of 3D ocean temperature fields from reanalysis data and satellite and buoy surface measurements | |
| Senyurek et al. | Assessment of interpolation errors of cygnss soil moisture estimations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220603 |