CN115236772A - Data quality control system and method for drifting observation instrument - Google Patents
Data quality control system and method for drifting observation instrument Download PDFInfo
- Publication number
- CN115236772A CN115236772A CN202210710047.1A CN202210710047A CN115236772A CN 115236772 A CN115236772 A CN 115236772A CN 202210710047 A CN202210710047 A CN 202210710047A CN 115236772 A CN115236772 A CN 115236772A
- Authority
- CN
- China
- Prior art keywords
- data
- quality control
- drifting
- module
- checking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/18—Testing or calibrating meteorological apparatus
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01W—METEOROLOGY
- G01W1/00—Meteorology
- G01W1/02—Instruments for indicating weather conditions by measuring two or more variables, e.g. humidity, pressure, temperature, cloud cover or wind speed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Environmental & Geological Engineering (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Ecology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- General Physics & Mathematics (AREA)
- Atmospheric Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention relates to a drifting observation instrument data quality control system and a drifting observation instrument data quality control method. The meteorological element quality control module of the system is used for performing quality control on meteorological data of the drifting observation instrument, the hydrological element quality control module is used for performing quality control on sea temperature data of the drifting observation instrument, reasonably accepting or rejecting data of relative deviation, integrating received meteorological element data of the drifting observation instrument, outputting relatively accurate meteorological element data of the drifting observation instrument, and improving monitoring precision of the instrument.
Description
Technical Field
The invention relates to the field of sea area data measurement, in particular to a data quality control system and a data quality control method for a drifting observation instrument.
Background
The world climate research project in 1982 proposed that buoy observations could not be worth estimating the value of ocean and climate research. In 2014, the weather drift visualizer is jointly developed by relevant units of the national oceanic office and the Chinese meteorological office, the weather drift visualizer developed by the project is based on a Beidou navigation communication satellite, the observation elements comprise marine elements and meteorological elements, and the drift visualizer needs to research the meteorological element data characteristics of the drift visualizer, determine the drift visualizer meteorological element data quality control method and improve the element data availability because the drift visualizer observes the differences of the geographic positions and environments of different time positions and causes different observation data quality control standards.
The data sources of the ocean elements and the meteorological elements are the meteorological drift visualizer thrown into the sea area, the meteorological elements observed by the meteorological drift visualizer comprise air temperature, air pressure and wind speed, and the observation error sources of the meteorological elements of the meteorological drift visualizer mainly comprise two aspects, namely system errors and random errors. The system errors include interference caused by factors such as sensor precision, sampling algorithm and instrument replacement, generally, the system errors cannot be avoided, and the values of the system errors can be reduced as much as possible only through data correction.
The random error is mainly determined by the running condition of the instrument, and when the weather condition is severe and the instrument is in an abnormal working state, the obtained observation data has larger deviation. Therefore, the data quality of the drift visualizer needs to be controlled, and data with relative deviation needs to be reasonably accepted and rejected, so that a control system specially aiming at the data quality of the drift visualizer does not exist at present.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides a drift visualizer data quality control system and a drift visualizer data quality control method.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a drifting observation instrument data quality control system comprises a working state quality control module, a meteorological element quality control module and a hydrological element quality control module.
Wherein, meteorological element quality control module includes:
the climatological boundary value inspection module is used for analyzing the climatological characteristics of global marine meteorological elements and determining the climatological boundary value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment;
the climate extreme value checking module is used for checking whether the value of each meteorological element exceeds the maximum value and the minimum value which are historically appeared;
the internal consistency checking module is used for checking the internal consistency between similar elements, and the relation between meteorological element records observed at the same time accords with the checking of certain physical relation;
and the time consistency checking module is used for checking whether the change of the data of the meteorological record in a certain time range has a specific rule.
Further, the working state quality control module comprises:
the data format checking module is used for preprocessing a binary satellite transmission data packet of the drifting observation instrument, checking the preprocessed data format and file size, and removing data files with abnormal format and abnormal size;
and the instrument state inspection module is used for judging the working state of the instrument according to the power supply voltage of the drifting observation instrument and rejecting all meteorological and hydrological data obtained when the instrument is in an abnormal working state, wherein the power supply voltage of the instrument is less than 6V.
Further, hydrologic elements matter accuse module includes:
the preprocessing module is used for taking the homologous data of the longitudes or the latitudes which have a certain distance in the preprocessing process as the repeated data;
the reasonability checking module is used for checking the reasonability of error data of which the longitude and latitude and the sea temperature are beyond the effective range and the sea temperature value appears in a land site in the source data;
and the internal consistency checking module is used for checking the moving speed rationality of the drifting observation instrument and judging the rationality of the time or space change gradient of the peak value appearing in the continuous sea surface temperature data sequence.
Furthermore, the hydrological element quality control module also comprises a Bayes consistency module, and the module is based on an objective quality control method of a Bayes theory, calculates the probability density of sea temperature errors by taking a high-resolution sea surface temperature analysis product obtained by processing satellite remote sensing sea surface temperature data by an optimal interpolation method as a background field, sets related parameters based on experience and sensitivity analysis, obtains Bayes interval estimation, and deletes data outside a drop area.
Further, the meteorological element quality control module is used for carrying out quality control on meteorological data of the drifting observation instrument.
Furthermore, the hydrological element quality control module is used for performing quality control on the sea temperature data of the drifting observation instrument.
And further, the data after the quality control of the meteorological element quality control module and the hydrological element quality control module are displayed through the display module.
Further, a control method of the drift visualizer data quality control system comprises the following steps:
s1, error analysis:
s2, controlling data quality;
wherein, the error analysis comprises system error analysis and random error analysis;
when the system error is analyzed, the value of the system error is reduced as much as possible through data correction; and (3) analyzing the random error: when the weather is severe and the instrument is in an abnormal working state, the observed air temperature value also changes suddenly, and the air temperature value at the sudden change point is regarded as error data and discarded.
S2, the data quality control comprises the following steps:
s21, performing quality control on meteorological data of the drifting observation instrument;
s22, performing quality control on the sea temperature data of the drifting observation instrument;
s21, the quality control of meteorological data of the drifting observation instrument comprises the following steps:
s211, analyzing the climatological characteristics of global oceanographic meteorological elements, determining the climatological limit value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment, and checking whether the data exceeds the climatological critical value which cannot be exceeded from the climatological perspective;
s212, checking whether the meteorological element values exceed the historical maximum value and minimum value;
s213, checking that the relation among meteorological element records observed at the same time accords with certain physical relation, and checking the internal consistency among similar elements;
s214, checking whether the change of the data of the weather record in a certain time range has a specific rule or not.
Further, the step S22 of performing quality control on the sea temperature data of the drift observer includes the following steps:
s221, in the data acquisition and transmission process, data repetition is caused when the data are subjected to repeated supplementary transmission, in the preprocessing, longitudes or latitudes are different by a certain distance, homologous data in similar time are regarded as repeated data, and for a series of repeated data, the data in the queue are retained or all the repeated data in the queue are deleted based on the sea surface temperature difference condition;
s222, carrying out rationality check on the longitude and latitude appearing in the source data, error data of which the sea temperature exceeds the effective range and the condition of the sea temperature value appearing in a land site;
s223, checking the moving speed reasonability of the drifting observation instrument, and judging the reasonability of the time or space change gradient of a peak value appearing in the continuous sea surface temperature data sequence;
s224, an objective quality control method based on Bayes theory is adopted, a high-resolution sea surface temperature analysis product obtained after satellite remote sensing sea surface temperature data are processed through an optimal interpolation method is used as a background field, the probability density of sea temperature errors is calculated, relevant parameters are set based on experience and sensitivity analysis, bayes interval estimation is obtained, and data outside a drop zone are deleted.
The invention has the beneficial effects that: the meteorological element quality control module of the system is used for performing quality control on meteorological data of the drifting observation instrument, the hydrological element quality control module is used for performing quality control on sea temperature data of the drifting observation instrument, reasonably accepting or rejecting data of relative deviation, integrating received meteorological element data of the drifting observation instrument, outputting relatively accurate meteorological element data of the drifting observation instrument, and improving detection precision.
Drawings
FIG. 1 is a graph of a power supply voltage variation during operation of a certain drift viewer;
FIG. 2 is a graph showing the change in air temperature observed by a drift viewer with a change in power supply voltage as shown in FIG. 1;
FIG. 3 is a graph of air temperature data of the drift visualizer in a first time period after data quality control;
FIG. 4 is a graph of air temperature data of the drift visualizer in a second time period after data quality control;
FIG. 5 is a graph of wind speed data obtained for a drift viewer over a first time period without data quality control;
FIG. 6 is a wind speed data graph of the drift viewer obtained over a second time period without data quality control;
fig. 7 is a schematic diagram of a control system corresponding to the control method of the present application.
Detailed Description
As shown in fig. 1 to 7, a drifting observation instrument data quality control system includes a working state quality control module 1, a meteorological element quality control module 11 and a hydrological element quality control module 12, wherein the working state quality control module includes:
the data format checking module 13 is used for preprocessing the binary satellite transmission data packet of the drift viewer, checking the preprocessed data format and file size, and removing the data files with abnormal format and abnormal size;
the instrument state checking module 14 is used for judging the working state of the instrument according to the power supply voltage of the drifting observation instrument and eliminating all meteorological and hydrological data obtained when the instrument is in an abnormal working state, wherein the power supply voltage of the instrument is less than 6V;
the working state quality control module 1 judges whether the instrument normally operates, and after the instrument normally operates, the instrument is divided into meteorological elements or hydrological elements for quality control, the meteorological elements are subjected to quality control through the meteorological element quality control module 11, and the hydrological elements are subjected to quality control through the hydrological element quality control module 12;
when the instrument state is checked, the working state of the instrument is judged according to the power supply voltage of the drifting observation instrument, and all data obtained when the instrument is in an abnormal working state (the power supply voltage is less than 6V) are eliminated.
Further, the meteorological element quality control module 11 includes:
the climatological boundary value inspection module 111 is used for analyzing the climatological characteristics of global marine meteorological elements and determining the climatological boundary value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment; it is checked whether the data exceeds a critical value of a meteorological element which it is not possible to exceed from a climatological point of view.
A climate extreme value checking module 112, configured to check whether the value of each meteorological element exceeds a historical maximum value and a historical minimum value;
the method comprises the steps of dividing a global sea area according to tropical seas, temperate seas and cold seas, classifying seasons according to spring, summer, autumn and winter, respectively counting historical meteorological element climatic extreme values of the seas in different temperature zones in different seasons, and checking the climatic extreme values of data according to the historical climatic statistical values in a specified sea area and time domain range.
The internal consistency checking module 113 is used for checking that the relation among meteorological element records observed at the same time accords with the checking of certain physical relation and is used for checking the internal consistency among similar elements;
the internal consistency check is generally a logical check, and for example, the positive point value is greater than or equal to the minimum value, and the positive point value is less than or equal to the maximum value, as shown in table 1.
TABLE 1 internal consistency Algorithm List of elements
And the time consistency checking module 114 is used for checking whether the change of the data of the weather record in a certain time range has a specific rule.
The change of meteorological elements such as air temperature, air pressure and the like has a certain rule, and abnormity can occur when jumping occurs or the meteorological elements are not changed for a long time, so that the maximum allowable change rate check and the minimum required change rate check are carried out.
(1) Maximum allowable rate of change of meteorological element
The difference between the current observed value and the previous value is checked for being less than a specified maximum allowable rate of change. In extreme weather conditions, meteorological variables may change unusually, in which case the correct data may be greater than or equal to a specified maximum allowable rate of change, and further verification may be performed.
(2) Minimum rate of change check of meteorological elements
The update period of the indication value of the observation value is 10min, and the minimum change rate which should be changed of the meteorological observation value is specified in the past 60min, so that whether the value is correct or not can be verified.
Further, the hydrological element quality control module 12 includes:
the preprocessing module 121 repeats data when multiple data supplementary transmissions are performed during data acquisition and transmission. In the preprocessing, the longitude or the latitude are different by a certain distance, and the homologous data in the similar time are regarded as the repeated data. And for a series of repeated data, reserving the data in the queue or deleting all the repeated data in the queue based on the sea table temperature difference condition.
And the reasonability checking module 122 is used for checking the reasonability of the error data of which the longitude and latitude and the sea temperature are beyond the effective range and the condition that the sea temperature value is at a land site. In the step, reasonable latitude and longitude and a reasonable sea surface temperature range are set, unreasonable data are removed, and data with unreasonable positioning (such as a sea surface temperature measurement value at a land site) are deleted.
The internal consistency check module 123 checks the moving speed rationality of the drifting observer and judges the rationality of the time or space variation gradient of the peak value appearing in the continuous sea surface temperature data sequence. The moving speed reasonableness check of the drifting observation instrument is based on different sea area moving track rules, and whether an abnormal position or an abnormal speed which is larger than the maximum speed setting exists in the moving track is judged; the maximum change gradient value of the sea surface temperature in space-time is set in the rationality check of the sea surface temperature change gradient, and different threshold values are set aiming at different sea areas in consideration of normal fluctuation existing between continuous records caused by instrument noise, so that abnormal values are eliminated.
Further, the hydrological element quality control module 12 further includes a bayesian consistency module 124, which uses an objective quality control method based on bayesian theory to calculate the probability density of the sea temperature error by using a high-resolution sea surface temperature analysis product obtained by processing satellite remote sensing sea surface temperature data by an optimal interpolation method as a background field, sets related parameters based on experience and sensitivity analysis to obtain bayesian interval estimation, and deletes data outside the drop zone. Compared with the traditional outlier detection method, the Bayesian probability theory better considers the factors such as the uncertainty of the background field, the error caused by the position difference of the observation point and the reference grid point, the field data measurement error caused by the instrument noise and the like.
In this embodiment, the meteorological element quality control module 11 is configured to perform quality control on meteorological data of the drifting observation instrument, the hydrological element quality control module 12 is configured to perform quality control on sea temperature data of the drifting observation instrument, and data after quality control of the meteorological element quality control module 11 and the hydrological element quality control module 12 is displayed through the display module 2.
In summary, the meteorological element quality control module of the system is used for performing quality control on meteorological data of the drifting observation instrument, the hydrological element quality control module is used for performing quality control on sea temperature data of the drifting observation instrument, reasonably accepting or rejecting data of relative deviation, integrating received meteorological element data of the drifting observation instrument, outputting relatively accurate meteorological element data of the drifting observation instrument, and improving detection precision.
A drift visualizer meteorological element data quality control method comprises the following steps:
s1, error analysis:
s2, controlling data quality;
wherein the error analysis comprises a systematic error analysis and a random error analysis.
Further, in the analysis of the system error, the value of the system error is reduced as much as possible by data correction.
Further, the random error analysis: when the weather is severe and the instrument is in an abnormal working state, the observed air temperature value also changes suddenly, and the air temperature value at the sudden change point is regarded as error data and discarded.
The random error is mainly determined by the running condition of the instrument, and when the weather condition is severe and the instrument is in an abnormal working state, the obtained observation data has larger deviation. As shown in fig. 1, it is a power supply voltage change graph when a certain drift visualizer operates, and as shown in fig. 2, it is a change graph of the observed air temperature when the power supply voltage changes as shown in fig. 1 when a certain drift visualizer operates.
It can be seen from comparison that, when the power voltage of the instrument changes suddenly, that is, the instrument is in an abnormal working state, the observed air temperature value changes suddenly, and the air temperature values at these points can be regarded as error data and discarded.
Further, the S2, controlling data quality includes:
s21, performing quality control on meteorological data of the drifting observation instrument;
and S22, performing quality control on the sea temperature data of the drift viewer.
Further, the step S21 of performing quality control on meteorological data of the drifting observation instrument includes the steps of:
s211, analyzing the climatological characteristics of global oceanographic meteorological elements, determining the climatological limit value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment, and checking whether the data exceeds the climatological critical value which cannot be exceeded from the climatological perspective;
s212, checking whether the meteorological element values exceed the historical maximum value and minimum value;
s213, checking that the relation among the meteorological element records observed at the same time accords with certain physical relation, and checking the internal consistency among the similar elements;
s214, checking whether the change of the data of the weather record in a certain time range has a specific rule or not.
Further, the step S22 of performing quality control on the sea temperature data of the drifting observation instrument includes the steps of:
s221, in the data acquisition and transmission process, data repetition is caused when the data are subjected to repeated supplementary transmission, in the preprocessing, longitudes or latitudes are different by a certain distance, homologous data in similar time are regarded as repeated data, and for a series of repeated data, the data in the queue are retained or all the repeated data in the queue are deleted based on the sea surface temperature difference condition;
s222, carrying out rationality check on the longitude and latitude appearing in the source data, error data of which the sea temperature exceeds the effective range and the condition of the sea temperature value appearing in a land site;
s223, checking the moving speed rationality of the drifting observation instrument, and judging the time or space change gradient rationality of the peak value appearing in the continuous sea surface temperature data sequence;
s224, an objective quality control method based on Bayes theory is adopted, a high-resolution sea surface temperature analysis product obtained after satellite remote sensing sea surface temperature data are processed through an optimal interpolation method is used as a background field, the probability density of sea temperature errors is calculated, relevant parameters are set based on experience and sensitivity analysis, bayes interval estimation is obtained, and data outside a drop area are deleted.
Further, in S224, based on an objective quality control method of the bayesian theory, a probability density of the sea temperature error is calculated, related parameters are set based on experience and sensitivity analysis, bayesian interval estimation is obtained, and data outside the drop zone is deleted.
FIG. 3 is a graph of air temperature data of the drift visualizer in a first time period after data quality control; FIG. 4 is a graph of air temperature data of the drift visualizer in a second time period after data quality control; FIG. 5 is a graph of wind speed data obtained from a drift viewer over a first time period without data quality control; FIG. 6 is a wind speed data graph of the obtained drift viewer in a second time period without data quality control.
The method performs system error analysis and random error analysis on meteorological element data of the drifting observation instrument, performs quality control on the meteorological data of the drifting observation instrument, performs quality control on sea temperature data of the drifting observation instrument, integrates received meteorological element data of the drifting observation instrument, outputs relatively accurate meteorological element data of the drifting observation instrument, and improves the precision of the meteorological data of the drifting observation instrument
The foregoing shows and describes the general principles, principal features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (10)
1. The utility model provides a drifting observation appearance data quality control system, includes operating condition matter accuse module, meteorological element matter accuse module and hydrology element matter accuse module, its characterized in that, meteorological element matter accuse module includes:
the climatological boundary value inspection module is used for analyzing the climatological characteristics of global marine meteorological elements and determining the climatological boundary value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment;
the weather extreme value checking module is used for checking whether the value of each meteorological element exceeds the historical maximum value and minimum value;
the internal consistency checking module is used for checking the internal consistency between similar elements, and the relation between meteorological element records observed at the same time accords with the checking of certain physical relation;
and the time consistency checking module is used for checking whether the change of the data of the weather record has a specific rule in a certain time range.
2. The data quality control system of the drifting observer according to claim 1, wherein the operating state quality control module comprises:
the data format checking module is used for preprocessing the binary satellite transmission data packet of the drift viewer, checking the preprocessed data format and file size, and removing the data files with abnormal format and abnormal size;
and the instrument state inspection module is used for judging the working state of the instrument according to the power supply voltage of the drifting observation instrument and rejecting all meteorological and hydrological data obtained when the instrument is in an abnormal working state, wherein the power supply voltage of the instrument is less than 6V.
3. The drifting observer data quality control system according to claim 1 or 2, wherein the hydrologic factor quality control module comprises:
the preprocessing module is used for taking the homologous data of the longitudes or the latitudes which have a certain distance in the preprocessing process as the repeated data;
the reasonability checking module is used for checking the reasonability of error data of which the longitude and latitude and the sea temperature exceed the effective range and the condition that the sea temperature value appears in a land site in the source data;
and the internal consistency checking module is used for checking the moving speed rationality of the drifting observation instrument and judging the rationality of the time or space change gradient of the peak value appearing in the continuous sea surface temperature data sequence.
4. The drifting observation instrument data quality control system according to claim 3, wherein the hydrological element quality control module further comprises a Bayesian consistency module, the Bayesian consistency module is based on an objective quality control method of Bayesian theory, a high-resolution sea surface temperature analysis product obtained by processing satellite remote sensing sea surface temperature data through an optimal interpolation method is used as a background field, the probability density of sea temperature errors is calculated, relevant parameters are set based on experience and sensitivity analysis, bayesian interval estimation is obtained, and data outside a landing area are deleted.
5. The data quality control system of claim 3, wherein the meteorological element quality control module is configured to perform quality control on meteorological data of the drifting observer, and the hydrological element quality control module is configured to perform quality control on sea temperature data of the drifting observer.
6. The drifting viewer data quality control system of claim 5, wherein the data after the meteorological element quality control module and the hydrological element quality control module are controlled is displayed through a display module.
7. The control method of the drift viewer data quality control system based on claim 3, characterized by comprising the following steps:
s1, error analysis:
s2, controlling data quality;
the error analysis comprises system error analysis and random error analysis;
when analyzing the system error, the value of the system error is reduced as much as possible through data correction; and (3) analyzing the random error: when the weather condition is severe and the instrument is in an abnormal working state, the observed temperature value also changes suddenly, and the temperature value at the sudden change point is regarded as error data and discarded.
8. The control method of the drifting observation instrument data quality control system according to claim 7, wherein the S2 data quality control comprises:
s21, performing quality control on meteorological data of the drifting observation instrument;
s22, performing quality control on the sea temperature data of the drifting observation instrument;
9. the method for controlling the data quality control system of the drifting observer according to claim 8, wherein the step S21 of performing quality control on the meteorological data of the drifting observer includes the following steps:
s211, analyzing the climatological characteristics of global oceanographic meteorological elements, determining the climatological limit value of the drift visualizer by combining the design principle of the drift visualizer and the performance of equipment, and checking whether the data exceeds the climatological critical value which cannot be exceeded from the climatological perspective;
s212, checking whether the meteorological element values exceed the historical maximum value and minimum value;
s213, checking that the relation among the meteorological element records observed at the same time accords with certain physical relation, and checking the internal consistency among the similar elements;
s214, checking whether the change of the data of the weather record in a certain time range has a specific rule or not.
10. The control method of the data quality control system of the drifting observation instrument according to claim 8, wherein the step S22 of performing quality control on the sea temperature data of the drifting observation instrument includes the steps of:
s221, in the data acquisition and transmission process, data repetition is caused when the data are subjected to repeated supplementary transmission, in the preprocessing, longitudes or latitudes are different by a certain distance, homologous data in similar time are regarded as repeated data, and for a series of repeated data, the data in the queue are retained or all the repeated data in the queue are deleted based on the sea surface temperature difference condition;
s222, carrying out rationality check on the longitude and latitude and error data of which the sea temperature exceeds the effective range in the source data and the condition that the sea temperature value appears in a land site;
s223, checking the moving speed reasonability of the drifting observation instrument, and judging the reasonability of the time or space change gradient of a peak value appearing in the continuous sea surface temperature data sequence;
s224, an objective quality control method based on Bayes theory is adopted, a high-resolution sea surface temperature analysis product obtained after satellite remote sensing sea surface temperature data are processed through an optimal interpolation method is used as a background field, the probability density of sea temperature errors is calculated, relevant parameters are set based on experience and sensitivity analysis, bayes interval estimation is obtained, and data outside a drop zone are deleted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710047.1A CN115236772A (en) | 2022-06-22 | 2022-06-22 | Data quality control system and method for drifting observation instrument |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710047.1A CN115236772A (en) | 2022-06-22 | 2022-06-22 | Data quality control system and method for drifting observation instrument |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115236772A true CN115236772A (en) | 2022-10-25 |
Family
ID=83669255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210710047.1A Pending CN115236772A (en) | 2022-06-22 | 2022-06-22 | Data quality control system and method for drifting observation instrument |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115236772A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115630878A (en) * | 2022-12-21 | 2023-01-20 | 国家卫星海洋应用中心 | Quality control method and quality control device for buoy observation data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160105065A (en) * | 2015-02-27 | 2016-09-06 | 순천대학교 산학협력단 | System and method for transmitting oceanic environment observation data in real time by using drifting buoy |
| CN110989046A (en) * | 2019-12-25 | 2020-04-10 | 中国气象局气象探测中心 | Data quality control method and system for anchorage buoy station |
| CN113761019A (en) * | 2021-07-20 | 2021-12-07 | 浙江省水利水电勘测设计院 | Statistical post-processing correction method for meteorological hydrological ensemble prediction |
| CN114235055A (en) * | 2021-12-25 | 2022-03-25 | 山东广为海洋科技有限公司 | Data acquisition system of ocean monitoring equipment |
| CN114490622A (en) * | 2022-03-10 | 2022-05-13 | 中国科学院大气物理研究所 | Automatic ocean temperature and salinity observation data quality control method and system based on climate state range threshold |
-
2022
- 2022-06-22 CN CN202210710047.1A patent/CN115236772A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160105065A (en) * | 2015-02-27 | 2016-09-06 | 순천대학교 산학협력단 | System and method for transmitting oceanic environment observation data in real time by using drifting buoy |
| CN110989046A (en) * | 2019-12-25 | 2020-04-10 | 中国气象局气象探测中心 | Data quality control method and system for anchorage buoy station |
| CN113761019A (en) * | 2021-07-20 | 2021-12-07 | 浙江省水利水电勘测设计院 | Statistical post-processing correction method for meteorological hydrological ensemble prediction |
| CN114235055A (en) * | 2021-12-25 | 2022-03-25 | 山东广为海洋科技有限公司 | Data acquisition system of ocean monitoring equipment |
| CN114490622A (en) * | 2022-03-10 | 2022-05-13 | 中国科学院大气物理研究所 | Automatic ocean temperature and salinity observation data quality control method and system based on climate state range threshold |
Non-Patent Citations (1)
| Title |
|---|
| 张灿: "海洋气象漂流观测仪资料应用***的设计与实现", 《南京信息工程大学硕士学位论文》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115630878A (en) * | 2022-12-21 | 2023-01-20 | 国家卫星海洋应用中心 | Quality control method and quality control device for buoy observation data |
| CN115630878B (en) * | 2022-12-21 | 2023-08-22 | 国家卫星海洋应用中心 | Quality control method and quality control device for buoy observation data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Casson et al. | Simulation and extremal analysis of hurricane events | |
| CN113919231B (en) | PM2.5 concentration space-time change prediction method and system based on space-time diagram neural network | |
| Hong et al. | Basis for recommending an update of wind velocity pressures in Canadian design codes | |
| Cummings | Ocean data quality control | |
| CN104820250A (en) | Processing method for detecting clouds on sea by polar orbit meteorological satellite visible and infrared radiometer (VIRR) | |
| CN106897957B (en) | Automatic weather station real-time data quality control method based on PCA and PSO-E L M | |
| CN113743013A (en) | XGboost-based temperature prediction data correction method | |
| Lucio-Eceiza et al. | Quality control of surface wind observations in northeastern North America. Part II: Measurement errors | |
| CN116205541B (en) | Method and device for evaluating influence of local pollution source on environmental air quality | |
| CN115100819B (en) | Landslide hazard early warning method and device based on big data analysis and electronic equipment | |
| CN115236772A (en) | Data quality control system and method for drifting observation instrument | |
| Lucio-Eceiza et al. | Quality control of surface wind observations in Northeastern North America. Part I: Data management issues | |
| CN113704693A (en) | High-precision effective wave height data estimation method | |
| Foresti et al. | On the spatial distribution of rainfall nowcasting errors due to orographic forcing | |
| CN114925047A (en) | Online filling method for Antarctic vision acuity missing value | |
| Zhu et al. | Validation of rainfall erosivity estimators for mainland China | |
| CN114882373A (en) | Multi-feature fusion sandstorm prediction method based on deep neural network | |
| CN106772697A (en) | Sea of clouds natural landscape forecasting procedure and system | |
| Young | Detecting and removing inhomogeneities from long-term monthly sea level pressure time series | |
| CN113742929B (en) | Data quality evaluation method for grid point weather condition | |
| CN114047508B (en) | Sea radar detection range statistical evaluation method and system | |
| CN115857056A (en) | Weather forecasting method and system for fast updating cyclic assimilation by strong convection | |
| CN115269568A (en) | Method for developing data set of oceanographic anchorage buoy | |
| Ebert et al. | Incorporating satellite observations of “no rain” in an Australian daily rainfall analysis | |
| CN113391376B (en) | Terrestrial cloud detection method for AMSU-A data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |