CN117785847A - Cloud filtering method for GPS static drift data, computer equipment and medium - Google Patents
Cloud filtering method for GPS static drift data, computer equipment and medium Download PDFInfo
- Publication number
- CN117785847A CN117785847A CN202311567470.1A CN202311567470A CN117785847A CN 117785847 A CN117785847 A CN 117785847A CN 202311567470 A CN202311567470 A CN 202311567470A CN 117785847 A CN117785847 A CN 117785847A
- Authority
- CN
- China
- Prior art keywords
- data
- gps
- cloud
- filtering method
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000003068 static effect Effects 0.000 title claims abstract description 29
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 28
- 238000004140 cleaning Methods 0.000 claims abstract description 10
- 238000003860 storage Methods 0.000 claims description 9
- 238000013500 data storage Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000010219 correlation analysis Methods 0.000 claims description 4
- 238000010801 machine learning Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000013079 data visualisation Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 27
- 230000007246 mechanism Effects 0.000 abstract description 4
- 238000003672 processing method Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 description 10
- 230000003044 adaptive effect Effects 0.000 description 5
- 238000009499 grossing Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000007794 visualization technique Methods 0.000 description 1
Landscapes
- Position Fixing By Use Of Radio Waves (AREA)
Abstract
The invention relates to the technical field of positioning, in particular to a GPS static drift data cloud filtering method, computer equipment and a medium. According to the invention, through a unique two-stage data processing mechanism, preliminary data cleaning and processing are performed, and then the Kalman filtering algorithm is utilized to perform filtering processing, so that the accuracy and reliability of data are further improved. By an efficient data processing method: by optimizing the data processing flow and algorithm, high-efficiency data processing is realized, and the real-time requirement is met. Through the combination of multiple filtering algorithms, data filtering can be better performed, rapid operation is performed, and accuracy and instantaneity of positioning service are improved.
Description
Technical Field
The invention relates to the technical field of positioning, in particular to a GPS static drift data cloud filtering method, computer equipment and a medium.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
With the popularization of Global Positioning Systems (GPS) and the expansion of application fields, a large amount of GPS data is collected in real time and uploaded to the cloud. These data typically contain noise and other interfering factors such as static drift errors. The static drift error means that the GPS receiver cannot accurately track satellite signals under static conditions, so that deviation of positioning data occurs. Therefore, the development of the cloud filtering method capable of effectively filtering the GPS static drift data has important significance. The existing static drift data filtering method is poor in filtering effect, still has a certain static drift error, is low in processing speed and poor in instantaneity, and has limited applicability to occasions with high requirements on position accuracy and instantaneity.
Disclosure of Invention
Aiming at the technical problems and the technical difficulties, the invention provides a GPS static drift data cloud filtering method, computer equipment and media, which can perform data filtering better and operate rapidly, and improve the accuracy and instantaneity of positioning service.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows.
Firstly, in a first aspect, the invention provides a cloud filtering method for GPS static drift data, which comprises the following steps:
acquiring GPS data; preprocessing, namely preprocessing the acquired GPS data to remove invalid and abnormal data; uploading data to a cloud end, and uploading the preprocessed GPS data to a cloud end server in real time; data queuing, namely, storing the data uploaded to a cloud server into a message queue for queuing according to a time sequence; after the data is consumed from the message queue, the obvious errors, singular points, noise points and repeated points of the data are filtered and cleaned, and the disorder occurring in a short time is sequenced; data filtering, namely obtaining data characteristics through the original data and the filtered data, and adaptively adjusting parameters and initial values of a Kalman filter through machine learning to form an algorithm closed loop; and storing the filtered GPS data into a database.
In some embodiments, the data storage further includes, before the data storage, performing data cleaning on the filtered data to remove residual static drift errors.
In some embodiments, the obtaining data characteristics from the raw data and the filtered data includes one or more of data inspection, data summarization, data visualization, correlation analysis, data grouping and clustering, and data comparison.
As a further technical scheme, peak clipping and valley filling are carried out on the data when the data are queued.
As a further technical scheme, the method adopts a distributed computing framework and a distributed file storage system or cloud storage service, and is provided with a backup scheme.
As a further technical scheme, the invention adopts a mode of combining one or more algorithms to carry out data filtering, and the data filtering adopts at least one of Kalman filtering algorithm, least square method and HILBERT-HUANG transformation.
As a further technical scheme, the preprocessing comprises at least one of data cleaning, format conversion and data screening.
As a further technical solution, the GPS data includes at least one of longitude, latitude, altitude, and speed.
On the other hand, the invention also provides a computer device based on the method, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the GPS static drift data cloud filtering method when executing the computer program.
The invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the GPS static drift data cloud filtering method.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, through a unique two-stage data processing mechanism, preliminary data cleaning and processing are performed, and then the Kalman filtering algorithm is utilized to perform filtering processing, so that the accuracy and reliability of data are further improved. By an efficient data processing method: by optimizing the data processing flow and algorithm, high-efficiency data processing is realized, and the real-time requirement is met. Through the combination of multiple filtering algorithms, data filtering can be better performed, rapid operation is performed, and accuracy and instantaneity of positioning service are improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
fig. 1 is a flowchart of a cloud filtering method for GPS static drift data according to an embodiment of the present invention.
Detailed Description
For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.
It will be understood that when an element is referred to as being "mounted" or "disposed" on another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or intervening elements may also be present. When an element is referred to as being "disposed" on another element, it can be disposed on the surface or in the interior of the element.
Throughout the specification and claims, unless explicitly stated otherwise, the term "comprise" or variations thereof such as "comprises" or "comprising", etc. will be understood to include the stated element or component without excluding other elements or components.
Aiming at the problems of poor filtering effect, low processing speed and poor instantaneity of the static drift data filtering method in the prior art, the invention can better filter data and rapidly operate by combining multiple filtering algorithms and a unique two-stage data processing mechanism, thereby improving the accuracy and instantaneity of positioning service.
Optionally, as an embodiment, a cloud filtering method for GPS static drift data, as shown in fig. 1, includes:
s1, acquiring GPS data; various positioning data are acquired by the GPS receiver including, but not limited to, longitude, latitude, altitude, speed, etc. The GPS data of the invention can come from any terminal equipment with GPS positioning function, such as mobile phones, IPAD, computers, vehicles, airplanes, ships, intelligent wearing equipment and the like, and is especially suitable for equipment with higher requirements on position precision and real-time performance. In addition, the GPS positioning data of the invention is not the position information in the traditional sense, and can also reflect the information of the height, speed and the like of the terminal equipment.
S2, preprocessing the acquired GPS data to remove invalid and abnormal data; in order to improve the subsequent data processing effect, the acquired GPS data is subjected to preliminary processing, such as format conversion, data screening and other operations, so that the data processing efficiency is improved.
And S3, uploading data to the cloud, and uploading the preprocessed GPS data to a cloud server in real time.
S4, queuing data, and storing the data uploaded to a cloud server into a message queue for queuing according to a time sequence; in some embodiments, during the data queuing process, peak clipping and valley filling are also performed on the data, so that the data with excessive obvious offset is removed, the data is smoother, and the operation amount is reduced.
And S5, data cleaning, namely after the data is consumed from the message queue, filtering and cleaning obvious errors, singular points, noise points and repeated points of the data, and sequencing disorder occurring in a short time.
And S6, filtering the data, obtaining data characteristics through the original data and the filtered data, and adaptively adjusting parameters and initial values of a Kalman filter through machine learning to form an algorithm optimization closed loop. In this embodiment, a kalman filter algorithm is applied to perform data filtering. And in the cloud server, a Kalman filtering algorithm is applied to filter the queued GPS data so as to remove noise and correct deviation, and data after static drift filtering is obtained. And collecting, storing and analyzing the original data and the filtered data to obtain data characteristics, and continuously adjusting parameters of the algorithm by using a machine method to form an algorithm optimization closed loop.
The method for obtaining the data characteristics through the original data and the filtered data comprises the following steps:
1) Checking data: and checking the original data and the filtered data, and knowing the structure and the content of the data. The columns and rows of data are checked and the data type and format in the dataset are checked.
2) Data summary: and calculating summary statistics of the data, such as mean value, median, standard deviation and the like, so as to know the distribution condition and the concentration trend of the data.
3) Data visualization: graph and visualization techniques are used to show the distribution and pattern of data. For example, abnormal values, concentration levels, and correlations of data are found by plotting histograms, scatter plots, box plots, or the like.
4) Correlation analysis: by calculating the correlation coefficient between the data, the relationship between the variables is known. Patterns and interdependencies present in the data can be determined by correlation analysis.
5) Data grouping and clustering: if the data has similar characteristics, grouping and clustering techniques can be used to divide the data into different groups to understand the inherent structure and population of the data.
6) And (3) establishing a model: and constructing a proper model by using the original data and the filtered data to predict or infer. From the results of the model, the importance and predictive power of the features can be further understood.
7) Data comparison: and comparing the original data with the filtered data, checking the influence of the filtering operation on the data, and determining the validity and applicability of the filtering operation.
In practice, the appropriate analysis methods may be adjusted and selected based on the particular data set and problem. The method of one of these steps may be used alone, the method of a plurality of steps may be used in combination, and is not limited to the order limitation of the above steps, thereby obtaining detailed information about the characteristics of data and exploring patterns and relationships existing in the data.
And S7, data storage, namely storing the filtered GPS data into a database for subsequent service use.
According to the embodiment, through a unique two-stage data processing mechanism, primary data cleaning and processing are performed first, and then the Kalman filtering algorithm is utilized for filtering processing, so that the accuracy and reliability of data are further improved. By an efficient data processing method: by optimizing the data processing flow and algorithm, high-efficiency data processing is realized, and the real-time requirement is met. Also by adaptive parameter adjustment: according to the actual condition of the data, the original data and characteristic extraction are collected, and parameters and initial values of a Kalman filter are adaptively adjusted through a machine learning method, so that the optimal processing effect is achieved. The neural network model can be built, a series of original GPS data and filtered data are used as a training set to train the network model, so that the training set can master the data filtering rule, and then the filtering processing of the original data is carried out through the trained model. Alternatively, the initial features may be filtered through a feature selection process, and the model trained using the filtered features.
In addition, in some other embodiments, after the data is filtered, the filtered data is subjected to secondary cleaning to remove residual static drift errors, so as to further improve the data processing effect and efficiency.
In some other embodiments, the data filtering uses a combination of multiple filtering algorithms, and it is contemplated that the data processing may be performed in combination with multiple filtering algorithms to improve filtering and reduce errors. For example, kalman filtering algorithms may be attempted to be processed in combination with algorithms such as least squares, hilbert-HUANG transforms, and the like. The following methods may be employed to perform data filtering in conjunction with a variety of filtering algorithms:
a cascaded filter: a plurality of filters are used in a cascade in sequence, with the output of one filter being the input of the next filter. The method can improve the data filtering effect by connecting different filters in series. For example, a low pass filter may be used to smooth and then a high pass filter may be used to remove low frequency noise.
Combination of averaging filters: the different types of averaging filters are combined for use, so that a better smoothing effect can be obtained. For example, a combination of a moving average filter and a weighted average filter may be used to balance the smoothness and response speed of the filter.
Combination of statistical filters: the data is filtered using statistical methods, such as median filters and mean filters. The median filter may effectively remove salt and pepper noise, while the mean filter may smooth the signal. By combining these two filters, the effects of denoising and smoothing can be achieved at the same time.
An adaptive filter: the filtering parameters can be dynamically adjusted according to the characteristics of the data using an adaptive filter. This approach can accommodate different filtering algorithms in different data portions to better handle different noise types and strengths. For example, an adaptive median filter or an adaptive weighted average filter may be used.
Multi-scale filtering: the data is processed using a plurality of filters of different scales to capture noise and signal features of different scales. This approach allows for efficient smoothing and denoising of data over different frequency ranges. Common methods include wavelet transformation and decomposition and reconstruction of the signal.
The choice of which combination of filters depends on the characteristics of the data and the application scenario. In practical applications, different combinations of filtering algorithms may be tried as needed, and the best combination may be selected according to the filtering effect and performance. At the same time, attention should also be paid to the influence of the filtering operation on the data and the possible side effects.
In some other embodiments, a distributed computing framework is employed: by using the distributed computing framework, multiple servers can be utilized to process data simultaneously, and processing efficiency is further improved.
In some embodiments, to address the problem of large-scale data storage and management, the data storage and backup scheme may employ a distributed file system or cloud storage service for data storage and management, while a reasonable backup scheme is designed to prevent data loss.
The invention can also be extended to other fields, for example processing other types of sensor data (e.g. barometers, gyroscopes, etc.) using the kalman filter algorithm. In addition, it is also contemplated that the technique may be applied to other global positioning systems (e.g., GLONASS, galileo, etc.), as well as in the fields of intelligent transportation, unmanned, etc. In future research, the method can further optimize algorithm performance, reduce calculation complexity and the like.
In the description of the present invention, it should be understood that the directions or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Finally, it should be pointed out that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting. Although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may be modified or some technical features may be equivalently replaced, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The cloud filtering method for the GPS static drift data is characterized by comprising the following steps of:
acquiring GPS data;
preprocessing, namely preprocessing the acquired GPS data to remove invalid and abnormal data;
uploading data to a cloud end, and uploading the preprocessed GPS data to a cloud end server in real time;
data queuing, namely, storing the data uploaded to a cloud server into a message queue for queuing according to a time sequence;
after the data is consumed from the message queue, the obvious errors, singular points, noise points and repeated points of the data are filtered and cleaned, and the disorder occurring in a short time is sequenced;
data filtering, namely obtaining data characteristics through the original data and the filtered data, and adaptively adjusting parameters and initial values of a Kalman filter through machine learning to form an algorithm closed loop;
and storing the filtered GPS data into a database.
2. The cloud filtering method of GPS static drift data according to claim 1, wherein the data storage is preceded by data cleaning of the filtered data to remove residual static drift errors.
3. The cloud filtering method for the static drift data of the GPS according to claim 2, wherein the method for obtaining the data characteristics through the original data and the filtered data comprises one or more of data inspection, data summarization, data visualization, correlation analysis, data grouping and clustering and data comparison.
4. The cloud filtering method for the GPS static drift data according to claim 3, wherein peak clipping and valley filling are performed on the data during data queuing.
5. The cloud filtering method for GPS static drift data according to claim 4, wherein the method adopts a distributed computing framework and a distributed file storage system or cloud storage service, and is provided with a backup scheme.
6. The cloud filtering method of GPS static drift data according to claim 5, wherein the data filtering adopts at least one of Kalman filtering algorithm, least square method and HILBERT-HUANG transformation.
7. The cloud filtering method for GPS static drift data according to claim 1, wherein the preprocessing includes at least one of data cleaning, format conversion and data screening.
8. The cloud filtering method for GPS static drift data according to claim 1, wherein the GPS data includes at least one of longitude, latitude, altitude, and speed.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the GPS dead-float data cloud filtering method of any of claims 1 to 8 when the computer program is executed by the processor.
10. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for performing the GPS dead-float data cloud filtering method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311567470.1A CN117785847A (en) | 2023-11-23 | 2023-11-23 | Cloud filtering method for GPS static drift data, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311567470.1A CN117785847A (en) | 2023-11-23 | 2023-11-23 | Cloud filtering method for GPS static drift data, computer equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117785847A true CN117785847A (en) | 2024-03-29 |
Family
ID=90398911
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311567470.1A Pending CN117785847A (en) | 2023-11-23 | 2023-11-23 | Cloud filtering method for GPS static drift data, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117785847A (en) |
-
2023
- 2023-11-23 CN CN202311567470.1A patent/CN117785847A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110413905B (en) | Method, device and equipment for acquiring road alignment and storage medium | |
| CN114330713B (en) | Convolutional neural network model pruning method and device, electronic equipment and storage medium | |
| CN113591380B (en) | Traffic flow prediction method, medium and equipment based on graph Gaussian process | |
| CN105590216B (en) | Method and system for monitoring transaction risk in real time | |
| CN109743356B (en) | Industrial internet data acquisition method and device, readable storage medium and terminal | |
| CN114282670A (en) | Neural network model compression method, device and storage medium | |
| CN110265052A (en) | The signal-to-noise ratio of radio equipment determines method, apparatus, storage medium and electronic device | |
| CN110443279B (en) | Unmanned aerial vehicle image vehicle detection method based on lightweight neural network | |
| CN111192313A (en) | Method for robot to construct map, robot and storage medium | |
| CN117557414B (en) | Cultivated land supervision method, device, equipment and storage medium based on automatic interpretation of remote sensing image | |
| CN114580572B (en) | Abnormal value identification method and device, electronic equipment and storage medium | |
| CN115546705A (en) | Target identification method, terminal device and storage medium | |
| CN113920255B (en) | High-efficient mapping system based on point cloud data | |
| CN114742237A (en) | Federal learning model aggregation method and device, electronic equipment and readable storage medium | |
| CN112399177B (en) | Video coding method, device, computer equipment and storage medium | |
| CN107817491A (en) | A kind of generation method of image, device and image forming apparatus | |
| CN117785847A (en) | Cloud filtering method for GPS static drift data, computer equipment and medium | |
| CN110988631B (en) | Partial discharge positioning detection method and device, readable storage medium and electrical equipment | |
| CN109558819A (en) | A kind of depth network light weight method for Remote Sensing Target detection | |
| CN109443367B (en) | RFID positioning method based on mean clustering random particle swarm algorithm | |
| CN116263735A (en) | Robustness assessment method, device, equipment and storage medium for neural network | |
| CN113596757A (en) | Rapid high-precision indoor fingerprint positioning method based on integrated width learning | |
| CN113780462A (en) | Vehicle detection network establishment method based on unmanned aerial vehicle aerial image and application thereof | |
| CN114691626A (en) | Point cloud data compression method, point cloud data transmission method, point cloud data decompression method, point cloud data compression device, point cloud data transmission device and point cloud data decompression device | |
| CN116108896B (en) | Model quantization method, device, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |