CN110781175A - Method for realizing monitoring data integration based on Promethues - Google Patents
Method for realizing monitoring data integration based on Promethues Download PDFInfo
- Publication number
- CN110781175A CN110781175A CN201911041910.3A CN201911041910A CN110781175A CN 110781175 A CN110781175 A CN 110781175A CN 201911041910 A CN201911041910 A CN 201911041910A CN 110781175 A CN110781175 A CN 110781175A
- Authority
- CN
- China
- Prior art keywords
- data
- promethues
- monitoring data
- monitoring
- framework
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a method for realizing monitoring data integration based on Promethues, which belongs to the technical field of operation and maintenance monitoring. The configuration is simple and flexible, and the expansion is easy.
Description
Technical Field
The invention relates to an operation and maintenance monitoring technology, in particular to a method for realizing monitoring data integration based on Promethues.
Background
Prometheus is an open source monitoring system whose predecessor is the sourdcloud's warning toolkit. Prometheus performed very well in recording pure digital time series. The method is suitable for monitoring hardware indexes such as server-oriented and the like, and is also suitable for monitoring a high-dynamic service-oriented architecture. For the microservices which are popular today, the multidimensional data collection and data screening query language of Prometheus is also very powerful. Prometheus is designed for reliability of service, and when a service fails, it allows you to quickly locate and diagnose the problem. The building process of the system has no strong dependence on hardware and services.
Prometheus, the value of which is reliability, even in a very harsh environment, you can access it and view the statistical information of various indexes of the system service at any time. Prometheus stores time-sequential data, i.e., sets of consecutive data are stored in the time dimension in the same time sequence (same name and label). Time sequences (time series) are defined by a name (Metric), and a set of key/value tags, having the same name and belonging to the same time sequence. The name of the time sequence consists of ASCII characters, numbers, underlines, and colons, which must satisfy the regular expression [ a-zA-Z: ] [ a-zA-Z0-9: ], whose name should be semantically expressed, generally representing a measurable index, such as http _ requests _ total, which may represent the total number of http requests.
The timing tag can enrich the data of Prometous, and can distinguish different specific examples, for example, http _ requests _ total { method ═ POST } can represent POST requests in all http. The tag name consists of ASCII characters, numbers, and an underline, where __ begins with Prometheus reserved, and the value of the tag can be any Unicode character, supporting chinese.
At present, the following problems exist in the prior art for realizing the integrated reporting of monitoring data:
1. prometheus' existing data format does not satisfy the monitoring data format
2. The content of the Prometheus existing data does not meet the requirement of monitoring data.
Disclosure of Invention
The invention aims to provide a middleware (hereinafter referred to as a monitoring data integration middleware) for realizing a function of integrating monitoring data and other data based on Promethues so as to solve the problem which cannot be solved by the prior art.
The method analyzes the monitoring data of Promethues through the monitoring data integration service, arranges the analyzed data, and integrates the data of the database and the arranged monitoring data of Promethues according to the corresponding relation, thereby realizing the integration of the monitoring data. According to the method, the Promethues data to be integrated are recursively analyzed and integrated in the data of the database through a configuration standard to obtain new data, and the integration of the Promethues monitoring data and the data of the database is realized.
The technical scheme of the invention is as follows:
the method for realizing monitoring data integration based on Promethues obtains built Prometheus monitoring data, analyzes the data, obtains database data, and integrates the two data sets according to the corresponding relation.
Further, in the above-mentioned case,
analyzing the monitoring data of Promethues through the monitoring data integration service, sorting the analyzed data, and integrating the data of the database and the sorted monitoring data of Promethues according to the corresponding relation to realize the integration of the monitoring data.
Further, in the above-mentioned case,
setting a configuration specification, carrying out recursive analysis on Promethues data to be integrated and integrating the Promethues data in the data of the database to obtain new data, and realizing the integration of Promethues monitoring data and database data.
In a still further aspect of the present invention,
the specification includes:
analyzing time sequence data, namely analyzing, namely storing a continuous data set in a time dimension according to the same time sequence, namely the same name and label;
formulating a target data format;
and integrating the monitoring data according to the target data format.
In a still further aspect of the present invention,
the monitoring data is based on the Promethues framework.
In a still further aspect of the present invention,
the Promethues framework monitoring data format is JSON format data.
In a still further aspect of the present invention,
the Promethues framework uses the yaml profile.
In a still further aspect of the present invention,
the invention is realized based on a SpringBoot framework.
In a still further aspect of the present invention,
the Springboot framework is realized based on a JAVA language.
The invention has the advantages that
Compared with the prior art, the method for realizing monitoring data integration based on Promethues, provided by the invention, has the following beneficial effects:
firstly, the analysis of the monitoring data is realized
Second, the definition of the target data format is achieved
Thirdly, the configuration is simple and flexible, and the expansion is easy.
Drawings
FIG. 1 is a schematic workflow diagram of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
The application discloses a method for realizing monitoring data integration based on Promethues. The method comprises the steps of obtaining built Prometheus monitoring data, analyzing the data, obtaining database data and integrating two data sets according to a corresponding relation.
The invention discloses a method for realizing monitoring data integration based on Promethues, which is based on a Promethues monitoring framework. The method analyzes the monitoring data of Promethues through the monitoring data integration service, arranges the analyzed data, and integrates the data of the database and the arranged monitoring data of Promethues according to the corresponding relation, thereby realizing the integration of the monitoring data.
According to the method, the Promethues data to be integrated are recursively analyzed and integrated in the data of the database through a configuration standard to obtain new data, and the integration of the Promethues monitoring data and the data of the database is realized. The method comprises the following steps:
parsing the time-series data, i.e. parsing stores a set of consecutive data in a time dimension, i.e. according to the same time-series (same name and label);
formulating a target data format;
integrating the monitoring data according to the target data format;
the monitoring data is based on the Promethues framework.
The Promethues framework monitoring data format is JSON format data.
The Promethues framework uses the yaml profile.
The method is realized based on a SpringBoot framework.
The Springboot framework is realized based on a JAVA language.
The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (9)
1. A method for realizing monitoring data integration based on Promethues is characterized in that,
and acquiring the constructed data monitored by Prometous, analyzing the data, acquiring database data, and integrating the two data sets according to the corresponding relation.
2. The method of claim 1,
analyzing the monitoring data of Promethues through the monitoring data integration service, sorting the analyzed data, and integrating the data of the database and the sorted monitoring data of Promethues according to the corresponding relation to realize the integration of the monitoring data.
3. The method of claim 2,
setting a configuration specification, carrying out recursive analysis on Promethues data to be integrated and integrating the Promethues data in the data of the database to obtain new data, and realizing the integration of Promethues monitoring data and database data.
4. The method of claim 3,
the specification includes:
analyzing time sequence data, namely analyzing, namely storing a continuous data set in a time dimension according to the same time sequence, namely the same name and label;
formulating a target data format;
and integrating the monitoring data according to the target data format.
5. The method according to any one of claims 1 to 4,
the monitoring data is based on the Promethues framework.
6. The method of any one of claim 5,
the Promethues framework monitoring data format is JSON format data.
7. The method according to any one of claims 1 to 4,
the Promethues framework uses the yaml profile.
8. The method of claim 1,
the method is realized based on a SpringBoot framework.
9. The method of claim 8,
the Springboot framework is realized based on a JAVA language.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041910.3A CN110781175A (en) | 2019-10-30 | 2019-10-30 | Method for realizing monitoring data integration based on Promethues |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041910.3A CN110781175A (en) | 2019-10-30 | 2019-10-30 | Method for realizing monitoring data integration based on Promethues |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110781175A true CN110781175A (en) | 2020-02-11 |
Family
ID=69387530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911041910.3A Pending CN110781175A (en) | 2019-10-30 | 2019-10-30 | Method for realizing monitoring data integration based on Promethues |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110781175A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797021A (en) * | 2020-07-06 | 2020-10-20 | 广州嘉为科技有限公司 | Prometheus standard format-based inspection method |
| CN115277952A (en) * | 2022-06-30 | 2022-11-01 | 重庆长安汽车股份有限公司 | Telephone receiver configuration method, device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215381A (en) * | 2011-06-23 | 2011-10-12 | 天地阳光通信科技（北京）有限公司 | Integrating device and method for integrating digital video monitoring systems |
| US8280014B1 (en) * | 2006-06-27 | 2012-10-02 | VoiceCaptionIt, Inc. | System and method for associating audio clips with objects |
| CN106210124A (en) * | 2016-08-03 | 2016-12-07 | 浪潮电子信息产业股份有限公司 | Unified cloud data center monitoring system |
| CN106302015A (en) * | 2016-08-16 | 2017-01-04 | 华青融天（北京）技术股份有限公司 | A kind of service condition monitoring method, device and system |
| CN107180072A (en) * | 2017-03-31 | 2017-09-19 | 北京奇艺世纪科技有限公司 | A kind of processing method and processing device of time series data |
| CN107729219A (en) * | 2017-11-17 | 2018-02-23 | 北京联想超融合科技有限公司 | Resource monitoring method, device and terminal based on super fusion storage system |
-
2019
- 2019-10-30 CN CN201911041910.3A patent/CN110781175A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8280014B1 (en) * | 2006-06-27 | 2012-10-02 | VoiceCaptionIt, Inc. | System and method for associating audio clips with objects |
| CN102215381A (en) * | 2011-06-23 | 2011-10-12 | 天地阳光通信科技（北京）有限公司 | Integrating device and method for integrating digital video monitoring systems |
| CN106210124A (en) * | 2016-08-03 | 2016-12-07 | 浪潮电子信息产业股份有限公司 | Unified cloud data center monitoring system |
| CN106302015A (en) * | 2016-08-16 | 2017-01-04 | 华青融天（北京）技术股份有限公司 | A kind of service condition monitoring method, device and system |
| CN107180072A (en) * | 2017-03-31 | 2017-09-19 | 北京奇艺世纪科技有限公司 | A kind of processing method and processing device of time series data |
| CN107729219A (en) * | 2017-11-17 | 2018-02-23 | 北京联想超融合科技有限公司 | Resource monitoring method, device and terminal based on super fusion storage system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797021A (en) * | 2020-07-06 | 2020-10-20 | 广州嘉为科技有限公司 | Prometheus standard format-based inspection method |
| CN115277952A (en) * | 2022-06-30 | 2022-11-01 | 重庆长安汽车股份有限公司 | Telephone receiver configuration method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wais | Gender prediction methods based on first names with genderizeR. | |
| CN106202561B (en) | Digitlization contingency management case base construction method and device based on text big data | |
| US10572494B2 (en) | Bootstrapping the data lake and glossaries with ‘dataset joins’ metadata from existing application patterns | |
| CN108182215B (en) | Structured Query Language (SQL) performance statistics method and device | |
| CN102891852B (en) | Message analysis-based protocol format automatic inferring method | |
| CN114746856A (en) | Converting a data stream into structured data | |
| CN108711443B (en) | Text data analysis method and device for electronic medical record | |
| CN109684052B (en) | Transaction analysis method, device, equipment and storage medium | |
| CN111459967A (en) | Structured query statement generation method and device, electronic equipment and medium | |
| CN102119385A (en) | Method and subsystem for searching media content within a content-search-service system | |
| CN109766525A (en) | A kind of sensitive information leakage detection framework of data-driven | |
| CN110781175A (en) | Method for realizing monitoring data integration based on Promethues | |
| CN103605651A (en) | Data processing showing method based on on-line analytical processing (OLAP) multi-dimensional analysis | |
| US20150205862A1 (en) | Method and device for recognizing and labeling peaks, increases, or abnormal or exceptional variations in the throughput of a stream of digital documents | |
| US10331648B2 (en) | Method, device and medium for knowledge base construction | |
| CN112532972B (en) | Fault detection method and device for live broadcast service, electronic equipment and readable storage medium | |
| CN112685433A (en) | Metadata updating method and device, electronic equipment and computer-readable storage medium | |
| CN114896305A (en) | Smart internet security platform based on big data technology | |
| CN113360603B (en) | Contract similarity and compliance detection method and device | |
| US11568344B2 (en) | Systems and methods for automated pattern detection in service tickets | |
| CN108616413B (en) | Information calibration method and device | |
| WO2023050967A1 (en) | System abnormality detection processing method and apparatus | |
| US20190370531A1 (en) | Data processing apparatus, data processing method, and non-transitory storage medium | |
| CN112306421B (en) | Method and system for storing MDF file in analysis and measurement data format | |
| CN114996080A (en) | Data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200211 |