CN109743230A - Monitoring data transmission system based on statistical information - Google Patents
Monitoring data transmission system based on statistical information Download PDFInfo
- Publication number
- CN109743230A CN109743230A CN201910124284.8A CN201910124284A CN109743230A CN 109743230 A CN109743230 A CN 109743230A CN 201910124284 A CN201910124284 A CN 201910124284A CN 109743230 A CN109743230 A CN 109743230A
- Authority
- CN
- China
- Prior art keywords
- host
- information
- sent
- data
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of monitoring data transmission systems based on statistical information, the system comprises: serviced component, it is installed on host node, for being dynamically grouped host according to user's operation during supervisory control system running, and a host in each grouping is determined as center element；After completing host packet, host in the group is dynamically specified to need the index monitored；Agent components are sent by host packet information and monitor control index information, start monitor component, the operating status of monitoring host computer；Agent components are installed on monitored host, for periodically collecting the corresponding index data of host according to the grouping information and the monitor control index information, and the achievement data are sent to center element；Center element carries out Pingdu distribution statistics to data, statistical data is regularly then sent to serviced component for the achievement data that regular Receiving Agent component is sent.
Description
Technical field
The present invention relates to the communications field more particularly to a kind of monitoring data transmission systems based on statistical information.
Background technique
Mass computing cluster monitoring system, by the parameter of the various working conditions of real-time collecting cluster interior joint, and with
The relevant parameter being collected into, is fed back to the operation maintenance personnel of cluster, so that the system of cluster by a kind of informing mechanism flexibly, friendly
Administrator can quickly locate and solve the problems, such as it is various in cluster operational process, to guarantee the safe and reliable of group system
Expeditiously run.Monitoring system provides the monitoring alarm service based on resource level, can be with by using monitoring alarm
Comprehensively understand resource utilization, application program capacity and the system operation situation of cluster interior joint.Wherein, monitoring system is supervised
The object of control is known as monitor control index, and common index includes: that resource metrics (e.g., CPU, memory, I/O and bandwidth etc.) and system refer to
It marks (e.g., concurrent user number, response time, affairs success rate and time-out error rate etc.).By these index parameters, cluster
System operation maintenance personnel can accurately and timely grasp the operation conditions of cluster.
Existing distributed monitoring system (e.g., zabbix), realizes the property such as host in cluster, the network equipment, database
It can monitor, have the function of the drafting of a variety of warning modes and powerful detailed account.Cluster monitoring system framework as shown in Fig. 1,
It is made of serviced component and Agent components, and wherein Agent components are mounted on monitored host, is responsible for periodically collecting every
Then achievement data sends data to serviced component；Serviced component is responsible for the report information of Receiving Agent component transmission and will
Index of correlation data are stored into database, and all configurations of system, statistical data and operation data are carried out by serviced component
Tissue.
In recent years, for high-performance and resource-intensive application field, mass computing cluster has changed into very heavy
The platform wanted.In group system, for different types of task, such as performance evaluation, adjustment, scheduling and error detection
With diagnosis etc. O&Ms task, require to obtain a large amount of monitoring data.For mass computing cluster, as scale constantly expands
Greatly, complexity is continuously increased, and transmission/preservation task of monitoring data consumes a large amount of network bandwidth and memory space, and
Single monitoring data is seldom used again, becomes the important bottleneck for influencing monitor system performance, more serious, monitoring system consumption
A large amount of system resource will seriously affect the availability of cluster, to the service ability of cluster propose serious challenge.
Summary of the invention
The embodiment of the present invention provides a kind of monitoring data transmission system based on statistical information, to solve in the prior art
The above problem.
The embodiment of the present invention provides a kind of monitoring data transmission system based on statistical information, comprising:
Serviced component is installed on host node, for during supervisory control system running dynamic according to user's operation by host
It is grouped, and a host in each grouping is determined as center element；After completing host packet, dynamically specifying should
Host needs the index monitored in group；Agent components are sent by host packet information and monitor control index information, start monitoring group
Part, the operating status of monitoring host computer；
Agent components are installed on monitored host, are used for according to the grouping information and the monitor control index information,
The corresponding index data of host are periodically collected, and the achievement data is sent to center element；
Center element carries out Pingdu distribution statistics to data, so for the achievement data that regular Receiving Agent component is sent
Statistical data is regularly sent to serviced component afterwards.
Preferably, dynamically host is grouped according to user's operation during supervisory control system running and is specifically included:
Dynamic is grouped host based on different host ids according to user's operation during supervisory control system running.
Preferably, Agent components are sent by host packet information and monitor control index information to specifically include:
Agent components are sent by host packet information and monitor control index information by REST-API.
Preferably dynamically specify host in the group that the index monitored is needed to specifically include:
Confirmation needs to modify the host and index modification information of monitor control index；
Newly-built grouping, group member are the above-mentioned host for needing to modify monitor control index, initialize the monitor control index being newly grouped；
Above-mentioned modification information is sent to Agent components, and starts monitor component.
Using the embodiment of the present invention, only transmission/preservation " publication data ", significantly reduce network bandwidth and storage overhead；
Secondly, the localization pretreatment of monitoring data can make full use of the advantage of distributed computing cluster, monitoring system is significantly reduced
Occupied system resource of system itself, improves the availability of cluster.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 is the schematic diagram of system monitoring service architecture in the prior art；
Fig. 2 is the configuration diagram of Pingdu distribution statistics principle in the embodiment of the present invention；
Fig. 3 is the schematic diagram of the monitoring system based on Pingdu distribution statistics in the embodiment of the present invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
The present invention provides a kind of monitoring data transmission method based on statistical information, joins for the index of monitoring system
Number carries out " Pingdu distribution statistics ".Compared with the data transmission policies of existing monitoring system, method provided by the present invention is first
First, only transmission/preservation " publication data ", significantly reduce network bandwidth and storage overhead；Secondly, the localization of monitoring data is pre-
Processing can make full use of the advantage of distributed computing cluster, significantly reduce the occupied system resource of monitoring system itself,
Improve the availability of cluster.
Pingdu distribution statistics principle framework is related to three components: serviced component, Agent components and central set as shown in Fig. 2
Part.Wherein, serviced component, a kind of programmable component, compared with the serviced component of existing monitoring system, increase " host packet " and
" index allocation " two API." host packet " is other than being responsible for dynamic, being on demand grouped host, it is also necessary to be responsible for bright
A host in the true grouping is center component.Refer to that user can will monitor different indexs according to specific requirements on demand
Host is grouped, and grouping process can be dynamically completed during supervisory control system running.For example, part host needs emphasis
The indexs such as monitoring CPU and response time, another part host need the indexs such as key monitoring memory and concurrent user number, at this time
User can call " host packet " API that host is grouped into group 1 and group 2 respectively.In turn, the monitoring of system can be with group
Unit hit the target monitoring, information collect etc. work." index allocation ", after completing host packet process, dynamically specifying should
Host needs the index monitored in group.Agent components are installed on monitored host, are responsible for periodically collecting indices data,
Then center element is sent data to.Center element, the achievement data that regular Receiving Agent component is sent, carries out data flat
Distribution statistics are spent, statistical data is regularly then sent to serviced component.
As shown in fig. 3, the concrete operations process or principle of monitoring system are such as monitoring system based on " Pingdu distribution statistics "
It is lower described:
1. disposing Agent components and serviced component respectively to node (monitored host) and host node；
2. completing the initial work of monitoring system:
1. serviced component calls " host packet " to be grouped monitored host, we carry out it according to node ID
Grouping.By taking Fig. 2 as an example, host is divided into four groups by us, respectively grouping 1={ node 1_1, node 2_1, node 3_1, node
4_1 }, it is grouped 2={ node 1_2, node 2_2, node 3_2, node 4_2 }, is grouped 3={ node 1_3, node 2_3, node 3_
3, node 4_3 }, it is grouped 4={ node 1_4, node 2_4, node 3_4, node 4_4 }.
2. serviced component is, it is specified that node 4_1, node 2_2, node 1_3 and node 3_4 are respectively to be grouped 1, grouping 2, divide
The center element node of group 3 and grouping 4.
3. serviced component, the monitor control index of initialisation packet.For example, index grouping 1={ CPU, memory, response time }.
3. serviced component, will be grouped using REST-API and grouping indication information is sent to Agent components, and start monitoring
Component starts the operating status of monitoring host computer.
4. Agent components, monitor the corresponding index parameter for the monitoring host computer being each responsible in real time, and regularly it is sent to point
Center element in group.
5. center element, the achievement data that Receiving Agent component is sent, and Pingdu distribution statistics are carried out, then by defined
Time interval sends statistical information to serviced component.
Compared with existing monitoring system, the monitoring data transmission method based on statistical information of the embodiment of the present invention, and
The beneficial effect that monitoring system based on " Pingdu distribution statistics " is embodied is:
The utilization rate of cluster resource is effectively promoted.Firstly, only transmission/preservation " publication data ", significantly reduces net
Network bandwidth and storage overhead；Secondly, the localization pretreatment of monitoring data can make full use of the advantage of distributed computing cluster,
The occupied system resource of monitoring system itself is significantly reduced, the availability of cluster has been risen.
Obviously, those skilled in the art should be understood that each module of the above invention or each step can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
It is performed by computing device in the storage device, and in some cases, it can be to be different from shown in sequence execution herein
Out or description the step of, perhaps they are fabricated to each integrated circuit modules or by them multiple modules or
Step is fabricated to single integrated circuit module to realize.In this way, the present invention is not limited to any specific hardware and softwares to combine.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (4)
1. a kind of monitoring data transmission system based on statistical information characterized by comprising
Serviced component is installed on host node, for dynamically being carried out host according to user's operation during supervisory control system running
Grouping, and a host in each grouping is determined as center element；After completing host packet, dynamically specify in the group
Host needs the index monitored；Agent components are sent by host packet information and monitor control index information, start monitor component, prison
Control the operating status of host；
Agent components are installed on monitored host, are used for according to the grouping information and the monitor control index information, periodically
The corresponding index data of host are collected, and the achievement data is sent to center element；
Center element carries out Pingdu distribution statistics to data for the achievement data that regular Receiving Agent component is sent, then fixed
Statistical data is sent to phase serviced component.
2. the system as claimed in claim 1, which is characterized in that dynamic will according to user's operation during supervisory control system running
Host, which is grouped, to be specifically included:
Dynamic is grouped host based on different host ids according to user's operation during supervisory control system running.
3. the system as claimed in claim 1, which is characterized in that send agency for host packet information and monitor control index information
Component specifically includes:
Agent components are sent by host packet information and monitor control index information by REST-API.
4. the system as claimed in claim 1, which is characterized in that dynamically specify the index that host needs to monitor in the group specific
Include:
Confirmation needs to modify the host and index modification information of monitor control index；
Newly-built grouping, group member are the above-mentioned host for needing to modify monitor control index, initialize the monitor control index being newly grouped；
Above-mentioned modification information is sent to Agent components, and starts monitor component.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124284.8A CN109743230A (en) | 2019-02-18 | 2019-02-18 | Monitoring data transmission system based on statistical information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910124284.8A CN109743230A (en) | 2019-02-18 | 2019-02-18 | Monitoring data transmission system based on statistical information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109743230A true CN109743230A (en) | 2019-05-10 |
Family
ID=66367743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910124284.8A Pending CN109743230A (en) | 2019-02-18 | 2019-02-18 | Monitoring data transmission system based on statistical information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109743230A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111400149A (en) * | 2020-01-19 | 2020-07-10 | 上海趣蕴网络科技有限公司 | Application performance data monitoring system, client and method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667034A (en) * | 2009-09-21 | 2010-03-10 | 北京航空航天大学 | Scalable monitoring system supporting hybrid clusters |
| CN103095598A (en) * | 2012-12-17 | 2013-05-08 | 华中科技大学 | Monitoring data aggregate method under large-scale cluster environment |
| CN104184819A (en) * | 2014-08-29 | 2014-12-03 | 城云科技（杭州）有限公司 | Multi-hierarchy load balancing cloud resource monitoring method |
| CN105915405A (en) * | 2016-03-29 | 2016-08-31 | 深圳市中博科创信息技术有限公司 | Large-scale cluster node performance monitoring system |
| WO2018109146A1 (en) * | 2016-12-16 | 2018-06-21 | INSERM (Institut National de la Santé et de la Recherche Médicale) | Methods and kits for detecting basophil activation |
| CN108390907A (en) * | 2018-01-09 | 2018-08-10 | 浙江航天恒嘉数据科技有限公司 | A kind of management monitoring system and method based on Hadoop clusters |
-
2019
- 2019-02-18 CN CN201910124284.8A patent/CN109743230A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101667034A (en) * | 2009-09-21 | 2010-03-10 | 北京航空航天大学 | Scalable monitoring system supporting hybrid clusters |
| CN103095598A (en) * | 2012-12-17 | 2013-05-08 | 华中科技大学 | Monitoring data aggregate method under large-scale cluster environment |
| CN104184819A (en) * | 2014-08-29 | 2014-12-03 | 城云科技（杭州）有限公司 | Multi-hierarchy load balancing cloud resource monitoring method |
| CN105915405A (en) * | 2016-03-29 | 2016-08-31 | 深圳市中博科创信息技术有限公司 | Large-scale cluster node performance monitoring system |
| WO2018109146A1 (en) * | 2016-12-16 | 2018-06-21 | INSERM (Institut National de la Santé et de la Recherche Médicale) | Methods and kits for detecting basophil activation |
| CN108390907A (en) * | 2018-01-09 | 2018-08-10 | 浙江航天恒嘉数据科技有限公司 | A kind of management monitoring system and method based on Hadoop clusters |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111400149A (en) * | 2020-01-19 | 2020-07-10 | 上海趣蕴网络科技有限公司 | Application performance data monitoring system, client and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10355959B2 (en) | Techniques associated with server transaction latency information | |
| CN107925612B (en) | Network monitoring system, network monitoring method, and computer-readable medium | |
| US10298477B2 (en) | Flexible and safe monitoring of computers | |
| CN104794047B (en) | Method and system for correlation analysis of performance indicators | |
| US20060277295A1 (en) | Monitoring system and monitoring method | |
| JP5474177B2 (en) | Distributed application monitoring | |
| US10503558B2 (en) | Adaptive resource management in distributed computing systems | |
| US20140096129A1 (en) | Systems and methods for installing, managing, and provisioning applications | |
| CN109271243B (en) | Cluster task management system | |
| US20140095694A1 (en) | Systems and methods for installing, managing, and provisioning applications | |
| US20080263556A1 (en) | Real-time system exception monitoring tool | |
| CN107729213B (en) | Background task monitoring method and device | |
| CN111897643B (en) | Thread pool configuration system, method, device and storage medium | |
| CN106941431A (en) | Monitoring system server | |
| US20140096125A1 (en) | Systems and methods for installing, managing, and provisioning applications | |
| CN109743230A (en) | Monitoring data transmission system based on statistical information | |
| CN111324460A (en) | Power monitoring control system and method based on cloud computing platform | |
| CN110198246B (en) | Method and system for monitoring flow | |
| CN107948001B (en) | Intelligent monitoring system and method for industrial network of ship manufacturing workshop | |
| CN115883376A (en) | Node dynamic capacity expansion method and system based on platform real-time access volume | |
| CN115422010A (en) | Node management method and device in data cluster and storage medium | |
| US20220027251A1 (en) | System for monitoring activity in a process and method thereof | |
| CN109901971A (en) | The Host Supervision System of dynamically programmable | |
| KR20170131007A (en) | Apparatus for monitoring communication based on data distribution service | |
| Gjorgjevikj et al. | System monitoring addon analysis in system load simulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190510 |