CN103200046B - The method and system of monitoring network element device performance - Google Patents
The method and system of monitoring network element device performance Download PDFInfo
- Publication number
- CN103200046B CN103200046B CN201310104796.0A CN201310104796A CN103200046B CN 103200046 B CN103200046 B CN 103200046B CN 201310104796 A CN201310104796 A CN 201310104796A CN 103200046 B CN103200046 B CN 103200046B
- Authority
- CN
- China
- Prior art keywords
- network element
- log data
- information
- performance
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000005540 biological transmission Effects 0.000 claims abstract description 32
- 238000004458 analytical method Methods 0.000 claims description 105
- 238000005070 sampling Methods 0.000 claims description 22
- 238000013500 data storage Methods 0.000 claims description 10
- 230000002776 aggregation Effects 0.000 claims description 6
- 238000004220 aggregation Methods 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 238000007405 data analysis Methods 0.000 abstract description 4
- 238000012546 transfer Methods 0.000 abstract description 4
- 238000011156 evaluation Methods 0.000 abstract 1
- 230000006399 behavior Effects 0.000 description 70
- 230000006870 function Effects 0.000 description 36
- 238000012545 processing Methods 0.000 description 13
- 230000009471 action Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 8
- 230000002085 persistent effect Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000002688 persistence Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of method and system of monitoring network element device performance.The method comprises: in described network element device, pre-sets the data format for performance monitoring and data transfer mode; Acquisition performance information, is encapsulated as daily record data according to the data format pre-set by performance information, according to the data transfer mode transmission daily record data pre-set; Receive the daily record data sent, according to the performance information collection period pre-set, cluster is carried out to daily record data; Merge the cycle according to the daily record data pre-set, the daily record data of cluster is merged, according to the daily record data that merging treatment obtains, carries out network element device performance evaluation.Application the present invention, can improve the reliability of daily record data analysis result.
Description
Technical Field
The present invention relates to communication monitoring technologies, and in particular, to a method and a system for monitoring performance of a network element device.
Background
With the development of communication technology, the number of users served by a communication system is increasing day by day, and in order to improve the efficiency of data transmission in the communication system and improve the service experience of the users, in the communication system, each network element device generally provides services to the users by adopting a concurrent data transmission mode. Taking the smart television system as an example, due to the wide application of the digital television technology, the number of users is large, so that the concurrent processing amount (performance) of each network element device in the smart television system is increasing day by day. Therefore, in order to improve user experience, the requirement on the parallel processing performance of each network element device in the smart television system is increasingly enhanced.
In order to guarantee the parallel processing capability of the smart television system and maintain the normal operation of the smart television system, the performance of each network element device in the smart television system needs to be monitored, for example, load information, CPU occupancy rate, and the like, so as to ensure the data transmission rate of the access user, for example, it needs to ensure that the network element device can provide the access user with a concurrent transmission amount of more than 10K/S. In the prior art, a log monitor is generally arranged in an intelligent television system, and the log monitor acquires performance information of network element equipment in the intelligent television system in real time, samples the acquired performance information, writes the performance information obtained by sampling into a hard disk in a log data mode, and analyzes the network element performance according to log data; and if the performance analysis result shows that the concurrency rate of the network element is lower than 10K/S, controlling the number of the accessed users so as to ensure the experience of the digital television service of the accessed users.
As can be seen from the above, in the conventional method for monitoring the performance of the network element device, performance information (log data) is collected and analyzed in a periodic sampling manner, so that the analyzed log data cannot cover the source data of the performance of the network element device in the smart television system, and the reliability of the analysis result of the performance of the network element device is low; further, since the analysis result cannot reflect the real performance status of the network element device, if the control is performed according to the analysis result, the control effect is poor. On the other hand, if the performance information of the network element device is not sampled, the storage space of the log monitor is difficult to satisfy for the log data generated by the network element device of the smart television system in such a scale, and the log monitor analyzes the log data, so that the resource consumption of the required data processing is very large, and the log data analysis is difficult to realize.
In the prior art, although performance information is sampled and analyzed to reduce the amount of transmitted log data and background calculation, the performance of the network element device cannot be well monitored because the processing mode of the performance information cannot maintain source data covering the performance of the network element device. In other words, there is a need in the art for a technique for monitoring network element device performance that can reduce the amount of transmitted log data and background computations, and at the same time can cover the source data of the network element device performance.
Disclosure of Invention
The embodiment of the invention provides a method for monitoring the performance of network element equipment, which improves the reliability of log data analysis results.
The embodiment of the invention also provides a system for monitoring the performance of the network element equipment, which improves the reliability of the log data analysis result.
In order to achieve the above object, an embodiment of the present invention provides a method for monitoring performance of a network element device, where a data format and a data transmission mode for performance monitoring are preset in the network element device, and the method includes:
acquiring performance information, packaging the performance information into log data according to a preset data format, and transmitting the log data according to a preset data transmission mode;
receiving the sent log data, and clustering the log data according to a preset performance information acquisition cycle;
and merging the clustered log data according to a preset log data merging period, and analyzing the performance of the network element equipment according to the log data obtained by merging.
Wherein the data format comprises: network element equipment identification code, function number, behavior identification, data time point, additional data and other information;
the data transmission mode is user datagram protocol transmission.
Wherein the behavior mark is a trigger behavior mark, a continuous behavior mark or an error behavior mark.
Wherein clustering log data comprises:
a11, analyzing the received log data in a preset performance information acquisition period, and acquiring network element equipment identification code information, network element equipment number information, function number information and behavior identification information contained in the log data;
a12, judging whether the behavior identification information obtained by analysis is continuous behavior identification information, if so, executing step A13, otherwise, executing step A15;
a13, recording data time point information corresponding to the beginning of a continuous behavior in log data, and placing the log data into a temporary cache;
a14, after the end of the continuous behavior is monitored, merging the log data of the continuous behavior in the temporary cache into a log data containing the duration, and executing the step A15;
a15, counting the total times, the mean value, the maximum value and the minimum value of the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information;
and A16, constructing clustered log data according to the total times, the mean value, the maximum value and the minimum value obtained through statistics.
Wherein the method further comprises:
extracting log data with a preset threshold value from the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information, recording data time point information and duration information of the extracted log data, and generating a sampling set.
Wherein the constructing log data of clusters comprises:
and judging whether the behavior identification information in the received log data is error behavior information, if so, constructing clustered error behavior log data, and otherwise, constructing clustered functional log data.
Wherein the clustered functional log data comprises: network element equipment identification code information, network element equipment number information, function number information, performance information acquisition time period information, total times information, mean value information, maximum value information, minimum value information, and { sampling set (time, value) } information.
Wherein the clustered error behavior log data comprises: the network element equipment identification code information, the network element equipment number information, the function number information, the performance information acquisition time period information and the error information.
Wherein the merging of clustered log data comprises:
and in a preset log data merging period, merging log data containing the same network element equipment identification code information, network element equipment number information and function number information according to the received clustered log data to obtain merged log data.
Wherein the method further comprises:
and displaying the combined log data or displaying the result of the performance analysis of the network element equipment.
Wherein the method further comprises:
and when the performance analysis result of the network element equipment shows that the concurrency rate is lower than a preset rate threshold, controlling the number of users accessing the intelligent television system.
A system for monitoring network element device performance, the system comprising: a network element device, a proxy server, a convergence server, and a performance analysis server, wherein,
the network element equipment stores a preset data format and a preset data transmission mode for performance monitoring; acquiring performance information, packaging the performance information into log data according to a preset data format, and transmitting the log data to a proxy server according to a preset data transmission mode;
the proxy server is used for receiving the log data sent by the network element equipment, clustering the log data according to a preset performance information acquisition cycle and outputting the log data to the convergence server;
the aggregation server is used for merging the received clustered log data according to a preset log data merging period and outputting the merged log data to the performance analysis server;
and the performance analysis server is used for analyzing the performance of the network element equipment according to the log data obtained by merging processing.
Preferably, the performance analysis server is further configured to display the merged log data or a result of performing the performance analysis of the network element device.
Preferably, the performance analysis server includes: a log data storage unit, an analysis result storage unit and an analysis result display unit, wherein,
the log data storage unit is used for storing the transmission control protocol data packet output by the convergence server;
the analysis unit is used for reading the transmission control protocol data packet stored in the log data storage unit, analyzing and outputting an analysis result to the analysis result storage unit;
an analysis result storage unit for storing the analysis result output by the analysis unit;
and the analysis result display unit is used for reading and displaying the analysis result from the analysis result storage unit.
Preferably, the proxy server is deployed on the network element device, and the network element device transmits the log data to the proxy server deployed on the network element device in a user datagram protocol manner.
Preferably, the system further comprises:
the configuration server is used for configuring configuration information containing a set data format, a data transmission mode and network element equipment attributes and sending the configuration information to the network element equipment; and monitoring the state of the proxy server, and when the downtime of the proxy server is monitored, repairing and starting the downtime of the proxy server through remote control.
Preferably, the system further comprises:
and the control server is used for reading the performance analysis result of the network element equipment obtained by the performance analysis server, and controlling the number of users accessing the intelligent television system if the performance analysis result of the network element equipment indicates that the concurrence rate is lower than a preset rate threshold value.
It can be seen from the foregoing technical solutions that, the method and system for monitoring performance of a network element device provided in the embodiments of the present invention monitor performance of a network element device at a service level. Because the performance information is collected in real time, the collected performance information is packaged into log data according to a preset data format, and the log data is transmitted according to a preset data transmission mode, the performance analysis is not performed on each log data. Then, the transmitted log data are received, and clustering such as total times, mean values, maximum values and minimum values is performed on the log data according to a preset performance information acquisition period, so that the transmitted log data volume and background calculation volume can be effectively reduced. And finally, merging the clustered log data according to a preset log data merging period, and analyzing the performance of the network element equipment according to the log data obtained by merging. Therefore, the log data subjected to clustering can cover the source data of the performance of the network element equipment in the intelligent television system, and the reliability of the analysis result is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of the invention and that other embodiments and drawings may be devised by those skilled in the art based on the exemplary embodiments shown in the drawings.
Fig. 1 is a flowchart illustrating a method for monitoring network element device performance according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a system for monitoring network element device performance according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a log data structure output by a network element device according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a data structure of a clustered functional log according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a merged functional log data structure according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of an analysis result shown in the embodiment of the present invention.
Detailed Description
The technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the existing method for monitoring the performance of the network element equipment, the performance information is collected in a sampling mode, log data is generated and analyzed, so that the analyzed log data cannot cover the source data of the real-time performance of the network element equipment in the intelligent television system, and the reliability of the analysis result is low.
The User Datagram Protocol (UDP) is a connectionless transport layer protocol in an Open System Interconnection (OSI) reference model, and UDP does not provide packet grouping, assembling and sequencing services, but only provides a transaction-oriented simple unreliable information transfer service. Through the UDP protocol, the network data flow can be compressed into a UDP data packet form, and the UDP does not belong to a connection type protocol, so that the method has the advantages of low resource consumption and high processing speed, and can be used for network application needing to transmit data between computers. For example, network video conferencing, audio, video and general data transfer (ICQ and QQ) applications, which do not have much influence on the reception result even if a small amount of UDP packets are lost when transmitting.
In the embodiment of the invention, a method for monitoring the performance of network element equipment is provided, wherein the network element equipment of an intelligent television system outputs log data reflecting the performance state in a UDP (user Datagram protocol) data packet mode. Therefore, the resource consumption of the UDP data packet mode is low, and the concurrency performance of the network element equipment is hardly influenced; after receiving the UDP data packet, the system for monitoring the performance of the network element device in the embodiment of the present invention merges log data in the UDP data packet in a flow counting manner, that is, merges log data in a calculation merging count, an average value, a merging manner, a maximum and minimum value, a sampling manner, and the like, and then transmits the merged log data. Therefore, the transmitted data volume can be greatly reduced, the merged log data can cover the source data of the network element equipment in the intelligent television system, the original appearance of the log data is kept to a great extent, and the requirement of the centralized performance real-time monitoring of the network element equipment is met.
Fig. 1 is a flowchart illustrating a method for monitoring network element device performance according to an embodiment of the present invention. Referring to fig. 1, the process includes:
step 101, presetting a data format and a data transmission mode for performance monitoring in network element equipment;
in this step, the data format and the data transmission mode may be set on the configuration server and then sent to each network element device, or may be directly set on each network element device. Wherein,
the data format includes: network element equipment identification code, function number, behavior identification, data time point, additional data and other information. Wherein,
the network element equipment identification code is a network element equipment identifier and is used for distinguishing network element equipment in the intelligent television system, and the network element equipment identification code and the network element equipment are in one-to-one correspondence;
the function number is used to identify the function of the performance information in the network element device, that is, the performance information is specific function data in the network element device, for example, audio, video, etc.;
the behavior identifier is used for identifying the behavior of the performance information in the network element equipment. May be a triggering action, a persistence action, or an error action. Wherein,
trigger behavior means that when an event occurs, for example, when a function is used, a corresponding piece of log data is recorded. The persistence behavior represents the duration of an event, including: start behavior information and end behavior information. And the error behavior indicates that the current operation has errors, and error data and reasons are recorded. The triggering behavior only needs to count the triggering times, and for the continuous behavior, the triggering times are counted, and the time for processing the behavior is counted. In practical application, which behavior needs to be reported may be determined by data statistics requirements. For example, if a function a is triggered, the function a is recorded once as a triggering action; if the duration of the function a needs to be recorded, for example, 1 millisecond is needed for completing the function, a persistent behavior start is declared when the function a starts, a persistent behavior end is declared again when the function a ends, and the proxy server automatically merges the persistent behavior start and the persistent behavior end.
The data time point is the time point information of recording performance information;
the additional data is specific performance information. For example, when error action occurs, the additional data is detailed information of the occurrence of an error; when the triggering action occurs, the additional data is debugging information and the like.
Preferably, the data format may further include a network element device number, which is used to identify a network element device deployed in the smart television system, where the network element device and the network element device number may be in a one-to-many relationship, that is, one network element device may have multiple network element device numbers.
The data transmission mode is UDP transmission.
Step 102, network element equipment collects performance information, packages the performance information into log data according to a preset data format, and transmits the log data to a proxy server according to a preset data transmission mode;
in this step, the log data includes: network element equipment identification code information, network element equipment number information, function number information, behavior identification information, data time point information, additional data information and the like.
The network element equipment transmits the log data in a UDP mode, so that the resource consumption is low, and the concurrency performance of the network element equipment is hardly influenced.
103, the proxy server receives the log data sent by the network element equipment, clusters the log data according to a preset performance information acquisition cycle, and outputs the cluster data to the convergence server;
in this step, the performance information acquisition period may be determined according to actual needs.
Clustering log data includes:
a11, analyzing the received log data in a preset performance information acquisition period, and acquiring network element equipment identification code information, network element equipment number information, function number information and behavior identification information contained in the log data;
a12, judging whether the behavior identification information obtained by analysis is continuous behavior identification information, if so, executing step A13, otherwise, executing step A15;
a13, recording data time point information corresponding to the beginning of a continuous behavior in log data, and placing the log data into a temporary cache;
a14, after the end of the continuous behavior is monitored, merging the log data of the continuous behavior in the temporary cache into a log data containing the duration, and executing the step A15;
in this step, for the log data of the continuous behavior, the log data from the beginning of the continuous behavior to the end of the continuous behavior is merged into one log data containing the duration, so that the amount of the log data to be transmitted can be effectively reduced. In practical applications, when the ending behavior waits for the timeout, the ending behavior can be merged into a piece of timeout continuous behavior data (log data). When the continuous behaviors are combined, the same continuous behavior corresponds to one piece of log data.
A15, counting the total times, the mean value, the maximum value and the minimum value of the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information;
in this step, in a preset performance information acquisition period, log data from the same function and the same behavior in the same network element device are counted, so that performance conditions of each function in the network element device can be monitored.
In the embodiment of the invention, the statistics of the average value, the maximum value and the minimum value of the log data is carried out according to the behavior identifier, the statistical granularity is larger, and in order to refine the statistical granularity, the log data can be sampled and collected for reserving some detailed performance information as a reference for carrying out the performance analysis of the network element equipment. For example, if ten thousand pieces of log data containing the same behavior occur in one second, it is difficult to record all the log data, but some details of the network element device can be reflected by a sampling set process, for example, by recording 20 pieces of log data in a sampling manner at fixed time intervals. Thus, preferably, the method may further include:
extracting log data with a preset threshold value from the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information, recording data time point information and duration information of the extracted log data, and generating a sampling set.
In this step, the sample set is expressed in { sample set (time, value) }, for example, the format of { sample set (time, value) } may be {2013-1-1018:10:20, 20 ms; 2013-1-1018:10:22, 21 ms; 2013-1-1018:10:22, 28ms }, where the value is a duration, e.g., 20 ms.
And A16, constructing clustered log data according to the total times, the mean value, the maximum value and the minimum value obtained through statistics.
In this step, constructing clustered log data includes:
and judging whether the behavior identification information in the received log data is error behavior information, if so, constructing clustered error behavior log data, and otherwise, constructing clustered functional log data.
In this step, the clustered functional log data includes: network element equipment identification code information, network element equipment number information, function number information, performance information acquisition time period information, total times information, mean value information, maximum value information, minimum value information, sampling set (time, value) information and the like. Wherein,
the performance information acquisition time period is from the action starting time to the action ending time, and the difference between the action ending time and the action ending time is a preset performance information acquisition period. For example, if the preset performance information acquisition period is 60 seconds, the performance information acquisition time period may be 2013-1-1018:10:20 to 2013-1-1018:11: 20.
Error behavior log data for a cluster, comprising: the network element equipment identification code information, the network element equipment number information, the function number information, the performance information acquisition time period information and the error information.
Of course, in practical applications, the function log data and the error behavior log data may further include the identification code information of the proxy server.
In the embodiment of the invention, after the log data are clustered, the log data are packaged into the TCP data packet and output to the convergence server in a TCP mode. In this way, by acquiring log data in real time, the acquired log data is packaged and clustered, for example, the total number of times, the mean value, the maximum value, and the minimum value are clustered according to the acquired log data, and then, performance analysis is performed based on the total number of times, the mean value, the maximum value, and the minimum value of the log data obtained by clustering, without performing performance analysis on each log data, that is, performance analysis performed on each log data acquired by sampling is completely different from that performed in the prior art by acquiring log data periodically. According to the method for monitoring the performance of the network element equipment, log data subjected to clustering processing can cover all source data of the performance of the network element equipment in the intelligent television system, and the reliability of an analysis result is improved; moreover, only the results (total times, mean value, maximum value and minimum value) of the clustering processing are transmitted for periodic collection, so that the transmitted log data volume and background calculation volume can be effectively reduced; furthermore, log data can be collected periodically to reserve some detailed performance information as a reference for network element equipment performance analysis.
And step 104, merging the received clustered log data by the convergence server according to a preset log data merging period, and analyzing the performance of the network element equipment by the performance analysis server according to the log data obtained by merging.
In this step, the log data merging period may be determined according to actual needs. Preferably, the log data merging period is an integer multiple of the performance information acquisition period.
Merging the clustered log data includes:
and in a preset log data merging period, merging log data containing the same network element equipment identification code information, network element equipment number information and function number information according to the received clustered log data to obtain merged log data.
The merged log data includes: merged error behavior log data and merged functional log data, wherein,
the merged functional log data includes: proxy server identification code information, network element equipment number information, function number information, log data merging time period information, merging total times information, merging mean information, merging maximum value information, merging minimum value information, and merging { sample set (time, value) } information. Wherein,
the log data merging time period is from merging start time to merging end time, and the difference between the merging end time and the merging start time is a preset log data merging period.
The total merging times are the times in each performance information acquisition period in the log data merging period, the average merging time is the average value in each performance information acquisition period, the maximum merging time is the maximum value in each performance information acquisition period, and the merging sampling set is the summary of all the data sampling sets.
The merged error behavior log data includes: the information of the identification code of the proxy server, the identification code of the network element equipment, the number information of the network element equipment, the function number information, the merging time period information of the log data and the error information.
Preferably, the aggregation server receives the clustered log data, merges the clustered log data, and stores the merged log data in the first MySQL database. Therefore, the performance analysis server reads the log data obtained by merging from the first MySQL database and performs performance analysis on the network element equipment.
Preferably, the performance analysis server stores the performance analysis result of the network element device in the second MySQL database after obtaining the performance analysis result of the network element device.
For a detailed process of analyzing the network element device performance according to the log data obtained by the merging process, reference may be made to related technical documents, which are not described herein again.
Preferably, the method may further include:
and 105, displaying the combined log data or the result of analyzing the performance of the network element equipment.
In this step, when the merged log data is displayed, some information in the merged log data may be filtered, that is, all information included in the merged log data does not need to be displayed. For example, for merged functional log data, the exposed information may include: network element equipment identification code information, log data merging time period information, merging total times information, merging mean information, merging maximum information, merging minimum information, and merging { sample set (time, value) } information.
Of course, in practical applications, the method may further include:
and if the network element equipment performance analysis result shows that the concurrency rate is lower than a preset rate threshold, controlling the number of users accessing the intelligent television system.
In this step, a detailed process of controlling the number of users accessing the smart television system may be specifically referred to in the related technical literature, and is not described herein again.
Fig. 2 is a schematic structural diagram of a system for monitoring network element device performance according to an embodiment of the present invention. Referring to fig. 2, the system includes: a network element device, a proxy server, a convergence server, and a performance analysis server, wherein,
the network element equipment stores a preset data format and a preset data transmission mode for performance monitoring; acquiring performance information, packaging the performance information into log data according to a preset data format, and transmitting the log data to a proxy server according to a preset data transmission mode;
fig. 3 is a schematic diagram of a log data structure output by a network element device according to an embodiment of the present invention. Referring to fig. 3, in the embodiment of the present invention, the log data includes: network element equipment identification code information, network element equipment number information, function number information, behavior identification information, data time point information, additional data information and the like.
In practical application, the network element device may automatically generate a random identifier for each persistent behavior. In the following, the proxy server sorts according to functions and random identifiers, and as the continuous behaviors appear in pairs, when only finishing behaviors are found, the log data are discarded; when only the starting behavior is found, judging whether the starting behavior is overtime or not, and if the starting behavior is overtime, discarding the starting behavior; otherwise, merging into a piece of log data.
Preferably, the network element device transmits the log data in a UDP manner.
The proxy server is used for receiving the log data sent by the network element equipment, clustering the log data according to a preset performance information acquisition cycle and outputting the log data to the convergence server;
in the embodiment of the present invention, the proxy server receives log data (UDP data) sent by the network element device, and performs clustering on the received UDP data according to a preset performance information acquisition period, for example, performs processing such as classification, summarization, merging, filtering, and sampling on the received UDP data to obtain total number information, mean information, maximum information, minimum information, and sample set information. The log data sent by each network element device is processed to generate clustered log data, that is, a Transmission Control Protocol (TCP) data packet, and the clustered log data is output to the convergence server.
The clustered log data includes: clustered error behavior log data and clustered functional log data.
Fig. 4 is a schematic diagram of a data structure of a clustered functional log according to an embodiment of the present invention. Referring to fig. 4, the clustered functional log data includes: network element equipment identification code information, network element equipment number information, function number information, performance information acquisition time period information, total times information, mean value information, maximum value information, minimum value information, and { sampling set (time, value) } information.
For sampling combination, in the embodiment of the present invention, average sampling is adopted, sampling points, for example, M points, of a preset threshold are extracted in a preset performance information acquisition period, and an average extraction interval is calculated according to a count in the performance information acquisition period. Specifically, M points are extracted uniformly in the preset performance information acquisition period, for example, if there are 10000 points in the preset performance information acquisition period and 20(M) points are required to be extracted now, one sampling point is extracted every 10000/20 ═ 500. Therefore, the calculation amount of the proxy server is relatively small, the required calculation resource is not large, but the high-concurrency log data can be summarized into the statistical data in the preset performance information acquisition period through the calculation of the proxy server, so that the centralized monitoring of the network element equipment becomes possible.
And when the preset performance information acquisition period is up, the proxy server carries out zero clearing treatment on the count.
Preferably, the proxy server is deployed on the network element device. The network element equipment transmits the log data to the proxy server deployed on the network element equipment in a UDP mode, so that the resource consumption is low, and the concurrency performance of the network element equipment is hardly influenced.
The aggregation server is used for merging the received clustered log data according to a preset log data merging period and outputting the merged log data to the performance analysis server;
in the embodiment of the invention, the convergence server receives the TCP data packet output by the proxy server, performs convergence processing and outputs the TCP data packet to the performance analysis server.
Preferably, the system may further comprise:
the configuration server (ConfigServer) is used for configuring configuration information containing set data format, data transmission mode and network element equipment attribute and sending the configuration information to the network element equipment; and monitoring the state of the proxy server, and when the downtime of the proxy server is monitored, repairing and starting the downtime of the proxy server through remote control.
In the embodiment of the invention, when the configuration server monitors that the proxy server is down, the proxy server can be remotely pulled up, namely, the proxy server is remotely controlled. Specifically, the method comprises the steps of accessing the internet in different places through a computer network, connecting a controlled proxy server, logging in the proxy server in a Secure Shell (SSH) mode, displaying a desktop environment of the controlled proxy server on a configuration server, configuring, installing a software program, modifying and the like the proxy server in different places with faults through the configuration server so as to eliminate the faults of the proxy server with the faults, and starting the proxy server with the faults eliminated in a remote awakening mode.
The network element device attributes may include: the network element equipment identification code, the network element equipment number, the function number, the performance information acquisition period, the sampling M value and other parameters.
In the embodiment of the present invention, the configuration server may be configured in a master-slave manner to ensure High Availability (HA).
And the performance analysis server is used for analyzing the performance of the network element equipment according to the log data obtained by merging processing.
In the embodiment of the present invention, the performance analysis server receives the TCP data packet output by the aggregation server, and stores the TCP data packet in a temporary storage, where the temporary storage may be a MySQL database established in a temporary file system (TMPFS). Because the high-concurrency data are clustered by the proxy server according to the preset performance information acquisition cycle, the resource overhead required by the clustering performed by the aggregation server and the resource overhead required by the storage of the performance analysis server are small.
In the embodiment of the invention, the performance analysis server can use the query function of the database to combine the functional log data according to the preset log data combination period.
The merged log data includes: merged error behavior log data and merged functional log data.
FIG. 5 is a diagram illustrating a merged functional log data structure according to an embodiment of the present invention. Referring to fig. 5, the merged functional log data includes: proxy server identification code information, network element equipment number information, function number information, log data merging time period information, merging total times information, merging mean information, merging maximum value information, merging minimum value information, and merging { sample set (time, value) } information.
Preferably, the performance analysis server may be further configured to present the merged log data or a result of performing the performance analysis of the network element device.
Of course, in practical applications, the system may further include:
and the control server is used for reading the performance analysis result of the network element equipment obtained by the performance analysis server, and controlling the number of users accessing the intelligent television system if the performance analysis result of the network element equipment indicates that the concurrence rate is lower than a preset rate threshold value.
In an embodiment of the present invention, the performance analysis server includes a log data storage unit, an analysis result storage unit, and an analysis result display unit (not shown in the figure), wherein,
the log data storage unit is used for storing the TCP data packet output by the convergence server;
the analysis unit is used for reading the TCP data packet stored in the log data storage unit, analyzing and outputting an analysis result to the analysis result storage unit;
an analysis result storage unit for storing the analysis result output by the analysis unit;
in the embodiment of the invention, the analysis result storage unit adopts a persistent database to store the analysis result. For example, the analysis results are persistently stored by adopting a relational database management system MySQL server. Of course, in practical applications, the log data storage unit may also store log data by using a MySql server built on the memory.
And the analysis result display unit is used for reading and displaying the analysis result from the analysis result storage unit.
In the embodiment of the invention, the analysis result display unit queries the analysis result in the analysis result storage unit (result database), refreshes the change of each data in a preset analysis time period in real time, and can further query the data.
Fig. 6 is a schematic structural diagram of an analysis result shown in the embodiment of the present invention. Referring to fig. 6, the illustrated functional log data includes: network element equipment identification code information, log data merging time period information, merging total times information, merging mean information, merging maximum information, merging minimum information, and merging { sample set (time, value) } information.
As can be seen from the above, in the method and system for monitoring performance of a network element device according to the embodiments of the present invention, by presetting a data format for performance monitoring of the network element device and a data transmission mode of UDP, the network element device collects performance information, packages the performance information into log data according to the preset data format, and transmits the log data by using UDP, so that performance of the network element device is monitored at a service level, and concurrent performance of the network element device is less affected; the proxy server receives log data sent by the network element equipment, clusters the log data according to a preset performance information acquisition cycle, can effectively reduce the amount of transmitted log data and background calculation amount, and can cover the source data of the performance of the network element equipment in the intelligent television system by the clustered log data, so that the reliability of an analysis result is improved, the real performance condition of the network element equipment can be reflected by the analysis result, and the control effect is good by controlling according to the analysis result; and merging the clustered log data, analyzing the merged log data, and easily monitoring the performance of the network element equipment, wherein the resource overhead required by analysis is low.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also encompasses these modifications and variations.
Claims (14)
1. A method for monitoring performance of network element equipment is provided, in the network element equipment, a data format and a data transmission mode for performance monitoring are preset, and the data format comprises: the network element equipment comprises a network element equipment identification code, a function number, a behavior identifier, a data time point and additional data information, wherein the behavior identifier is a trigger behavior identifier, a continuous behavior identifier or an error behavior identifier; the method comprises the following steps:
acquiring performance information, packaging the acquired performance information into log data according to a preset data format, and transmitting the log data according to a preset data transmission mode;
receiving the sent log data, and clustering the log data according to a preset performance information acquisition cycle;
merging the clustered log data according to a preset log data merging period, and analyzing the performance of the network element equipment according to the log data obtained by merging;
wherein clustering the log data comprises:
a11, analyzing the received log data in a preset performance information acquisition period, and acquiring network element equipment identification code information, network element equipment number information, function number information and behavior identification information contained in the log data;
a12, judging whether the behavior identification information obtained by analysis is continuous behavior identification information, if so, executing step A13, otherwise, executing step A15;
a13, recording data time point information corresponding to the beginning of a continuous behavior in log data, and placing the log data into a temporary cache;
a14, after the end of the continuous behavior is monitored, merging the log data of the continuous behavior in the temporary cache into a log data containing the duration, and executing the step A15;
a15, counting the total times, the mean value, the maximum value and the minimum value of the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information;
and A16, constructing clustered log data according to the total times, the mean value, the maximum value and the minimum value obtained through statistics.
2. The method of monitoring network element device performance of claim 1, wherein the method further comprises:
extracting log data with a preset threshold value from the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information, recording data time point information and duration information of the extracted log data, and generating a sampling set.
3. The method of monitoring network element device performance of claim 2, wherein said constructing log data of clusters comprises:
and judging whether the behavior identification information in the received log data is error behavior information, if so, constructing clustered error behavior log data, and otherwise, constructing clustered functional log data.
4. The method of monitoring network element device performance of claim 3, wherein the clustered functional log data comprises: network element equipment identification code information, network element equipment number information, function number information, performance information acquisition time period information, total times information, mean value information, maximum value information, minimum value information, and { sampling set (time, value) } information.
5. The method of monitoring network element device performance of claim 3, wherein the clustered error behavior log data comprises: the network element equipment identification code information, the network element equipment number information, the function number information, the performance information acquisition time period information and the error information.
6. The method of monitoring network element device performance of claim 1, wherein the merging clustered log data comprises:
and in a preset log data merging period, merging log data containing the same network element equipment identification code information, network element equipment number information and function number information according to the received clustered log data to obtain merged log data.
7. The method of monitoring network element device performance of any of claims 1 to 6, wherein the method further comprises:
and displaying the combined log data or displaying the result of the performance analysis of the network element equipment.
8. The method of monitoring network element device performance of any of claims 1 to 6, wherein the method further comprises:
and when the performance analysis result of the network element equipment shows that the concurrency rate is lower than a preset rate threshold, controlling the number of users accessing the intelligent television system.
9. A system for monitoring performance of network element devices, the system comprising: a network element device, a proxy server, a convergence server, and a performance analysis server, wherein,
the network element equipment stores a preset data format and a preset data transmission mode for performance monitoring; acquiring performance information, packaging the performance information into log data according to a preset data format, and transmitting the log data to a proxy server according to a preset data transmission mode; wherein the data format comprises: the network element equipment comprises a network element equipment identification code, a function number, a behavior identifier, a data time point and additional data information, wherein the behavior identifier is a trigger behavior identifier, a continuous behavior identifier or an error behavior identifier;
the proxy server is used for receiving the log data sent by the network element equipment, clustering the log data according to a preset performance information acquisition cycle and outputting the log data to the convergence server;
the aggregation server is used for merging the received clustered log data according to a preset log data merging period and outputting the merged log data to the performance analysis server;
the performance analysis server is used for analyzing the performance of the network element equipment according to the log data obtained by merging;
wherein clustering the log data comprises:
analyzing the received log data in a preset performance information acquisition period to acquire network element equipment identification code information, network element equipment number information, function number information and behavior identification information contained in the log data;
judging whether the behavior identification information obtained by analysis is the continuous behavior identification information,
if so, recording data time point information corresponding to the beginning of the continuous behavior in the log data, putting the log data into a temporary cache, and combining the log data of the continuous behavior in the temporary cache into a piece of log data containing the duration after the end of the continuous behavior is monitored; counting the total times, the mean value, the maximum value and the minimum value of received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information;
otherwise, counting the total times, the average value, the maximum value and the minimum value of the received log data with the same network element equipment identification code information, network element equipment number information, function number information and behavior identification information;
and constructing clustered log data according to the total times, the mean value, the maximum value and the minimum value obtained by statistics.
10. The system for monitoring performance of network element devices of claim 9, wherein the performance analysis server is further configured to display the merged log data or the result of performing the performance analysis of the network element device.
11. The system for monitoring network element device performance of claim 10, wherein the performance analysis server comprises: a log data storage unit, an analysis result storage unit and an analysis result display unit, wherein,
the log data storage unit is used for storing the transmission control protocol data packet output by the convergence server;
the analysis unit is used for reading the transmission control protocol data packet stored in the log data storage unit, analyzing and outputting an analysis result to the analysis result storage unit;
an analysis result storage unit for storing the analysis result output by the analysis unit;
and the analysis result display unit is used for reading and displaying the analysis result from the analysis result storage unit.
12. The system for monitoring network element device performance of claim 9, wherein the proxy server is deployed on the network element device, and the network element device transmits log data to the proxy server deployed on the network element device in a user datagram protocol manner.
13. The system for monitoring network element device performance of any one of claims 9 to 12, wherein the system further comprises:
the configuration server is used for configuring configuration information containing a set data format, a data transmission mode and network element equipment attributes and sending the configuration information to the network element equipment; and monitoring the state of the proxy server, and when the downtime of the proxy server is monitored, repairing and starting the downtime of the proxy server through remote control.
14. The system for monitoring network element device performance of any one of claims 9 to 12, wherein the system further comprises:
and the control server is used for reading the performance analysis result of the network element equipment obtained by the performance analysis server, and controlling the number of users accessing the intelligent television system if the performance analysis result of the network element equipment indicates that the concurrence rate is lower than a preset rate threshold value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310104796.0A CN103200046B (en) | 2013-03-28 | 2013-03-28 | The method and system of monitoring network element device performance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310104796.0A CN103200046B (en) | 2013-03-28 | 2013-03-28 | The method and system of monitoring network element device performance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103200046A CN103200046A (en) | 2013-07-10 |
| CN103200046B true CN103200046B (en) | 2016-01-20 |
Family
ID=48722412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310104796.0A Active CN103200046B (en) | 2013-03-28 | 2013-03-28 | The method and system of monitoring network element device performance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103200046B (en) |
Families Citing this family (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500071B (en) * | 2013-09-18 | 2017-01-25 | 湖南蚁坊软件有限公司 | Method and device for storing performance index data quantitatively |
| CN105101281A (en) * | 2014-05-16 | 2015-11-25 | 中兴通讯股份有限公司 | Performance data processing method and device |
| CN104579767B (en) * | 2014-12-29 | 2018-01-02 | 山石网科通信技术有限公司 | The sending method and system of gateway log information |
| CN105071946A (en) * | 2015-07-03 | 2015-11-18 | 北京奇虎科技有限公司 | System monitoring method and device |
| CN105357584B (en) * | 2015-12-11 | 2020-09-22 | Tcl科技集团股份有限公司 | User behavior log classification uploading method and system based on smart television |
| CN105791049B (en) * | 2016-03-08 | 2018-12-25 | 成都广达新网科技股份有限公司 | A method of EOC local-side device historical traffic is monitored using network management system |
| CN107515807B (en) * | 2016-06-17 | 2021-05-07 | 阿里巴巴集团控股有限公司 | Method and device for storing monitoring data |
| CN106453223B (en) * | 2016-07-15 | 2020-03-31 | 珠海金山网络游戏科技有限公司 | Seamless world game server system |
| CN106452867A (en) * | 2016-08-10 | 2017-02-22 | 贵阳朗玛信息技术股份有限公司 | Log message processing method and system |
| CN106294866B (en) * | 2016-08-23 | 2020-02-11 | 北京奇虎科技有限公司 | Log processing method and device |
| CN108259255B (en) * | 2016-12-29 | 2020-05-15 | 大唐移动通信设备有限公司 | Network element data analysis method and device |
| CN107196879B (en) * | 2017-05-18 | 2020-10-09 | 杭州敦崇科技股份有限公司 | UDP message processing method and device and network forwarding device |
| CN107493315B (en) * | 2017-06-28 | 2021-04-23 | 武汉斗鱼网络科技有限公司 | Behavior data collection method, resource server and storage medium |
| CN107888445B (en) * | 2017-10-18 | 2020-08-18 | 金蝶软件（中国）有限公司 | Method and device for analyzing performance state, computer equipment and storage medium |
| CN108460161B (en) * | 2018-03-30 | 2021-11-19 | 腾讯科技（深圳）有限公司 | Hierarchical sampling method and device and computer equipment |
| CN109492008A (en) * | 2018-11-13 | 2019-03-19 | 上海理想信息产业（集团）有限公司 | A kind of network big data design methods and system based on HBase |
| CN109861843B (en) * | 2018-11-28 | 2021-11-23 | 阿里巴巴集团控股有限公司 | Method, device and equipment for completely collecting and confirming log files |
| CN111159129A (en) * | 2019-12-31 | 2020-05-15 | 北京神州绿盟信息安全科技股份有限公司 | Statistical method and device for log report |
| CN112256653B (en) * | 2020-11-06 | 2024-02-02 | 网易（杭州）网络有限公司 | Data sampling method and device |
| CN112685248B (en) * | 2020-12-24 | 2024-07-05 | 北京浪潮数据技术有限公司 | Intelligent network card monitoring log acquisition method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101174972A (en) * | 2006-11-03 | 2008-05-07 | 盛趣信息技术（上海）有限公司 | System and method for analysis data collection and statistics |
| CN101291399A (en) * | 2007-09-25 | 2008-10-22 | 深圳市同洲电子股份有限公司 | Digital television receiver, log system and log management method thereof |
| CN102902752A (en) * | 2012-09-20 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and system for monitoring log |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070066297A1 (en) * | 2005-09-20 | 2007-03-22 | Ghobad Heidari-Bateni | Network monitoring system and method |
| CN102231130B (en) * | 2010-01-11 | 2015-06-17 | 国际商业机器公司 | Method and device for analyzing computer system performances |
-
2013
- 2013-03-28 CN CN201310104796.0A patent/CN103200046B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101174972A (en) * | 2006-11-03 | 2008-05-07 | 盛趣信息技术（上海）有限公司 | System and method for analysis data collection and statistics |
| CN101291399A (en) * | 2007-09-25 | 2008-10-22 | 深圳市同洲电子股份有限公司 | Digital television receiver, log system and log management method thereof |
| CN102902752A (en) * | 2012-09-20 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and system for monitoring log |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103200046A (en) | 2013-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103200046B (en) | The method and system of monitoring network element device performance | |
| CN108156006B (en) | Buried point data reporting method and device and electronic equipment | |
| CN112671560B (en) | High-availability distributed real-time alarm processing method and system | |
| CN109271412A (en) | The real-time streaming data processing method and system of smart city | |
| EP2852097B1 (en) | Efficient data center monitoring | |
| CN105376335B (en) | Collected data uploading method and device | |
| CN105610648B (en) | A kind of acquisition method and server of O&M monitoring data | |
| CN101997925A (en) | Server monitoring method with early warning function and system thereof | |
| CN108471366A (en) | A kind of stereoscopic monitoring system of facing cloud native applications | |
| CN109684052B (en) | Transaction analysis method, device, equipment and storage medium | |
| CN112311617A (en) | Configured data monitoring and alarming method and system | |
| CN109977089A (en) | Blog management method, device, computer equipment and computer readable storage medium | |
| CN109560989B (en) | Link monitoring system | |
| CN110320891A (en) | Rail vehicle braking system CAN bus message maintenance monitoring system and monitoring method | |
| CN110727556A (en) | BMC health state monitoring method, system, terminal and storage medium | |
| CN108737549A (en) | A kind of log analysis method and device of big data quantity | |
| CN110858192A (en) | Log query method and system, log checking system and query terminal | |
| CN110620699A (en) | Message arrival rate determination method, device, equipment and computer readable storage medium | |
| CN103517292A (en) | Mobile terminal information reporting method and apparatus | |
| CN103916463B (en) | Network access statistical analysis method and system | |
| CN108334429A (en) | Method, apparatus and system for investigating front end page problem | |
| CN111817865A (en) | Method for monitoring network management equipment and monitoring system | |
| CN106161339B (en) | Obtain the method and device of IP access relations | |
| CN106933718B (en) | Method for monitoring performance and device | |
| CN114189480B (en) | Flow sampling method, device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20170213 Address after: 266100 Shandong Province, Qingdao city Laoshan District Songling Road No. 399 Patentee after: Poly Polytron Technologies Inc Address before: 266061 Laoshan, Qingdao province Hongkong District No. East Road, room 248, room 131 Patentee before: Qingdao Hisense Media Networks Co., Ltd. |