CN104092623A - Method and device used for carrying out overload protection on network node or server - Google Patents
Method and device used for carrying out overload protection on network node or server Download PDFInfo
- Publication number
- CN104092623A CN104092623A CN201310110430.4A CN201310110430A CN104092623A CN 104092623 A CN104092623 A CN 104092623A CN 201310110430 A CN201310110430 A CN 201310110430A CN 104092623 A CN104092623 A CN 104092623A
- Authority
- CN
- China
- Prior art keywords
- server
- data flow
- cluster
- flow
- network node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention provides a method which transmits data from a terminal to a server through a transmission network and carries out overload protection on a network node or a server in a data transmission and processing system. The method comprises the steps that when the network node or the server is overloaded, the flow characteristics of a processed data flow are collected; when the network node or the server is overloaded, clustering is carried out on a data flow with the similar flow characteristics in the processed data flow to generate a cluster to be processed; and according to the flow characteristics of the data flow in the cluster to be processed, a reverse control policy is applied to the data flow in the cluster to be processed, so as to eliminate the overload of the network node or the server.
Description
Technical field
The present invention relates to mass data transfers and treatment system; be specifically related to a kind of by data from terminal by transmission network to the method and apparatus that network node or server is carried out the data transmission processing system of server transmission overload protection; can prevention service device and flow surge and the negative effect thereof of network side, and avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
Background technology
Along with the development of network and information technology, the application with mass data more and more occurs.As shown in Figure 4, data transmission processing system comprises mass data terminal 201, transmission network 202 and application server 203.Distributed mass data terminal 201 is sent to data on application server 203 and processes by transmission network 202.Due to the surge of data volume, in current data transmission processing system, there is overload in the server 205 in network node 204 and application server 203 again and again.
In the disclosed network traffics surge protection of patent application (US2010/0020687) technology; obtain multiple network traffics and network resource utilization information; determine a best allocated bandwidth to these network flows, in the time that flow has exceeded best allocated bandwidth, carry out packet loss processing according to priority.Only consider resource and the surge control of network side, do not considered resource and the flow surge of end side and server side.And single packet loss processing brings the risk of loss of data.
Therefore, need a kind of utilization to carry out comprehensive monitoring to the resource of terminal, transmission network and server, terminal flow is carried out to clustering control, and comprehensive various strategy, the overload problem that the surge of mass data terminal and generation thereof is caused is carried out the technology of network and server protection.
Summary of the invention
For the above-mentioned defect that overcomes prior art has proposed the present invention.Therefore; the object of the invention is to propose a kind of by data from terminal by transmission network to the method and apparatus that network node or server is carried out the data transmission processing system of server transmission overload protection; can prevention service device and flow surge and the negative effect thereof of network side, and avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
To achieve these goals, according to the present invention, proposed a kind of by data from terminal by transmission network to the method for network node or server being carried out the data transmission processing system of server transmission overload protection, comprise: in the time of network node or server generation overload, collect the properties of flow of handled data flow; In the time of network node or server generation overload, the data flow with similar properties of flow in handled data flow is carried out to clustering and generate pending cluster; And according to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the overload of network node or server.
Preferably, described properties of flow comprises: time-delay characteristics, packet loss characteristic or network congestion characteristic, described time-delay characteristics, packet loss characteristic and network congestion characteristic are characterized by delay parameter, packet loss parameter and the path congestion node parameter of data flow respectively.
Preferably, for according to the pending cluster of time-delay characteristics clustering, carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.
Preferably, for according to the pending cluster of packet loss characteristic clustering, carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.
Preferably, for according to the pending cluster of network congestion characteristic clustering, carry out at least one in following oppositely control strategy: the terminal that carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster, is sending at least one data flow in this pending cluster is carried out packet loss re-transmission policy and at least one the data flow execution route optimisation strategy in this pending cluster.
Preferably, described pending cluster comprises at least one data flow.
In addition, according to the present invention, proposed a kind of by data from terminal by transmission network to the device that network node or server is carried out the data transmission processing system of server transmission overload protection, comprise: in the time of network node or server generation overload, collect the unit of the properties of flow of handled data flow; In the time that overload occurs for network node or server, the data flow with similar properties of flow in handled data flow is carried out clustering and is generated the unit of pending cluster; And according to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the unit of overload of network node or server.
Invention effect
According to the present invention, can prevention service device and flow surge and the negative effect thereof of network side, avoided the negative effect of the phase mutual interference of terminal room by reverse cluster control.
Brief description of the drawings
By reference to following combination accompanying drawing, to adopted detailed description of the preferred embodiment, it is more apparent that above-mentioned purpose of the present invention, advantage and feature will become, wherein:
Fig. 1 shows the pie graph that network node or server is carried out to the device of overload protection according to of the present invention.
Fig. 2 shows the flow chart that network node or server is carried out to the method for overload protection according to of the present invention.
Fig. 3 shows when when overload occurs server and treats Processing Cluster and apply the schematic diagram of reverse control strategy.
Fig. 4 shows the signal pie graph of the data transmission processing system of prior art.
Embodiment
The preferred embodiments of the present invention are described below with reference to the accompanying drawings.
Fig. 1 shows the pie graph that network node or server is carried out to the device of overload protection according to of the present invention.
As shown in Figure 1, the device 1 that network node or server are carried out to overload protection according to the present invention comprises status information capture unit 101, flow label unit 102, cluster generation unit 103, reverse flow control unit 104 and control strategy storehouse 105.
Fig. 2 shows the flow chart that network node or server is carried out to the method for overload protection according to of the present invention.
In this supposition: in the time that mass data terminal is transmitted data by transmission network to application server, network node N1 or Ni, server S 1 or Sk exist the risk of overload.
Now, server and network state information (step 401) are collected in status information capture unit 101, and collected server and network state information are as shown in table 1 below:
Table 1
| State | S1 | Sk | N1 | Ni |
| Server parameter 1:CPU | 80％ | 90％ | ? | ? |
| Server parameter 2:RAM | 60％ | 100％ | ? | ? |
| Server parameter 3:I/O | 40％ | 100％ | ? | ? |
| Network parameter: congested node buffer memory | ? | ? | 90％ | 96％ |
In table 1, data are carried out to real-time update.When reaching pre-overload threshold value, the relevant parameter of server S k (or network node) enters pre-overload judgement (step 402).As above in table 1, system can be at least according to a certain server parameter, cpu resource load condition 90% > 80% of for example Sk, or RAM, or I/O resource is in full load condition 100%, judges that Sk enters pre-overload.Comparatively speaking, the CPU of S1, RAM and I/O resource use value are lower, therefore can not be judged as pre-overload.System also can be used the combination of above parameter to carry out the judgement of pre-overload.
Then, the parameter (step 403) of object flow is collected in flow label unit 102.This collection operation is also analyzed for the handled data flow of server S k, marks relevant properties of flow, and for example time-delay characteristics, packet loss characteristic or network congestion characteristic, characterized by delay parameter, packet loss parameter, path congestion node parameter respectively.For example can utilize the time field of host-host protocol to know time delay, the timestamp of such as RSVP etc.Utilize the data sequence number UID of transmission or message field (MFLD) to know packet loss.Can know network path congested node according to IETF/CONEX RFC.Shown in relevant properties of flow table 2 specific as follows:
Table 2
Here P4 other parameters 4 that use that express possibility, Pi other parameter I that use that express possibility, representation parameter P4 and the different values such as x, y, z.
In step 404, whether determining server Sk transships.Value in reference shown in table 1, system can be at least according to a certain server parameter, and the cpu resource load condition of for example Sk increases to 95% >=90%, or RAM, or I/O resource is in full load condition 100%, judges that Sk enters overload.System also can be used the combination of above parameter to carry out the judgement of overload.
When the relevant parameter of server S k reaches overload protection thresholding;, while there is overload (step 404 be), cluster generation unit 103 carries out clustering to the data flow with similar properties of flow in handled data flow and generates pending cluster (step 405).
Here can carry out clustering mainly for single parameter to the application of some single parameter sensitivity.For example: some application is in real time very sensitive to delay parameter.Therefore, can detect for the delay parameter of numerous streams specially, select time delay grade not homogeneous turbulence generate different clusters and apply reverse control strategy after a while.
Certainly, also can carry out comprehensive selection to multiple parameters and carry out clustering
Data in reference table 2, F1 and F3 can become cluster 1, and this cluster has obvious time-delay characteristics, forms the pending cluster according to time-delay characteristics clustering.F1 and F2 can become cluster 2, and this cluster has obvious packet loss characteristic, form according to the pending cluster of packet loss characteristic clustering.F3 and Fn can become cluster 3, and this cluster has network congestion characteristic, according to the pending cluster of network congestion characteristic clustering.In this example, each cluster has been lifted two streams and has been generated as cluster, but the present invention is not limited thereto, and can be that single or multiple stream generates different clusters.
Reverse flow control unit 104 obtains corresponding oppositely control strategy from control strategy storehouse 105, and the data flow in pending cluster is applied reverse control strategy, to eliminate the overload (step 406 and 407) of network node or server.Relevant reverse control strategy can be as follows but be not limited to this:
Time delay sending strategy: send time shift message to making a start of object flow, the transmission delay differing in size in the insertion of making a start.
Packet loss re-transmission policy: part stream is carried out to packet loss processing, then retransmit in the time of the non-overload of server.
Routing optimality strategy: the stream that route is occurred to congested node carries out routing optimality.For example, in table 2, there is network congestion node N1 or Ni in F3 and F4, avoids N1 or Ni node thereby the transmission route of F3 and F4 can be updated to new route.For example use MPLS or Software Defined Network (SDN) technology etc.
Here for according to the pending cluster of time-delay characteristics clustering, can carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.For according to the pending cluster of packet loss characteristic clustering, can carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.For according to the pending cluster of network congestion characteristic clustering, can carry out at least one in time delay sending strategy, packet loss re-transmission policy and path optimization's strategy.
Although the above situation that server k is occurred to transship is described, the present invention is not limited thereto, and the situation that can also occur to transship to network node similarly be processed.
Fig. 3 shows when when overload occurs server and treats Processing Cluster and apply the schematic diagram of reverse control strategy.
As shown in Figure 3, in the time of server overload, there are five cluster A, B, C, D and E with different characteristic.Because the stream in cluster B and D has obvious network congestion node diagnostic, therefore the reverse control that cluster B and D are correlated with.Relevant reverse control strategy can be one or several the combination in above strategy (time delay sending strategy, packet loss re-transmission policy and path optimization's strategy).
By the present invention; in the time of mass data processing, server and network node occur when congested; the stream by handle with similar features carries out cluster and the reverse control for cluster employing corresponding strategies, can protect server or network over loading, the normal operation of protection server or network.
Although below show the present invention in conjunction with the preferred embodiments of the present invention, one skilled in the art will appreciate that without departing from the spirit and scope of the present invention, can carry out various amendments, replacement and change to the present invention.Therefore, the present invention should not limited by above-described embodiment, and should be limited by claims and equivalent thereof.
Claims (7)
- By data from terminal by transmission network to a method of network node or server being carried out the data transmission processing system of server transmission overload protection, comprising:In the time of network node or the pre-overload of server generation, collect the properties of flow of handled data flow;In the time of network node or server generation overload, the data flow with similar properties of flow in handled data flow is carried out to clustering and generate pending cluster; AndAccording to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the overload of network node or server.
- 2. method according to claim 1, wherein,Described properties of flow comprises: time-delay characteristics, packet loss characteristic or network congestion characteristic, described time-delay characteristics, packet loss characteristic and network congestion characteristic are characterized by delay parameter, packet loss parameter and the path congestion node parameter of data flow respectively.
- 3. method according to claim 2, wherein,For according to the pending cluster of time-delay characteristics clustering, carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster.
- 4. method according to claim 2, wherein,For according to the pending cluster of packet loss characteristic clustering, carry out packet loss re-transmission policy in the terminal that sends at least one data flow in this pending cluster.
- 5. method according to claim 2, wherein,For according to the pending cluster of network congestion characteristic clustering, carry out at least one in following oppositely control strategy: the terminal that carry out time delay sending strategy in the terminal that sends at least one data flow in this pending cluster, is sending at least one data flow in this pending cluster is carried out packet loss re-transmission policy and at least one the data flow execution route optimisation strategy in this pending cluster.
- 6. method according to claim 1, wherein,Described pending cluster comprises at least one data flow.
- By data from terminal by transmission network to a device that network node or server is carried out the data transmission processing system of server transmission overload protection, comprising:In the time of network node or the pre-overload of server generation, collect the unit of the properties of flow of handled data flow;In the time that overload occurs for network node or server, the data flow with similar properties of flow in handled data flow is carried out clustering and is generated the unit of pending cluster; AndAccording to the properties of flow of the data flow in pending cluster, the data flow for the treatment of in Processing Cluster is applied reverse control strategy, to eliminate the unit of overload of network node or server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310110430.4A CN104092623A (en) | 2013-04-01 | 2013-04-01 | Method and device used for carrying out overload protection on network node or server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310110430.4A CN104092623A (en) | 2013-04-01 | 2013-04-01 | Method and device used for carrying out overload protection on network node or server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104092623A true CN104092623A (en) | 2014-10-08 |
Family
ID=51640305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310110430.4A Pending CN104092623A (en) | 2013-04-01 | 2013-04-01 | Method and device used for carrying out overload protection on network node or server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104092623A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1689278A (en) * | 2001-12-18 | 2005-10-26 | 思科技术公司 | Methods and apparatus for network congestion control |
| CN1984043A (en) * | 2006-06-07 | 2007-06-20 | 华为技术有限公司 | Method and system for controlling flow based on bandwidth adjusting mechanism |
| CN101060475A (en) * | 2006-05-23 | 2007-10-24 | 华为技术有限公司 | A method and system for regulating the link loads of media gateway control interface |
| US7403988B1 (en) * | 2001-12-28 | 2008-07-22 | Nortel Networks Limited | Technique for autonomous network provisioning |
| CN101321173A (en) * | 2008-07-21 | 2008-12-10 | 华为技术有限公司 | Method, system and device for preventing network attack |
| US20100142524A1 (en) * | 2007-07-02 | 2010-06-10 | Angelo Garofalo | Application data flow management in an ip network |
| CN102271092A (en) * | 2011-09-09 | 2011-12-07 | 西安电子科技大学 | Congestion control system and method based on node load quantization level |
-
2013
- 2013-04-01 CN CN201310110430.4A patent/CN104092623A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1689278A (en) * | 2001-12-18 | 2005-10-26 | 思科技术公司 | Methods and apparatus for network congestion control |
| US7403988B1 (en) * | 2001-12-28 | 2008-07-22 | Nortel Networks Limited | Technique for autonomous network provisioning |
| CN101060475A (en) * | 2006-05-23 | 2007-10-24 | 华为技术有限公司 | A method and system for regulating the link loads of media gateway control interface |
| CN1984043A (en) * | 2006-06-07 | 2007-06-20 | 华为技术有限公司 | Method and system for controlling flow based on bandwidth adjusting mechanism |
| US20100142524A1 (en) * | 2007-07-02 | 2010-06-10 | Angelo Garofalo | Application data flow management in an ip network |
| CN101321173A (en) * | 2008-07-21 | 2008-12-10 | 华为技术有限公司 | Method, system and device for preventing network attack |
| CN102271092A (en) * | 2011-09-09 | 2011-12-07 | 西安电子科技大学 | Congestion control system and method based on node load quantization level |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106062726B (en) | Flow aware buffer management for data center switches | |
| JP4774357B2 (en) | Statistical information collection system and statistical information collection device | |
| JP2017034678A5 (en) | ||
| JP5659125B2 (en) | Relay device and relay method | |
| US10033644B2 (en) | Controlling congestion controlled flows | |
| JP5904908B2 (en) | Communication system and quality control server | |
| EP2688256A1 (en) | Network system, and switching method | |
| US9705745B2 (en) | System and method for virtualizing software defined network (SDN)-based network monitoring | |
| JP2012257010A (en) | Communication device, communication method, and remote monitoring system | |
| CN101350765B (en) | Network flow detection method | |
| Petrangeli et al. | Network-based dynamic prioritization of http adaptive streams to avoid video freezes | |
| CN109995664B (en) | Method, equipment and system for transmitting data stream | |
| CN109167734B (en) | Method and device for identifying transmission control protocol state | |
| CN114513467B (en) | Network traffic load balancing method and device of data center | |
| CN105007235B (en) | A kind of jamming control method in wireless multimedia sensor network | |
| US20220053373A1 (en) | Communication apparatus, communication method, and program | |
| CN104734905A (en) | Data flow detection method and device | |
| Siekkinen et al. | A root cause analysis toolkit for TCP | |
| JP5440200B2 (en) | Relay device and bandwidth control method | |
| JP2013179678A (en) | Router device | |
| EP3637692A1 (en) | Method, apparatus, and system for measuring transmission performance | |
| JP5742549B2 (en) | Packet capture processing method and apparatus | |
| CN104092623A (en) | Method and device used for carrying out overload protection on network node or server | |
| CN111385667A (en) | Video data processing method, device and computer readable storage medium | |
| JP2008079138A (en) | Communication monitoring system, flow collection apparatus, analysis manager apparatus, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141008 |