CN105450434A - Internet traffic analysis method based on traffic graphs - Google Patents
Internet traffic analysis method based on traffic graphs Download PDFInfo
- Publication number
- CN105450434A CN105450434A CN201410425596.XA CN201410425596A CN105450434A CN 105450434 A CN105450434 A CN 105450434A CN 201410425596 A CN201410425596 A CN 201410425596A CN 105450434 A CN105450434 A CN 105450434A
- Authority
- CN
- China
- Prior art keywords
- flow
- traffic
- node
- bare
- spirogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an Internet traffic analysis method based on traffic graphs, specifically comprising the following steps: (1) collecting flow information generated at different time through traffic monitoring equipment in a network, wherein each piece of the collected traffic information corresponds to a traffic record; (2) establishing a traffic graph for the collected traffic information; (3) establishing a core traffic graph G2 based on the basic traffic graph G1; and (4) comparatively analyzing the statistical characteristics of the basic traffic graphs and the core traffic graphs formed based on different application traffics to obtain the distribution between important nodes and the connectivity between important nodes and non-important nodes. According to the invention, the traffic graphs are established by starting from actual traffic data, and the interaction behavior of network users is characterized accurately; and important nodes and edges are extracted from the basic traffic graph, the essential law of traffic interaction is easy to grasp, and the complexity of large-scale data analysis is reduced.
Description
Technical field
The present invention relates to a kind of Internet streaming analysis method based on flow diagram, belong to internet traffic analysis technical field.
Background technology
The Internet develops rapidly under the promotion of technology and market, and business demand drives speed, the kind rapid growth of flow.Internet traffic analytical technology is intended to the behavioral trait holding Internet user by excavating traffic characteristic, contribute to the network planning of science and dilatation, differentiated service quality control and network security and abnormality detection, management, planning, safety etc. for current real network and business all have obvious realistic meaning.
Flow analysis in the Internet is the hot issue in internet measurement field always, and domestic and international researcher has carried out long-term research.Large quantifier elimination concentrates on the analysis of flow microscopic characteristics, and the data set based on package level in real network or stream rank observes the feature of flow.Early stage report points out that data traffic is different from the Poisson distribution characteristic of phone traffic, has self similarity (self-similar) and fractal (fractal) feature.Follow-up research shows that in single stream, Inter-arrival Time obeys Gamma distribution respectively, and sudden significantly the reduction in convergence flow of bag length, the size of stream presents heavy-tailed (heavy-tailed) characteristic etc.Along with the development of flow monitoring technology and instrument, the flow of larger Time and place scale is collected and analyze, and find that flow shared by internet, applications has a great difference along with the difference of region, and P2P flow reduces to some extent, and video flow significantly increases.
The traffic characteristic that various application produces is paid close attention in research in recent years more, comprises web traffic, P2P flow, YouTube flow, game on line flow, online social networks flow etc.In addition, mobile Internet application more and more receives publicity, and by analyzing video flow characteristic in 3G cellular network, find that HLS accounts for 1/3 of whole video flow, most of video content is with the speed transmission lower than 255Kbps, and only the video of 40% is completely downloaded.
Above flow analysis technology mostly from the feature of application traffic itself (as)s such as port, fingerprint, statistical natures, observe the microscopic characteristics of internet traffic, as wrapped length, bag due in, packet interarrival times, bag amount of bursts etc., and then set up corresponding Mathematical Modeling.Prior art does not consider that flow produces, the natural characteristic with multiple participant for network interaction, and is not only the problem of communicating pair.
Summary of the invention
For the deficiency that prior art exists, the object of the invention is to provide a kind of Internet streaming analysis method based on flow diagram, by setting up flow diagram from actual flow data, the accurate characterization interbehavior of the network user, in bare flow figure, extract important node, limit is analyzed, both be easy to grasp the mutual essential laws of flow, again reduced the complexity of large-scale data analyzing and processing.
To achieve these goals, the present invention realizes by the following technical solutions:
A kind of Internet streaming analysis method based on flow diagram of the present invention, specifically comprises following step:
(1) by the traffic monitoring equipment in network, the stream information do not produced in the same time is gathered, the corresponding stream record of each stream information collected;
(2) stream information collected according to step (1) sets up bare flow figure G1, described bare flow figure G1 to build drawing method as follows:
Using stream record in source host and destination host as node, using the flow between source host and destination host alternately as limit, mutual for the flow on described limit summation is set to the weights on limit, the intensity of described node is the weights summation on all limits be connected with it;
(3) on the basis of described bare flow figure G1, set up core flow spirogram G2, described core flow spirogram G2 to build drawing method as follows:
Calculate the degree of each node in described bare flow figure G1, according to degree order from big to small, node is sorted; Choose the forward node of rank as important node, only retain the important node in bare flow figure G1 and the limit between them, delete the insignificant node in bare flow figure G1 and the limit between them, thus form core flow spirogram G2; Described core flow spirogram G2 interior joint number is the x% of bare flow figure G1 interior joint number;
(4) statistical property of the bare flow figure that comparative analysis different application flow is formed and core flow spirogram, can draw the distribution situation between important node, and the connectivity power between important node and insignificant node.
In step (1), the content flowing record described in every bar comprises time of origin, source and destination IP address, source and destination port, bag number and byte number and application type.
In step (1), in fixed, described traffic monitoring equipment can be arranged on the link between Access Network and backbone network;
In a mobile network, described traffic monitoring equipment can be installed on the link in the gprs networks between SGSN and GGSN;
By all stream informations of these links all by described traffic monitoring equipment records and analysis.
In step (3), by the quantitative analysis of P2PDownload, P2PStream, HTTP, VideoStream, IM different application stream, x% can be set to 1% to 10%.
In step (4), the statistical property of described bare flow figure and core flow spirogram comprises the change in bare flow figure and core flow spirogram moderate of nodes, limit number, average degree, maximal degree/minimum degree, mean intensity, maximum intensity/minimum strength, degree distribution and important node.
In step (4), the statistical property of the bare flow figure that comparative analysis different application flow is formed and core flow spirogram, can draw and connect closely between the important node in HTTP, VideoStream, IM, and connectivity between insignificant node is weak; And the important node in P2PDownload, P2PStream is evenly distributed, and connectivity between insignificant node is strong.
(1) the present invention sets up flow diagram from actual flow data, the accurate characterization interbehavior of the network user, is easy to excavate global traffic feature by graph structure;
(2) consider the actual operating mechanism of network, in bare flow figure, extract important node and important limit is analyzed, be both easy to grasp the mutual essential laws of flow, again reduced the complexity of large-scale data analyzing and processing;
(3) bare flow figure and core flow spirogram are contrasted, contribute to excavating the mutual difference of different application flow.
Accompanying drawing explanation
Fig. 1 is the bare flow figure G1 in the present embodiment;
Fig. 2 is the core flow spirogram G2 in the present embodiment.
Embodiment
The technological means realized for making the present invention, creation characteristic, reaching object and effect is easy to understand, below in conjunction with embodiment, setting forth the present invention further.
A kind of Internet streaming analysis method based on flow diagram of the present invention specifically comprises following step:
(1) the traffic monitoring equipment that network flow data passage is deployed in carrier network gathers.In fixed, traffic monitoring equipment can be deployed on the link between Access Network and backbone network; And in a mobile network, traffic monitoring equipment can be deployed on the link in GPRS network between SGSN and GGSN.By all stream informations of these links all by traffic monitoring equipment records and analysis, within one day 24 hours, just can produce more than one hundred million and flow records.
(2) bare flow figure and core flow spirogram is set up based on the data on flows collected, see Fig. 1, its node of bare flow figure G1 is the source/destination IP address in stream record, and the flow between source and destination transmits and forms limit, and the weights on limit are the uninterrupted transmitted.
(3) see Fig. 2, core flow spirogram G2 is the subgraph of G1.First sorted from big to small according to degree by the node in G1, choosing the node that rank is forward, is important node; Retain the important node in G1 and the limit between them, delete other node in G1 and limit, form G2.Nodes in G2 is the x% of G1 interior joint number.
The important parameter of core flow spirogram is the ratio x% of important node, can be configured according to actual conditions.By flowing quantitative analysis to actual P2PDownload, P2PStream, HTTP, VideoStream, IM etc., x% can be set to 1% to 10%.
(4) statistical property of the bare flow figure that comparative analysis different application flow is formed and core flow spirogram, as nodes, limit number, average degree, maximum/minimum degree, mean intensity, maximum/minimum strength, degree distribution, important node, in the change etc. of bare flow figure and core flow spirogram moderate, can observe the difference between different application.Such as, connect tightr between the important node in HTTP, VideoStream, IM, and connectivity between insignificant node is more weak; And the important node of P2PDownload and P2PStream is more evenly distributed, and connectivity between insignificant node is stronger.
Beneficial effect of the present invention is as follows:
(1) the present invention sets up flow diagram from actual flow data, the accurate characterization interbehavior of the network user, is easy to excavate global traffic feature by graph structure;
(2) consider the actual operating mechanism of network, in bare flow figure, extract important node and important limit is analyzed, be both easy to grasp the mutual essential laws of flow, again reduced the complexity of large-scale data analyzing and processing;
(3) bare flow figure and core flow spirogram are contrasted, contribute to excavating the mutual difference of different application flow.
More than show and describe general principle of the present invention and principal character and advantage of the present invention.The technical staff of the industry should understand; the present invention is not restricted to the described embodiments; what describe in above-described embodiment and specification just illustrates principle of the present invention; without departing from the spirit and scope of the present invention; the present invention also has various changes and modifications, and these changes and improvements all fall in the claimed scope of the invention.Application claims protection range is defined by appending claims and equivalent thereof.
Claims (6)
1. based on an Internet streaming analysis method for flow diagram, it is characterized in that, specifically comprise following step:
(1) by the traffic monitoring equipment in network, the stream information do not produced in the same time is gathered, the corresponding stream record of each stream information collected;
(2) stream information collected according to step (1) sets up bare flow figure G1, described bare flow figure G1 to build drawing method as follows:
Using stream record in source host and destination host as node, using the flow between source host and destination host alternately as limit, mutual for the flow on described limit summation is set to the weights on limit, the intensity of described node is the weights summation on all limits be connected with it;
(3) on the basis of described bare flow figure G1, set up core flow spirogram G2, described core flow spirogram G2 to build drawing method as follows:
Calculate the degree of each node in described bare flow figure G1, according to degree order from big to small, node is sorted; Choose the forward node of rank as important node, only retain the important node in bare flow figure G1 and the limit between them, delete the insignificant node in bare flow figure G1 and the limit between them, thus form core flow spirogram G2; Described core flow spirogram G2 interior joint number is the x% of bare flow figure G1 interior joint number;
(4) statistical property of the bare flow figure that comparative analysis different application flow is formed and core flow spirogram, can draw the distribution situation between important node, and the connectivity power between important node and insignificant node.
2. the Internet streaming analysis method based on flow diagram according to claim 1, is characterized in that,
In step (1), the content flowing record described in every bar comprises time of origin, source and destination IP address, source and destination port, bag number and byte number and application type.
3. the Internet streaming analysis method based on flow diagram according to claim 1, is characterized in that,
In step (1), in fixed, described traffic monitoring equipment can be arranged on the link between Access Network and backbone network;
In a mobile network, described traffic monitoring equipment can be installed on the link in the gprs networks between SGSN and GGSN;
By all stream informations of these links all by described traffic monitoring equipment records and analysis.
4. the Internet streaming analysis method based on flow diagram according to claim 1, is characterized in that,
In step (3), by the quantitative analysis of P2PDownload, P2PStream, HTTP, VideoStream, IM different application stream, x% can be set to 1% to 10%.
5. the Internet streaming analysis method based on flow diagram according to claim 4, is characterized in that,
In step (4), the statistical property of described bare flow figure and core flow spirogram comprises the change in bare flow figure and core flow spirogram moderate of nodes, limit number, average degree, maximal degree/minimum degree, mean intensity, maximum intensity/minimum strength, degree distribution and important node.
6. the Internet streaming analysis method based on flow diagram according to claim 5, is characterized in that,
In step (4), the statistical property of the bare flow figure that comparative analysis different application flow is formed and core flow spirogram, can draw and connect closely between the important node in HTTP, VideoStream, IM, and connectivity between insignificant node is weak; And the important node in P2PDownload, P2PStream is evenly distributed, and connectivity between insignificant node is strong.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410425596.XA CN105450434A (en) | 2014-08-27 | 2014-08-27 | Internet traffic analysis method based on traffic graphs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410425596.XA CN105450434A (en) | 2014-08-27 | 2014-08-27 | Internet traffic analysis method based on traffic graphs |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105450434A true CN105450434A (en) | 2016-03-30 |
Family
ID=55560244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410425596.XA Pending CN105450434A (en) | 2014-08-27 | 2014-08-27 | Internet traffic analysis method based on traffic graphs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105450434A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106059830A (en) * | 2016-07-18 | 2016-10-26 | 安徽农业大学 | Automatic analysis method for traffic performance of PTN (Packet Transport Network) ring network |
| CN106941419A (en) * | 2017-03-13 | 2017-07-11 | 中国科学院深圳先进技术研究院 | The visual analysis method and system of network architecture and network communication mode |
| WO2017190488A1 (en) * | 2016-05-05 | 2017-11-09 | 腾讯科技（深圳）有限公司 | User interaction parameter acquisition method and device, and computer storage medium |
| WO2018165823A1 (en) * | 2017-03-13 | 2018-09-20 | 中国科学院深圳先进技术研究院 | Visual analysis method and system for network architecture and network communication mode |
| CN110933101A (en) * | 2019-12-10 | 2020-03-27 | 腾讯科技（深圳）有限公司 | Security event log processing method, device and storage medium |
| CN113037775A (en) * | 2021-03-31 | 2021-06-25 | 上海天旦网络科技发展有限公司 | Network application layer full-flow vectorization record generation method and system |
| CN114928545A (en) * | 2022-03-31 | 2022-08-19 | 中国电子科技集团公司第十五研究所 | Spark-based large-scale flow data key node calculation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101801036A (en) * | 2010-03-03 | 2010-08-11 | 华为终端有限公司 | Network traffic management method and system and common node |
| US20130013534A1 (en) * | 2011-07-07 | 2013-01-10 | International Business Machines Corporation | Hardware-assisted approach for local triangle counting in graphs |
| CN103001814A (en) * | 2011-09-09 | 2013-03-27 | 湖南神州祥网科技有限公司 | Method for describing network flow characteristic statistics |
-
2014
- 2014-08-27 CN CN201410425596.XA patent/CN105450434A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101801036A (en) * | 2010-03-03 | 2010-08-11 | 华为终端有限公司 | Network traffic management method and system and common node |
| US20130013534A1 (en) * | 2011-07-07 | 2013-01-10 | International Business Machines Corporation | Hardware-assisted approach for local triangle counting in graphs |
| CN103001814A (en) * | 2011-09-09 | 2013-03-27 | 湖南神州祥网科技有限公司 | Method for describing network flow characteristic statistics |
Non-Patent Citations (2)
| Title |
|---|
| 《NETWORK INFRASTRUCTURE AND DIGITAL CONTENT (IC-NIDC), 2012 3RD IEEE INTERNATIONAL CONFERENCE ON》 * |
| 《WIRELESS PERSONAL MULTIMEDIA COMMUNICATIONS (WPMC), 2013 16TH INTERNATIONAL SYMPOSIUM ON》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017190488A1 (en) * | 2016-05-05 | 2017-11-09 | 腾讯科技（深圳）有限公司 | User interaction parameter acquisition method and device, and computer storage medium |
| CN107346517A (en) * | 2016-05-05 | 2017-11-14 | 腾讯科技（深圳）有限公司 | User-interaction parameter acquisition methods and acquisition device in customer relationship network |
| CN106059830A (en) * | 2016-07-18 | 2016-10-26 | 安徽农业大学 | Automatic analysis method for traffic performance of PTN (Packet Transport Network) ring network |
| CN106059830B (en) * | 2016-07-18 | 2020-10-13 | 安徽农业大学 | Automatic analysis method for traffic performance of PTN (packet transport network) ring network |
| US10833964B2 (en) | 2017-03-13 | 2020-11-10 | Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences | Visual analytical method and system for network system structure and network communication mode |
| CN106941419A (en) * | 2017-03-13 | 2017-07-11 | 中国科学院深圳先进技术研究院 | The visual analysis method and system of network architecture and network communication mode |
| WO2018165823A1 (en) * | 2017-03-13 | 2018-09-20 | 中国科学院深圳先进技术研究院 | Visual analysis method and system for network architecture and network communication mode |
| CN106941419B (en) * | 2017-03-13 | 2019-12-06 | 中国科学院深圳先进技术研究院 | visual analysis method and system for network architecture and network communication mode |
| CN110933101A (en) * | 2019-12-10 | 2020-03-27 | 腾讯科技（深圳）有限公司 | Security event log processing method, device and storage medium |
| CN110933101B (en) * | 2019-12-10 | 2022-11-04 | 腾讯科技（深圳）有限公司 | Security event log processing method, device and storage medium |
| CN113037775A (en) * | 2021-03-31 | 2021-06-25 | 上海天旦网络科技发展有限公司 | Network application layer full-flow vectorization record generation method and system |
| CN113037775B (en) * | 2021-03-31 | 2022-07-29 | 上海天旦网络科技发展有限公司 | Network application layer full-flow vectorization record generation method and system |
| CN114928545A (en) * | 2022-03-31 | 2022-08-19 | 中国电子科技集团公司第十五研究所 | Spark-based large-scale flow data key node calculation method |
| CN114928545B (en) * | 2022-03-31 | 2024-02-06 | 中国电子科技集团公司第十五研究所 | Spark-based large-scale flow data key node calculation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105450434A (en) | Internet traffic analysis method based on traffic graphs | |
| CN104796348B (en) | IDC network egress flow equalizations method of adjustment, equipment and system based on SDN | |
| CN102035698B (en) | HTTP tunnel detection method based on decision tree classification algorithm | |
| CN106656616A (en) | Whole network flow analysis method of computer network | |
| CN102739457B (en) | Network flow recognition system and method based on DPI (Deep Packet Inspection) and SVM (Support Vector Machine) technology | |
| CN109495317B (en) | Data network flow prediction method and device | |
| Gürsun et al. | On traffic matrix completion in the internet | |
| CN105490834B (en) | A kind of probe deployment method based on vertex covering and weak vertex cover | |
| CN108111361B (en) | Transmission network fault positioning analysis method and system based on big data analysis | |
| CN103200133A (en) | Flow identification method based on network flow gravitation cluster | |
| CN104753732A (en) | Distribution based network traffic analysis system and method | |
| CN106559407A (en) | A kind of Network traffic anomaly monitor system based on SDN | |
| JP6290849B2 (en) | Traffic analysis system and traffic analysis method | |
| CN107147514A (en) | A kind of powerline network is optimized allocation of resources method and system | |
| CN110417729A (en) | A kind of service and application class method and system encrypting flow | |
| Lad et al. | Link-rank: A graphical tool for capturing bgp routing dynamics | |
| CN103973589A (en) | Network traffic classification method and device | |
| CN111935063A (en) | System and method for monitoring abnormal network access behavior of terminal equipment | |
| CN106535240A (en) | Mobile APP centralized performance analysis method based on cloud platform | |
| CN105227548A (en) | Based on the abnormal flow screening technique of ' Office LAN steady-state model | |
| Mori et al. | Flow analysis of internet traffic: World Wide Web versus peer‐to‐peer | |
| CN104079452A (en) | Data monitoring technology and network traffic abnormality classifying method | |
| Siska et al. | A flow trace generator using graph-based traffic classification techniques | |
| CN108540443A (en) | A kind of computer Traffic anomaly detection analysis system | |
| CN106257867A (en) | A kind of business recognition method encrypting flow and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160330 |