CN110719286A - Network optimization scheme sharing system and method based on big data - Google Patents
Network optimization scheme sharing system and method based on big data Download PDFInfo
- Publication number
- CN110719286A CN110719286A CN201910964870.3A CN201910964870A CN110719286A CN 110719286 A CN110719286 A CN 110719286A CN 201910964870 A CN201910964870 A CN 201910964870A CN 110719286 A CN110719286 A CN 110719286A
- Authority
- CN
- China
- Prior art keywords
- network
- module
- data
- ring
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0882—Utilisation of link capacity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of network optimization, and discloses a big data-based network optimization scheme sharing system and a method thereof, wherein the big data-based network optimization scheme sharing system comprises: the system comprises a data dynamic detection module, a vulnerability detection module, an intrusion detection module, a network equipment parameter acquisition module, a main control module, a network communication module, a data analysis module, a performance evaluation module, an optimization module, a sharing module, a big data processing module and a display module. The invention can improve the success rate of network intrusion detection through the intrusion detection module; meanwhile, the performance evaluation module evaluates the network condition by taking the ring as a basic unit, is closer to practical application, and can avoid blind whole-ring expansion caused by single-section heavy load from the perspective of network optimization through the flow balance in the ring, thereby realizing deep excavation of network bandwidth resources and improving the utilization rate of network resources.
Description
Technical Field
The invention belongs to the technical field of network optimization, and particularly relates to a big data-based network optimization scheme sharing system and method.
Background
Network optimization refers to bringing network performance to the best balance point we need through various hardware or software techniques. Network optimization is also SEO. The network optimization mainly comprises two aspects of equipment and service, wherein planning, evaluation and optimization belong to the service industry; under the existing network state, users often encounter the problems and troubles that broadband congestion, low application performance, worm virus, DDoS abuse, malicious intrusion and the like have negative influences on network use and resources, the network optimization function is to supplement the existing equipment and network problems such as firewall, security and intrusion detection, load balancing, bandwidth management, network antivirus and the like, parameter acquisition and data analysis can be carried out in a mode of accessing hardware and software operation to find out the reasons influencing the network quality, and the network can achieve the best operation state by technical means or a method for increasing corresponding hardware equipment and adjusting to enable the network resources to obtain the best benefits. Meanwhile, the method can realize the acceleration of network application performance, the management of security content, the management of security events, the management of users, the management and optimization of network resources, the management of desktop systems, the monitoring, measurement, tracking, analysis and management of traffic patterns, and improve the performance of application transmission on the wide area network. The product mainly comprises a network resource manager, an application performance accelerator and a webpage performance accelerator, and network optimization is carried out according to different requirements and functional requirements. The network optimization device also has functions such as supported protocols, network integration functions (tandem mode, bypass mode), device monitoring functions, compressed data statistics, QOS, bandwidth management, data export, application reporting, uninterrupted operation in case of failure, or upgrading over the network, etc. However, an attacker in the existing network optimization process can initiate an attack to a target server through the broiler chicken or the proxy server; the broiler chicken or the proxy server can respond to the verification message returned by the skip detection equipment, so that the network intrusion detection mode in the prior art can be penetrated; meanwhile, the blind whole-loop capacity expansion and performance improvement caused by network performance evaluation cannot be realized.
In summary, the problems of the prior art are as follows:
the network intrusion detection mode in the prior art is easily penetrated by the broiler or the proxy server in a mode of responding to the verification message returned by the jump detection equipment; meanwhile, the blind whole-loop capacity expansion and performance improvement caused by network performance evaluation cannot be realized.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a network optimization scheme sharing system and a network optimization scheme sharing method based on big data.
The invention is realized in such a way that a big data-based network optimization scheme sharing system comprises:
the data dynamic detection module is connected with the main control module and is used for detecting the network data dynamic information through a dynamic detection program;
the vulnerability detection module is connected with the main control module and used for detecting vulnerability information of the network system through a security detection program;
the intrusion detection module is connected with the main control module and used for detecting network intrusion information through an intrusion detection program;
the network equipment parameter acquisition module is connected with the main control module and is used for acquiring network equipment data through the monitoring chip;
the main control module is connected with the data dynamic detection module, the vulnerability detection module, the intrusion detection module, the network equipment parameter acquisition module, the network communication module, the data analysis module, the performance evaluation module, the optimization module, the sharing module, the big data processing module and the display module and is used for controlling each module to normally work through the single chip microcomputer;
the network communication module is connected with the main control module and is used for accessing the internet through a network card to carry out network communication;
the data analysis module is connected with the main control module and used for analyzing the detected data through an analysis program;
the performance evaluation module is connected with the main control module and used for evaluating the network performance through an evaluation program;
the optimization module is connected with the main control module and used for optimizing the network through an optimization program;
the sharing module is connected with the main control module and is used for sharing the network resources through a sharing program;
the big data processing module is connected with the main control module and used for carrying out big data processing on the detected network data by centralizing big data resources through the cloud server;
and the display module is connected with the main control module and used for displaying the detected network data dynamic state, the network loophole, the network intrusion information and the acquired network equipment data information through the display.
Further, the data analysis module includes:
the system comprises a dividing and sampling unit, a data processing unit and a data processing unit, wherein the dividing and sampling unit is used for dividing and investigating contents to be monitored from a database server, sampling in a layering manner, and extracting abstract data after data arrangement;
the parameter generating unit is used for generating a scatter diagram and historical data of data indexes to be monitored and associated data indexes according to the abstract data and selecting monitoring parameters;
the trend graph drawing unit is used for drawing a trend graph of the network data through a computer according to the generated monitoring parameters;
and the summarizing unit is used for summarizing the fluctuation condition of the network transmission data according to the drawn trend graph.
Another object of the present invention is to provide a big data based network optimization scheme sharing method for executing the big data based network optimization scheme sharing system, the big data based network optimization scheme sharing method comprising the steps of:
firstly, detecting network data dynamic information by using a dynamic detection program through a data dynamic detection module; detecting vulnerability information of the network system by a vulnerability detection module through a security detection program;
secondly, network intrusion information is detected by an intrusion detection program through an intrusion detection module; acquiring network equipment data by using a monitoring chip through a network equipment parameter acquisition module;
thirdly, the main control module accesses the internet by using the network card through the network communication module to carry out network communication;
analyzing the detected data by using an analysis program through a data analysis module; evaluating the network performance by a performance evaluation module by using an evaluation program;
fifthly, optimizing the network by utilizing an optimization program through an optimization module; sharing the network resource by using a sharing program through a sharing module;
step six, the big data processing module is used for carrying out big data processing on the detected network data by utilizing the cloud server to centralize big data resources; the big data processing adopts an encryption function to carry out bitwise encryption on the prefix member, the port number and the protocol number generated in the last step by using the secret key t, and transmits ciphertext data and the secret key to the middleware; the middleware carries out different processing on the IP data, the port number data and the protocol data; the port and the protocol can authorize the pseudo-random function to encrypt directly according to an encryption mode; obtaining the port number corresponding to the rule and the ciphertext data Enc of the protocolK(sourceport)、Enck(destport)、Enck(protocol); encrypting the IP address according to the length agreed by the protocol; enc generating the filter ruleK(F(sourceIP))、EncK(f (destip)) a set of ciphertext IP fields; the middleware processes the rule data according to prefix length convention and an encryption function, and the processing only needs to be processed once on the premise of not replacing the rule and the convention length; the data packet adopts a double-layer bloom filter; the bloom filter adopts a bit array V with the length of m and k mutually independent Haxi functions h1、h2、…、hk(ii) a When the element s needs to be stored to the bloom filter, the setting h is calculated separately1(s)、h2(s)、…、hk(s) and setting the bit value of the corresponding position in V to be '1'; when the element u needs to be judged whether to be in the bloom filter or not, checking the h-th element in the V1(u)、h2(u)、…、hk(u) whether the bit values of the positions are all 1, if all 1, the element u is in S with a high probability, and if not all 1, u is not in the bloom filter;
and seventhly, displaying the detected network data dynamic state, the network loophole, the network intrusion information and the acquired network equipment data information by using a display through a display module.
Further, the method for detecting the network intrusion information by the intrusion detection module comprises the following steps:
(1) acquiring access request records from a preset number of source IPs to a target IP through an intrusion detection program;
(2) counting a target access request record with a request source page identifier being empty in the access request records;
(3) respectively setting weight values for a plurality of characteristics of the target access request record, and writing the target access request record into a preset state table based on the set weight values;
(4) and matching the written target access request records in the preset state table, and taking the access request records in the matching result as network intrusion records.
Further, the obtaining of the access request records from the preset number of source IPs to the target IPs includes:
acquiring a preset number of access request records from real source IP to target IP based on a determination rule of the real source IP;
the determination rule of the real source IP specifically includes:
when an x-forward-for field of the access request record is empty, taking an IP address in a source IP field as a real source IP;
and when the x-forward-for field of the access request record is non-empty, determining the real source IP according to the IP address in the x-forward-for field.
Further, the setting of the weight values by the plurality of characteristics of the target access request record respectively comprises:
and determining a classification error rate corresponding to a target feature of the target access request record, and determining a weight value of the target feature based on the classification error rate.
Further, the method for evaluating the network performance by using the evaluation program is as follows:
1) acquiring the bandwidth utilization rate of each ring internal link of the current ring network through an evaluation program; determining the average value of the intra-loop bandwidth utilization rate of the current ring network according to the bandwidth utilization rate of the internal link of each ring; determining the intra-loop link load balance degree according to the intra-loop bandwidth utilization rate average value of the current ring network and the bandwidth utilization rate of each intra-loop internal link;
2) determining the maximum value of the bandwidth utilization rate of each loop internal link as the highest bandwidth utilization rate of the loop internal link according to the bandwidth utilization rate of each loop internal link; determining the network performance of the current ring network according to the load balance degree of the intra-ring link and the highest bandwidth utilization rate of the intra-ring link;
3) acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
4) determining the busyness of each ring internal link according to the sampling times of the super-early warning value and the total sampling times of each ring internal link, and determining the maximum value in the busyness of each ring internal link as the link busyness;
5) the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the link busyness degree;
further, the evaluating the network performance by using the evaluating program further comprises the steps of:
acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
determining the busyness of the current ring network according to the sampling times of the super-early warning value and the total sampling times of the internal links of each ring;
the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the busyness degree of the current ring network;
the method for determining the network performance of the current ring network comprises the following steps:
comparing the intra-loop link load balance degree with a preset balance degree threshold value, and determining an intra-loop link balance level corresponding to the intra-loop link load balance degree;
comparing the highest bandwidth utilization rate of the intra-loop link with a preset bandwidth utilization rate threshold value, and determining a highest bandwidth utilization rate level corresponding to the highest bandwidth utilization rate of the intra-loop link;
and determining the network performance of the current looped network according to the pairing relationship between the intra-loop link balancing level and the highest bandwidth utilization level.
Further, the optimization module optimizes network resources according to the dynamic network resource demand at each time point;
setting the dynamic resource demand at a certain time point as Y;
wherein n is the number of network processes started or ongoing at the same time at a certain point in time, XiNumber of types of network resources required for the ith process, CS, to be started or carried out at the same time at a certain point in timeiThe required amount of each network resource per unit time for the ith network process that is started or ongoing at the same time at a certain point in time.
Another object of the present invention is to provide an information data processing terminal applying the big data based network optimization scheme sharing method.
The invention has the advantages and positive effects that: according to the invention, target access request records with blank source page identifications can be screened out from a large number of access request records through the intrusion detection module, the target access request records are directly accessed to a target IP (Internet protocol) and are not obtained after other page jumps, and the behavior pattern of network intrusion is better met; then, weight values can be set for a plurality of characteristics of the target access request, the weight values can indicate the degree value of the target access request as a network intrusion request, and then a target access request record with a higher weight value can be written into a preset state table; the success rate of network intrusion detection can be improved; meanwhile, the performance evaluation module evaluates the network condition by taking the ring as a basic unit, is closer to practical application, and can avoid blind whole-ring expansion caused by single-section heavy load from the perspective of network optimization through the flow balance in the ring, thereby realizing deep excavation of network bandwidth resources and improving the utilization rate of network resources.
Drawings
Fig. 1 is a flowchart of a method for sharing a big data-based network optimization scheme according to an embodiment of the present invention.
Fig. 2 is a block diagram of a big data based network optimization scheme sharing system according to an embodiment of the present invention.
In fig. 2: 1. a data dynamic detection module; 2. a vulnerability detection module; 3. an intrusion detection module; 4. a network equipment parameter acquisition module; 5. a main control module; 6. a network communication module; 7. a data analysis module; 8. a performance evaluation module; 9. an optimization module; 10. a sharing module; 11. a big data processing module; 12. and a display module.
Detailed Description
In order to further understand the contents, features and effects of the present invention, the following embodiments are illustrated and described in detail with reference to the accompanying drawings.
The structure of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the method for sharing a big data-based network optimization scheme provided by the present invention includes the following steps:
step S101, detecting network data dynamic information by a dynamic detection program through a data dynamic detection module; detecting vulnerability information of the network system by a vulnerability detection module through a security detection program;
step S102, network intrusion information is detected by an intrusion detection program through an intrusion detection module; acquiring network equipment data by using a monitoring chip through a network equipment parameter acquisition module;
step S103, the main control module accesses the Internet by using a network card through the network communication module to carry out network communication;
step S104, analyzing the detected data by using an analysis program through a data analysis module; evaluating the network performance by a performance evaluation module by using an evaluation program;
step S105, optimizing the network by utilizing an optimization program through an optimization module; sharing the network resource by using a sharing program through a sharing module;
step S106, carrying out big data processing on the detected network data by utilizing the cloud server to centralize big data resources through a big data processing module;
and S107, displaying the detected network data dynamics, the network bugs, the network intrusion information and the collected network equipment data information by using a display through a display module.
As shown in fig. 2, the system for sharing a big data-based network optimization scheme according to an embodiment of the present invention includes: the system comprises a data dynamic detection module 1, a vulnerability detection module 2, an intrusion detection module 3, a network equipment parameter acquisition module 4, a main control module 5, a network communication module 6, a data analysis module 7, a performance evaluation module 8, an optimization module 9, a sharing module 10, a big data processing module 11 and a display module 12.
The data dynamic detection module 1 is connected with the main control module 5 and is used for detecting network data dynamic information through a dynamic detection program;
the vulnerability detection module 2 is connected with the main control module 5 and is used for detecting vulnerability information of the network system through a security detection program;
the intrusion detection module 3 is connected with the main control module 5 and is used for detecting network intrusion information through an intrusion detection program;
the network equipment parameter acquisition module 4 is connected with the main control module 5 and is used for acquiring network equipment data through the monitoring chip;
the main control module 5 is connected with the data dynamic detection module 1, the vulnerability detection module 2, the intrusion detection module 3, the network equipment parameter acquisition module 4, the network communication module 6, the data analysis module 7, the performance evaluation module 8, the optimization module 9, the sharing module 10, the big data processing module 11 and the display module 12 and is used for controlling each module to normally work through a single chip microcomputer;
the network communication module 6 is connected with the main control module 5 and is used for accessing the internet through a network card to carry out network communication;
the data analysis module 7 is connected with the main control module 5 and is used for analyzing the detected data through an analysis program;
the performance evaluation module 8 is connected with the main control module 5 and used for evaluating the network performance through an evaluation program;
the optimization module 9 is connected with the main control module 5 and is used for optimizing the network through an optimization program;
the sharing module 10 is connected with the main control module 5 and is used for sharing network resources through a sharing program;
the big data processing module 11 is connected with the main control module 5 and is used for carrying out big data processing on the detected network data by centralizing big data resources through the cloud server; the big data processing adopts an encryption function to carry out bitwise encryption on the prefix member, the port number and the protocol number generated in the last step by using the secret key t, and transmits ciphertext data and the secret key to the middleware; the middleware carries out different processing on the IP data, the port number data and the protocol data; the port and the protocol can authorize the pseudo-random function to encrypt directly according to an encryption mode; obtaining the port number corresponding to the rule and the ciphertext data Enc of the protocolK(sourceport)、Enck(destport)、Enck(protocol); encrypting the IP address according to the length agreed by the protocol; enc generating the filter ruleK(F(sourceIP))、EncK(f (destip)) a set of ciphertext IP fields; the middleware processes the rule data according to prefix length convention and an encryption function, and the processing only needs to be processed once on the premise of not replacing the rule and the convention length; the data packet adopts a double-layer bloom filter; the bloom filter adopts a bit array V with the length of m and k mutually independent Haxi functions h1、h2、…、hk(ii) a When the element s needs to be stored to the bloom filter, the setting h is calculated separately1(s)、h2(s)、…、hk(s) and setting the bit value of the corresponding position in V to be '1'; when the element u needs to be judged whether to be in the bloom filter or not, checking the h-th element in the V1(u)、h2(u)、…、hk(u) whether the bit values of the positions are all 1, if all 1, the element u is in S with a high probability, and if not all 1, u is not in the bloom filter;
and the display module 12 is connected with the main control module 5 and is used for displaying the detected network data dynamic state, network vulnerability and network intrusion information and the acquired network equipment data information through a display.
Further, the data analysis module includes:
the system comprises a dividing and sampling unit, a data processing unit and a data processing unit, wherein the dividing and sampling unit is used for dividing and investigating contents to be monitored from a database server, sampling in a layering manner, and extracting abstract data after data arrangement;
the parameter generating unit is used for generating a scatter diagram and historical data of data indexes to be monitored and associated data indexes according to the abstract data and selecting monitoring parameters;
the trend graph drawing unit is used for drawing a trend graph of the network data through a computer according to the generated monitoring parameters;
and the summarizing unit is used for summarizing the fluctuation condition of the network transmission data according to the drawn trend graph.
Further, the optimization module optimizes network resources mainly according to the dynamic network resource demand at each time point;
if the dynamic resource requirement at a certain time point is Y, then
Wherein n is the number of network processes started or ongoing at the same time at a certain point in time, XiNumber of types of network resources required for the ith process, CS, to be started or carried out at the same time at a certain point in timeiThe required amount of each network resource per unit time for the ith network process that is started or ongoing at the same time at a certain point in time.
The detection method of the intrusion detection module 3 provided by the invention comprises the following steps:
(1) acquiring access request records from a preset number of source IPs to a target IP through an intrusion detection program;
(2) counting a target access request record with a request source page identifier being empty in the access request records;
(3) respectively setting weight values for a plurality of characteristics of the target access request record, and writing the target access request record into a preset state table based on the set weight values;
(4) and matching the written target access request records in the preset state table, and taking the access request records in the matching result as network intrusion records.
The access request record for obtaining the preset number of source IPs to the target IP comprises the following steps:
acquiring a preset number of access request records from real source IP to target IP based on a determination rule of the real source IP;
the determination rule of the real source IP specifically includes:
when an x-forward-for field of the access request record is empty, taking an IP address in a source IP field as a real source IP;
and when the x-forward-for field of the access request record is non-empty, determining the real source IP according to the IP address in the x-forward-for field.
The step of setting the weight values for the plurality of characteristics of the target access request record respectively includes:
and determining a classification error rate corresponding to a target feature of the target access request record, and determining a weight value of the target feature based on the classification error rate.
The performance evaluation module 8 provided by the invention has the following evaluation method:
1) acquiring the bandwidth utilization rate of each ring internal link of the current ring network through an evaluation program; determining the average value of the intra-loop bandwidth utilization rate of the current ring network according to the bandwidth utilization rate of the internal link of each ring; determining the intra-loop link load balance degree according to the intra-loop bandwidth utilization rate average value of the current ring network and the bandwidth utilization rate of each intra-loop internal link;
2) determining the maximum value of the bandwidth utilization rate of each loop internal link as the highest bandwidth utilization rate of the loop internal link according to the bandwidth utilization rate of each loop internal link; determining the network performance of the current ring network according to the load balance degree of the intra-ring link and the highest bandwidth utilization rate of the intra-ring link;
3) acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
4) determining the busyness of each ring internal link according to the sampling times of the super-early warning value and the total sampling times of each ring internal link, and determining the maximum value in the busyness of each ring internal link as the link busyness;
5) the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the link busyness degree;
the evaluation method provided by the invention further comprises the following steps:
acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
determining the busyness of the current ring network according to the sampling times of the super-early warning value and the total sampling times of the internal links of each ring;
and the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the busyness degree of the current ring network.
The method for determining the network performance of the current ring network provided by the invention comprises the following steps:
comparing the intra-loop link load balance degree with a preset balance degree threshold value, and determining an intra-loop link balance level corresponding to the intra-loop link load balance degree;
comparing the highest bandwidth utilization rate of the intra-loop link with a preset bandwidth utilization rate threshold value, and determining a highest bandwidth utilization rate level corresponding to the highest bandwidth utilization rate of the intra-loop link;
and determining the network performance of the current looped network according to the pairing relationship between the intra-loop link balancing level and the highest bandwidth utilization level.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent changes and modifications made to the above embodiment according to the technical spirit of the present invention are within the scope of the technical solution of the present invention.
Claims (10)
1. A big data based network optimization scheme sharing system, comprising:
the data dynamic detection module is connected with the main control module and is used for detecting the network data dynamic information through a dynamic detection program;
the vulnerability detection module is connected with the main control module and used for detecting vulnerability information of the network system through a security detection program;
the intrusion detection module is connected with the main control module and used for detecting network intrusion information through an intrusion detection program;
the network equipment parameter acquisition module is connected with the main control module and is used for acquiring network equipment data through the monitoring chip;
the main control module is connected with the data dynamic detection module, the vulnerability detection module, the intrusion detection module, the network equipment parameter acquisition module, the network communication module, the data analysis module, the performance evaluation module, the optimization module, the sharing module, the big data processing module and the display module and is used for controlling each module to normally work through the single chip microcomputer;
the network communication module is connected with the main control module and is used for accessing the internet through a network card to carry out network communication;
the data analysis module is connected with the main control module and used for analyzing the detected data through an analysis program;
the performance evaluation module is connected with the main control module and used for evaluating the network performance through an evaluation program;
the optimization module is connected with the main control module and used for optimizing the network through an optimization program;
the sharing module is connected with the main control module and is used for sharing the network resources through a sharing program;
the big data processing module is connected with the main control module and used for carrying out big data processing on the detected network data by centralizing big data resources through the cloud server;
and the display module is connected with the main control module and used for displaying the detected network data dynamic state, the network loophole, the network intrusion information and the acquired network equipment data information through the display.
2. The big-data based network optimization scheme sharing system of claim 1, wherein the data analysis module comprises:
the system comprises a dividing and sampling unit, a data processing unit and a data processing unit, wherein the dividing and sampling unit is used for dividing and investigating contents to be monitored from a database server, sampling in a layering manner, and extracting abstract data after data arrangement;
the parameter generating unit is used for generating a scatter diagram and historical data of data indexes to be monitored and associated data indexes according to the abstract data and selecting monitoring parameters;
the trend graph drawing unit is used for drawing a trend graph of the network data through a computer according to the generated monitoring parameters;
and the summarizing unit is used for summarizing the fluctuation condition of the network transmission data according to the drawn trend graph.
3. A big data based network optimization scheme sharing method for executing the big data based network optimization scheme sharing system of claim 1, wherein the big data based network optimization scheme sharing method comprises the following steps:
firstly, detecting network data dynamic information by using a dynamic detection program through a data dynamic detection module; detecting vulnerability information of the network system by a vulnerability detection module through a security detection program;
secondly, network intrusion information is detected by an intrusion detection program through an intrusion detection module; acquiring network equipment data by using a monitoring chip through a network equipment parameter acquisition module;
thirdly, the main control module accesses the internet by using the network card through the network communication module to carry out network communication;
analyzing the detected data by using an analysis program through a data analysis module; evaluating the network performance by a performance evaluation module by using an evaluation program;
fifthly, optimizing the network by utilizing an optimization program through an optimization module; sharing the network resource by using a sharing program through a sharing module;
step six, the big data processing module is used for carrying out big data processing on the detected network data by utilizing the cloud server to centralize big data resources; the big data processing adopts an encryption function to carry out bitwise encryption on the prefix member, the port number and the protocol number generated in the last step by using the secret key t, and transmits ciphertext data and the secret key to the middleware; the middleware carries out different processing on the IP data, the port number data and the protocol data; the port and the protocol can authorize the pseudo-random function to encrypt directly according to an encryption mode; obtaining the port number corresponding to the rule and the ciphertext data Enc of the protocolK(sourceport)、Enck(destport)、Enck(protocol); encrypting the IP address according to the length agreed by the protocol; enc generating the filter ruleK(F(sourceIP))、EncK(f (destip)) a set of ciphertext IP fields; the middleware processes the rule data according to prefix length convention and an encryption function, and the processing only needs to be processed once on the premise of not replacing the rule and the convention length; the data packet adopts a double-layer bloom filter; the bloom filter adopts a bit array V with the length of m and k mutually independent Haxi functions h1、h2、…、hk(ii) a When the element s needs to be stored to the bloom filter, the setting h is calculated separately1(s)、h2(s)、…、hk(s) and setting the bit value of the corresponding position in V to be '1'; when it is necessary to determine whether element u is in the bloom filter, check VH th1(u)、h2(u)、…、hk(u) whether the bit values of the positions are all 1, if all 1, the element u is in S with a high probability, and if not all 1, u is not in the bloom filter;
and seventhly, displaying the detected network data dynamic state, the network loophole, the network intrusion information and the acquired network equipment data information by using a display through a display module.
4. The big data-based network optimization scheme sharing method according to claim 3, wherein the intrusion detection module detects the network intrusion information as follows:
(1) acquiring access request records from a preset number of source IPs to a target IP through an intrusion detection program;
(2) counting a target access request record with a request source page identifier being empty in the access request records;
(3) respectively setting weight values for a plurality of characteristics of the target access request record, and writing the target access request record into a preset state table based on the set weight values;
(4) and matching the written target access request records in the preset state table, and taking the access request records in the matching result as network intrusion records.
5. The big data based network optimization scheme sharing method according to claim 4, wherein the obtaining of the access request records from the preset number of source IPs to the target IPs comprises:
acquiring a preset number of access request records from real source IP to target IP based on a determination rule of the real source IP;
the determination rule of the real source IP specifically includes:
when an x-forward-for field of the access request record is empty, taking an IP address in a source IP field as a real source IP;
and when the x-forward-for field of the access request record is non-empty, determining the real source IP according to the IP address in the x-forward-for field.
6. The big data based network optimization scheme sharing method of claim 4, wherein the setting of the weight values for the plurality of characteristics of the target access request record respectively comprises:
and determining a classification error rate corresponding to a target feature of the target access request record, and determining a weight value of the target feature based on the classification error rate.
7. The big data based network optimization scheme sharing system of claim 3, wherein the method for evaluating the network performance by the evaluation program comprises:
1) acquiring the bandwidth utilization rate of each ring internal link of the current ring network through an evaluation program; determining the average value of the intra-loop bandwidth utilization rate of the current ring network according to the bandwidth utilization rate of the internal link of each ring; determining the intra-loop link load balance degree according to the intra-loop bandwidth utilization rate average value of the current ring network and the bandwidth utilization rate of each intra-loop internal link;
2) determining the maximum value of the bandwidth utilization rate of each loop internal link as the highest bandwidth utilization rate of the loop internal link according to the bandwidth utilization rate of each loop internal link; determining the network performance of the current ring network according to the load balance degree of the intra-ring link and the highest bandwidth utilization rate of the intra-ring link;
3) acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
4) determining the busyness of each ring internal link according to the sampling times of the super-early warning value and the total sampling times of each ring internal link, and determining the maximum value in the busyness of each ring internal link as the link busyness;
5) and the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the link busyness degree.
8. The big data based network optimization scheme sharing method of claim 7, wherein the evaluating the network performance using the evaluation program further comprises the steps of:
acquiring the total sampling times of flow sampling of each ring internal link in the current ring network within a preset time period, and the sampling times of the super-early warning value when the acquired flow exceeds the flow limit value of each ring internal link when the flow sampling is carried out on each ring internal link;
determining the busyness of the current ring network according to the sampling times of the super-early warning value and the total sampling times of the internal links of each ring;
the network performance of the current ring network is determined according to the load balance degree of the intra-ring link, the highest bandwidth utilization rate of the intra-ring link and the busyness degree of the current ring network;
the method for determining the network performance of the current ring network comprises the following steps:
comparing the intra-loop link load balance degree with a preset balance degree threshold value, and determining an intra-loop link balance level corresponding to the intra-loop link load balance degree;
comparing the highest bandwidth utilization rate of the intra-loop link with a preset bandwidth utilization rate threshold value, and determining a highest bandwidth utilization rate level corresponding to the highest bandwidth utilization rate of the intra-loop link;
and determining the network performance of the current looped network according to the pairing relationship between the intra-loop link balancing level and the highest bandwidth utilization level.
9. The big data based network optimization scheme sharing method according to claim 3, wherein the optimization module performs network resource optimization according to the dynamic network resource demand at each time point;
setting the dynamic resource demand at a certain time point as Y;
wherein n is a certainNumber of network processes, X, started or ongoing at the same time in timeiNumber of types of network resources required for the ith process, CS, to be started or carried out at the same time at a certain point in timeiThe required amount of each network resource per unit time for the ith network process that is started or ongoing at the same time at a certain point in time.
10. An information data processing terminal applying the big data based network optimization scheme sharing method of any one of claims 3 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964870.3A CN110719286A (en) | 2019-10-11 | 2019-10-11 | Network optimization scheme sharing system and method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964870.3A CN110719286A (en) | 2019-10-11 | 2019-10-11 | Network optimization scheme sharing system and method based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110719286A true CN110719286A (en) | 2020-01-21 |
Family
ID=69211443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910964870.3A Pending CN110719286A (en) | 2019-10-11 | 2019-10-11 | Network optimization scheme sharing system and method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110719286A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113450078A (en) * | 2021-07-07 | 2021-09-28 | 国网山西省电力公司信息通信分公司 | Processing system of communication big data |
| CN113783740A (en) * | 2021-10-26 | 2021-12-10 | 北京字节跳动网络技术有限公司 | Network optimization method, device, equipment and medium |
| CN114124376A (en) * | 2021-11-23 | 2022-03-01 | 中国标准化研究院 | Data processing method and system based on network data acquisition |
| CN114978629A (en) * | 2022-05-12 | 2022-08-30 | 北京神州慧安科技有限公司 | Safety monitoring, early warning and emergency disposal system based on industrial internet |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105703994A (en) * | 2016-04-29 | 2016-06-22 | 广东省电信规划设计院有限公司 | Network performance assessment method and system |
| CN107046548A (en) * | 2017-05-22 | 2017-08-15 | 东莞理工学院 | A kind of packet filtering method under secret protection |
| CN108596436A (en) * | 2018-03-28 | 2018-09-28 | 郑州铁路职业技术学院 | Computer based economic indicator monitors analysis method in real time |
| CN108650274A (en) * | 2018-05-21 | 2018-10-12 | 中国科学院计算机网络信息中心 | A kind of network inbreak detection method and system |
-
2019
- 2019-10-11 CN CN201910964870.3A patent/CN110719286A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105703994A (en) * | 2016-04-29 | 2016-06-22 | 广东省电信规划设计院有限公司 | Network performance assessment method and system |
| CN107046548A (en) * | 2017-05-22 | 2017-08-15 | 东莞理工学院 | A kind of packet filtering method under secret protection |
| CN108596436A (en) * | 2018-03-28 | 2018-09-28 | 郑州铁路职业技术学院 | Computer based economic indicator monitors analysis method in real time |
| CN108650274A (en) * | 2018-05-21 | 2018-10-12 | 中国科学院计算机网络信息中心 | A kind of network inbreak detection method and system |
Non-Patent Citations (4)
| Title |
|---|
| 张子林等: "网络优化模型及程序设计", 《工业技术经济》 * |
| 李煜民等: "从网络资源共享到网络服务", 《电脑知识与技术》 * |
| 毕强等: "实现网络资源共享及其技术研究", 《图书馆论坛》 * |
| 陈琳等: "一种面向大规模***域网络性能管理***", 《计算机工程与科学》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113450078A (en) * | 2021-07-07 | 2021-09-28 | 国网山西省电力公司信息通信分公司 | Processing system of communication big data |
| CN113783740A (en) * | 2021-10-26 | 2021-12-10 | 北京字节跳动网络技术有限公司 | Network optimization method, device, equipment and medium |
| CN113783740B (en) * | 2021-10-26 | 2022-08-16 | 北京字节跳动网络技术有限公司 | Network optimization method, device, equipment and medium |
| CN114124376A (en) * | 2021-11-23 | 2022-03-01 | 中国标准化研究院 | Data processing method and system based on network data acquisition |
| CN114978629A (en) * | 2022-05-12 | 2022-08-30 | 北京神州慧安科技有限公司 | Safety monitoring, early warning and emergency disposal system based on industrial internet |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11316878B2 (en) | System and method for malware detection | |
| US20200344246A1 (en) | Apparatus, system and method for identifying and mitigating malicious network threats | |
| US9860154B2 (en) | Streaming method and system for processing network metadata | |
| CN110719286A (en) | Network optimization scheme sharing system and method based on big data | |
| US9386028B2 (en) | System and method for malware detection using multidimensional feature clustering | |
| US8955091B2 (en) | Systems and methods for integrating cloud services with information management systems | |
| US8438639B2 (en) | Apparatus for detecting and filtering application layer DDoS attack of web service | |
| CN103179132B (en) | A kind of method and device detecting and defend CC attack | |
| CN113079143A (en) | Flow data-based anomaly detection method and system | |
| US11184387B2 (en) | Network attack defense system and method | |
| CN108809749B (en) | Performing upper layer inspection of a stream based on a sampling rate | |
| US7903657B2 (en) | Method for classifying applications and detecting network abnormality by statistical information of packets and apparatus therefor | |
| CN107623685B (en) | Method and device for rapidly detecting SYN Flood attack | |
| EP2767056A1 (en) | A method and a system to detect malicious software | |
| KR20140027616A (en) | Apparatus and method for detecting http botnet based on the density of web transaction | |
| US20140259140A1 (en) | Using learned flow reputation as a heuristic to control deep packet inspection under load | |
| WO2014110293A1 (en) | An improved streaming method and system for processing network metadata | |
| Fenil et al. | Towards a secure software defined network with adaptive mitigation of dDoS attacks by machine learning approaches | |
| CN115664833B (en) | Network hijacking detection method based on local area network safety equipment | |
| Nakahara et al. | Malware Detection for IoT Devices using Automatically Generated White List and Isolation Forest. | |
| CN109922083A (en) | A kind of network protocol flow control system | |
| CN112287252B (en) | Method, device, equipment and storage medium for detecting website domain name hijacking | |
| CN114584356A (en) | Network security monitoring method and network security monitoring system | |
| KR100728446B1 (en) | Hardware based intruding protection device, system and method | |
| Boonyopakorn | Applying Data Analytics to Findings of User Behaviour Usage in Network Systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200121 |
|
| RJ01 | Rejection of invention patent application after publication |