CN115622720A - Network anomaly detection method and device and detection equipment - Google Patents
Network anomaly detection method and device and detection equipment Download PDFInfo
- Publication number
- CN115622720A CN115622720A CN202110789429.3A CN202110789429A CN115622720A CN 115622720 A CN115622720 A CN 115622720A CN 202110789429 A CN202110789429 A CN 202110789429A CN 115622720 A CN115622720 A CN 115622720A
- Authority
- CN
- China
- Prior art keywords
- network
- abnormal
- traffic data
- characteristic parameters
- anomaly detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 194
- 230000002159 abnormal effect Effects 0.000 claims abstract description 133
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000004458 analytical method Methods 0.000 claims description 51
- 238000000034 method Methods 0.000 claims description 32
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 238000007621 cluster analysis Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 11
- 241000700605 Viruses Species 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 5
- 230000005856 abnormality Effects 0.000 abstract description 10
- 238000004891 communication Methods 0.000 abstract description 2
- 239000000284 extract Substances 0.000 abstract 1
- 238000005553 drilling Methods 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 3
- 238000003064 k means clustering Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 102100024607 DNA topoisomerase 1 Human genes 0.000 description 1
- 101000830681 Homo sapiens DNA topoisomerase 1 Proteins 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/145—Countermeasures against malicious traffic the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/1458—Denial of Service
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Virology (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a network anomaly detection method, a device and a detection device, belonging to the technical field of wireless communication, wherein the network anomaly detection method comprises the following steps: preprocessing network flow data of an access layer to obtain characteristic parameters of the network flow data; detecting the characteristic parameters by adopting a network anomaly detection model in the access layer to obtain a detection result, wherein the detection result is abnormal or normal; and analyzing the characteristic parameters with abnormal detection results to determine the network abnormal type of the network traffic data. The embodiment of the invention extracts the network flow data at the access layer for preprocessing, and automatically identifies the abnormality by adopting the network abnormality detection model, so that the network abnormality type can be quickly and accurately identified, and the detection efficiency and the real-time performance are effectively improved.
Description
Technical Field
The present invention relates to the field of network anomaly detection technologies, and in particular, to a network anomaly detection method, device and detection apparatus.
Background
In recent years, with the development of network technology, computer networks have become one of the most important infrastructures for human society, are widely and deeply influencing the economic and social activities of people, and are becoming an indispensable part of daily life of people. However, the increasing size of networks has caused the possibility of failures in the networks to increase greatly, causing more and more serious safety problems and economic losses. Therefore, the method has important practical significance for accurately and quickly detecting the abnormality in the network.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus and a device for detecting network anomalies, which are used to solve the problem that anomalies occurring in the current network cannot be detected accurately in time.
In order to solve the above technical problem, in a first aspect, the present invention provides a method for detecting a network anomaly, where the method includes:
preprocessing network traffic data of an access layer to obtain characteristic parameters of the network traffic data;
detecting the characteristic parameters by adopting a network anomaly detection model at the access layer to obtain a detection result, wherein the detection result is abnormal or normal;
and analyzing the characteristic parameters with abnormal detection results to determine the network abnormal type of the network traffic data.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm.
Optionally, the detecting the characteristic parameter by using the network anomaly detection model to obtain a detection result further includes:
and updating the network anomaly detection model by adopting the characteristic parameters with normal detection results at the edge layer, and issuing the updated network anomaly detection model to the access layer.
Optionally, the characteristic parameter includes at least one of: the information entropy of the source IP address, the information entropy of the destination port, the information entropy of the network flow duration, the information entropy of the data packet size, and the number ratio of syn packets to ack packets.
Optionally, the analyzing the characteristic parameter of which the detection result is abnormal, and determining the network abnormal type of the network traffic data includes at least one of the following:
if the information entropy of the source IP address is increased, determining that the network abnormal type of the network flow data is distributed denial of service attack;
if the information entropy of the destination port is increased, determining that the network abnormal type of the network traffic data is port scanning;
and if the information entropy of the destination IP address is increased, determining that the network abnormal type of the network traffic data is a worm virus.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one class cluster, and before analyzing the characteristic parameter of which the detection result is abnormal and determining the network anomaly type of the network traffic data, the method further includes:
calculating a target parameter of each class of clusters in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the class of clusters plus three times of the standard deviation of all classified characteristic parameters in the class of clusters;
and if the characteristic parameter of the abnormal detection result is larger than the target parameter of the corresponding class cluster, determining that the characteristic parameter of the abnormal detection result is increased.
Optionally, after analyzing the characteristic parameter of which the detection result is abnormal and determining the network abnormal type of the network traffic data, the method further includes:
performing drill-down analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drill-down analysis results;
and verifying the network abnormal type of the determined network traffic data according to the drilling analysis result, and acquiring the abnormal information of the abnormal network traffic data.
In a second aspect, the present invention further provides a network anomaly detection apparatus, including:
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing network traffic data of an access layer to acquire characteristic parameters of the network traffic data;
the detection module is used for detecting the characteristic parameters by adopting a network anomaly detection model at the access layer to obtain a detection result, and the detection result is abnormal or normal;
and the analysis module is used for analyzing the characteristic parameters of which the detection results are abnormal and determining the network abnormal type of the network flow data.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm.
Optionally, the apparatus further comprises:
and the updating module is used for updating the network abnormity detection model by adopting the characteristic parameters with normal detection results at the edge layer and sending the updated network abnormity detection model to the access layer.
Optionally, the characteristic parameter includes at least one of: the information entropy of the source IP address, the information entropy of the destination port, the information entropy of the network flow duration, the information entropy of the data packet size, and the number ratio of syn packets to ack packets.
Optionally, the analysis module comprises at least one of:
the first analysis unit is used for determining that the network abnormal type of the network traffic data is distributed denial of service attack if the information entropy of the source IP address is increased;
the second analysis unit is used for determining that the network abnormal type of the network traffic data is port scanning if the information entropy of the destination port is increased;
and the third analysis unit is used for determining that the network abnormal type of the network traffic data is a worm virus if the information entropy of the destination IP address is increased.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one class cluster, and the apparatus further includes:
the calculation module is used for calculating a target parameter of each class of clusters in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the class of clusters plus three times of standard deviation of all classified characteristic parameters in the class of clusters;
and the determining module is used for determining that the characteristic parameter of the abnormal detection result is increased if the characteristic parameter of the abnormal detection result is larger than the target parameter of the corresponding cluster.
Optionally, the apparatus further comprises:
the drilling analysis module is used for performing drilling analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drilling analysis results;
and the verification module is used for verifying the network abnormal type of the determined network traffic data according to the drilling analysis result and acquiring the abnormal information of the abnormal network traffic data.
In a third aspect, the present invention also provides a detection apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor; the processor implements any of the above network anomaly detection methods when executing the computer program.
In a fourth aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of any one of the above-mentioned network anomaly detection methods.
The technical scheme of the invention has the following beneficial effects:
in the embodiment of the invention, the network flow data is extracted from the access layer for preprocessing, and the network anomaly detection model is adopted to automatically identify the anomaly, so that the network anomaly type can be quickly and accurately identified, and the detection efficiency and the real-time performance are effectively improved.
Drawings
Fig. 1 is a schematic flowchart of a network anomaly detection method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an exception determining process according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a drill-down analysis according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a network anomaly detection apparatus according to a second embodiment of the present invention;
fig. 5 is a schematic structural diagram of a detection apparatus according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the embodiments of the present invention. It should be apparent that the described embodiments are only some of the embodiments of the present invention, and not all of them. All other embodiments, which can be derived by a person skilled in the art from the description of the embodiments of the invention given above, are within the scope of protection of the invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a network anomaly detection method according to an embodiment of the present invention, where the method includes the following steps:
step 11: preprocessing network traffic data of an access layer to obtain characteristic parameters of the network traffic data.
In this step, the acquisition of network traffic data is directly performed in the access layer, and specifically, the acquisition may be performed on an important link or node of the network. For the acquired original network Flow data, the acquired original network Flow data can be aggregated into a data structure of a Flow format based on a NetFlow five-tuple (a source posture, a target address, a source port, a target port and a transport layer protocol), and for different network exception types, corresponding characteristic fields can be selected from the aggregated data of the Flow format, and information entropies of different characteristics are calculated to perform detection and analysis. After the network traffic data is preprocessed, the characteristic parameters of the network traffic data can be obtained, and the characteristic parameters reflect different characteristics of the corresponding network traffic data.
The access stratum is the part of the network directly facing the user connection or access. The access layer uses transmission media such as optical fiber, twisted pair, coaxial cable, wireless access technology, etc. to realize connection with users, and performs service and bandwidth allocation, allowing end users to connect to the network.
Step 12: and detecting the characteristic parameters by adopting a network anomaly detection model at the access layer to obtain a detection result, wherein the detection result is abnormal or normal.
After the characteristic parameters of the network traffic data are obtained, a network anomaly detection model is directly adopted to detect the characteristic parameters at an access layer to obtain detection results, the detection results are divided into two types, namely anomaly or normal, if the detection results of the characteristic parameters are abnormal, the network traffic data corresponding to the characteristic parameters are abnormal, and if the detection results of the characteristic parameters are normal, the network traffic data corresponding to the characteristic parameters are normal. Through directly carrying out anomaly detection on the access layer, namely directly carrying out anomaly detection on the data acquisition side, compared with a cloud platform detection scheme adopting a cloud computing framework, the method can greatly improve the detection efficiency and the real-time property.
The network anomaly detection model can be established in advance and trained, so that model parameters of the network anomaly detection model meet corresponding requirements, and characteristic parameters can be accurately detected. Optionally, the network anomaly detection model may be constructed in an Edge layer, where the Edge layer uses a Mobile Edge Computing (MEC) architecture, and the network anomaly detection model is established in the Edge layer and then sent to each data acquisition point of the access layer.
Step 13: analyzing the characteristic parameters with abnormal detection results to determine the network abnormal type of the network flow data
After the network anomaly detection model is used for detecting the abnormal characteristic parameters, only whether the network traffic data is abnormal or normal can be judged at the moment, but the specific network anomaly type under the abnormal condition cannot be judged. Due to the fact that the change conditions of the characteristic parameters caused by different network abnormal types are different, the abnormal characteristic parameters can be further analyzed to determine the network abnormal type of the abnormal network traffic data, so that more accurate abnormal information can be obtained, and users can be informed conveniently or corresponding countermeasures can be taken conveniently.
Therefore, in the embodiment of the invention, the network flow data is extracted from the access layer for preprocessing, and the network anomaly detection model is adopted to automatically identify the anomaly, so that the network anomaly type can be quickly and accurately identified, and the detection efficiency and the real-time performance are effectively improved.
The above network anomaly detection method is exemplified below.
In some embodiments of the invention, the characteristic parameter comprises at least one of: information entropy H (src _ IP) of a source IP address, information entropy H (dst _ IP) of a destination IP address, information entropy H (dst _ port) of a destination port, information entropy H (duration) of network flow duration, information entropy H (pktsize) of a packet size, and a number ratio of syn packets to ack packets n _ syn/n _ ack.
Wherein the H function represents an information entropy function; the NetWork Flow, namely NetWork Flow; the syn, i.e., synchronize Sequence Numbers, is numbered; the ack, an Acknowledge character. The number ratio of the syn packet to the ack packet can describe the half-connection state of a session in a network or a link, a large number of false source IPs cause a large number of half-connection sessions, and by analyzing a three-way handshake mechanism of a TCP (Transmission Control Protocol), it can be known that the large number of half-connection states can cause the proportion of the syn packet in a TCP FLAG field to increase, and the proportion of the ack packet required in the data Transmission process to decrease.
It can be known that, when the characteristic parameters include the information entropy H (src _ IP) of the source IP address, the information entropy H (dst _ IP) of the destination IP address, the information entropy H (dst _ port) of the destination port, the information entropy H (duration) of the network flow duration, the information entropy H (pktsize) of the size of the data packet, and the number of syn packets and ack packets is greater than the number of items in n _ syn/n _ ack, the anomaly detection result of the subsequent network anomaly detection model on the characteristic parameters is more accurate. Illustratively, when an anomaly occurs, the expression of three characteristic parameters, namely the information entropy H (duration) of the network flow duration, the information entropy H (pktsize) of the data packet size and the number ratio of the syn packet to the ack packet n _ syn/n _ ack, is different from that of normal flow, so that the expression can be used as a basis for judging whether the network flow data is abnormal or not; when the concentrated abnormality occurs, the duration of the attack packet is generally short, H (duration) is reduced, when the concentrated abnormality occurs, the size of the attack packet is generally small, H (pktsize) is reduced, and when the concentrated abnormality occurs, n _ syn/n _ ack is increased (only for DDoS).
In the embodiment of the present invention, in the characteristic parameters, each information entropy may be calculated by using the following formula:
wherein x represents characteristics, the characteristics are source IP address, destination port, network flow duration, data packet size, and the like, S is the total number of data packets, N is the number of different values appearing in characteristic x, and N is the number of different values appearing in characteristic x i The number of occurrences when the feature x is a certain value.
In an optional specific embodiment, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm; the so-called K-means clustering algorithm (K-means clustering algorithm) is a clustering algorithm for iterative solution.
In the embodiment of the present invention, a network anomaly detection model may be established and optimized on the MEC platform of the edge layer based on a cluster analysis algorithm, wherein a training set for training the network anomaly detection model may use characteristic parameters obtained by preprocessing of the access layer, and at this time, original network traffic data needs to be uploaded to the MEC platform in real time after being preprocessed, and of course, the training set may also use an open source data set, for example.
The following description will be made by taking the characteristic parameters obtained after preprocessing of the access stratum as a training set to establish the network anomaly detection model.
Assuming that a training set is divided into K clusters after being clustered by a K-means cluster analysis algorithm, and recording the following parameters for each cluster for anomaly detection and judgment:
(1) Cluster center C:
(2) Cluster radius r: r is the maximum distance from the cluster center to the cluster sample;
(3) Mean and standard deviation of the characteristic parameters.
And clustering and dividing the data when the network is normal through the K-means clustering analysis algorithm to obtain a network anomaly detection model (clustering model) of the normal network flow. When the abnormal traffic of the network arrives, the network traffic data will change to some extent, and any sample which does not belong to any cluster is considered as abnormal for the obtained network abnormal detection model of the normal traffic of the network, so that the obtained network abnormal detection model can identify whether the network traffic data is normal or abnormal.
Fig. 2 is a schematic diagram of an anomaly determination process according to an embodiment of the present invention. As shown in fig. 2, in some embodiments of the present invention, when the access stratum uses a network anomaly detection model to detect the characteristic parameters, the method specifically includes the following steps:
step 21: for the network flow data in unit time, the characteristic parameters are obtained through the preprocessing step, and a characteristic representation sample X to be detected is formed;
step 22: calculating sample X and center C of each cluster i Distance d of i Selecting the minimum distance d t ＝min{d i Assign the sample X to the class;
step 23: comparison d i Radius r of the cluster t If d is i >r t If the detection result of the characteristic parameter is abnormal, namely the corresponding network flow data is abnormal flow, and then further judging the network abnormal type; if d is i ≤r t If the detection result of the characteristic parameter is normal, that is, the corresponding network traffic data is normal traffic, ending this timeAnd detecting, reporting the characteristic parameters and corresponding network flow data to the MEC platform, and updating the network anomaly detection model by using the new characteristic parameters when the number of new network normal data reaches a certain number.
In some embodiments of the present invention, after the detecting the characteristic parameter by using the network anomaly detection model and obtaining the detection result, the method further includes:
and updating the network anomaly detection model by adopting the characteristic parameters with normal detection results at the edge layer, and issuing the updated network anomaly detection model to the access layer.
That is to say, if the detection result obtained after the network anomaly detection model is adopted for detection is normal, the normal characteristic parameters can be uploaded to the MEC platform of the edge layer, the MEC platform updates the network anomaly detection model by using the received characteristic parameters, and then issues the updated network anomaly detection model to each node of the access layer, so that the network anomaly detection model is continuously optimized, and the detection accuracy is improved. When the MEC platform updates the network anomaly detection model by using the received characteristic parameters, the MEC platform can update the network anomaly detection model after the received characteristic parameters reach a certain number.
In some embodiments of the present invention, the analyzing the characteristic parameter of which the detection result is abnormal, and determining the network abnormal type of the network traffic data includes at least one of:
if the information entropy of the source IP address is increased, determining that the network abnormal type of the network flow data is distributed denial of service attack;
if the information entropy of the destination port is increased, determining that the network abnormal type of the network traffic data is port scanning;
and if the information entropy of the destination IP address is increased, determining that the network abnormal type of the network flow data is a worm virus.
Referring to table 1 below, table 1 shows the main characteristics of the network anomaly type.
As can be seen from the content in table 1 above, the main performance characteristics of different network anomaly types are different, specifically, the characteristics are different for the destination IP address, the source IP address, the destination port, and the like.
Referring to table 2 below, table 2 shows the principle of determining the type of network anomaly.
| DDoS | Port scanning | Worm virus | |
| H(src_ip) | ↑ | ↓ | ↓ |
| H(dst_ip) | ↓ | ↓ | ↑ |
| H(dst_port) | ↓ | ↑ | ↓ |
As can be seen from the above table 2, only DDoS (Distributed denial of service attack) increases the information entropy H (src _ IP) of the source IP address, only port scanning increases the information entropy H (dst _ port) of the destination port, and only worm virus increases the information entropy H (dst _ IP) of the destination IP address.
Therefore, when analyzing the characteristic parameters of which the detection result is abnormal and determining the network abnormal type of the network traffic data: if the information entropy of the source IP address is increased, determining that the network abnormal type of the network traffic data is distributed denial of service attack; if the information entropy of the destination port is increased, determining that the network abnormal type of the network traffic data is port scanning; if the information entropy of the destination IP address is increased, it may be determined that the network anomaly type of the network traffic data is a worm virus. Therefore, the embodiment of the invention can specifically acquire the specific type of the network abnormity by further analyzing the characteristic parameters of the abnormity, and is convenient for informing a user or taking corresponding countermeasures.
In some embodiments of the present invention, optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one cluster, and before analyzing the characteristic parameter of which the detection result is an anomaly and determining the network anomaly type of the network traffic data, the method further includes:
calculating a target parameter of each class of clusters in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the class of clusters plus three times of the standard deviation of all classified characteristic parameters in the class of clusters;
and if the characteristic parameter of the abnormal detection result is larger than the target parameter of the corresponding cluster, determining that the characteristic parameter of the abnormal detection result is increased.
That is, it is necessary to set a reference for comparison so as to determine whether the characteristic parameter of the abnormality is increased or decreased, such as an increase in the entropy of the source IP address, an increase in the entropy of the destination port, and an increase in the entropy of the destination IP address. Specifically, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, so that the network anomaly detection model comprises at least one class cluster, one class cluster corresponds to one class of characteristic parameters, and the target parameters are set as comparison references by calculating the target parameters of each class cluster in the network anomaly detection model, so that whether the abnormal characteristic parameters are increased or decreased can be determined. The target parameter is equal to the average value of all classified characteristic parameters in the cluster plus three times of standard deviation of all classified characteristic parameters in the cluster, taking the information entropy of the source IP address as an example, the target parameter of the cluster corresponding to the information entropy of the source IP address is equal to the average value of all classified characteristic parameters in the cluster corresponding to the information entropy of the source IP address plus three times of standard deviation of all classified characteristic parameters in the cluster corresponding to the information entropy of the source IP address. By the method, whether the abnormal characteristic parameter is increased or decreased can be accurately known, so that a judgment basis is provided for subsequently determining the network abnormal type of the network traffic data corresponding to the abnormal characteristic parameter, and the accuracy of judging the network abnormal type is improved.
In other embodiments of the present invention, after analyzing the characteristic parameter whose detection result is abnormal and determining a network abnormality type of the network traffic data, the method further includes:
performing drill-down analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drill-down analysis results;
and verifying the network abnormal type of the determined network traffic data according to the drilling analysis result, and acquiring the abnormal information of the abnormal network traffic data.
That is, after the network anomaly type of the network traffic data corresponding to the abnormal characteristic parameter is determined, drill-down analysis (drill-down analysis) may be performed on the abnormal network traffic data to obtain a drill-down analysis result, so that whether the network anomaly type determined in the foregoing step is correct or not is verified by using the drill-down analysis result, and specific anomaly information of the abnormal network traffic data may be further obtained, which is beneficial to analysis and processing of the abnormal network traffic data.
Among them, the so-called drill-down analysis, which is an analysis operation in a data warehouse, falls down along the hierarchy of a specific attribute dimension to obtain more detailed data.
Optionally, the process of performing drill-down analysis may be performed on a cloud platform. For example, the MEC platform at the edge layer may upload the obtained abnormal feature parameters and the network traffic data corresponding to the abnormal feature parameters to the cloud platform, and the cloud platform performs drill-down analysis based on the offline network traffic data.
Fig. 3 is a schematic flow chart of drill-down analysis according to an embodiment of the present invention. As shown in fig. 3, for example, when performing a drill-down analysis on the network traffic data corresponding to the characteristic parameter whose detection result is abnormal, the drill-down analysis may be performed on the network traffic data corresponding to a certain time period in which the abnormality is detected, and the specific steps may include the following steps:
(1) Protocol analysis: and counting the protocol distribution conditions of the application layer and the transmission layer in the unit time in the period of time to be used as a evidence for subsequent analysis. For example, analyzing that the TCP protocol occupies 95% of the total Session, TCP network anomalies such as TCP Flood, etc. are later verified.
(2) TopN user analysis: from the perspective of the user, listing a TopN sending user and a TopN receiving user so as to help analyze suspected attack IP or attacked IP;
(3) Selecting a specific user, and counting the distribution of the characteristics, such as: and selecting one dst _ IP, analyzing which Top10 communication source IPs are provided, which Top10 destination ports are provided, the size distribution condition of data packets/transmission packets, the numerical value of the semi-connection state, the Flow (network Flow) duration distribution and the like, and if the source IPs are dispersed, the destination ports are concentrated, the data packets/transmission packets are generally small, the semi-connection state data is large, and the network Flow duration is short, considering that the network abnormal type is likely to be DDoS attack.
Therefore, according to the drill-down analysis result, the network anomaly type of the abnormal network traffic data determined in the foregoing step can be verified, whether the network anomaly detection model is correctly detected or not can be verified, and specific anomaly information of the abnormal network traffic data can be obtained, for example, different source IPs are analyzed for the destination IP of TOP1 to be uniformly distributed, destination ports are also concentrated, the size of a data packet is small, and the like, so that the DDoS attack can be determined according to the drill-down analysis results, and the destination IP and the destination port are attacked addresses.
In the embodiment of the invention, the network flow data is extracted from the access layer for preprocessing, and the network anomaly detection model is adopted to automatically identify the anomaly, so that the network anomaly type can be quickly and accurately identified, and the detection efficiency and the real-time performance are effectively improved.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a network anomaly detection apparatus according to a second embodiment of the present invention, where the apparatus 40 includes:
a preprocessing module 41, configured to preprocess network traffic data of an access stratum, and obtain a characteristic parameter of the network traffic data;
a detection module 42, configured to detect the feature parameter by using a network anomaly detection model in the access stratum to obtain a detection result, where the detection result is abnormal or normal;
and the analysis module 43 is configured to analyze the characteristic parameter of which the detection result is abnormal, and determine a network abnormal type of the network traffic data.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm.
Optionally, the apparatus further comprises:
and the updating module is used for updating the network abnormity detection model by adopting the characteristic parameters with normal detection results at the edge layer and sending the updated network abnormity detection model to the access layer.
Optionally, the characteristic parameter includes at least one of: the information entropy of the source IP address, the information entropy of the destination port, the information entropy of the network flow duration, the information entropy of the data packet size, and the number ratio of syn packets to ack packets.
Optionally, the analysis module comprises at least one of:
the first analysis unit is used for determining that the network abnormal type of the network traffic data is distributed denial of service attack if the information entropy of the source IP address is increased;
the second analysis unit is used for determining that the network abnormal type of the network traffic data is port scanning if the information entropy of the destination port is increased;
and the third analysis unit is used for determining that the network abnormal type of the network traffic data is a worm virus if the information entropy of the destination IP address is increased.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one class cluster, and the apparatus further includes:
the calculation module is used for calculating a target parameter of each class of cluster in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the class of cluster plus three times of standard deviation of all classified characteristic parameters in the class of cluster;
and the determining module is used for determining that the characteristic parameters of the abnormal detection results are increased if the characteristic parameters of the abnormal detection results are larger than the target parameters of the corresponding cluster.
Optionally, the apparatus further comprises:
the drilling analysis module is used for performing drilling analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drilling analysis results;
and the verification module is used for verifying the network abnormal type of the determined network traffic data according to the drill-down analysis result and acquiring the abnormal information of the abnormal network traffic data.
The embodiment of the present invention is a product embodiment corresponding to the above method embodiment, and therefore, detailed description is omitted here, and please refer to the first embodiment in detail.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a detection apparatus according to a third embodiment of the present invention, where the detection apparatus 50 includes a processor 51, a memory 52, and a computer program stored in the memory 52 and capable of running on the processor 51; the processor 51, when executing the computer program, implements the steps of:
preprocessing network flow data of an access layer to obtain characteristic parameters of the network flow data;
detecting the characteristic parameters by adopting a network anomaly detection model at the access layer to obtain a detection result, wherein the detection result is abnormal or normal;
and analyzing the characteristic parameters with abnormal detection results to determine the network abnormal type of the network flow data.
In the embodiment of the invention, optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm.
In this embodiment of the present invention, optionally, when the processor 51 executes the computer program, the following steps may also be implemented: the detecting the characteristic parameters by using the network anomaly detection model and after obtaining the detection result, the method further comprises the following steps:
and updating the network anomaly detection model by adopting the characteristic parameters with normal detection results at the edge layer, and issuing the updated network anomaly detection model to the access layer.
Optionally, the characteristic parameter includes at least one of: the information entropy of the source IP address, the information entropy of the destination port, the information entropy of the network flow duration, the information entropy of the data packet size, and the number ratio of syn packets to ack packets.
Optionally, the analyzing the characteristic parameter of which the detection result is abnormal, and determining the network abnormal type of the network traffic data includes at least one of the following:
if the information entropy of the source IP address is increased, determining that the network abnormal type of the network flow data is distributed denial of service attack;
if the information entropy of the destination port is increased, determining the network abnormal type of the network traffic data as port scanning;
and if the information entropy of the destination IP address is increased, determining that the network abnormal type of the network traffic data is a worm virus.
Optionally, the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one class cluster, and before analyzing the characteristic parameter of which the detection result is abnormal and determining the network anomaly type of the network traffic data, the method further includes:
calculating a target parameter of each cluster in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the cluster plus three times of standard deviation of all classified characteristic parameters in the cluster;
and if the characteristic parameter of the abnormal detection result is larger than the target parameter of the corresponding class cluster, determining that the characteristic parameter of the abnormal detection result is increased.
Optionally, after analyzing the characteristic parameter of which the detection result is abnormal and determining the network abnormal type of the network traffic data, the method further includes:
performing drill-down analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drill-down analysis results;
and verifying the network abnormal type of the determined network traffic data according to the drill-down analysis result, and acquiring the abnormal information of the abnormal network traffic data.
The specific working process of the embodiment of the present invention is the same as that of the first embodiment of the method, and therefore, detailed description is not repeated here, and please refer to the description of the method steps in the first embodiment.
A fourth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in any one of the network anomaly detection methods in the first embodiment. Please refer to the above description of the method steps in the corresponding embodiments.
The computer-readable storage media described above, including non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
While the foregoing is directed to the preferred embodiment of the present invention, it will be appreciated by those skilled in the art that various changes and modifications may be made therein without departing from the principles of the invention as set forth in the appended claims.
Claims (10)
1. A network anomaly detection method is characterized by comprising the following steps:
preprocessing network traffic data of an access layer to obtain characteristic parameters of the network traffic data;
detecting the characteristic parameters by adopting a network anomaly detection model in the access layer to obtain a detection result, wherein the detection result is abnormal or normal;
and analyzing the characteristic parameters with abnormal detection results to determine the network abnormal type of the network traffic data.
2. The method of claim 1, wherein the network anomaly detection model is constructed based on a K-means cluster analysis algorithm.
3. The method according to claim 2, wherein the detecting the characteristic parameter by using the network anomaly detection model further comprises, after obtaining a detection result:
and updating the network anomaly detection model by adopting the characteristic parameters with normal detection results at the edge layer, and issuing the updated network anomaly detection model to the access layer.
4. The method of claim 1, wherein the characteristic parameter comprises at least one of: the information entropy of the source IP address, the information entropy of the destination port, the information entropy of the network flow duration, the information entropy of the data packet size, and the number ratio of syn packets to ack packets.
5. The method according to claim 4, wherein the analyzing the characteristic parameter whose detection result is abnormal, and determining the network abnormal type of the network traffic data includes at least one of:
if the information entropy of the source IP address is increased, determining that the network abnormal type of the network traffic data is distributed denial of service attack;
if the information entropy of the destination port is increased, determining that the network abnormal type of the network traffic data is port scanning;
and if the information entropy of the destination IP address is increased, determining that the network abnormal type of the network traffic data is a worm virus.
6. The method according to claim 5, wherein the network anomaly detection model is constructed based on a K-means cluster analysis algorithm, the network anomaly detection model includes at least one cluster, and before analyzing the characteristic parameters of which the detection result is an anomaly and determining the network anomaly type of the network traffic data, the method further includes:
calculating a target parameter of each class of clusters in the network anomaly detection model, wherein the target parameter is equal to the average value of all classified characteristic parameters in the class of clusters plus three times of the standard deviation of all classified characteristic parameters in the class of clusters;
and if the characteristic parameter of the abnormal detection result is larger than the target parameter of the corresponding cluster, determining that the characteristic parameter of the abnormal detection result is increased.
7. The method according to claim 5, wherein after analyzing the characteristic parameter whose detection result is abnormal and determining the network abnormal type of the network traffic data, further comprising:
performing drill-down analysis on the network flow data corresponding to the characteristic parameters with abnormal detection results to obtain drill-down analysis results;
and verifying the network abnormal type of the determined network traffic data according to the drill-down analysis result, and acquiring the abnormal information of the abnormal network traffic data.
8. A network anomaly detection device, comprising:
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing network traffic data of an access layer to acquire characteristic parameters of the network traffic data;
the detection module is used for detecting the characteristic parameters by adopting a network anomaly detection model at the access layer to obtain a detection result, wherein the detection result is abnormal or normal;
and the analysis module is used for analyzing the characteristic parameters of which the detection results are abnormal and determining the network abnormal type of the network flow data.
9. A detection apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor; characterized in that the processor, when executing the computer program, implements the network anomaly detection method according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the network anomaly detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110789429.3A CN115622720A (en) | 2021-07-13 | 2021-07-13 | Network anomaly detection method and device and detection equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110789429.3A CN115622720A (en) | 2021-07-13 | 2021-07-13 | Network anomaly detection method and device and detection equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115622720A true CN115622720A (en) | 2023-01-17 |
Family
ID=84854868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110789429.3A Pending CN115622720A (en) | 2021-07-13 | 2021-07-13 | Network anomaly detection method and device and detection equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115622720A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101795215A (en) * | 2010-01-28 | 2010-08-04 | 哈尔滨工程大学 | Network traffic anomaly detection method and detection device |
| CN104202336A (en) * | 2014-09-22 | 2014-12-10 | 浪潮电子信息产业股份有限公司 | DDoS (distributed denial of service) attach detection method based on information entropy |
| CN104660464A (en) * | 2015-01-22 | 2015-05-27 | 贵州电网公司信息通信分公司 | Network anomaly detection method based on non-extensive entropy |
| CN105553998A (en) * | 2015-12-23 | 2016-05-04 | 中国电子科技集团公司第三十研究所 | Network attack abnormality detection method |
| US20180167404A1 (en) * | 2016-12-08 | 2018-06-14 | Cisco Technology, Inc. | Filtering onion routing traffic from malicious domain generation algorithm (dga)-based traffic classification |
| WO2019184131A1 (en) * | 2018-03-29 | 2019-10-03 | 清华大学 | Entropy method and density clustering method integrated power stealing detection method and apparatus |
| CN111339297A (en) * | 2020-02-21 | 2020-06-26 | 广州天懋信息***股份有限公司 | Network asset anomaly detection method, system, medium, and device |
| CN112398779A (en) * | 2019-08-12 | 2021-02-23 | 中国科学院国家空间科学中心 | Network traffic data analysis method and system |
-
2021
- 2021-07-13 CN CN202110789429.3A patent/CN115622720A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101795215A (en) * | 2010-01-28 | 2010-08-04 | 哈尔滨工程大学 | Network traffic anomaly detection method and detection device |
| CN104202336A (en) * | 2014-09-22 | 2014-12-10 | 浪潮电子信息产业股份有限公司 | DDoS (distributed denial of service) attach detection method based on information entropy |
| CN104660464A (en) * | 2015-01-22 | 2015-05-27 | 贵州电网公司信息通信分公司 | Network anomaly detection method based on non-extensive entropy |
| CN105553998A (en) * | 2015-12-23 | 2016-05-04 | 中国电子科技集团公司第三十研究所 | Network attack abnormality detection method |
| US20180167404A1 (en) * | 2016-12-08 | 2018-06-14 | Cisco Technology, Inc. | Filtering onion routing traffic from malicious domain generation algorithm (dga)-based traffic classification |
| WO2019184131A1 (en) * | 2018-03-29 | 2019-10-03 | 清华大学 | Entropy method and density clustering method integrated power stealing detection method and apparatus |
| CN112398779A (en) * | 2019-08-12 | 2021-02-23 | 中国科学院国家空间科学中心 | Network traffic data analysis method and system |
| CN111339297A (en) * | 2020-02-21 | 2020-06-26 | 广州天懋信息***股份有限公司 | Network asset anomaly detection method, system, medium, and device |
Non-Patent Citations (3)
| Title |
|---|
| RODRIGO SIQUEIRA MARTINS: ""Automatic Detection of Computer Network Traffic Anomalies based on Eccentricity Analysis"", 《2018 IEEE INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS (FUZZ-IEEE)》, 14 October 2018 (2018-10-14) * |
| 庄芳仪: ""基于信息熵的异常流量分布式检测方法的研究"", 24 August 2011 (2011-08-24) * |
| 董刚、余伟、玄光哲: ""高级持续性威胁中攻击特征的分析与检测"", 《吉林大学学报(理学版)》, vol. 57, no. 02, 26 March 2019 (2019-03-26), pages 339 - 344 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111935170B (en) | Network abnormal flow detection method, device and equipment | |
| US9386028B2 (en) | System and method for malware detection using multidimensional feature clustering | |
| US9769190B2 (en) | Methods and apparatus to identify malicious activity in a network | |
| US11316878B2 (en) | System and method for malware detection | |
| CN108667856B (en) | Network anomaly detection method, device, equipment and storage medium | |
| CN111277570A (en) | Data security monitoring method and device, electronic equipment and readable medium | |
| CN108965347B (en) | Distributed denial of service attack detection method, device and server | |
| CN109450721B (en) | Network abnormal behavior identification method based on deep neural network | |
| CN107579986B (en) | Network security detection method in complex network | |
| CN109361673B (en) | Network anomaly detection method based on flow data sample statistics and balance information entropy estimation | |
| CN113206860B (en) | DRDoS attack detection method based on machine learning and feature selection | |
| CN111835681B (en) | Large-scale flow abnormal host detection method and device | |
| JP2018148350A (en) | Threshold determination device, threshold level determination method and program | |
| Amoli et al. | A real time unsupervised NIDS for detecting unknown and encrypted network attacks in high speed network | |
| CN110602109A (en) | Application layer DDoS attack detection and defense method based on multi-feature entropy | |
| Patcha et al. | Network anomaly detection with incomplete audit data | |
| Mohan et al. | Complex event processing based hybrid intrusion detection system | |
| CN107231377B (en) | BGP-LDoS attack detection method based on mutation equilibrium state theory | |
| CN113055333B (en) | Network flow clustering method and device capable of adaptively and dynamically adjusting density grid | |
| CN110912933B (en) | Equipment identification method based on passive measurement | |
| CN112291213A (en) | Abnormal flow analysis method and device based on intelligent terminal | |
| CN115622720A (en) | Network anomaly detection method and device and detection equipment | |
| Sheng et al. | How to fingerprint attack traffic against industrial control system network | |
| CN107566187B (en) | SLA violation monitoring method, device and system | |
| CN117395070B (en) | Abnormal flow detection method based on flow characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |