CN117040824A - Network threat detection method and system - Google Patents
Network threat detection method and system Download PDFInfo
- Publication number
- CN117040824A CN117040824A CN202310970741.1A CN202310970741A CN117040824A CN 117040824 A CN117040824 A CN 117040824A CN 202310970741 A CN202310970741 A CN 202310970741A CN 117040824 A CN117040824 A CN 117040824A
- Authority
- CN
- China
- Prior art keywords
- data
- flow
- network
- standard
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 86
- 238000012549 training Methods 0.000 claims abstract description 36
- 238000012360 testing method Methods 0.000 claims abstract description 30
- 238000007781 pre-processing Methods 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 20
- 238000010606 normalization Methods 0.000 claims description 32
- 238000010586 diagram Methods 0.000 claims description 17
- 238000013507 mapping Methods 0.000 claims description 13
- 238000001228 spectrum Methods 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 10
- 238000005520 cutting process Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 description 27
- 238000012545 processing Methods 0.000 description 7
- 230000001934 delay Effects 0.000 description 5
- 239000000654 additive Substances 0.000 description 4
- 230000000996 additive effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computer Hardware Design (AREA)
- Probability & Statistics with Applications (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a network threat method and system, and belongs to the technical field of network security. Firstly, collecting network traffic data and physical signal data, and preprocessing the network traffic data and the physical signal data to obtain standard network traffic data and standard physical signal data. And then converting the standard network flow data into a flow image, and fusing the standard physical signal data with the flow image to obtain a multi-channel flow image. And inputting the multichannel flow images into a flow network for training and testing to obtain a flow detection model. And finally, deploying the flow detection model on a platform to detect the flow, and obtaining a detection result. The invention provides remarkable benefits for the network security field by comprehensively utilizing the abundant characteristic representation of the network traffic data and the physical signal data, plays an important role in improving the accuracy and the instantaneity of network threat detection, and provides safer and more reliable network environment for network users and enterprises.
Description
Technical Field
The invention belongs to the technical field of network security, and particularly relates to a network threat detection method and system.
Background
With the rapid development of the internet, network threats and attacks are increasing, and great challenges are brought to network security. Network threats include, but are not limited to, malware, hacking, denial of service attacks, and the like. In order to effectively address these cyber threats, cyber threat detection is a very important technology. The network threat detection aims at timely discovering potential threat behaviors and taking corresponding defensive measures by monitoring and analyzing network traffic data.
Traditional network threat detection methods are mainly based on rules, signatures or feature matching, and can detect known network threats, but often cannot effectively identify unknown novel threats. Therefore, in order to improve the accuracy and timeliness of network threat detection, a convolutional neural network method is adopted for threat detection.
In the prior art, chinese patent publication No.: CN115643086a provides an unknown threat detection method based on a deep neural network. The patent includes: disposing information collectors on all nodes of the big data platform, collecting network flow data with different dimensionalities, and carrying out normalization processing on the network flow data to obtain sample set data; constructing a convolutional neural network, and carrying out optimization training on the convolutional neural network by using a sample set data set based on a deep learning algorithm to obtain an unknown threat detection model; and testing the acquired network flow data through an unknown threat detection model to obtain a corresponding unknown threat detection result. The patent can effectively improve the recognition success rate of the information system to the known threat and the unknown threat by applying the deep learning technology, and accurately and timely discover the invasion so as to avoid the influence of the unknown threat on the safety of the information system to the greatest extent.
The patent relies solely on traffic data for testing, does not analyze other data, and lacks more comprehensive network data. Therefore, a more comprehensive network threat detection method needs to be proposed to provide more abundant information and enhance the perception capability of the network threat.
Disclosure of Invention
Based on the technical problems, the invention provides a network threat detection method and system, which can provide more comprehensive and multidimensional network data information by combining physical signal data (such as time delay, frequency spectrum and power) with network traffic data, so that a network threat detection model can better sense abnormal behaviors and potential threats in network traffic.
The invention provides a network threat detection method, which comprises the following steps:
step S1: collecting network traffic data and physical signal data; the physical signal data comprises time delay, frequency spectrum and power;
step S2: respectively preprocessing the network traffic data and the physical signal data to obtain standard network traffic data and standard physical signal data;
step S3: converting the standard network flow data to obtain a flow image;
step S4: fusing the standard physical signal data with the flow image to obtain a multi-channel flow image;
step S5: inputting the multichannel flow image into a flow network for training and testing to obtain a flow detection model;
step S6: and deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
Optionally, the preprocessing operation is performed on the network traffic data and the physical signal data to obtain standard network traffic data and standard physical signal data, which specifically includes:
sequentially performing data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network traffic data to obtain standard network traffic data;
and carrying out data normalization and pixel value mapping operation on the physical signal data in sequence to obtain standard physical signal data.
Optionally, the converting the standard network traffic data to obtain a traffic image specifically includes:
performing data cutting on the standard network flow data to obtain segmented flow data; the segmented flow data comprises flow data of a fixed time window;
and normalizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on an image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the data points into the image with the specified size from left to right and from top to bottom to obtain a flow image.
Optionally, the fusing the standard physical signal data with the flow image to obtain a multi-channel flow image specifically includes:
combining the standard physical signal data serving as channel characteristics with the flow image to obtain a multi-channel flow image; the standard physical signal data includes physical signal data of a fixed time window.
Optionally, the inputting the multi-channel flow image into a flow network for training and testing to obtain a flow detection model specifically includes:
marking the multichannel flow image to obtain a marked sample; the marked samples comprise malicious traffic and normal traffic;
dividing the marked sample into a training set and a testing set according to a certain proportion, and inputting the training set into the flow network for training to obtain an initial flow detection model; and inputting the test set into an initial flow detection model for testing to obtain a flow detection model.
Optionally, the training set is input to the traffic network to perform training, so as to obtain an initial traffic detection model, which specifically includes:
the flow network comprises a standard convolution module, two residual error modules, a global average pooling module, a full connection layer and a Softmax classifier; the two residual error modules are a first residual error module and a second residual error module respectively;
inputting the multichannel flow image of the training set to the standard convolution module for convolution operation to obtain a feature map O4;
inputting the characteristic diagram O4 into the first residual error module to perform residual error connection operation to obtain a characteristic diagram O14;
inputting the feature map O14 into the second residual error module to perform residual error connection operation to obtain a feature map O25;
and sequentially inputting the feature map O25 into the global average pooling layer, the full-connection layer and the Softmax classifier for classification to obtain an initial flow detection model.
The invention also provides a network threat detection system, comprising:
the data acquisition module is used for collecting network flow data and physical signal data; the physical signal data comprises time delay, frequency spectrum and power;
the data preprocessing module is used for respectively preprocessing the network flow data and the physical signal data to obtain standard network flow data and standard physical signal data;
the flow data conversion module is used for converting the standard network flow data to obtain a flow image;
the data fusion module is used for fusing the standard physical signal data with the flow image to obtain a multi-channel flow image;
the model generation module is used for inputting the multichannel flow image into a flow network for training and testing to obtain a flow detection model;
and the flow detection module is used for deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
Optionally, the data preprocessing module specifically includes:
the flow data preprocessing sub-module is used for sequentially performing data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network flow data to obtain standard network flow data;
and the physical signal data preprocessing sub-module is used for sequentially carrying out data normalization and pixel value mapping operation on the physical signal data to obtain standard physical signal data.
Optionally, the flow data conversion module specifically includes:
the data cutting sub-module is used for carrying out data cutting on the standard network flow data to obtain segmented flow data;
and the flow image generation sub-module is used for standardizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on an image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the data points into the image with the specified size from left to right and from top to bottom to obtain a flow image.
Optionally, the model generating module specifically includes:
the standard convolution sub-module is used for inputting the multichannel flow image of the training set into the standard convolution module to carry out convolution operation to obtain a feature map O4;
the first residual sub-module is used for inputting the characteristic diagram O4 into the first residual module to carry out residual connection operation to obtain a characteristic diagram O14;
the second residual sub-module is used for inputting the characteristic diagram O14 into the second residual module to carry out residual connection operation to obtain a characteristic diagram O25;
and the classification sub-module is used for sequentially inputting the feature map O25 into the global average pooling layer, the full-connection layer and the Softmax classifier for classification to obtain an initial flow detection model.
Compared with the prior art, the invention has the following beneficial effects:
the invention uses a network threat detection method, which fuses network traffic data and physical signal data to obtain a multichannel traffic image. Through the fusion of the multi-channel information, more comprehensive and multi-dimensional network data information can be provided, so that the network threat detection model can better sense abnormal behaviors and potential threats in network traffic; traditional cyber threat detection methods typically rely on manually defined features or rules that may not fully and accurately reflect the complexity of the cyber threat. By converting the network traffic data into a traffic image and fusing the physical signal data as channel characteristics, the method can automatically learn more abundant and effective characteristic representation, and improves the accuracy of network threat detection; the deep learning technology is utilized, and the network threat detection model has stronger self-adaptive learning capability. The method can automatically adjust and optimize according to the actual network environment and threat characteristics, so that the identification capability of the novel network threat is maintained, and the increasingly complex and changeable network threat situation is effectively treated; the construction mode and the deep learning model of the multi-channel flow image used by the method can be flexibly adapted to different network environments and application scenes. Therefore, the method has stronger compatibility and expansibility, can be widely applied to various network equipment and systems, and is continuously optimized and upgraded along with the development of technology.
Drawings
FIG. 1 is a flow chart of a method for detecting a network threat according to the present invention;
FIG. 2 is a flow network structure diagram of a network threat detection method of the present invention;
fig. 3 is a block diagram of a network threat detection system of the invention.
Detailed Description
The invention is further described below in connection with specific embodiments and the accompanying drawings, but the invention is not limited to these embodiments.
Example 1
As shown in fig. 1, the invention discloses a network threat detection method, which comprises the following steps:
step S1: collecting network traffic data and physical signal data; the physical signal data includes time delay, frequency spectrum and power.
Step S2: and respectively preprocessing the network flow data and the physical signal data to obtain standard network flow data and standard physical signal data.
Step S3: and converting the standard network flow data to obtain a flow image.
Step S4: and fusing the standard physical signal data with the flow image to obtain a multi-channel flow image.
Step S5: and inputting the multichannel flow images into a flow network for training and testing to obtain a flow detection model.
Step S6: and deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
The steps are discussed in detail below:
step S1: collecting network traffic data and physical signal data; the physical signal data includes time delay, frequency spectrum and power.
The step S1 specifically comprises the following steps:
step S11: the collection of network traffic data is mainly performed by passive collection, which is by monitoring data packets on the network, usually using network monitoring tools or data packet capturing software, or active collection.
In this embodiment, wireshark is a commonly used network packet analysis tool that captures packets on the network and stores them in the form of data files. Passive collection does not actively generate traffic, but rather analyzes existing traffic in the network; active collection is by sending test traffic to the network to measure network performance and may be accomplished through the use of specialized network performance testing tools. In this embodiment, the ping command is used to test the delay (round trip delay) and packet loss rate with the target host, and the tools such as the iperf or ttcp can also be used to test the bandwidth and throughput of the network.
Step S12: collecting physical signal data: physical signal data includes information such as time delay, frequency spectrum, and power, which play a critical role when the data packets are transmitted over a transmission medium (e.g., cable, fiber, wireless channel, etc.).
In this embodiment, the time delay refers to the time required for the data packet from the transmitting end to the receiving end. In network communications, there are various types of delays, including propagation delays, transmission delays, queuing delays, processing delays, and the like. The time delay data can be collected by adding a time stamp to the data packet and recording the time stamp at the receiving end. Spectrum is an important concept in wireless communication, representing the distribution of signals over different frequencies. The wireless signals are transmitted in a certain frequency range, and the knowledge of the spectrum distribution is important for avoiding interference and optimizing the performance of the wireless network. Collecting spectral data requires the use of a spectrum analyzer or software radio (SDR) device. Power data refers to the strength or power level of a wireless signal. In wireless communications, knowledge of the power condition of a signal is critical to coverage, transmission distance, and interference control. Collecting power data may be accomplished through the use of a power meter or power sensor.
Step S2: and respectively preprocessing the network flow data and the physical signal data to obtain standard network flow data and standard physical signal data.
The step S2 specifically comprises the following steps:
step S21: and carrying out data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network traffic data in sequence to obtain standard network traffic data.
The step S21 specifically includes:
(1) data cleansing refers to removing unnecessary information and outliers from the raw network traffic data. This includes processing duplicate data, deleting missing or outliers, etc. Repeated data may occur due to too high a sampling frequency or for other reasons, which needs to be deduplicated. The missing values may be due to network failure or other factors, and linear interpolation is filled in or deleted. Outliers may be caused by network attacks, equipment failures, etc., using box graphs to detect and process outliers.
(2) Redundancy elimination refers to eliminating duplicate or redundant information to reduce the size of a data set and increase data processing efficiency. For example, there may be multiple repeated requests or responses in the traffic data, and the repeated portions may be removed, leaving only one portion.
(3) In network traffic data, different devices, protocols, and data sources may use different data formats. Data format normalization is the conversion of all data into a unified format for subsequent processing and analysis, including unified timestamp format, IP address format, data header format, and the like.
(4) Feature extraction is the extraction of useful features or attributes from the raw data. In network traffic data, the characteristics may be information about packet size, protocol type, traffic distribution, transmission rate, etc. By feature extraction, the data dimension can be reduced and more focused on critical information.
(5) Data normalization is the conversion of data of different dimensions into the same range to avoid some features from having too much or too little impact on the results during computation. Common normalization methods include Min-Max normalization, which scales the data to within the [0,1] range, and normalization, which converts the data to a distribution with a mean of 0 and standard deviation of 1.
Step S22: and carrying out data normalization and pixel value mapping operation on the physical signal data in sequence to obtain standard physical signal data.
The step S22 specifically includes:
and carrying out normalization processing on the physical signal data, and mapping the physical signal data to a certain range. A common normalization method is to scale the signal values to between 0,1 or-1, 1 so that all features are within the same scale. The pixel brightness in the image channel, i.e. the gray value in the range of the gray image 0,255, is mapped according to the specific normalized value. The value 0 (black) represents the minimum value of feature a, while the value 255 (white) represents the maximum value of feature a.
Step S3: and converting the standard network flow data to obtain a flow image.
The step S3 specifically comprises the following steps:
step S31: cutting the standard network flow data to obtain segmented flow data; the segmented traffic data comprises traffic data for a fixed time window. The method specifically comprises the following steps:
the standard network traffic data is data cut into traffic data of a specified time window, to obtain segmented traffic data, for example, 1 hour of traffic data, and the time window is set to 2 minutes, and then 30 windows are cut, and each time window comprises 30 data points, and each data point is 4 seconds.
Step S32: the method comprises the steps of standardizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on an image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the standard segmented flow data into the image with the specified size from left to right from top to bottom to obtain a flow image. The method specifically comprises the following steps:
the segmented flow data is normalized, i.e. the flow data is normalized at the maximum and minimum values of all time windows, the normalized flow data is mapped to the brightness of the image pixels, the normalized value of each data point is between [0,1], 0 can be mapped to black, 1 to white, and the middle value is mapped to the middle gray according to linearity. For example, for a 1 hour flow rate data, the time window is set to 2 minutes, then 30 windows are cut, each time window including 30 data points, one data point every 4 seconds. Each data point represents the flow value size (packet size) within 4 seconds, an average value can be taken, then 30 data points of each time window are mapped into 30 pixel values to form one row of an image, and 30 rows are stacked in time sequence from left to right and from top to bottom to form a 30×30 flow image.
Step S4: and fusing the standard physical signal data with the flow image to obtain a multi-channel flow image.
The step S4 specifically comprises the following steps:
combining the standard physical signal data as channel characteristics with the flow image to obtain a multi-channel flow image; the standard physical signal data includes physical signal data of a fixed time window, and specifically includes:
the physical signal features are consistent with the data points of each time window of the flow data, the physical signal is regarded as an auxiliary feature of each data point, the physical signal features are taken as channels for introducing additional information, and the features of the physical signal are fused into the image representation of the flow data. In this embodiment, there are 3 physical signal features, each of which can be used as a channel for the image. The multi-channel image will have 4 channels: wherein 3 channels represent the characteristics of the physical signal and another channel represents the network traffic data. In constructing the multi-channel image, the flow data image and the image channels of the 3 physical signal features are combined to form a multi-channel image.
Step S5: and inputting the multichannel flow images into a flow network for training and testing to obtain a flow detection model.
The step S5 specifically comprises the following steps:
step S51: marking the multichannel flow image to obtain a marked sample; the marked samples include malicious traffic and normal traffic.
Step S52: dividing a marked sample into a training set and a testing set according to a certain proportion, inputting the training set into a flow network for training to obtain an initial flow detection model; and inputting the test set into the initial flow detection model for testing to obtain the flow detection model.
In fig. 2, conv2D represents a standard convolution layer, with convolution kernel sizes of 3×3 and 1×1; strides represents the step size, and takes on the value of 1 or 2; the normalized activation layer comprises a batch normalization layer (Batch Normalization) and an activation function layer (Activation (Relu)), wherein the normalized activation function layer selects a Relu activation function; sepConv2D represents a depth separable convolutional layer with convolutional kernel sizes of 3 x 3 and 1 x 1; dense stands for full connectivity layer; globalaeragepooling 2D represents a global average pooling layer; add,/>) Representation->，/>Performing element-by-element addition; o->Representing the characteristics obtained in the traffic network, < >>The value range is [1,25 ]]，/>Is an integer.
As shown in fig. 2, step S52 specifically includes:
A. dividing the marked sample into a training set and a testing set according to a certain proportion, inputting the training set into a flow network for training to obtain an initial flow detection model, and specifically comprising the following steps:
dividing the marked sample into a training set and a testing set according to a certain proportion, inputting a multichannel flow image of the training set into a standard convolution module for convolution operation to obtain a feature map O4, wherein the method specifically comprises the following steps:
the training set and the test data set of the invention are as follows 8:2, inputting the multichannel flow image of the training set into a first standard convolution layer for convolution operation to obtain a feature map O1, wherein the number of convolution kernels of the first standard convolution layer is 64, the size of the convolution kernels is 3 multiplied by 3, and the step length is 2; feature map O1 is 15×15 for 64 channels; inputting the feature map O1 into a first normalized activation layer to perform batch normalization and activation operation to obtain a feature map O2; the feature map O2 is 15×15 for 64 channels; inputting the feature map O2 into a second standard convolution layer to carry out convolution operation to obtain a feature map O3, wherein the number of convolution kernels of the second standard convolution layer is 128, the size of the convolution kernels is 3 multiplied by 3, and the step length is 2; feature map O3 is 8×8 for 128 channels; inputting the feature map O3 into a second normalized activation layer to perform batch normalization and activation operation to obtain a feature map O4; the feature map O4 is 8×8 for 128 channels.
In this embodiment, the first standard convolution module includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, and a second normalized activation layer.
Inputting the feature map O4 to a first residual module for residual connection operation to obtain a feature map O14, which specifically includes:
(1) Inputting the feature map O4 into a third standard convolution layer for convolution operation to obtain a feature map O5, wherein the number of convolution kernels of the third standard convolution layer is 128, the size of the convolution kernels is 1 multiplied by 1, and the step length is 1; feature map O5 is 8 x 8 for 128 channels; inputting the feature map O5 into a third normalized activation layer to perform batch normalization and activation operation to obtain a feature map O6; feature map O6 is 8 x 8 for 128 channels; inputting the feature map O6 into a fourth standard convolution layer for convolution operation to obtain a feature map O7, wherein the number of convolution kernels of the fourth standard convolution layer is 128, the size of the convolution kernels is 3 multiplied by 3, and the step length is 1; feature map O7 is 8×8 for 128 channels; inputting the feature map O7 into a fourth normalized activation layer to perform batch normalization and activation operation to obtain a feature map O8; the feature map O8 is 8×8 for 128 channels.
(2) Inputting the feature map O4 into a first depth separable convolution layer to perform depth separable convolution operation to obtain a feature map O9, wherein the number of convolution kernels of the first depth separable convolution layer is 128, the size of the convolution kernels is 3 multiplied by 3, and the step length is 1; feature map O9 is 8×8 for 128 channels; inputting the feature map O9 into a fifth normalized activation layer for batch normalization and activation operation to obtain a feature map O10; feature map O10 is 8×8 for 128 channels; inputting the feature map O10 into a second depth-separable convolution layer to perform depth-separable convolution operation to obtain a feature map O11, wherein the number of convolution kernels of the second depth-separable convolution layer is 128, the size of the convolution kernels is 3 multiplied by 3, and the step length is 1; feature map O11 is 8×8 for 128 channels; inputting the feature map O11 into a fourth normalized activation layer for batch normalization and activation operation to obtain a feature map O12; the feature map O12 is 8×8 for 128 channels.
(3) Inputting the feature map O8 and the feature map O12 into a first element-by-element adding layer to perform element-by-element adding operation to obtain a feature map O13; feature map O13 is 8×8 for 128 channels; inputting the feature map O13 and the feature map O4 into a second element-by-element addition layer to perform element-by-element addition operation to obtain a feature map O14; the feature map O14 is 8×8 for 128 channels.
In this embodiment, the first residual module includes a third standard convolutional layer, a third normalized active layer, a fourth standard convolutional layer, a fourth normalized active layer, a first depth separable convolutional layer, a fifth normalized active layer, a second depth separable convolutional layer, a sixth normalized active layer, a first element-by-element additive layer, and a second element-by-element additive layer.
Inputting the feature map O14 to a second residual module for residual connection operation to obtain a feature map O25, which specifically includes:
1) Inputting the feature map O14 into a fifth standard convolution layer for convolution operation to obtain a feature map O15, wherein the number of convolution kernels of the fifth standard convolution layer is 256, the size of the convolution kernels is 1 multiplied by 1, and the step length is 1; feature map O15 is 8×8 for 256 channels; inputting the feature map O15 into a seventh normalized activation layer for batch normalization and activation operation to obtain a feature map O16; feature map O16 is 8×8 for 256 channels; inputting the feature map O16 into a sixth standard convolution layer for convolution operation to obtain a feature map O17, wherein the number of convolution kernels of the sixth standard convolution layer is 256, the size of the convolution kernels is 3 multiplied by 3, and the step length is 2; feature map O17 is 4×4 for 256 channels; inputting the feature map O17 into an eighth normalized activation layer for batch normalization and activation operation to obtain a feature map O18; the feature map O18 is 4×4 of 256 channels.
2) Inputting the feature map O14 into a third depth separable convolution layer to perform depth separable convolution operation to obtain a feature map O19, wherein the number of convolution kernels of the third depth separable convolution layer is 256, the size of the convolution kernels is 3 multiplied by 3, and the step length is 1; feature map O19 is 8×8 for 256 channels; inputting the feature map O19 into a ninth normalized activation layer for batch normalization and activation operation to obtain a feature map O20; feature map O20 is 8×8 for 256 channels; inputting the feature map O20 into a fourth depth separable convolution layer to perform depth separable convolution operation to obtain a feature map O21, wherein the number of convolution kernels of the fourth depth separable convolution layer is 256, the size of the convolution kernels is 3 multiplied by 3, and the step length is 2; feature map O21 is 4×4 for 256 channels; inputting the feature map O21 into a tenth normalized activation layer for batch normalization and activation operation to obtain a feature map O22; the feature map O22 is 4×4 of 256 channels.
3) Inputting the feature map O18 and the feature map O22 into a third element-by-element adding layer to perform element-by-element adding operation to obtain a feature map O23; feature map O23 is 4×4 for 256 channels; inputting the feature map O14 into a seventh standard convolution layer for convolution operation to obtain a feature map O24; the number of convolution kernels of the seventh standard convolution layer is 256, the size of the convolution kernels is 3 multiplied by 3, and the step length is 2; feature map O24 is 4×4 for 256 channels; inputting the feature map O23 and the feature map O24 into a fourth element-by-element adding layer to perform element-by-element adding operation to obtain a feature map O25; the feature map O25 is 4×4 of 256 channels.
In this embodiment, the second residual module includes a fifth standard convolutional layer, a seventh normalized active layer, a sixth standard convolutional layer, an eighth normalized active layer, a third depth separable convolutional layer, a ninth normalized active layer, a fourth depth separable convolutional layer, a tenth normalized active layer, a seventh standard convolutional layer, a third element-by-element additive layer, and a fourth element-by-element additive layer.
And IV, sequentially inputting the feature map O25 into a global average pooling layer, a full connection layer and a Softmax classifier for classification to obtain an initial flow detection model.
In this embodiment, the traffic network includes a standard convolution module, two residual modules, a global average pooling module, a full connection layer, and a Softmax classifier; the two residual error modules are a first residual error module and a second residual error module respectively.
B. And inputting the test set into the initial flow detection model for testing to obtain the flow detection model.
Step S6: and deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
The step S6 specifically comprises the following steps:
and exporting the trained flow detection model into a format which can be deployed. Typically, the model is saved as a file with specific model structures and parameters, such as TensorFlow SavedModel or ONNX format; selecting a proper deployment platform, which can be a Cloud platform (such as AWS, azure, *** Cloud, etc.), a local server, an edge device or other hardware platforms; before deploying the model, ensuring that an appropriate production environment, including an operating system, a software library and dependent items, is set on the target platform; ensuring that there are sufficient computing resources and memory space on the platform to run the model; the exported model is loaded on the deployment platform and ready to receive traffic data for reasoning. According to the input requirement of the model, converting the flow data into a proper input format, for example converting the data cut by a time window into an image format; and inputting the preprocessed flow data into the model for reasoning. The model will return a classification result indicating whether the traffic within the time window is abnormal. And according to the model output, storing the detection result in a database, sending an alarm or performing other subsequent processing.
The invention can also monitor and optimize the performance of the model, including the detection accuracy, the reasoning time and the resource utilization rate, in the production environment. And (3) carrying out model optimization and adjustment according to the monitoring result so as to ensure efficient operation of the model in the production environment.
The traffic detection model needs to be continually updated and maintained to accommodate new traffic patterns and threats. The model is updated regularly, and potential problems and loopholes are processed in time, so that the high efficiency and accuracy of flow detection are maintained.
Example 2
As shown in fig. 3, the present invention further provides a cyber threat detection system, including:
a data acquisition module 10 for collecting network traffic data and physical signal data; the physical signal data includes time delay, frequency spectrum and power.
The data preprocessing module 20 is configured to perform preprocessing operation on the network traffic data and the physical signal data, so as to obtain standard network traffic data and standard physical signal data.
The traffic data conversion module 30 is configured to convert standard network traffic data to obtain a traffic image.
The data fusion module 40 is configured to fuse the standard physical signal data with the flow image to obtain a multi-channel flow image.
The model generating module 50 is configured to input the multi-channel flow image into a flow network for training and testing, so as to obtain a flow detection model.
The flow detection module 60 is configured to deploy the flow detection model on the platform and perform flow detection to obtain a detection result.
As an alternative embodiment, the data preprocessing module 20 of the present invention specifically includes:
and the flow data preprocessing sub-module is used for sequentially performing data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network flow data to obtain standard network flow data.
And the physical signal data preprocessing sub-module is used for sequentially carrying out data normalization and pixel value mapping operation on the physical signal data to obtain standard physical signal data.
As an alternative embodiment, the flow data conversion module 30 of the present invention specifically includes:
and the data cutting sub-module is used for carrying out data cutting on the standard network flow data to obtain segmented flow data.
The flow image generation sub-module is used for standardizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on the image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the standard segmented flow data into the image with the specified size from left to right and from top to bottom to obtain the flow image.
As an alternative embodiment, the model generating module 50 of the present invention specifically includes:
and the standard convolution sub-module is used for inputting the multichannel flow image of the training set into the standard convolution module to carry out convolution operation to obtain a characteristic diagram O4.
And the first residual sub-module is used for inputting the characteristic diagram O4 into the first residual module to perform residual connection operation to obtain the characteristic diagram O14.
And the second residual sub-module is used for inputting the characteristic map O14 into the second residual module to perform residual connection operation, so as to obtain a characteristic map O25.
And the classification sub-module is used for sequentially inputting the feature map O25 into the global average pooling layer, the full-connection layer and the Softmax classifier for classification to obtain an initial flow detection model.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of detecting a cyber threat, the method comprising:
step S1: collecting network traffic data and physical signal data; the physical signal data comprises time delay, frequency spectrum and power;
step S2: respectively preprocessing the network traffic data and the physical signal data to obtain standard network traffic data and standard physical signal data;
step S3: converting the standard network flow data to obtain a flow image;
step S4: fusing the standard physical signal data with the flow image to obtain a multi-channel flow image;
step S5: inputting the multichannel flow image into a flow network for training and testing to obtain a flow detection model;
step S6: and deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
2. The method for detecting a network threat according to claim 1, wherein the preprocessing operation is performed on the network traffic data and the physical signal data respectively to obtain standard network traffic data and standard physical signal data, and the method specifically comprises:
sequentially performing data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network traffic data to obtain standard network traffic data;
and carrying out data normalization and pixel value mapping operation on the physical signal data in sequence to obtain standard physical signal data.
3. The method for detecting a network threat according to claim 1, wherein the converting the standard network traffic data to obtain a traffic image specifically includes:
performing data cutting on the standard network flow data to obtain segmented flow data; the segmented flow data comprises flow data of a fixed time window;
and normalizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on an image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the data points into the image with the specified size from left to right and from top to bottom to obtain a flow image.
4. The method for detecting cyber threat according to claim 1, wherein the fusing the standard physical signal data with the traffic image to obtain a multi-channel traffic image specifically comprises:
combining the standard physical signal data serving as channel characteristics with the flow image to obtain a multi-channel flow image; the standard physical signal data includes physical signal data of a fixed time window.
5. The method for detecting a network threat according to claim 1, wherein the inputting the multi-channel traffic image into a traffic network for training and testing, to obtain a traffic detection model, specifically comprises:
marking the multichannel flow image to obtain a marked sample; the marked samples comprise malicious traffic and normal traffic;
dividing the marked sample into a training set and a testing set according to a certain proportion, and inputting the training set into the flow network for training to obtain an initial flow detection model; and inputting the test set into an initial flow detection model for testing to obtain a flow detection model.
6. The method for detecting cyber threat of claim 5, wherein the inputting the training set into the traffic network for training to obtain an initial traffic detection model specifically comprises:
the flow network comprises a standard convolution module, two residual error modules, a global average pooling module, a full connection layer and a Softmax classifier; the two residual error modules are a first residual error module and a second residual error module respectively;
inputting the multichannel flow image of the training set to the standard convolution module for convolution operation to obtain a feature map O4;
inputting the characteristic diagram O4 into the first residual error module to perform residual error connection operation to obtain a characteristic diagram O14;
inputting the feature map O14 into the second residual error module to perform residual error connection operation to obtain a feature map O25;
and sequentially inputting the feature map O25 into the global average pooling layer, the full-connection layer and the Softmax classifier for classification to obtain an initial flow detection model.
7. A cyber threat detection system, the system comprising:
the data acquisition module is used for collecting network flow data and physical signal data; the physical signal data comprises time delay, frequency spectrum and power;
the data preprocessing module is used for respectively preprocessing the network flow data and the physical signal data to obtain standard network flow data and standard physical signal data;
the flow data conversion module is used for converting the standard network flow data to obtain a flow image;
the data fusion module is used for fusing the standard physical signal data with the flow image to obtain a multi-channel flow image;
the model generation module is used for inputting the multichannel flow image into a flow network for training and testing to obtain a flow detection model;
and the flow detection module is used for deploying the flow detection model on a platform and detecting the flow to obtain a detection result.
8. The cyber threat detection system of claim 7, wherein the data preprocessing module specifically comprises:
the flow data preprocessing sub-module is used for sequentially performing data cleaning, redundancy removal, data format standardization, feature extraction and data normalization on the network flow data to obtain standard network flow data;
and the physical signal data preprocessing sub-module is used for sequentially carrying out data normalization and pixel value mapping operation on the physical signal data to obtain standard physical signal data.
9. The cyber threat detection system of claim 7, wherein the traffic data conversion module specifically comprises:
the data cutting sub-module is used for carrying out data cutting on the standard network flow data to obtain segmented flow data;
and the flow image generation sub-module is used for standardizing the segmented flow data to obtain standard segmented flow data, mapping the standard segmented flow data to pixel points on an image, wherein each pixel point represents each data point in the standard segmented flow data, and filling the data points into the image with the specified size from left to right and from top to bottom to obtain a flow image.
10. The cyber threat detection system of claim 7, wherein the model generation module specifically comprises:
the standard convolution sub-module is used for inputting the multichannel flow image of the training set into the standard convolution module to carry out convolution operation to obtain a feature map O4;
the first residual sub-module is used for inputting the characteristic diagram O4 into the first residual module to carry out residual connection operation to obtain a characteristic diagram O14;
the second residual sub-module is used for inputting the characteristic diagram O14 into the second residual module to carry out residual connection operation to obtain a characteristic diagram O25;
and the classification sub-module is used for sequentially inputting the feature map O25 into the global average pooling layer, the full-connection layer and the Softmax classifier for classification to obtain an initial flow detection model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310970741.1A CN117040824A (en) | 2023-08-03 | 2023-08-03 | Network threat detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310970741.1A CN117040824A (en) | 2023-08-03 | 2023-08-03 | Network threat detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117040824A true CN117040824A (en) | 2023-11-10 |
Family
ID=88632933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310970741.1A Pending CN117040824A (en) | 2023-08-03 | 2023-08-03 | Network threat detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117040824A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117252488A (en) * | 2023-11-16 | 2023-12-19 | 国网吉林省电力有限公司经济技术研究院 | Industrial cluster energy efficiency optimization method and system based on big data |
-
2023
- 2023-08-03 CN CN202310970741.1A patent/CN117040824A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117252488A (en) * | 2023-11-16 | 2023-12-19 | 国网吉林省电力有限公司经济技术研究院 | Industrial cluster energy efficiency optimization method and system based on big data |
| CN117252488B (en) * | 2023-11-16 | 2024-02-09 | 国网吉林省电力有限公司经济技术研究院 | Industrial cluster energy efficiency optimization method and system based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112003870B (en) | Network encryption traffic identification method and device based on deep learning | |
| Cordero et al. | Analyzing flow-based anomaly intrusion detection using replicator neural networks | |
| CN113612763B (en) | Network attack detection device and method based on network security malicious behavior knowledge base | |
| CN110868409A (en) | Passive operating system identification method and system based on TCP/IP protocol stack fingerprint | |
| CN117040824A (en) | Network threat detection method and system | |
| CN114050979B (en) | Industrial control protocol safety test system and device | |
| CN111786951B (en) | Traffic data feature extraction method, malicious traffic identification method and network system | |
| CN114448830B (en) | Equipment detection system and method | |
| CN112165484B (en) | Network encryption traffic identification method and device based on deep learning and side channel analysis | |
| CN111935185B (en) | Method and system for constructing large-scale trapping scene based on cloud computing | |
| CN113206860A (en) | DRDoS attack detection method based on machine learning and feature selection | |
| Janabi et al. | Convolutional neural network based algorithm for early warning proactive system security in software defined networks | |
| US20190124094A1 (en) | Active prioritization of investigation targets in network security | |
| CN116723058B (en) | Network attack detection and protection method and device | |
| CN110574348B (en) | Data processing apparatus and method | |
| CN114205855A (en) | Feeder automation service network anomaly detection method facing 5G slices | |
| Wan et al. | DevTag: A benchmark for fingerprinting IoT devices | |
| US9049170B2 (en) | Building filter through utilization of automated generation of regular expression | |
| CN112929364A (en) | Data leakage detection method and system based on ICMP tunnel analysis | |
| Huabing et al. | Real-time detection method for mobile network traffic anomalies considering user behavior security monitoring | |
| WO2018142704A1 (en) | Feature amount generation device, feature amount generation method, and program | |
| CN117014192A (en) | Data processing method and related equipment | |
| CN113810372B (en) | Low-throughput DNS hidden channel detection method and device | |
| CN116662184B (en) | Industrial control protocol fuzzy test case screening method and system based on Bert | |
| CN114915599B (en) | Dark website point conversation identification method and system based on semi-supervised cluster learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |