CN115643086A - Unknown threat detection method based on deep neural network - Google Patents

Unknown threat detection method based on deep neural network Download PDF

Info

Publication number
CN115643086A
CN115643086A CN202211290952.2A CN202211290952A CN115643086A CN 115643086 A CN115643086 A CN 115643086A CN 202211290952 A CN202211290952 A CN 202211290952A CN 115643086 A CN115643086 A CN 115643086A
Authority
CN
China
Prior art keywords
data
neural network
deep neural
threat detection
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211290952.2A
Other languages
Chinese (zh)
Inventor
王胜
张凌浩
张颉
唐超
王海
张菊玲
向思屿
唐勇
梁晖辉
柴继文
吴�民
熊晓雯
刘涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electric Power Research Institute of State Grid Sichuan Electric Power Co Ltd
Original Assignee
Electric Power Research Institute of State Grid Sichuan Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electric Power Research Institute of State Grid Sichuan Electric Power Co Ltd filed Critical Electric Power Research Institute of State Grid Sichuan Electric Power Co Ltd
Priority to CN202211290952.2A priority Critical patent/CN115643086A/en
Publication of CN115643086A publication Critical patent/CN115643086A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses an unknown threat detection method based on a deep neural network, which comprises the following steps: deploying an information collector on each node of the big data platform, collecting network flow data with different dimensionalities, and carrying out normalization processing on the network flow data to obtain sample set data; constructing a convolutional neural network, and performing optimization training on the convolutional neural network by using a sample set data set based on a deep learning algorithm to obtain an unknown threat detection model; and testing and processing the acquired network flow data through an unknown threat detection model to obtain a corresponding unknown threat detection result. By applying the deep learning technology, the invention can effectively improve the identification success rate of the information system to the known threats and the unknown threats, accurately and timely find the intrusion, protect high-value assets, provide technical support and guarantee for guaranteeing and improving the safe and stable operation of the information system, and avoid the influence of the unknown threats on the safety of the information system to the maximum extent.

Description

Unknown threat detection method based on deep neural network
Technical Field
The invention relates to the technical field of information security and big data, in particular to an unknown threat detection method based on a deep neural network.
Background
With the increasing development and popularization of internet technology and mobile communication technology, large data platforms also face network threats and data security problems. The information exchange quantity between the big data platform and the user is greatly improved, and the data security and the risk prevention in the field of the big data platform are more complex than those of the traditional network. Such as the most common enterprise information systems, are at all times subject to threats and attacks from the outside and from the inside. Therefore, how to improve the detection and prevention capability of the enterprise information system to the external unknown threats is very important.
Currently, intrusion detection techniques commonly used in enterprise information systems include: (1) detection technology based on statistical method: the traditional intrusion detection technology based on a statistical method is the earliest detection method, and the method considers that a 'credible interval' of behavior of a protection subject in the current time period can be obtained by observing the normal behavior of the protection subject in a specific time period in the past, a monitoring system can count the deviation generated by related variables during the protection period, whether the abnormality occurs is judged according to the deviation, if the deviation exceeds a safety threshold value set by the system, the abnormality occurs, and at the moment, the system needs to give an alarm or perform corresponding treatment. (2) detection technology based on hidden Markov model: the Hidden Markov Model (HMM) is good at mining the context of data and can effectively process time series data. Many data in the network security field also belong to time sequence data, such as the call sequence of the system, the operation command sequence, and the like. The key point of HMMs is to determine the implicit parameters of the process from the observable data and then use these parameters for further analysis, such as pattern recognition. The HMM is widely applied in the field of network security to train normal behavior data collected in a system by using a normal behavior training module to form an HMM model capable of describing normal behaviors, and the anomaly detection module compares real-time data in the system with the HMM of a normal event to determine whether an abnormal behavior occurs. (3) detection technology based on a support vector machine: SVMs have also been used by many researchers in recent years for intrusion detection. The essence of the method is that an optimal hyperplane among different types of samples is constructed through a support vector, and the optimal hyperplane is used as a segmentation plane of sample data.
However, the currently used intrusion detection methods generally have the following disadvantages: (1) it is difficult to cope with large data volumes: (2) when the detection system faces a novel attack, the detection system does not have characteristic samples, and the system appears to be stranded when facing the novel attack; (3) the false alarm rate and the false alarm rate are high, the false alarm rate and the alarm rate are the most serious problems of the intrusion detection system, due to the complexity of the network environment and the variety of attack means, the intrusion detection system always has the false alarms with different degrees, and a large amount of information is missed and reported, the efficiency and the performance of the intrusion detection system can be reduced, the energy of personnel is consumed, and the sensitivity is reduced.
Disclosure of Invention
In order to solve the problems, the invention provides an unknown threat detection method based on a deep neural network.
The invention achieves the above purpose through the following technical scheme:
an unknown threat detection method based on a deep neural network comprises the following steps:
the method comprises the following steps: deploying an information collector on each node of the big data platform, collecting network flow data with different dimensionalities, and carrying out normalization processing on the network flow data to obtain sample set data;
step two: constructing a convolutional neural network, and performing optimization training on the convolutional neural network by using a sample set data set based on a deep learning algorithm to obtain an unknown threat detection model;
step three: and testing and processing the acquired network flow data through an unknown threat detection model to obtain a corresponding unknown threat detection result.
In a further improvement, the input of the neural network neuron is x 1 、x 2 、x 3 、x 4 The output is Hw, b (x) = f (WTX), where f is an activation function of the neuron, the activation function including a Sigmoid function, tanh function, and ReLU function;
wherein, sigmoid function:
Figure BDA0003901359270000031
tan h function:
Figure BDA0003901359270000032
ReLU function: f (x) = max (0,x).
In a further refinement, the optimization training comprises
(1) Unsupervised learning from the bottom up: gradually inputting sample set data into a deep neural network structure from an input layer, separating data types from data labels, and performing non-supervised learning without labels;
(2) Top-down supervised fine tuning: and performing discriminative training on the neural network by using the data with the labels from top to bottom, adjusting the output error from top to bottom, and finely adjusting the parameters of each layer so as to realize global optimization.
In a further refinement, the unknown threat detection model includes a data preprocessor, a feature selector, and a deep neural network classifier.
In a further improvement, the data preprocessor reads the training data set and the test data set and the respective categories respectively, and then performs normalization processing.
The further improvement is that the feature selector performs dimensionality reduction on the preprocessed training set data and test set data, removes redundant data, and forms a network traffic data feature set.
The further improvement is that the deep neural network classifier receives the data characteristics and the categories of the training set, the deep neural network is trained, and then the characteristics of the test set are loaded to predict the categories of the training set, so that the data classification prediction result of the test set is obtained.
The further improvement is that the workflow of the deep neural network classifier specifically comprises:
(1) Respectively loading training data and test data into a classifier of the model, wherein the training data enter a training module of the deep neural network classifier, and the test data enter a test module of the deep neural network classifier;
(2) Training the training data by a deep neural network classifier to obtain an effective result, loading the testing data, predicting the class of the testing data set according to the training effective result to obtain a prediction result, namely completing a classification prediction process;
(3) And loading the class marks of the test set to be compared with the predicted value of the deep neural network classifier, and evaluating the model performance according to a confusion matrix method.
In a further refinement, the deep learning is configured to: automatically learning the connection relation between the devices in the network, and generating an alarm when finding abnormal access to the phantom device and device counterfeiting; the network flow behavior characteristics of the equipment in the network are automatically learned, and when abnormal flow behavior characteristics of the equipment occur, an alarm is generated.
The invention has the beneficial effects that: by applying the deep learning technology, the success rate of the information system for identifying known threats and unknown threats can be effectively improved, invasion can be accurately and timely found, high-value assets are protected, data destruction and secret leakage are prevented, the efficiency is improved, safety compliance and safety management work are simpler due to automation and intellectualization, technical support and guarantee are provided for guaranteeing and improving the safe and stable operation of the information system, and the influence of the unknown threats on the safety of the information system is avoided to the greatest extent.
Drawings
FIG. 1 is a flowchart of the operation of an unknown threat detection model;
fig. 2 is a schematic diagram of information collection distributed deployment.
Detailed Description
The present application will now be described in further detail with reference to the drawings, and it should be noted that the following detailed description is given for purposes of illustration only and should not be construed as limiting the scope of the present application, as these numerous insubstantial modifications and variations can be made by those skilled in the art based on the teachings of the present application.
Referring to fig. 1 and fig. 2, a method for detecting an unknown threat based on a deep neural network includes the steps of:
the method comprises the following steps: deploying an information collector on each node of the big data platform, collecting network flow data with different dimensions, and carrying out normalization processing on the network flow data to obtain sample set data;
step two: constructing a convolutional neural network, and performing optimization training on the convolutional neural network by using a sample set data set based on a deep learning algorithm to obtain an unknown threat detection model;
step three: and testing and processing the acquired network flow data through an unknown threat detection model to obtain a corresponding unknown threat detection result.
When information acquisition deploys, 1 set of network intelligent defense management and control platform is deployed beside a headquarter core switch, the platform is composed of 2 high-end devices, and dual-computer hot standby mode deployment is adopted. Each branch organization is respectively provided with 1 set of network intelligent defense management and control platform, which consists of 2 pieces of middle and low-end equipment and is arranged in a dual-machine hot standby mode. The network intelligent defense equipment receives network flow in a port mirror image mode, and realizes access control of the terminal at the location and detection of abnormal/malicious behaviors. The headquarters and each branch adopt an intelligent phantom technology to actively capture abnormal/malicious behaviors and carry out control in linkage admission.
The input of the neural network neuron is x 1 、x 2 、x 3 、x 4 The output is Hw, b (x) = f (WTX), where f is an activation function of the neuron, the activation function including a Sigmoid function, tanh function, and ReLU function;
wherein, sigmoid function:
Figure BDA0003901359270000061
tan h function:
Figure BDA0003901359270000062
ReLU function: f (x) = max (0,x).
The deep neural network algorithm is mainly divided into an input layer, a hidden layer and an output layer. The excellent feature learning process of deep learning depends on the effective training mode of deep learning. There are many learning ways to realize deep learning, but all have three common characteristics, which are deep level, nonlinear and layer-by-layer feature extraction respectively. Training multiple layers of neural network nodes simultaneously results in high time and space complexity, and deviations caused by layered learning are transmitted among layers.
The optimization training comprises
(1) Unsupervised learning from the bottom up: the method comprises the steps of gradually inputting sample set data into a deep neural network structure from an input layer, separating data types from data labels, and carrying out non-supervised learning without labels, is also called a greedy training method, and can obtain parameters of each layer by using sequential training of the method.
(2) Top-down supervised fine tuning: and performing discriminative training on the neural network by using the data with the labels from top to bottom, adjusting the output error from top to bottom, and finely adjusting the parameters of each layer so as to realize global optimization. After the previous unsupervised learning from bottom to top, the labeled data is used for carrying out discriminative training on the neural network from top to bottom, the unsupervised learning process from bottom to top in the first step can only ensure that the weight matrix of each layer is locally optimal but not globally optimal, so the supervised learning process in the second step is needed, the output error is adjusted from top to bottom, and parameters of each layer are finely adjusted, so that the global optimal is realized.
In the invention, the unknown threat detection model comprises a data preprocessor, a feature selector and a deep neural network classifier.
The data preprocessor respectively reads the training data set, the testing data set and the respective categories, and then carries out normalized processing.
And the feature selector performs dimensionality reduction on the preprocessed training set data and test set data, removes redundant data and forms a network traffic data feature set.
And the deep neural network classifier receives the data characteristics and the categories of the training set, and after the deep neural network training, the characteristics of the test set are loaded to predict the categories of the training set, so that a data classification prediction result of the test set is obtained.
The workflow of the deep neural network classifier specifically comprises the following steps:
(1) Respectively loading training data and test data into a classifier of a model, wherein the training data enters a training module of a deep neural network classifier, and the test data enters a test module of the deep neural network classifier;
(2) Training the training data by a deep neural network classifier to obtain an effective result, loading the testing data, predicting the class of the testing data set according to the training effective result to obtain a prediction result, namely completing a classification prediction process;
(3) And (4) loading the class mark of the test set to be compared with the predicted value of the deep neural network classifier, and evaluating the performance of the model according to a confusion matrix method.
The deep learning is configured to: automatically learning the connection relation between the devices in the network, and generating an alarm when finding abnormal access to the phantom device and device counterfeiting; the network flow behavior characteristics of the equipment in the network are automatically learned, and when abnormal flow behavior characteristics of the equipment occur, an alarm is generated.
The functions of the invention include:
(1) Accurate equipment identification and classification management
Intelligent acquisition: based on active and passive information acquisition technologies, equipment information is intelligently acquired.
Intelligent identification: based on equipment portrait technique, intelligent recognition equipment type and producer.
(2) Intelligent admission management
And intelligent admission based on AD domain, email and fingerprint is supported. The authentication mode supports LDAP/RADIUS/AD and the like.
(3) And (3) access compliance detection:
software compliance detection, configuration compliance detection, anonymity detection, NAT equipment detection, violation sharing, equipment access time check and Telnet compliance check.
(4) Detection of aggressive behavior
C & C attacks, doS attacks, brute force attacks, lessovirus, botnet, worms, trojan attacks, network scanning, shellcode attacks, malicious software attacks, authority cracking attacks, video voice protocol attacks and other behaviors are detected. Device spoofing, abnormal connections, abnormal traffic, abnormal protocols, abnormal online time
(5) Vulnerability detection
Weak password detection, vulnerability detection and patch installation condition detection.
(6) Abnormal behavior detection
The method mainly applies a deep learning method to detect unknown attacks, A, automatically learns the connection relation between devices in the network and the behavior of accessing the Internet, automatically constructs a normal access behavior pattern of a user, intelligently discovers abnormal connection, and sends an alarm or automatically blocks the abnormal connection when discovering the abnormal connection. B. The traffic behavior characteristics among devices in the network and the traffic characteristics of accessing the Internet are automatically learned, and when traffic is abnormal within a certain time period, the system can give an alarm or automatically block. C. And discovering the access of the abnormal protocol in time. d. The online time of the equipment is automatically learned, and once the online time of the equipment is found to be abnormal, the alarm can be given or blocked immediately. E. For devices with relatively fixed access positions such as a server, a dumb terminal device, an IoT device and the like, once the access positions of the devices are found to change, the devices can immediately alarm or block. F. And analyzing the suspicious degree of the domain name by adopting a mode of combining machine learning and threat intelligence, and immediately alarming or blocking when a malicious domain name is found. G. The attack of unknown type is discovered, the attack is discovered through a trapping mode, and the attack can be captured without knowing the attack type. H. Unknown types of attacks are discovered through a deep learning method, and the method comprises portrait type discovery, association analysis, suspicious domain name analysis and the like.
(7) Flow analysis
The method supports displaying real-time flow, total flow analysis, active session analysis, host flow analysis, network segment list and internet flow map.
(8) Risk handling and visualization management
Intelligent treatment: to the equipment that has the risk, the system can adopt following several kinds of modes to carry out intelligence according to the factor of safety change condition and deal with: active alarm, network control, third party interface; visual management: and displaying the risk state of the whole network: the system can display the equipment and the risk state of the whole network in a qualitative or quantitative mode according to the severity and distribution condition of the abnormal behavior, the attack behavior, the compliance and the vulnerability of the equipment.
By applying the deep learning technology, the success rate of the information system for identifying known threats and unknown threats can be effectively improved, invasion can be accurately and timely found, high-value assets are protected, data destruction and secret leakage are prevented, the efficiency is improved, safety compliance and safety management work are simpler due to automation and intellectualization, technical support and guarantee are provided for guaranteeing and improving the safe and stable operation of the information system, and the influence of the unknown threats on the safety of the information system is avoided to the greatest extent.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

Claims (9)

1. An unknown threat detection method based on a deep neural network is characterized in that: comprises the steps of
The method comprises the following steps: deploying an information collector on each node of the big data platform, collecting network flow data with different dimensionalities, and carrying out normalization processing on the network flow data to obtain sample set data;
step two: constructing a convolutional neural network, and performing optimization training on the convolutional neural network by using a sample set data set based on a deep learning algorithm to obtain an unknown threat detection model;
step three: and testing and processing the acquired network flow data through an unknown threat detection model to obtain a corresponding unknown threat detection result.
2. The unknown threat detection method based on the deep neural network as claimed in claim 1, characterized in that: the input of the neural network neuron is x 1 、x 2 、x 3 、x 4 The output is Hw, b (x) = f (WTX), where f is the activation function of the neuron,the activation function comprises a Sigmoid function, a tanh function and a ReLU function;
wherein, sigmoid function:
Figure FDA0003901359260000011
tan h function:
Figure FDA0003901359260000012
ReLU function: f (x) = max (0,x).
3. The unknown threat detection method based on the deep neural network as claimed in claim 1, characterized in that: the optimization training comprises
(1) Unsupervised learning from the bottom up: gradually inputting sample set data into a deep neural network structure from an input layer, separating data types from data labels, and performing non-supervised learning without labels;
(2) Top-down supervised fine tuning: and performing discriminative training on the neural network by using the data with the labels from top to bottom, adjusting the output error from top to bottom, and finely adjusting the parameters of each layer so as to realize global optimization.
4. The unknown threat detection method based on the deep neural network as claimed in claim 1, characterized in that: the unknown threat detection model includes a data preprocessor, a feature selector, and a deep neural network classifier.
5. The unknown threat detection method based on the deep neural network as claimed in claim 4, wherein: the data preprocessor respectively reads the training data set, the test data set and respective categories, and then carries out standardized processing.
6. The unknown threat detection method based on the deep neural network as claimed in claim 4, wherein: and the feature selector performs dimensionality reduction on the preprocessed training set data and test set data, removes redundant data and forms a network traffic data feature set.
7. The unknown threat detection method based on the deep neural network as claimed in claim 4, wherein: the deep neural network classifier receives the data characteristics and the categories of the training set, the deep neural network is trained, and then the characteristics of the test set are loaded to predict the categories of the training set, so that the data classification prediction result of the test set is obtained.
8. The unknown threat detection method based on the deep neural network as claimed in claim 7, wherein: the workflow of the deep neural network classifier specifically comprises the following steps:
(1) Respectively loading training data and test data into a classifier of a model, wherein the training data enters a training module of a deep neural network classifier, and the test data enters a test module of the deep neural network classifier;
(2) Training the training data by a deep neural network classifier to obtain an effective result, loading the testing data, predicting the class of the testing data set according to the training effective result to obtain a prediction result, namely completing a classification prediction process;
(3) And (4) loading the class mark of the test set to be compared with the predicted value of the deep neural network classifier, and evaluating the performance of the model according to a confusion matrix method.
9. The unknown threat detection method based on the deep neural network as claimed in claim 1, characterized in that: the deep learning is configured to: automatically learning the connection relation between the devices in the network, and generating an alarm when finding abnormal access to the phantom device and device counterfeiting; the network flow behavior characteristics of the equipment in the network are automatically learned, and when abnormal flow behavior characteristics of the equipment occur, an alarm is generated.
CN202211290952.2A 2022-10-21 2022-10-21 Unknown threat detection method based on deep neural network Pending CN115643086A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211290952.2A CN115643086A (en) 2022-10-21 2022-10-21 Unknown threat detection method based on deep neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211290952.2A CN115643086A (en) 2022-10-21 2022-10-21 Unknown threat detection method based on deep neural network

Publications (1)

Publication Number Publication Date
CN115643086A true CN115643086A (en) 2023-01-24

Family

ID=84944375

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211290952.2A Pending CN115643086A (en) 2022-10-21 2022-10-21 Unknown threat detection method based on deep neural network

Country Status (1)

Country Link
CN (1) CN115643086A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117579400A (en) * 2024-01-17 2024-02-20 国网四川省电力公司电力科学研究院 Industrial control system network safety monitoring method and system based on neural network

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110995652A (en) * 2019-11-04 2020-04-10 中国电子科技网络信息安全有限公司 Big data platform unknown threat detection method based on deep migration learning
CN113422771A (en) * 2021-06-22 2021-09-21 北京华圣龙源科技有限公司 Threat early warning method and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110995652A (en) * 2019-11-04 2020-04-10 中国电子科技网络信息安全有限公司 Big data platform unknown threat detection method based on deep migration learning
CN113422771A (en) * 2021-06-22 2021-09-21 北京华圣龙源科技有限公司 Threat early warning method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
蔡洪民;王庆香;: "基于深度学习的入侵检测技术研究", 网络安全技术与应用, no. 11, 15 November 2017 (2017-11-15), pages 2 - 3 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117579400A (en) * 2024-01-17 2024-02-20 国网四川省电力公司电力科学研究院 Industrial control system network safety monitoring method and system based on neural network
CN117579400B (en) * 2024-01-17 2024-03-29 国网四川省电力公司电力科学研究院 Industrial control system network safety monitoring method and system based on neural network

Similar Documents

Publication Publication Date Title
Hodo et al. Threat analysis of IoT networks using artificial neural network intrusion detection system
US7941855B2 (en) Computationally intelligent agents for distributed intrusion detection system and method of practicing same
US7639806B2 (en) Fingerprinting digital devices using electromagnetic characteristics of their communications
Al-Janabi Pragmatic miner to risk analysis for intrusion detection (PMRA-ID)
CN117081868B (en) Network security operation method based on security policy
Chen et al. An effective metaheuristic algorithm for intrusion detection system
CN113904795A (en) Rapid and accurate flow detection method based on network security probe
CN115643086A (en) Unknown threat detection method based on deep neural network
Lee et al. AI-based network security enhancement for 5G industrial Internet of things environments
CN112287345A (en) Credible edge computing system based on intelligent risk detection
CN116366319A (en) Method and system for detecting network security
Zhang et al. Research on unknown threat detection method of information system based on deep learning
Agrawal et al. A SURVEY ON ATTACKS AND APPROACHES OF INTRUSION DETECTION SYSTEMS.
CN113468555A (en) Method, system and device for identifying client access behavior
Suresh et al. Detection of malicious activities by AI-Supported Anomaly-Based IDS
CN116743507B (en) Intrusion detection method and system based on intelligent door lock
Moudoud et al. Enhancing Open RAN Security with Zero Trust and Machine Learning
Sharma et al. Advances and challenges in cryptography using artificial intelligence
CN117544420B (en) Fusion system safety management method and system based on data analysis
Youm et al. An authorized access attack detection method for realtime intrusion detection system
CN117118745B (en) Network security dynamic early warning system based on deep learning
Wen [Retracted] Design of Network Intrusion Detection Model Based on TCA
Cai Research on Network Traffic Anomaly Detection Method Based on Python
Selvam et al. An Improving Intrusion Detection Model Based on Novel CNN Technique Using Recent CIC-IDS Datasets
CN116938525A (en) Novel electric power business terminal safety identification method based on LeNet-5 model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination