CN117201275A - Internet threat information monitoring system and method based on big data - Google Patents
Internet threat information monitoring system and method based on big data Download PDFInfo
- Publication number
- CN117201275A CN117201275A CN202311209181.4A CN202311209181A CN117201275A CN 117201275 A CN117201275 A CN 117201275A CN 202311209181 A CN202311209181 A CN 202311209181A CN 117201275 A CN117201275 A CN 117201275A
- Authority
- CN
- China
- Prior art keywords
- information
- edge computing
- monitoring
- computing node
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 173
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000005259 measurement Methods 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 7
- 238000007670 refining Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an Internet threat information monitoring system and method based on big data, comprising an edge layer and a platform layer; the platform layer and the edge layer form a topological structure; the edge layer comprises edge computing nodes; the edge computing node is provided with a monitoring unit; each edge computing node is in signal connection with at least one edge computing node; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the monitoring unit of each edge computing node is in signal connection with the monitoring unit of the cooperative node; the platform layer comprises a central cloud platform; the central cloud platform is provided with a monitoring center. The monitoring unit acquires fault and network security threat information of the edge computing server system and the cooperative nodes in real time, and reports the fault and network security threat information to the edge computing node system state monitoring center cloud platform, so that the cooperative monitoring capability among the edge computing nodes is realized.
Description
Technical Field
The invention relates to the technical field of information security, in particular to an Internet threat information monitoring system and method based on big data.
Background
With the development of 5G networks, for three special requirements of high bandwidth (emmbb), low latency (URLLC), and wide connection (mctc), a large amount of computing storage resources of edge computing servers (nodes) need to be deployed at the edge side to assist in achieving the above three special requirements of 5G networks.
The edge nodes can realize cloud edge cooperation with the central cloud platform and can also realize connection and intercommunication of adjacent edge nodes. Especially for the scene related to low time delay and wide connection, the edge nodes use more resources for processing the accessed edge side terminal information and distributing and self-adaptive scheduling the resources between the adjacent edge nodes due to the huge number of the edge nodes, the interaction frequency with the central cloud platform is not high, and meanwhile, the central cloud platform cannot support frequent interaction of a large number of the edge nodes. Therefore, when the edge computing node system is subjected to malicious attack or fails to lose connection with the central cloud platform, the central cloud platform cannot discover the failure in time or acquire detailed results of related failure or threat detection in time, and therefore operation and maintenance personnel are affected to perform failure diagnosis on the related system. Particularly, when the edge computing node is located in a remote place (such as some mountain mobile communication base stations), a great deal of time and labor are required to carry out diagnosis and maintenance on site, and if the detailed result of related fault or threat detection cannot be timely obtained, the service application of the edge terminal (such as unmanned driving supported by the edge computing technology) is greatly influenced.
Disclosure of Invention
The invention provides an Internet threat information monitoring system and method based on big data, which are used for solving the problems in the prior art.
The invention provides an internet threat information monitoring method based on big data, which comprises the following steps:
s100, connecting a central cloud platform with at least one edge computing node by each edge computing node signal; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
s200, a monitoring unit of the edge computing node monitors and analyzes data of the edge computing node in real time, and records the detected and diagnosed security threat and fault information;
s300, a monitoring unit of the edge computing node communicates with a monitoring unit of the cooperative node, and sends a request for acquiring system fault information and network security threat information of the cooperative node; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
S400, judging whether a monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if not: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
s500, the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
Preferably, in the step S100, the method for determining the cooperative node is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
Preferably, the method further comprises:
s600, the monitoring center compares the reported information with the recently received data and judges the repeatability of the reported information; according to the comparison result, if the repeated representation of the information has been reported by other cooperative nodes, step S700 is entered; otherwise, step S800 is entered;
S700, directly discarding repeated information without treatment;
s800, updating the reported information of the monitoring center into a center cloud platform, and sending monitoring result alarm information to a user;
s900, the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
Preferably, in the S600, for each system fault information and network security threat information found by monitoring, a unique identification code of an edge computing node, a time for information acquisition, and a specific content of the information are encoded to form a unique fingerprint information code; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
the unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
the specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
Preferably, in S600, the step of determining the repeatability of the reported information by the monitoring center is as follows:
extracting information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, uniformly converting the information data into a data format with a unique identifier number, and creating a corresponding feature library and a feature value table;
according to the value set of one feature of the information data, other information data with the same feature value as the feature is found out, and the other information data is called synchronous information data;
setting a definition information data set;
and creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all characteristic values of the characteristics by each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
The invention provides an Internet threat information monitoring system based on big data, comprising: edge computing nodes and a central cloud platform;
each edge computing node signal connects the central cloud platform with at least one edge computing node; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
The monitoring unit of the edge computing node monitors and analyzes the data of the edge computing node in real time and records the detected and diagnosed security threat and fault information;
the monitoring unit of the edge computing node is communicated with the monitoring unit of the cooperative node, and a request for acquiring system fault information and network security threat information of the cooperative node is sent; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
judging whether the monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if no: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
and the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
Preferably, the process of cooperating nodes is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
Preferably, the method further comprises:
the judging unit is used for comparing the reported information with the recently received data by the monitoring center and judging the repeatability of the reported information;
the result obtaining unit is used for directly discarding repeated information without treatment if the repeated information is repeatedly indicated to be reported by other cooperative nodes according to the comparison result; if the comparison result is that the report information of the monitoring center is not updated, the report information enters the center cloud platform, and warning information of the monitoring result is sent to the user; and the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
Preferably, in the judging unit, for each system fault information and network security threat information found by monitoring, a unique identification code of an edge computing node, time for information acquisition and specific content of the information are encoded to form a unique fingerprint information code; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
the unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
the specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
Preferably, the determining unit further includes:
the characteristic block creation subunit is used for refining the information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, unifying the information data into a data format with a unique identifier number, and creating a corresponding characteristic library and a characteristic value table;
A data synchronization subunit, configured to find other information data having the same feature value as the feature according to the value set of the feature of the information data, and refer to these other information data as synchronization information data;
a data set setting subunit for setting a definition information data set;
and the data numbering subunit is used for creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all the characteristic values of the characteristics in each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
Compared with the prior art, the invention has the following advantages:
the invention provides an Internet threat information monitoring system and method based on big data, wherein a monitoring unit of system faults and network security threats is deployed into each edge computing server node, the monitoring unit is used for acquiring the system-related faults and network security threat information of the edge computing node in real time, simultaneously requesting to acquire the system faults and network security threat information of 2 cooperative nodes with the node in real time, and then reporting the acquired faults and threat information to a cloud platform of a system state monitoring center of the edge computing node, so that the cooperative monitoring capability among the edge computing nodes is realized. In addition, if the network fault of the adjacent node cannot be acquired, the adjacent node can be judged to be the network interruption fault, and the node information is reported to the central cloud platform. The method and the system realize multi-node collaborative monitoring of system faults and network security threat information of the edge computing nodes, ensure stable and safe operation of the edge computing nodes, and solve the problems that the problem is difficult to trace to source and repair in time due to out-of-control of the edge nodes caused by the fact that the edge nodes are separated from an edge cloud collaborative network due to faults or network attacks.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flow chart of an Internet threat information monitoring method based on big data in an embodiment of the invention;
FIG. 2 is a flow chart of another method for monitoring Internet threat information for big data in accordance with an embodiment of the invention;
fig. 3 is a schematic structural diagram of an internet threat information monitoring system based on big data in an embodiment of the invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
The embodiment of the invention provides an internet threat information monitoring method for big data, referring to fig. 1, the method comprises the following steps:
s100, connecting a central cloud platform with at least one edge computing node by each edge computing node signal; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
s200, a monitoring unit of the edge computing node monitors and analyzes data of the edge computing node in real time, and records the detected and diagnosed security threat and fault information;
s300, a monitoring unit of the edge computing node communicates with a monitoring unit of the cooperative node, and sends a request for acquiring system fault information and network security threat information of the cooperative node; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
S400, judging whether a monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if not: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
s500, the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
The working principle of the technical scheme is as follows: the scheme adopted by the embodiment is that each edge computing node signal connects a central cloud platform with at least one edge computing node; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit; the monitoring unit of the edge computing node monitors and analyzes the data of the edge computing node in real time and records the detected and diagnosed security threat and fault information; the monitoring unit of the edge computing node is communicated with the monitoring unit of the cooperative node, and a request for acquiring system fault information and network security threat information of the cooperative node is sent; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node; judging whether the monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering the steps of: and the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform. And (3) no: and the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
The monitoring system based on the edge multipoint cooperation fault or safety threat comprises an application layer, an edge layer and a platform layer; the platform layer and the edge layer form a topological structure.
The application layer comprises an edge user terminal, wherein the edge user terminal is intelligent terminal equipment, and the intelligent terminal equipment comprises, but is not limited to, a mobile phone, a computer and a traffic signal lamp.
The edge layer comprises edge computing nodes; and the edge computing node is in signal connection with the edge user terminal and collects and analyzes data uploaded by the edge user terminal.
The edge computing node is provided with a monitoring unit; the monitoring unit acquires system fault information and safety threat information of the edge computing node and comprises a system fault monitoring function module and a network safety threat monitoring function module.
The system fault monitoring function module collects and analyzes the system logs of the nodes, namely: the method and the system realize the acquisition and analysis of fault logs of windows and Linux operating system levels (the log types mainly comprise kernel and system logs, user logs and program use logs) so as to confirm the fault information of the edge computing node.
The network security threat monitoring functional module is used for collecting and analyzing node flow, analyzing and restoring node data and tracing analysis threat events, namely: analyzing the flow entering and exiting the edge computing node, restoring the data, dynamically analyzing the restored file to identify a malicious file, and analyzing the security threat behavior of the restored request event to identify a security threat event.
Each edge computing node is connected with at least one edge computing node in a signal manner; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the monitoring unit of each edge computing node is in signal connection with the monitoring unit of the cooperative node.
The beneficial effects of the technical scheme are as follows: the scheme provided by the embodiment is adopted to deploy the monitoring units of the system faults and the network security threats into the edge computing server nodes, the monitoring units are used for acquiring the system related faults and the network security threat information of the edge computing nodes in real time, simultaneously, the system faults and the network security threat information of 2 cooperative nodes with the nodes are also requested to be acquired in real time, and then the acquired fault and threat information is reported to the edge computing node system state monitoring center cloud platform, so that the cooperative monitoring capability among the edge computing nodes is realized. In addition, if the network fault of the adjacent node cannot be acquired, the adjacent node can be judged to be the network interruption fault, and the node information is reported to the central cloud platform. The method and the system realize multi-node collaborative monitoring of system faults and network security threat information of the edge computing nodes, ensure stable and safe operation of the edge computing nodes, and solve the problems that the problem is difficult to trace to source and repair in time due to out-of-control of the edge nodes caused by the fact that the edge nodes are separated from an edge cloud collaborative network due to faults or network attacks.
In another embodiment, the method for determining the cooperative node is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
The working principle of the technical scheme is as follows: the scheme adopted in the embodiment is that the method for determining the cooperative node is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
The beneficial effects of the technical scheme are as follows: by adopting the scheme provided by the embodiment, through a data polling detection mode, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, generally perform network speed measurement for about 10 times, perform average calculation on speed measurement results, and take two adjacent edge computing nodes 2 with the fastest average speed measurement results as cooperative nodes for cooperative monitoring of the edge computing nodes.
For example, edge computing node a finds, by averaging, that the data interaction time with both edge computing node B and edge computing node D is shorter than edge computing node C, so edge computing node a selects edge computing nodes B and D as cooperative nodes. Similarly, by pairing every two edge computing nodes B select edge computing nodes A and C as cooperative nodes, edge computing node C selects edge computing nodes B and D as cooperative nodes, edge computing node D selects edge computing nodes A and C as cooperative nodes, and finally the architecture of the monitoring system based on edge multipoint cooperation designed by the scheme is formed.
In another embodiment, referring to fig. 2, the method further includes:
S600, the monitoring center compares the reported information with the recently received data and judges the repeatability of the reported information; according to the comparison result, if the repeated representation of the information has been reported by other cooperative nodes, step S700 is entered; otherwise, step S800 is entered;
s700, directly discarding repeated information without treatment;
s800, updating the reported information of the monitoring center into a center cloud platform, and sending monitoring result alarm information to a user;
s900, the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
The working principle of the technical scheme is as follows: the scheme adopted by the embodiment is to ensure that only one part of the same data reported by the plurality of edge computing nodes in cooperation is collected and stored, the central cloud platform identifies and filters repeated information of the unique fingerprint information code of the reported information, the reported information records are classified and stored, the data records of the unique fingerprint information code under the same classification are called for repeated comparison and identification, and the repeated data of the unique fingerprint information code are discarded.
In another embodiment, aiming at each system fault information and network security threat information found by monitoring, a unique identification code of an edge computing node, the time for information acquisition and the specific content of the information are encoded to form a unique fingerprint information code; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
The unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
the specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
In another embodiment, in S600, the step of determining the repeatability of the reported information by the monitoring center is as follows:
extracting information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, uniformly converting the information data into a data format with a unique identifier number, and creating a corresponding feature library and a feature value table;
according to the value set of one feature of the information data, other information data with the same feature value as the feature is found out, and the other information data is called synchronous information data;
setting a definition information data set;
and creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all characteristic values of the characteristics by each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
The beneficial effects of the technical scheme are as follows: by adopting the scheme provided by the embodiment, according to the steps, the method judges whether the two reported information data are repeated or not, namely, whether the similarity is larger than a set threshold U or not, namely, the sum of all the relative similarities is larger than the threshold U. The similarity can be seen as the final result of judging whether the two information data are synchronous information data or not in each feature, if so, adding weight, and not adding 0.
The judging method is to compare the new information data with all the old information data, and if one comparison calculation amount is larger. And the two information data are asynchronous information data in a certain characteristic, the relative similarity of the two information data is 0, and the relative similarity is found out to be not 0 and the sum of the two information data is the similarity of the two information data. Meanwhile, only synchronous information data with the relative similarity not being 0 can find corresponding synchronous information data through the established characteristic relation library, and the similarity is found by utilizing the synchronous information data. Because the synchronization information data is very small in relation to all the information data, the calculation amount can be greatly reduced by utilizing the characteristic.
In another embodiment, the present invention provides an internet threat information monitoring system based on big data, please refer to fig. 3, the system comprising: edge computing nodes and a central cloud platform;
Each edge computing node signal connects the central cloud platform with at least one edge computing node; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
the monitoring unit of the edge computing node monitors and analyzes the data of the edge computing node in real time and records the detected and diagnosed security threat and fault information;
the monitoring unit of the edge computing node is communicated with the monitoring unit of the cooperative node, and a request for acquiring system fault information and network security threat information of the cooperative node is sent; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
Judging whether the monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if no: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
and the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
In another embodiment, the process of coordinating nodes is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
In another embodiment, the method further comprises:
the judging unit is used for comparing the reported information with the recently received data by the monitoring center and judging the repeatability of the reported information;
The result obtaining unit is used for directly discarding repeated information without treatment if the repeated information is repeatedly indicated to be reported by other cooperative nodes according to the comparison result; if the comparison result is that the report information of the monitoring center is not updated, the report information enters the center cloud platform, and warning information of the monitoring result is sent to the user; and the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
In another embodiment, in the judging unit, for each piece of system fault information and network security threat information found by monitoring, a unique fingerprint information code is formed by encoding a unique identification code of an edge computing node, information acquisition time and information specific content; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
the unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
The specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
In another embodiment, the determining unit further includes:
the characteristic block creation subunit is used for refining the information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, unifying the information data into a data format with a unique identifier number, and creating a corresponding characteristic library and a characteristic value table;
a data synchronization subunit, configured to find other information data having the same feature value as the feature according to the value set of the feature of the information data, and refer to these other information data as synchronization information data;
a data set setting subunit for setting a definition information data set;
and the data numbering subunit is used for creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all the characteristic values of the characteristics in each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
In conclusion, the technical scheme has the advantages that:
1, identifying two adjacent nearest nodes of each edge computing node through an edge network deployment architecture. The identifying, by the edge network deployment architecture, two nodes that are nearest to each edge computing node, comprising: in the deployment of the edge computing nodes, the physical networks are close, and the network communication among the edge computing nodes ensures that each node participates in collaborative monitoring by three edge computing nodes including the node itself.
2, identifying system faults and network security threat information of the self node through an edge computing built-in monitoring unit, and acquiring system faults or network security threat information of adjacent edge computing nodes, wherein the system faults or network security threat information comprises periodic collection of fault information of the edge computing system; detecting the network request flow data of the edge computing node in real time, and finding out malicious files and network security malicious request events in the flow; according to the request of the adjacent edge computing node, the monitored system fault information and the network security threat information are returned to the requesting node; and sending a monitoring information acquisition request to the adjacent edge computing node to acquire own system fault information and network security threat information monitored by the adjacent edge computing node.
And 3, completing the data reporting of the central cloud platform by utilizing a fault and security event driven mode, and reducing frequent interaction between the edge node and the cloud platform.
And 4, rapidly judging the repeatability of the reported information by utilizing the event fingerprint, and filtering the repeated data reported by a plurality of cooperative edge computing nodes, wherein the method comprises the following steps: aiming at each system fault information and network security threat information found by monitoring, unique fingerprint information is formed by encoding a unique identification code of an edge computing node, information acquisition time and specific content of the information; reporting each piece of system fault information and network security threat information found by monitoring together with fingerprint information thereof; and identifying and filtering repeated information of the reported information through fingerprint information, so that only one part of the same data reported by the plurality of edge computing nodes in a cooperated manner is collected and stored. And the reported information records are classified and stored, a corresponding data characteristic relation library is established, the data records under the same classification are called for comparison, the algorithm is guaranteed not to repeatedly compare the data records subjected to discarding treatment, the data calculation amount of the cloud platform is greatly reduced while the duplicate removal efficiency is improved, and the resource utilization rate of the central cloud platform is indirectly improved.
And 5, the unique fingerprint information is formed by combining unique identification codes of edge computing nodes, time for information acquisition and specific content of the information, the method is encoded, the international universal unique identification code (UUID) standard is consulted, optimization and improvement are carried out, the edge computing node identification codes are defined by self, the information acquisition time of the edge nodes is acquired, the information content (mainly comprising adjacent side edge node identification codes and system fault/network security threat information) is simplified, encryption is carried out through MD5 before data transmission, and the uniqueness of the fingerprint information reported by the edge nodes is ensured.
And 6, the unique fingerprint information code in the scheme fully ensures the uniqueness of the information uploaded by the node and the accuracy of the information content, and converts the plaintext into the ciphertext through an encryption means so that the ciphertext cannot be identified by general personnel, thereby ensuring the confidentiality of the transmitted data. Meanwhile, the balance of memory capacity and encryption is considered, four output ciphertext forms of 16 bits (English lowercase), 32 bits (English lowercase) and 32 bits (English uppercase) are provided, and the output ciphertext forms can be switched according to actual use conditions during deployment.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. The Internet threat information monitoring method based on big data is characterized by comprising the following steps:
s100, connecting a central cloud platform with at least one edge computing node by each edge computing node signal; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
s200, a monitoring unit of the edge computing node monitors and analyzes data of the edge computing node in real time, and records the detected and diagnosed security threat and fault information;
s300, a monitoring unit of the edge computing node communicates with a monitoring unit of the cooperative node, and sends a request for acquiring system fault information and network security threat information of the cooperative node; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
S400, judging whether a monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if not: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
s500, the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
2. The big data based internet threat information monitoring method of claim 1, wherein in S100, the method for determining the cooperative node is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
3. The big data based internet threat information monitoring method of claim 1, further comprising:
S600, the monitoring center compares the reported information with the recently received data and judges the repeatability of the reported information; according to the comparison result, if the repeated representation of the information has been reported by other cooperative nodes, step S700 is entered; otherwise, step S800 is entered;
s700, directly discarding repeated information without treatment;
s800, updating the reported information of the monitoring center into a center cloud platform, and sending monitoring result alarm information to a user;
s900, the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
4. The internet threat information monitoring method based on big data according to claim 3, wherein in S600, for each system fault information and network security threat information discovered by monitoring, a unique fingerprint information code is formed by encoding a unique identification code of an edge computing node, time of information acquisition, and specific content of information; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
The unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
the specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
5. The internet threat information monitoring method based on big data according to claim 3, wherein in S600, the step of determining the repeatability of the reported information by the monitoring center is as follows:
extracting information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, uniformly converting the information data into a data format with a unique identifier number, and creating a corresponding feature library and a feature value table;
according to the value set of one feature of the information data, other information data with the same feature value as the feature is found out, and the other information data is called synchronous information data;
setting a definition information data set;
and creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all characteristic values of the characteristics by each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
6. Internet threat information monitoring system based on big data, which is characterized by comprising: edge computing nodes and a central cloud platform;
each edge computing node signal connects the central cloud platform with at least one edge computing node; each edge computing node is a cooperative node of the edge computing nodes connected with the signal of the edge computing node; the edge computing node is provided with a monitoring unit; the central cloud platform is provided with a monitoring center; the central cloud platform is in signal connection with all edge computing nodes; the monitoring center is in signal connection with the monitoring unit;
the monitoring unit of the edge computing node monitors and analyzes the data of the edge computing node in real time and records the detected and diagnosed security threat and fault information;
the monitoring unit of the edge computing node is communicated with the monitoring unit of the cooperative node, and a request for acquiring system fault information and network security threat information of the cooperative node is sent; after the monitoring unit of the cooperative node acquires the request, the system fault information and the network security threat information of the node where the monitoring unit of the cooperative node is positioned are returned to the request unit; in addition, the monitoring unit of the edge computing node requests to acquire system fault information and network security threat information of the cooperative node, and if the network interruption request is found, the network interruption result is used as the fault information of the cooperative node;
Judging whether the monitoring unit of the edge computing node acquires information of faults and security threats, if yes, entering a step S500, and if no: the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered;
and the monitoring unit of the edge computing node reports the acquired system fault information and network security threat information to a monitoring center of the central cloud platform.
7. The big data based internet threat information monitoring system of claim 6, wherein the process of cooperating nodes is as follows: selecting a plurality of physically adjacent edge computing nodes of each edge computing node, and ensuring that each edge computing node has at least two or more adjacent edge computing nodes; then, by means of data polling detection, the edge computing nodes send network speed measurement data requests to each physically adjacent edge computing node, average calculation is carried out on speed measurement results, and two adjacent edge computing nodes with the fastest average speed measurement results are used as cooperative nodes for cooperative monitoring of the edge computing nodes.
8. The big data based internet threat information monitoring system of claim 6, further comprising:
The judging unit is used for comparing the reported information with the recently received data by the monitoring center and judging the repeatability of the reported information;
the result obtaining unit is used for directly discarding repeated information without treatment if the repeated information is repeatedly indicated to be reported by other cooperative nodes according to the comparison result; if the comparison result is that the report information of the monitoring center is not updated, the report information enters the center cloud platform, and warning information of the monitoring result is sent to the user; and the primary system fault and network security monitoring execution process of the edge computing node is completed, and then the next monitoring period is entered.
9. The internet threat information monitoring system based on big data according to claim 8, wherein in the judging unit, for each system fault information and network security threat information discovered by monitoring, a unique fingerprint information code is formed by encoding a unique identification code of an edge computing node, time of information acquisition and specific information content; the unique fingerprint information code comprises a unique identification code of an edge computing node, information acquisition time and specific information content, and the output coding format is as follows: unique identification code-information acquisition time-information specific content;
The unique identification code is used for identifying the edge computing node reporting the information and can be defined by the user;
the information acquisition time takes the current date and time when the edge computing node collects the information data of the cooperative node and finishes processing as a record;
the specific content of the information is used for representing specific system fault information and network security threat information discovered by monitoring, and the information mainly comprises a problem node identification code and problem information.
10. The big data based internet threat information monitoring system of claim 3, wherein the determining unit further comprises:
the characteristic block creation subunit is used for refining the information data of the unique fingerprint information code according to the unique fingerprint information code reported by the edge computing node, unifying the information data into a data format with a unique identifier number, and creating a corresponding characteristic library and a characteristic value table;
a data synchronization subunit, configured to find other information data having the same feature value as the feature according to the value set of the feature of the information data, and refer to these other information data as synchronization information data;
a data set setting subunit for setting a definition information data set;
And the data numbering subunit is used for creating a characteristic relation table according to the information data, dividing all the characteristics into different characteristic libraries, recording all the characteristic values of the characteristics in each characteristic library, merging the characteristic values into a table, and storing numbers of all the information data with the characteristic values in each characteristic value table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311209181.4A CN117201275A (en) | 2023-09-19 | 2023-09-19 | Internet threat information monitoring system and method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311209181.4A CN117201275A (en) | 2023-09-19 | 2023-09-19 | Internet threat information monitoring system and method based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117201275A true CN117201275A (en) | 2023-12-08 |
Family
ID=89005017
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311209181.4A Pending CN117201275A (en) | 2023-09-19 | 2023-09-19 | Internet threat information monitoring system and method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117201275A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190104138A1 (en) * | 2017-10-04 | 2019-04-04 | New Context Services, Inc. | Autonomous edge device for monitoring and threat detection |
| CN112688822A (en) * | 2021-02-07 | 2021-04-20 | 浙江御安信息技术有限公司 | Edge computing fault or security threat monitoring system and method based on multi-point cooperation |
| CN112804348A (en) * | 2021-02-07 | 2021-05-14 | 浙江御安信息技术有限公司 | Method for judging repeatability of reported data of edge computing node by cloud monitoring center |
| CN113709114A (en) * | 2021-08-05 | 2021-11-26 | 浪潮云信息技术股份公司 | Edge node safety monitoring method under edge computing scene |
| CN116192459A (en) * | 2022-12-29 | 2023-05-30 | 浙江御安信息技术有限公司 | Edge node network security threat monitoring method based on edge-to-edge cooperation |
| CN116455607A (en) * | 2022-12-09 | 2023-07-18 | 苏州鹰目电子科技有限公司 | Edge node network protection system and method based on edge calculation |
-
2023
- 2023-09-19 CN CN202311209181.4A patent/CN117201275A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190104138A1 (en) * | 2017-10-04 | 2019-04-04 | New Context Services, Inc. | Autonomous edge device for monitoring and threat detection |
| CN112688822A (en) * | 2021-02-07 | 2021-04-20 | 浙江御安信息技术有限公司 | Edge computing fault or security threat monitoring system and method based on multi-point cooperation |
| CN112804348A (en) * | 2021-02-07 | 2021-05-14 | 浙江御安信息技术有限公司 | Method for judging repeatability of reported data of edge computing node by cloud monitoring center |
| CN113709114A (en) * | 2021-08-05 | 2021-11-26 | 浪潮云信息技术股份公司 | Edge node safety monitoring method under edge computing scene |
| CN116455607A (en) * | 2022-12-09 | 2023-07-18 | 苏州鹰目电子科技有限公司 | Edge node network protection system and method based on edge calculation |
| CN116192459A (en) * | 2022-12-29 | 2023-05-30 | 浙江御安信息技术有限公司 | Edge node network security threat monitoring method based on edge-to-edge cooperation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112688822B (en) | Edge computing fault or security threat monitoring system and method based on multi-point cooperation | |
| CN112804348B (en) | Method for judging repeatability of reported data of edge computing node by cloud monitoring center | |
| CN108964995A (en) | Log correlation analysis method based on time shaft event | |
| CN107872457B (en) | Method and system for network operation based on network flow prediction | |
| CN113553210A (en) | Alarm data processing method, device, equipment and storage medium | |
| CN109150869B (en) | Switch information acquisition and analysis system and method | |
| CN112769605B (en) | Heterogeneous multi-cloud operation and maintenance management method and hybrid cloud platform | |
| EP2907085A2 (en) | Autonomic network sentinels | |
| CN114363151A (en) | Fault detection method and device, electronic equipment and storage medium | |
| CN117221088A (en) | Computer network intensity detection system and device | |
| CN110609761B (en) | Method and device for determining fault source, storage medium and electronic equipment | |
| CN113411209A (en) | Distributed password service full-link detection system and method | |
| CN110224872B (en) | Communication method, device and storage medium | |
| CN117201275A (en) | Internet threat information monitoring system and method based on big data | |
| CN115484326A (en) | Method, system and storage medium for processing data | |
| CN115664992A (en) | Network operation data processing method and device, electronic equipment and medium | |
| CN114598480A (en) | Method and system for processing machine data of network security operation platform | |
| CN115114316A (en) | Processing method, device, cluster and storage medium for high-concurrency data | |
| CN113852984A (en) | Wireless terminal access monitoring system and method, electronic equipment and readable storage device | |
| EP3756310B1 (en) | Method and first node for managing transmission of probe messages | |
| CN111988172A (en) | Network information management platform, device and security management method | |
| CN110995500A (en) | Node log management and control method, system and related components | |
| CN113890814B (en) | Fault perception model construction and fault perception method and system, equipment and medium | |
| CN115686381B (en) | Prediction method and device for storage cluster running state | |
| CN110225543B (en) | Mobile terminal software quality situation perception system and method based on network request data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |