CN113591134B - Threat intelligence big data sharing method and system - Google Patents
Threat intelligence big data sharing method and system Download PDFInfo
- Publication number
- CN113591134B CN113591134B CN202111139244.4A CN202111139244A CN113591134B CN 113591134 B CN113591134 B CN 113591134B CN 202111139244 A CN202111139244 A CN 202111139244A CN 113591134 B CN113591134 B CN 113591134B
- Authority
- CN
- China
- Prior art keywords
- dbs
- data
- tables
- authority
- homology
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/604—Tools and structures for managing or administering access control systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2141—Access rights, e.g. capability lists, access control lists, access tables, access matrices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The method and the system for sharing the big threat intelligence data acquire the big intelligence data from a server cluster, calculate the homology of each table to construct a data tree, distribute the sharing authority of each table by the data tree, further determine the access sequence according to the sharing authority, and achieve the beneficial effect of effectively processing the exclusive authority distribution of each table in a database.
Description
Technical Field
The disclosure belongs to the technical field of data security, and particularly relates to a threat intelligence big data sharing method and system.
Background
The application of technologies such as big data, internet of things, cloud computing and mobile internet is gradually popularized due to informatization of modern society and high-speed construction of the big data, and threat intelligence big data is structured database information data with exclusivity and encryption in the process of social management. The situation that the mass production and large-scale storage of big intelligence data are more complicated than the past is threatened, and the new information safety problem is continuously emerged in the process of sharing and transmitting the big intelligence data. Patent document No. CN109344941A discloses a method and apparatus for secure sharing of big data, which can not effectively handle exclusive right assignment of each table in a database, although cross-system and cross-platform data sharing can be performed to realize data sharing between different platforms.
Disclosure of Invention
The invention aims to provide a method and a system for sharing big data of threat intelligence, which are used for solving one or more technical problems in the prior art and at least providing a beneficial selection or creation condition.
The situation that the mass generation and large-scale storage of big intelligence data are more complicated than the prior situation is threatened, new information safety problems are continuously generated in the process of sharing and transmitting the big intelligence data, and the exclusive authority distribution of each table in a database needs to be effectively processed.
The method comprises the steps of obtaining big intelligence data from a server cluster, calculating the homology of each table to construct a data tree, distributing the sharing authority of each table by the data tree, and determining an access sequence according to the sharing authority.
In order to achieve the above object, according to an aspect of the present disclosure, there is provided a threat intelligence big data sharing method, the method including:
s100, acquiring big intelligence data from a server cluster, wherein the big intelligence data are a plurality of tables, and the tables are objects used for storing data in a database;
s200, calculating the homology among the tables;
s300, constructing a data tree according to the homology of each table;
s400, distributing sharing authorities of all tables by using a data tree;
s500, determining an access sequence according to the sharing authority.
Further, in S100, the method for obtaining the big intelligence data from the server cluster, where the big intelligence data is a plurality of tables, and the tables are objects used for storing data in the database includes: acquiring big intelligence data from the server cluster, wherein the big intelligence data is text data obtained by identifying the text content on pictures and photos through user input, web crawler software crawling or OCR (optical character recognition) and is stored in a plurality of tables of a database on the server cluster, a collection of tables for storing informative big data, the data in said tables organized in a format of rows and columns, is denoted Dbs, wherein, one row of the table is a record, each serial number recorded in the table is the serial number of the row, each column of the table has the serial number of the column in the table and the column name of the column, the column name is a character string, an element defining a column in a certain row of a table is called a data field of the table, the data field is a character string, the data field has the serial number of the row where the data field is located and the column name of the column where the data field is located, and the table obtains the modification time of the table by reading the record of the data field in the server cluster; let n represent the number of elements in the set Dbs, represent the sequence number of the elements in the set Dbs by variable i, i belongs to [1, n ], and take the element with the sequence number i in the set Dbs as a table Dbs _ i; the number of rows of the element table Dbs _ i with sequence number i in the set Dbs is row _ i, the number of columns of the element table Dbs _ i with sequence number i in the set Dbs is col _ i, the sequence number of a row in any table in the set Dbs is represented by variable k, and k e [1, row _ i ] exists in the table Dbs _ i in which it is located, the row with sequence number k in a row in the table Dbs _ i is denoted Dbs _ i (k), the sequence number of a column in any table in the set Dbs is represented by variable q, and q e [1, col _ i ] exists in the table Dbs _ i in which it is located, the column with sequence number q in the table Dbs _ i is denoted as Dbs _ i (, q), the column of Dbs _ i (, q) in the table Dbs _ i is denoted as Dbs _ i (, q) _ col, the column in _ i in the table Dbs _ i is denoted as Dbs _ i, the data k in the column of the Dbs _ i is denoted as Dbs _ i (, q) in the column in the table Dbs _ i), q) _ col.
Further, in S200, the method for calculating the degree of homology between the tables is: defining the degree of homology as representing the degree of coincidence between data contained in two tables, recording a function equals () as a function for judging whether two character strings are the same, if the two character strings are the same, outputting the function equals () as a value 1, otherwise outputting a value 0, taking any two serial numbers from [1, n ] as a, b, or respectively recording the serial numbers of two tables to be calculated for the degree of homology as a, b, obtaining a table Dbs _ a and a table Dbs _ b, making a function Lap () as a function for calculating the degree of homology between the two tables, wherein the calculation process of the function Lap () is as follows:
s201, starting a program;
s202, the number of the rows in the acquisition table Dbs _ a is row _ a; the number of columns in the acquisition table Dbs _ a is col _ a; the number of rows in the acquisition table Dbs _ b is row _ b; the number of columns in the acquisition table Dbs _ b is col _ b;
s203, setting a variable ra, and enabling the value of ra to be 1; setting a variable ca, and enabling the value of ca to be 1; setting a variable rb, and enabling the value of rb to be 1; setting a variable cb, and enabling the value of cb to be 1; setting a null array elist;
s204, obtaining Dbs _ a (ra); b (ra) is set, and b (ra) represents the overlapping degree of Dbs _ a (ra) and Dbs _ b;
s2051, defining a function Comp () as a function for determining and calculating whether two data fields are the same according to the column names of the two input data fields, wherein Dbs _ a (ra, ca) _ col represents the column name of the column where Dbs _ a (ra, ca) is located, Dbs _ b (rb, cb) _ col represents the column name of the column where Dbs _ b (rb, cb) is located, and the calculation procedure of inputting Dbs _ a (ra, ca) and Dbs _ b (rb, cb) into the function Comp () is as follows: the Comp () function outputs the result of calculation of equals (Dbs _ a (ra, ca), Dbs _ b (rb, cb)) if Dbs _ a (ra, ca) _ col is equal to Dbs _ b (rb, cb) _ col, and Comp () outputs 0 if Dbs _ a (ra, ca) _ col is not equal to Dbs _ b (rb, cb) _ col;
s2052, calculating the value of b (ra), and obtaining the value of b (ra) according to the following formula:
adding the value of b (ra) to the array elist;
s2053, judging whether the number of elements in the array elist is larger than or equal to the numerical value of row _ a, if so, going to S2062, otherwise, going to S2061;
s2061, increasing the value of ra by 1, and turning to S204;
s2062, outputting the arithmetic mean of the numerical values of all dimensions in the array elist as el; ending the program;
the value of each dimension in the array elist represents the degree of homology of each row corresponding table Dbs _ b in Dbs _ a, the arithmetic mean of the values of each dimension in the array elist represents the degree of homology of the Dbs _ a corresponding table Dbs _ b, and el is the result of calculating the degree of homology between the two tables by using the function Lap ().
Further, in S300, the method for constructing the data tree according to the degree of homology of each table is as follows: the data tree is a data structure formed by taking each table in the Dbs as a node, the modification time of the table Dbs _ i with the sequence number i in the Dbs is recorded as t (i), the modification time is the latest modification time of the table, the sequence number of the table with the latest modification time in the Dbs is recorded as al by comparing the sequence of the modification time of each table in the Dbs, and the table with the latest modification time in the Dbs is recorded as Dbs _ al; calculating the degree of homology of each table in the Dbs except the Dbs _ al and the Dbs _ al through a function Lap (),
taking the set formed by each table except Dbs _ al in Dbs as the al complement, respectively calculating the homology of each element in the second complement with the Dbs _ al through a function Lap (), calculating the arithmetic mean of the homology of each element in the second complement with the Dbs _ al and recording the arithmetic mean as a second threshold, taking each table in Dbs as each node of the data tree, taking Dbs _ al as a starting node, the starting point node is the first node for traversing the data tree, the element with the homology being less than or equal to the al threshold in the al complement set is taken as the leaf node on the left side of the starting point node, the set of the leaf nodes on the left side is recorded as the left set, the element with the homology being greater than the al threshold in the al complement set is taken as the leaf node on the right side of the starting point node, the set of the leaf nodes on the right side is recorded as the right set, and the starting point node, the left set and the right set form the data tree.
Further, in S400, the method for allocating the sharing authority of each table by using the data tree is as follows: selecting any node in a data tree to be recorded as a table Dbs _ bt, wherein the sharing authority is the authority of modifying or accessing another table by one table in the data tree, the number of tables which can be modified by one table in the data tree is defined as the authority number of the table, or the number of tables which can be modified by one table in the data tree is defined as the authority number of the table, namely the authority number is the number of tables which have sharing authority by one table, and the step of judging and distributing the sharing authority of each table is as follows:
s401, obtaining a table Dbs _ bt; go to S402;
s402, judging whether the Dbs _ bt is a starting point node, if so, turning to S4041, and if not, turning to S403;
s403, judging whether the Dbs _ bt is in the left set or the right set, if so, turning to S4042, and if so, turning to S4043;
s4041, distributing the sharing authority of all nodes in the data tree to Dbs _ bt, and acquiring the number of tables which are owned by Dbs _ bt as the authority number (if only one table is in Dbs, the authority number of Dbs _ bt is 0); go to S405;
s4042, distributing sharing authority of nodes with the homology of the Dbs _ al smaller than or equal to the homology of the tables Dbs _ bt and Dbs _ al in the left set to the Dbs _ bt, and acquiring the number of the tables with sharing authority owned by the Dbs _ bt as authority number; go to S405;
s4043, distributing sharing authority of nodes with homology of Dbs _ al smaller than or equal to that of tables Dbs _ bt and Dbs _ al in the right set to Dbs _ bt, and acquiring the number of the tables with sharing authority owned by Dbs _ bt as authority number; go to S405;
s405, finishing judgment to obtain sharing authority and authority number distributed to the Dbs _ bt;
thus, according to the data tree, the assignment of sharing authority to each table is obtained in steps S401 to S405.
Further, in S500, the method for determining the access order according to the sharing authority includes: and sequencing the nodes according to the sequence of the authority number of each node in the data tree from large to small according to the numerical value of the authority number, wherein the sequencing obtains an ordered sequence of each node in the data tree, namely an access sequence, the access sequence determines the sequence of accessing each node, the access refers to the operation of querying the table by using a structured query language, and the table is queried according to the sequence of the access sequence and printed on an output device.
The present disclosure also provides a threat intelligence big data sharing system, the threat intelligence big data sharing system includes: the threat intelligence big data sharing system can be operated in computing equipment such as desktop computers, notebooks, palm computers and cloud data centers, and the operable system can include, but is not limited to, a processor, a memory and a computer program stored in the memory and operable on the processor, the processor executes the computer program and operates in the following units of the system:
the information big data acquisition unit is used for acquiring the information big data from the server cluster;
a homology calculation unit for calculating a homology between the tables;
the data tree construction unit is used for constructing a data tree according to the homology of each table;
the sharing authority distributing unit is used for distributing the sharing authority of each table by a data tree;
and the sequential access unit is used for determining the access sequence according to the sharing authority.
The beneficial effect of this disclosure does: the method and the system for sharing the big threat intelligence data acquire the big intelligence data from a server cluster, calculate the homology of each table to construct a data tree, distribute the sharing authority of each table by the data tree, further determine the access sequence according to the sharing authority, and achieve the beneficial effect of effectively processing the exclusive authority distribution of each table in a database.
Drawings
The foregoing and other features of the present disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the drawings in which like reference characters designate the same or similar elements throughout the several views, and it is apparent that the drawings in the following description are merely some examples of the present disclosure and that other drawings may be derived therefrom by those skilled in the art without the benefit of any inventive faculty, and in which:
FIG. 1 is a flow chart of a threat intelligence big data sharing method;
fig. 2 is a system structure diagram of a threat intelligence big data sharing system.
Detailed Description
The conception, specific structure and technical effects of the present disclosure will be clearly and completely described below in conjunction with the embodiments and the accompanying drawings to fully understand the objects, aspects and effects of the present disclosure. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
Fig. 1 is a flow chart of a method for sharing big data of threat intelligence according to the present invention, and a method and a system for sharing big data of threat intelligence according to an embodiment of the present invention are described below with reference to fig. 1.
The disclosure provides a threat intelligence big data sharing method, which specifically comprises the following steps:
s100, acquiring big intelligence data from a server cluster, wherein the big intelligence data are a plurality of tables, and the tables are objects used for storing data in a database;
s200, calculating the homology among the tables;
s300, constructing a data tree according to the homology of each table;
s400, distributing sharing authorities of all tables by using a data tree;
s500, determining an access sequence according to the sharing authority.
Further, in S100, the method for obtaining the big intelligence data from the server cluster, where the big intelligence data is a plurality of tables, and the tables are objects used for storing data in the database includes: acquiring large intelligence data from a server cluster, wherein the large intelligence data are a plurality of tables, the Dbs is used as a set consisting of the tables, the data in the tables are organized according to a row and column format, one row of the table is a record, each serial number recorded in the table is the serial number of the row in which the table is located, each column of the table has the serial number of the column in the table and the column name of the column, the column name is a character string, an element of a certain column determined in a certain row of the table is called a data field of the table, the data field has the serial number of the row in which the data field is located and the column name of the column in which the data field is located, and the table acquires the modification time of the table by reading the record of the table in the server cluster; let n represent the number of elements in the set Dbs, represent the sequence number of the elements in the set Dbs by variable i, i belongs to [1, n ], and take the element with the sequence number i in the set Dbs as a table Dbs _ i; the number of rows of the element table Dbs _ i with sequence number i in the set Dbs is row _ i, the number of columns of the element table Dbs _ i with sequence number i in the set Dbs is col _ i, the sequence number of a row in any table in the set Dbs is represented by variable k and there is k e [1, row _ i ] in the table Dbs _ i where it is located, the row with sequence number k in the table Dbs _ i is denoted Dbs (k), the sequence number of a column in any table in the set Dbs is represented by variable q and there is q e [1, col _ i ] in the table Dbs _ i where it is located, the column with sequence number q in the table Dbs _ i is denoted as Dbs _ i (, q), the column of the Dbs _ i (, q) in the table Dbs _ i is denoted as Dbs _ i (, q) _ col), the data in the row of the table Dbs _ i is denoted as Dbs _ i (, q) in the column in _ i), the column k _ k in the table Dbs _ i is denoted as Dbs _ i (, q) is denoted as Dbs _ i _j _, q _ (k _, the column (k _, the name of the table Dbs _ i (k _, q) in the column (k _ i) is denoted as Dbs _ i _, q _, q) _ col.
Further, in S200, the method for calculating the degree of homology between the tables is: defining the degree of homology as representing the degree of coincidence between data contained in two tables, recording a function equals () as a function for judging whether two character strings are the same, if the two character strings are the same, outputting the function equals () to be a value 1, otherwise outputting a value 0, taking any two serial numbers from [1, n ] as a and b, obtaining a table Dbs _ a and a table Dbs _ b, and making a function Lap () be a function for calculating the degree of homology between the two tables, wherein the calculation process of the function Lap () is as follows:
s201, starting a program;
s202, the number of the rows in the acquisition table Dbs _ a is row _ a; the number of columns in the acquisition table Dbs _ a is col _ a; the number of rows in the acquisition table Dbs _ b is row _ b; the number of columns in the acquisition table Dbs _ b is col _ b;
s203, setting a variable ra, and enabling the value of ra to be 1; setting a variable ca, and enabling the value of ca to be 1; setting a variable rb, and enabling the value of rb to be 1; setting a function cb, and enabling the value of cb to be 1; setting a null array elist;
s204, obtaining Dbs _ a (ra); setting a function b (ra) for representing the coincidence degree of Dbs _ a (ra) and Dbs _ b;
s2051, defining a function Comp () as a function for determining and calculating whether two data fields are the same according to the column names of the two input data fields, wherein Dbs _ a (ra, ca) _ col represents the column name of the column where Dbs _ a (ra, ca) is located, Dbs _ b (rb, cb) _ col represents the column name of the column where Dbs _ b (rb, cb) is located, and the calculation procedure of inputting Dbs _ a (ra, ca) and Dbs _ b (rb, cb) into the function Comp () is as follows: if Dbs _ a (ra, ca) _ col is equal to Dbs _ b (rb, cb) _ col, the function outputs the calculation result of equals (Dbs _ a (ra, ca), Dbs _ b (rb, cb)), and if Dbs _ a (ra, ca) _ col is not equal to Dbs _ b (rb, cb) _ col, 0 is output;
s2052, calculating the value of function b (ra), and obtaining the value of function b (ra) according to the following formula:
adding the value of b (ra) to the array elist;
s2053, judging whether the number of elements in the array elist is larger than or equal to the numerical value of row _ a, if so, going to S2062, otherwise, going to S2061;
s2061, increasing the value of ra by 1, and turning to S204;
s2062, outputting the arithmetic mean of the numerical values of all dimensions in the array elist as el; ending the program;
the value of each dimension in the array elist represents the degree of homology of each row corresponding table Dbs _ b in Dbs _ a, the arithmetic mean of the values of each dimension in the array elist represents the degree of homology of the Dbs _ a corresponding table Dbs _ b, and el is the result of calculating the degree of homology between the two tables by using the function Lap ().
Further, in S300, the method for constructing the data tree according to the degree of homology of each table is as follows: the data tree is a data structure formed by taking each table in the Dbs as a node, the modification time of the table Dbs _ i with the sequence number i in the Dbs is recorded as t (i), the sequence number of the table with the first modification time in the Dbs is recorded as al by comparing the sequence of the modification time of each table in the Dbs, and the table with the first modification time in the Dbs is recorded as Dbs _ al; respectively calculating different homologies of each table except Dbs _ al in the Dbs and each Dbs _ al through a function Lap (), marking a set formed by each table except Dbs _ al in the Dbs and each different homologies of the Dbs _ al as a second complement set by using the serial number al of the Dbs _ al as a mark, or marking a set formed by each table except Dbs _ al in the Dbs as a second complement set, respectively calculating the homologies of each element in the second complement set and the Dbs _ al through the function Lap (), calculating an arithmetic mean of the homologies of each element in the second complement set and the Dbs _ al as a second threshold value, using each table in the Dbs as each node of the data tree, using the Dbs _ al as a starting node which is a first node for traversing the data tree, taking an element with the homologies smaller than or equal to the second threshold value in the second complement set as a leaf node on the left of the starting node and marking a left set as a left set of the leaf node, and taking the elements with the homology greater than the al threshold value in the al complement set as leaf nodes on the right side of the starting point node, recording the set of the leaf nodes on the right side as a right set, and forming the data tree by the starting point node, the left set and the right set.
Further, in S400, the method for allocating the sharing authority of each table by using the data tree is as follows: selecting any node in the data tree to be recorded as a table Dbs _ bt, or recording a table Dbs _ bt to be judged and allocated with sharing authority, wherein the sharing authority is the authority of modifying one table to another table in the data tree, the number of tables which can be modified in one table in the data tree is defined as the authority number of the table, namely the authority number is the number of tables with sharing authority of one table, and the step of judging and allocating the sharing authority of each table is as follows:
s401, obtaining a table Dbs _ bt; go to S402;
s402, judging whether the Dbs _ bt is a starting point node, if so, turning to S4041, and if not, turning to S403;
s403, judging whether the Dbs _ bt is in the left set or the right set, if so, turning to S4042, and if so, turning to S4043;
s4041, distributing the sharing authority of all nodes in the data tree to Dbs _ bt, and acquiring the number of tables with sharing authority owned by Dbs _ bt as the authority number; go to S405;
s4042, distributing sharing authority of nodes with the homology of the Dbs _ al smaller than or equal to the homology of the tables Dbs _ bt and Dbs _ al in the left set to the Dbs _ bt, and acquiring the number of the tables with sharing authority owned by the Dbs _ bt as authority number; go to S405;
s4043, distributing sharing authority of nodes with homology of Dbs _ al smaller than or equal to that of tables Dbs _ bt and Dbs _ al in the right set to Dbs _ bt, and acquiring the number of the tables with sharing authority owned by Dbs _ bt as authority number; go to S405;
s405, finishing judgment to obtain sharing authority and authority number distributed to the Dbs _ bt;
preferably, the key part of the Python implementation code of the step of determining and allocating the sharing authority of each table may include:
Class DataTree:
"""
building a data tree
"""
def __init__(self,x,dbs_bt=None, dbs_al=None):
self.row_a=x
self.left=dbs_bt
self.right=dbs_al
"""。
Assigning to Dbs _ bt the sharing permissions of nodes in the left set having a degree of homology to Dbs _ al that is less than or equal to the degree of homology of tables Dbs _ bt and Dbs _ al,
and obtaining the number of tables with shared permission owned by Dbs _ bt as the permission number;
assigning to Dbs _ bt the sharing permission for nodes in the right set having a degree of homology to Dbs _ al that is less than or equal to the degree of homology of tables Dbs _ bt and Dbs _ al,
and obtaining the number of tables with shared permission owned by Dbs _ bt as the permission number;
"""
def preOrder(root):
if not root:
return None
print(root.row_a)
preOrder(root.left)
preOrder(root.right)
"""。
the shared permissions of all nodes in the data tree are assigned to Dbs _ bt,
and acquires the number of the tables with the shared permission owned by the Dbs _ bt as the permission number
"""
def tree(root):
elist = [root]
while elist:
n = len(elist)
for ra in range(n)；
q = elist.pop(0)
if q:
print(q.row_a)；
elist.append(q.left if q.left else None)；
elist.append(q.right if q.right else None)。
Thus, according to the data tree, the assignment of sharing authority to each table is obtained in steps S401 to S405.
Further, in S500, the method for determining the access order according to the sharing authority includes: and sequencing the nodes according to the sequence of the authority number of each node in the data tree from large to small according to the numerical value of the authority number, wherein the sequencing obtains an ordered sequence of each node in the data tree, namely an access sequence, the access sequence determines the sequence of accessing each node, the access refers to the operation of querying the table by using a structured query language, and the table is queried according to the sequence of the access sequence and printed on an output device.
The threat intelligence big data sharing system comprises: the threat intelligence big data sharing system can be operated in computing equipment such as desktop computers, notebooks, palm computers, cloud data centers and the like, and the operable systems can include, but are not limited to, processors, memories and server clusters.
As shown in fig. 2, the threat intelligence big data sharing system according to the embodiment of the present disclosure includes: a processor, a memory and a computer program stored in the memory and capable of running on the processor, the processor implementing the steps in the above embodiment of the threat intelligence big data sharing method when executing the computer program, the processor executing the computer program running in the units of the following system:
the information big data acquisition unit is used for acquiring the information big data from the server cluster;
a homology calculation unit for calculating a homology between the tables;
the data tree construction unit is used for constructing a data tree according to the homology of each table;
the sharing authority distributing unit is used for distributing the sharing authority of each table by a data tree;
and the sequential access unit is used for determining the access sequence according to the sharing authority.
The threat information big data sharing system can be operated in computing equipment such as desktop computers, notebooks, palm computers and cloud data centers. The threat intelligence big data sharing system comprises a processor and a memory. Those skilled in the art will appreciate that the example is only an example of a threat intelligence big data sharing method and system, and does not constitute a limitation of a threat intelligence big data sharing method and system, and may include more or less components than a ratio, or combine some components, or different components, for example, the threat intelligence big data sharing system may further include an input output device, a network access device, a bus, etc.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete component Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor, and the processor is a control center of the threat intelligence big data sharing system, and various interfaces and lines are used to connect various sub-areas of the whole threat intelligence big data sharing system.
The memory can be used for storing the computer program and/or the module, and the processor realizes various functions of the threat intelligence big data sharing method and system by operating or executing the computer program and/or the module stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
The method and the system for sharing the big threat intelligence data acquire the big intelligence data from a server cluster, calculate the homology of each table to construct a data tree, distribute the sharing authority of each table by the data tree, further determine the access sequence according to the sharing authority, and achieve the beneficial effect of effectively processing the exclusive authority distribution of each table in a database.
Although the description of the present disclosure has been rather exhaustive and particularly described with respect to several illustrated embodiments, it is not intended to be limited to any such details or embodiments or any particular embodiments, so as to effectively encompass the intended scope of the present disclosure. Furthermore, the foregoing describes the disclosure in terms of embodiments foreseen by the inventor for which an enabling description was available, notwithstanding that insubstantial modifications of the disclosure, not presently foreseen, may nonetheless represent equivalent modifications thereto.
Claims (3)
1. A threat intelligence big data sharing method is characterized by comprising the following steps:
s100, acquiring big intelligence data from a server cluster, wherein the big intelligence data are a plurality of tables, and the tables are objects used for storing data in a database;
s200, calculating the homology among the tables;
s300, constructing a data tree according to the homology of each table;
s400, distributing sharing authorities of all tables by using a data tree;
s500, determining an access sequence according to the sharing authority;
in S100, obtaining big intelligence data from a server cluster, where the big intelligence data is a plurality of tables, and the table is an object used for storing data in a database by: acquiring big intelligence data from the server cluster, wherein the big intelligence data is text data obtained by identifying the text content on pictures and photos through user input, web crawler software crawling or OCR (optical character recognition) and is stored in a plurality of tables of a database on the server cluster, a collection of tables for storing informative big data, the data in said tables organized in a format of rows and columns, is denoted Dbs, wherein, one row of the table is a record, each serial number recorded in the table is the serial number of the row, each column of the table has the serial number of the column in the table and the column name of the column, the column name is a character string, an element defining a column in a certain row of a table is called a data field of the table, the data field is a character string, the data field has the serial number of the row where the data field is located and the column name of the column where the data field is located, and the table obtains the modification time of the table by reading the record of the data field in the server cluster; let n represent the number of elements in the set Dbs, represent the sequence number of the elements in the set Dbs by variable i, i belongs to [1, n ], and take the element with the sequence number i in the set Dbs as a table Dbs _ i; the number of rows of the element table Dbs _ i with sequence number i in the set Dbs is row _ i, the number of columns of the element table Dbs _ i with sequence number i in the set Dbs is col _ i, the sequence number of a row in any table in the set Dbs is represented by variable k, and k e [1, row _ i ] exists in the table Dbs _ i in which it is located, the row with sequence number k in a row in the table Dbs _ i is denoted Dbs _ i (k), the sequence number of a column in any table in the set Dbs is represented by variable q, and q e [1, col _ i ] exists in the table Dbs _ i in which it is located, the column with sequence number q in the table Dbs _ i is denoted as Dbs _ i (, q), the column of Dbs _ i (, q) in the table Dbs _ i is denoted as Dbs _ i (, q) _ col, the column in _ i in the table Dbs _ i is denoted as Dbs _ i, the data k in the column of the Dbs _ i is denoted as Dbs _ i (, q) in the column in the table Dbs _ i), q) _ col;
in S200, the method for calculating the degree of homology between tables includes: defining the degree of homology as representing the degree of coincidence between data contained in two tables, recording a function equals () as a function for judging whether two character strings are the same, if the two character strings are the same, outputting the function equals () to be a value 1, otherwise outputting a value 0, taking any two serial numbers from [1, n ] as a and b, obtaining a table Dbs _ a and a table Dbs _ b, and making a function Lap () be a function for calculating the degree of homology between the two tables, wherein the calculation process of the function Lap () is as follows:
s201, starting a program;
s202, the number of the rows in the acquisition table Dbs _ a is row _ a; the number of columns in the acquisition table Dbs _ a is col _ a; the number of rows in the acquisition table Dbs _ b is row _ b; the number of columns in the acquisition table Dbs _ b is col _ b;
s203, setting a variable ra, and enabling the value of ra to be 1; setting a variable ca, and enabling the value of ca to be 1; setting a variable rb, and enabling the value of rb to be 1; setting a variable cb, and enabling the value of cb to be 1; setting a null array elist;
s204, obtaining Dbs _ a (ra); setting a function b (ra) for representing the coincidence degree of Dbs _ a (ra) and Dbs _ b;
s2051, defining a function Comp () as a function for determining and calculating whether two data fields are the same according to the column names of the two input data fields, wherein Dbs _ a (ra, ca) _ col represents the column name of the column where Dbs _ a (ra, ca) is located, Dbs _ b (rb, cb) _ col represents the column name of the column where Dbs _ b (rb, cb) is located, and the calculation procedure of inputting Dbs _ a (ra, ca) and Dbs _ b (rb, cb) into the function Comp () is as follows: if Dbs _ a (ra, ca) _ col is equal to Dbs _ b (rb, cb) _ col, the function outputs the calculation result of equals (Dbs _ a (ra, ca), Dbs _ b (rb, cb)), and if Dbs _ a (ra, ca) _ col is not equal to Dbs _ b (rb, cb) _ col, 0 is output;
s2052, calculating the value of function b (ra), and obtaining the value of function b (ra) according to the following formula:
adding the value of b (ra) to the array elist;
s2053, judging whether the number of elements in the array elist is larger than or equal to the numerical value of row _ a, if so, going to S2062, otherwise, going to S2061;
s2061, increasing the value of ra by 1, and turning to S204;
s2062, outputting the arithmetic mean of the numerical values of all dimensions in the array elist as el; ending the program;
the value of each dimension in the array elist represents the degree of homology of each row corresponding table Dbs _ b in Dbs _ a, the arithmetic mean of the values of each dimension in the array elist represents the degree of homology of the Dbs _ a corresponding table Dbs _ b, and el is the result obtained by calculating the degree of homology between the two tables through a function Lap ();
in S300, the method for constructing the data tree according to the degree of homology of each table includes: the data tree is a data structure formed by taking each table in the Dbs as a node, the modification time of the table Dbs _ i with the sequence number i in the Dbs is recorded as t (i), the sequence number of the table with the first modification time in the Dbs is recorded as al by comparing the sequence of the modification time of each table in the Dbs, and the table with the first modification time in the Dbs is recorded as Dbs _ al; respectively calculating different degrees of homology of each table except Dbs _ al in the Dbs and each Dbs _ al through a function Lap (), respectively taking a set consisting of each table except Dbs _ al in the Dbs as a second complement, respectively calculating the degrees of homology of each element in the second complement with the Dbs _ al through the function Lap (), calculating the arithmetic mean value of the degrees of homology of each element in the second complement and the Dbs _ al as a second threshold, taking each table in the Dbs as each node of the data tree, taking the Dbs _ al as a starting node which is a first node for traversing the data tree, taking an element with the degree of homology smaller than or equal to the second threshold in the second complement as a leaf node on the left side of the starting node and taking a set of the leaf nodes on the left side as a set, taking an element with the degree of homology larger than the second threshold in the second complement as a leaf node on the right side of the starting node and taking a set of the leaf nodes on the right side as a set of the right side of the starting node, a data tree is formed by the starting point node, the left side set and the right side set;
in S400, the method for allocating the sharing authority of each table by using the data tree includes: selecting any node in the data tree to be recorded as a table Dbs _ bt, wherein the sharing authority is the authority of modifying one table to another table in the data tree, the number of tables which can be modified by one table in the data tree is defined as the authority number of the table, or the number of tables which can be modified by one table in the data tree is defined as the authority number of the table, namely the authority number is the number of tables with sharing authority of one table, and the steps of judging and distributing the sharing authority of each table are as follows:
s401, obtaining a table Dbs _ bt; go to S402;
s402, judging whether the Dbs _ bt is a starting point node, if so, turning to S4041, and if not, turning to S403;
s403, judging whether the Dbs _ bt is in the left set or the right set, if so, turning to S4042, and if so, turning to S4043;
s4041, distributing the sharing authority of all nodes in the data tree to Dbs _ bt, and acquiring the number of tables with sharing authority owned by Dbs _ bt as the authority number; go to S405;
s4042, distributing sharing authority of nodes with the homology of the Dbs _ al smaller than or equal to the homology of the tables Dbs _ bt and Dbs _ al in the left set to the Dbs _ bt, and acquiring the number of the tables with sharing authority owned by the Dbs _ bt as authority number; go to S405;
s4043, distributing sharing authority of nodes with homology of Dbs _ al smaller than or equal to that of tables Dbs _ bt and Dbs _ al in the right set to Dbs _ bt, and acquiring the number of the tables with sharing authority owned by Dbs _ bt as authority number; go to S405;
s405, finishing judgment to obtain sharing authority and authority number distributed to the Dbs _ bt;
thus, according to the data tree, the assignment of sharing authority to each table is obtained in steps S401 to S405.
2. The method for sharing big data of threat intelligence according to claim 1, wherein in S500, the method for determining the access sequence according to the sharing authority is as follows: and sequencing the nodes according to the sequence of the authority number of each node in the data tree from large to small according to the numerical value of the authority number, wherein the sequencing obtains an ordered sequence of each node in the data tree, namely an access sequence, the access sequence determines the sequence of accessing each node, the access refers to the operation of querying the table by using a structured query language, and the table is queried according to the sequence of the access sequence and printed on an output device.
3. A threat intelligence big data sharing system, wherein the threat intelligence big data sharing system comprises: the system comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps in the threat intelligence big data sharing method in claim 1, and the threat intelligence big data sharing system runs in computing equipment of desktop computers, notebooks, palm computers and cloud data centers.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111139244.4A CN113591134B (en) | 2021-09-28 | 2021-09-28 | Threat intelligence big data sharing method and system |
| PCT/CN2022/118573 WO2023051235A1 (en) | 2021-09-28 | 2022-09-14 | Threat intelligence big data sharing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111139244.4A CN113591134B (en) | 2021-09-28 | 2021-09-28 | Threat intelligence big data sharing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113591134A CN113591134A (en) | 2021-11-02 |
| CN113591134B true CN113591134B (en) | 2021-12-14 |
Family
ID=78242130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111139244.4A Active CN113591134B (en) | 2021-09-28 | 2021-09-28 | Threat intelligence big data sharing method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113591134B (en) |
| WO (1) | WO2023051235A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113591134B (en) * | 2021-09-28 | 2021-12-14 | 广东机电职业技术学院 | Threat intelligence big data sharing method and system |
| CN114241535B (en) * | 2021-12-01 | 2022-09-27 | 佛山市红狐物联网科技有限公司 | Rapid palm vein feature extraction method and system |
| CN114896600B (en) * | 2022-04-29 | 2024-06-25 | 苏州浪潮智能科技有限公司 | Server threat assessment method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107391598A (en) * | 2017-06-30 | 2017-11-24 | 北京航空航天大学 | One kind threatens information automatic generation method and system |
| CN108600212A (en) * | 2018-04-19 | 2018-09-28 | 北京邮电大学 | Threat information credibility method of discrimination and device based on the credible feature of various dimensions |
| WO2019028341A1 (en) * | 2017-08-03 | 2019-02-07 | T-Mobile Usa, Inc. | Similarity search for discovering multiple vector attacks |
| CN111935131A (en) * | 2020-08-06 | 2020-11-13 | 中国工程物理研究院计算机应用研究所 | SaaS resource access control method based on resource authority tree |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11182476B2 (en) * | 2016-09-07 | 2021-11-23 | Micro Focus Llc | Enhanced intelligence for a security information sharing platform |
| US11005869B2 (en) * | 2017-11-24 | 2021-05-11 | Korea Internet & Security Agency | Method for analyzing cyber threat intelligence data and apparatus thereof |
| US11055420B2 (en) * | 2018-02-05 | 2021-07-06 | International Business Machines Corporation | Controlling access to data requested from an electronic information system |
| CN113591134B (en) * | 2021-09-28 | 2021-12-14 | 广东机电职业技术学院 | Threat intelligence big data sharing method and system |
-
2021
- 2021-09-28 CN CN202111139244.4A patent/CN113591134B/en active Active
-
2022
- 2022-09-14 WO PCT/CN2022/118573 patent/WO2023051235A1/en unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107391598A (en) * | 2017-06-30 | 2017-11-24 | 北京航空航天大学 | One kind threatens information automatic generation method and system |
| WO2019028341A1 (en) * | 2017-08-03 | 2019-02-07 | T-Mobile Usa, Inc. | Similarity search for discovering multiple vector attacks |
| CN108600212A (en) * | 2018-04-19 | 2018-09-28 | 北京邮电大学 | Threat information credibility method of discrimination and device based on the credible feature of various dimensions |
| CN111935131A (en) * | 2020-08-06 | 2020-11-13 | 中国工程物理研究院计算机应用研究所 | SaaS resource access control method based on resource authority tree |
Non-Patent Citations (2)
| Title |
|---|
| A Review of Artificial Intelligence to Enhance the Security of Big Data Systems: State-of-Art, Methodologies, Applications, and Challenges;Duan Dai等;《ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING》;20210712;全文 * |
| 大数据环境下的威胁情报分析;李超等;《情报杂志》;20170930;第24-30页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113591134A (en) | 2021-11-02 |
| WO2023051235A1 (en) | 2023-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113591134B (en) | Threat intelligence big data sharing method and system | |
| CN106021541B (en) | Distinguish the anonymous Privacy preserving algorithms of secondary k of standard identifier attribute | |
| TWI706280B (en) | Data reading and writing method and device, electronic equipment | |
| US20140040262A1 (en) | Techniques for cloud-based similarity searches | |
| US10289702B2 (en) | Image retrieval method | |
| US20100223240A1 (en) | System and method for composite record keys ordered in a flat key space for a distributed database | |
| CN104572785B (en) | A kind of distributed method and apparatus for creating index | |
| CN109460406B (en) | Data processing method and device | |
| JP2020501254A (en) | Method and system for anonymizing data stock | |
| TW202020756A (en) | Data permission control method and system thereof, computer device, and readable storage medium | |
| CN112925792B (en) | Data storage control method, device, computing equipment and medium | |
| Patgiri et al. | Role of bloom filter in big data research: A survey | |
| CN113609715B (en) | Multivariate model data fusion method and system under digital twin background | |
| CN115905630A (en) | Graph database query method, device, equipment and storage medium | |
| CN116719822B (en) | Method and system for storing massive structured data | |
| CN117499124A (en) | Access control method and device | |
| CN116842012A (en) | Method, device, equipment and storage medium for storing Redis cluster in fragments | |
| CN111399898A (en) | Management method and device for multi-module system code version and computer equipment | |
| CN115328950A (en) | Secondary index-based hbase query method, terminal device and storage medium | |
| CN115221360A (en) | Tree structure configuration method and system | |
| CN112528189A (en) | Data-based component packaging method and device, computer equipment and storage medium | |
| CN114268476B (en) | Data security protection method and system based on node encryption | |
| CN115408491B (en) | Text retrieval method and system for historical data | |
| CN115102920B (en) | Individual transmission and management control method based on relational network | |
| CN114595030A (en) | Cloud container resource allocation method and system based on auction algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |