CN115905150A - Log management system, method, device and storage medium - Google Patents
Log management system, method, device and storage medium Download PDFInfo
- Publication number
- CN115905150A CN115905150A CN202111094785.XA CN202111094785A CN115905150A CN 115905150 A CN115905150 A CN 115905150A CN 202111094785 A CN202111094785 A CN 202111094785A CN 115905150 A CN115905150 A CN 115905150A
- Authority
- CN
- China
- Prior art keywords
- tree
- log
- log entries
- database
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 98
- 238000012795 verification Methods 0.000 claims abstract description 88
- 230000008569 process Effects 0.000 claims abstract description 47
- 238000010276 construction Methods 0.000 claims abstract description 24
- 238000013473 artificial intelligence Methods 0.000 claims description 189
- 238000012550 audit Methods 0.000 claims description 131
- 238000007726 management method Methods 0.000 claims description 89
- 230000015654 memory Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 23
- 238000004891 communication Methods 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A log management system and method belong to the technical field of computers. According to the method, the related information corresponding to the log data generated by the AI system in the first time period is respectively stored through two different databases, so that the safety is ensured. The first log entries of the AI system are stored in a first database, and the root value of a first Merckel tree, the root value of a target Merckel tree and the verification information generated based on the first log entries of the AI system are stored in a second database. In the process, the target Mercker tree is constructed based on the first Mercker tree and other second Mercker trees constructed in the previous time period, and the hierarchical construction method of the Mercker trees can support storage of relevant information corresponding to mass log data generated by the AI system, so that the log data generated by the AI system are uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety.
Description
The present invention claims priority from chinese patent application No. 202110997928.1 entitled "method, apparatus and system for log management" filed on 27/08/2021, the entire contents of which are incorporated herein by reference.
Technical Field
The present application relates to the field of computer technologies, and in particular, to a log management system, method, device, and storage medium.
Background
With the rapid development of Artificial Intelligence (AI) technology, various AI systems have come into play, which can provide various AI services to users. At present, laws and regulations of various countries require that AI systems should continuously improve transparency to ensure safety. In the related art, a log audit mode is usually adopted to provide technical support for improving the transparency of the AI system. The log audit refers to standardized audit and inspection of log data generated by the AI system, and consideration of authenticity and integrity of the log data to discover risks in the aspects of potential system security, data security, service security and the like. However, in the related art, there is no technical scheme for uniformly tracking and recording the log data generated by the AI system, so that the transparency of the AI system is poor, and potential safety hazards are easily caused.
Therefore, a log management system capable of effectively improving transparency of the AI system while ensuring security is needed.
Disclosure of Invention
The embodiment of the application provides a log management system, a log management method, log management equipment and a log management storage medium, which can effectively improve the transparency of an AI system on the premise of ensuring the safety. The technical scheme is as follows:
in a first aspect, a log management system is provided, the system comprising a first database and a second database;
the first database is used for storing a plurality of first log entries of the artificial intelligence AI system, and the plurality of first log entries correspond to log data generated by the AI system in a first time period;
the second database is used for storing the root value of the first Merckel tree, the root value of the target Merckel tree and the verification information of the target Merckel tree;
wherein the first Merckel tree is constructed based on the first plurality of log entries, the target Merckel tree is constructed based on the first Merckel tree and at least one second Merckel tree, the second Merckel tree is constructed based on a second plurality of log entries, the second plurality of log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and verification information of the target Merckel tree indicates a construction process of the target Merckel tree.
In the system, related information corresponding to log data generated by an AI system for a first period of time is stored through two different databases, respectively, to ensure security. The method includes storing a plurality of first log entries of the AI system in a first database, and storing a root value of a first Mercker tree, a root value of a target Mercker tree, and check information, which are generated based on the plurality of first log entries of the AI system, in a second database. In the process, the target Merckel tree is constructed on the basis of the first Merckel tree and other second Merckel trees constructed in the previous time period, and the hierarchical construction mode of the Merckel trees can support storage of relevant information corresponding to mass log data generated by the AI system, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety. Further, since the merkel tree has the characteristics of non-tampering, transparency and credibility, whether log entries are tampered or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
In some embodiments, the first database is further configured to store a verification path of the plurality of first log entries, where the verification path includes an intermediate node of the corresponding first log entry, and the intermediate node is a node on a path for constructing the first mercker tree starting from the corresponding first log entry.
The verification path for storing the log entry can be used for verifying the existence of the log entry, namely, verifying whether the log entry is tampered. Illustratively, in a scenario where the plurality of first log entries are audited, if the plurality of first log entries are not audited, the log management system can verify the existence of each first log entry through a verification path of each first log entry, thereby locating a tampered log entry and improving transparency of the AI system.
In some embodiments, the first database is further configured to store log data, hash values, and generation times of the first plurality of log entries.
In some embodiments, the first database is an object-type database; the second database is a relational database.
The object type database has large storage capacity, is not limited in storage format and is not limited in permission of reading data and writing data, so that the performance problem of instant storage/query of massive log entries can be supported under the condition that the first database is the object type database, and the log management efficiency is effectively improved. In addition, because the storage format of the relational database is relatively fixed, and the permission for reading and writing data needs to be limited, the security of the data in the database can be guaranteed, and therefore, the security can be effectively improved under the condition that the second database is the relational database.
In some embodiments, the system further comprises a processing center; the processing center is used for:
processing log data generated by the AI system in the first time period, generating a plurality of first log entries, and storing the plurality of first log entries in the first database;
constructing the first Merck tree by using a plurality of hash values corresponding to the first log entries as leaf nodes, and storing the root value of the first Merck tree in the second database;
constructing the target Merckel tree by using the root value of the first Merckel tree and the root value of the at least one second Merckel tree as leaf nodes, and storing the root value of the target Merckel tree and the verification information of the target Merckel tree in the second database.
Through the hierarchical establishment of the Mercker tree, the storage of the related information corresponding to the mass log data generated by the AI system can be supported, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety.
In some embodiments, the processing center is further configured to:
if the first log entries meet a first condition, constructing the first Merckel tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or after the first log entries are generated, log data generated by the AI system is not received within a first time period.
By constructing the first Merck tree under the condition that the number of the first log entries meets the number requirement, the problem that one Merck tree is constructed every time the storage server receives log data of the AI system can be avoided, so that the data processing amount of the storage server is reduced, the load consumption of the storage server is reduced, and the log management efficiency is improved. By constructing the first Mercker tree under the condition that a plurality of first log entries meet the time requirement, the storage server can be prevented from waiting for log data generated by the AI system for a long time until the number of leaf nodes for constructing the first Mercker tree meets the number requirement, and therefore the storage efficiency of the storage server is improved.
In some embodiments, the processing center is further configured to:
if the at least one second Mercker tree meets a second condition, constructing the target Mercker tree by taking the root value of the first Mercker tree and the root value of the at least one second Mercker tree as leaf nodes;
wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second duration.
By constructing the target Merck tree under the condition that the number of the at least one second Merck tree meets the number requirement, the storage server can be prevented from frequently constructing the target Merck tree, so that the data processing amount of the storage server is reduced, the load consumption of the storage server is reduced, and the log management efficiency is improved. By constructing the target Mercker tree under the condition that at least one second Mercker tree meets the time requirement, the storage server can be prevented from waiting for the number of the Mercker trees to meet the requirement for a long time, and therefore the storage efficiency of the storage server is improved.
In some embodiments, the processing center is further configured to:
receiving a first log audit request, the first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, acquiring the plurality of first log entries from the first database, and acquiring a root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Merck tree.
By the method, the log auditing function is realized, and because the Mercker tree has the characteristics of non-falsification, transparency and credibility, whether log entries are falsified or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
In some embodiments, the processing center is configured to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes;
and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
In some embodiments, the processing center is further configured to store the tree root value of the target merck tree and the verification information of the target merck tree in the form of blocks on a block chain.
By the method, the properties of transparent credibility, tamper resistance, traceability and the like of the block chain are utilized, the safety of the root value and the check information of the target Mercker tree is ensured, and the transparency of the AI system is further improved on the premise of ensuring the safety.
In some embodiments, the system further comprises a verifier; the checker is used for acquiring the root value of the target Mercker tree and the checking information of the target Mercker tree from the block chain, and checking the simple increment of the target Mercker tree.
By the mode, the checker can check the simple increment of the target Merckel tree, namely check the simple increment of the log of the AI system, so that the purpose of monitoring the log storage process of the storage server is achieved, the log is ensured to be only added, and the previously stored log is not covered, so that the transparency of the AI system is effectively improved.
In some embodiments, the verifier is to:
and acquiring the root value of the target Mercker tree and the checking information of the target Mercker tree from the block chain in the form of an intelligent contract, and checking the monotonicity of the target Mercker tree, wherein the intelligent contract indicates that the monotonicity of the target Mercker tree is checked periodically.
By means of the intelligent contract, the checker periodically and automatically checks the target Merckel tree, the purpose of monitoring the log storage process is achieved, and transparency of an AI system is improved.
In some embodiments, the verifier is to:
acquiring a root value of a fourth Mercker tree and verification information of the fourth Mercker tree from the block chain, wherein the fourth Mercker tree is the latest Mercker tree which passes the single increment verification on the block chain, and the verification information of the fourth Mercker tree indicates the construction process of the fourth Mercker tree;
checking the simple increase of the target Merckel tree based on the root value of the fourth Merckel tree, the verification information of the fourth Merckel tree, the root value of the target Merckel tree and the verification information of the target Merckel tree.
In some embodiments, the system further comprises an audit center;
the audit center is used for:
sending a second log audit request to the processing center, the second log audit request indicating that the plurality of first log entries are audited;
receiving a log audit result, or receiving log audit information; wherein the log audit result indicates whether the first plurality of log entries are audited, and the log audit information includes the first plurality of log entries and a root value of the first Mercker tree.
In some embodiments, where the log audit information is received, the audit center is further to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a fifth Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes;
and if the root value of the fifth Mercker tree is the same as the root value of the first Mercker tree, determining that the plurality of first log entries are audited to pass.
Through the method, the audit center can send the log audit request to the storage server to audit the log entries of the AI system, wherein the audit center can receive the log audit result to reduce the load consumption of the audit center and improve the log management efficiency, and the audit center can also receive the log audit information and automatically audit to ensure the safety and improve the transparency of the AI system.
In a second aspect, a log management method is provided, where the method includes:
processing log data generated by an AI system in a first time period to generate a plurality of first log entries, and storing the plurality of first log entries in a first database;
constructing a first Mercker tree by taking a plurality of Hash values corresponding to the first log entries as leaf nodes, and storing a tree root value of the first Mercker tree in a second database;
taking the root value of the first Merck tree and the root value of at least one second Merck tree as leaf nodes, constructing a target Merck tree, storing the root value of the target Merck tree and the verification information of the target Merck tree in the second database, wherein the second Merck tree is constructed based on a plurality of second log entries, the plurality of second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target Merck tree indicates the construction process of the target Merck tree.
In some embodiments, the method further comprises:
storing a verification path of the plurality of first log entries in the first database, the verification path including an intermediate node corresponding to the first log entry, the intermediate node being a node on a path constructing the first merkel tree starting from the corresponding first log entry.
In some embodiments, the method further comprises:
the log data, hash value, and generation time of the plurality of first log entries are stored in the first database.
In some embodiments, the first database is an object-type database; the second database is a relational database.
In some embodiments, the method further comprises:
if the first log entries meet a first condition, constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
In some embodiments, the method further comprises:
if the at least one second Mercker tree meets a second condition, constructing the target Mercker tree by taking the root value of the first Mercker tree and the root value of the at least one second Mercker tree as leaf nodes;
wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second duration.
In some embodiments, the method further comprises:
receiving a first log audit request, the first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
In some embodiments, the auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first merkel tree includes:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
In some embodiments, the method further comprises:
storing the root value of the target Merck tree and the verification information of the target Merck tree on a block chain in the form of blocks.
In a third aspect, there is provided a log management apparatus, including:
the first storage module is used for processing log data generated by the AI system in a first time period, generating a plurality of first log entries and storing the plurality of first log entries in a first database;
the second storage module is used for taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes, constructing a first Mercker tree, and storing a tree root value of the first Mercker tree in a second database;
the second storage module is further configured to use the root value of the first merck tree and the root value of at least one second merck tree as leaf nodes to construct a target merck tree, store the root value of the target merck tree and verification information of the target merck tree in the second database, where the second merck tree is constructed based on a plurality of second log entries, the plurality of second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target merck tree indicates a construction process of the target merck tree.
In some embodiments, the first storage module is further configured to store a verification path of the plurality of first log entries in the first database, where the verification path includes an intermediate node corresponding to the first log entry, and the intermediate node is a node on a path for constructing the first mercker tree starting from the corresponding first log entry.
In some embodiments, the first storage module is further configured to store the log data, the hash value, and the generation time of the first plurality of log entries in the first database.
In some embodiments, the first database is an object-type database; the second database is a relational database.
In some embodiments, the first storage module is further configured to: if the first log entries meet a first condition, constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes; the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
In some embodiments, the second storage module is further configured to: if the at least one second Mercker tree meets a second condition, constructing the target Mercker tree by taking the root value of the first Mercker tree and the root value of the at least one second Mercker tree as leaf nodes; wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second duration.
In some embodiments, the apparatus further comprises an audit module; the audit module is configured to:
receiving a first log audit request, the first log audit request indicating that the plurality of first log entries are audited; based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database; auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
In some embodiments, the audit module is to: determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries; constructing a third Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes; and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
In some embodiments, the second storage module is further configured to store the root value of the target mercker tree and the check information of the target mercker tree in the form of blocks on a block chain.
In a fourth aspect, a computing device is provided, the computing device comprising a processor and a memory, the memory being configured to store at least one piece of program code, the at least one piece of program code being loaded by the processor and performing the log management method as provided in the second aspect or any one of the alternative implementations of the second aspect.
In a fifth aspect, a computer-readable storage medium is provided, which is used for storing at least one program code for executing the log management method provided in the second aspect or any one of the optional implementation manners of the second aspect.
A sixth aspect provides a computer program product for causing a computer to perform the log management method as provided in the second aspect or any one of the alternative implementations of the second aspect, when the computer program product runs on the computer.
Drawings
Fig. 1 is a schematic architecture diagram of a log management system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an architecture of a storage server according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a computing device according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a computing device according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a computing device according to an embodiment of the present application;
fig. 6 is a flowchart illustrating a log management method according to an embodiment of the present application;
FIG. 7 is a schematic structural diagram of a first Mercker tree according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a log management method provided in an embodiment of the present application;
FIG. 9 is a schematic diagram of a log audit process provided by an embodiment of the present application;
FIG. 10 is a schematic diagram of a simple incremental verification provided by an embodiment of the present application;
fig. 11 is a schematic diagram of a log management method provided in an embodiment of the present application;
fig. 12 is a schematic structural diagram of a log management apparatus according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.
Before describing the technical solutions provided by the embodiments of the present application, the following description will be made on the key terms related to the present application.
An AI system refers to software developed using one or more AI techniques and methodologies that generate outputs (e.g., content, predictions, suggestions, or decisions, etc.) that affect an interactive environment to achieve a particular goal specified by a human.
Log is a kind of information that records hardware, software and system problems in a system. In some embodiments, the user can check the cause of the system error through the log generated by the system, and can also inquire the trace left by the system during illegal operation or attack through the log generated by the system.
Transparency (transparency) is a security mechanism in the field of information security for describing that an attacker cannot realize intentional detection or hiding, that is, all operations in the system are transparent to an authorized auditor, and a system with higher transparency makes malicious behaviors easier to discover.
A database (database), an electronic file cabinet, that is, a place for storing electronic files, where a user can add, query, update, delete, etc. data in the files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.
A hash function is a function that maps data of an arbitrary length into data of a fixed length.
A merkel tree (hash tree), which is a type of hashed binary tree, also called hash tree, stores data in leaf nodes of a tree structure, and ensures non-tamper-ability of the data through a progressive hash operation on the data. Wherein, any father node of the Merck tree is equal to the result of Hash after data connection of left and right child nodes, and the topmost node of the Merck tree is called Merck tree root.
Public Key Infrastructure (PKI), an identity authentication system based on public-private key algorithm. In some embodiments, the PKI includes a Certificate Authority (CA), a certificate Registration Authority (RA), a digital certificate library, a certificate issuing system, a PKI application, and the like, which are not limited in this embodiment of the present invention.
The block chain (blockchain) is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like, and has the properties of transparency, credibility, tamper resistance, traceability and the like. The blockchain is essentially a decentralized database, which is a string of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating the next block. In other words, blockchain technology is a decentralized architecture and computing paradigm that utilizes blockchain data structures to verify and store data, distributed node consensus algorithms to generate and update data, cryptographic means to secure data transmission and access, and intelligent contracts composed of automated script code to program and manipulate data.
The intelligent contract is a computerized protocol, can execute the terms of a certain contract, is realized by codes which are deployed on a shared account and are used for executing when certain conditions are met, is used for completing automatic transaction according to actual business requirement codes, such as inquiring the logistics state of goods purchased by a buyer, and transferring the electronic money of the buyer to the address of a merchant after the buyer signs goods; of course, smart contracts are not limited to executing contracts for trading, but may also execute contracts that process received information.
The consensus mechanism is an algorithm for block chain transactions to achieve distributed consensus. In the embodiment of the present application, a user may select a suitable consensus algorithm according to an actual service scenario, where the consensus algorithm includes, but is not limited to: proof of work (POW), proof of entitlement (POS), proof of Delegation (DPOS), practicalized Byzantine Fault Tolerance (PBFT), raft, kafka, and hotspot, among others.
The following briefly introduces an application scenario of the technical solution provided by the present application.
At present, laws and regulations of various countries require that the transparency of an AI system be continuously improved so as to provide technical support for log audit of the AI system. For example, legal regulations indicate that "AI systems should be increasingly transparent, interpretable, reliable, controllable, auditable, supervisoble, traceable, trustworthy" in a step-by-step manner. Also for example, by issuing an AI regulation proposal, placing the AI system in a legislative program, requiring the AI system to be transparent to the user, and so forth. However, the AI system has a complex structure, many related components and modules, can generate a large amount of log data, and the visibility of the whole process is poor, so that the transparency of the AI system is difficult to be ensured.
Based on this, the embodiment of the present application provides a log management system and method, which can be applied to various scenarios requiring unified tracking and recording of log data generated by an AI system, in which relevant information corresponding to log data generated by the AI system in a first time period is stored through two different databases, respectively, so as to ensure security. The first log entries of the AI system are stored in a first database, and the root value of a first Merckel tree, the root value of a target Merckel tree and the verification information generated based on the first log entries of the AI system are stored in a second database. In the process, the target Mercker tree is constructed based on the first Mercker tree and other second Mercker trees constructed in the previous time period, and the hierarchical construction method of the Mercker trees can support storage of relevant information corresponding to mass log data generated by the AI system, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety. Further, since the mercker tree has the characteristics of non-tampering, transparency and credibility, whether log entries are tampered or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
The following describes an architecture of the log management system provided in the present application.
Fig. 1 is a schematic architecture diagram of a log management system according to an embodiment of the present application. As shown in fig. 1, the log management system includes an AI system 101 and a storage server 102, and the storage server 102 is directly or indirectly connected to the AI system 101 through a wired network or a wireless network. In some embodiments, the log management system further comprises a verifier 103 and an audit center 104. The storage server 102 and the verifier 103 can be directly or indirectly connected through a wired network or a wireless network, and the storage server 102 and the audit center 104 can be directly or indirectly connected through a wired network or a wireless network, which is not limited in the embodiment of the present application.
The AI system 101 serves to transmit the generated log data to the storage server 102. The AI system 101 may be any type of AI system, such as an AI model-based operating system, an AI technology-based attack detection system, and the like, and the embodiment of the present application is not limited to the type of the AI system. In some embodiments, the log data indicates records of all system operations/system operations in the AI system, for example, for a certain operation in the AI system, the log data of the operation indicates a date, a time, an operator, and contents of the operation, etc., which are not limited in this embodiment. It should be noted that, in the embodiment of the present application, an AI system is introduced as an example that belongs to a log management system, and in some embodiments, the AI system refers to any AI system that is independent from the log management system, that is, the log management system can provide a log management service for any AI system, which is not limited in the embodiment of the present application.
The storage server 102 is configured to process log data generated by the AI system and store the processed log data. The storage server 102 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like. The number of the storage servers 102 may be more or less, and the embodiment of the present application is not limited thereto.
In some embodiments, the storage server 102 is further configured to send verification data related to the log data of the AI system to the verifier 103. The checker 103 is configured to check, based on the received check data, whether or not the new log is added after the old log in the processed log data stored in the storage server 102, the log singleness of the AI system. In some embodiments, the verifier 103 is implemented by one server or a server cluster consisting of a plurality of servers. In some embodiments, the verifier 103 is implemented by a server cluster composed of a plurality of servers, where the server cluster is a cluster based on a consensus mechanism, and this is not limited in this embodiment of the present application.
In some embodiments, storage server 102 is further configured to receive a log audit request sent by audit center 104 for the AI system, and send audit data related to log data of the AI system to audit center 104 according to the log audit request. The audit center 104 is configured to determine an audit result of the AI system according to the received audit data, or audit the AI system according to the received audit data. In some embodiments, the audit center 104 is implemented by a terminal, such as but not limited to a smart phone, a tablet, a laptop or desktop computer. In some embodiments, the audit center 104 runs a log audit application, through which a user can audit the AI system. It should be noted that audit center 104 may refer to one of a plurality of audit centers in general, and the embodiment of the present application is illustrated only by audit center 104.
In some embodiments, the wired or wireless networks described above use standard communication techniques and/or protocols. The network is typically the internet, but can be any network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wired or wireless network, a private network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including hypertext markup language (HTML), extensible markup language (XML), and the like. In addition, all or some of the links can be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), transport Layer Security (TLS), virtual Private Network (VPN), internet protocol security (IPsec), and the like. In other embodiments, custom and/or dedicated data communication techniques can also be used in place of or in addition to the data communication techniques described above.
The following describes the architecture of the storage server 102 in the log management system.
Fig. 2 is a schematic architecture diagram of a storage server according to an embodiment of the present application. It should be understood that fig. 2 is only an exemplary diagram illustrating the structure of the storage server 102, and the present application does not limit the division of the various parts in the storage server 102. As shown in fig. 2, the storage server 102 includes a first database 1021, a second database 1022, and a processing center 1023. The first database 1021 and the second database 1022 are different types of databases, and are disposed in different security areas of the storage server 102. The processing center 1023 is used for realizing various functions of the storage server 102. In some embodiments, the storage server 102 includes functionality including, but not limited to: log storage function and log audit function. The functions of the respective sections in the storage server 102 will be briefly described below. It should be noted that, in order to improve readability of the embodiments, detailed description of implementation processes is omitted, and reference is made to relevant description of the following method embodiments.
The first database 1021 is used for storing a plurality of first log entries of the AI system, which correspond to log data generated by the AI system during a first time period. In some embodiments, the first database 1021 is an object-type database, for example, an elastic search-based database (ES database), which is not limited in this embodiment.
The second database 1022 is used for storing the tree root value of the first mercker tree, the tree root value of the target mercker tree and the verification information of the target mercker tree; the first Mercker tree is constructed based on the first plurality of log entries, the target Mercker tree is constructed based on the first Mercker tree and at least one second Mercker tree, the second Mercker tree is constructed based on a second plurality of log entries, the second plurality of log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and verification information of the target Mercker tree indicates a construction process of the target Mercker tree. In some embodiments, the second database 1022 is a relational database, such as an Oracle database, a DB2 database, or a MySQL database, etc., which is not limited in this embodiment.
The processing center 1023 is configured to process log data generated by the AI system in a first time period, generate a plurality of first log entries, and store the plurality of first log entries in the first database 1021; constructing a first merkel tree by using a plurality of hash values corresponding to the plurality of first log entries as leaf nodes, and storing a tree root value of the first merkel tree in a second database 1022; taking the root value of the first merkel tree and the root value of the at least one second merkel tree as leaf nodes, constructing a target merkel tree, and storing the root value of the target merkel tree and the verification information of the target merkel tree in a second database 1022. It should be noted that this process is also a process in which the processing center 1023 implements the log storage function described above.
In some embodiments, the processing center 1023 runs the first database 1021 and the second database 1022 through virtual machines. In other embodiments, the processing center 1023 runs the first database 1021 and the second database 1022 through a container engine (docker), which is not limited in this embodiment.
In some embodiments, the first and at least one second merkel trees that are directly constructed from the log entries are collectively referred to as a small merkel tree (or first-level merkel tree), and accordingly, the tree root values of these merkel trees are referred to as small tree root values; a target mercker tree constructed from the root values of a plurality of mercker trees is called a large mercker tree (or a second-level mercker tree), and accordingly, the root value of the target mercker tree is called a large root value.
Some optional functions of the first database 1021, the second database 1022 and the processing center 1023 are described below.
In some embodiments, the first database 1021 is further configured to store a verification path of the plurality of first log entries, where the verification path includes an intermediate node of the corresponding first log entry, and the intermediate node is a node on a path of the first merkel tree constructed by taking the corresponding first log entry as a starting point.
In some embodiments, the first database 1021 is also used to store log data, hash values, and generation times for the plurality of first log entries.
In some embodiments, the processing center 1023 is further configured to construct the first merkel tree by taking a plurality of hash values corresponding to the first plurality of log entries as leaf nodes if the first plurality of log entries satisfy a first condition; the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
In some embodiments, the processing center 1023 is further configured to construct the target merkel tree by taking the root value of the first merkel tree and the root value of the at least one second merkel tree as leaf nodes if the at least one second merkel tree satisfies a second condition; wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second duration.
In some embodiments, the processing center 1023 is further configured to receive a first log audit request indicating that the plurality of first log entries were audited; based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database; auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree. This process is a process in which the processing center 1023 realizes the log audit function described above.
In some embodiments, the processing center 1023 is configured to determine a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries; constructing a third Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes; determining that the first plurality of log entries audit passed if the root value of the third Merck tree is the same as the root value of the first Merck tree.
In some embodiments, the processing center 1023 is further configured to store the root value of the target mercker tree and the check information of the target mercker tree in the form of blocks on a block chain.
In addition, the storage server 102 may also include other functional units, for example, the storage server 102 further includes a PKI signature center for generating digital signatures for the tiles on the blockchain. For another example, the storage server 102 further includes a timestamp center, configured to generate a corresponding timestamp for each log entry of the AI system, so that the storage server numbers each log entry based on the timestamp to obtain a log identifier of each log entry. The embodiment of the present application does not limit the specific architecture of the storage server 102.
The log management system provided by the embodiment of the present application is described with reference to fig. 1 and fig. 2, and the deployment environments of the storage server 102, the verifier 103, and the audit center 104 in the log management system are described below.
The embodiment of the present application provides a computing device, which can be configured as the storage server 102 in the log management system.
Schematically, fig. 3 is a schematic structural diagram of a computing device provided in an embodiment of the present application. As shown in fig. 3, the computing device 300 includes a memory 301, a processor 302, a communication interface 303, and a bus 304. The memory 301, the processor 302 and the communication interface 303 are connected to each other through a bus 304.
The memory 301 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 301 may store at least one piece of program code, and when the program code stored in the memory 301 is executed by the processor 302, the processor 302 and the communication interface 303 are used to execute the log management method executed by the storage server in the method embodiments described below.
The processor 302 may be a Network Processor (NP), a Central Processing Unit (CPU), an application-specific integrated circuit (ASIC), or an integrated circuit for controlling the execution of programs according to the present disclosure. The processor 302 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The number of the processors 302 may be one or more. The communication interface 303 enables communication between the computing device 300 and other devices or communication networks using transceiver modules, such as transceivers. For example, the data may be acquired through the communication interface 303.
The memory 301 and the processor 302 may be separately provided or integrated together.
Bus 304 may include a path that transfers information between components of computing device 300 (e.g., memory 301, processor 302, communication interface 303).
It should be noted that in some embodiments, the storage server is deployed on a single computing device. In some embodiments, the storage server may also be distributively deployed in different environments, for example, the storage server is logically divided into multiple parts, each part has different functions, each part is separately deployed on one computing device, and the embodiment of the present application does not limit the deployment manner of the storage server.
The embodiment of the present application further provides a computing device, which can be configured as the verifier 103 in the log management system.
Schematically, fig. 4 is a schematic structural diagram of a computing device provided in an embodiment of the present application. The computing device 400 may have a large difference due to different configurations or performances, and includes one or more processors 401 and one or more memories 402, where the memory 402 stores at least one program code, and the at least one program code is loaded by the processor 401 and executes a log management method executed by a checker in the method embodiments described below. Certainly, the computing device 400 can also have components such as a wired or wireless network interface, a keyboard, an input/output interface, and the like, so as to perform input and output, and the computing device 400 further includes other components for implementing device functions, which are not described herein again.
It should be noted that, in some embodiments, the verifier is separately deployed on a computing device. In some embodiments, the verifier is deployed on multiple computing devices. In some embodiments, the checkers are distributively deployed in the form of intelligent contracts. The embodiment of the present application does not limit the deployment manner of the checker.
The embodiment of the present application further provides a computing device, which can be configured as the audit center 104 in the log management system.
Schematically, fig. 5 is a schematic structural diagram of a computing device provided in an embodiment of the present application. The computing device 500 may have a large difference due to different configurations or performances, and includes one or more processors 501 and one or more memories 502, where the memory 502 stores at least one program code, and the at least one program code is loaded and executed by the processor 501 to implement the log management method performed by the audit center in the method embodiments described below. Certainly, the computing device 500 may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the computing device 500 further includes other components for implementing functions of the device, which are not described herein again.
It should be noted that in some embodiments, the audit center is deployed separately on a single computing device. In some embodiments, the audit center is deployed on one of the computing devices in the audit cluster. In some embodiments, the audit center may also be distributively deployed in different environments, for example, the audit center is logically divided into a plurality of portions, each portion has a different function, and each portion is separately deployed on one computing device.
With reference to fig. 6, a log management method provided in the embodiment of the present application is exemplarily described below on the basis of introducing the architecture of the log management system provided in the embodiment of the present application.
Fig. 6 is a schematic flowchart of a log management method according to an embodiment of the present application. As shown in fig. 6, the log management method is applied to the log management system shown in fig. 1, and is described below by taking the interaction among the AI system 101, the storage server 102, the verifier 103, and the audit center 104 as an example. The steps performed by the log management method storage server 102 described below are illustratively performed by the processing center 1023 shown in fig. 2. The log management method includes the following steps 601 to 610.
601. The AI system sends log data generated by the AI system during a first time period to a storage server.
In the embodiment of the present application, the first time period is a time period of any duration. The log data produced by the AI system for the first time period indicates a record of all system operations/system operations produced by the AI system for the first time period. In other words, the AI system may send log data of all log types generated by the AI system for the first time period to the storage server. Among them, log types include but are not limited to: the log includes a running log, an alarm log, an operation log, a message log, a status log, and the like, which is not limited in this embodiment of the present application.
In some embodiments, the first time period is a preset time period. For example, the first time period is one day, that is, the AI system transmits log data generated by the AI system every day to the storage server in units of days. For another example, the first time period is one hour, that is, the AI system sends the log data generated by the AI system every hour to the storage server in units of hours, and the specific defining manner of the first time period in the embodiment of the present application is not limited. In some embodiments, the AI system sends the log data generated by the AI system to the storage server in real time, and the timing for sending the log data by the AI system is not limited in the embodiments of the present application.
In some embodiments, the log data is raw log data. That is, the AI system sends the raw log data generated by the AI system for the first time period directly to the storage server. In this way, the storage server receives all log data generated by the AI system in the first time period, and the integrity of log storage is ensured. In other embodiments, the AI system has a log data processing function, and is configured to classify the original log data, where the log data carries an identifier of a log type to which the log data belongs. For example, when the log data includes log data corresponding to two log types, the AI system adds, according to the log types, respective identifiers of the log types to which the AI system belongs to the log data, and then sends the log data to the storage server. By the method, the data processing amount of the subsequent storage server can be reduced, and the log management efficiency is improved.
602. The storage server receives log data generated by the AI system during a first time period.
603. The storage server processes log data generated by the AI system in a first time period, generates a plurality of first log entries, and stores the plurality of first log entries in a first database.
In the embodiment of the present application, the plurality of first log entries carry log identifiers of respective log entries, and the log identifiers are used for uniquely identifying each log entry and are monotonically increased. The storage server receives log data generated by the AI system in a first time period, normalizes the log data to obtain a plurality of first log entries, and stores the plurality of first log entries in a first database. In some embodiments, the first database is an object-type database. Because the object-type database has large storage capacity, the storage format is not limited, and the permission for reading and writing data is not limited, the log management method provided by the embodiment of the application can support the performance problem of instant storage/query of mass log entries under the condition that the first database is the object-type database, thereby effectively improving the log management efficiency.
In some embodiments, the normalization process includes parsing the log data to obtain corresponding log entries, and normalizing the log entries into a unified normalized log format to obtain the first plurality of log entries.
In some embodiments, the normalization process includes sorting the first plurality of log entries by generation time of the log entries, numbering each log entry, and obtaining a corresponding log identification. In some embodiments, when the storage server includes a timestamp center, the timestamp center is configured to generate a corresponding timestamp for each log entry, so as to reflect a precedence relationship between the log entries in real time, which is convenient for the storage server to number each log entry based on the timestamp, and obtain a globally unique and monotonically increasing log identifier. For example, the log generation timestamp uses at least one of a physical clock, a logical clock, a mixed physical clock, or a mixed logical clock (HLC), which is not limited in this embodiment.
Illustratively, taking the first database as the ES database as an example, the storage server uses the log identifier of each first log entry as a primary key index, and stores the plurality of first log entries in the ES database. For example, for any first Log entry, the Log identifier (e.g., log id: 001) of the first Log entry is a primary key index, and the contents of the first Log entry include: the behavior occurrence time (such as XXXX year XX month XX day XX minute XX second), the identification of the log type (such as alarm log: type 1), the user identification (such as user id: user A), the generation time (such as time stamp), and the log data (such as RAW data), etc., and the format and the content of the first log entry are not limited in the embodiments of the present application. In some embodiments, in a case that the number of log entries stored in the ES database is greater than a preset value, the log entries are divided by further creating an index, for example, the index is created according to a specified time period, which is not limited in this application.
604. The storage server constructs a first Merck tree by using the hash values corresponding to the first log entries as leaf nodes, and stores the root value of the first Merck tree in a second database.
In this embodiment, after generating a plurality of first log entries, the storage server calls a hash function to generate a plurality of hash values corresponding to the plurality of first log entries, constructs a first merkel tree using the plurality of hash values as leaf nodes, and stores a tree root value of the first merkel tree in a second database. In some embodiments, the root value of the first merkel tree can be used as audit data (also referred to as a log audit voucher) for the purpose of auditing the AI system. In some embodiments, the second database is a relational database. Because the storage format of the relational database is relatively fixed, the permission for reading and writing data needs to be limited, and the security of the data in the database can be guaranteed, the log management method provided by the embodiment of the application can effectively improve the security under the condition that the second database is the relational database.
In some embodiments, the storage server also stores information about the first Mercker tree in a second database. Wherein, the related information of the first Merck tree comprises: the log identifier of the log entry corresponding to each leaf node, the generation time of the log entry corresponding to each leaf node, the number of all leaf nodes, and the like. Illustratively, taking the second database as a MySQL database as an example, the MySQL database is used to maintain a target log list, and after the storage server constructs the first merck tree, the storage server stores the root value and the related information of the first merck tree in the target log list.
In some embodiments, the storage server constructs the first merkel tree if a first plurality of log entries satisfies a first condition, wherein the first condition is that a number of the first plurality of log entries is greater than or equal to a first threshold, or that log data generated by the AI system is not received for a first period of time after the first plurality of log entries are generated.
Schematically, the following describes an alternative case in which the storage server builds the first merkel tree based on different first conditions.
The first condition is that the number of the plurality of first log entries is greater than or equal to a first threshold.
Wherein, the first threshold is a preset threshold. For example, the first threshold is 1024, which is not limited in the embodiment of the present application. The storage server, after generating the plurality of first log entries, determines whether or not the number of the plurality of first log entries is equal to or greater than a first threshold value based on the number of the plurality of first log entries, and constructs a first Merck tree using a plurality of hash values corresponding to the plurality of first log entries as leaf nodes when the number of the plurality of first log entries is equal to or greater than the first threshold value.
In some embodiments, in a case where the number of the plurality of first log entries is less than a first threshold, the storage server caches the plurality of first log entries to a first data block (chunk), while the storage server continues to receive log data sent by the AI system and generates corresponding log entries until a sum of the plurality of first log entries and a newly generated log entry is greater than or equal to the first threshold, the storage server building a first merck tree based on the plurality of first log entries and the newly generated log entry.
By the method, the first Merck tree is constructed under the condition that the number of the first log entries meets the number requirement, so that the problem that one Merck tree is constructed every time the storage server receives the log data of the AI system can be avoided, the data processing amount of the storage server is reduced, the load consumption of the storage server is reduced, and the log management efficiency is improved.
In the second case, the first condition is that the log data generated by the AI system is not received within the first time period after the plurality of first log entries are generated.
Wherein, the first duration is a preset duration. For example, the first time period is 1 hour, which is not limited in the examples of the present application. After the storage server generates the first log entries, the storage server does not receive the log data generated by the AI system within a first time period, and constructs a first Merckel tree by using a plurality of hash values corresponding to the first log entries as leaf nodes.
In some embodiments, after the storage server generates the first log entries, the storage server caches the first log entries in a second data block (chunk), and if the log data generated by the AI system is not received within the first duration, the storage server takes hash values corresponding to the first log entries as leaf nodes to construct a first merkel tree.
By the method, the first Mercker tree is constructed under the condition that the plurality of first log entries meet the time requirement, the storage server can be prevented from waiting for the log data generated by the AI system for a long time until the number of leaf nodes for constructing the first Mercker tree meets the number requirement, and therefore the storage efficiency of the storage server is improved.
It is noted that in some embodiments, the storage server can build the first Merck tree based on other conditions. For example, after generating a plurality of first log entries, if the number of the plurality of first log entries is smaller than a first threshold, the storage server does not receive log data generated by the AI system for a certain period of time, and constructs a first mercker tree by using a plurality of hash values corresponding to the plurality of first log entries as leaf nodes.
In some embodiments, the storage server stores, in the first database, a verification path of the plurality of first log entries after constructing the first merkel tree, the verification path including an intermediate node of a corresponding first log entry, the intermediate node being a node on a path from which the corresponding first log entry is constructed. The storage server correspondingly stores the verification path of each first log entry based on the log identifiers of the plurality of first log entries. For any first log entry, the verification path of the first log entry can be used to verify the existence of the first log entry (which can also be understood as verifying whether the first log entry is tampered with).
Referring to fig. 7, fig. 7 is a schematic structural diagram of a first merkel tree according to an embodiment of the present disclosure. As shown in fig. 7, c1 to c6 are hash values corresponding to the first log entries, and the first merkel tree root value is h (h (h (c 1, c 2), h (c 3, c 4)), h (c 5, c 6)). When the existence of the first log entry corresponding to c1 is checked, the root value of the first mercker tree can be recalculated by the intermediate nodes, the root value is compared with h (h (h (c 1, c 2), h (c 3, c 4)), and h (c 5, c 6)), if the root value is the same as h (h (h (c 1, c 2), h (c 3, c 4)), the first log entry corresponding to c1 exists, that is, the first log entry is not tampered, and if the root value is different from h, the first log entry corresponding to c1 does not exist, that is, the first log entry is tampered.
In some embodiments, the storage server stores a plurality of hash values corresponding to the plurality of first log entries in the first database after building the first merkel tree. The storage server correspondingly stores the hash value of each first log entry based on the log identifiers of the plurality of first log entries.
It should be noted that, after the above steps 603 and 604, the storage server stores the plurality of first log entries in the first database, and in some embodiments, the storage server also stores the verification paths and the hash values of the plurality of first log entries in the first database. In addition, the storage server stores the tree root value of the first mercker tree in a second database, and in some embodiments, the storage server also stores information about the first mercker tree in the second database. Schematically, the above process is understood as: storing the plurality of first log entries, the verification paths of the plurality of first log entries, the hash values, and the like as log original texts of the AI system in a first database (the first database may also be referred to as a log original text database); and storing the tree root value and the related information of the first Mercker tree as audit data of the AI system in a second database (the second database can also be called a log audit database).
605. The storage server takes the tree root value of the first Merckel tree and the tree root value of at least one second Merckel tree as leaf nodes, constructs a target Merckel tree, and stores the tree root value of the target Merckel tree and the verification information of the target Merckel tree in a second database.
In this embodiment, the second merkel tree is constructed based on a plurality of second log entries, where the plurality of second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target merkel tree indicates a construction process of the target merkel tree. That is, the at least one second merkel tree is constructed based on log data generated by the AI system during a previous time period. Wherein the second time period is a time period of any duration. In some embodiments, the second time period is the same as the first time period in duration, which is not limited in this application. In some embodiments, the verification information of the target mercker tree can be used as verification data (also called a single increment voucher) to achieve the purpose of verifying the single increment of the target mercker tree, i.e., to verify whether a new log is added after an old log in log entries stored in the storage server.
In some embodiments, the second database of the storage server stores the tree root value of the at least one second merkel tree. After the storage server constructs the first Mercker tree, the storage server acquires the root value of the at least one second Mercker tree from a second database, the root value of the first Mercker tree and the root value of the at least one second Mercker tree are used as leaf nodes, a target Mercker tree is constructed, and the root value of the target Mercker tree and the verification information of the target Mercker tree are stored in the second database. Illustratively, taking the second database as a MySQL database for maintaining a target log list, after receiving log data generated by the AI system in a previous time period, the storage server constructs at least one second mercker tree, stores a root value and related information of the at least one mercker tree in the target log list, and after constructing the first mercker tree, the storage server obtains a root value of the at least one second mercker tree from the target log list, and constructs the target mercker tree.
In some embodiments, the storage server builds the target merkel tree if the at least one second merkel tree satisfies a second condition, wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold, or that a time interval between building the at least one second merkel tree and building the first merkel tree is greater than or equal to a second duration.
In the following, an alternative case is schematically described in which the storage server builds the second merkel tree on the basis of different second conditions.
The first and second conditions mean that the number of the at least one second merkel tree is greater than or equal to a second threshold value.
Wherein, the second threshold is a preset threshold. For example, the second threshold is 256, which is not limited in the embodiment of the present application. The storage server judges whether the number of the at least one second Merckel tree is larger than or equal to a second threshold value or not based on the number of the at least one second Merckel tree, and if the number of the at least one second Merckel tree is larger than or equal to the second threshold value, the storage server takes the root value of the first Merckel tree and the root value of the at least one second Merckel tree as leaf nodes to construct a target Merckel tree.
In some embodiments, in a case that the number of the at least one second merkel tree is smaller than a second threshold value, the storage server continues to receive the log data sent by the AI system, generates a corresponding log entry, and continues to construct a small merkel tree according to the newly generated log entry until the sum of the numbers of the at least one second merkel tree, the first merkel tree, and the newly constructed small merkel tree is greater than or equal to the second threshold value, and the storage server constructs a target merkel tree. Illustratively, taking the second database as a MySQL database for maintaining a target log list, in the case that the number of the at least one second mercker tree is smaller than a second threshold, the storage server continues to build a new small mercker tree and store corresponding tree root values until the number of tree root values in the target log list is larger than the second threshold, and builds a target mercker tree based on the tree root values in the target log list.
By the method, the target Merck tree is constructed under the condition that the quantity of the at least one second Merck tree meets the quantity requirement, and the storage server can be prevented from frequently constructing the target Merck tree, so that the data processing quantity of the storage server is reduced, the load consumption of the storage server is reduced, and the log management efficiency is improved.
The second condition is that the time interval between the construction of the at least one second merkel tree and the construction of the first merkel tree is greater than or equal to a second time period.
Wherein the second time length is a preset time length. For example, the second time period is 24 hours, which is not limited in the embodiment of the present application. And the storage server acquires the construction time of the at least one second Mercker tree, and if the time interval between the construction of the at least one second Mercker tree and the construction of the first Mercker tree is greater than or equal to a second time length, the tree root value of the first Mercker tree and the tree root value of the at least one second Mercker tree are used as leaf nodes to construct the target Mercker tree.
By the method, the target Merck tree is constructed under the condition that at least one second Merck tree meets the time requirement, so that the storage server can be prevented from waiting for a long time for the number of the Merck trees to meet the requirement, and the storage efficiency of the storage server is improved.
It should be noted that the above two cases are explained based on the number of leaf nodes of the target merkel tree being greater than the second threshold. In some embodiments, there is no limitation on the number of leaf nodes of the target mercker tree, in which case the storage server can construct the target mercker tree based on other conditions, which is not limited in the embodiments of the present application.
In some embodiments, the storage server stores the tree root value of the target merkel tree and the verification information of the target merkel tree in the form of blocks on a blockchain, and the data on the blockchain is stored by the second database. The storage server obtains the block information of the current block in the block chain, generates a target block based on the block information of the current block, the tree root value of the target merkel tree and the verification information of the target merkel tree, and stores the target block in the block chain of the second database. In some embodiments, where the storage server comprises a PKI signature center, the PKI signature center is configured to generate a respective digital signature for each tile on the chain of tiles, and to store the digital signature in each tile. Illustratively, the block information of the target block includes: the block identifier, the digital signature, the tree root value of the target mercker tree, the verification information of the target mercker tree, information of each leaf node of the target mercker tree (for example, the tree root value of the mercker tree corresponding to each leaf node, the total number of leaf nodes, and time period information corresponding to each leaf node, etc.), the offset, and the hash value of the previous block, etc. embodiments of the present application do not limit specific contents of the block information. By the method, the properties of transparent credibility, tamper resistance, traceability and the like of the block chain are utilized, the safety of the tree root value and the verification information of the target Merckel tree is ensured, and the transparency of the AI system is further improved on the premise of ensuring the safety.
It should be noted that, through the above step 605, the storage server stores the tree root value of the target mercker tree and the verification information of the target mercker tree in the second database, so as to facilitate the subsequent verification of the simple increment of the target mercker tree.
Through the above steps 603 to 605, the storage server stores the related information corresponding to the log data generated by the AI system in the first time period through two different databases, respectively, so as to ensure security. The method includes storing a plurality of first log entries of the AI system in a first database, and storing a root value of a first Mercker tree, a root value of a target Mercker tree, and check information, which are generated based on the plurality of first log entries of the AI system, in a second database. In the process, the target Merckel tree is constructed on the basis of the first Merckel tree and other second Merckel trees constructed in the previous time period, and the hierarchical construction mode of the Merckel trees can support storage of relevant information corresponding to mass log data generated by the AI system, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety. Further, since the mercker tree has the characteristics of non-tampering, transparency and credibility, whether log entries are tampered or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
Illustratively, the steps 603 to 605 may also be understood as a process in which the storage server implements a log storage function. The above steps 603 to 605 will be exemplified with reference to fig. 8. Fig. 8 is a schematic diagram of a log management method according to an embodiment of the present application. As shown in fig. 8, the storage server receives log data generated by the AI system in a first time period, normalizes the log data to obtain 1024 first log entries, numbers each first log entry according to the generation time of each first log entry to generate a corresponding log id, and stores the plurality of first log entries in the first database. Further, the storage server constructs a first merkel tree (i.e., a small merkel tree) using the hash values of the 1024 first log entries as leaf nodes, and stores a tree root value (i.e., a small tree root value) of the first merkel tree in the second database. In the case that the number of at least one second merkel tree in the second database meets the number requirement, the tree root value of the first merkel tree and the tree root value of at least one second merkel tree are used as leaf nodes, a target merkel tree (i.e. a large merkel tree) is constructed, and the tree root value (i.e. the large tree root value) of the target merkel tree and the verification information of the target merkel tree are stored on a block chain in the form of blocks.
In some embodiments, in the case that the storage server has a log auditing function, the storage server can audit corresponding log entries according to the received log auditing request. The following describes a process of implementing the log audit function by the storage server. The method comprises the following steps of one step to three steps.
Step one, a storage server receives a first log audit request, and the first log audit request indicates that the plurality of first log entries are audited.
The first log audit request carries an identifier of the AI system and log audit time information, and the time information indicates a first time period. In some embodiments, the first log audit request is sent by an audit center in the log management system. In other embodiments, the first journal audit request is sent by any third party audit organization, and the source of the first journal audit request is not limited in the embodiments of the present application.
And step two, the storage server acquires the plurality of first log entries from a first database based on the first log audit request, and acquires the root value of the first Merckel tree from a second database.
The storage server obtains log identifiers of a plurality of first log entries based on the identifiers of the AI system and the time information of log audit carried in the first log audit request, obtains the plurality of first log entries from a first database based on the log identifiers of the plurality of first log entries, and obtains a root value of a first Merck tree from a second database.
In some embodiments, in a case where the storage server stores, in the form of a block, a root value of a target mercker tree and check information of the target mercker tree on a block chain, the storage server parses, based on the first log audit request, the block information stored on the block chain, locates log identifiers of a plurality of first log entries corresponding to the first log audit request, obtains, based on the log identifiers of the plurality of first log entries, the plurality of first log entries from a first database, and obtains, from a second database, the root value of the first mercker tree, which is not limited in this embodiment of the present application.
And step three, auditing the first log entries by the storage server based on the first log entries and the root value of the first Mercker tree.
The storage server calls a hash function based on log data corresponding to the first log entries, recalculates a plurality of hash values corresponding to the first log entries, constructs a Mercker tree by taking the hash values as leaf nodes, compares the root value of the constructed Mercker tree with the root value of the first Mercker tree, determines that the first log entries are audited to pass if the root values of the two trees are the same, and determines that the first log entries are not audited to pass if the root values of the two trees are different, namely, tampered log entries exist in the first log entries. In this step three, the storage server determines a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries; constructing a third Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes; and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
In some embodiments, the storage server is capable of locating a tampered log entry of the first plurality of log entries in the event that the first plurality of log entries are not audited. Illustratively, the first database stores therein verification paths of the plurality of first log entries, and the storage server is capable of verifying the existence of each first log entry based on the verification path of each first log entry, thereby locating a tampered log entry and improving transparency of the AI system. It should be noted that the verification process here is the same as the corresponding text segment of fig. 7, and therefore is not described herein again.
Referring to fig. 9, fig. 9 is a schematic diagram of a log auditing process provided by an embodiment of the present application. As shown in fig. 9, when auditing log entries in the AI system, the storage server obtains log data corresponding to the log entries from the first database, recalculates the hash value, constructs a merkel tree, obtains a tree root value of the merkel tree corresponding to the log entries from the second database, which is also referred to as a log audit certificate, and then compares the tree root value of the newly constructed merkel tree with the tree root value read from the second database to obtain an audit result.
The first log audit request is described by way of example as auditing a plurality of first log entries. It should be understood that, in practical application, the first log audit request may indicate that the audit AI system audits log entries corresponding to any time period, and the specific process is the same as the first to third steps, that is, the storage server may locate the log identifier of the log entry to be audited through the first log audit request, which is not described herein again. In some embodiments, the first log audit instruction audits log entries corresponding to a target user, in which case, the storage server obtains the log entries corresponding to the target user from the first database according to an identifier of the target user carried in the first log audit request, calculates hash values respectively, constructs a mercker tree, obtains tree root values of small tree roots corresponding to the log entries from the second database, and verifies whether the tree root values of the newly constructed mercker tree are the same as the tree root values stored in the second database. Similarly, the first log audit request may also indicate that log entries corresponding to the log type are audited, e.g., all alarm logs in the AI system are audited, and so on. In some embodiments, the first log audit request indicates a log entry of an audit complex condition, for example, an alarm log generated by the audit AI system in a first time period, for example, a log entry corresponding to a target user in the audit AI system in the first time period, and the like, which is not limited in this embodiment of the present application.
The above embodiment describes the log management method provided in the embodiment of the present application based on the interaction between the AI system and the storage server in the log management system shown in fig. 1. In some embodiments, the log management system shown in fig. 1 further comprises a verifier and an audit center. The calibrator is used for verifying the single increment of the log of the AI system, and the auditing center is used for auditing the AI system. Based on this, some optional implementation steps of the log management method are described below through steps 606 to 610.
606. The checker obtains the tree root value of the target Mercker tree and the checking information of the target Mercker tree from the block chain.
When the storage server stores the root value of the target mercker tree and the check information of the target mercker tree in the form of the block on the block chain, the checker obtains the root value of the target mercker tree and the check information of the target mercker tree from the block chain to check the single increment of the target mercker tree, namely check the single increment of the log of the AI system.
In some embodiments, the verifier operates in the form of an intelligent contract that obtains a root value of the target mercker tree and verification information of the target mercker tree from the block chain, the intelligent contract indicating that the monotonicity of the target mercker tree is to be verified periodically. By means of the intelligent contract, the checker periodically and automatically checks the target Merck tree, the purpose of monitoring the log storage process is achieved, single-point equipment is avoided, and therefore transparency of the AI system is effectively improved on the premise that safety is guaranteed.
607. The verifier verifies the singleton of the target merck tree.
The checker obtains the block information of the previous block from the block chain, wherein the previous block is the newest block on the block chain before storing the root value of the target merkel tree and the verification information of the target merkel tree, the checker obtains the root value of the fourth merkel tree and the verification information of the fourth merkel tree based on the block information of the previous block, the fourth merkel tree is the newest merkel tree that passes single increment verification on the block chain, and the verification information of the fourth merkel tree indicates the construction process of the fourth merkel tree; and checking the single increment of the target Mercker tree based on the root value of the fourth Mercker tree, the checking information of the fourth Mercker tree, the root value of the target Mercker tree and the checking information of the target Mercker tree.
Referring now to FIG. 10, an embodiment of the verifier verifying the singleton property of the target Mercker tree is illustrated. Fig. 10 is a schematic diagram of a simple increase verification provided in an embodiment of the present application. As shown in fig. 10 (a), the root value of the fourth mercker tree is h (h (c 1, c 2), h (c 3, c 4)), h (c 5, c 6)), and the verification information of the fourth mercker tree is c2, h (c 3, c 4), and h (c 5, c 6), and as shown in fig. 10 (b), the root value of the target mercker tree is h21, and the verification information of the target mercker tree is h4, h17, and h20. Illustratively, the verifier verifies the singleness of the target mercker tree by three steps:
1) The checker calculates the latest version of the root value of the fourth mercker tree based on c1, c2, h (c 3, c 4), h (c 5, c 6) and h4, judges whether the latest version is equal to h16, if so, executes the next step, and if not, exits the current process, and determines that the single increment check of the target mercker tree fails.
2) And the checker calculates the hash value based on h16 and h17, judges whether the hash value is equal to h19 or not, if so, executes the next step, and if not, exits the current process and determines that the singly-augmented verification of the target Mercker tree fails.
3) The checker calculates the hash value based on h19 and h20, judges whether the hash value is equal to h21, if so, determines that the singly-augmented check of the target tacle tree passes, and if not, determines that the singly-augmented check of the target tacle tree fails.
Through the above steps 606 and 607, when the storage server stores the root value of the target mercker tree and the verification information of the target mercker tree in the form of a block on the block chain, the verifier of the log management system can verify the single increment of the target mercker tree, that is, verify the single increment of the log of the AI system, thereby achieving the purpose of monitoring the log storage process of the storage server, ensuring that the log is only appended, and not covering the previously stored log, and thus effectively improving the transparency of the AI system.
608. And the auditing center sends a second log auditing request to the storage server, wherein the second log auditing request indicates that the plurality of first log entries are audited.
Under the condition that the log management system comprises the audit center, the audit center can send corresponding log audit requests to the storage server according to user requirements. The second log audit request carries the identifier of the AI system and the log audit time information. In some embodiments, the second log audit request further carries an identifier of an audit type, where the identifier of the audit type indicates that the audit center needs the storage server to feed back the log audit result.
609. And the storage server sends a log audit result to the audit center based on the second log audit request.
Wherein the log audit result indicates whether the first plurality of log entries are audited. And the storage server determines that the audit center needs to feed back the log audit result to the storage server based on the identifier of the audit type carried in the second log audit request, and sends the log audit result to the audit center.
610. And the auditing center receives the log auditing result.
In some embodiments, the audit type flag carried in the second log audit request indicates that the audit center needs the storage server to feed back the log audit information. In this case, the storage server sends log audit information to the audit center based on the second log audit request, the log audit information including the first plurality of log entries and a root value of the first Merck tree. After receiving the log audit information, the audit center determines a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries; constructing a fifth Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes; determining that the first plurality of log entries audit passed if the root value of the fifth Merck tree is the same as the root value of the first Merck tree. This process is similar to the process of auditing the first log entries by the storage server in step 605, and therefore, will not be described herein again.
Through the steps 608 to 610, the audit center can send a log audit request to the storage server to audit log entries of the AI system, wherein the audit center can receive a log audit result fed back by the storage server to reduce load consumption of the audit center and improve log management efficiency, and the audit center can also receive log audit information fed back by the storage server and automatically audit to ensure safety and improve transparency of the AI system.
In addition, the execution sequence and execution timing of the steps 606 to 610 are not limited in the embodiment of the present application. That is, in some embodiments, the log management method includes steps 601 to 607 described above. In some embodiments, the log management method includes steps 601 to 605 and steps 608 to 610 described above. In some embodiments, in the case that the log management method includes the above steps 601 to 610, the verifier performs the steps 608 to 610 while performing the steps 606 and 607, and the audit center and the storage server perform the above steps 608 to 610.
In summary, the embodiment of the present application provides a log management method, which respectively stores, through two different databases, related information corresponding to log data generated by an AI system in a first time period, so as to ensure security. The method includes storing a plurality of first log entries of the AI system in a first database, and storing a root value of a first Mercker tree, a root value of a target Mercker tree, and check information, which are generated based on the plurality of first log entries of the AI system, in a second database. In the process, the target Mercker tree is constructed based on the first Mercker tree and other second Mercker trees constructed in the previous time period, and the hierarchical construction method of the Mercker trees can support storage of relevant information corresponding to mass log data generated by the AI system, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety. Further, since the merkel tree has the characteristics of non-tampering, transparency and credibility, whether log entries are tampered or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
Through the embodiment shown in fig. 6, a specific implementation of the log management method provided in the embodiment of the present application is described. The following describes, with reference to fig. 11, a log management method provided in an embodiment of the present application.
Fig. 11 is a schematic diagram of a log management method according to an embodiment of the present application. As shown in fig. 11, the log management method is applied to a log management system including an AI system, a storage server, a checker, and an audit center.
The AI system is used for sending the generated log data to the storage server.
The storage server is used for carrying out standardization processing on log data generated by the AI system to obtain a plurality of log entries, obtaining a small Mercker tree and a large Mercker tree by a way of hierarchically constructing the Mercker trees based on the plurality of log entries, and storing the plurality of log entries, verification paths of the plurality of log entries, hash values and the like as log original texts of the AI system in a first database (namely a log original text database); the tree root value and the related information of the small Mercker tree are used as auditing data of the AI system and stored in a second database (namely a log auditing database), and the tree root value and the verification information of the large Mercker tree are used as verification data of the AI system and stored in a block chain.
The checker is used for operating in the form of an intelligent contract, periodically obtaining checking data from the block chain, namely the root value and the checking information of the large Merck tree, and checking the simple increment of the large Merck tree to ensure that the new log is added after the old log.
The auditing center is used for sending a log auditing request to the storage server to audit log entries of the AI system, wherein the auditing center can receive a log auditing result fed back by the storage server to reduce load consumption of the auditing center and improve log management efficiency, and can also receive log auditing information fed back by the storage server and automatically audit to ensure safety and improve transparency of the AI system.
In the process, the log management system can be applied to various scenes in which the log data generated by the AI system needs to be uniformly tracked and recorded, and the related information corresponding to the log data generated by the AI system is respectively stored through two different databases so as to ensure the safety. Moreover, the hierarchical establishment of the Merckel tree can support the storage of relevant information corresponding to massive log data generated by the AI system, so that the log data generated by the AI system is uniformly tracked and recorded, and the transparency of the AI system is effectively improved on the premise of ensuring the safety. Further, since the merkel tree has the characteristics of non-tampering, transparency and credibility, whether log entries are tampered or not can be determined by comparing the contents stored in the two databases, so that the transparency of the AI system is effectively improved.
Fig. 12 is a schematic structural diagram of a log management device according to an embodiment of the present application. The log management apparatus 1200 shown in fig. 12 realizes, for example, the function of a storage server in the log management method shown in fig. 6. As shown in fig. 12, the log management apparatus 1200 includes a first storage module 1201 and a second storage module 1202.
A first storage module 1201, configured to process log data generated by the AI system in a first time period, generate a plurality of first log entries, and store the plurality of first log entries in a first database;
a second storage module 1202, configured to construct a first merkel tree by using a plurality of hash values corresponding to the plurality of first log entries as leaf nodes, and store a tree root value of the first merkel tree in a second database;
the second storage module 1202 is further configured to construct a target mercker tree by using the tree root value of the first mercker tree and the tree root value of at least one second mercker tree as leaf nodes, store the tree root value of the target mercker tree and verification information of the target mercker tree in the second database, where the second mercker tree is constructed based on a plurality of second log entries, the plurality of second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target mercker tree indicates a construction process of the target mercker tree.
In some embodiments, the first storage module 1201 is further configured to:
storing a verification path of the plurality of first log entries in the first database, the verification path including an intermediate node corresponding to the first log entry, the intermediate node being a node on a path constructing the first merkel tree starting from the corresponding first log entry.
In some embodiments, the first storage module 1201 is further configured to:
the log data, hash value, and generation time of the plurality of first log entries are stored in the first database.
In some embodiments, the first database is an object-type database; the second database is a relational database.
In some embodiments, the first storage module 1201 is further configured to:
if the first log entries meet a first condition, constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
In some embodiments, the second storage module 1202 is further configured to:
if the at least one second Mercker tree meets a second condition, constructing the target Mercker tree by taking the root value of the first Mercker tree and the root value of the at least one second Mercker tree as leaf nodes;
wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second duration.
In some embodiments, the apparatus further comprises an audit module; the audit module is configured to:
receiving a first log audit request, the first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
In some embodiments, the audit module is to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Merck tree by taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes;
and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
In some embodiments, the second storage module 1202 is further configured to store the tree root value of the target merck tree and the verification information of the target merck tree in a block form on a block chain.
It should be noted that: in the log management apparatus 1200 according to the above embodiment, only the division of the functional modules is illustrated when performing log management, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to complete all or part of the functions described above. In addition, the log management device and the log management method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.
The terms "first," "second," and the like in this application are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it should be understood that "first," "second," and "nth" do not have any logical or temporal dependency or limitation on the number or order of execution. It will be further understood that, although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first merkel tree may be referred to as a second merkel tree, and similarly, a second merkel tree may be referred to as a first merkel tree, without departing from the scope of various described examples. Both the first and second merkel trees may be merkel trees, and in some cases, may be separate and distinct merkel trees.
The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more, for example, a plurality of merkel trees means two or more merkel trees.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, and these modifications or substitutions should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a program product. The program product includes one or more program instructions. When loaded and executed on a computing device, cause the flow or functions according to embodiments of the application to occur, in whole or in part.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (36)
1. A log management system, characterized in that the system comprises a first database and a second database;
the first database is used for storing a plurality of first log entries of an Artificial Intelligence (AI) system, and the plurality of first log entries correspond to log data generated by the AI system in a first time period;
the second database is used for storing the root value of the first Mercker tree, the root value of the target Mercker tree and the verification information of the target Mercker tree;
wherein the first Mercker tree is constructed based on the first plurality of log entries, the target Mercker tree is constructed based on the first Mercker tree and at least one second Mercker tree, the second Mercker tree is constructed based on a second plurality of log entries, the second plurality of log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target Mercker tree indicates a construction process of the target Mercker tree.
2. The system of claim 1,
the first database is further configured to store a verification path of the plurality of first log entries, where the verification path includes an intermediate node corresponding to the first log entry, and the intermediate node is a node on a path that constructs the first merck tree starting from the corresponding first log entry.
3. The system according to claim 1 or 2,
the first database is further configured to store log data, hash values, and generation times of the plurality of first log entries.
4. The system according to any one of claims 1 to 3,
the first database is an object type database;
the second database is a relational database.
5. The system of any one of claims 1 to 4, further comprising a processing center;
the processing center is used for:
processing log data generated by the AI system in the first time period, generating a plurality of first log entries, and storing the plurality of first log entries in the first database;
constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes, and storing a tree root value of the first Mercker tree in the second database;
and constructing the target Merckel tree by taking the tree root value of the first Merckel tree and the tree root value of the at least one second Merckel tree as leaf nodes, and storing the tree root value of the target Merckel tree and the verification information of the target Merckel tree in the second database.
6. The system of claim 5, wherein the processing center is further configured to:
if the first log entries meet a first condition, constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
7. The system of claim 5 or 6, wherein the processing center is further configured to:
if the at least one second Merckel tree meets a second condition, constructing the target Merckel tree by taking the root value of the first Merckel tree and the root value of the at least one second Merckel tree as leaf nodes;
wherein the second condition means that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or that a time interval between the construction of the at least one second merkel tree and the construction of the first merkel tree is greater than or equal to a second period of time.
8. The system of any one of claims 5 to 7, wherein the processing center is further configured to:
receiving a first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
9. The system of claim 8, wherein the processing center is configured to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Merck tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
10. The system of any of claims 5 to 9, wherein the processing center is further configured to:
storing the root value of the target Merck tree and the check information of the target Merck tree on a block chain in the form of blocks.
11. The system of claim 10, further comprising a verifier;
the checker is configured to obtain a tree root value of the target mercker tree and check information of the target mercker tree from the block chain, and check the monotonicity of the target mercker tree.
12. The system of claim 11, wherein the verifier is to:
the method is operated in the form of an intelligent contract, the tree root value of the target Mercker tree and the checking information of the target Mercker tree are obtained from the block chain, the single increment of the target Mercker tree is checked, and the intelligent contract indicates that the single increment of the target Mercker tree is checked periodically.
13. The system of claim 11 or 12, wherein the verifier is configured to:
obtaining a root value of a fourth Merck tree and verification information of the fourth Merck tree from the block chain, wherein the fourth Merck tree is the latest Merck tree which passes single increment verification on the block chain, and the verification information of the fourth Merck tree indicates the construction process of the fourth Merck tree;
checking the monotonicity of the target Merckel tree based on the root value of the fourth Merckel tree, the verification information of the fourth Merckel tree, the root value of the target Merckel tree and the verification information of the target Merckel tree.
14. The system of any one of claims 5 to 13, further comprising an audit center;
the audit center is used for:
sending a second log audit request to the processing center, wherein the second log audit request indicates that the plurality of first log entries are audited;
receiving a log audit result, or receiving log audit information; wherein the log audit result indicates whether the first plurality of log entries are audited, and the log audit information includes the first plurality of log entries and a root value of the first Mercker tree.
15. The system of claim 14, wherein upon receiving the log audit information, the audit center is further configured to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a fifth Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
determining that the first plurality of log entries audit pass if the root value of the fifth Merck tree is the same as the root value of the first Merck tree.
16. A method of log management, the method comprising:
processing log data generated by an AI system in a first time period, generating a plurality of first log entries, and storing the plurality of first log entries in a first database;
taking a plurality of hash values corresponding to the plurality of first log entries as leaf nodes, constructing a first Merckel tree, and storing a tree root value of the first Merckel tree in a second database;
taking the tree root value of the first Mercker tree and the tree root value of at least one second Mercker tree as leaf nodes, constructing a target Mercker tree, storing the tree root value of the target Mercker tree and verification information of the target Mercker tree in the second database, wherein the second Mercker tree is constructed based on a plurality of second log entries, the second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target Mercker tree indicates the construction process of the target Mercker tree.
17. The method of claim 16, further comprising:
storing a verification path of the plurality of first log entries in the first database, the verification path including an intermediate node corresponding to the first log entry, the intermediate node being a node on a path constructing the first merkel tree starting from the corresponding first log entry.
18. The method according to claim 16 or 17, further comprising:
storing the log data, hash value, and generation time of the plurality of first log entries in the first database.
19. The method according to any one of claims 16 to 18,
the first database is an object-type database;
the second database is a relational database.
20. The method of any one of claims 16 to 19, further comprising:
if the first log entries meet a first condition, taking a plurality of hash values corresponding to the first log entries as leaf nodes to construct the first Merck tree;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
21. The method of any one of claims 16 to 20, further comprising:
if the at least one second Mercker tree meets a second condition, constructing the target Mercker tree by taking the tree root value of the first Mercker tree and the tree root value of the at least one second Mercker tree as leaf nodes;
wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second time length.
22. The method of any one of claims 16 to 21, further comprising:
receiving a first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a tree root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
23. The method according to claim 22, wherein said auditing the plurality of first log entries based on the plurality of first log entries and a root value of the first merkel tree comprises:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
and if the root value of the third Merck tree is the same as the root value of the first Merck tree, determining that the plurality of first log entries are audited to pass.
24. The method of any one of claims 16 to 23, further comprising:
storing the root value of the target Merck tree and the check information of the target Merck tree on a block chain in the form of blocks.
25. An apparatus for log management, the apparatus comprising:
the first storage module is used for processing log data generated by the AI system in a first time period, generating a plurality of first log entries and storing the plurality of first log entries in a first database;
a second storage module, configured to construct a first merkel tree by using a plurality of hash values corresponding to the plurality of first log entries as leaf nodes, and store a tree root value of the first merkel tree in a second database;
the second storage module is further configured to construct a target mercker tree by using the tree root value of the first mercker tree and the tree root value of at least one second mercker tree as leaf nodes, store the tree root value of the target mercker tree and verification information of the target mercker tree in the second database, where the second mercker tree is constructed based on a plurality of second log entries, the plurality of second log entries correspond to log data generated by the AI system in a second time period, the second time period is a time period before the first time period, and the verification information of the target mercker tree indicates a construction process of the target mercker tree.
26. The apparatus of claim 25, wherein the first storage module is further configured to:
storing, in the first database, a verification path of the plurality of first log entries, the verification path including an intermediate node corresponding to the first log entry, the intermediate node being a node on a path that constructs the first merkel tree starting from the corresponding first log entry.
27. The apparatus of claim 25 or 26, wherein the first storage module is further configured to:
storing the log data, hash value, and generation time of the plurality of first log entries in the first database.
28. The apparatus of any one of claims 25 to 27,
the first database is an object-type database;
the second database is a relational database.
29. The apparatus of any one of claims 25 to 28, wherein the first storage module is further configured to:
if the first log entries meet a first condition, constructing the first Mercker tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
the first condition is that the number of the first log entries is greater than or equal to a first threshold, or log data generated by the AI system is not received within a first time period after the first log entries are generated.
30. The apparatus according to any one of claims 25 to 29, wherein the second storage module is further configured to:
if the at least one second Merckel tree meets a second condition, constructing the target Merckel tree by taking the root value of the first Merckel tree and the root value of the at least one second Merckel tree as leaf nodes;
wherein the second condition is that the number of the at least one second merkel tree is greater than or equal to a second threshold value, or the time interval between the building of the at least one second merkel tree and the building of the first merkel tree is greater than or equal to a second time length.
31. The apparatus of any one of claims 25 to 30, further comprising an auditing module;
the audit module is configured to:
receiving a first log audit request indicating that the plurality of first log entries are audited;
based on the first log audit request, obtaining the plurality of first log entries from the first database, and obtaining a root value of the first Merck tree from the second database;
auditing the first plurality of log entries based on the first plurality of log entries and a root value of the first Mercker tree.
32. The apparatus of claim 31, wherein the audit module is configured to:
determining a plurality of hash values corresponding to the plurality of first log entries based on the plurality of first log entries;
constructing a third Merck tree by taking a plurality of hash values corresponding to the first log entries as leaf nodes;
determining that the first plurality of log entries audit pass if the root value of the third Merck tree is the same as the root value of the first Merck tree.
33. The apparatus according to any one of claims 25 to 32, wherein the second storage module is further configured to:
storing the root value of the target Merck tree and the check information of the target Merck tree on a block chain in the form of blocks.
34. A computing device comprising a processor and a memory, the memory to store at least one piece of program code, the at least one piece of program code to be loaded by the processor and to perform the log management method of any of claims 16 to 24.
35. A computer-readable storage medium for storing at least one program code for performing the log management method of any of claims 16 to 24.
36. A computer program product, which, when run on a computer, causes the computer to perform the log management method of any one of claim 16 to claim 24.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110997928 | 2021-08-27 | ||
CN2021109979281 | 2021-08-27 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115905150A true CN115905150A (en) | 2023-04-04 |
Family
ID=86471455
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111094785.XA Pending CN115905150A (en) | 2021-08-27 | 2021-09-17 | Log management system, method, device and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115905150A (en) |
-
2021
- 2021-09-17 CN CN202111094785.XA patent/CN115905150A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10965445B2 (en) | Blockchain-based unexpected data detection | |
US11204939B2 (en) | Data manifest as a blockchain service | |
US20190207751A1 (en) | Blockchain enterprise data management | |
US11924323B2 (en) | On-chain governance of blockchain | |
US11200260B2 (en) | Database asset fulfillment chaincode deployment | |
US20190207750A1 (en) | Blockchain enterprise data management | |
US11194555B2 (en) | Optimization of execution of smart contracts | |
CN111414413A (en) | Block chain endorsement verification | |
CN112703499A (en) | Distributed platform for computing and trust verification | |
US11269863B2 (en) | Index structure for blockchain ledger | |
US11645268B2 (en) | Database world state performance improvement | |
CN114128214A (en) | Security layer for configuring block chains | |
CN111931220B (en) | Consensus processing method, device, medium and electronic equipment for block chain network | |
EP3744071B1 (en) | Data isolation in distributed hash chains | |
US11455598B2 (en) | Automated conflict resolution | |
US11475365B2 (en) | Verification of stochastic gradient descent | |
JP2023530594A (en) | Permitted Event Processing in Distributed Databases | |
US12033144B2 (en) | Systems and methods for improved indexing of non-standardized, custom smart contracts | |
JP2023520634A (en) | Maintaining contextual integrity | |
US20240232381A1 (en) | Systems and methods for conducting blockchain actions based on network mappings of self-executing program characteristics | |
US11792022B2 (en) | Resolution of conflicting data | |
US20210150597A1 (en) | Automated invoicing | |
US11321298B1 (en) | Automated merge of DLT networks | |
CN115905150A (en) | Log management system, method, device and storage medium | |
CN116070191A (en) | Information processing method and device, storage medium, and program product |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination |