CN110532293B - Data stream life cycle management method and system based on block chain technology - Google Patents
Data stream life cycle management method and system based on block chain technology Download PDFInfo
- Publication number
- CN110532293B CN110532293B CN201910823516.9A CN201910823516A CN110532293B CN 110532293 B CN110532293 B CN 110532293B CN 201910823516 A CN201910823516 A CN 201910823516A CN 110532293 B CN110532293 B CN 110532293B
- Authority
- CN
- China
- Prior art keywords
- data
- data block
- block
- tracing
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2141—Access rights, e.g. capability lists, access control lists, access tables, access matrices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/2151—Time stamp
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data stream life cycle management method and system based on a block chain technology, and belongs to the technical field of data stream life cycle management. The data stream life cycle management method based on the block chain technology comprises the following steps: s1, performing data preprocessing on original data according to a provided data preprocessing method; s2, performing data access on the preprocessed data; s3, cleaning the data by using a data cleaning service; s4, after the data are cleaned, the data are analyzed and used by using a user identity authentication mechanism; and S5, when the data is destroyed, storing administrator information and data block information of the destruction operation into the data block tracing private chain. The data flow life cycle management method based on the block chain technology can solve the problems of disordered data access, association interruption after data processing, unretraceable historical data and potential safety hazard in the use process, and has good popularization and application values.
Description
Technical Field
The invention relates to the technical field of data stream life cycle management, and particularly provides a data stream life cycle management method and system based on a block chain technology.
Background
The existing management software generally only carries out rough and simple management on data stream life cycle management, and mainly has the following problems: firstly, no pretreatment is carried out during data access; after the accessed data is cleaned, the relationship between the data before and after operation cannot be processed, so that the data is discarded or archived and stored, and the data circulation process cannot be continuously recorded; thirdly, when the data is used, the identity authentication of the user is not carried out, the data process cannot be effectively recorded, and potential safety hazards exist; and fourthly, the data cannot be traced after being destroyed, so that the historical data flow information cannot be inquired.
Disclosure of Invention
The technical task of the invention is to provide a data flow life cycle management method based on the block chain technology, which can solve the problems of disordered data access, association interruption after data processing, unretraceable historical data and potential safety hazard in the use process.
A further technical task of the present invention is to provide a data stream lifecycle management system based on the blockchain technique.
In order to realize the purpose, the invention provides the following technical scheme:
a data flow life cycle management method based on block chain technology provides data uplink function, data block information is stored in a data flow tracing private chain, a random number is stored in the data block tracing private chain, a hash value is generated and stored in the data block tracing private chain, data are connected through a hash algorithm, and a time stamp characteristic is provided, the method comprises the following steps:
s1, performing data preprocessing on original data according to a provided data preprocessing method;
s2, performing data access on the preprocessed data, acquiring data source information of each data block, generating description information, and storing the description information in a data block tracing private chain;
s3, cleaning the data by using a data cleaning service, generating a new data module, and generating new description information which is stored in a data block tracing private chain;
s4, after the data are cleaned, the data are analyzed and used by using a user identity authentication mechanism, and each time the data block is used, the user information and use information generation block is stored in the data block tracing private chain;
and S5, when the data is destroyed, storing administrator information and data block information of the destruction operation into the data block tracing private chain.
Preferably, the data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
Preferably, in step S2, when accessing data, a time interval of file aggregation and a threshold of a size of a data block generated by data aggregation are set, and data access processing is performed according to the size of the data block.
Preferably, when data is accessed, in a set time interval, if the file size is larger than a set threshold value, the data is directly accessed without preprocessing; and if the size of the data blocks aggregated in the time interval does not reach the threshold value, performing data access on the aggregated data blocks.
Preferably, in step S4, when the administrator and the developer user in the user identity authentication mechanism register in the system, a pair of public key and private key is generated, and the developer user can operate after performing key authentication each time.
A data flow life cycle management system based on block chain technology is disclosed, the system stores data block information in a data flow tracing private chain based on providing a data chaining function, a random number is stored on the data block tracing private chain, a hash value is generated and stored on the data block tracing private chain, data are connected through a hash algorithm, and a time stamp characteristic is provided, the system comprises the following modules:
the data block tracing private chain module is used for storing data block information;
the identity authentication module is used for authenticating the identity of the user;
the data preprocessing module is used for preprocessing the original data according to the provided data preprocessing method;
the data access module is used for performing data access on the preprocessed data, acquiring data source information of each data block, generating description information and storing the description information in a data block tracing private chain;
the data cleaning module is used for cleaning data by using a data cleaning service, generating a new data module and generating new description information at the same time, and storing the new description information in a data block tracing private chain;
the data use module is used for analyzing and using the user identity authentication mechanism after the data is cleaned, and the user information and use information generation block is stored in the data block tracing private chain when the data block is used each time;
and the data destruction module is used for storing administrator information and data block information of destruction operation into the data block tracing private chain when data is destroyed.
Preferably, the data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
Preferably, when the data access module accesses data, the data access module sets a time interval of file aggregation and a threshold of a data block size generated by data aggregation, and performs data access processing according to the data block size. When the original data is imported, in a set time interval, if the size of the file is larger than a set threshold value, the data is directly accessed without preprocessing; and if the size of the file is smaller than the threshold value, performing aggregation, performing data access after the aggregated data block reaches the set size, and if the size of the aggregated data block does not reach the threshold value in a time interval, performing data access on the aggregated data block.
Preferably, the data use module generates a pair of public key and private key when the administrator and the developer user register in the system in the user identity authentication mechanism, and the developer user can operate after performing key authentication each time.
Compared with the prior art, the data flow life cycle management method based on the block chain technology has the following outstanding beneficial effects: the data flow life cycle management method based on the block chain technology realizes the record management of the whole life cycle of preprocessing, accessing, cleaning, using and destroying the data flow, can standardize the data access, improve the usability by cleaning the data, support the identity authentication and improve the safety, ensure the data to be stored on the data flow tracing chain without being falsified and traceable, ensure the safety of the data and have good popularization and application values.
Drawings
Fig. 1 is a flowchart of a data flow lifecycle management method based on a blockchain technique according to the present invention.
Detailed Description
The method and system for managing data stream lifecycle based on blockchain technology according to the present invention will be described in detail with reference to the accompanying drawings and embodiments.
Examples
The data flow life cycle management method based on the block chain technology provides a data uplink function, data block information is stored in a data flow tracing private chain, random numbers are stored in the data block tracing private chain, hash values are generated and stored in the data block tracing private chain, data are connected through a hash algorithm, a time stamp characteristic is provided, and the characteristic that the data cannot be tampered is guaranteed. The data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
As shown in fig. 1, the method for managing a data stream lifecycle based on a blockchain technique specifically includes the following steps:
s1, performing data preprocessing on original data according to a provided data preprocessing method.
And S2, performing data access on the preprocessed data, acquiring data source information of each data block, generating description information, and storing the description information in a data block tracing private chain.
When data access is carried out, setting a time interval of file aggregation and a threshold value of the size of a data block generated by data aggregation, and carrying out data access processing according to the size of the data block. In a set time interval, if the size of the file is larger than a set threshold value, the data access is directly carried out without preprocessing; and if the size of the file is smaller than the threshold value, performing aggregation, performing data access after the aggregated data block reaches the set size, and if the size of the aggregated data block does not reach the threshold value in a time interval, performing data access on the aggregated data block.
And S3, cleaning the data by using the data cleaning service, generating a new data module, and generating new description information which is stored in the data block tracing private chain.
And S4, the data is analyzed and used by using a user identity authentication mechanism after being cleaned, and the user information and use information generation block is stored in the data block tracing private chain every time the data block is used.
In the user identity authentication mechanism, when an administrator and a developer user register in a system, a pair of public key and private key is generated, and the developer user can operate after key authentication each time.
And S5, when the data is destroyed, storing administrator information and data block information of the destruction operation into the data block tracing private chain.
The data flow life cycle management system based on the block chain technology stores data block information in a data flow tracing private chain based on the function of providing data chaining, stores random numbers on the data block tracing private chain, generates hash values and stores the hash values on the data block tracing private chain, and connects data through a hash algorithm to provide a timestamp characteristic and ensure that the data cannot be tampered. The data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
The system comprises the following modules:
and the data block tracing private chain module is used for storing data block information.
And the identity authentication module is used for authenticating the identity of the user.
And the data preprocessing module is used for preprocessing the original data according to the provided data preprocessing method.
And the data access module is used for performing data access on the preprocessed data, acquiring data source information of each data block, generating description information and storing the description information in the data block tracing private chain.
When the data access module accesses data, setting a time interval of file aggregation and a threshold value of the size of a data block generated by data aggregation, and performing data access processing according to the size of the data block. When the original data is imported, in a set time interval, if the size of the file is larger than a set threshold value, the data is directly accessed without preprocessing; and if the size of the file is smaller than the threshold value, performing aggregation, performing data access after the aggregated data block reaches the set size, and if the size of the aggregated data block does not reach the threshold value in a time interval, performing data access on the aggregated data block.
And the data cleaning module is used for cleaning the data by using the data cleaning service, generating a new data module, and generating new description information which is stored in the data block tracing private chain.
And the data use module is used for analyzing and using the user identity authentication mechanism after the data is cleaned, and the data block is used each time to store the user information and the use information generation block into the data block tracing private chain.
The data use module generates a pair of public key and private key when an administrator and a developer user register in a system in a user identity authentication mechanism, and the developer user can operate after performing key authentication each time.
And the data destruction module is used for storing administrator information and data block information of destruction operation into the data block tracing private chain when data is destroyed.
The above-described embodiments are merely preferred embodiments of the present invention, and general changes and substitutions by those skilled in the art within the technical scope of the present invention are included in the protection scope of the present invention.
Claims (9)
1. A data stream life cycle management method based on block chain technology is characterized in that: the method provides a data chaining function, data block information is stored in a data flow tracing private chain, random numbers are stored on the data block tracing private chain, a hash value is generated and stored on the data block tracing private chain, data are connected through a hash algorithm, and a time stamp characteristic is provided, and the method comprises the following steps:
s1, performing data preprocessing on original data according to a provided data preprocessing method;
s2, performing data access on the preprocessed data, acquiring data source information of each data block, generating description information, and storing the description information in a data block tracing private chain;
s3, cleaning the data by using a data cleaning service, generating a new data module, and generating new description information which is stored in a data block tracing private chain;
s4, after the data are cleaned, the data are analyzed and used by using a user identity authentication mechanism, and each time the data block is used, the user information and use information generation block is stored in the data block tracing private chain;
and S5, when the data is destroyed, storing administrator information and data block information of the destruction operation into the data block tracing private chain.
2. The method of claim 1, wherein the data stream lifecycle management method based on blockchain technology comprises: the data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
3. The method of claim 2, wherein the method comprises: in step S2, when data is accessed, a time interval of file aggregation and a threshold of a size of a data block generated by data aggregation are set, and data access processing is performed according to the size of the data block.
4. The method of claim 3, wherein the data stream lifecycle management method based on blockchain technology comprises: when data access is carried out, in a set time interval, if the size of a file is larger than a set threshold value, data access is directly carried out without preprocessing; and if the size of the data blocks aggregated in the time interval does not reach the threshold value, performing data access on the aggregated data blocks.
5. The method of claim 4, wherein the data flow lifecycle management method based on a blockchain technique comprises: in step S4, when the administrator and the developer user in the user identity authentication mechanism are registered in the system, a pair of public key and private key is generated, and the developer user can operate after performing key authentication each time.
6. A data stream life cycle management system based on block chain technology is characterized in that: the system stores data block information in a data flow tracing private chain based on the function of providing data chaining, stores random numbers on the data block tracing private chain, generates hash values to store the hash values on the data block tracing private chain, connects data through a hash algorithm, provides a timestamp characteristic, and comprises the following modules:
the data block tracing private chain module is used for storing data block information;
the identity authentication module is used for authenticating the identity of the user;
the data preprocessing module is used for preprocessing the original data according to the provided data preprocessing method;
the data access module is used for performing data access on the preprocessed data, acquiring data source information of each data block, generating description information and storing the description information in a data block tracing private chain;
the data cleaning module is used for cleaning data by using a data cleaning service, generating a new data module and generating new description information at the same time, and storing the new description information in a data block tracing private chain;
the data use module is used for analyzing and using the user identity authentication mechanism after the data is cleaned, and the user information and use information generation block is stored in the data block tracing private chain when the data block is used each time;
and the data destruction module is used for storing administrator information and data block information of destruction operation into the data block tracing private chain when data is destroyed.
7. The system according to claim 6, wherein: the data stream tracing private chain is divided into a plurality of data block tracing subchains, and each data block corresponds to one data block tracing subchain.
8. The system according to claim 7, wherein: when the data access module accesses data, setting a time interval of file aggregation and a threshold value of the size of a data block generated by data aggregation, and performing data access processing according to the size of the data block.
9. The system according to claim 8, wherein: the data use module generates a pair of public key and private key when the administrator and the developer user register in the system in the user identity authentication mechanism, and the developer user can operate after performing key authentication each time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910823516.9A CN110532293B (en) | 2019-09-02 | 2019-09-02 | Data stream life cycle management method and system based on block chain technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910823516.9A CN110532293B (en) | 2019-09-02 | 2019-09-02 | Data stream life cycle management method and system based on block chain technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110532293A CN110532293A (en) | 2019-12-03 |
| CN110532293B true CN110532293B (en) | 2023-04-07 |
Family
ID=68666250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910823516.9A Active CN110532293B (en) | 2019-09-02 | 2019-09-02 | Data stream life cycle management method and system based on block chain technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110532293B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563130A (en) * | 2020-07-15 | 2020-08-21 | 成都无右区块链科技有限公司 | Data credible data management method and system based on block chain technology |
| CN112148790A (en) * | 2020-09-07 | 2020-12-29 | 西安纸贵互联网科技有限公司 | Digital space directory chain and block chain platform |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018046009A1 (en) * | 2016-09-12 | 2018-03-15 | 上海鼎利信息科技有限公司 | Block chain identity system |
| CN107947922B (en) * | 2017-11-29 | 2020-07-21 | 中国科学院合肥物质科学研究院 | Digital file management method and system based on block chain technology |
| CN108564182B (en) * | 2018-04-16 | 2021-10-08 | 安徽中科智链信息科技有限公司 | Equipment full life cycle management system and method based on block chain technology |
-
2019
- 2019-09-02 CN CN201910823516.9A patent/CN110532293B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110532293A (en) | 2019-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9990507B2 (en) | Adapting decoy data present in a network | |
| US8671449B1 (en) | Systems and methods for identifying potential malware | |
| CN111934879B (en) | Encryption method, device, equipment and medium for data transmission of internal and external network system | |
| CN106789964B (en) | Cloud resource pool data security detection method and system | |
| WO2019217212A1 (en) | Systems and methods for attributing security vulnerabilities to a configuration of a client device | |
| US8955143B1 (en) | Use of decoy data in a data store | |
| TW201411367A (en) | Method and system for monitoring execution of user request in distributed system | |
| CN109474583B (en) | Data security management system | |
| CN110532293B (en) | Data stream life cycle management method and system based on block chain technology | |
| CN114826553A (en) | Cloud storage data security protection method and device based on group signature and homomorphic encryption | |
| CN112036995A (en) | Large-scale enterprise financial data management method and system based on block chain and readable storage medium | |
| CN106682521B (en) | File transparent encryption and decryption system and method based on driver layer | |
| CN111914300A (en) | Document encryption device and method for preventing file leakage | |
| CN113420049A (en) | Data circulation method and device, electronic equipment and storage medium | |
| CN111639355A (en) | Data security management method and system | |
| CN109254893B (en) | Service data auditing method, device, server and storage medium | |
| CN113364590A (en) | Block chain encryption method for production process data of multiple types of enterprises | |
| CN113239401A (en) | Big data analysis system and method based on power Internet of things and computer storage medium | |
| CN112910883B (en) | Data transmission method and device and electronic equipment | |
| CN114153838A (en) | Encryption storage and query method for member information | |
| CN110933064B (en) | Method and system for determining user behavior track | |
| CN114492491A (en) | Data management system for NFC label application | |
| CN113259348A (en) | Heterogeneous data processing method and device, computer equipment and storage medium | |
| CN115221125A (en) | File processing method and device, electronic equipment and readable storage medium | |
| CN105681428A (en) | User information synchronizing system, method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 271000 Langchao science and Technology Park, 527 Dongyue street, Tai'an City, Shandong Province Applicant after: INSPUR SOFTWARE Co.,Ltd. Address before: No. 1036, Shandong high tech Zone wave road, Ji'nan, Shandong Applicant before: INSPUR SOFTWARE Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |