CN115438123A - Data sharing exchange method and system based on Docker and micro-service architecture - Google Patents
Data sharing exchange method and system based on Docker and micro-service architecture Download PDFInfo
- Publication number
- CN115438123A CN115438123A CN202211063183.2A CN202211063183A CN115438123A CN 115438123 A CN115438123 A CN 115438123A CN 202211063183 A CN202211063183 A CN 202211063183A CN 115438123 A CN115438123 A CN 115438123A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- layer
- data sharing
- sharing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6227—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database where protection concerns the structure of data, e.g. records, types, queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data sharing exchange method and a data sharing exchange system based on a Docker and micro-service architecture, wherein the data sharing exchange system is mainly divided into a service discovery layer, a storage layer, a data processing layer, a service distribution layer, a gateway security layer and the like and is respectively responsible for functions of service registration, data extraction and processing, data storage management, data automation service, gateway request authentication control and the like; the data processing layer comprises the functions of data extraction, cleaning, conversion, quality inspection, warehousing and the like. The invention provides a data sharing and exchanging platform which supports multiple data sources, multiple dimensions and multiple systems and has safety, reliability and openness by developing a data sharing and exchanging method and constructing a corresponding webpage platform system.
Description
Technical Field
The invention relates to the technical field of micro-service architectures, in particular to a data sharing and exchanging method and system based on Docker and a micro-service architecture.
Background
The microservice architecture is an architecture style of a complete application system consisting of a plurality of loosely coupled microservices, wherein each microservice realizes single responsibility and function, can be independently developed, tested and deployed, and interacts among the microservices through a lightweight communication protocol, and with the development of business, more and more enterprises adopt the microservice architecture to develop the application system, but the traditional architecture system does not support a many-to-many mode, and the data exchange mode has low security, poor reliability and openness, cannot be improved from a technical level, and also reduces the efficiency and the security of data exchange.
Disclosure of Invention
The present invention provides a data sharing and exchanging method and system based on a Docker and a micro-service architecture, so as to solve the problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme: a data sharing and exchanging method based on Docker and micro service architecture is disclosed, which comprises the following steps:
1. system architecture
The data sharing exchange system is mainly divided into a service discovery layer, a storage layer, a data processing layer, a service distribution layer, a gateway security layer and the like, and is respectively responsible for functions of service registration, data extraction and processing, data storage management, data automation service, gateway request authentication control and the like;
the data processing layer comprises functions of data extraction, cleaning, conversion, quality inspection, warehousing and the like, and the data distribution layer is used for carrying out data management on the existing data, automatically issuing data services and the like;
2. data sharing flow
If the data acquirer needs to use the data, the data acquirer needs to perform double approval through the data sharing system and the data provider, and can acquire the data only by carrying the token to perform a data request after obtaining permission.
Preferably, the data source supports multiple databases including a mainstream relational database, hive storage and the like, and for unstructured data, minIO object type storage is adopted, so that efficiency is higher.
Preferably, the gateway security layer controls request access, data rights, and the like.
Preferably, the data sharing system is developed in a background based on a Spring Cloud micro-service architecture by adopting Java language, the front-end display layer is developed by adopting a Vue framework, and the database is a MySQL relational database:
(1) Data directory: the data directory function establishes a resource cataloging system, generalizes and indexes various resources, encodes various resources according to classification, provides fields such as areas, departments, industries, scenes, types and the like to index the directory, and can be used for users to check details and subscribe;
(2) Data service: the service generated by database sharing or other external data services is provided to the outside in a service registration mode, and a user can search, check, collect, apply for use, call on line and the like;
(3) The personal center: operations such as personal information modification, application authentication, message viewing, data sharing, data authorization, log viewing, token management and the like can be performed;
(4) And (3) approval management: managing application of user use data, user authentication and user data sharing;
(5) And (3) system management: user rights, system logs, etc. are managed.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention provides a platform for data sharing and exchange, which supports multiple data sources, multiple dimensions and multiple systems and has safety, reliability and openness by developing a data sharing and exchanging method and constructing a corresponding webpage platform system.
2. According to the invention, the sharing efficiency is improved, and the data sharing safety is improved, through centralized data sharing, data resources are uniformly registered to the resource center, and then data services are issued to users with permission, so that the complicated step of data supply and demand one-to-one butt joint is avoided; meanwhile, when the data service details are displayed outwards, the data sharing system is encrypted, so that the safety of the data service is ensured; when accessing data, the data access security is ensured by multi-layer token verification and check of a system gateway; in the data transmission process, data is encrypted, and the data security of a transmission layer is ensured.
3. In the digital country solution scheme, the data service and file data of other application systems in the scheme are registered to a data sharing exchange system, the data are safely shared and continuously transmitted to other government affair systems, the data sharing exchange system also plays a role in social public inquiry service, a common user can conveniently inquire relevant data after registration and authentication, and meanwhile, the data sharing exchange system also well supports the problem that data exchange is needed due to upgrading between the established government affair applications, and the time and labor cost for butting of developers of the application systems are greatly reduced.
Drawings
FIG. 1 is a block diagram of a data sharing switching system according to the present invention;
FIG. 2 is a flow chart of the data sharing exchange of the present invention;
FIG. 3 is a functional diagram of a data sharing system according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-3, the present invention provides the following technical solutions: a data sharing and exchanging method based on Docker and micro service architecture is disclosed, which comprises the following steps:
1. system architecture
The data sharing exchange system is mainly divided into a service discovery layer, a storage layer, a data processing layer, a service distribution layer, a gateway security layer and the like, and is respectively responsible for functions of service registration, data extraction and processing, data storage management, data automation service, gateway request authentication control and the like;
the data processing layer comprises functions of data extraction, cleaning, conversion, quality inspection, warehousing and the like, and the data distribution layer is used for carrying out data management on the existing data, automatically issuing data services and the like;
2. data sharing process
If the data acquirer needs to use the data, the data acquirer needs to perform double approval through the data sharing system and the data provider, and can acquire the data only by carrying the token to perform a data request after obtaining permission.
A data sharing exchange system based on a Docker and a micro-service architecture is characterized in that the data sharing system adopts Java language, background development is carried out based on a Spring Cloud micro-service architecture, a front-end display layer adopts a Vue framework for development, and a MySQL relational database is selected as a database:
(1) Data directory: the data directory function establishes a resource cataloging system, summarizes and indexes various resources, encodes various resources according to classification, provides fields such as areas, departments, industries, scenes, types and the like to index the directory, and can be used for users to check details and subscribe;
(2) Data service: for the service generated by database sharing or other external data services, the service is provided in a service registration mode, and a user can search and view, collect the service, apply for use, call online and the like;
(3) The personal center: operations such as personal information modification, authentication application, message viewing, data sharing, data authorization, log viewing, token management and the like can be performed;
(4) And (3) approval management: managing application of user use data, user authentication and user data sharing;
(5) And (3) system management: user rights, system logs, etc. are managed.
The first embodiment is as follows:
a data sharing and exchanging method based on Docker and micro service architecture is disclosed, which comprises the following steps:
1. system architecture
The data sharing exchange system is mainly divided into a service discovery layer, a storage layer, a data processing layer, a service distribution layer, a gateway security layer and the like, and is respectively responsible for functions of service registration, data extraction and processing, data storage management, data automation service, gateway request authentication control and the like;
the data source supports various databases, including a mainstream relational database, hive storage and the like, for unstructured data, minIO object type storage is adopted, so that the efficiency is higher, a data processing layer comprises functions of data extraction, cleaning, conversion, quality inspection, warehousing and the like, a data distribution layer is used for carrying out data management on existing data, automatically issuing data services and the like, and a gateway security layer is used for controlling request access, data permission and the like;
2. data sharing flow
If the data acquirer needs to use the data, the data acquirer needs to perform double approval through the data sharing system and the data provider, and can acquire the data only by carrying the token to perform a data request after obtaining permission.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (4)
1. A data sharing exchange method based on Docker and micro-service architecture is characterized in that: the data sharing and exchanging method comprises the following steps:
1. system architecture
The data sharing exchange system is mainly divided into a service discovery layer, a storage layer, a data processing layer, a service distribution layer, a gateway security layer and the like, and is respectively responsible for functions of service registration, data extraction and processing, data storage management, data automation service, gateway request authentication control and the like;
the data processing layer comprises functions of data extraction, cleaning, conversion, quality inspection, warehousing and the like, and the data distribution layer is used for carrying out data management on the existing data, automatically issuing data services and the like;
2. data sharing process
If the data acquirer needs to use the data, the data acquirer needs to perform double approval through the data sharing system and the data provider, and can acquire the data only by carrying the token to perform data request after obtaining permission.
2. The data sharing and exchanging method based on the Docker and the micro service architecture as claimed in claim 1, wherein: the data source supports various databases including a mainstream relational database, hive storage and the like, and for non-structural data, minIO object type storage is adopted, so that the efficiency is higher.
3. The data sharing and exchanging method based on the Docker and the microservice architecture as claimed in claim 1, wherein: the gateway security layer controls request access, data rights, and the like.
4. A data sharing exchange system based on Docker and micro service architecture is characterized in that: the data sharing system adopts Java language, background development is carried out based on Spring Cloud micro-service architecture, the front-end display layer adopts Vue framework for development, and the database adopts MySQL relational database:
(1) Data directory: the data directory function establishes a resource cataloging system, generalizes and indexes various resources, encodes various resources according to classification, provides fields such as areas, departments, industries, scenes, types and the like to index the directory, and can be used for users to check details and subscribe;
(2) Data service: the service generated by database sharing or other external data services is provided to the outside in a service registration mode, and a user can search, check, collect, apply for use, call on line and the like;
(3) The personal center: operations such as personal information modification, authentication application, message viewing, data sharing, data authorization, log viewing, token management and the like can be performed;
(4) And (3) approval management: managing application of user use data, user authentication and user data sharing;
(5) And (3) system management: user rights, system logs, etc. are managed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211063183.2A CN115438123A (en) | 2022-08-31 | 2022-08-31 | Data sharing exchange method and system based on Docker and micro-service architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211063183.2A CN115438123A (en) | 2022-08-31 | 2022-08-31 | Data sharing exchange method and system based on Docker and micro-service architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115438123A true CN115438123A (en) | 2022-12-06 |
Family
ID=84245396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211063183.2A Pending CN115438123A (en) | 2022-08-31 | 2022-08-31 | Data sharing exchange method and system based on Docker and micro-service architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115438123A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117176379A (en) * | 2023-06-25 | 2023-12-05 | 广东电网有限责任公司佛山供电局 | Unstructured data sharing method and device based on micro-service architecture |
-
2022
- 2022-08-31 CN CN202211063183.2A patent/CN115438123A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117176379A (en) * | 2023-06-25 | 2023-12-05 | 广东电网有限责任公司佛山供电局 | Unstructured data sharing method and device based on micro-service architecture |
| CN117176379B (en) * | 2023-06-25 | 2024-02-09 | 广东电网有限责任公司佛山供电局 | Unstructured data sharing method and device based on micro-service architecture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111488595B (en) | Method for realizing authority control and related equipment | |
| US9082091B2 (en) | Unified user login for co-location facilities | |
| CN110188573B (en) | Partition authorization method, partition authorization device, partition authorization equipment and computer readable storage medium | |
| CN104160381A (en) | Managing tenant-specific data sets in a multi-tenant environment | |
| CN111460506A (en) | Data access control method and device | |
| CN101908163A (en) | Expert-supported application system platform used for government affair and business affair decision and establishment method thereof | |
| CN111984717A (en) | Big data intelligent government affair platform information management method | |
| CN111625510A (en) | Multi-source data sharing system and method based on cloud mapping | |
| Du | Application of information communication network security management and control based on big data technology | |
| CN106993049A (en) | A kind of General Aviation enterprise operation total management system based on cloud computing | |
| CN115438123A (en) | Data sharing exchange method and system based on Docker and micro-service architecture | |
| Li et al. | Granularity decision of microservice splitting in view of maintainability and its innovation effect in government data sharing | |
| CN104881749A (en) | Data management method and data storage system for multiple tenants | |
| CN113553381A (en) | Distributed data management system based on novel pipeline scheduling algorithm | |
| CN111181833B (en) | Enterprise interconnection realization method and device | |
| US11042514B2 (en) | Collaboration computer system | |
| WO2022214227A1 (en) | A system and method for exchanging and managing data stored in heterogeneous data sources | |
| CN114463111A (en) | Gridding marketing platform | |
| CN114611861A (en) | Enterprise resource informatization management system | |
| CN113221177A (en) | Data access method, device and system in distributed system | |
| Yan et al. | Innovative research on intelligent enterprise management mode under the background of smart city | |
| WO2022206089A1 (en) | Data internet method and system | |
| US20230138622A1 (en) | Emergency Access Control for Cross-Platform Computing Environment | |
| Xie | The Application and Implementation of Clustering Algorithm in New Small-and-medium Sized Enterprise Management System | |
| CN115473915A (en) | Intelligent financial solution method based on distributed micro-service architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |