CN117544507A - Multi-region distributed configuration method and system based on cloud object storage service - Google Patents
Multi-region distributed configuration method and system based on cloud object storage service Download PDFInfo
- Publication number
- CN117544507A CN117544507A CN202311496499.5A CN202311496499A CN117544507A CN 117544507 A CN117544507 A CN 117544507A CN 202311496499 A CN202311496499 A CN 202311496499A CN 117544507 A CN117544507 A CN 117544507A
- Authority
- CN
- China
- Prior art keywords
- service
- configuration
- configuration data
- file
- snapshot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000007726 management method Methods 0.000 claims description 22
- 238000012986 modification Methods 0.000 claims description 8
- 230000004048 modification Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000007792 addition Methods 0.000 claims description 5
- 238000013500 data storage Methods 0.000 claims description 5
- 238000012217 deletion Methods 0.000 claims description 5
- 230000037430 deletion Effects 0.000 claims description 5
- 238000012423 maintenance Methods 0.000 claims description 4
- 230000000977 initiatory effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 description 6
- 238000011084 recovery Methods 0.000 description 4
- 238000013523 data management Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000872 buffer Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/084—Configuration by using pre-existing information, e.g. using templates or copying from other elements
- H04L41/0846—Configuration by using pre-existing information, e.g. using templates or copying from other elements based on copy from other elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a multi-region distributed configuration method and system based on cloud object storage service, wherein the method comprises the following steps: single-area deployment configuration center service; the data snapshot and the incremental data managed by the configuration center service are stored in a data file at regular time; uploading the data file to a cloud object storage service in the region; the cloud object storage service copies and distributes the uploaded configuration data file to other areas where the application distributed system is deployed; the application distributed system service deployed in each region obtains configuration data snapshot files and incremental data files by accessing cloud object storage service of the region where the application distributed system service is located, and further obtains configuration data required by the system. The invention solves the problem of configuration data synchronization when the application distributed system is deployed in a plurality of areas, and avoids the problems of data acquisition failure, overtime and the like caused by unstable network environment and the problem of configuration data consistency of the application distributed system deployed in a plurality of areas.
Description
Technical Field
The application relates to the technical field of distributed system configuration center data management, in particular to a multi-region distributed configuration method and system based on cloud object storage service.
Background
In globalized waves, users of distributed system services are no longer limited to a certain region, but are oriented to global users. Therefore, the system needs to be deployed in different areas respectively, so that the user can access nearby conveniently, and the network delay is reduced; meanwhile, as system functions are increasingly complex, configuration data are increasingly more complex, and simple local configuration cannot cope with challenges brought by current efficient deployment and maintenance requirements.
Therefore, in a complex multi-zone deployed application distributed system, a distributed configuration center solution that can serve multiple zones is needed.
Disclosure of Invention
The application provides a multi-region distributed configuration method and system based on cloud object storage service, which are used for solving the problem that a distributed system deployed by complex multi-region cannot acquire consistent configuration data rapidly, efficiently and safely.
In a first aspect, a method for multi-region distributed configuration based on cloud object storage services, the method comprising:
deploying a high-availability distributed configuration center service in a single area, wherein the distributed configuration center service is used for performing addition, deletion, modification and check operations on configuration data required in a distributed system;
the method comprises the steps of writing configuration data set snapshots and configuration data increments managed by configuration center service in a file mode to obtain configuration data snapshot files and increment files;
compressing the configuration data snapshot file and the incremental file and uploading the configuration data snapshot file and the incremental file to a cloud object storage service;
the cloud object storage service copies the snapshot file and the increment file and distributes the snapshot file and the increment file to other areas through a special network connection;
the distributed system deployed in each region obtains configuration data required by the service by accessing the cloud object storage service in the region.
Optionally, the distributed configuration center service supports high availability, high concurrency, and high performance, and the configuration data managed by the distributed configuration center service comprises application configuration data and service configuration data; the application configuration data are used for guiding the behavior and the function of the service, and assisting the service in solving the actual application problem; the service configuration data is a service initiation parameter set that is consolidated for ease of deployment and maintenance of the service.
Optionally, the configuration data set snapshot and the configuration data increment managed by the configuration center service are written into the configuration data snapshot file and the configuration data increment file at fixed time; compressing the configuration data snapshot file and the increment file and uploading the compressed configuration data snapshot file and the increment file to a cloud object storage service, wherein the method specifically comprises the following steps of:
the low-frequency timer is used for timing the snapshot of the configuration data set managed by the configuration center, writing the snapshot into different snapshot files according to the distributed system service, version and deployment environment in a binary byte stream format, and then compressing the snapshot files and uploading the compressed snapshot files to the cloud object storage service.
Optionally, the configuration data set snapshot and the configuration data increment managed by the configuration center service are written into the configuration data snapshot file and the configuration data increment file at fixed time; compressing the configuration data snapshot file and the increment file and uploading the compressed configuration data snapshot file and the increment file to a cloud object storage service, wherein the method specifically comprises the following steps of:
the high-frequency timer is used for regularly writing the increment data set of the relative snapshot file generation time managed by the configuration center into different increment files according to the distributed system service, version and deployment environment in a binary byte stream format, and then compressing the increment files and uploading the increment files to the cloud object storage service.
Optionally, the cloud object storage service copies the snapshot file and the incremental file and distributes the snapshot file and the incremental file to other areas through a private network connection, and specifically includes:
and copying and distributing the configuration data snapshot file and the configuration data increment file from the area where the configuration center is located to other areas where the application distributed system is deployed by using a special network connection provided by a cloud object storage service provider.
Optionally, when the application distributed system service deployed in each area is started, the cloud service object storage service in the area where the application distributed system service is deployed accesses the cloud service object storage service in the area to obtain the configuration data snapshot file and the configuration data incremental file through the HTTP protocol, so that the required configuration data is obtained, and simultaneously, the ETag value returned by the incremental configuration file is saved for inquiring new incremental data.
Optionally, after the application distributed system deployed in each area is started, the cloud service object storage service of the area is accessed through the HTTP protocol with the ETag at regular time, whether new incremental data exists is queried, and the new incremental data is processed.
In a second aspect, a multi-region distributed configuration system based on cloud object storage services, the system comprising:
the distributed configuration center management module is used for deploying high-availability distributed configuration center services in a single area, and the distributed configuration center services are used for performing addition, deletion, modification and check operations on configuration data required in a distributed system;
the configuration data storage module is used for regularly writing the configuration data set snapshot and the configuration data increment managed by the configuration center service into the configuration data snapshot file and the increment file; compressing the configuration data snapshot file and the incremental file and uploading the configuration data snapshot file and the incremental file to a cloud object storage service; the cloud object storage service copies the snapshot file and the increment file and distributes the snapshot file and the increment file to other areas through a special network connection;
and the distributed system configuration data acquisition module is used for acquiring configuration data required by the service by accessing the cloud object storage service of the region where the distributed system deployed in each region is located.
Optionally, the distributed configuration center management module specifically includes:
the configuration center snapshot unit is used for storing versioned application configuration data and service configuration data required by each service in the application distributed system in respective environments;
and the configuration center increment unit is used for storing versioned application configuration increment data and service configuration increment data required by each service in respective environments after snapshot generation in the application distributed system.
In a third aspect, a computer readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the multi-zone distributed configuration method of any of the above first aspects.
Compared with the prior art, the application has the following beneficial effects:
the single-area deployment configuration center service is used for operating and managing configuration data required by the application distributed system; the data snapshot and the incremental data managed by the configuration center service are stored in a data file at regular time; uploading the data file to a cloud object storage service in the region; the cloud object storage service copies and distributes the uploaded configuration data file to other areas where the application distributed system is deployed; the application distributed system service deployed in each region obtains configuration data snapshot files and incremental data files by accessing cloud object storage service of the region where the application distributed system service is located, and further obtains configuration data required by the system. The method and the system realize the data sharing of the multi-region configuration center based on the cloud object storage service, solve the problem of configuration data synchronization when the application distributed system is deployed in the multi-region, and avoid the problems of data acquisition failure, overtime and the like caused by unstable network environment and the problem of the consistency of the configuration data of the application distributed system deployed in the multi-region.
Drawings
In order to more simply demonstrate the manner of use or technical solutions of the prior art of the present invention, the following description will briefly introduce the drawings that are required to be used in the description of the manner of use or the prior art. It will be apparent to those skilled in the art from this disclosure that the drawings described below are merely exemplary and that other embodiments may be derived from the drawings provided without undue effort.
The structures, proportions, sizes, etc. shown in the present specification are shown only for the purposes of illustration and description, and are not intended to limit the scope of the invention, which is defined by the claims, so that any structural modifications, changes in proportions, or adjustments of sizes, which do not affect the efficacy or the achievement of the present invention, should fall within the ambit of the technical disclosure.
FIG. 1 is a flowchart of a method for a multi-region distributed configuration center based on cloud object storage services provided in an embodiment of the present invention;
fig. 2 is a schematic diagram of a multi-region distributed configuration center method based on a cloud object storage service according to an embodiment of the present invention.
Detailed Description
Other advantages and advantages of the present invention will become apparent to those skilled in the art from the following detailed description, which, by way of illustration, is to be read in connection with certain specific embodiments, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The abbreviations in the examples below have the following meanings:
AWS: amazon cloud service platform;
s3: AWS simple storage services;
ETag: entity tags, part of the world wide web protocol HTTP;
gzip: a compressed file format;
ProtoBuf: protocol Buffers, a Protocol that opens a source cross-platform serialized data structure.
In this application, as shown in fig. 1, the method specifically includes:
step one, deploying a high availability distributed configuration center service in a single area,
the distributed configuration center service is used for performing addition, deletion, modification and check operations on configuration data required in the distributed system. The distributed configuration center service supports high availability, high concurrency and high performance, and the managed configuration data comprises application configuration data and service configuration data; the application configuration data are used for guiding the behavior and the function of the service, and assisting the service in solving the actual application problem; the service configuration data is a service initiation parameter set that is consolidated for ease of deployment and maintenance of the service.
And step two, performing file writing on the configuration data set snapshot and the configuration data increment managed by the configuration center service at regular time to obtain a configuration data snapshot file and an increment file.
Specifically, the low-frequency timer periodically snapshots the configuration data set managed by the configuration center, writes the configuration data set snapshots into different snapshot files according to distributed system service, version and deployment environment in a binary byte stream format, and then compresses the snapshot files and uploads the compressed snapshot files to the cloud object storage service. The high-frequency timer is used for regularly writing the increment data set of the relative snapshot file generation time managed by the configuration center into different increment files according to the distributed system service, version and deployment environment in a binary byte stream format, and then compressing the increment files and uploading the increment files to the cloud object storage service.
And thirdly, compressing the configuration data snapshot file and the incremental file and uploading the compressed configuration data snapshot file and the compressed incremental file to the cloud object storage service.
And fourthly, the cloud object storage service copies the snapshot file and the increment file and distributes the snapshot file and the increment file to other areas through special network connection.
Specifically, the configuration data snapshot file and the configuration data delta file are copied and distributed from the area where the configuration center is located to other areas where the application distributed system is deployed by using a special network connection provided by the cloud object storage service provider.
And fifthly, the distributed system deployed in each region obtains configuration data required by the service by accessing the cloud object storage service in the region.
Specifically, when the application distributed system service deployed in each area is started, the cloud service object storage service in the area where the application distributed system service is deployed accesses the cloud service object storage service in the area to obtain the configuration data snapshot file and the configuration data increment file through the HTTP protocol, so that the required configuration data is obtained, and simultaneously, the ETag value returned by the increment configuration file is saved and used for inquiring new increment data. After the application distributed system deployed in each area is started, the cloud service object storage service of the area is accessed at regular time through the HTTP protocol with ETag, whether new incremental data exist or not is inquired, and the new incremental data are processed.
The following is a specific implementation procedure based on the above steps in the AWS platform:
example 1
Referring to fig. 1, the embodiment provides a multi-region distributed configuration center method based on an AWS platform S3 service, which comprises the following steps:
deploying a high availability distributed configuration center service in an area, namely deploying a complete distributed configuration center service in an AWS and Meadobe area, wherein the service center provides the following functions;
supporting management of application configuration data of an application distributed system;
supporting management of application distributed system service configuration data;
the management of the configuration data comprises the operations of adding, deleting, modifying and checking the configuration data items, supports version control of the configuration data, supports isolation management of the configuration data under multiple environments, supports release state management of the configuration data, supports load balancing and disaster recovery, provides security and authority management for a configuration data center, and supports service monitoring and alarming for the configuration data center;
all application and service configuration data set snapshots of the configuration center service management are encoded through a ProtoBuf format and stored into a file at 10 minutes in 2 am every day, and the file is compressed into a Gzip format;
uploading a Gzip file storing the configuration data snapshot file to the eastern region S3 service by using an AWS account with S3 uploading file authority;
detecting whether the configuration center has incremental data relative to the snapshot file every minute, encoding the incremental data through a ProtoBuf format, storing the encoded incremental data into the file, and compressing the file into a Gzip format;
uploading a Gzip file storing the configuration data increment file to the eastern American area S3 by using an AWS account with S3 uploading file authority;
the Meeastern area S3 service copies and distributes the configuration data snapshot file and the incremental data file to European areas, japanese areas and Singapore areas where the application distributed system is deployed;
when the application distributed system service deployed in the eastern, european, japanese and Singapore regions is started, an AWS account with the authority of reading the S3 file is used, the S3 service in the region is accessed through the HTTP protocol, the required configuration data snapshot file and configuration data increment file compression file are downloaded, the latest configuration data is obtained after the file is decompressed, and the service is started by using the obtained configuration data.
After the application distributed system service deployed in each area is started, the ETag value returned by the current incremental file is saved and used for acquiring a new incremental data file next time;
after the application distributed system service deployed in each area is started, HTTP request with ETag is sent to S3 service at regular time every minute, whether new incremental data files exist is inquired, if so, the configuration is updated, and the new ETag is stored for next inquiry;
in summary, the invention realizes unified configuration data management of the application distributed system of multi-area deployment through the configuration center service of single-area deployment and the cloud object storage service. High availability of the force configuration center is guaranteed, and load balancing and disaster recovery and authority management are supported. The consistency of system configuration data in multi-region distribution can be efficiently and real-timely ensured.
Example 2
Referring to fig. 2, the present embodiment provides a multi-area distributed configuration center system based on the AWS platform S3 service, which includes a high availability distributed configuration center management module 1, a configuration data storage module 2, and an application distributed system configuration data acquisition module 3.
The high availability distributed configuration center management module 1 provides the following functions:
supporting management of application configuration data of an application distributed system;
supporting management of application distributed system service configuration data;
the management of the configuration data comprises the operations of adding, deleting, modifying and checking the configuration data items, supports version control of the configuration data, supports isolation management of the configuration data under multiple environments, supports release state management of the configuration data, supports load balancing and disaster recovery, provides security and authority management for a configuration data center, and supports service monitoring and alarming for the configuration data center;
the configuration data storage module 2 includes:
writing configuration center data into a file;
compressing the configuration data file;
storing the compressed configuration data file;
carrying out multi-region distribution on the stored configuration data file;
the distributed system configuration data acquisition module 3 includes:
the application distributed system service deployed in each region pulls the required configuration files;
analyzing the obtained configuration file;
the distributed configuration center management module specifically comprises:
the configuration center snapshot unit is used for storing versioned application configuration data and service configuration data required by each service in the application distributed system in respective environments;
and the configuration center increment unit is used for storing versioned application configuration increment data and service configuration increment data required by each service in respective environments after snapshot generation in the application distributed system.
In summary, the invention realizes unified configuration data management of the application distributed system of multi-area deployment through the configuration center service of single-area deployment and the cloud object storage service. High availability of the force configuration center is guaranteed, and load balancing and disaster recovery and authority management are supported. The consistency of system configuration data in multi-region distribution can be efficiently and real-timely ensured.
Example 3
Embodiment 3 of the present invention provides a non-transitory computer readable storage medium, in which a program code of a multi-region distributed configuration method based on a cloud object storage service is stored, the program code including instructions for executing the multi-region distributed configuration method based on a cloud object storage service of the above embodiment or any possible implementation thereof.
Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc., that contain an integration of one or more available media. Usable media may be magnetic media (e.g., floppy Disk, hard Disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid state Disk, SSD), etc.
Example 4
Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor; the processor and the memory complete the communication with each other through a bus; the memory stores program instructions executable by the processor, which invoke the program instructions to perform the implementation of the above-described embodiments or any of their possible implementations. Specifically, the processor may be implemented by hardware or software, and when implemented by hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor, implemented by reading software code stored in a memory, which may be integrated in the processor, or may reside outside the processor, and exist separately. In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.) means from one website, computer, server, or data center.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
While the invention has been described in detail in the foregoing general description and specific examples, it will be apparent to those skilled in the art that modifications and improvements can be made thereto. Accordingly, such modifications or improvements may be made without departing from the spirit of the invention and are intended to be within the scope of the invention as claimed. The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
Claims (10)
1. A multi-region distributed configuration method based on cloud object storage service, the method comprising:
deploying a high-availability distributed configuration center service in a single area, wherein the distributed configuration center service is used for performing addition, deletion, modification and check operations on configuration data required in a distributed system;
the method comprises the steps of writing configuration data set snapshots and configuration data increments managed by configuration center service in a file mode to obtain configuration data snapshot files and increment files;
compressing the configuration data snapshot file and the incremental file and uploading the configuration data snapshot file and the incremental file to a cloud object storage service;
the cloud object storage service copies the snapshot file and the increment file and distributes the snapshot file and the increment file to other areas through a special network connection;
the distributed system deployed in each region obtains configuration data required by the service by accessing the cloud object storage service in the region.
2. The multi-region distributed configuration method of claim 1, wherein the distributed configuration center service supports high availability, high concurrency, high performance, and its managed configuration data includes application configuration data and service configuration data; the application configuration data are used for guiding the behavior and the function of the service, and assisting the service in solving the actual application problem; the service configuration data is a service initiation parameter set that is consolidated for ease of deployment and maintenance of the service.
3. The multi-region distributed configuration method according to claim 1, wherein the writing of configuration data set snapshots and configuration data increments managed by a configuration center service into configuration data snapshot files and increment files is performed at fixed time; compressing the configuration data snapshot file and the increment file and uploading the compressed configuration data snapshot file and the increment file to a cloud object storage service, wherein the method specifically comprises the following steps of:
the low-frequency timer is used for timing the snapshot of the configuration data set managed by the configuration center, writing the snapshot into different snapshot files according to the distributed system service, version and deployment environment in a binary byte stream format, and then compressing the snapshot files and uploading the compressed snapshot files to the cloud object storage service.
4. The multi-region distributed configuration method according to claim 1, wherein the writing of configuration data set snapshots and configuration data increments managed by a configuration center service into configuration data snapshot files and increment files is performed at fixed time; compressing the configuration data snapshot file and the increment file and uploading the compressed configuration data snapshot file and the increment file to a cloud object storage service, wherein the method specifically comprises the following steps of:
the high-frequency timer is used for regularly writing the increment data set of the relative snapshot file generation time managed by the configuration center into different increment files according to the distributed system service, version and deployment environment in a binary byte stream format, and then compressing the increment files and uploading the increment files to the cloud object storage service.
5. The multi-region distributed configuration method of claim 1, wherein the cloud object storage service distributes the snapshot file and the delta file to other regions through a dedicated network connection after copying, and specifically comprises:
and copying and distributing the configuration data snapshot file and the configuration data increment file from the area where the configuration center is located to other areas where the application distributed system is deployed by using a special network connection provided by a cloud object storage service provider.
6. The multi-region distributed configuration method according to claim 1, wherein when the application distributed system service deployed in each region is started, the cloud service object storage service accessing the region where the application distributed system service is located through an HTTP protocol obtains a configuration data snapshot file and a configuration data delta file, so as to obtain required configuration data, and simultaneously stores an ETag value returned by the delta configuration file, so as to query new delta data.
7. The multi-region distributed configuration method according to claim 1, wherein the application distributed system deployed in each region accesses the cloud service object storage service of the region by HTTP protocol with ETag at regular time after being started, inquires whether new incremental data exists, and processes the new incremental data.
8. A multi-region distributed configuration system based on cloud object storage services, the system comprising:
the distributed configuration center management module is used for deploying high-availability distributed configuration center services in a single area, and the distributed configuration center services are used for performing addition, deletion, modification and check operations on configuration data required in a distributed system;
the configuration data storage module is used for regularly writing the configuration data set snapshot and the configuration data increment managed by the configuration center service into the configuration data snapshot file and the increment file; compressing the configuration data snapshot file and the incremental file and uploading the configuration data snapshot file and the incremental file to a cloud object storage service; the cloud object storage service copies the snapshot file and the increment file and distributes the snapshot file and the increment file to other areas through a special network connection;
and the distributed system configuration data acquisition module is used for acquiring configuration data required by the service by accessing the cloud object storage service of the region where the distributed system deployed in each region is located.
9. The multi-zone distributed configuration system of claim 8, wherein the distributed configuration center management module specifically comprises:
the configuration center snapshot unit is used for storing versioned application configuration data and service configuration data required by each service in the application distributed system in respective environments;
and the configuration center increment unit is used for storing versioned application configuration increment data and service configuration increment data required by each service in respective environments after snapshot generation in the application distributed system.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311496499.5A CN117544507A (en) | 2023-11-10 | 2023-11-10 | Multi-region distributed configuration method and system based on cloud object storage service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311496499.5A CN117544507A (en) | 2023-11-10 | 2023-11-10 | Multi-region distributed configuration method and system based on cloud object storage service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117544507A true CN117544507A (en) | 2024-02-09 |
Family
ID=89785413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311496499.5A Pending CN117544507A (en) | 2023-11-10 | 2023-11-10 | Multi-region distributed configuration method and system based on cloud object storage service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117544507A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117651043A (en) * | 2024-01-30 | 2024-03-05 | 天津创意星球网络科技股份有限公司 | Multimedia file uploading method, device and system based on OSS service |
-
2023
- 2023-11-10 CN CN202311496499.5A patent/CN117544507A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117651043A (en) * | 2024-01-30 | 2024-03-05 | 天津创意星球网络科技股份有限公司 | Multimedia file uploading method, device and system based on OSS service |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10387451B2 (en) | Synchronization system for multiple client devices | |
| KR102042723B1 (en) | Methods for updating applications | |
| US20120150930A1 (en) | Cloud storage and method for managing the same | |
| CN109492013B (en) | Data processing method, device and system applied to database cluster | |
| CN110597910A (en) | Remote data synchronization method, device and system | |
| KR101871383B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN110019104B (en) | File resource exchange method, file resource server and file exchange system | |
| CN117544507A (en) | Multi-region distributed configuration method and system based on cloud object storage service | |
| US20150227605A1 (en) | Information processing terminal, synchronization control method, and computer-readable recording medium | |
| US20170054803A1 (en) | Information processing device, method, and system | |
| CN111338834A (en) | Data storage method and device | |
| CN112948498A (en) | Method and device for generating global identification of distributed system | |
| CN102594874A (en) | Synchronization processing method and device | |
| CN111343220B (en) | Repeater, distributed file transmission method, distributed file transmission system, medium and electronic equipment | |
| CN105022779A (en) | Method for realizing HDFS file access by utilizing Filesystem API | |
| CN114096962A (en) | Block chain cache system | |
| CN115982133A (en) | Data processing method and device | |
| CN111382132A (en) | Medical image data cloud storage system | |
| KR20120073799A (en) | Data synchronizing and servicing apparatus and method based on cloud storage | |
| CN113391759B (en) | Communication method and equipment | |
| CN111083182B (en) | Distributed Internet of things equipment management method and device | |
| CN111371606A (en) | Method for specifying monitor ip when using look to deploy ceph cluster | |
| CN106407320B (en) | File processing method, device and system | |
| CN112667607B (en) | Historical data management method and related equipment | |
| CN117112500B (en) | Resource management method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |