CN117874143A

CN117874143A - Cloud edge database middleware synchronization method in distributed environment

Info

Publication number: CN117874143A
Application number: CN202410111676.1A
Authority: CN
Inventors: 邓晓衡; 谷子豪; 池来新; 梁晶晶; 桂劲松
Original assignee: Central South University
Current assignee: Central South University
Priority date: 2024-01-26
Filing date: 2024-01-26
Publication date: 2024-04-12

Abstract

The embodiment of the disclosure provides a cloud edge database synchronization middleware method in a distributed environment, which belongs to the technical field of data processing and specifically comprises the following steps: constructing an application scene according to an edge database and a cloud database corresponding to the plurality of distributed edge servers; designing a data detection model to update data of an edge database; encrypting the data updating content by utilizing an encryption algorithm; establishing a cache model of each edge server by using the non-relational database as temporary storage of encrypted data; determining a time node for establishing a data transmission request by combining a priority processing algorithm and a monitoring system state, asynchronously storing transmitted messages by using a message queue middleware model when encrypting data transmission, and transmitting corresponding messages in a queue form; and carrying out decryption processing on the transmitted encrypted data by using a decryption model, and storing the decrypted plaintext data into a cloud database. Through the scheme of the present disclosure, safety and adaptability are improved.

Description

Cloud edge database middleware synchronization method in distributed environment

Technical Field

The embodiment of the disclosure relates to the technical field of data processing, in particular to a cloud edge database synchronization middleware method in a distributed environment.

Background

Currently, as enterprise informatization evolves, more and more enterprises begin to use databases to store and manage data. Traditional databases can assist in informatization management of individuals and businesses while providing rich database operations, but with the continued expansion of geographic locations and the continued expansion of business sizes. These databases may be located on different servers or even in different geographical locations, and conventional single server databases cannot meet the problems of data transmission, real-time processing of multiple data, data consistency, and the like. Therefore, to ensure consistency and integrity of the data, these databases need to be synchronized.

However, a common database synchronization is a synchronization of a master database and a slave database, which is a slave logical master device and a slave device, and the slave device is a complete copy of the master device, and the master database is identical to the slave database, but a master-slave relationship exists from the management point of view. Although the consistency and synchronization of the databases can be ensured by the traditional master-slave databases, a plurality of solutions for real-time synchronization exist at present, and most of the solutions utilize a caching technology to read the databases in real time and synchronously change the databases in real time. However, the problems of data transmission, data access, data security and the like are still difficult to solve in a distributed environment, and if a plurality of databases to be synchronized are mutually contained, the problem of incapability of synchronization is caused not only by the simple replication of the main database. In actual business, it is often necessary to conduct data interactions and integration between different database systems.

Therefore, there is a need for a cloud-edge database synchronization middleware method in a distributed environment with high security and adaptability.

Disclosure of Invention

In view of the above, the embodiments of the present disclosure provide a method for synchronizing middleware of cloud-edge databases in a distributed environment, which at least partially solves the problems of poor security and adaptability in the prior art.

The embodiment of the disclosure provides a cloud edge database middleware synchronization method in a distributed environment, which comprises the following steps:

step 1, building an application scene according to an edge database and a cloud database corresponding to a plurality of distributed edge servers;

step 2, updating data of an edge database in an application scene design data detection model;

step 3, encrypting the data updating content of the edge database by utilizing an encryption algorithm;

step 4, establishing a cache model of each edge server by using the non-relational database as temporary storage of encrypted data;

step 5, determining a time node for establishing a data transmission request by combining a priority processing algorithm and a monitoring system state, asynchronously storing transmitted messages by using a message queue middleware model when encrypting data transmission, and transmitting corresponding messages in a queue form;

and 6, decrypting the transmitted encrypted data by using a decryption model, and storing the decrypted plaintext data into a cloud database.

According to a specific implementation manner of the embodiment of the present disclosure, the step 1 specifically includes:

and connecting the edge databases corresponding to the edge servers with a cloud database, wherein the cloud database gathers basic information of all the edge databases, monitors update data of all the edge databases, carries out subsequent processing, and finally synchronizes to the cloud database as a summation to form an application scene.

According to a specific implementation manner of the embodiment of the present disclosure, the step 2 specifically includes:

step 2.1, disguising the detection model as a database Slave node, and acquiring updated log information by sending dump protocol to the master database;

step 2.2, analyzing the updated log information by a data analyzer in the detection model, storing the log information in a data memory, and returning an ACK (acknowledgement) to finish the acquisition and analysis of the data log once;

and 2.3, summarizing and storing the acquired data.

According to a specific implementation manner of the embodiment of the present disclosure, the step 3 specifically includes:

step 3.1, acquiring a corresponding key format according to a preset algorithm model, and generating a key file in an I/O stream form and storing the key file;

step 3.2, splitting the data into a plurality of plaintext data blocks and performing chained encryption in an encryption mode of a CBC chained password block;

and 3.3, splicing the encrypted plaintext data blocks to obtain ciphertext data.

According to a specific implementation manner of the embodiment of the present disclosure, the step 4 specifically includes:

step 4.1, using Redis as the basis of a non-relational database model, and deploying the Redis servers on different servers;

and 4.2, storing the encrypted ciphertext data into a non-relational database Redis corresponding to the local machine.

According to a specific implementation manner of the embodiment of the present disclosure, the step 5 specifically includes:

step 5.1, judging ciphertext data through a priority algorithm, and creating transmission requests at different times according to different solutions corresponding to the type of the ciphertext data;

step 5.2, selecting the time with lower system resource utilization rate or the time with lower system pressure to transmit data by monitoring the system state;

and 5.3, transmitting the message of the transmission request in a queue manner through the message queue middleware.

According to a specific implementation manner of the embodiment of the present disclosure, the step 6 specifically includes:

step 6.1, obtaining a transmitted plaintext data block by using a decryption model;

and 6.2, splicing the decrypted plaintext data blocks to obtain plaintext data and summarizing the plaintext data to a cloud database.

The cloud edge database synchronization middleware scheme in the distributed environment in the embodiment of the disclosure comprises the following steps: step 1, building an application scene according to an edge database and a cloud database corresponding to a plurality of distributed edge servers; step 2, updating data of an edge database in an application scene design data detection model; step 3, encrypting the data updating content of the edge database by utilizing an encryption algorithm; step 4, establishing a cache model of each edge server by using the non-relational database as temporary storage of encrypted data; step 5, determining a time node for establishing a data transmission request by combining a priority processing algorithm and a monitoring system state, asynchronously storing transmitted messages by using a message queue middleware model when encrypting data transmission, and transmitting corresponding messages in a queue form; and 6, decrypting the transmitted encrypted data by using a decryption model, and storing the decrypted plaintext data into a cloud database.

The beneficial effects of the embodiment of the disclosure are that: according to the scheme, based on traditional data synchronization, a synchronization middleware of a cloud edge database in a distributed environment is provided, an encryption and decryption model, a distributed cache, a priority algorithm and a message queue middleware are used for processing data, and a system is monitored. The system model can be applied to synchronization of the edge database and the cloud database in the distributed environment, meanwhile, on the basis of ensuring synchronization, safety, asynchronism and high throughput rate are realized, and the system model is applicable to both a small data environment and a distributed large data environment. The priority algorithm can be well matched with message transmission in the scene, and meanwhile, load balancing in edge calculation is applied, so that the resource utilization rate is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.

Fig. 1 is a schematic flow chart of a cloud edge database synchronization middleware method in a distributed environment according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a specific implementation of a cloud edge database synchronization middleware method in a distributed environment according to an embodiment of the present disclosure;

fig. 3 is a communication flow chart of a Master node and a Slave node provided in an embodiment of the disclosure;

FIG. 4 is a diagram of a camouflage model architecture provided in an embodiment of the present disclosure;

FIG. 5 is a flow chart of a chained cipher block encryption mode provided in an embodiment of the present disclosure;

fig. 6 is a flowchart of a priority algorithm provided in an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

Other advantages and effects of the present disclosure will become readily apparent to those skilled in the art from the following disclosure, which describes embodiments of the present disclosure by way of specific examples. It will be apparent that the described embodiments are merely some, but not all embodiments of the present disclosure. The disclosure may be embodied or practiced in other different specific embodiments, and details within the subject specification may be modified or changed from various points of view and applications without departing from the spirit of the disclosure. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. All other embodiments, which can be made by one of ordinary skill in the art without inventive effort, based on the embodiments in this disclosure are intended to be within the scope of this disclosure.

It is noted that various aspects of the embodiments are described below within the scope of the following claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the present disclosure, one skilled in the art will appreciate that one aspect described herein may be implemented independently of any other aspect, and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. In addition, such apparatus may be implemented and/or such methods practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.

It should also be noted that the illustrations provided in the following embodiments merely illustrate the basic concepts of the disclosure by way of illustration, and only the components related to the disclosure are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.

In addition, in the following description, specific details are provided in order to provide a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.

As enterprise informatization has evolved, more and more enterprises began using databases to store and manage data. Traditional databases can assist in informatization management of individuals and businesses while providing rich database operations, but with the continued expansion of geographic locations and the continued expansion of business sizes. These databases may be located on different servers or even in different geographical locations, and conventional single server databases cannot meet the problems of data transmission, real-time processing of multiple data, data consistency, and the like. Therefore, to ensure consistency and integrity of the data, these databases need to be synchronized.

Meanwhile, as clouds become an unprecedented data supply channel, the importance of data accuracy, consistency, and privacy increases. A seemingly slight data error or malfunction may have a significant negative impact. However, ordering and synchronizing data in real time with existing data, and then periodically parsing the data to achieve real-time synchronization while maintaining data integrity can be tedious and expensive, and data real-time synchronization is also one of the most valuable data management tools.

The real-time synchronization of the data can ensure accurate, safe and compliant data. Consistency between each data source and its different endpoints is ensured. When data is entered, it is cleaned, checked for errors, duplicates, and consistency before being put into use. Local synchronization involves devices and computers that are adjacent to each other, while remote synchronization is performed through a mobile network. The data must remain consistent throughout the data record. If the data is modified in any way, it must be updated in real time in each system to avoid errors, prevent privacy disclosure, and ensure that the latest data is the only information available. The real-time synchronization of data ensures that all records remain consistent throughout.

Middleware (Middleware) is a software layer located in a computer software system whose main functions are to connect, support and coordinate distributed, independently functioning components and services. The middleware is positioned above the operating system and below the application software to provide services and support for the application software, and meanwhile, the application software is connected with the underlying hardware, the network, the database and the like so that the application software and the underlying hardware, the network, the database and the like can cooperatively work. The middleware plays a role of 'intermediation', connects and coordinates different software components, and enables the whole system to be more flexible, maintainable and extensible.

The advent of database synchronization middleware has solved the data synchronization problem, by using database synchronization middleware, enterprises can better manage distributed databases and process data more efficiently. The method can ensure real-time synchronization of the data and ensure consistency and integrity of the data among different databases. This is important for improving the informatization management level and data processing capacity of enterprises. In practical applications, the database synchronization middleware may perform a variety of functions, such as data replication, data conflict resolution, data backup and recovery, and the like. It can be flexibly configured according to the needs and scale of enterprises, providing a highly available and scalable solution. The meaning of the database synchronization middleware is that:

1. data consistency can be achieved between different systems. Data consistency is achieved between databases by automatically replicating changes to the data. When data in one database is changed, the synchronization process automatically applies the change to other related databases, so as to ensure that the contents of the databases are consistent. This not only prevents errors and collisions due to data inconsistencies, but also helps to improve the reliability of the data and the accuracy of the business decisions.

2. And the service efficiency and flexibility are improved. Through database synchronization, enterprises can acquire latest data information in time wherever the data are stored. This enables enterprises to make decisions quickly, improving work efficiency. At the same time, many database synchronization tools provide flexible deployment and configuration options that enable enterprises to customize according to their own business needs.

3. The disaster tolerance and high reliability of the database are solved. Real-time synchronization provides a guarantee for achieving disaster recovery and high availability of the database. When one database fails or is not available, the synchronous process can automatically switch the flow to the standby database, so that the continuity and stability of the service are ensured. This provides an enterprise with higher data reliability and business recovery capabilities.

4. Multiple database types under different environments are supported. Enterprises typically use different types of databases, including relational databases, non-relational databases (NoSQL), and the like. The design of the database synchronization middleware is to provide a unified synchronization solution in consideration of diversified database types, and the complexity of system integration is reduced.

5. And the system integration cost is reduced. Database middleware may help reduce the cost of system integration. Without middleware, business systems may take more time and resources to handle integration and data interaction problems between different database systems.

However, a common database synchronization is a synchronization of a master database and a slave database, which is a slave logical master device and a slave device, and the slave device is a complete copy of the master device, and the master database is identical to the slave database, but a master-slave relationship exists from the management point of view. Although the consistency and synchronization of the databases can be ensured by the traditional master-slave databases, a plurality of solutions for real-time synchronization exist at present, and most of the solutions utilize a caching technology to read the databases in real time and synchronously change the databases in real time. However, the problems of data transmission, data access, data security and the like are still difficult to solve in a distributed environment, and if a plurality of databases to be synchronized are mutually contained, the problem of incapability of synchronization is caused not only by the simple replication of the main database. In actual business, it is often necessary to conduct data interactions and integration between different database systems. For example, data from different databases is integrated into a unified view.

The embodiment of the disclosure provides a cloud edge database synchronization middleware method in a distributed environment, which can be applied to a data management process of enterprises or governments.

Referring to fig. 1, a flow diagram of a cloud edge database synchronization middleware method in a distributed environment is provided in an embodiment of the present disclosure. As shown in fig. 1 and 2, the method mainly comprises the following steps:

optionally, the step 1 specifically includes:

In the implementation, a new application environment can be built, edge databases are arranged in different servers, the edge databases are a sub-database of the cloud databases, and in the application, the edge databases are mostly changed and synchronized to the cloud databases for summarization, so that the purpose of ensuring the safety consistency of data in a distributed environment is achieved.

on the basis of the above embodiment, the step 2 specifically includes:

and 2.3, summarizing and storing the acquired data.

In practice, a masquerading model is established (the model masquerades as a Slave node (Slave) of the edge database), and a database Binary Log (Binary Log) of the edge database is started. The Binary Log is the most important Log system in MySQL database, and records all DDL statements and DML statements (except data query statements), the DML statements contain SELECT, UPDATE, INSERT, DELETE, which are languages used to operate on data in the database, and DDL contains CREATE, ALTER, DROP mainly, etc. for initializing operations of defining or changing the structure of TABLEs (TABLEs), data types, connections and constraints between TABLEs, etc. The Binary Log is recorded in the form of an event and is transaction-safe.

The Master node of MySQL writes the change of data into a Binary file (Binary Log), and records various update information thereof. The masquerading model simulates a Slave node of MySQL and sends dump protocol to a Master node. After receiving the request, the Master node pushes the Binary Log to the Slave node, and the Slave node copies the Binary Log to the own Relay Log (Relay Log), then analyzes the Log and uses the needed Log information. The overall flow is shown in fig. 3.

The overall architecture of the disguised model includes a Server, an Instance queue. The queue end is a data queue corresponding to one database, and one server end can monitor a plurality of databases and can comprise 1-n queue ends. There are also different sub-modules (Event Parser Event Parser, event Sink connector, event Store and Meta Manager message Manager) under the queue side. The Event Parser Event Parser obtains the last time the Log was successfully obtained from the Log Position before sending the dump command to the Master node (if the server is started for the first time, it starts from the file header). After receiving the data, the Master node of MySQL pushes the Binary Log to the Slave node, and after receiving the data, the Slave node analyzes the data and transmits the data to a corresponding Event Sink connector for data storage, and the data is stored to the Event Store. After all transmission is successful, the Event Store returns Ack confirmation, and records the Log Position of the new round. The architecture and flow chart is shown in fig. 4.

further, the step 3 specifically includes:

In the specific implementation, considering that the safety and reliability of the data need to be ensured, if the transmission of the original data is directly carried out, the safety problem of data theft can occur, so that an encryption algorithm model is established. A complete encryption step includes generating a key file, setting an encryption mode, and reading the key to encrypt the data.

Two encryption algorithms can be considered to be taken as the basis of the model, namely an RSA encryption algorithm and an AES encryption algorithm, and the two algorithms can be selected to be used or a single algorithm can be used. The RSA algorithm is an asymmetric encryption algorithm that uses two keys (public and private), the public key of which encrypts data and the private key of which decrypts data. The AES algorithm is a symmetric encryption algorithm, which uses a 16-bit single key to divide a plaintext into fixed-length data blocks by using a block cipher, and encrypts each data block to obtain a ciphertext. The corresponding secret key is generated according to the selected encryption algorithm, the file is generated in the form of an I/O stream, and the file is conveniently stored and then applied.

The encryption mode (ciper-block encryption) of the CBC chained Cipher block may be adopted. The data to be encrypted is divided into data blocks with fixed length, and the traditional ECB (Electronic Codebook) mode is to encrypt the encrypted data blocks individually, so that the encryption sequence of the data blocks is not required, and the encrypted data blocks can be recombined according to the transmitted sequence. However, the ciphertext encrypted by the plaintext is the same, so that the data security cannot be well ensured. While CBC chained encryption modes are fast, first generate a 16 byte initialization vector (Initialization Vector)With iv as the first block of data, each block of plaintext data is first exclusive-ored with the block of ciphertext and then encrypted. The encrypted ciphertext is taken as an exclusive or object of the next group of plaintext data blocks, finally the encrypted data is obtained, and the generated iv vector is transmitted to a subsequent module together for use in decryption. The embodiment of the disclosure ensures that the result of each encryption is different, and increases the security of data encryption. The chained cipher block encryption mode is shown in fig. 5.

on the basis of the above embodiment, the step 4 specifically includes:

In specific implementation, after explaining how to encrypt and decrypt the data detected by the Slave node, we also need to consider the problem of storing and transmitting multiple pieces of data later. Not only the reconnaissance of the data is considered, but also the parallel storage and transmission of the reconnaissance data of multiple servers are ensured. Because conventional relational databases structure stored data in a rank-by-rank format, the I/O and query efficiency of a hard disk is quite low if we accept a large amount of data. The database synchronization middleware is designed to simplify the storage and improve the transmission efficiency, and if the relational database is still used for storing the data to be transmitted, the efficiency problem is reduced. Therefore, the method proposes to store the data to be transmitted after encryption by using the non-relational database. The non-relational database is stored in key value pairs, can be custom-structured, is not stored in a single row and column any more, and can reduce unnecessary space, time and cost.

Redis can be used as the basis of a non-relational database model, redis servers are deployed on different servers, a single Redis can support data reading and writing for tens of thousands of times per second, after scout data are acquired, encryption and decryption algorithms are used for encryption and then are stored in a local non-relational database Redis, different edge databases are provided with different Redis non-relational databases in different servers, and a plurality of Redis non-relational databases can be connected into a cluster to achieve the effect of distributed data processing. Meanwhile, a AOF (Append Only File) persistent operation is configured, an independent file is set for each Redis non-relational database, each write command is recorded in a snapshot mode, namely, each data storage is recorded, when an abnormal condition such as power failure or downtime occurs to one edge server node, the Redis non-relational database can load stored data according to the snapshot content in the file after restarting, and the purpose of recovering the data is achieved.

Redis of different edge servers in a distributed environment stores information of corresponding edge databases, namely information of the encrypted edge databases which are changed. After the stored information is processed by a priority algorithm described later, the stored information is transmitted to the message queue middleware in the form of an asynchronous producer for subsequent consumption.

further, the step 5 specifically includes:

In practice, after storing encrypted data by using the Redis non-relational database, we need to consider under what conditions the data is transmitted. In the conventional synchronization model, the timing transmission of the message is mostly adopted, that is, a fixed time is set, and the transmission is performed at the time point, but there are other problems that a proper time cannot be selected. To accomplish this, we set a corresponding priority handling algorithm PA (Priority Algorithm).

The priority algorithm summarizes the current data information and the network resource conditions, and determines when to transmit based on this. The algorithm comprises three parts, wherein the first part is the case of self information, the second part is the case of buffer quantity, and the third part is the utilization condition of network resources. The self message is as described above, and the obtained edge database change message is stored in the non-relational database Redis after being encrypted. In practical application, besides some basic information data, there is also alarm information for recording abnormal state of the system, and this information is more important than the basic information data, which we call special information, and the information needs to be transmitted in time. When analyzing the acquired data, recording the type of the data, encrypting the information and storing the information into a non-relational database Redis queue if the data type is basic information data, and waiting for the next judgment. If the data type is special information, a data transmission request is created, and basic information data in the Redis queue are added into the transmission queue together.

The buffer number is the number of basic information stored in the Redis, and given a suitable threshold value P, when the buffer number exceeds P and no special message with higher priority is available before the buffer number, a transmission request is created to transmit all basic information in the Redis buffer.

The network resource utilization is determined in order to select an appropriate time for processing the transmission request task created in the first and second parts by the algorithm PA. In a practical environment, a service layer of a server may have a large number of resource requests at the same time, and if we perform data transmission, a great pressure is applied to the server, and in a serious case, the server is down or crashed. So we will create a monitoring task after receiving the transmission requests of the first and second parts in the foregoing, the task starts to detect the network resource utilization, when the resource utilization of the system reaches the load balance in a proper range, we will process the transmission requests and transmit data, and close the monitoring task after the end. The priority algorithm flow is shown in fig. 6.

After the task of data transmission is completed using the priority algorithm, we need to consider how to receive the data. The conventional method directly transmits data to a required system through an interface, so that the problem is that a target system is coupled with a transmission system, the data is lost when the target system fails, the transmission system fails or is abnormal, and the code of the original system needs to be changed frequently when the system is expanded.

The message queue middleware is used as a solution model, information is transmitted in the form of a message queue, so that the system is decoupled, the transmission system directly transmits the message to the message queue, and an interface of a target system is not required to be called, so that if the target interface is problematic, the secondary transmission cannot be affected, and when the system is expanded, a code is not required to be changed, and a consumer can directly consume the message from the message queue. The model takes an edge database as a Producer (Producer), takes data processed by a priority algorithm as a product, uploads the data in a form of a message queue, and can apply resources and other places without receiving a confirmation Ack transmitted by a cloud after transmitting information by an edge terminal. The cloud database is used as a Consumer (Consumer), and can adopt a priority algorithm to asynchronously process products in the message queue according to different scheduling strategies. Meanwhile, the edge databases on different servers can build message queue clusters, each server configures own message queue instance, and message communication can be carried out between a plurality of instances if the need exists.

On the basis of the above embodiment, the step 6 specifically includes:

In the specific implementation, considering that the previous module performs encryption processing on the data in order to ensure the security of the data, when the data is finally acquired and stored, the data is decrypted and classified according to the type of the data and stored, so that a decryption model is designed and stored after decryption.

Similar to the encryption model, we also adopt the decryption mode of CBC chained Cipher blocks (Cipher-block decryption). Dividing the data to be decrypted into fixed-length ciphertext data blocks, and first obtaining a 16-byte initialization vector (Initialization Vector) delivered from the encryption modelThe first ciphertext data block is first algorithmically decrypted and then exclusive-ored with the initialization vector iv to obtain a first set of plaintext data blocks. Each group of ciphertext data blocks is decrypted by an algorithm, then exclusive OR operation is carried out on the ciphertext data blocks of the previous group to obtain plaintext data blocks, and the generated plaintext data blocks are spliced to obtain decrypted data. And finishing the decryption operation.

After obtaining the decrypted plaintext data, the data update of the edge end needs to be stored and stored in a cloud database. The cloud database is a whole total database, which contains all data information of all edge databases, the data information is summarized to the cloud database and distinguished by table names, so that when data are stored in the cloud database, the data are spliced and updated according to the basic information of the updated table and the naming standards of the table names of the database. Based on basic database sentences, updating an edge database information table representing the data update sentences in the cloud database in a database sentence splicing mode, so that the data update operation of the edge database is completely synchronized to the cloud database in a specific mode, and the cloud database synchronization middleware is constructed in a distributed environment.

According to the cloud edge database synchronization middleware method under the distributed environment, a new application environment is provided, edge databases are arranged in different servers, the edge databases are one sub-database of the cloud database, the edge databases are mostly changed in application and synchronized to the cloud databases for summarization, and the purpose of guaranteeing the safety consistency of data under the distributed environment is achieved. And establishing a data detection model in the edge database, detecting the change of the data of the database by changing the log information of the database, and obtaining the data change information. The encryption and decryption algorithm is adopted while monitoring the data, so that the transmitted data is encrypted before transmission and decrypted when acquired after transmission, and the safety of data transmission is ensured. And a non-relational database model is established, lateral expansion and load balancing are supported, and a large amount of data and high concurrent access are processed. The middleware transmission model is built, encrypted data is transmitted in the form of message queue middleware, asynchronous communication is achieved, reliability and throughput rate of the system are improved, and the method is applicable to small-sized data environments and distributed large-sized data environments. The priority algorithm can be well matched with message transmission in the scene, and meanwhile, load balancing in edge calculation is applied, so that the resource utilization rate is improved.

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.

The foregoing is merely specific embodiments of the disclosure, but the protection scope of the disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the disclosure are intended to be covered by the protection scope of the disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. The cloud edge database middleware synchronization method in the distributed environment is characterized by comprising the following steps of:

2. The method according to claim 1, wherein the step 1 specifically comprises:

3. The method according to claim 2, wherein the step 2 specifically comprises:

and 2.3, summarizing and storing the acquired data.

4. A method according to claim 3, wherein said step 3 comprises:

5. The method according to claim 4, wherein the step 4 specifically includes:

6. The method according to claim 5, wherein the step 5 specifically comprises:

7. The method according to claim 6, wherein the step 6 specifically includes: