CN112181939A - Method, device and system for switching sub-base and sub-table based on proxy mode - Google Patents
Method, device and system for switching sub-base and sub-table based on proxy mode Download PDFInfo
- Publication number
- CN112181939A CN112181939A CN202010854101.0A CN202010854101A CN112181939A CN 112181939 A CN112181939 A CN 112181939A CN 202010854101 A CN202010854101 A CN 202010854101A CN 112181939 A CN112181939 A CN 112181939A
- Authority
- CN
- China
- Prior art keywords
- data source
- data
- instruction
- database
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application relates to a method, a device and a system for switching a sub-library and a sub-table based on an agent mode, wherein the method comprises the following steps: loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set; receiving a pre-configured data source identifier transmitted by a service logic layer; acquiring a data source link corresponding to the data source identifier from the data source set according to the current data read/write operation instruction and the data source identifier, and acquiring data in the data source according to the data source link; and executing the read/write operation of the sub-base and sub-table according to the current data read/write operation instruction. By the mode, decoupling between the data read/write logic layer and the service logic layer is achieved, the multi-data-source switching assembly can be flexibly plugged and pulled out, intrusiveness is avoided, repeated code development amount is greatly reduced, reliability, rapidness and simplicity are really achieved, and therefore the process of switching the sub-database and the sub-table is greatly accelerated.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a method, a device and a system for switching a sub-library and a sub-table based on an agent mode.
Background
With the development of time and business, more and more tables are in the database, and the data volume in the tables is larger and larger. This undoubtedly places a considerable load on the database, affecting the stability and performance of the system. The most common and effective method for solving the problems is to switch the original single-library single table into the sub-library table. In the prior art, each time a single library list needs to be switched into a sub-library list, two sets of logic codes need to be edited again, wherein one set of logic codes is mainly written into an old library for service reading/writing and is asynchronously written into a new library. The other set is a logic code which takes the new library as the main for the read/write service and asynchronously writes into the old library. The two sets of logic codes are customized logic codes, cannot be used universally for different services, and are poor in flexibility. Once the service code is adjusted, the two sets of codes also need to be adjusted, that is, the service code is coupled, which increases the complexity in the switching process, resulting in a long switching time.
Therefore, how to realize the universal read/write logic codes of the service sub-library and the sub-table can improve the flexibility, and the logic codes are decoupled with the service codes, so that the complexity in the switching process is reduced, and the switching time is shortened.
Disclosure of Invention
In view of this, to solve the technical problems in the prior art, embodiments of the present application provide a method, an apparatus, and a system for switching between a library and a table based on a proxy mode.
The embodiment of the application provides a database-based and table-based switching method based on an agent mode, which is executed by a pre-configured data agent component, and comprises the following steps:
loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set;
receiving a pre-configured data source identifier transmitted by a service logic layer;
acquiring a data source link corresponding to the data source identifier from the data source set according to a pre-configured data read/write operation instruction and the data source identifier, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
and performing sub-library and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and a pre-configured data source indication instruction, wherein the data source indication instruction is used for reading/writing the data in the data source corresponding to the current data read/write operation instruction when indicating that the read/write operation of different service data is performed. In one possible embodiment, when the number of the links generated according to the first data source configuration file includes a plurality of links, the first link pool is configured in the data source set to carry the link corresponding to the first data source configuration file, where the first data source configuration file is any one of the loaded preconfigured data source configuration files.
In one possible embodiment, the preconfigured data read/write operation instructions include: an old database logic read instruction, an old database logic write instruction, an asynchronous new database write logic instruction, a new database read logic instruction, a new database write logic instruction, and an asynchronous old database write logic instruction.
In one possible embodiment, the data sources include a new data source corresponding to data in a new database and an old data source corresponding to data in an old database; the data source indication instructions include: a new data source calling instruction and an old data source calling instruction; according to the preconfigured data read/write operation instruction and the data source identifier, acquiring a data source link corresponding to the data source identifier from the data source set, and acquiring data in the data source according to the data source link, specifically including:
in the first stage, acquiring a new data source link from a data source set according to a new data source calling instruction and a data source identifier corresponding to a new data source, and acquiring data in the new data source according to the new data source link;
and in the second stage, acquiring an old data source link from the data source set according to the old data source calling instruction and the data source identifier corresponding to the old data source, and acquiring data in the old data source according to the old data source link.
In a possible embodiment, according to a data read/write operation instruction and a preconfigured data source instruction, performing a sub-library and sub-table read/write operation on data in a data source, specifically including:
in the first stage, according to the logic instruction of the old database, writing the data in the new data source into the old database; according to the logic instruction of the new database written asynchronously and the logic instruction of the old database read, after the data in the old database is read, the data in the old database is written into the new database asynchronously;
entering a second stage until the data in the old database and the data in the new database reach data consistency;
in the second stage, according to the logic instruction of the new data writing database, writing the data in the new data source into the new data base;
reading the data in the new database and asynchronously writing the data in the old database according to the logic instruction of the read new database and the logic instruction of the asynchronously written old database;
and when the data in the new database is determined to be written into the database, deleting the old database, and deleting all operation instructions for executing the operation on the old database.
In a possible embodiment, the data read/write operation instruction further includes specific location information of the data to be written, so that when performing a sub-base sub-table read/write operation on the data in the data source according to the preconfigured data read/write operation instruction and the data source indication instruction, the data corresponding to the data read/write operation instruction is written to the specific location of the sub-base sub-table.
In one possible embodiment, in the first stage, when data in the new data source is written into the old database according to the logic instruction for writing the old database, and after data in the old database is read according to the logic instruction for asynchronously writing the new database and the logic instruction for reading the old database, the data in the old database is asynchronously written into the new database, and if the first data in the old database is stored in the new database, the updating time of the first data in the old database and the updating time of the first data in the new database are respectively determined;
when the updating time of the first data in the old data is the latest updating time, overwriting the first data in the old database into the new database;
or when the updating time of the first data in the old database is determined to be later than the updating time of the first data in the new database, the first data is abandoned to be written into the new database, wherein the first data is any one of the old database.
The embodiment of the application provides a device for switching between a warehouse and a table based on an agent mode, which comprises:
the processing unit is used for loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set;
the receiving unit is used for receiving the preconfigured data source identifier transmitted by the service logic layer;
the acquisition unit is used for acquiring a data source link corresponding to the data source identifier from the data source set according to the preconfigured data read/write operation instruction and the data source identifier, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
the processing unit is further configured to perform a sub-library and sub-table read/write operation on data in the data source according to the data read/write operation instruction and a preconfigured data source indication instruction, where the data source indication instruction is used to indicate that data in the data source corresponding to the current data read/write operation instruction is read/written when different service data read/write operations are performed.
In one possible embodiment, when the number of the links generated according to the first data source configuration file includes a plurality of links, the first link pool is configured in the data source set to carry the link corresponding to the first data source configuration file, where the first data source configuration file is any one of the loaded preconfigured data source configuration files.
The embodiment of the application provides a database-based and table-based switching system based on an agent mode, which comprises:
the storage module is used for storing a pre-configured data source configuration file;
the data source switching module is used for calling a pre-configured data source configuration file stored in the memory, generating a link according to the data source configuration file and storing the link into a data source set;
the service logic module is used for transmitting the pre-configured data source identification to the data source switching module;
the data source switching module is further used for acquiring a data source link corresponding to the data source identifier from the data source set according to the preconfigured data read/write operation instruction and the data source identifier transmitted by the service logic module, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
and the data source switching module is also used for executing the sub-library and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and the pre-configured data source indication instruction.
An embodiment of the present application provides a computer device, including:
at least one processor and memory;
the processor is configured to execute a sub-library and sub-table switching program based on the proxy mode stored in the memory to implement the sub-library and sub-table switching method based on the proxy mode as described in any embodiment of the first aspect.
An embodiment of the present application provides a computer storage medium, where one or more programs are stored in the computer storage medium, and the one or more programs are executable by the computer device described in the third aspect, so as to implement the method for switching between the libraries and the tables based on the proxy mode described in any implementation manner of the first aspect.
According to the method for switching the databases and the tables based on the proxy mode, decoupling between the data read/write logic layer and the service logic layer is achieved, the data read/write logic is completely transparent to the service logic layer, the service logic layer does not care about data read/write operation any more, only a data source identifier is transmitted to the downstream, the data read/write operation is completed by taking the multi-data source switching component as a proxy, and heavy multi-data source operation is made to be transparent to the service. Moreover, the multi-data source switching component can be flexibly plugged and pulled out, can be completely universal, avoids the defects that codes need to be modified when a new data source is added, and a data read/write logic layer corresponding to the new data source is added, is flexible to apply, accords with the switching principle, is isolated from service codes, and is non-invasive, thereby greatly reducing the development amount of repeated codes, meanwhile, in the double-writing switching process, the data source can be switched by only switching the configuration of the data source, thereby really achieving the purposes of reliability, rapidness and conciseness, and greatly accelerating the switching process of the sub-database and sub-table.
Drawings
Fig. 1 is a schematic flow chart of a method for switching between a plurality of libraries and a plurality of tables based on an agent model according to an embodiment of the present disclosure;
FIG. 2 is a block diagram of a structure for acquiring data to be read/written before a multi-data source proxy component performs a switch between a database and a table;
FIG. 3 is a schematic diagram of a proxy model architecture provided herein;
FIG. 4 is a block diagram illustrating a process for performing a switch between a plurality of data sources and a plurality of tables by the multi-data source proxy component according to the present application;
fig. 5 is a schematic structural diagram of a device for switching between a plurality of libraries and a plurality of tables based on a proxy mode according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a system architecture for switching between a warehouse and a table based on a proxy mode according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
For the purpose of facilitating understanding of the embodiments of the present application, the following description will be made in terms of specific embodiments with reference to the accompanying drawings, which are not intended to limit the embodiments of the present application.
Fig. 1 is a schematic flowchart of a method for switching between a plurality of libraries and tables based on a proxy mode according to an embodiment of the present application, as shown in fig. 1, the method is executed by a multi-data source switching component, and the method includes:
and step 110, loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set.
Specifically, the data source configuration file may include information such as a user name, a password, and a specific location of the data source. For generating links based on the information and storing the links in the data source set. The purpose of generating the link is to directly find a data source and acquire corresponding data from the data source according to the link information.
Alternatively, considering that there may be more than one proxy component calling the same data source each time, multiple links may be generated based on the data source configuration file, and then the multiple links generated based on the same data source configuration file are added to one link pool. For example, in fig. 2, a plurality of links generated by the data source a configuration file are all added to the data source a link pool, and a plurality of links generated by the data source B configuration file are all added to the data source B link pool. Further alternatively, as shown in fig. 2, the data source may include not only one, but also a plurality. Therefore, included in the data source set may be a plurality of data source link pools, each of which establishes a connection with its corresponding data source.
That is, when the number of the links generated according to the first data source configuration file is multiple, the first link pool is configured in the data source set to carry the link corresponding to the first data source configuration file, where the first data source configuration file is any one of the loaded preconfigured data source configuration files.
Referring specifically to fig. 2, fig. 2 is a block diagram illustrating a structure of acquiring data to be read/written before the multiple data source proxy component performs a bank-to-table switch.
The left side of fig. 2 shows the configuration file corresponding to each data source, and fig. 2 shows the configuration files corresponding to data source a and data source B. The middle part is a data source switching agent component which is used for initializing the process of the data source link pool, in fact, the process of establishing the connection between the data source combination and the data source. And routing the data to a service background to finish the final sub-database and sub-table reading/writing task of the data.
Described in step 110 is the process of generating links from the data source profiles and storing them to the data source collection. In the following, the processes of obtaining data, routing to the service background, and completing data reading/writing will be further described. See in particular the following steps:
Specifically, the service layer only needs to transmit the preconfigured data source identifier to the multi-data source switching component. The service layer does not pay attention to the data read/write operation condition any more, and only downloads the pre-configured data source identification.
Optionally, a database operation handle parameter may be downloaded to indicate what operation the multi-data source switching component performs on data in the data source other than read/write. The multi-data source switching component replaces a service background to perform multi-data source switching operation, so that heavy multi-data source operation is transparent to services. It is also in this way that the decoupling between data read/write operations and service codes is achieved. The data read/write operation instruction is stored in the multi-data source switching assembly, is irrelevant to the service, can be suitable for each data source switching service, and has strong universality and flexibility.
Fig. 3 shows a structure diagram of a proxy mode, in an object-oriented system, some objects may bring much trouble to a user or a system architecture due to some reasons, such as that the object creation overhead is large, some operations require security control, or extra-process access is required, so that an access layer for the object may be added when accessing the object, and the access layer is a proxy for the object. Proxy mode is to provide a proxy for other objects to control access to this object. The proxy object acts as an intermediary between the client and the target object. The proxy mode can separate the proxy object from the called target object of the real object, and reduces the coupling degree of the system to a certain extent, thereby improving the expansibility of the system. In this embodiment, the multiple data source switching component plays a role of a proxy. The general class diagram of the proxy mode is shown in fig. 3:
the object is an abstract class or an interface, and is the most common service type definition without special requirements.
RealSubject is a subclass or implementation class of Subject, which is the proxied role. Is a concrete executor of business logic.
Proxy is called Proxy class, which is responsible for the application of real objects. It does preprocessing and postprocessing before and after the real object is processed.
In this embodiment, the Subject in fig. 3 can be understood as a business logic layer. The real execution body of the business logic layer is RealSubject, that is, real data read-write operation is performed by the RealSubject according to the database operation handle parameters. The Proxy in fig. 3 is a multiple data source switching component in this implementation.
The specific operation flow may include:
1. and the business logic layer (Subject) acquires the data source identification and the database operation handle from the client (client), and transmits the data source identification and the database operation handle to the agent, namely a multi-data source switching component (Proxy) through the docking interface.
2. And the multi-data source switching component (Proxy) acquires corresponding data source information according to the data source identification, connects the database according to the data source information, and calls a read-write module (RealSubject) to execute the database operation handle to perform data read-write operation.
3. And the multi-data source switching component (Proxy) returns the data read-write operation result to the business logic layer (Subject).
In this embodiment, the specific processes of acquiring the data source information, connecting the database according to the data source information, and performing the read-write operation may refer to steps 130 to 140.
And step 130, acquiring a data source link corresponding to the data source identifier from the data source set according to the preconfigured data read/write operation instruction and the data source identifier, and acquiring data in the data source according to the data source link.
And step 140, performing the sub-library and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and the pre-configured data source indication instruction.
Specifically, the data source identifier is used to indicate the data source that needs to be acquired. And matching corresponding data source links from the data source set or matching corresponding data source link pools from the data source set according to the data source identification, and then extracting one link from the link pools. The data source identification comprises: a data source name, or a data source ID, etc.
The data source indication instruction is used for indicating data in a data source corresponding to the current data read/write operation instruction to be read/written when different service data read/write operations are executed;
the preconfigured data read/write operation instructions include: an old database logic read instruction, an old database logic write instruction, an asynchronous new database write logic instruction, a new database read logic instruction, a new database write logic instruction, and an asynchronous old database write logic instruction.
The data source comprises a new data source corresponding to the data in the new database and an old data source corresponding to the data in the old database, and the data source indication instruction comprises: a new data source call instruction and an old data source call instruction.
In the specific implementation of step 130, two phases are mainly involved.
In the first stage, acquiring a new data source link from a data source set according to a new data source calling instruction and a data source identifier corresponding to a new data source, and acquiring data in the new data source according to the new data source link;
and in the second stage, acquiring an old data source link from the data source set according to the old data source calling instruction and the data source identifier corresponding to the old data source, and acquiring data in the old data source according to the old data source link.
The first stage is mainly writing old database and assisting writing new database. And writing the data in the new data table into the old database. At this point, all the data in the old database is asynchronously written to the new database.
And entering a second stage when the data in the old database and the data in the new database reach data consistency.
The new database is written as the main, and the old database is written as the auxiliary. The purpose is to prevent logical errors after data is written to a new database, when an old database is also used as an alternative. Data in the database can be read/written to ensure that the service can be executed normally.
Specifically, for completing the data read/write task in the above two phases, see the following:
in the first stage, according to the logic instruction of the old database, writing the data in the new data source into the old database; and asynchronously writing the data in the old database into the new database after reading the data in the old database according to the logic instruction of asynchronously writing the new database and the logic instruction of the old database.
Entering a second stage until the data in the old database and the data in the new database reach data consistency;
in the second stage, according to the logic instruction of the new data writing database, writing the data in the new data source into the new data base;
reading the data in the new database according to the logic instruction of the read new database and the logic instruction of the asynchronous write old database, and then asynchronously writing the data in the new database source into the old database;
when it is determined that the data in the new database completes the writing work, the old database is deleted, and all operation instructions for performing operations on the old database, such as the old database reading logic instruction, the old database writing logic instruction, the old database asynchronous writing logic instruction, the old data source calling instruction, and the like, described above, are deleted. Also, what needs to be removed is the asynchronous write new database logic instruction. Because the writing of data to the old database has been stopped, followed by the writing to the new database, the asynchronous write to new database logical instruction is no longer used.
Specifically, how to judge that the data in the old database and the data in the new database reach data consistency and when the data is no longer written into the old database needs to be implemented according to the following method:
in the first stage, data is written into an old database, the data in the old database is asynchronously read, when the data is asynchronously written into a new database, if the data in the old database is not written yet but is stored in the new database, the data updating time is judged, and if the data updating time in the old database is the latest updating time, the data in the old database is overwritten and written into the new database. If the updating time is later than that of the new database, the data in the old database is completely written into the new database, and the data is abandoned from being written into the new database. After the data in the old database completes one round of operation (no matter all writing or abandoning according to the comparison operation, etc.), the data in the new database and the old database are compared again to determine whether the inconsistency exists.
If the data are inconsistent, the data synchronization operation is carried out again, the operation is repeated in a circulating mode until the data are completely consistent, and then the second stage is carried out.
In the second stage, when the 'reading and writing are performed mainly by the new library and asynchronously written into the old library', the 'asynchronously written into the old library' can be removed as long as the normal business processing is determined and the data is normal, and at the moment, the data is not written into the old library any more.
Referring specifically to fig. 4, fig. 4 is a block diagram illustrating a structure of the multiple data source proxy component performing a sub-library and sub-table switching process provided in the present application, and fig. 4 illustrates that a data double-write process is completed at different stages.
The left side of fig. 4 is the data double write process for completing the first phase, and the right side of fig. 4 is the data double write process for completing the second phase.
It should be noted that the multiple data source switching component is a component that can be embedded in the service code. As introduced above, the multiple data source switching components may include data read/write operations instructions, as well as data source indication instructions. Before executing the multi-data source logic switching and the read/write operation, a worker only needs to adaptively adjust the data source information indicated in the data source indication instruction, then inserts the multi-data source switching component into the upper layer of the data read/write interface, and the service logic layer only needs to transmit the preconfigured data source identifier and optionally can also include a database operation handle parameter. By the mode, decoupling of data reading/writing logic and service operation logic is achieved, dynamic plugging and unplugging can be conducted on the multi-data-source switching assembly, service codes are isolated, and the multi-data-source switching assembly is non-invasive, so that repeated code development amount is greatly reduced, meanwhile, in the double-writing switching process, the reading and writing data sources can be switched only by switching data source configuration, reliability, rapidness and conciseness are really achieved, and the switching process of sub-database and sub-table is greatly accelerated. Moreover, the assembly replaces a service background to perform multi-data source switching operation, so that heavy multi-data source operation is transparent to the service.
For example, when a data source is newly added, the data read/write operation instruction described above can be directly used without rewriting a set of data read/write logic corresponding to the new data source. And additionally configuring a set of data source indication instructions, namely informing the service data read/write instructions when to read the new data source. And generating a link corresponding to the new data source, adding the link into the data source set, configuring a data source identifier in the data read/write logic layer, finding the link corresponding to the new data source identifier in the data source set according to the data source identifier corresponding to the new data source, and acquiring the data in the new data source according to the link.
According to the method for switching the databases and the tables based on the proxy mode, decoupling between the data read/write logic layer and the service logic layer is achieved, the data read/write logic is completely transparent to the service logic layer, the service logic layer does not care about data read/write operation any more, only data source identification is transmitted to the downstream, and the data read/write operation is completed by taking the multi-data source switching component as a proxy, so that heavy multi-data source operation is transparent to the service. Moreover, the multi-data source switching component can be flexibly plugged and pulled out, can be completely universal, avoids the defects that codes need to be modified when a new data source is added, and a data read/write logic layer corresponding to the new data source is added, is flexible to apply, accords with the switching principle, is isolated from service codes, and is non-invasive, thereby greatly reducing the development amount of repeated codes, meanwhile, in the double-writing switching process, the data source can be switched by only switching the configuration of the data source, thereby really achieving the purposes of reliability, rapidness and conciseness, and greatly accelerating the switching process of the sub-database and sub-table.
Fig. 5 is a device for switching between libraries and tables based on a proxy mode according to an embodiment of the present application, where the device includes: a processing unit 501, a receiving unit 502 and an obtaining unit 503.
The processing unit 501 is configured to load a preconfigured data source configuration file, generate a link according to the data source configuration file, and store the link into a data source set;
a receiving unit 502, which receives a preconfigured data source identifier delivered by a service logic layer;
an obtaining unit 503, configured to obtain, according to a preconfigured data read/write operation instruction and a data source identifier, a data source link corresponding to the data source identifier from the data source set, and obtain, according to the data source link, data in the data source corresponding to the data source identifier;
the processing unit 501 is further configured to, according to a data read/write operation instruction and a preconfigured data source indication instruction, perform a sub-library and sub-table read/write operation on data in a data source, where the data source indication instruction is used to indicate that data in the data source corresponding to the current data read/write operation instruction is read/written when different service data read/write operations are performed.
Optionally, when the number of links generated according to the first data source configuration file includes a plurality of links, configuring a first link pool in the data source set to carry a link corresponding to the first data source configuration file, where the first data source configuration file is any one of the data source configuration files loaded with the pre-configuration.
Optionally, the preconfigured data read/write operation instruction includes: an old database logic read instruction, an old database logic write instruction, an asynchronous new database write logic instruction, a new database read logic instruction, a new database write logic instruction, and an asynchronous old database write logic instruction.
Optionally, the obtaining unit 503 is specifically configured to enable the data source to include a new data source corresponding to data in a new database and an old data source corresponding to data in an old database; the data source indication instructions include: a new data source calling instruction and an old data source calling instruction; according to the preconfigured data read/write operation instruction and the data source identifier, acquiring a data source link corresponding to the data source identifier from the data source set, and acquiring data in the data source according to the data source link, specifically including:
in the first stage, acquiring a new data source link from a data source set according to a new data source calling instruction and a data source identifier corresponding to a new data source, and acquiring data in the new data source according to the new data source link;
and in the second stage, acquiring an old data source link from the data source set according to the old data source calling instruction and the data source identifier corresponding to the old data source, and acquiring data in the old data source according to the old data source link.
Optionally, the processing unit 501 is specifically configured to, in the first stage, write data in the new data source into the old database according to the old database writing logic instruction; according to the logic instruction of the new database written asynchronously and the logic instruction of the old database read, after reading the data in the old database, the data in the old database is written into the new database asynchronously;
entering a second stage until the data in the old database and the data in the new database reach data consistency;
in the second stage, according to the logic instruction of the new data writing database, writing the data in the new data source into the new data base;
reading the data in the new database according to the logic instruction of the read new database and the logic instruction of the asynchronous write old database, and then asynchronously writing the data in the new database source into the old database;
and when the data in the new database is determined to be written into the database, deleting the old database, and deleting all operation instructions for executing the operation on the old database.
Optionally, the data read/write operation instruction further includes specific location information of the data to be written, so that when performing a sub-library sub-table read/write operation on the data in the data source according to the preconfigured data read/write operation instruction and the data source indication instruction, the data corresponding to the data read/write operation instruction is written into the specific location of the sub-library sub-table.
The functions executed by each functional component in the library-based sub-table switching device based on the proxy mode provided in this embodiment have been described in detail in the embodiment corresponding to fig. 1, and therefore are not described herein again.
According to the database-based and table-based switching device based on the proxy mode, decoupling between the data read/write logic layer and the service logic layer is achieved, the data read/write logic is completely transparent to the service logic layer, the service logic layer does not care about data read/write operation any more, only the data source identification is transmitted to the downstream, the data read/write operation is completed by taking the multi-data source switching component as a proxy, and heavy multi-data source operation is transparent to the service. Moreover, the multi-data source switching component can be flexibly plugged and pulled out, can be completely universal, avoids the defects that codes need to be modified when a new data source is added, and a data read/write logic layer corresponding to the new data source is added, is flexible to apply, accords with the switching principle, is isolated from service codes, and is non-invasive, thereby greatly reducing the development amount of repeated codes, meanwhile, in the double-writing switching process, the data source can be switched by only switching the configuration of the data source, thereby really achieving the purposes of reliability, rapidness and conciseness, and greatly accelerating the switching process of the sub-database and sub-table.
Fig. 6 is a schematic diagram of a system architecture for switching between libraries and tables based on a proxy mode according to an embodiment of the present application, specifically referring to fig. 6, the system includes: a storage module 601, a data source switching module 602, and a service logic module 603.
A storage module 601, configured to store a preconfigured data source profile;
the data source switching module 602 is configured to call a preconfigured data source configuration file stored in the storage module 601, generate a link according to the data source configuration file, and store the link into a data source set;
a service logic module 603, configured to transmit a preconfigured data source identifier to the data source switching module 602;
the data source switching module 602 is further configured to obtain a data source link corresponding to the data source identifier from the data source set according to the preconfigured data read/write operation instruction and the data source identifier transmitted by the service logic module 603, and obtain data in the data source corresponding to the data source identifier according to the data source link;
the data source switching module 602 is further configured to perform a sub-library and sub-table read/write operation on data in the data source according to the data read/write operation instruction and the preconfigured data source indication instruction.
The functions executed by each functional component in the library-based sub-table switching device based on the proxy mode provided in this embodiment have been described in detail in the embodiment corresponding to fig. 1, and therefore are not described herein again.
According to the database-based and table-based switching system based on the proxy mode, decoupling between the data read/write logic layer and the service logic layer is achieved, the data read/write logic is completely transparent to the service logic layer, the service logic module does not care about data read/write operation any more, only the data source identification is transmitted to the downstream, the data read/write operation is completed by taking the multi-data source switching module as a proxy, and heavy multi-data source operation is transparent to the service. Moreover, the multi-data source switching module can be flexibly plugged and pulled out, can be completely universal, avoids the defects that codes need to be modified when a new data source is added, and a data read/write logic layer corresponding to the new data source is added, is flexible to apply, accords with the switching principle, is isolated from service codes, and is non-invasive, thereby greatly reducing the development amount of repeated codes, meanwhile, in the double-writing switching process, the data source can be switched by only switching the configuration of the data source, thereby really achieving the purposes of reliability, rapidness and conciseness, and greatly accelerating the switching process of the sub-database and sub-table.
Fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application, where the computer device 700 shown in fig. 7 includes: at least one processor 701, memory 702, at least one network interface 703, and other user interfaces 704. The various components of the proxy mode based sub-pool sub-table switching computer device 700 are coupled together by a bus system 705. It is understood that the bus system 705 is used to enable link pool communication between these components. The bus system 705 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various busses are labeled in figure 7 as the bus system 705.
The user interface 704 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, trackball, touch pad, or touch screen, among others.
It will be appreciated that the memory 702 in the subject embodiment can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (ddr Data Rate SDRAM, ddr SDRAM), Enhanced Synchronous SDRAM (ESDRAM), synclink pool SDRAM (SLDRAM), and Direct memory bus RAM (DRRAM). The memory 702 described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
In some embodiments, memory 702 stores the following elements, executable units or data structures, or a subset thereof, or an expanded set thereof: an operating system 7021 and application programs 7022.
The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application 7022 includes various applications, such as a Media Player (Media Player), a Browser (Browser), and the like, for implementing various application services. A program for implementing the methods according to embodiments of the present application may be included in application 7022.
In this embodiment of the application, the processor 701 is configured to execute the method steps provided by the method embodiments by calling a program or an instruction stored in the memory 702, specifically, a program or an instruction stored in the application 7022, for example, including:
loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set;
receiving a pre-configured data source identifier transmitted by a service logic layer;
acquiring a data source link corresponding to the data source identifier from the data source set according to a pre-configured data read/write operation instruction and the data source identifier, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
and performing sub-library and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and a pre-configured data source indication instruction, wherein the data source indication instruction is used for reading/writing the data in the data source corresponding to the current data read/write operation instruction when indicating that the read/write operation of different service data is performed.
Optionally, when the number of links generated according to the first data source configuration file includes a plurality of links, configuring a first link pool in the data source set to carry a link corresponding to the first data source configuration file, where the first data source configuration file is any one of the data source configuration files loaded with the pre-configuration.
Optionally, the preconfigured data read/write operation instruction includes: an old database logic read instruction, an old database logic write instruction, an asynchronous new database write instruction, a new database read logic instruction, a new database write logic instruction, and an asynchronous old database write logic instruction.
Optionally, the data source includes a new data source corresponding to data in the new database and an old data source corresponding to data in the old database; the data source indication instructions include: a new data source calling instruction and an old data source calling instruction;
in the first stage, acquiring a new data source link from a data source set according to a new data source calling instruction and a data source identifier corresponding to a new data source, and acquiring data in the new data source according to the new data source link;
and in the second stage, acquiring an old data source link from the data source set according to the old data source calling instruction and the data source identifier corresponding to the old data source, and acquiring data in the old data source according to the old data source link.
Optionally, in the first stage, writing data in the new data source into the old database according to the logic instruction of the old database; according to the logic instruction of the new database written asynchronously and the logic instruction of the old database read, after reading the data in the old database, the data in the old database is written into the new database asynchronously;
entering a second stage until the data in the old database and the data in the new database reach data consistency;
in the second stage, according to the logic instruction of the new data writing database, writing the data in the new data source into the new data base;
reading the data in the new database according to the logic instruction of the read new database and the logic instruction of the asynchronous write old database, and then asynchronously writing the data in the new database source into the old database;
and when the data in the new database is determined to be written into the database, deleting the old database, and deleting all operation instructions for executing the operation on the old database.
Optionally, the data read/write operation instruction further includes specific location information of the data to be written, so that when performing a sub-library sub-table read/write operation on the data in the data source according to the preconfigured data read/write operation instruction and the data source indication instruction, the data corresponding to the data read/write operation instruction is written into the specific location of the sub-library sub-table.
The method disclosed in the embodiments of the present application may be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 701. The Processor 701 may be a general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software elements in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and performs the steps of the above method in combination with the hardware thereof.
It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the Processing unit 701701 may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units configured to perform the functions of the present Application, or a combination thereof.
For a software implementation, the techniques herein may be implemented by means of units performing the functions herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
The computer device provided in this embodiment may be the computer device shown in fig. 7, and may execute all the steps of the method for switching between the sub-libraries and the sub-lists based on the proxy mode in fig. 1, so as to achieve the technical effect of the method for switching between the sub-libraries and the sub-lists based on the proxy mode in fig. 1.
An embodiment of the present application further provides a computer storage medium, where one or more programs are stored in the computer storage medium, and the one or more programs may be executed by a computer device corresponding to fig. 7, so as to implement the method for switching between libraries and tables based on the proxy mode described in any embodiment of embodiment 1.
Embodiments of the present application further provide a computer program comprising computer instructions, where the computer instructions are stored in a computer-readable storage medium; the processor in the computer device as described above reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, so that the computer device executes the computer instructions to implement the method for switching between the libraries and the tables based on the proxy mode as described in any embodiment of embodiment 1.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above embodiments are provided to explain the purpose, technical solutions and advantages of the present application in further detail, and it should be understood that the above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present application should be included in the scope of the present application.
Claims (10)
1. A method for switching between a database and a table based on a proxy mode is characterized by comprising the following steps:
loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set;
receiving a pre-configured data source identifier transmitted by a service logic layer;
acquiring a data source link corresponding to the data source identifier from the data source set according to a preconfigured data read/write operation instruction and the data source identifier, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
and performing sub-library and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and a pre-configured data source indication instruction, wherein the data source indication instruction is used for reading/writing the data in the data source corresponding to the data read/write operation instruction when indicating that the read/write operation of different service data is performed.
2. The method according to claim 1, wherein when there are a plurality of links generated according to the first data source configuration file, configuring a first link pool in the data source set to carry a link corresponding to the first data source configuration file, wherein the first data source configuration file is any one of the loaded preconfigured data source configuration files.
3. The method of claim 1 or 2, wherein the preconfigured data read/write operation instructions comprise: an old database logic read instruction, an old database logic write instruction, an asynchronous new database write logic instruction, a new database read logic instruction, a new database write logic instruction, and an asynchronous old database write logic instruction.
4. The method of claim 3, wherein the data sources include a new data source corresponding to data in a new database and an old data source corresponding to data in an old database; the data source indication instructions include: a new data source calling instruction and an old data source calling instruction;
the acquiring, according to the preconfigured data read/write operation instruction and the data source identifier, a data source link corresponding to the data source identifier from the data source set, and acquiring, according to the data source link, data in the data source specifically includes:
in the first stage, according to the new data source calling instruction and a data source identifier corresponding to a new data source, obtaining a new data source link from the data source set, and obtaining data in the new data source according to the new data source link;
and at the second stage, acquiring an old data source link from the data source set according to the old data source calling instruction and a data source identifier corresponding to an old data source, and acquiring data in the old data source according to the old data source link.
5. The method according to claim 4, wherein the performing a sub-bank and sub-table read/write operation on the data in the data source according to the data read/write operation instruction and a preconfigured data source indication instruction specifically comprises:
in the first stage, writing the data in the new data source into the old database according to the logic instruction of the old database; according to the logic instruction for asynchronously writing the new database and the logic instruction for reading the old database, after reading the data in the old database, asynchronously writing the data in the old database into the new database;
entering the second stage until the data in the old database and the data in the new database reach data consistency;
in the second stage, writing the data in the new data source into the new database according to the logic instruction for writing the new database;
according to the logic instruction for reading the new database and the logic instruction for asynchronously writing the old database, after the data in the new database is read, the data in the new database is asynchronously written into the old database;
and when the data in the new database is determined to be written into the database, deleting the old database, and deleting all operation instructions for executing the operation on the old database.
6. The method according to claim 1 or 2, wherein the data read/write operation instruction further includes specific location information of data to be written, so that when performing a sub-library sub-table read/write operation on data in the data source according to a pre-configured data read/write operation instruction and a data source indication instruction, data corresponding to the data read/write operation instruction is written to a specific location of a sub-library sub-table.
7. An apparatus for switching between sub-libraries and sub-tables based on a proxy mode, the apparatus comprising:
the processing unit is used for loading a pre-configured data source configuration file, generating a link according to the data source configuration file and storing the link into a data source set;
the receiving unit is used for receiving the preconfigured data source identifier transmitted by the service logic layer;
the acquisition unit is used for acquiring a data source link corresponding to the data source identifier from the data source set according to a preconfigured data read/write operation instruction and the data source identifier, and acquiring data in the data source corresponding to the data source identifier according to the data source link;
the processing unit is further configured to perform a sub-library and sub-table read/write operation on data in the data source according to the data read/write operation instruction and a preconfigured data source indication instruction, where the data source indication instruction is used to indicate that data in the data source corresponding to the current data read/write operation instruction is read/written when different service data read/write operations are performed.
8. A system for switching between a warehouse and a table based on a proxy mode is characterized by comprising:
the storage module is used for storing a pre-configured data source configuration file;
the data source switching module is used for calling a pre-configured data source configuration file stored in the storage module, generating a link according to the data source configuration file and storing the link into a data source set;
the service logic module is used for transmitting a pre-configured data source identifier to the data source switching module;
the data source switching module is further configured to obtain a data source link corresponding to the data source identifier from the data source set according to a preconfigured data read/write operation instruction and the data source identifier transmitted by the service logic module, and obtain data in the data source corresponding to the data source identifier according to the data source link;
the data source switching module is further configured to perform a sub-library and sub-table read/write operation on data in the data source according to the data read/write operation instruction and a preconfigured data source indication instruction.
9. A computer device, characterized in that the computer device comprises: at least one processor and memory;
the processor is used for executing the library and table splitting switching program based on the proxy mode stored in the memory so as to realize the library and table splitting switching method based on the proxy mode in any one of claims 1-6.
10. A computer storage medium storing one or more programs executable by the computer device of claim 9 to implement the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010854101.0A CN112181939A (en) | 2020-08-20 | 2020-08-20 | Method, device and system for switching sub-base and sub-table based on proxy mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010854101.0A CN112181939A (en) | 2020-08-20 | 2020-08-20 | Method, device and system for switching sub-base and sub-table based on proxy mode |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112181939A true CN112181939A (en) | 2021-01-05 |
Family
ID=73925068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010854101.0A Pending CN112181939A (en) | 2020-08-20 | 2020-08-20 | Method, device and system for switching sub-base and sub-table based on proxy mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112181939A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113468148A (en) * | 2021-08-13 | 2021-10-01 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
| CN113835690A (en) * | 2021-09-23 | 2021-12-24 | 中邮科通信技术股份有限公司 | Read-write separation multi-data source component and working method thereof |
-
2020
- 2020-08-20 CN CN202010854101.0A patent/CN112181939A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113468148A (en) * | 2021-08-13 | 2021-10-01 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
| CN113468148B (en) * | 2021-08-13 | 2023-02-17 | 上海浦东发展银行股份有限公司 | Data migration method and device of database, electronic equipment and storage medium thereof |
| CN113835690A (en) * | 2021-09-23 | 2021-12-24 | 中邮科通信技术股份有限公司 | Read-write separation multi-data source component and working method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10261869B2 (en) | Transaction processing using torn write detection | |
| CN110471690B (en) | Loading method and device for SO file of dynamic link library and storage medium | |
| JP4510345B2 (en) | How to dynamically verify resource compatibility | |
| US9262237B2 (en) | Automating software availability management based on API versioning | |
| US6542167B1 (en) | System and method for flexible software linking | |
| WO2017057111A1 (en) | In-vehicle control device, program update system, and program update software | |
| CN112181939A (en) | Method, device and system for switching sub-base and sub-table based on proxy mode | |
| WO2015154538A1 (en) | Memory booting method and device | |
| CN112882734A (en) | Upgrading method and device, computer equipment and medium | |
| CN112783558B (en) | Method for accelerating self-decompression phase of Linux kernel start | |
| CN112306533A (en) | Front-end framework automatic conversion method and device, computer equipment and storage medium | |
| CN112732300A (en) | Data packet updating method and device, electronic equipment and readable storage medium | |
| CN112416710A (en) | User operation recording method and device, electronic equipment and storage medium | |
| CN112787795A (en) | Cloud server clock synchronization method, computer storage medium and electronic device | |
| CN108664255B (en) | Software upgrading method and device | |
| CN111400256B (en) | Method and device for calling resource file by application program | |
| CN112925796A (en) | Write consistency control method, device, equipment and storage medium | |
| CN109614383B (en) | Data copying method and device, electronic equipment and storage medium | |
| CN114780173B (en) | Method for loading plug-in application, computing device and storage medium | |
| US7886279B2 (en) | Information processing apparatus, control method therefor, computer program, and storage medium | |
| US20050027977A1 (en) | Method and system for maintaining the boot order of mass storage devices in a computer system | |
| CN113126998B (en) | Incremental source code acquisition method and device, electronic equipment and storage medium | |
| CN115129360A (en) | Code file management method and device, electronic equipment and storage medium | |
| CN114171205A (en) | Medical data processing method and device and digital model editor | |
| CN109558181B (en) | Method for enabling non-IE browser to be compatible with OCX control, computer storage medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |