CN115033551A - Database migration method and device, electronic equipment and storage medium - Google Patents
Database migration method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115033551A CN115033551A CN202111652659.1A CN202111652659A CN115033551A CN 115033551 A CN115033551 A CN 115033551A CN 202111652659 A CN202111652659 A CN 202111652659A CN 115033551 A CN115033551 A CN 115033551A
- Authority
- CN
- China
- Prior art keywords
- data
- migration
- target
- database
- verification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a database migration method, a database migration device, an electronic device and a storage medium, and particularly, corresponding migration services are executed respectively by adopting multiple threads or multiple processes, so that the migration efficiency of a database is improved, and historical version views aiming at historical change states of related migration data are stored in a source database respectively according to a set flash-back time period before the migration services of the multiple threads or multiple processes are executed by setting a flash-back time parameter and a historical version query data parameter, so that the migration data are verified through the corresponding historical version views, the consistency and integrity of the migration data are ensured, and the time cost and the calculation cost caused by data verification one by one are greatly reduced.
Description
Technical Field
The present invention relates to the field of data migration technologies, and in particular, to a database migration method and apparatus, an electronic device, and a storage medium.
Background
With the development of the internet era, various data storage technologies are in endless, wherein Database (DB) storage is widely used due to its efficient searching characteristic. With the rapid development of industries such as integrated circuits, there are often situations in an actual scene where data migration of stored data in a source Database is required, for example, part of the data is organized to pursue higher storage capacity of the Database, the stored data in the source Database often needs to be migrated to a target Database with better corresponding storage performance, and for example, to meet actual business requirements, a source Database Management System (DBMS) needs to be upgraded or switched correspondingly, and before System upgrade/switching, each Database under the source DBMS also needs to be migrated to the target DBMS, respectively, so as to avoid damage to important data.
Specifically, the database migration may be divided into two manners, namely physical migration and logical migration, where the logical migration manner mainly uses a third-party data synchronization or data stream synchronization manner to migrate the stored data in the source database to the target database, and this synchronization manner does not need to occupy the running process of the source DBMS, and can perform synchronous migration of the database when the service of the source DBMS is normally executed, so that the method has a wide focus, however, the following defects still exist:
1. the migration efficiency is low.
In the related art, the logical migration of the database for different source DBMSs is usually performed in a single-thread manner, and when the amount of corresponding stored data in the database to be migrated is large, the manner often needs to consume a large amount of time to implement the full migration of the database, that is, in the related art, there is a problem of low database migration efficiency.
2. The data consistency is poor.
In the related art, it is impossible to ensure that the data format of the migrated database is consistent with the data format of the source database in a third-party data synchronization or data stream synchronization manner, and therefore, in the related art, data verification is often required to be performed on migrated data in the target database, which results in additional time cost and calculation cost.
Disclosure of Invention
The embodiment of the application provides a database migration method and device, an electronic device and a storage medium, which are used for improving the migration efficiency of a database while ensuring the data consistency.
In a first aspect, an embodiment of the present application provides a database migration method, including:
and acquiring target data to be migrated from a source database.
The target data is divided into at least one corresponding migration data based on a preset division rule, and corresponding migration check information is generated based on the at least one migration data.
And migrating the at least one migration data and the migration check information to a specified target database through corresponding data migration channels respectively.
And performing data verification on each migration data in the target database based on the migration verification information, and determining the migration result of the target data.
In a second aspect, an embodiment of the present application provides a database migration apparatus, including:
and the acquisition module is used for acquiring the target data to be migrated from the source database.
The dividing module is used for dividing the target data into at least one corresponding migration data based on a preset dividing rule, and generating corresponding migration check information based on the at least one migration data.
And the migration module is used for migrating the at least one migration data and the migration check information to the designated target database through the corresponding data migration channels respectively.
And the verification module is used for performing data verification on each migration data in the target database based on the migration verification information and determining the migration result of the target data.
In an optional embodiment, before the target data to be migrated is obtained from the source database, the obtaining module is further configured to:
and acquiring a source database identifier of the source database and a target database identifier of the target database.
And establishing data source connection between the source database and the target database based on the source database identifier and the target database identifier.
In an optional embodiment, before obtaining the target data to be migrated from the source database, the method further includes: acquiring a target database format of a target database; when the target data to be migrated is acquired from the source database, the acquiring module is further configured to:
and acquiring the specified target original data from the source database.
And carrying out format conversion on the target original data to obtain target conversion data corresponding to the format of the target database, and taking the target conversion data as target data to be migrated.
In an optional embodiment, when generating corresponding migration check information based on at least one migration data, the partitioning module is specifically configured to:
acquiring a historical data view of the source database, wherein the historical data view at least comprises: historical migration data of the source database;
and generating a corresponding data verification table based on the respective data type of the at least one migration data.
And generating corresponding migration check information based on the historical data view and the data verification table.
In an optional embodiment, when performing data verification on each migration data in the target database based on the migration verification information and determining a migration result of the target data, the verification module is specifically configured to:
and performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain corresponding at least one check analysis data, and generating a check analysis table corresponding to the at least one check analysis data.
And comparing the verification analysis table with the data verification table in the migration verification information to obtain a corresponding comparison result, and determining the migration result of the target data based on the comparison result and the historical data view.
In an optional embodiment, when performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain at least one corresponding verification analysis data, the verification module is specifically configured to:
and based on a HASH algorithm HASH, performing data analysis on each migration data in the target database to obtain at least one corresponding HASH record, and taking the at least one HASH record as at least one corresponding check analysis data respectively.
Alternatively, the first and second electrodes may be,
and performing data analysis on each migration data in the target database based on a standard deviation algorithm SD to obtain at least one corresponding standard deviation statistical record, and taking the at least one standard deviation statistical record as at least one corresponding check analysis data respectively.
In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, the processor is enabled to implement any one of the database migration methods in the first aspect.
In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method for migrating a database in the first aspect is implemented.
The technical effect brought by any one implementation manner in the second aspect to the fourth aspect may refer to the technical effect brought by the corresponding implementation manner in the first aspect, and details are not described here again.
The embodiment of the application provides a database migration method, a device, an electronic device and a storage medium, and particularly, migration data in a source database is divided, multithreading or multiple processes are adopted, corresponding migration services are executed on the divided data respectively, so that migration efficiency for the source database is greatly improved, on the other hand, in the embodiment of the application, a flash time parameter and a historical version query data parameter are set, so that the source database respectively stores historical version views aiming at historical change states of related migration data according to a set flash time period before the migration services of the multithreading or multiple processes are executed, further, by acquiring the historical version views related to current migration data and verifying the migration data based on the historical version views, consistency and integrity of the migration data are ensured, the method reduces a large amount of time cost and calculation cost caused by data verification item by item.
Drawings
FIG. 1 is a block diagram of a data migration system according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a data migration method according to an embodiment of the present application;
fig. 3 is a flowchart of a method for determining migration check information according to an embodiment of the present disclosure;
fig. 4 is a flowchart of a historical data view determination method according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of a data migration scenario provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of a data migration apparatus according to an embodiment of the present application;
fig. 7 is a schematic view of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the drawings in the embodiments of the present invention, and it is obvious that the embodiments described are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the problems of poor data consistency and low database migration efficiency in related database migration schemes, embodiments of the present application provide a database migration method, an apparatus, an electronic device, and a storage medium, specifically, partition migration data in a source database, and respectively execute corresponding migration services on the partitioned data by using multiple threads or multiple processes, so as to greatly improve the migration efficiency for the source database, on the other hand, in the embodiments of the present application, by setting a flashback time parameter and a historical version query data parameter, before the migration services of multiple threads or multiple processes are executed, the source database respectively stores historical version views for historical change states of related migration data according to a set flashback time period, and further, by acquiring a historical version view related to current migration data, and verifying the migration data based on the historical version view, thereby ensuring the consistency and the integrity of the migration data and reducing a large amount of time cost and calculation cost caused by data verification item by item.
The preferred embodiments of the present application will be described and illustrated in conjunction with the appended drawings.
Referring to FIG. 1, a diagram of a possible architecture of a data migration system according to an embodiment of the present application is shown. The system architecture diagram includes a first terminal device 110, a second terminal device 120, and a server 130.
In the embodiment of the present application, the first terminal device 110 and the second terminal device 120 include, but are not limited to, a mobile phone, a tablet computer, a notebook computer, a desktop computer, an electronic book reader, an intelligent voice interaction device, an intelligent household appliance, a vehicle-mounted terminal, and the like; the first terminal device 110 and the second terminal device 120 may have clients related to database migration installed thereon, where the clients may be software (e.g., a browser), or may also be a web page, an applet, and the like, and the first terminal device 110 and the second terminal device 120 may interact with the server 130 based on the clients related to database migration, further, the first terminal device 110 deploys a source database to be migrated, and the second terminal device 120 deploys a specified target database, and in a possible scenario, it should be understood that the first terminal device 110 and the second terminal device 120 may be the same electronic device, which is not specifically limited in this application.
On the other hand, the server 130 is a background server corresponding to software, a web page, an applet, or the like, or a server specially used for database migration, which is not limited in this application. The server 130 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform.
It should be noted that the database migration method in the embodiment of the present application may be executed by an electronic device, and the electronic device may be the server 130, the first terminal device 110, or the second terminal device 120, that is, the method may be executed by the server 130, the first terminal device 110, or the second terminal device 120 alone, or may be executed by any combination of the server 130, the first terminal device 110, or the second terminal device 120 together.
In an alternative embodiment, the first terminal device 110, the second terminal device 120 and the server 130 may communicate with each other through a communication network.
In an alternative embodiment, the communication network is a wired network or a wireless network.
It should be noted that fig. 1 is only an example, and the number of the terminal devices and the servers is not limited in practice, and is not specifically limited in the embodiment of the present application.
Based on the application scenario, before the database is migrated, in order to specify the source database to be migrated and the designated target database, the server 130 obtains the source database identifier of the source database and the target database identifier of the target database, and establishes a data source connection between the source database and the target database based on the source database identifier and the target database identifier, thereby ensuring fast migration of related data based on the established data source connection.
In the following, in combination with the application scenarios described above, the database migration method provided in the exemplary embodiment of the present application is described with reference to the accompanying drawings.
Referring to fig. 2, an embodiment of the present application provides a database migration method, including:
s201: and acquiring target data to be migrated from the source database.
Specifically, the target data may be any database object in the source database, including but not limited to: in an optional embodiment, because the database format of the target database may not be consistent with the database format of the source database, the server 130 obtains specified target raw data from the source database based on the obtained target database format, where the target raw data may be any database object in the source database, and performs format conversion on the target raw data, and uses target conversion data conforming to the target database format as target data to be migrated.
S202: and dividing the target data into at least one corresponding migration data based on a preset division rule, and generating corresponding migration check information based on the at least one migration data.
Specifically, the target data is divided into corresponding migration data, and for each divided migration data, a corresponding process or thread is used to execute the related migration service, optionally, the current idle thread number multiThreadNum or the current idle process number multiProssNum is queried, so that the target data is equally divided based on the idle thread number or the idle process number to obtain corresponding migration data.
Further, referring to fig. 3, based on each of the divided migration data, the following method is adopted to generate corresponding migration check information, including:
s2021: acquiring a historical data view of a source database, wherein the historical data view at least comprises the following components: historical migration data of source databases.
In order to ensure consistency of current migration data, a historical data view for current target data in a source database is obtained, wherein the historical data view is determined by setting a preset flashback historical time period allow _ flash _ time and a preset flashback historical time point flash, specifically, the source database performs full-scale data consistency verification on data stored in the source database at each set flashback historical time point flash, and stores a corresponding historical data view, and the server 130 can determine a historical modification state of the target data by obtaining the corresponding historical data view, so that consistency of the target data is determined based on the determined historical modification state.
Optionally, referring to fig. 4, determining the corresponding historical data view in the following manner includes:
s401: and determining the flash history time point flash, and determining the corresponding time period between the current time and the flash history time point flash.
S402: and judging whether the time period belongs to a preset flash-back history time period, if so, executing the following step S403, otherwise, executing the step S404.
S403: determining the corresponding relation between the flash time point flash and the corresponding service storage time in a preset service balance binary tree (service storage time, service ID), wherein:
if the flash time point of the last flash is less than the corresponding minimum service storage time in the preset service balance binary tree < service storage time, service ID >, executing the following step S406;
if the latest flashback time point flash is consistent with any one service storage time in the preset service balanced binary tree < service storage time, service ID >, executing the following S407;
if the flash time point of the last flash is the maximum service storage time corresponding to the preset service balanced binary tree < service storage time, service ID >, the following S408 is executed.
S404: and returning the set misoperation data state.
S406: and taking the service ID corresponding to the minimum service storage time as a target service ID to be inquired, and acquiring a historical data view corresponding to the target service ID from a preset historical data view set based on the target service ID.
S407: and taking the service ID corresponding to the consistent service storage time as a target service ID to be inquired, and acquiring a historical data view corresponding to the target service ID from a preset historical data view set based on the target service ID.
S408: and taking the service ID corresponding to the maximum service storage time as a target service ID to be inquired, and acquiring a historical data view corresponding to the target service ID from a preset historical data view set based on the target service ID.
Further, after the execution of the above S406, S407, and S408 is completed, the above S404 is executed to avoid the related service from being repeatedly executed.
Based on the steps, the corresponding service ID can be obtained from the preset service balanced binary tree based on the corresponding time length between the current time and the set flash time point flash, and the historical data view of the version corresponding to the target data is obtained based on the determined service ID, so that the consistency and the integrity of the data are ensured.
S2022: and generating a corresponding data verification table based on the respective data type of the at least one migration data.
Specifically, to further ensure the consistency of the data, a corresponding data verification table is generated based on the respective data type of each migration data, where the data verification table at least includes the respective data type of each migration data, and optionally, the data verification table may further include an index, a default value, a rule, a trigger, a user, a function, and the like.
S2023: and generating corresponding migration check information based on the historical data view and the data verification table.
Further, based on the historical data view and the data verification table, migration check information for the target data to be migrated is generated, and the migration check information represents the storage state of the target data in the source database.
S203: and migrating the at least one migration data and the migration check information to a specified target database through corresponding data migration channels respectively.
Furthermore, each migration data and the corresponding migration check information are migrated to the specified target database through the data migration channel of the corresponding idle thread or idle process, respectively, so as to implement data migration for the current target data.
S204: and performing data verification on each migration data in the target database based on the migration verification information, and determining the migration result of the target data.
Optionally, based on a preset analysis algorithm, data analysis is performed on each migration data in the target database to obtain corresponding at least one verification analysis data, and a verification analysis table corresponding to the at least one verification analysis data is generated.
For example, based on the HASH algorithm HASH, data analysis is performed on each migration data in the target database to obtain at least one corresponding HASH record, and the at least one HASH record is respectively used as at least one corresponding check analysis data.
For another example, based on the standard deviation algorithm SD in the statistical analysis algorithm, data analysis is performed on each migration data in the target database to obtain at least one corresponding standard deviation statistical record, and the at least one standard deviation statistical record is respectively used as at least one corresponding verification analysis data.
And further, generating a corresponding verification analysis table based on each obtained verification analysis data to reflect the data change condition of each migration data.
Further, in order to ensure data consistency, a check analysis table is compared with a data verification table to obtain a corresponding comparison result, if the check analysis table is consistent with the data verification table, the data comparison is determined to be correct, further, the historical state of the target data is determined based on the comparison result and a historical data view, and if the historical state of the target data is determined to be consistent with the current state, the target data is further determined to realize data migration from the source database to the target database while the data consistency is ensured.
Referring to fig. 5, in a scenario provided in this embodiment of the present application, in the scenario, corresponding target data is obtained from a source database, and the target data is divided into n equal preset portions to obtain n migration data, and further, the n migration data are migrated to a specified target database through n threads or processes, so as to implement data migration for the target data in the source database.
Referring to fig. 6, a database migration apparatus provided in this embodiment of the present application includes an obtaining module 601, a dividing module 602, a migration module 603, and a checking module 604, where:
the obtaining module 601 is configured to obtain target data to be migrated from a source database.
The dividing module 602 is configured to divide the target data into at least one corresponding migration data based on a preset dividing rule, and generate corresponding migration check information based on the at least one migration data.
The migration module 603 is configured to migrate the at least one migration data and the migration check information to the designated target database through the corresponding data migration channels, respectively.
The checking module 604 is configured to perform data checking on each migration data in the target database based on the migration checking information, and determine a migration result of the target data.
In an alternative embodiment, before obtaining the target data to be migrated from the source database, the obtaining module 601 is further configured to:
and acquiring a source database identifier of the source database and a target database identifier of the target database.
And establishing data source connection between the source database and the target database based on the source database identifier and the target database identifier.
In an optional embodiment, before obtaining the target data to be migrated from the source database, the method further includes: acquiring a target database format of a target database; when the target data to be migrated is acquired from the source database, the acquiring module 601 is further configured to:
and acquiring the specified target original data from the source database.
And carrying out format conversion on the target original data to obtain target conversion data corresponding to the format of the target database, and taking the target conversion data as target data to be migrated.
In an optional embodiment, when generating corresponding migration check information based on at least one migration data, the partitioning module 602 is specifically configured to:
acquiring a historical data view of the source database, wherein the historical data view at least comprises: historical migration data of the source database;
and generating a corresponding data verification table based on the respective data type of the at least one migration data.
And generating corresponding migration checking information based on the historical data view and the data verification table.
In an optional embodiment, when performing data verification on each migration data in the target database based on the migration verification information and determining the migration result of the target data, the verification module 604 is specifically configured to:
and performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain corresponding at least one check analysis data, and generating a check analysis table corresponding to the at least one check analysis data.
And comparing the verification analysis table with the data verification table in the migration verification information to obtain a corresponding comparison result, and determining the migration result of the target data based on the comparison result and the historical data view.
In an optional embodiment, when performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain at least one corresponding verification analysis data, the verification module 604 is specifically configured to:
and based on a HASH algorithm HASH, performing data analysis on each migration data in the target database to obtain at least one corresponding HASH record, and taking the at least one HASH record as at least one corresponding check analysis data respectively.
Alternatively, the first and second liquid crystal display panels may be,
and performing data analysis on each migration data in the target database based on a standard deviation algorithm SD to obtain at least one corresponding standard deviation statistical record, and taking the at least one standard deviation statistical record as at least one corresponding check analysis data respectively.
Based on the same inventive concept as the embodiment of the application, the embodiment of the application also provides electronic equipment which can be used for database migration. In one embodiment, the electronic device may be a server, a terminal device, or other electronic device. In this embodiment, the electronic device may be configured as shown in fig. 7, and includes a memory 701, a communication interface 703 and one or more processors 702.
A memory 701 for storing a computer program executed by the processor 702. The memory 701 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, a program required for running an instant messaging function, and the like; the storage data area can store various instant messaging information, operation instruction sets and the like.
The memory 701 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory (non-volatile memory) such as, but not limited to, a read-only memory (rom), a flash memory (flash memory), a hard disk (HDD) or solid-state drive (SSD), or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 901 may be a combination of the above memories.
The processor 702 may include one or more Central Processing Units (CPUs), a digital Processing Unit, or the like. A processor 702, configured to implement the above-described database migration method when calling the computer program stored in the memory 701.
The communication interface 703 is used for communicating with terminal devices and other servers.
In the embodiment of the present application, the specific connection medium among the memory 701, the communication interface 703, and the processor 702 is not limited. In the embodiment of the present application, the memory 701 and the processor 702 are connected by a bus 704 in fig. 7, the bus 704 is represented by a thick line in fig. 7, and the connection manner between other components is merely illustrative and is not limited. The bus 704 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.
According to an aspect of the application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any of the database migration methods in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The embodiment of the application provides a database migration method, a device, an electronic device and a storage medium, and particularly, migration data in a source database is divided, multithreading or multiple processes are adopted, corresponding migration services are executed on the divided data respectively, so that migration efficiency for the source database is greatly improved, on the other hand, in the embodiment of the application, a flash time parameter and a historical version query data parameter are set, so that the source database respectively stores historical version views aiming at historical change states of related migration data according to a set flash time period before the migration services of the multithreading or multiple processes are executed, further, by acquiring the historical version views related to current migration data and verifying the migration data based on the historical version views, consistency and integrity of the migration data are ensured, and a large amount of time cost and calculation cost caused by data verification item by item are reduced.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method of database migration, comprising:
acquiring target data to be migrated from a source database;
dividing the target data into at least one corresponding migration data based on a preset division rule, and generating corresponding migration check information based on the at least one migration data;
migrating the at least one migration data and the migration verification information to a specified target database through corresponding data migration channels respectively;
and performing data verification on each migration data in the target database based on the migration verification information, and determining the migration result of the target data.
2. The method of claim 1, wherein before obtaining the target data to be migrated from the source database, further comprising:
acquiring a source database identifier of the source database and acquiring a target database identifier of the target database;
and establishing data source connection between the source database and the target database based on the source database identifier and the target database identifier.
3. The method as claimed in claim 1 or 2, wherein before the obtaining the target data to be migrated from the source database, further comprising: acquiring a target database format of the target database; then, the obtaining target data to be migrated from the source database further includes:
acquiring specified target original data from the source database;
and carrying out format conversion on the target original data to obtain target conversion data corresponding to the target database format, and taking the target conversion data as target data to be migrated.
4. The method of claim 1 or 2, wherein generating the corresponding migration check information based on the at least one migration data comprises:
acquiring a historical data view of the source database, wherein the historical data view at least comprises: historical migration data of the source database;
generating a corresponding data verification table based on the respective data type of the at least one migration data;
and generating corresponding migration checking information based on the historical data view and the data verification table.
5. The method according to claim 3, wherein the performing data verification on each migration data in the target database based on the migration verification information to determine the migration result of the target data comprises:
performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain corresponding at least one check analysis data, and generating a check analysis table corresponding to the at least one check analysis data;
comparing the verification analysis table with the data verification table in the migration verification information to obtain a corresponding comparison result, and determining the migration result of the target data based on the comparison result and the historical data view.
6. The method of claim 5, wherein the performing data analysis on each migration data in the target database based on a preset analysis algorithm to obtain at least one corresponding verification analysis data comprises:
performing data analysis on each migration data in the target database based on a HASH algorithm HASH to obtain at least one corresponding HASH record, and taking the at least one HASH record as at least one corresponding check analysis data respectively;
alternatively, the first and second electrodes may be,
and performing data analysis on each migration data in the target database based on a standard deviation algorithm SD to obtain at least one corresponding standard deviation statistical record, and taking the at least one standard deviation statistical record as at least one corresponding check analysis data respectively.
7. A database migration apparatus, comprising:
the acquisition module is used for acquiring target data to be migrated from a source database;
the dividing module is used for dividing the target data into at least one corresponding migration data based on a preset dividing rule and generating corresponding migration check information based on the at least one migration data;
the migration module is used for migrating the at least one migration data and the migration verification information to a specified target database through corresponding data migration channels respectively;
and the verification module is used for performing data verification on each migration data in the target database based on the migration verification information and determining the migration result of the target data.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the database migration method according to any one of claims 1-6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that it, when called by a computer, causes the computer to carry out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652659.1A CN115033551A (en) | 2021-12-30 | 2021-12-30 | Database migration method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111652659.1A CN115033551A (en) | 2021-12-30 | 2021-12-30 | Database migration method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115033551A true CN115033551A (en) | 2022-09-09 |
Family
ID=83118282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111652659.1A Pending CN115033551A (en) | 2021-12-30 | 2021-12-30 | Database migration method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115033551A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115639972A (en) * | 2022-12-23 | 2023-01-24 | 北京志翔科技股份有限公司 | Data migration method and device, electronic equipment and storage medium |
-
2021
- 2021-12-30 CN CN202111652659.1A patent/CN115033551A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115639972A (en) * | 2022-12-23 | 2023-01-24 | 北京志翔科技股份有限公司 | Data migration method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108287669B (en) | Date storage method, device and storage medium | |
| CN109669709B (en) | Data migration method and data migration system for block chain upgrading | |
| CN107622091B (en) | Database query method and device | |
| CN108462760B (en) | Electronic device, automatic cluster access domain name generation method and storage medium | |
| US11296940B2 (en) | Centralized configuration data in a distributed file system | |
| CN102857578B (en) | A kind of file uploading method of network hard disc, system and net dish client | |
| CN107870802B (en) | Virtual machine migration method and device | |
| US11709815B2 (en) | Retrieving index data from an object storage system | |
| CN115277566B (en) | Load balancing method and device for data access, computer equipment and medium | |
| CN112269661B (en) | Partition migration method and device based on Kafka cluster | |
| CN111885184A (en) | Method and device for processing hot spot access keywords in high concurrency scene | |
| CN112256433A (en) | Partition migration method and device based on Kafka cluster | |
| CN115033551A (en) | Database migration method and device, electronic equipment and storage medium | |
| CN111694703A (en) | Cache region management method and device and computer equipment | |
| CN112988062A (en) | Metadata reading limiting method and device, electronic equipment and medium | |
| CN116842012A (en) | Method, device, equipment and storage medium for storing Redis cluster in fragments | |
| US9378230B1 (en) | Ensuring availability of data in a set being uncorrelated over time | |
| CN110377665B (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN108763498B (en) | User identity identification method and device, electronic equipment and readable storage medium | |
| CN111966286A (en) | Multi-data pool hierarchical migration method and system | |
| CN112527780A (en) | Redis cluster creating and managing method, system, computer device and storage medium | |
| CN112148461A (en) | Application scheduling method and device | |
| CN112433891A (en) | Data processing method and device and server | |
| CN112749189A (en) | Data query method and device | |
| CN112860694B (en) | Service data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |