CN114372450A - Data migration testing method and device - Google Patents
Data migration testing method and device Download PDFInfo
- Publication number
- CN114372450A CN114372450A CN202210034083.0A CN202210034083A CN114372450A CN 114372450 A CN114372450 A CN 114372450A CN 202210034083 A CN202210034083 A CN 202210034083A CN 114372450 A CN114372450 A CN 114372450A
- Authority
- CN
- China
- Prior art keywords
- data
- migration
- test
- comparison
- data migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/186—Templates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/194—Calculation of difference between files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention belongs to the technical field of data processing, in particular to a data migration test method and a device, wherein the data migration test method is used for uniformly scheduling corresponding data in a source system and a target system by establishing a universal data migration test platform, and comparing data in the source system and the target system according to a data structure test template and a data content test template by establishing a data structure test template and a data content test template, so that the data comparison efficiency of the source system and the template system is improved; and the checking result can be uniformly output for checking, so that the labor and time cost is saved. The established data migration test platform has the functions of static query comparison test and dynamic migration verification, static data is tested through the static query comparison test function, dynamic data is tested and verified through the dynamic migration verification function, updating of the dynamic data can be achieved, and the data migration test platform can be used after the data migration.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a data migration testing method and device.
Background
In the technical field of computer application, migration of data stored in a computer is often involved, the data migration is to use original existing data, and specific use modes include storage, statistics, analysis, viewing and the like.
For example, when switching between a new system and an old system, data migration is required, that is, data in a source system is migrated to a target system, where the source system is the old system and also is a data migration system, and the target system is the new system and also is a data migration system. It is common that data of business management systems, financial management systems and the like of enterprises move forward, and when a new system and an old system are switched, various business data and financial data are migrated from a source system to a target system, so that the business range of the old system is protected, and the application function of the new system is met.
After migration, the correctness, consistency and availability of the migrated data need to be tested, where the correctness refers to that the migrated data is correct, the consistency refers to that data migrated from an original database is consistent with data stored in a new database table in the data migration process, and the availability refers to that the migrated data must be available.
In the prior art, chinese patent publication No. CN111459415A describes a method, an apparatus, a device, and a storage medium for migrating data between cloud platforms; chinese patent publication No. CN111966665A describes a data migration test method and apparatus for migrating data between old and new systems; chinese patent publication No. CN113010489A describes a data migration method and system, which solves the technical problems of great difference between old and new systems and uncertainty in data migration. None of the technologies disclosed in the above patent documents unifies the function of testing the data after migration, so that the correctness, consistency, and usability after data migration cannot be known.
Disclosure of Invention
The invention aims to provide a data migration testing method, which solves the technical problem that migrated data cannot be tested efficiently in the prior art.
To solve the above technical problem, a first aspect of the present invention is:
a data migration testing method is provided, which is used for testing data migrated from a source system to a target system, and comprises the following steps:
(1) establishing a universal data migration test platform, wherein the data migration test platform is used for uniformly scheduling corresponding data in a source system and a target system;
(2) establishing a data structure test template and a data content test template by analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system and combining a data migration test range and a test strategy;
(3) importing the established data structure test template and the data content test template into the data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case;
(4) and importing the data migration test case into the data migration test platform, after receiving a test instruction, the data migration test platform simultaneously inquires a source table, an intermediate table and a target table, and automatically comparing the data migration test case serving as a template.
Preferably, in the step (4), the comparison contents include comparison of a database table structure, comparison of a data amount, and comparison of data contents, and the results of the test comparison are output in a unified manner.
Preferably, the data migration test platform has a static query comparison test function and a dynamic migration verification function, the static data is tested through the static query comparison test function, and the dynamic data is tested and verified through the dynamic migration verification function.
Preferably, in the step (4), after the automatic comparison, a data comparison result report for checking is derived, the data comparison result report is transmitted to a proofreader, and the proofreader compares the service content in the target system and the service content in the source system by spot-checking the correct and wrong contents in the data comparison result report, so as to verify the correctness of the migration data.
Preferably, in the step (4), during automatic comparison, the data migration test platform performs a large number of high-frequency query transactions on data in the source system and data in the target system, and compares data results of the query transactions.
The second aspect of the present invention is:
provided is a data migration test apparatus including:
a data scheduling module: a universal data migration test platform is established in the data scheduling module, and the data migration test platform is used for uniformly scheduling corresponding data in the source system and the target system;
a data analysis module: the data analysis module is used for analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system, and establishing a data structure test template and a data content test template by combining a data migration test range and a test strategy;
the data migration test case establishing module: the data migration test case establishing module is used for importing the established data structure test template and the data content test template into the data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case;
a data comparison module: the data comparison module is used for importing the data migration test case into the data migration test platform, after the data migration test platform receives a test instruction, the data migration test platform simultaneously inquires a source table, an intermediate table and a target table, and the data migration test case is used as a template for automatic comparison.
Preferably, the contents compared by the data comparison module include comparison of database table structures, comparison of data volumes and comparison of data contents, and the results of the test comparison are output in a unified manner.
Preferably, the data migration test platform established by the data scheduling module has a static query comparison test function and a dynamic migration verification function, the static data is tested through the static query comparison test function, and the dynamic data is tested and verified through the dynamic migration verification function.
Preferably, after the data comparison module automatically compares the data, a data comparison result report for checking can be derived, the data comparison result report is transmitted to a proofreader, and the proofreader compares the service content in the target system and the service content in the source system by spot-checking the correct content and the wrong content in the data comparison result report so as to verify the correctness of the migration data.
Preferably, when the data comparison module compares the data, the data migration test platform performs a large number of high-frequency query transactions on the data in the source system and the data in the target system, and compares the data results of the query transactions through the data comparison module.
Compared with the prior art, the invention has the beneficial effects that:
1. the data migration test method is used for uniformly scheduling corresponding data in a source system and a target system by establishing a universal data migration test platform, and comparing data in the source system and the target system according to a data structure test template and a data content test template by establishing a data structure test template and a data content test template, so that the data comparison efficiency of the source system and the template system is improved; and the checking result can be uniformly output for checking, so that the labor and time cost is saved.
2. The established data migration testing platform has the functions of static inquiry comparison testing and dynamic migration verification, static data are tested through the static inquiry comparison testing function, dynamic data are tested and verified through the dynamic migration verification function, updating of the dynamic data can be achieved, and the data can be used after migration.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flowchart illustrating a data migration testing method according to an embodiment of the present invention.
FIG. 2 is a block diagram of a data migration testing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
a data migration testing method, please refer to fig. 1.
The data migration testing method is used for testing the data migrated from the source system to the target system so as to verify the correctness, consistency and availability of the migrated data.
As shown in fig. 1, the data migration testing method includes:
step S100, establishing a universal data migration test platform, wherein the data migration test platform is used for uniformly scheduling corresponding data in the source system and the target system.
The data migration test platform in the step has universality, namely all data in a source system and all data in a target system can be tested, and the data migration test platform has static inquiry comparison test and dynamic migration verification functions, static data is tested through the static inquiry comparison test function, and dynamic data is tested and verified through the dynamic migration verification function. The dynamic data refers to data that changes with time, for example, in a financial management system used by an asset management system, a fund company, a bank, and the like, data related to interest changes with time, and thus the migrated data is the latest interest data.
For some scenarios of data migration of the managed system, due to the fact that the matching degree of the migration party is poor, the managed system is suitable for passing through a static query comparison test, the static query comparison test is used for carrying out data verification by comparing query results in a new system and an old system, the problem that business report data are correct and migration data are wrong can be found, and the correctness of data migration business verification is improved to a great extent.
Application example of live migration verification function:
for example, the data migration test method for performing data migration on a management system of a financial company such as a bank specifically includes the following operations: (1) loan settlement operation is carried out on loan products of the new and old systems through data migration T day (the same day), deposit accounting and account holder operation is carried out on deposit products of the new and old systems, and the loan and deposit inventory interest migration and account migration of the new and old systems are verified to be correct; (2) loan settlement operation is carried out on loan products of the new and old system by the day T + N (the Nth day after the current day), deposit accounting and account-cancelling operation is carried out on deposit products of the new and old system, so as to verify that interest settlement of the loan and deposit increment of the new and old system is correct, and verify the correctness of migration data and the correctness of an interest rule; (3) and carrying out batch settlement operation on new and old system loan-saving products through the settlement day so as to verify the correctness of settlement data and settlement rules of the new and old system loans and deposits. Through the layer-by-layer verification, the service continuity verification is carried out on the migration data which is verified to be correct, and the data after the migration can be confirmed to carry out subsequent transaction. The dynamic migration verification can solve the problem of interest verification before and after migration, and the interest migration result is fully verified by comparing the difference of interest before and after migration in the modes of batch interest settlement, batch account cancellation and batch acquisition in the new and old systems.
Step S200, a data structure test template and a data content test template are established by analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system and combining a data migration test range and a test strategy.
Here, the source table represents the data structure and data content of the source system, the intermediate table represents the data structure and data content for transition, and the target table represents the data structure and data content of the target system; since it is difficult for the source table to exactly correspond to the target table in the case where the difference in data structures between the source system and the target system is large, the intermediate table is set as the transition table.
The data structure is a way of storing and organizing data by a computer, and the source system and the target system may adopt different data structures to store the same data, so that a data structure test template is established to compare the database table structure and the data volume and test whether the data structures and the data volume of the source system and the target system have one-to-one correspondence; the established data content test template is used for comparing the data content of the source system and the data content of the target system and testing whether the data content of the target system is correct relative to the source system.
Step S300, importing the established data structure test template and the data content test template into a data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case.
In this step, the actual result data represents data in the source system, the expected result data represents data in the template system, the generated data migration test case is used for data testing of the source system and the target system as a case template, and the case is used as a template in the subsequent data testing process.
And S400, importing the data migration test case into a data migration test platform, after receiving the test instruction, the data migration test platform simultaneously inquires a source table, an intermediate table and a target table, and automatically comparing the data migration test case serving as a template.
In this step, the compared contents include the comparison of the database table structure, the comparison of the data volume and the comparison of the data content, and the result of the test comparison is uniformly output. In addition, during automatic comparison, the data migration test platform performs a large number of high-frequency query transactions on data in the source system and the target system, compares data results of the query transactions, and can effectively search data structures and data contents through the high-frequency query transactions, thereby ensuring the integrity and the test depth of data test.
And after automatic comparison, exporting a data comparison result report for checking, transmitting the data comparison result report to a proofreader, and comparing the service contents in the target system and the source system by the proofreader through spot checking of correct and wrong contents in the data comparison result report so as to verify the correctness of the migration data.
The proofreading personnel can be personnel who initially acquire data, for example, data related to a service is proofread by a service personnel, and the proofreading process belongs to secondary checking of correctness of the data, so that a sampling mode is adopted, and the effect of the data migration test platform on data after migration is mainly known.
Example 2:
a data migration testing device.
As shown in fig. 2, the data migration testing apparatus includes a data scheduling module, a data analysis module, a data migration testing case establishing module, and a data comparison module, where the data scheduling module, the data analysis module, the data migration testing case establishing module, and the data comparison module are stored in a computer readable medium and can be implemented in the form of a computer executable program. Wherein:
and a universal data migration test platform is established in the data scheduling module and used for uniformly scheduling corresponding data in the source system and the target system. The data migration test platform established by the data scheduling module has the functions of static query comparison test and dynamic migration verification, the static data is tested through the static query comparison test function, and the dynamic data is tested and verified through the dynamic migration verification function.
The data analysis module is used for analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system, and establishing a data structure test template and a data content test template by combining a data migration test range and a test strategy.
The data migration test case establishing module: the data migration test case establishing module is used for importing the established data structure test template and the data content test template into the data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case.
The data comparison module is used for importing the data migration test cases into the data migration test platform, after the data migration test platform receives the test instruction, the data migration test platform simultaneously inquires the source table, the intermediate table and the target table, and the data migration test cases are used as templates for automatic comparison. Specifically, when the data comparison module compares the data in the source system and the target system, the data migration test platform performs a large number of high-frequency query transactions on the data in the source system and the target system, compares data results of the query transactions through the data comparison module, compares contents compared by the data comparison module with the database table structure, the data volume and the data content, and uniformly outputs test comparison results; after the data comparison module automatically compares the data, a data comparison result report for checking can be derived, the data comparison result report is transmitted to a proofreader, and the proofreader compares the service content in the target system and the service content in the source system through spot checking of the correct content and the wrong content in the data comparison result report so as to verify the correctness of the migration data.
It should be noted that, in this document, terms such as "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. A data migration testing method is used for testing data migrated from a source system to a target system, and is characterized by comprising the following steps:
(1) establishing a universal data migration test platform, wherein the data migration test platform is used for uniformly scheduling corresponding data in a source system and a target system;
(2) establishing a data structure test template and a data content test template by analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system and combining a data migration test range and a test strategy;
(3) importing the established data structure test template and the data content test template into the data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case;
(4) and importing the data migration test case into the data migration test platform, after receiving a test instruction, the data migration test platform simultaneously inquires a source table, an intermediate table and a target table, and automatically comparing the data migration test case serving as a template.
2. The data migration testing method according to claim 1, characterized in that: in the step (4), the compared contents include the comparison of database table structures, the comparison of data volumes and the comparison of data contents, and the results of the test comparison are output uniformly.
3. The data migration testing method according to claim 1, characterized in that: the data migration testing platform has the functions of static inquiry comparison testing and dynamic migration verification, static data are tested through the static inquiry comparison testing function, and dynamic data are tested and verified through the dynamic migration verification function.
4. The data migration testing method according to claim 1, characterized in that: in the step (4), after the automatic comparison, a data comparison result report for checking is derived, the data comparison result report is transmitted to a proofreader, and the proofreader compares the service contents in the target system and the source system by spot-checking the contents of 'correct' and 'wrong' in the data comparison result report to verify the correctness of the migration data.
5. The data migration testing method according to claim 1, characterized in that: in the step (4), during automatic comparison, the data migration test platform performs a large number of high-frequency query transactions on data in the source system and the target system, and compares data results of the query transactions.
6. A data migration testing apparatus, comprising:
a data scheduling module: a universal data migration test platform is established in the data scheduling module, and the data migration test platform is used for uniformly scheduling corresponding data in the source system and the target system;
a data analysis module: the data analysis module is used for analyzing mapping documents from a source table of a source system to an intermediate table and from the intermediate table to a target table of a target system, and establishing a data structure test template and a data content test template by combining a data migration test range and a test strategy;
the data migration test case establishing module: the data migration test case establishing module is used for importing the established data structure test template and the data content test template into the data migration test platform, and filling expected result data and actual result data into the data migration test platform according to requirements to generate a data migration test case;
a data comparison module: the data comparison module is used for importing the data migration test case into the data migration test platform, after the data migration test platform receives a test instruction, the data migration test platform simultaneously inquires a source table, an intermediate table and a target table, and the data migration test case is used as a template for automatic comparison.
7. The data migration testing device of claim 6, wherein: the contents compared by the data comparison module comprise comparison of database table structures, comparison of data volumes and comparison of data contents, and the comparison results of tests are output in a unified mode.
8. The data migration testing device of claim 6, wherein: the data migration test platform established by the data scheduling module has the functions of static query comparison test and dynamic migration verification, the static data is tested through the static query comparison test function, and the dynamic data is tested and verified through the dynamic migration verification function.
9. The data migration testing device of claim 6, wherein: after the data comparison module automatically compares the data, a data comparison result report for checking can be derived, the data comparison result report is transmitted to a proofreader, and the proofreader compares the service content in the target system and the service content in the source system through randomly checking the correct content and the wrong content in the data comparison result report so as to verify the correctness of the migration data.
10. The data migration testing method according to claim 6, wherein: when the data comparison module compares the data, the data migration test platform carries out a large number of high-frequency query transactions on the data in the source system and the data in the target system, and the data comparison module compares the data results of the query transactions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210034083.0A CN114372450A (en) | 2022-01-12 | 2022-01-12 | Data migration testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210034083.0A CN114372450A (en) | 2022-01-12 | 2022-01-12 | Data migration testing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114372450A true CN114372450A (en) | 2022-04-19 |
Family
ID=81143831
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210034083.0A Pending CN114372450A (en) | 2022-01-12 | 2022-01-12 | Data migration testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114372450A (en) |
-
2022
- 2022-01-12 CN CN202210034083.0A patent/CN114372450A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105739968B (en) | Method and device for reviewing updated content based on distributed version control system Git | |
| CN110941630A (en) | Database operation and maintenance method, device and system | |
| EP4083897A1 (en) | Data management system, management method, and management program | |
| CN110188103A (en) | Data account checking method, device, equipment and storage medium | |
| US7607045B2 (en) | System and method for testing a modification to a process using a simulator | |
| CN110321339A (en) | A kind of data migration method, device, equipment and storage medium | |
| CN111199379A (en) | Examination and approval method, examination and approval device and storage medium of workflow engine | |
| CN109670943B (en) | Settlement system, settlement method, computer equipment and storage medium | |
| US20150378828A1 (en) | Test data management | |
| CN112256672B (en) | Database change approval method and device | |
| US20190236504A1 (en) | Technical field service inventory management | |
| CN113486358A (en) | Vulnerability detection method and device | |
| CN114372450A (en) | Data migration testing method and device | |
| CN117036062A (en) | Accounting resource calculation method and device | |
| CN114721943A (en) | Method and device for determining test range | |
| CN114780807A (en) | Service detection method, device, computer system and readable storage medium | |
| CN114037523A (en) | Letter service test method, device, equipment, medium and program product | |
| CN111338935A (en) | Method and system for joint debugging | |
| KR101737575B1 (en) | Method and device for verifying data based on sql sentences generated automatically | |
| CN110517010A (en) | A kind of data processing method, system and storage medium | |
| CN111966700B (en) | Information processing method, system, device and electronic equipment | |
| CN115543969B (en) | Data migration method, device, equipment and medium | |
| CN110032569B (en) | Method, device, equipment and system for checking changed data | |
| CN109767191B (en) | Data sharing method and device among registered users, server, equipment and medium | |
| CN118154131A (en) | Data processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |