CN105183852A - Database migration method and device - Google Patents
Database migration method and device Download PDFInfo
- Publication number
- CN105183852A CN105183852A CN201510566453.5A CN201510566453A CN105183852A CN 105183852 A CN105183852 A CN 105183852A CN 201510566453 A CN201510566453 A CN 201510566453A CN 105183852 A CN105183852 A CN 105183852A
- Authority
- CN
- China
- Prior art keywords
- database migration
- database
- connection request
- data
- service connection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a database migration method and device. The database migration method comprises the steps of locking a migrated data sheet before conducting the data exporting process of database migration; in the data exporting process, monitoring the locking duration of a database and the service connection request number of requested database operation; if it is monitored that the locking duration exceeds a set locking duration threshold value and/or the service connection request number exceeds a set service connection request number threshold value, stopping database migration operation, and unlocking the migrated data sheet. According to the database migration method and device, by monitoring two indexes, namely the locking duration and the service connection request number, the unlocking can be conducted without needing to wait till the data exporting process is ended, influence on database service processing is reduced, and better experience is brought to users. In addition, the embodiment of the invention further provides the database migration device.
Description
Technical field
Embodiments of the present invention relate to database migration technical field, and more specifically, embodiments of the present invention relate to a kind of database migration method and apparatus.
Background technology
This part embodiments of the present invention be intended to for stating in claims provide background or context.Description is not herein because be included in just admit it is prior art in this part.
In the use procedure of database, often need to carry out Data Migration, its reason may be server resource deficiency, Network Isolation, data disaster tolerance etc.On the other hand, along with the development of cloud computing technology, on database, cloud also becomes more and more general, and what first will do in the process is exactly to high in the clouds by database data migration.
Database migration refers to the database instance existed, by certain mode, export in file by list structure wherein and information such as table data etc., then recover a new database instance by certain mode, database migration includes the process of statistical conversion and data importing.
In database migration process, in order to ensure before and after database migration, by the consistance of tables of data of moving, needing to lock to by the tables of data of moving in database derivation process, avoiding database manipulation person modifies to it, deleting, increase etc.Lock period in tables of data, the operation needed mainly comprises:
1, snapshot is created to by the tables of data of moving, to ensure before and after database migration, consistent by the tables of data of moving;
2, resolve one by one by the tables of data of moving, the transfer parameter according to configuration will be split into many parts by the tables of data of moving and encapsulate;
3, non-transactional tables of data is derived, such as MyISAM (storage engines that MySQL database is primary).
It should be noted that, for transactional data table, carry out locking in above-mentioned steps 1,2, and for non-transactional tables of data, need to lock in above-mentioned steps 1,2,3.
In database migration process, usual step 1 is comparatively quick, but the time needed for step 2 affects by multiple factor, comprises the IO ability of database server, the quantity of tables of data, the record number of tables of data and fractionation granularity etc.From the practice production environment, to the large storehouse of hundreds of GB of similar blog applications, sometimes need the processing time of minute level.If there is larger non-transactional tables of data, step 3 also needs to expend many time.
Summary of the invention
For in the derivation process of database migration; to the reason cannot accurately estimated by the lock times of tables of data moved; in prior art, carry out database migration when business usually can be selected idle, but business absolute during cannot ensureing database migration is idle.
Therefore, in the prior art, to being locked by the tables of data of moving period in database table derivation process, cannot operate database table, terminate until database table derives process, but the service request number of connection that long lock times may cause requested database to operate explodes, and causes service request to be blocked, this is very bothersome process.
For this reason, be starved of a kind of database migration method of improvement, conservative control data library locking duration, reduces the impact on database business processing.
In the present context, embodiments of the present invention are expected to provide a kind of database migration method and apparatus.
In the first aspect of embodiment of the present invention, provide a kind of database migration method, comprising:
Cross Cheng Qian at the statistical conversion carrying out database migration, lock to by the tables of data of moving;
In described statistical conversion process, monitor lock duration and the service connection request quantity of asking to carry out database manipulation;
If locked described in monitoring, duration exceedes the duration threshold value and/or monitor the service connection request amount threshold that described service connection request quantity exceedes setting of locking of setting, then stop database migration operation and unlocked by the tables of data of moving described.
In the second aspect of embodiment of the present invention, provide a kind of database migration device, comprising:
Adding lock unit, for crossing Cheng Qian at the statistical conversion carrying out database migration, locking to by the tables of data of moving;
Monitoring unit, in described database derivation process, the service connection request quantity of database manipulation is carried out in lock duration and request of monitor database;
Control module, for the duration that locks described in monitoring at described monitoring unit exceed setting lock duration threshold value or monitor described service connection request quantity exceed the service connection request amount threshold of setting time, stop database migration operation also to be unlocked by the tables of data of moving described.
In the third aspect of embodiment of the present invention, provide a kind of database migration equipment, such as, storer and processor can be comprised, wherein, processor may be used for reading the program in storer, performs following process: cross Cheng Qian at the statistical conversion carrying out database migration, locks to by the tables of data of moving; In described statistical conversion process, monitor lock duration and the service connection request quantity of asking to carry out database manipulation; If locked described in monitoring, duration exceedes the duration threshold value and/or monitor the service connection request amount threshold that described service connection request quantity exceedes setting of locking of setting, then stop database migration operation and unlocked by the tables of data of moving described.
According to the database migration method and apparatus of embodiment of the present invention, to being locked by the tables of data of moving period in the statistical conversion process of database migration, monitor the service connection request quantity two indices of lock duration and requested database operation simultaneously, stop database migration operating and unlock to by the tables of data of moving when arbitrary index reaches the threshold value of its correspondence, and without the need to waiting for that statistical conversion process unlocks after terminating again, decrease the impact on database business processing; Simultaneously when the duration that locks exceedes predetermined threshold value, according to the concrete reason of time-out, adaptive adjustment affects the parameter of statistical conversion duration or the duration threshold value that locks, thus while reducing the impact on service request, decrease the number of times of the statistical conversion interruption caused because locking time-out, improve the efficiency of statistical conversion, for user brings better experience.
Accompanying drawing explanation
By reference to accompanying drawing reading detailed description hereafter, above-mentioned and other objects of exemplary embodiment of the invention, feature and advantage will become easy to understand.In the accompanying drawings, show some embodiments of the present invention by way of example, and not by way of limitation, wherein:
Fig. 1 schematically shows the application scenarios schematic diagram of the embodiment of the present invention;
Fig. 2 schematically shows the database migration embodiment of the method schematic diagram according to embodiment of the present invention;
Fig. 3 schematically shows the schematic flow sheet re-starting database migration operation according to database migration mode of operation according to further embodiment of this invention;
Fig. 4 data migration operation state schematically shown according to yet another embodiment of the invention re-executes the schematic flow sheet of database migration operation when being tables of data disassembled form;
Fig. 5 schematically shows database migration apparatus structure schematic diagram according to yet another embodiment of the invention;
Fig. 6 schematically shows the structural representation of database migration equipment according to yet another embodiment of the invention;
Fig. 7 schematically shows the program product schematic diagram for database migration according to yet another embodiment of the invention.
In the accompanying drawings, identical or corresponding label represents identical or corresponding part.
Embodiment
Below with reference to some illustrative embodiments, principle of the present invention and spirit are described.Should be appreciated that providing these embodiments is only used to enable those skilled in the art understand better and then realize the present invention, and not limit the scope of the invention by any way.On the contrary, provide these embodiments to be to make the disclosure more thorough and complete, and the scope of the present disclosure intactly can be conveyed to those skilled in the art.
Art technology technician know, embodiments of the present invention can be implemented as a kind of system, device, equipment, method or computer program.Therefore, the disclosure can be implemented as following form, that is: hardware, completely software (comprising firmware, resident software, microcode etc.) completely, or the form that hardware and software combines.
According to the embodiment of the present invention, a kind of database migration method and apparatus is proposed.
In this article, it is to be appreciated that in involved term:
Statistical conversion represents and the data in database is saved in backup file.
Data importing represents and the data in backup file is write in new database instance.
Service connection request refers to be initiated by the user of database, database is carried out to the operations such as data insertion, renewal, deletion and inquiry.
RPD (RowsPerDump) is each record number derived, and this value is estimated value, is realized by assigned indexes interval.
In addition, any number of elements in accompanying drawing is all unrestricted for example, and any name is all only for distinguishing, and does not have any limitation.
Below with reference to some representative embodiments of the present invention, explaination principle of the present invention and spirit in detail.
summary of the invention
The present inventor finds, to being locked by the tables of data of moving period in the statistical conversion process of database migration, owing to accurately cannot estimate lock times, if lock times arranges long, business may be caused to block.In order to conservative control lock times, reduce the impact on database business processing, in the embodiment of the present invention, the service connection request two indices arranging lock duration and requested database process is monitored, if monitor arbitrary index when reaching the threshold value of its correspondence, then immediately stopping database migration operation and unlock by the database table moved, avoiding causing business to block to being locked for a long time by the tables of data of moving.After database migration operation stops, the monitor control index reaching threshold value according to the duration that locks judges whether to re-execute database migration operation, if determine to need to re-execute database migration operation, then according to this database migration mode of operation, after adjustment correlation parameter, re-start database migration operation; In an embodiment of the present invention, when the duration that locks exceedes predetermined threshold value, according to the concrete reason of time-out, adaptive adjustment affects the parameter of statistical conversion duration or the duration threshold value that locks, thus while reducing the impact on service request, the number of times of the statistical conversion interruption caused because locking time-out can being reduced, improve the efficiency of statistical conversion.
After describing ultimate principle of the present invention, lower mask body introduces various non-limiting embodiment of the present invention.
application scenarios overview
First with reference to figure 1, if needed the database migration in server 11 in server 12, first statistical conversion operation is performed, list structure by database in server 11 exports in backup file with table data etc., by in the list structure of database in server 11 and table statistical conversion process, need to lock to by the tables of data of moving, to ensure by the consistance of tables of data before and after migration of moving; Then data import operation is performed, in the database of the write servers 12 such as the list structure in the backup file obtain statistical conversion process and table data, in tables of data importing process, without the need to being locked by the database table moved in server 11.
illustrative methods
Below in conjunction with the application scenarios of Fig. 1, be described with reference to Figure 2 the method for database migration according to exemplary embodiment of the invention.It should be noted that above-mentioned application scenarios is only that embodiments of the present invention are unrestricted in this regard for the ease of understanding spirit of the present invention and principle and illustrating.On the contrary, embodiments of the present invention can be applied to applicable any scene.
In the embodiment of the present invention, pre-set lock duration threshold value and the service connection request amount threshold of asking to carry out database manipulation, such as, time initial, the duration that locks can be set to 60s, and service connection request amount threshold can be set to 100.During concrete enforcement, can according to locked by parameter adjustments such as the tables of data quantity in migration data storehouse, tables of data sizes duration threshold value and service connection request amount threshold, the embodiment of the present invention does not limit this.
Based on lock duration threshold value and the service connection request amount threshold that arrange, in the embodiment of the present invention, database migration operation can be carried out according to flow process shown in Fig. 2:
S21, cross Cheng Qian at the statistical conversion carrying out database migration, lock to by the tables of data of moving.
S22, in statistical conversion process, the service connection request quantity of database manipulation is carried out in lock duration and request of monitoring.
Preferably, in statistical conversion process, lock duration and the service connection request quantity of asking to carry out database manipulation can be monitored in real time, also can monitor according to the monitoring period of setting lock duration and the service connection request quantity of asking to carry out database manipulation.
If S23 monitors the duration threshold value and/or monitor the service connection request amount threshold that service connection request quantity exceedes setting of locking that the duration that locks exceedes setting, then stop database migration operation and unlock by the tables of data of moving.
With the duration threshold value that locks arranged for 60s, service connection request amount threshold is 100 is example, in statistical conversion process, if monitor the threshold value that arbitrary index reaches its correspondence, namely, when monitoring the duration that locks and reaching 60s, the instant database migration that stops operates and unlocks by the tables of data of moving; Or, when monitoring service connection request quantity and reaching 100, also will immediately stop database migration operating and unlock by the tables of data of moving; Or when monitoring the duration that locks and arriving 60s, service connection request quantity also reaches 100, in this case, also will database migration be immediately stopped to operate and unlock by the tables of data of moving.
By above-mentioned steps, can the conservative control statistical conversion stage to by the lock times of migration data table, avoiding locks for a long time causes business to be blocked, and impacts data bank service.
After the operation of stopping database migration, judge whether to re-execute database migration operation according to the reason of database migration operation, concrete, can implement according to following either type:
The first embodiment
If determine to stop the reason of database migration operation to be the threshold value that service connection request quantity reaches that the threshold value of setting or lock duration and service connection request quantity reach setting according to step S23 simultaneously, then illustrate that present period is business processing busy section, in this case, reminder-data storehouse is moved unsuccessfully, failure cause can also be moved in reminder-data storehouse simultaneously, for reference.User according to prompting reselect the suitable time re-execute again database migration operation.
The second embodiment
If determine to stop the reason of database migration operation to be that the duration that locks reaches the duration threshold value that locks of setting according to step S23, when namely stopping database migration operation, service connection request quantity does not reach the threshold value of setting, in this case, can according to user arrange automatically re-execute database migration operation.Concrete, database migration operation can be re-started according to flow process as shown in Figure 3:
S31, acquisition database migration operation status information.
During concrete enforcement, stopping database migration operation after, if according to step S23 determine stop database migration operation reason for the duration that locks exceed threshold value time, then can obtain database migration operation status information from database migration daily record.
S32, according to obtain database migration operational status information, determine stop database migration operation time database migration mode of operation.
Wherein, database migration mode of operation can comprise database table fractured operation state and non-transactional tables of data derivation mode of operation.
The database migration mode of operation that S33, basis are determined, re-executes database migration operation.
Different according to the database migration mode of operation determined in step S32, the operation performed by step S33 is also different, introduces it respectively below.
If determine that database migration mode of operation when stopping database migration operation is database table fractured operation state according to step S32, namely the duration time-out that locks is operated by fractionation tables of data and causes, this explanation may cause because tables of data splits undersized, it is less that tables of data splits granularity, split by the dump_jobs quantity that goes out more, thus it is oversize to cause tables of data to split the time, causes carrying out dump_jobs thus and create time-out.
In database migration process, tables of data splits granularity by RPD (RowsPerDump, the record number of each derivation) state modulator, for the database instance of hundreds of GB, this parameter causes the overtime main cause that locks often, and this parameter is also a very important parameter in database migration simultaneously.RPD arranges meeting less than normal and causes the above-mentioned timeout issue that locks, but arranges bigger than normal, then can cause divide the dump_jobs number split out less, the data volume of each thread migration will become large, thus can bring impact to the parallel migration of follow-up data.For these reasons, in the invention process, can arrange RPD parameter value interval, user arranges acceptable RPD minimum value and maximal value when starting to move.
The situation of the time-out that causes locking less than normal is set for RPD, in the invention process, RPD can be increased and carry out retry, until reach the RPD maximal value of setting.The value of RPD parameter is linear with the dump_jobs number finally generated, and RPD doubles, then corresponding dump_jobs number reduces half, and then the time completed needed for dump_jobs establishment also can reduce accordingly.Generally lock times can be controlled locking in duration threshold value of arranging by which.
Based on foregoing description, can implement according to the step shown in Fig. 4 in step S33:
S41, judge arrange tables of data split granularity whether reach default fractionation upper size boundary value, if so, then perform step S42, if not, then perform step S43.
After S42, the duration threshold value that locks according to the adjustment of service connection request quantity, re-execute database migration operation, flow process terminates.
After S43, increase tables of data split granularity, re-execute database migration operation.
Wherein, in step S42, namely tables of data splits the maximal value that granularity has reached setting, still there is the duration time-out that locks, then can again judge according to service connection request quantity when stopping database migration operation further, concrete, service connection request quantity when stopping database migration operation and the ratio of service connection request amount threshold can be determined, if when this ratio is no more than default proportion threshold value, this illustrates that the number of services that request operates database is few, in this case, after then can increasing the duration threshold value that locks, re-execute database migration operation.Such as, above-mentioned proportion threshold value can be set to 50%, namely stops service connection request quantity during database migration operation to be no more than the half of service connection request amount threshold, then, after can increasing the duration threshold value that locks, re-execute database migration operation.
If determine in step S32 to stop database migration mode of operation during database migration operation to be that non-transactional tables of data derives mode of operation, namely the duration time-out that locks is operated by the derivation of expense transactional tables of data and causes, and this normally causes because non-transactional table is excessive.
In this case, in the embodiment of the present invention, after non-transactional tables of data can being changed into transactional data table, re-execute database migration operation.Because transactional data table only needs setting up in snapshot and tables of data splitting step in statistical conversion process to lock, and derive step without the need to locking to transactional data table in the non-transactional tables of data of statistical conversion, after non-transactional tables of data is converted into transactional data table, the duration time-out that locks caused because non-transactional tables of data derives operation can be avoided.
During concrete enforcement, also may there is following situation: when determining to stop database migration operation according to the database migration Operation Log obtained, database migration mode of operation is that database table fractured operation state and non-transactional tables of data derive mode of operation, namely when stopping database migration operation, database table fractured operation and non-transactional tables of data derive and operate parallel carrying out, in this case, because database table fractured operation is not over yet, therefore, when can determine in the embodiment of the present invention to stop database migration operation, database migration mode of operation is database table fractured operation state.
By repeating above-mentioned database migration operating process, can gradually by the duration adjusting thresholds that locks in statistical conversion process for rational value, duration threshold value of avoiding locking arranges the unreasonable business that causes and blocks, and reduces the impact on database business processing.
During concrete enforcement, in the embodiment of the present invention, user can also setting data storehouse transit time voluntarily, and such as, according to the feature of most of Internet service, be business idle period about 2:00 AM, user can be arranged on 2:00 AM and start to carry out database migration.
In the embodiment of the present invention, in the statistical conversion process of database migration, set up locking mechanisms with good conditionsi, judge whether to need to unlock by the tables of data of moving from lock times and service connection request quantity two dimensions, like this, can avoid to greatest extent causing data bank service to be affected in database migration process.On the other hand, also establish database migration retry mechanism, by dynamic adjusting data storehouse transfer parameter, as RPD and the duration threshold value etc. that locks, or large non-transactional tables of data is changed into the modes such as transactional data table to carry out the automatic retry of database migration with good conditionsi, improves database migration efficiency.Avoid user's frequent intervention after database migration failure, save the database migration time.
example devices
After the method describing exemplary embodiment of the invention, next, with reference to figure 5 pairs of exemplary embodiment of the invention, for the device of database migration.
As shown in Figure 5, be the structural representation of the data migration device that the embodiment of the present invention provides, can comprise:
Adding lock unit 51, for crossing Cheng Qian at the statistical conversion carrying out database migration, locking to by the tables of data of moving;
Monitoring unit 52, in database derivation process, the service connection request quantity of database manipulation is carried out in lock duration and request of monitor database;
Control module 53, for monitor at monitoring unit 52 duration that locks exceed setting lock duration threshold value or monitor service connection request quantity exceed the service connection request amount threshold of setting time, stop database migration operation also to unlock by the tables of data of moving.
During concrete enforcement, the database migration device that the embodiment of the present invention provides, can also comprise:
Acquiring unit, exceedes the duration threshold value that locks of setting for monitoring the duration that locks at monitoring unit 52, and after control module 53 stops database migration operating, obtains database migration operation status information;
Determining unit, for the database migration operational status information obtained according to acquiring unit, determines database migration mode of operation when stopping database migration operation;
Processing unit, for the database migration mode of operation determined according to determining unit, re-executes database migration operation.
Wherein, database migration mode of operation can comprise database table fractured operation state and non-transactional tables of data derivation mode of operation.
The database migration device that the embodiment of the present invention provides is also to comprise:
First judging unit, for judging that the tables of data arranged splits granularity and whether reaches default fractionation upper size boundary value;
First performance element, for when the judged result of the first judging unit is for being, after the duration threshold value that locks according to the adjustment of service connection request quantity, re-executes database migration operation; When the judged result of the first judging unit is no, after increasing described tables of data fractionation granularity, re-execute database migration operation.
The database migration device that the embodiment of the present invention provides is also to comprise:
Second judging unit, for judging whether the ratio of service connection request quantity and service connection request amount threshold exceedes preset ratio threshold value;
Second performance element, for when the judged result of the second judging unit is no, after increasing the duration threshold value that locks, re-executes database migration operation.
The database migration device that the embodiment of the present invention provides is also to comprise: the 3rd performance element, after non-transactional table is converted to transactional table, re-executes database migration operation.
After the method and apparatus describing exemplary embodiment of the invention, next, the device for database migration according to another illustrative embodiments of the present invention is introduced.
Person of ordinary skill in the field can understand, and various aspects of the present invention can be implemented as system, method or program product.Therefore, various aspects of the present invention can be implemented as following form, that is: hardware embodiment, completely Software Implementation (comprising firmware, microcode etc.) completely, or the embodiment that hardware and software aspect combines, " circuit ", " module " or " system " can be referred to as here.
In the embodiment that some are possible, can at least comprise at least one processing unit and at least one storage unit according to the equipment for database migration of the present invention.Wherein, described cell stores has program code, when described program code is performed by described processing unit, described processing unit is performed describe in this instructions above-mentioned " illustrative methods " part according to the various illustrative embodiments of the present invention for the step in the method for database migration.Such as, described processing unit can perform step S21 as shown in Figure 2, cross Cheng Qian at the statistical conversion carrying out database migration, lock to by the tables of data of moving, with step S22, in statistical conversion process, monitor lock duration and the service connection request quantity of asking to carry out database manipulation, if and step S23 monitors the duration threshold value and/or monitor the service connection request amount threshold that service connection request quantity exceedes setting of locking that the duration that locks exceedes setting, then stop database migration operation and unlock by the tables of data of moving.
Referring to Fig. 6, the device 60 for database migration according to this embodiment of the present invention is described.The device for database migration of Fig. 6 display is only an example, should not bring any restriction to the function of the embodiment of the present invention and usable range.
As shown in Figure 6, the device 60 for database migration can show with the form of universal computing device, and such as it can be server apparatus.Assembly for the device 60 of database migration can include but not limited to: the bus 63 of at least one processing unit 61 above-mentioned, at least one storage unit 62 above-mentioned, connection different system assembly (comprising storage unit 62 and processing unit 61).
Bus 63 represent in a few class bus structure one or more, comprise memory bus or Memory Controller, peripheral bus, processor or use any bus-structured local bus in multiple bus structure.
Storage unit 62 can comprise the computer-readable recording medium of volatile memory form, such as random access memory (RAM) 621 and/or cache memory 622, can further include ROM (read-only memory) (ROM) 623.
Storage unit 62 can also comprise the program/utility 625 with one group of (at least one) program module 624, such program module 624 includes but not limited to: operating system, one or more application program, other program module and routine data, may comprise the realization of network environment in each or certain combination in these examples.
Device 60 for database migration also can communicate with one or more external unit 64 (such as keyboard, sensing equipment etc.), also can with one or more make user can with this for the mutual devices communicating of the device 60 of database migration, and/or to communicate with any equipment (such as router, modulator-demodular unit etc.) making this device 60 for database migration can carry out communicating with other computing equipment one or more.This communication can be passed through I/O (I/O) interface 65 and carry out.Further, the device 60 for database migration can also such as, be communicated by network adapter 66 and one or more network (such as LAN (Local Area Network) (LAN), wide area network (WAN) and/or public network, the Internet).As shown in the figure, network adapter 66 is by bus 63 and other module communication for the device 60 of database migration.Be understood that, although not shown, other hardware and/or software module can be used in conjunction with the device 60 for database migration, include but not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and data backup storage system etc.
exemplary process product
In the embodiment that some are possible, various aspects of the present invention can also be embodied as a kind of form of program product, it comprises program code, when described program product runs on server apparatus, described program code be used for that described server apparatus is performed and describe in this instructions above-mentioned " illustrative methods " part according to the various illustrative embodiments of the present invention for the step in the method for database migration, such as, described server apparatus can perform step S21 as shown in Figure 2, Cheng Qian is crossed at the statistical conversion carrying out database migration, lock to by the tables of data of moving, with step S22, in statistical conversion process, monitor lock duration and the service connection request quantity of asking to carry out database manipulation, and step S23, if monitor the duration threshold value and/or monitor the service connection request amount threshold that service connection request quantity exceedes setting of locking that the duration that locks exceedes setting, then stop database migration operation and unlock by the tables of data of moving.
Described program product can adopt the combination in any of one or more computer-readable recording medium.Computer-readable recording medium can be readable signal medium or readable storage medium storing program for executing.Readable storage medium storing program for executing such as may be-but not limited to-the system of electricity, magnetic, optical, electrical magnetic, infrared ray or semiconductor, device or device, or combination above arbitrarily.The example more specifically (non exhaustive list) of readable storage medium storing program for executing comprises: the combination with the electrical connection of one or more wire, portable disc, hard disk, random access memory (RAM), ROM (read-only memory) (ROM), erasable type programmable read only memory (EPROM or flash memory), optical fiber, Portable, compact dish ROM (read-only memory) (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate.
As shown in Figure 7, describe the program product 70 for instant messaging application according to the embodiment of the present invention, it can adopt Portable, compact dish ROM (read-only memory) (CD-ROM) and comprise program code, and can run on server apparatus.But program product of the present invention is not limited thereto, in this document, readable storage medium storing program for executing can be any comprising or stored program tangible medium, and this program can be used by instruction execution system, device or device or be combined with it.
The data-signal that readable signal medium can comprise in a base band or propagate as a carrier wave part, wherein carries readable program code.The data-signal of this propagation can adopt various ways, comprises the combination of---but being not limited to---electromagnetic signal, light signal or above-mentioned any appropriate.Readable signal medium can also be any computer-readable recording medium beyond readable storage medium storing program for executing, and this computer-readable recording medium can send, propagates or transmit the program for being used by instruction execution system, device or device or be combined with it.
The program code that computer-readable recording medium comprises can with any suitable medium transmission, comprises that---but being not limited to---is wireless, wired, optical cable, RF etc., or the combination of above-mentioned any appropriate.
The program code operated for performing the present invention can be write with the combination in any of one or more programming languages, described programming language comprises object oriented program language-such as Java, C++ etc., also comprises conventional process type programming language-such as " C " language or similar programming language.Program code can fully perform on the user computing device, partly perform on a user device, as one, independently software package performs, partly part performs on a remote computing or performs in remote computing device or server completely on the user computing device.In the situation relating to remote computing device, remote computing device can by the network of any kind---comprise LAN (Local Area Network) (LAN) or wide area network (WAN)-be connected to user's computing equipment, or, external computing device (such as utilizing ISP to pass through Internet connection) can be connected to.
Although it should be noted that the some devices or sub-device that to be referred in above-detailed for the equipment of database migration, this division is only not enforceable.In fact, according to the embodiment of the present invention, the Characteristic and function of two or more devices above-described can be specialized in one apparatus.Otherwise, the Characteristic and function of an above-described device can Further Division for be specialized by multiple device.
In addition, although describe the operation of the inventive method in the accompanying drawings with particular order, this is not that requirement or hint must perform these operations according to this particular order, or must perform the result that all shown operation could realize expectation.Additionally or alternatively, some step can be omitted, multiple step be merged into a step and perform, and/or a step is decomposed into multiple step and perform.
Although describe spirit of the present invention and principle with reference to some embodiments, but should be appreciated that, the present invention is not limited to disclosed embodiment, can not combine to be benefited to the feature that the division of each side does not mean that in these aspects yet, this division is only the convenience in order to state.The present invention is intended to contain the interior included various amendment of spirit and scope and the equivalent arrangements of claims.
Claims (10)
1. a database migration method, comprising:
Cross Cheng Qian at the statistical conversion carrying out database migration, lock to by the tables of data of moving;
In described statistical conversion process, monitor lock duration and the service connection request quantity of asking to carry out database manipulation;
If locked described in monitoring, duration exceedes the duration threshold value and/or monitor the service connection request amount threshold that described service connection request quantity exceedes setting of locking of setting, then stop database migration operation and unlocked by the tables of data of moving described.
2. method according to claim 1, if the duration threshold value that locks monitoring that the duration that locks exceedes setting, after the described database migration operation of stopping, also comprising:
Obtain database migration operation status information;
According to the database migration operational status information obtained, determine database migration mode of operation when stopping described database migration to operate;
According to described database migration mode of operation, re-execute the operation of described database migration.
3. method according to claim 2, described database migration mode of operation comprises database table fractured operation state; And
According to described database migration mode of operation, re-execute the operation of described database migration, specifically comprise:
Judge that the tables of data arranged splits granularity and whether reaches default fractionation upper size boundary value;
If so, then after the duration threshold value that locks according to described service connection request quantity adjustment, the operation of described database migration is re-executed;
If not, then after increasing described tables of data fractionation granularity, the operation of described database migration is re-executed.
4. method according to claim 3, re-executes the operation of described database migration after the duration threshold value that locks, specifically comprise according to described service connection request quantity adjustment:
When determining that the ratio of described service connection request quantity and described service connection request amount threshold is no more than preset ratio threshold value, after the duration threshold value that locks described in increase, re-execute database migration operation.
5. method according to claim 2, described database migration mode of operation comprises non-transactional tables of data derives mode of operation; And
According to described database migration mode of operation, re-execute database migration operation, specifically comprise:
After described non-transactional table is converted to transactional table, re-execute database migration operation.
6. a database migration device, comprising:
Adding lock unit, for crossing Cheng Qian at the statistical conversion carrying out database migration, locking to by the tables of data of moving;
Monitoring unit, in described database derivation process, the service connection request quantity of database manipulation is carried out in lock duration and request of monitor database;
Control module, for the duration that locks described in monitoring at described monitoring unit exceed setting lock duration threshold value or monitor described service connection request quantity exceed the service connection request amount threshold of setting time, stop database migration operation also to be unlocked by the tables of data of moving described.
7. device according to claim 6, also comprises:
Acquiring unit, exceedes the duration threshold value that locks of setting for monitoring the duration that locks at described monitoring unit, and after described control module stops described database migration to operate, obtains database migration operation status information;
Determining unit, for the database migration operational status information obtained according to described acquiring unit, determines database migration mode of operation when stopping described database migration to operate;
Processing unit, for the database migration mode of operation determined according to described determining unit, re-executes the operation of described database migration.
8. device according to claim 7, described database migration mode of operation comprises database table fractured operation state; And
Described device, also comprises:
First judging unit, for judging that the tables of data arranged splits granularity and whether reaches default fractionation upper size boundary value;
First performance element, for when the judged result of described first judging unit is for being, after the duration threshold value that locks, re-executes the operation of described database migration according to described service connection request quantity adjustment; When the judged result of described first judging unit is no, after increasing described tables of data fractionation granularity, re-execute the operation of described database migration.
9. device according to claim 8, also comprises:
Second judging unit, for judging whether the ratio of described service connection request quantity and described service connection request amount threshold exceedes preset ratio threshold value;
Second performance element, for when the judged result of described second judging unit is no, after the duration threshold value that locks described in increase, re-executes the operation of described database migration.
10. device according to claim 7, described database migration mode of operation comprises non-transactional tables of data derives mode of operation; And
Described device, also comprises:
3rd performance element, after described non-transactional table is converted to transactional table, re-executes the operation of described database migration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510566453.5A CN105183852B (en) | 2015-09-08 | 2015-09-08 | A kind of database migration method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510566453.5A CN105183852B (en) | 2015-09-08 | 2015-09-08 | A kind of database migration method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105183852A true CN105183852A (en) | 2015-12-23 |
| CN105183852B CN105183852B (en) | 2018-05-22 |
Family
ID=54905933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510566453.5A Active CN105183852B (en) | 2015-09-08 | 2015-09-08 | A kind of database migration method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105183852B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107247639A (en) * | 2017-05-03 | 2017-10-13 | 上海动联信息技术股份有限公司 | A kind of efficient backup method of mysql databases |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107391634A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107391628A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Method of data synchronization and device |
| CN108021585A (en) * | 2016-10-28 | 2018-05-11 | 腾讯科技（深圳）有限公司 | Distributed data storage method and device |
| CN108573019A (en) * | 2018-01-23 | 2018-09-25 | 北京金山云网络技术有限公司 | A kind of data migration method, device, electronic equipment and readable storage medium storing program for executing |
| CN109801171A (en) * | 2018-12-14 | 2019-05-24 | 中国平安财产保险股份有限公司 | Non- renewal policy processing method, device, computer equipment and storage medium |
| CN111026734A (en) * | 2019-12-10 | 2020-04-17 | 北京北龙云海网络数据科技有限责任公司 | Database management method, system, device and storage medium |
| CN111125053A (en) * | 2019-10-31 | 2020-05-08 | 苏州浪潮智能科技有限公司 | Method and equipment for optimizing database writing rate |
| CN112463132A (en) * | 2020-11-13 | 2021-03-09 | 四川新网银行股份有限公司 | Database switching tool and switching method |
| CN112612792A (en) * | 2020-12-24 | 2021-04-06 | 中国联合网络通信集团有限公司 | Database management method and system |
| CN113656512A (en) * | 2021-10-21 | 2021-11-16 | 浙江太美医疗科技股份有限公司 | Data unlocking method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101861572A (en) * | 2007-11-19 | 2010-10-13 | 国际商业机器公司 | Technique of controlling access of database |
| US20120117030A1 (en) * | 2010-11-09 | 2012-05-10 | Ayako Sakuma | Database migration management apparatus and method thereof |
| CN103154901A (en) * | 2010-10-06 | 2013-06-12 | 国际商业机器公司 | Automated and self-adjusting data backup operations |
| CN103514164A (en) * | 2012-06-15 | 2014-01-15 | 阿里巴巴集团控股有限公司 | Data migration method and system |
| CN103530290A (en) * | 2012-07-03 | 2014-01-22 | 深圳市腾讯计算机***有限公司 | Method and system for data migration among databases |
-
2015
- 2015-09-08 CN CN201510566453.5A patent/CN105183852B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101861572A (en) * | 2007-11-19 | 2010-10-13 | 国际商业机器公司 | Technique of controlling access of database |
| CN103154901A (en) * | 2010-10-06 | 2013-06-12 | 国际商业机器公司 | Automated and self-adjusting data backup operations |
| US20120117030A1 (en) * | 2010-11-09 | 2012-05-10 | Ayako Sakuma | Database migration management apparatus and method thereof |
| CN103514164A (en) * | 2012-06-15 | 2014-01-15 | 阿里巴巴集团控股有限公司 | Data migration method and system |
| CN103530290A (en) * | 2012-07-03 | 2014-01-22 | 深圳市腾讯计算机***有限公司 | Method and system for data migration among databases |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021585A (en) * | 2016-10-28 | 2018-05-11 | 腾讯科技（深圳）有限公司 | Distributed data storage method and device |
| CN107247639A (en) * | 2017-05-03 | 2017-10-13 | 上海动联信息技术股份有限公司 | A kind of efficient backup method of mysql databases |
| CN107357883A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107391634A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data migration method and device |
| CN107391628A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Method of data synchronization and device |
| CN108573019B (en) * | 2018-01-23 | 2021-04-23 | 北京金山云网络技术有限公司 | Data migration method and device, electronic equipment and readable storage medium |
| CN108573019A (en) * | 2018-01-23 | 2018-09-25 | 北京金山云网络技术有限公司 | A kind of data migration method, device, electronic equipment and readable storage medium storing program for executing |
| CN109801171A (en) * | 2018-12-14 | 2019-05-24 | 中国平安财产保险股份有限公司 | Non- renewal policy processing method, device, computer equipment and storage medium |
| CN111125053A (en) * | 2019-10-31 | 2020-05-08 | 苏州浪潮智能科技有限公司 | Method and equipment for optimizing database writing rate |
| CN111026734A (en) * | 2019-12-10 | 2020-04-17 | 北京北龙云海网络数据科技有限责任公司 | Database management method, system, device and storage medium |
| CN111026734B (en) * | 2019-12-10 | 2020-11-13 | 北京北龙云海网络数据科技有限责任公司 | Database management method, system, device and storage medium |
| CN112463132A (en) * | 2020-11-13 | 2021-03-09 | 四川新网银行股份有限公司 | Database switching tool and switching method |
| CN112463132B (en) * | 2020-11-13 | 2023-06-06 | 四川新网银行股份有限公司 | Database switching tool and switching method |
| CN112612792A (en) * | 2020-12-24 | 2021-04-06 | 中国联合网络通信集团有限公司 | Database management method and system |
| CN112612792B (en) * | 2020-12-24 | 2023-05-30 | 中国联合网络通信集团有限公司 | Database management method and system |
| CN113656512A (en) * | 2021-10-21 | 2021-11-16 | 浙江太美医疗科技股份有限公司 | Data unlocking method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105183852B (en) | 2018-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105183852A (en) | Database migration method and device | |
| US9934105B2 (en) | Fault tolerance for complex distributed computing operations | |
| US10860447B2 (en) | Database cluster architecture based on dual port solid state disk | |
| CN101441582A (en) | Backup and recovery method of computer data object and system and program product for implementing the same | |
| EP3039534B1 (en) | Generating an idempotent workflow | |
| CN111984465A (en) | Database remote backup method, device, medium and electronic equipment | |
| US20210271583A1 (en) | Hyper-converged infrastructure (hci) log system | |
| CN110633046A (en) | Storage method and device of distributed system, storage equipment and storage medium | |
| US10241699B2 (en) | Method, apparatus or software for transferring a storage replication system between storage systems | |
| US10019182B2 (en) | Management system and management method of computer system | |
| WO2024131366A1 (en) | Cluster repair method and apparatus | |
| US9588998B2 (en) | Protecting storage data during system migration | |
| CN115827265A (en) | Transaction processing method, device, equipment and storage medium | |
| CN116974983A (en) | Data processing method, device, computer readable medium and electronic equipment | |
| WO2008039618A1 (en) | Persistent locks/resources for concurrency control | |
| CN103440325B (en) | Operation method of high-efficiency multi-concurrence and auto-adaptation database | |
| TW202305589A (en) | Storage device and operation method thereof | |
| US11940973B1 (en) | Limiting concurrent database updates | |
| US11430534B2 (en) | Method for testing storage systems, electronic device, and computer program product | |
| US10169136B2 (en) | Dynamic monitoring and problem resolution | |
| US20230161664A1 (en) | Method of responding to operation, electronic device, and storage medium | |
| KR101757249B1 (en) | Method and apparatus for processing query | |
| CN117312453A (en) | Online copying method, device and equipment for distributed data table and storage medium | |
| CN116450032A (en) | Disk creation method, data drop method and computing device | |
| CN118069168A (en) | Database deployment method, device and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Wen Zhenghu Inventor after: Guo Yi Inventor after: Jiang Chengyao Inventor after: Yu Lihua Inventor before: Wen Zhenghu Inventor before: Guo Yi |
|
| COR | Change of bibliographic data | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |