CN104899117A - Memory database parallel logging method for nonvolatile memory - Google Patents
Memory database parallel logging method for nonvolatile memory Download PDFInfo
- Publication number
- CN104899117A CN104899117A CN201510336836.3A CN201510336836A CN104899117A CN 104899117 A CN104899117 A CN 104899117A CN 201510336836 A CN201510336836 A CN 201510336836A CN 104899117 A CN104899117 A CN 104899117A
- Authority
- CN
- China
- Prior art keywords
- data
- nonvolatile memory
- log
- page
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a memory database parallel logging method for a nonvolatile memory and belongs to memory database parallel logging methods. The memory database parallel logging method comprises constantly modifying data in the process of operation of a memory database through a transaction, wherein the data can be distributed on different pages; recording data page IDs (Identity) for the modification operation of the data; performing decomposition on the logging information according to the page IDs to be written into a buffer area in the nonvolatile memory of the page IDs; still maintaining the data of the nonvolatile memory under the condition that the power failure is produced in the nonvolatile memory so as to ensure the recoverability of the memory database; submitting the transaction once the logging information arrives in the nonvolatile memory and namely that the persistence of the logging information is implemented; performing parallel execution on the log buffering disk writing operation of the pages by a multithreading technology due to the fact that the log buffering disk writing operation of the pages is independent of each other. According to the memory database parallel logging method for the nonvolatile memory, log records are organized according to the data pages through the associative law and the commutative law of differential logs to be temporarily cached in the nonvolatile memory and accordingly the recoverability of the transaction is ensured.
Description
Technical field
The present invention relates to a kind of memory database parallel logging method, particularly a kind of memory database parallel logging method towards Nonvolatile memory.
Background technology
At present, memory database, by data memory-resident, obtains the performance of some data volume faster than disk database.Power-off or software error cause hardware delay machine time, the content in internal memory can disappear.In order to support the restorability of memory database, all modifications history of affairs to data, in the mode of log recording, must be recorded on disk.
The content of log recording comprises value (being called front picture, Before Image) before data modification and the later value (being called after image, After Image) of data modification.Such as when affairs data X from 5, be modified as 7, then 10 are modified as again, so must record two log recordings, be the position of log recording 1=<X data respectively, 5,7>, the position of log recording 2=<X data, 7,10>.After memory database failure, a upper checkpoint (Checkpoint), internal memory can be loaded into, then the daily record since a upper checkpoint be re-applied in data, can date restoring to up-to-date state.The initial value of the data X in such as checkpoint loads, and be 5, then log recording 1, and log recording 2 is applied on data value, is modified as 75, is then modified as 10 7, and data can return to last state.
Database also can from initial state, and namely database is empty situation, the daily record since initial conditions has been replayed one time, has returned to last state.The setting of checkpoint every now and then the last state of database is preserved a complete image, during recovery, and the daily record since only need playing checkpoint last time, quickening rejuvenation.
The problem of prior art and shortcoming: the log approach of log recording being carried out to front picture and after image record, its shortcoming is that daily record application must perform in serial, such as in a upper example, action 1 (being modified as 7 5) and action 2 (being modified as 10 7), must apply in strict accordance with the order of log, carry out log scan and application serially, limit the efficiency of recovery of MMDB.
On the other hand, due to the serial of log recording application, each page of data (page), before the application of all log recordings is complete, can not ensure that it returns to last state, in so whole rejuvenation, all page of data are all disabled.
Summary of the invention
The object of the invention is to provide a kind of memory database parallel logging method towards Nonvolatile memory, solve the problem limiting recovery of MMDB efficiency in the database recovery process of prior art.
The object of the present invention is achieved like this: this memory database parallel logging method, specific as follows:
In memory database operational process, affairs are Update Table constantly, and these data may be distributed on the different pages; For the retouching operation of each data, record page of data ID, the XOR value of picture and after image before the positional information of data and side-play amount and difference daily record namely;
Then these log informations are decomposed according to page ID, write the buffer zone be subordinate in the Nonvolatile memory of page ID; Nonvolatile memory, when power down, still keeps its data, so ensure that the restorability of memory database; Once log information arrives at Nonvolatile memory, namely log information achieves persistence, and affairs can have submitted;
For each page prepares the buffer zone of two Nonvolatile memories; One, for constantly receiving log information, ensures the submission of affairs; Another one is full buffering, and for writing disk, after log recording transfers to disk, this buffering is standby; When the Nonvolatile memory buffering of certain page is full time, these log informations are transferred to disk, namely write disk; If now there is this page of transactions modify, generate new log information, these log informations write another one buffer zone, one, two buffer zones for receiving the daily record of current active affairs, one for shifting daily record to disk, both are used alternatingly;
The log buffer writing disk manipulation of each page, not Existence dependency relationship, by multithreading executed in parallel.
Beneficial effect, owing to have employed such scheme, based on picture before data and after image, calculates difference value, as log information; The application compound law of commutation of difference value and law of association, so daily record can carry out arbitrarily grouping and out of order application.Utilize this feature, adopt parallel logging recording method, utilize multichannel I/O to realize parallel logging record; When recovering in units of page of data, after applying the log information of this page, the page just reaches upstate, can accept the access of new affairs.Solve the problem limiting recovery of MMDB efficiency in the database recovery process of prior art, reach object of the present invention.
Advantage compared with prior art:
(1) utilize difference daily record to meet the feature of law of association and law of commutation, log recording is organized according to page of data, temporary cache, in Nonvolatile memory, ensures the restorability of affairs.When Nonvolatile memory is full time, daily record is dumped in disk.The log recording of each page, can be written in parallel to disk, accelerates affairs and submits process to.
(2) in rejuvenation, the recovery of each page of data is independently, after the checkpoint of the page is loaded, the daily record of the page is loaded and is applied in data, this page can return to last state, the new affairs accessing this page can continue to run, without the need to waiting for that all log informations are disposed.
Accompanying drawing illustrates:
Fig. 1 is law of association and the law of commutation figure of difference daily record of the present invention.
Fig. 2 is the memory database parallel logging method frame figure towards Nonvolatile memory of the present invention.
Embodiment
This memory database parallel logging method, specific as follows:
In memory database operational process, affairs are Update Table constantly, and these data may be distributed on the different pages; For the retouching operation of each data, record page of data ID, the XOR value of picture and after image before the positional information of data and side-play amount and difference daily record namely;
Then these log informations are decomposed according to page ID, write the buffer zone be subordinate in the Nonvolatile memory of page ID; Nonvolatile memory, when power down, still keeps its data, so ensure that the restorability of memory database; Once log information arrives at Nonvolatile memory, namely log information achieves persistence, and affairs can have submitted;
For each page prepares the buffer zone of two Nonvolatile memories; One, for constantly receiving log information, ensures the submission of affairs; Another one is full buffering, and for writing disk, after log recording transfers to disk, this buffering is standby; When the Nonvolatile memory buffering of certain page is full time, these log informations are transferred to disk, namely write disk; If now there is this page of transactions modify, generate new log information, these log informations write another one buffer zone, one, two buffer zones for receiving the daily record of current active affairs, one for shifting daily record to disk, both are used alternatingly;
The log buffer writing disk manipulation of each page, not Existence dependency relationship, by multithreading executed in parallel.
Embodiment 1: parallel logging method of the present invention depends on the feature of difference daily record, illustrates the concrete calculating of this difference daily record below by an example and meets the feature of law of commutation, law of association.Method is as follows:
The initial value of data X is 5, and scale-of-two is 0101, through first revision, becomes 7, and binary value is 0111; And then through second revision, become 10, binary value is 1010; The front picture that first time changes is 0101, and after image is 0111, and so both XOR values are 0010; The front picture that second time changes is 0111, and after image is 1010, and so both XOR values are 1101; Two log recordings are the positional information of log recording 1=< data X respectively, 0010>, the positional information of log recording 2=< data X, 1101>;
The truth table of xor operation is, 0 XOR 0=0,1 XOR 1=0,0 XOR 1=1,1 XOR 0=1;
Difference daily record meets law of association, two combining log records, namely first carries out xor operation 0010 and 1101, obtain 1111, then in the initial value being applied to data 5, obtain binary value 1010, this numerical value is just in time the last look 10 i.e. binary value 1010 of data;
Difference daily record compound law of commutation, log recording according to < log recording 1, the order of log recording 2> is applied in data, or according to < log recording 2, the order of log recording 1> is applied in data, result is the same, the i.e. 0101 i.e. decimal system 5, first XOR 0010, then XOR 1101, obtain the 1010 i.e. decimal systems 10, the 0101 i.e. first XOR 1101 of the decimal system 5, XOR 0010 again, also obtains the 1010 i.e. decimal systems 10, see Fig. 2.
Embodiment 2: embodiment
(1) for Data Update, the XOR value of picture and after image before calculating.
(2) in conjunction with the positional information of Update Table, form log recording, shape as < data positional information, the XOR value > of front picture and after image.
(3) daily record is divided into groups according to page ID, is recorded to Nonvolatile memory.
(4) when the log information of certain affairs all writes Nonvolatile memory, affairs can be submitted to.
(5), time full for the daily record Nonvolatile memory buffering of certain page, log information buffer zone is whole is written into disk.The log buffer of each page can write different disk units by multichannel I/O.
(6) while daily record writes disk from Nonvolatile memory buffering, if there are affairs access and revise this page, so log information is written into another one Nonvolatile memory buffering.We cushion for each page has prepared two Nonvolatile memories, write disk for one, and one accepts new log information, is used alternatingly.
Claims (1)
1. towards a memory database parallel logging method for Nonvolatile memory, it is characterized in that: towards the parallel logging method based on differential technique of Nonvolatile memory, specific as follows:
In memory database operational process, affairs are Update Table constantly, and these data may be distributed on the different pages; For the retouching operation of each data, record page of data ID, the XOR value of picture and after image before the positional information of data and side-play amount and difference daily record namely;
Then these log informations are decomposed according to page ID, write the buffer zone be subordinate in the Nonvolatile memory of page ID; Nonvolatile memory, when power down, still keeps its data, so ensure that the restorability of memory database; Once log information arrives at Nonvolatile memory, namely log information achieves persistence, and affairs can have submitted;
For each page prepares the buffer zone of two Nonvolatile memories; One, for constantly receiving log information, ensures the submission of affairs; Another one is full buffering, and for writing disk, after log recording transfers to disk, this buffering is standby; When the Nonvolatile memory buffering of certain page is full time, these log informations are transferred to disk, namely write disk; If now there is this page of transactions modify, generate new log information, these log informations write another one buffer zone, one, two buffer zones for receiving the daily record of current active affairs, one for shifting daily record to disk, both are used alternatingly;
The log buffer writing disk manipulation of each page, not Existence dependency relationship, by multithreading executed in parallel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510336836.3A CN104899117B (en) | 2015-06-17 | 2015-06-17 | Memory database parallel logging method towards Nonvolatile memory |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510336836.3A CN104899117B (en) | 2015-06-17 | 2015-06-17 | Memory database parallel logging method towards Nonvolatile memory |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104899117A true CN104899117A (en) | 2015-09-09 |
| CN104899117B CN104899117B (en) | 2019-04-16 |
Family
ID=54031794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510336836.3A Active CN104899117B (en) | 2015-06-17 | 2015-06-17 | Memory database parallel logging method towards Nonvolatile memory |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104899117B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117308A (en) * | 2015-09-29 | 2015-12-02 | 联想(北京)有限公司 | Data processing method, device and system |
| CN105677511A (en) * | 2015-12-30 | 2016-06-15 | 首都师范大学 | Data writing method and apparatus for reducing synchronization overheads |
| CN110019506A (en) * | 2017-09-21 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Log recording treating method and apparatus |
| CN110597663A (en) * | 2019-09-16 | 2019-12-20 | 阿里巴巴集团控股有限公司 | Transaction processing method and device |
| CN111324665A (en) * | 2020-01-23 | 2020-06-23 | 阿里巴巴集团控股有限公司 | Log playback method and device |
| CN112162833A (en) * | 2020-09-24 | 2021-01-01 | 苏宁云计算有限公司 | Transaction log processing method, device and system |
| CN113296683A (en) * | 2020-04-07 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data storage method, device, server and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7093157B2 (en) * | 2004-06-17 | 2006-08-15 | International Business Machines Corporation | Method and system for autonomic protection against data strip loss |
| CN102521269A (en) * | 2011-11-22 | 2012-06-27 | 清华大学 | Index-based computer continuous data protection method |
| US20130013602A1 (en) * | 2011-07-06 | 2013-01-10 | International Business Machines Corporation | Database system |
| CN103092903A (en) * | 2011-11-07 | 2013-05-08 | Sap股份公司 | Database Log Parallelization |
| CN103309767A (en) * | 2012-03-08 | 2013-09-18 | 阿里巴巴集团控股有限公司 | Method and device for processing client log |
-
2015
- 2015-06-17 CN CN201510336836.3A patent/CN104899117B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7093157B2 (en) * | 2004-06-17 | 2006-08-15 | International Business Machines Corporation | Method and system for autonomic protection against data strip loss |
| US20130013602A1 (en) * | 2011-07-06 | 2013-01-10 | International Business Machines Corporation | Database system |
| CN103092903A (en) * | 2011-11-07 | 2013-05-08 | Sap股份公司 | Database Log Parallelization |
| CN102521269A (en) * | 2011-11-22 | 2012-06-27 | 清华大学 | Index-based computer continuous data protection method |
| CN103309767A (en) * | 2012-03-08 | 2013-09-18 | 阿里巴巴集团控股有限公司 | Method and device for processing client log |
Non-Patent Citations (2)
| Title |
|---|
| QIN XIONGPAI: ""A Parallel Recovery Scheme for Update Intensive Main Memory Database Systems"", 《2008 NINTH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING, APPLICATIONS AND TECHNOLOGIES》 * |
| XIONGPAI QIN: ""Simulation of main memory database parallel recovery"", 《SPRINGSIM "09 PROCEEDINGS OF THE 2009 SPRING SIMULATION MULTICONFERENCE》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117308A (en) * | 2015-09-29 | 2015-12-02 | 联想(北京)有限公司 | Data processing method, device and system |
| CN105117308B (en) * | 2015-09-29 | 2020-06-23 | 联想(北京)有限公司 | Data processing method, device and system |
| CN105677511A (en) * | 2015-12-30 | 2016-06-15 | 首都师范大学 | Data writing method and apparatus for reducing synchronization overheads |
| CN105677511B (en) * | 2015-12-30 | 2018-08-17 | 首都师范大学 | A kind of method for writing data and device reducing synchronization overhead |
| CN110019506A (en) * | 2017-09-21 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Log recording treating method and apparatus |
| CN110019506B (en) * | 2017-09-21 | 2023-03-31 | 阿里云计算有限公司 | Log record processing method and device |
| CN110597663A (en) * | 2019-09-16 | 2019-12-20 | 阿里巴巴集团控股有限公司 | Transaction processing method and device |
| CN111324665A (en) * | 2020-01-23 | 2020-06-23 | 阿里巴巴集团控股有限公司 | Log playback method and device |
| CN111324665B (en) * | 2020-01-23 | 2023-06-27 | 阿里巴巴集团控股有限公司 | Log playback method and device |
| CN113296683A (en) * | 2020-04-07 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data storage method, device, server and storage medium |
| CN112162833A (en) * | 2020-09-24 | 2021-01-01 | 苏宁云计算有限公司 | Transaction log processing method, device and system |
| CN112162833B (en) * | 2020-09-24 | 2022-11-15 | 苏宁云计算有限公司 | Transaction log processing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104899117B (en) | 2019-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104899117A (en) | Memory database parallel logging method for nonvolatile memory | |
| US9087008B1 (en) | Replicating a volume using snapshots | |
| US9411821B1 (en) | Block-based backups for sub-file modifications | |
| US9916211B2 (en) | Relational database recovery | |
| US10970173B2 (en) | Logging process in a data storage system | |
| US9792345B1 (en) | Parallel database mirroring | |
| EP3441890A1 (en) | Transaction recovery method in database system and database management system | |
| US10884876B2 (en) | Systems and methods for performing a database backup for repairless restore | |
| US10275481B2 (en) | Updating of in-memory synopsis metadata for inserts in database table | |
| CN107665219B (en) | Log management method and device | |
| CN109086425B (en) | Data processing method and device for database | |
| US9542279B2 (en) | Shadow paging based log segment directory | |
| CN108763508B (en) | Data page access method, storage engines and computer readable storage medium | |
| CN101364166B (en) | Method and device for simulating Nand flash of 2048 byte page into hard disk | |
| CN102541691B (en) | Log check point recovery method applied to memory data base OLTP (online transaction processing) | |
| CN109690522B (en) | Data updating method and device based on B+ tree index and storage device | |
| CN107391544A (en) | Processing method, device, equipment and the computer storage media of column data storage | |
| CN110597663A (en) | Transaction processing method and device | |
| US11409798B2 (en) | Graph processing system including different kinds of memory devices, and operation method thereof | |
| US8015375B1 (en) | Methods, systems, and computer program products for parallel processing and saving tracking information for multiple write requests in a data replication environment including multiple storage devices | |
| CN110019063A (en) | Method, terminal device and the storage medium of calculate node data disaster tolerance playback | |
| US11301210B2 (en) | Merging multiple sorted lists in a distributed computing system | |
| CN112764662B (en) | Method, apparatus and computer program product for storage management | |
| CN105808451A (en) | Data caching method and related apparatus | |
| US11010091B2 (en) | Multi-tier storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |