CN113254271A

CN113254271A - Data sequence recovery method, device, equipment and storage medium

Info

Publication number: CN113254271A
Application number: CN202110642309.0A
Authority: CN
Inventors: 李金龙; 王巍; 韩朱忠
Original assignee: Shanghai Dameng Database Co Ltd
Current assignee: Shanghai Dameng Database Co Ltd
Priority date: 2021-06-09
Filing date: 2021-06-09
Publication date: 2021-08-13
Anticipated expiration: 2041-06-09

Abstract

The application discloses a data sequence recovery method, a device, equipment and a storage medium, wherein the method comprises the following steps: periodically storing at least one sequence value in the memory into a disk according to preset time; wherein each sequence value corresponds to a physical log; and under the condition that the sequence value in the memory is lost, recovering the sequence value in the memory according to the physical log. Therefore, the modification of the current value of the sequence is recorded in the physical log, so that the current value of the sequence can be automatically recovered by replaying the physical log after the database fails.

Description

Data sequence recovery method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of data processing, and in particular, to a method, an apparatus, a device, and a storage medium for recovering a data sequence.

Background

A Sequence (also known as a Sequence generator) is a type of object in a database that multiple users can generate and use a set of non-repeating ordered integer values using a Sequence. Sequences simplify programming by providing unique numerical values in a sequence, such as may be used to automatically generate primary key values. The current mainstream database realizes sequences, such as Oracle, SQL Server and the like, and can control the generation mode of the sequences by configuring the attributes of the sequence, such as initial value, step length, maximum value, minimum value, whether to circulate and the like. When the current value of the sequence is known, the next value of the sequence can always be known unambiguously. For example, if the sequence _ demo is known, the configuration start value is 0, the step size is 2, the maximum value is 1000, and no loop is performed. Knowing that the current value of sequence _ demo is 100, it is clear that the subsequent sequence values are first 102, 104, 106. Currently, a simple way to maintain the sequence is to record the current value of the sequence in memory, while updating the current value with the latest value each time a new sequence value is obtained. However, during the database operation process, the database may crash due to unpredictable problems such as system defects, external failures, etc., and by using this sequence maintenance method, the current value of the sequence is lost after the database fails, and the uniqueness of the sequence value cannot be guaranteed.

Disclosure of Invention

The embodiment of the application provides a data sequence recovery method, a data sequence recovery device, data sequence recovery equipment and a storage medium, and aims to record modification of a current value of a sequence into a physical log, so that the current value of the sequence can be automatically recovered in a mode of replaying the physical log after a database fails.

In a first aspect, an embodiment of the present application provides a data sequence recovery method, where the method includes:

periodically storing at least one sequence value in the memory into a disk according to preset time;

wherein each sequence value corresponds to a physical log;

and under the condition that the sequence value in the memory is lost, recovering the sequence value in the memory according to the physical log.

In a second aspect, an embodiment of the present application further provides a data sequence recovery apparatus, where the apparatus includes:

the storage module is used for periodically storing at least one sequence value in the memory into a disk according to preset time;

wherein each sequence value corresponds to a physical log;

and the recovery module is used for recovering the sequence value in the memory according to the physical log under the condition that the sequence value in the memory is lost.

In a third aspect, an embodiment of the present application further provides a computer device, including: the data sequence recovery method comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and when the computer program is executed by the processor, the data sequence recovery method provided by any embodiment of the application is realized.

In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data sequence recovery method as provided in any embodiment of the present application.

The embodiment of the application provides a data sequence recovery method, a device, equipment and a storage medium, wherein the method comprises the following steps: periodically storing at least one sequence value in the memory into a disk according to preset time; wherein each sequence value corresponds to a physical log; and under the condition that the sequence value in the memory is lost, recovering the sequence value in the memory according to the physical log. Therefore, the modification of the current value of the sequence is recorded in the physical log, so that the current value of the sequence can be automatically recovered by replaying the physical log after the database fails.

Drawings

FIG. 1 is a diagram illustrating relationships among tablespaces, data files, segments, clusters, and pages in an embodiment of the present application;

FIG. 2 is a flow chart of a data sequence recovery method in an embodiment of the present application;

FIG. 3 is a flowchart of a method for determining a sequence value in a memory according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating loading a data page into a memory according to an embodiment of the present application;

FIG. 5 is a flowchart of a method for recovering a sequence value in a memory according to a physical log in an embodiment of the present application;

FIG. 6 is a flow chart of a method of creating a sequence in an embodiment of the present application;

fig. 7 is a schematic structural diagram of a data sequence recovery apparatus in an embodiment of the present application;

fig. 8 is a schematic structural diagram of another data sequence recovery apparatus in the embodiment of the present application;

fig. 9 is a schematic structural diagram of a computer device in an embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.

In addition, in the embodiments of the present application, the words "optionally" or "exemplarily" are used for indicating as examples, illustrations or explanations. Any embodiment or design described herein as "optionally" or "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "optionally" or "exemplarily" etc. is intended to present the relevant concepts in a concrete fashion.

In order to facilitate a clearer understanding of the solutions provided in the embodiments of the present application, related concepts related to the embodiments of the present application are further described herein, specifically as follows:

all data in the database is stored in a set of data files, and all the data files form the storage space of the database. In a da Dream (DM) database, in order to facilitate management of data, a storage space is divided into five levels of a table space, a data file, a segment, a cluster, and a page. As shown in fig. 1, a database is composed of one or more tablespaces, each tablespace is composed of one or more data files, and the tablespace in the database is uniquely identified by a tablespace Identification (ID). Each data file is composed of a plurality of continuous data pages on a disk, the data files in the table space are uniquely identified through file IDs, the data pages are the smallest distribution units in the database and are also the smallest Input/Output (I/O) units used in the database, and the data pages in the data files are uniquely identified through page numbers. Clusters are the upper logical units of data pages, each cluster consisting of 16 or 32 or 64 consecutive data pages in the same data file. Segments are upper level logical units of clusters, each segment consisting of one or more clusters, as shown in fig. 1, where the clusters may be clusters in different data files.

Fig. 2 is a flowchart of a data sequence recovery method provided in an embodiment of the present application, where the method may be applied to a database, as shown in fig. 2, the method may include, but is not limited to, the following steps:

s201, periodically storing at least one sequence value in the memory into a disk according to preset time.

The preset time can be a time length set artificially, that is, one or more sequence values in the memory are stored into the disk periodically, so that the performance loss of the disk I/O caused by frequent data reading and writing in the disk can be avoided. Further, in the embodiment of the present application, each sequence value stored in the memory has its corresponding physical log, and the physical log is used for recording the update operation on the sequence value and the current value of the sequence after the operation is modified.

Illustratively, the structure of the physical log may be:

DDWORD: (Table space ID, File ID, Page number, offset, number of bytes, sequence value)

Wherein "DDWORD" represents a physical log, "table space ID", "file ID", "page number" and "offset" are used to locate the storage space position of the sequence, "byte number" represents the byte number occupied by "sequence value", and "sequence value" represents the current value of the sequence updated by the current update operation.

S202, under the condition that the sequence value in the memory is lost, recovering the sequence value in the memory according to the physical log.

Because a sequence value in a memory for storing a data sequence may be lost due to various reasons such as internal failures of computer hardware, an operating system, or software, a physical log is generated before each operation on the sequence in the memory to record the current update operation and the current modification of the sequence. Therefore, when the database is abnormally closed and the sequence value in the memory is lost, the sequence value in the memory can be recovered in a mode of replaying the physical log.

The embodiment of the application provides a data sequence recovery method, which comprises the following steps: periodically storing at least one sequence value in the memory into a disk according to preset time; wherein each sequence value corresponds to a physical log; and under the condition that the sequence value in the memory is lost, recovering the sequence value in the memory according to the physical log. Therefore, the modification of the current value of the sequence is recorded in the physical log, so that the current value of the sequence can be automatically recovered by replaying the physical log after the database fails.

As shown in fig. 3, in an example, the implementation manner of determining the sequence value in the memory in step S201 may include, but is not limited to, the steps of:

s301, obtaining sequence metadata from a system dictionary.

Each sequence in the database comprises two parts of data, wherein one part is the current value of the sequence, the other part is the metadata of the sequence, which is recorded as META, the metadata is stored in a system dictionary, and the attribute configuration information of the sequence, such as the initial value, the step length, the maximum value, the minimum value and the like, is recorded.

S302, acquiring a sequence current value of the sequence from a data page in the memory.

Each sequence value of the sequence is stored in a memory space allocated for the sequence, the memory space is denoted as BLOCK, the BLOCK is located by a table space ID, a file ID, a page number and an offset in a page, the table space ID, the file ID, the page number and the offset in the page where the BLOCK is located are memory space position information, and the memory space position information can be recorded in metadata, that is, the metadata is used for recording two parts of contents of attribute configuration information and memory space position information.

Further, as shown in fig. 4, before the sequence current value of the sequence is obtained from the data page in the memory, the data page where the storage space is located may also be obtained from the data file of the disk according to the storage space location information in the metadata, and the data page is loaded into the memory.

And S303, calculating a new sequence value of the sequence according to the attribute configuration information in the metadata and the current sequence value of the sequence.

Since the step size of the sequence is recorded in the attribute configuration information, a new sequence value of the sequence can be calculated according to the step size and the current sequence value of the sequence.

Illustratively, the implementation process may include:

SQL>SELECT SEQUENCE_DEMO.NEXTVAL；

row number NEXTVAL

--------------------

1 102

And S304, storing the new sequence value into a data page of the memory, and taking the new sequence value as a current sequence value.

And after calculating to obtain a new sequence value based on the step length in the attribute configuration information, storing the new sequence value into a data page BLOCK of the memory, and taking the calculated new sequence value as a current sequence value to prepare for next calculation and update.

The above-described steps S302 to S304 are repeated within the range of the sequence value of the attribute configuration information, i.e., the range of the maximum value and the minimum value of the metadata record.

Further, after calculating a new sequence value of the sequence, an implementation manner is further provided in the embodiments of the present application, where a physical log corresponding to the new sequence value is generated according to the new sequence value, that is, the updated and modified sequence value is recorded in the physical log. This process of generating a physical log corresponding to the new sequence value may also be performed prior to storing the new sequence value in a data page of memory to ensure that every time a sequence in the BLOCK is updated, the associated update operation is recorded in the physical log. The physical log may be written to disk for recording immediately after the corresponding modification to BLOCK is completed.

As shown in fig. 5, in an example, in the step S202, an implementation manner of recovering a sequence value in a memory according to a physical log may include, but is not limited to, the following steps:

s501, determining a physical log to be recovered.

Since the sequence values stored in the memory are periodically stored in the disk, it may be determined that at least one sequence value in the memory is stored at the latest time point in the disk, and a time period after the latest time point has not reached the preset time of the periodic storage, which indicates that the sequence values operated and updated in the memory have not been stored in the disk, and then all the physical logs generated after the latest time point are determined as the physical logs to be recovered.

S502, analyzing the current physical log and acquiring the position information of the storage space in the record of the current physical log.

The current physical log in this step is any one of the physical logs to be restored determined in step S501.

S503, acquiring a data page where the storage space is located from the data file of the magnetic disk according to the storage space position information.

S504, loading the data page into the memory.

And S505, writing the sequence value recorded in the current physical log into a storage space of the data page, and determining the next physical log as the current physical log.

And repeating the steps S502 to S505 until all the physical logs to be recovered are recovered.

As shown in fig. 6, in an example, an implementation manner of creating a sequence provided by the embodiment of the present application further includes, but is not limited to, the following steps:

and S601, allocating a storage space for the sequence.

S602, loading the data page of the storage space into the memory.

And S603, initializing a storage space according to the initial value of the sequence, and generating a corresponding physical log.

And S604, writing the metadata of the sequence into a system dictionary.

Illustratively, the manner in which the sequence is created includes:

SQL>CREATE SEQUENCE SEQUENCE_DEMO START WITH 100INCREMENT BY 2MAXVALUE 1000；

where 100 is the start of the sequence, 2 is the step size, and 1000 is the maximum.

Fig. 7 is a schematic structural diagram of a data sequence recovery apparatus according to an embodiment of the present application, and as shown in fig. 7, the apparatus may include: a storage module 701 and a recovery module 702;

the storage module is used for periodically storing at least one sequence value in the memory into a disk according to preset time, wherein each sequence value corresponds to a physical log;

In an example, as shown in fig. 8, the apparatus may further include a determining module 703;

the determining module is used for executing the following processes to determine the sequence value in the memory, and comprises the following steps:

the method comprises the following steps: acquiring metadata of the sequence from a system dictionary;

step two: acquiring a sequence current value of a sequence from a data page in a memory;

step three: calculating a new sequence value of the sequence according to the attribute configuration information in the metadata and the current sequence value of the sequence;

step four: storing the new sequence value into a data page of the memory, and taking the new sequence value as a current sequence value;

and repeating the second step to the fourth step within the range of the sequence value of the attribute configuration information.

Further, as shown in fig. 8, the apparatus may further include a loading module 704;

and the loading module is used for acquiring a data page where the storage space is located from the data file of the disk according to the storage space position information in the metadata and loading the data page into the memory.

Optionally, as shown in fig. 8, the apparatus may further include a generating module 705;

and the generating module is used for generating a physical log corresponding to the new sequence value according to the new sequence value.

In one example, the recovery module includes a determining unit, an analyzing unit, an obtaining unit, and a storing unit;

the determining unit is used for determining a physical log to be recovered;

the analysis unit is used for analyzing the current physical log;

the acquisition unit is used for acquiring the storage space position information in the current physical log record; acquiring a data page where the storage space is located from a data file of the disk according to the storage space position information;

the storage unit is used for loading the data page into the memory; writing the sequence value recorded in the current physical log into the storage space of the data page;

and the determining unit is further used for determining the next physical log as the current physical log.

Further, the determining unit is configured to determine a latest time point at which at least one sequence value in the memory is stored in the disk, and determine all physical logs generated after the latest time point as physical logs to be recovered.

As shown in fig. 8, in an example, the apparatus may further include an initialization module 706, where the initialization module includes an allocation unit, a loading unit, a generation unit, and a storage unit;

the allocation unit is used for allocating storage space for the sequence;

the loading unit is used for loading the data page where the storage space is located into the memory;

the generating unit is used for initializing the storage space according to the initial value of the sequence and generating a corresponding physical log;

and the storage unit is used for writing the metadata of the sequence into the system dictionary.

The data sequence recovery device can execute the data sequence recovery method provided by the figures 1-6, and has corresponding devices and beneficial effects in the method.

Fig. 9 is a schematic structural diagram of a computer apparatus according to an embodiment of the present invention, as shown in fig. 9, the computer apparatus includes a controller 801, a memory 802, an input device 803, and an output device 804; the number of the controllers 801 in the computer device may be one or more, and one controller 801 is illustrated in fig. 9; the controller 801, the memory 802, the input device 803, and the output device 804 may be connected by a bus or other means, and the bus connection is exemplified in fig. 9.

The memory 802 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules (e.g., the storage module 701 and the recovery module 702 in the data sequence recovery apparatus) corresponding to the data sequence recovery method in the embodiments of fig. 1-6. The controller 801 executes various functional applications of the computer device and data processing by running software programs, instructions, and modules stored in the memory 802, that is, implements the data sequence recovery method described above.

The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 802 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 802 may further include memory located remotely from the controller 801, which may be connected to a terminal/server through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input unit 803 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus. The output device 804 may include a display device such as a display screen.

Embodiments of the present application also provide a storage medium containing computer-executable instructions for performing a data sequence recovery method when executed by a computer controller, the method including the steps shown in fig. 1-6.

From the above description of the embodiments, it is obvious for those skilled in the art that the present application can be implemented by software and necessary general hardware, and certainly can be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods described in the embodiments of the present application.

It should be noted that the modules included in the data sequence recovery apparatus are only divided according to functional logic, but are not limited to the above division manner as long as the corresponding functions can be implemented, and are not used to limit the scope of the present application.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for recovering a data sequence, comprising:

wherein each sequence value corresponds to a physical log;

2. The method of claim 1, wherein determining a sequence value in memory comprises:

step two: acquiring a sequence current value of the sequence from a data page in a memory;

step three: calculating a new sequence value of the sequence according to attribute configuration information in the metadata and the current sequence value of the sequence;

step four: storing the new sequence value into a data page of the memory, and taking the new sequence value as the current sequence value;

3. The method of claim 2, wherein prior to obtaining the sequence current value of the sequence from the data page in memory, the method further comprises:

acquiring a data page where a storage space is located from a data file of the disk according to the storage space position information in the metadata;

and loading the data page into the memory.

4. The method of claim 2, wherein after calculating the new sequence value for the sequence, the method further comprises:

and generating a physical log corresponding to the new sequence value according to the new sequence value.

5. The method of claim 1, wherein recovering the in-memory sequence value from the physical log comprises:

the method comprises the following steps: determining a physical log to be recovered;

step two: analyzing a current physical log to obtain storage space position information in a record of the current physical log;

step three: acquiring a data page where a storage space is located from a data file of the magnetic disk according to the storage space position information;

step four: loading the data page into the memory;

step five: writing the sequence value recorded in the current physical log into the storage space of the data page, and determining the next physical log as the current physical log;

and repeating the second step to the fifth step until all the physical logs to be recovered are recovered.

6. The method of claim 5, wherein determining the physical log to be recovered comprises:

determining a most recent point in time at which at least one sequence value in memory was stored to disk:

and determining all physical logs generated after the latest time point as physical logs to be recovered.

7. The method of claim 1, wherein prior to periodically storing the at least one sequence value in the memory to the disk at the predetermined time, the method further comprises:

allocating storage space for the sequence;

loading the data page where the storage space is located into a memory;

initializing the storage space according to the initial value of the sequence and generating a corresponding physical log;

writing the metadata of the sequence to a system dictionary.

8. A data sequence recovery apparatus, comprising:

wherein each sequence value corresponds to a physical log;

9. A computer device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements a data sequence recovery method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a data sequence recovery method according to any one of claims 1 to 7.