CN117591013A - Request processing method, device, equipment and readable storage medium - Google Patents
Request processing method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN117591013A CN117591013A CN202311541223.4A CN202311541223A CN117591013A CN 117591013 A CN117591013 A CN 117591013A CN 202311541223 A CN202311541223 A CN 202311541223A CN 117591013 A CN117591013 A CN 117591013A
- Authority
- CN
- China
- Prior art keywords
- locking
- bitmap
- target file
- request
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 19
- 238000012545 processing Methods 0.000 claims abstract description 98
- 238000000034 method Methods 0.000 claims abstract description 43
- 230000004048 modification Effects 0.000 claims description 21
- 238000012986 modification Methods 0.000 claims description 21
- 230000007717 exclusion Effects 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 4
- 230000010076 replication Effects 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 5
- 230000008859 change Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, equipment and a readable storage medium for processing a request, which belongs to the technical field of storage, and comprises the following steps: after receiving an IO request for a target file, acquiring a bitmap identifier of the target file from a local copy bitmap; determining a locking requirement and a locking application under the condition of the locking requirement by using the bitmap identification and the request type of the IO request; applying for locking to a main node of the target file by using a locking application; IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined; after IO processing is completed, the local copy bitmap is updated and the target file is unlocked. The technical effects of this application: the IO of the local replication volume can be efficiently and correctly processed by each node, multi-node backup is ensured, and the failure of one node of the storage system does not affect the local replication backup data.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for processing a request.
Background
The local copy function in the storage system is to make a data backup in the storage system. When the local copy is created, an empty local copy backup volume (target volume) and a bitmap are created, when the source volume is written, data is first copied to the target volume, and the bitmap is modified to be 1, so that the complete local copy backup data can be ensured to exist all the time, and whether the data is on the source volume or the target volume is determined according to the result of reading the bitmap.
After a user creates a local copy of a volume on a storage system, IOs may be processed based on the locally copied bitmap. If IO can be processed only at one node, the efficiency of the system is greatly reduced, the redundancy of multi-node backup is lost, and when the node is lost, the backup data is invalid; if the IO is processed on each node, and the correct processing strategy is not adopted, the bitmap on each node is possibly inconsistent, or when different nodes simultaneously receive the conflicting IO, the IO is processed in parallel by errors, and the local copy backup is also invalid.
In summary, how to effectively and accurately perform the IO processing is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The purpose of the application is to provide a request processing method, a device, equipment and a readable storage medium, which ensure that each node can efficiently and correctly process IO of a local replication volume, ensure multi-node backup, and ensure that the failure of one node of a storage system does not affect local replication backup data.
In order to solve the technical problems, the application provides the following technical scheme:
a request processing method, comprising:
after receiving an IO request for a target file, acquiring a bitmap identifier of the target file from a local copy bitmap;
determining a locking requirement and a locking application under the condition of the locking requirement by utilizing the bitmap identification and the request type of the IO request;
applying for locking to the main node of the target file by using the locking application;
IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined;
and after IO processing is completed, updating the local copy bitmap and unlocking the target file.
Preferably, applying for locking to the master node of the target file by using the locking application includes:
if the node is the master node of the target file, adding the locking application into a locking application queue, and sequentially processing the locking application in the locking application queue;
and if the node is an auxiliary node of the target file, sending the locking application to a main node of the target file.
Preferably, the processing of the locking applications in the locking application queue sequentially includes:
if the target file is unlocked, reading a first locking application from the lock application queue in sequence;
if the first locking application is a shared lock, after the shared lock is allocated, reading a second locking application from the lock application queue in sequence;
if the second locking application is a sharing lock, continuing to distribute the sharing lock;
if the second locking application is a mutual exclusion lock, after the target file is unlocked, distributing the mutual exclusion lock;
and after the mutual exclusion lock is unlocked, reading a third locking application from the lock application queue in sequence.
Preferably, determining the locking requirement and the locking application under the condition of the locking requirement by using the bitmap identifier and the request type of the IO request includes:
if the request type is a source volume read request, determining that locking is not needed;
if the request type is a target volume read request and the bitmap identification is true, determining that locking is not needed;
if the request type is a target volume read request and the bitmap mark is false, determining that locking is needed and the locking application is an application sharing lock;
if the request type is a source volume write request and the bitmap identification is true, determining that locking is not needed;
if the request type is a source volume write request and the bitmap mark is false, determining that locking is needed, wherein the locking application is application mutex;
if the request type is a target volume write request and the bitmap mark is false, determining that locking is needed, wherein the locking application is application mutex;
if the request type is a target volume write request and the bitmap identification is true, determining that locking is not needed;
the target volume is a local copy backup volume for performing local copy on the target file; the bitmap is identified as true corresponding to the target file having been copied to the target volume, and the bitmap is identified as false corresponding to the target file not having been copied to the target volume.
Preferably, the performing IO processing on the target file in the case of obtaining a lock or determining that locking is not required includes:
after the lock is acquired, acquiring the bitmap identification of the target file from the local copy bitmap again;
and carrying out IO processing on the target file according to the bitmap identification obtained at the time.
Preferably, the performing IO processing on the target file according to the bitmap identifier obtained this time includes:
if the request type is a target volume read request and the read bitmap mark is true, directly reading the target file from the target volume;
and if the request type is a target volume read request and the read bitmap mark is false, reading the target file from the source volume corresponding to the target file in a redirection mode.
Preferably, updating the local copy bitmap and unlocking the target file includes:
if the node is a master node of the target file and the target file is copied to a target volume in the IO processing process, modifying the bitmap identification in the local copy bitmap, sending a bitmap modification message to auxiliary nodes of the target file, and unlocking after receiving bitmap modification success messages fed back by all the auxiliary nodes;
if the node is an auxiliary node of the target file and the target file is copied to the target volume in the IO processing process, a bitmap modification request is sent to a main node of the target file, after a bitmap modification message sent by the main node of the target file is received, the bitmap identification in the local copy bitmap is synchronously modified, and a bitmap modification success message is fed back to the main node of the target file so that the main node is unlocked.
A request processing apparatus comprising:
the bitmap identification reading module is used for acquiring the bitmap identification of the target file from the local copy bitmap after receiving the IO request aiming at the target file;
the lock demand analysis module is used for determining a locking demand and a locking application under the condition of the locking demand by utilizing the bitmap identification and the request type of the IO request;
the locking application module is used for applying for locking to the main node of the target file by utilizing the locking application;
the request processing module is used for carrying out IO processing on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined;
and the unlocking processing module is used for updating the local copy bitmap and unlocking the target file after the IO processing is completed.
An electronic device, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the request processing method when executing the computer program.
A readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the request processing method described above.
After receiving an IO request for a target file, the method provided by the embodiment of the application acquires a bitmap identifier of the target file from a local copy bitmap; determining a locking requirement and a locking application under the condition of the locking requirement by using the bitmap identification and the request type of the IO request; applying for locking to a main node of the target file by using a locking application; IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined; after IO processing is completed, the local copy bitmap is updated and the target file is unlocked.
In the application, after receiving an IO request for a target file, a bitmap identifier of the target file is first obtained. Then, a locking requirement is determined based on the bitmap identification and the request type of the IO request, and a locking application is correspondingly determined under the condition that the locking requirement is definitely existed. And under the condition that locking is needed, applying for locking to the main node of the target file by using a locking application. Thus, IO processing is performed on the target file in the event that a lock is obtained or it is determined that locking is not required. After IO processing is completed, the local copy bitmap is updated, and the target file is unlocked. It can be seen that in this application, the file itself is taken as a unit, and there is a corresponding master node to perform lock management, and the locking requirement and the specific locking application are determined by the bitmap identifier and the request type. Therefore, a plurality of nodes can be enabled to perform IO processing, and the exclusive operation can be effectively prevented from being executed in parallel by mistake based on the lock. The technical effects of this application: the IO of the local replication volume can be efficiently and correctly processed by each node, multi-node backup is ensured, and the failure of one node of the storage system does not affect the local replication backup data.
Accordingly, the embodiments of the present application further provide a request processing device, a device, and a readable storage medium corresponding to the above request processing method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for a person having ordinary skill in the art.
FIG. 1 is a flowchart illustrating a request processing method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a request processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a request processing device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to provide a better understanding of the present application, those skilled in the art will now make further details of the present application with reference to the drawings and detailed description. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Referring to fig. 1 and fig. 2, fig. 1 is a flowchart of a request processing method in an embodiment of the present application; fig. 2 is a specific flow chart of a request processing method in an embodiment of the present application, where the method may be applied to the primary node and the secondary node shown in fig. 2, and the method includes the following steps:
s101, after receiving an IO request for a target file, obtaining a bitmap identifier of the target file from a local copy bitmap.
The target file may be any file in the storage system, and the file may be a file that has been copied into the target volume, or may be a file that has not been copied into the target volume.
The IO request may be embodied as a read request or a write request, which may include in particular a source volume read request or a target volume read request, and which may include in particular a source volume write request or a target volume write request.
The bitmap of each file needing to be locally copied is recorded in the local copy bitmap, wherein the bitmap is true (1) to indicate that the file is copied to the target volume, and the bitmap is false (0) to indicate that the file is not copied to the target volume.
In the application, after receiving the IO request of the target file, the bitmap identifier of the target file may be read from the local copy bitmap. The bitmap identification may be true or false.
S102, determining locking requirements and locking applications under the condition of the locking requirements by using the bitmap identification and the request type of the IO request.
Because the target file has different storage states in the source volume and the target volume, different request conditions of IO requests may need to carry out operations such as content rewriting or state change on the target file, so that mutually exclusive IOs need to be ensured to be unable to be carried out simultaneously. For example, two nodes cannot write to one target file at the same time. Therefore, in the embodiment of the present application, it is required to firstly make an obvious whether the processing of the present IO request needs to be locked based on the bitmap identifier and the request type of the IO request, and in the case that the locking is required, it is required to make a specific locking application clear.
Specifically, different judging conditions of locking requirements and locking application conditions of different locks can be preset. For example, in the case that read IO needs to be locked, a shared lock is added; in the case that the write IO needs to be locked, the exclusive lock is added. The shared lock and the shared lock can acquire the lock according to a formula, but the mutual exclusion lock and the mutual exclusion lock cannot acquire the lock at the same time.
In one specific embodiment of the present application, determining a locking requirement by using a bitmap identifier and a request type of an IO request, and applying for locking in a case of the locking requirement includes:
if the request type is a source volume read request, determining that locking is not needed;
if the request type is a target volume read request and the bitmap mark is true, determining that locking is not needed;
if the request type is a target volume read request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for sharing the lock;
if the request type is a source volume write request and the bitmap identification is true, determining that locking is not needed;
if the request type is a source volume write request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for mutual exclusion locking;
if the request type is a target volume write request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for mutual exclusion locking;
if the request type is a target volume write request and the bitmap mark is true, determining that locking is not needed;
the target volume is a local copy backup volume for locally copying the target file; the bitmap identifies that the true corresponding target file has been copied to the target volume, and the bitmap identifies that the false corresponding target file has not been copied to the target volume.
For convenience of description, the above steps are described in combination.
There are two types of read IOs, a read IO of the source volume and a read IO of the target volume. For read IO of the source volume, as data is necessarily existed in the source volume, mutual exclusivity with other IO exists, and locking is not needed. For the read IO of the target volume, the bitmap of the node needs to be read first, if the bitmap is 1, the representative data is copied to the target volume, and locking is not needed at this time, so that the read IO is directly issued; if the bitmap is 0, it represents that redirection to the source volume for reading may be required, that a write IO (trigger data copy) with another source volume may collide, and that a locking process is required.
The write IO is also divided into two types, namely, a write IO of the source volume and a write IO of the target volume. For write IO of a source volume, the bitmap of the node needs to be read first, if the bitmap is 0, the data is possibly required to be copied from the source volume to a target volume first, and locking is also required; if 1, the representative data is copied to the target volume, and IO is directly issued without locking. For the write IO of the target volume, the bitmap of the node needs to be read first, and if the bitmap is 0, the bitmap needs to be set to 1 so as to indicate that data copy is not needed later and locking is needed; if the bitmap is 1, no locking is required.
The shared lock is added to the read IO application, and the exclusive lock is added to the write IO application. The shared lock and the shared lock can acquire the lock at the same time, because the IO of the redirection reading has no conflict and does not need mutual exclusion processing; redirecting read and write IOs requires exclusive processing.
S103, applying for locking to the main node of the target file by using the locking application.
After the locking is definitely needed and the locking application is determined, the main node of the target file can be applied for locking.
In the application, a master node can be determined for each file, and other nodes are all auxiliary nodes. When the master node is selected for the file, the method can be performed in a random selection mode, and the node for writing the file for the first time can be selected as the master node preferentially.
When the node is the main node of the target file, the locking application is directly processed; otherwise, the locking application is sent to the auxiliary node of the target file.
That is, the locking process needs to be completed in the master node, and if the process is the IO of the slave node, a message needs to be sent to the master node to apply for locking.
In a specific embodiment of the present application, applying for locking to a master node of a target file using a locking application includes:
if the node is the master node of the target file, adding a locking application into a locking application queue, and sequentially processing the locking application in the locking application queue;
and if the node is an auxiliary node of the target file, sending a locking application to a main node of the target file.
The method for sequentially processing the locking application in the locking application queue comprises the following steps:
if the target file is unlocked, reading a first locking application from a locking application queue in sequence;
if the first locking application is a shared lock, after the shared lock is allocated, reading a second locking application from a lock application queue in sequence;
if the second locking application is the sharing lock, continuing to distribute the sharing lock;
if the second locking application is a mutual exclusion lock, after the target file is unlocked, distributing the mutual exclusion lock;
and after the mutual exclusion lock is unlocked, reading a third locking application from the lock application queue in sequence.
That is, if the node is the master node of the target file, the locking application may be directly added to the lock application queue. If the node is an auxiliary node of the target file, the locking application needs to be sent to a main node of the target file. Therefore, the locking application processing is all responsible for the master node, confusion is avoided, and each node can be guaranteed to perform IO processing.
The master node processes the locking application, and can determine the lock allocation according to the lock allocation condition of the current target file and the specific condition of each locking application in the lock application queue. In particular, a shared lock may be allocated simultaneously, but a exclusive lock may be allocated to only one node. For example, for when the target file is in an unlocked state at this time, the earliest incoming one, i.e., the first locking application, may be read from the lock application queue. The allocation may be made whether the first locking application is a shared lock or a exclusive lock. Under the condition that the first locking application is the sharing lock, the second locking application can be read, under the condition that the second locking application is the sharing lock, the sharing lock can still be continuously allocated, and under the condition that the second locking application file is the mutual exclusion lock, the sharing lock can be continuously allocated after waiting for the unlocking of the target file. According to the allocation rule, the master node can enable mutually exclusive IO to be processed successively, not in parallel, and can effectively avoid the error parallel execution of the mutually exclusive IO.
That is, all the IOs needing locking process apply for locking at the main node, and the main node grants the locks according to the order of applying the locks, so that each IO is ensured to be sequentially processed according to the locking order.
S104, carrying out IO processing on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined.
When the node obtains the lock or confirms that the IO request does not need to be locked, IO processing can be carried out on the target file. I.e. to read or write the target file.
In a specific embodiment of the present application, it is considered that during a period when the bit icon is acquired for the first time and the lock is acquired, possibly other nodes may perform IO processing on the target file, which may cause a change in bitmap identifier, where the change in bitmap identifier may affect processing logic of the IO. Thus, in the present application, in the case of obtaining a lock or determining that locking is not required, performing IO processing on the target file includes:
after the lock is acquired, acquiring the bitmap identification of the target file from the local copy bitmap again;
and carrying out IO processing on the target file according to the bitmap identification obtained at the time.
That is, after the lock is obtained, processing is required according to the processing logic corresponding to the new bitmap identification.
The IO processing of the target file is carried out according to the bitmap identification obtained at the time, and the IO processing comprises the following steps:
if the request type is a target volume read request and the read bitmap mark is true, directly reading the target file from the target volume;
if the request type is a target volume read request and the read bitmap mark is false, the target file is read from the source volume corresponding to the redirected target file.
After the IO acquires the lock, the bitmap needs to be read again at the master node, because the bitmap identification may have been modified in the period from the time of reading the bitmap, sending the message to apply for the lock until the lock is acquired, and the logic of IO processing is changed again.
Illustrating: for the same data block, there is a read IO to the target volume, if the read bitmap is 0 before locking is applied, the read is required to be redirected to the source volume, but at the same time, there is a source volume write IO issued by different nodes, the lock is applied for first, IO processing is performed after the lock is obtained, and the data is copied from the source volume to the target volume. At this time, the read IO does not need to be read again, if the bitmap is not read again, the read still executes the read again, the error data can be read, and after the bitmap is read again to be 1, the target volume can be read directly.
After obtaining the lock and the latest IO processing logic, if the lock is the IO of the main node, IO processing is directly carried out; if the IO is the IO of the auxiliary node, the main node needs to send a message back to the auxiliary node after obtaining the lock, and the auxiliary node completes IO processing. For write IOs of source volumes, IO processing may include copying of data, and for other types of IOs, IO processing is not performed.
S105, after IO processing is completed, updating the local copy bitmap and unlocking the target file.
After the IO processing is completed, the local copy bitmap can also be updated in consideration of the change of the copy condition of the target file. And unlocking the target file after updating is completed or after determining that the local copy bitmap is the latest bitmap.
In one specific embodiment of the present application, updating the local copy bitmap and unlocking the target file includes:
if the node is a master node of the target file and the target file is copied to the target volume in the IO processing process, modifying the bitmap identification in the local copy bitmap, sending a bitmap modification message to auxiliary nodes of the target file, and unlocking after receiving bitmap modification success messages fed back by all the auxiliary nodes;
if the node is an auxiliary node of the target file and the target file is copied to the target volume in the IO processing process, a bitmap modification request is sent to a master node of the target file, after a bitmap modification message sent by the master node of the target file is received, bitmap identification in the local copy bitmap is synchronously modified, and a bitmap modification success message is fed back to the master node of the target file so as to facilitate unlocking of the master node.
For convenience of description, the above steps are described in combination.
After IO processing is completed, the bitmap is required to be written into 1 for the source volume write IO and the target volume write IO with the bitmap of 0. For redirected reads, no bitmap is written. If the IO is from the auxiliary node, a message is sent to the main node again, and the IO is processed; if the IO comes from the master node, processing is complete at the present node.
The request for writing the bitmap is initiated from the master node, the master node sends a message to all the auxiliary nodes to request the bitmap to be written, the auxiliary nodes inform the master node after writing the bitmap, and the master node unlocks and locks the next IO. And after unlocking, the IO is processed by the local replication module.
After the IO is processed, snapshot volume IO processing can be performed, and for how to perform snapshot volume IO processing, reference may be made to snapshot technology, which is not described in detail herein.
After receiving an IO request for a target file, the method provided by the embodiment of the application acquires a bitmap identifier of the target file from a local copy bitmap; determining a locking requirement and a locking application under the condition of the locking requirement by using the bitmap identification and the request type of the IO request; applying for locking to a main node of the target file by using a locking application; IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined; after IO processing is completed, the local copy bitmap is updated and the target file is unlocked.
In the application, after receiving an IO request for a target file, a bitmap identifier of the target file is first obtained. Then, a locking requirement is determined based on the bitmap identification and the request type of the IO request, and a locking application is correspondingly determined under the condition that the locking requirement is definitely existed. And under the condition that locking is needed, applying for locking to the main node of the target file by using a locking application. Thus, IO processing is performed on the target file in the event that a lock is obtained or it is determined that locking is not required. After IO processing is completed, the local copy bitmap is updated, and the target file is unlocked. It can be seen that in this application, the file itself is taken as a unit, and there is a corresponding master node to perform lock management, and the locking requirement and the specific locking application are determined by the bitmap identifier and the request type. Therefore, a plurality of nodes can be enabled to perform IO processing, and the exclusive operation can be effectively prevented from being executed in parallel by mistake based on the lock. The technical effects of this application: the IO of the local replication volume can be efficiently and correctly processed by each node, multi-node backup is ensured, and the failure of one node of the storage system does not affect the local replication backup data.
Corresponding to the above method embodiments, the embodiments of the present application further provide a request processing device, where the request processing device described below and the request processing method described above may be referred to correspondingly.
Referring to fig. 3, the apparatus includes the following modules:
the bitmap identifier reading module 101 is configured to obtain a bitmap identifier of a target file from a local copy bitmap after receiving an IO request for the target file;
the lock demand analysis module 102 is configured to determine a lock demand and a lock application under the condition of the lock demand by using the bitmap identifier and a request type of the IO request;
a locking application module 103, configured to apply for locking to a master node of the target file by using a locking application;
a request processing module 104, configured to perform IO processing on the target file if a lock is obtained or it is determined that locking is not required;
and the unlocking processing module 105 is used for updating the local copy bitmap and unlocking the target file after the IO processing is completed.
After receiving an IO request for a target file, the device provided by the embodiment of the application acquires a bitmap identifier of the target file from a local copy bitmap; determining a locking requirement and a locking application under the condition of the locking requirement by using the bitmap identification and the request type of the IO request; applying for locking to a main node of the target file by using a locking application; IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined; after IO processing is completed, the local copy bitmap is updated and the target file is unlocked.
In the application, after receiving an IO request for a target file, a bitmap identifier of the target file is first obtained. Then, a locking requirement is determined based on the bitmap identification and the request type of the IO request, and a locking application is correspondingly determined under the condition that the locking requirement is definitely existed. And under the condition that locking is needed, applying for locking to the main node of the target file by using a locking application. Thus, IO processing is performed on the target file in the event that a lock is obtained or it is determined that locking is not required. After IO processing is completed, the local copy bitmap is updated, and the target file is unlocked. It can be seen that in this application, the file itself is taken as a unit, and there is a corresponding master node to perform lock management, and the locking requirement and the specific locking application are determined by the bitmap identifier and the request type. Therefore, a plurality of nodes can be enabled to perform IO processing, and the exclusive operation can be effectively prevented from being executed in parallel by mistake based on the lock. The technical effects of this application: the IO of the local replication volume can be efficiently and correctly processed by each node, multi-node backup is ensured, and the failure of one node of the storage system does not affect the local replication backup data.
In a specific embodiment of the present application, the locking application module is specifically configured to add a locking application to a locking application queue if the node is a master node of a target file, and sequentially process the locking applications in the locking application queue;
and if the node is an auxiliary node of the target file, sending a locking application to a main node of the target file.
In one specific embodiment of the present application, the locking application module is specifically configured to sequentially read a first locking application from the locking application queue if the target file is unlocked;
if the first locking application is a shared lock, after the shared lock is allocated, reading a second locking application from a lock application queue in sequence;
if the second locking application is the sharing lock, continuing to distribute the sharing lock;
if the second locking application is a mutual exclusion lock, after the target file is unlocked, distributing the mutual exclusion lock;
and after the mutual exclusion lock is unlocked, reading a third locking application from the lock application queue in sequence.
In one embodiment of the present application, the lock requirement analysis module is specifically configured to determine that locking is not required if the request type is a source volume read request;
if the request type is a target volume read request and the bitmap mark is true, determining that locking is not needed;
if the request type is a target volume read request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for sharing the lock;
if the request type is a source volume write request and the bitmap identification is true, determining that locking is not needed;
if the request type is a source volume write request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for mutual exclusion locking;
if the request type is a target volume write request and the bitmap mark is false, determining that locking is needed, and applying for locking is applying for mutual exclusion locking;
if the request type is a target volume write request and the bitmap mark is true, determining that locking is not needed;
the target volume is a local copy backup volume for locally copying the target file; the bitmap identifies that the true corresponding target file has been copied to the target volume, and the bitmap identifies that the false corresponding target file has not been copied to the target volume.
In one specific embodiment of the present application, the request processing module is specifically configured to acquire, after acquiring the lock, a bitmap identifier of the target file from the local copy bitmap again;
and carrying out IO processing on the target file according to the bitmap identification obtained at the time.
In a specific embodiment of the present application, the request processing module is specifically configured to directly read the target file from the target volume if the request type is a target volume read request and the bitmap identifier read this time is true;
if the request type is a target volume read request and the read bitmap mark is false, the target file is read from the source volume corresponding to the redirected target file.
In a specific embodiment of the present application, the unlocking processing module is specifically configured to modify a bitmap identifier in a local copy bitmap if the local node is a master node of a target file and the target file is copied to a target volume in an IO processing process, send a bitmap modification message to auxiliary nodes of the target file, and unlock after receiving bitmap modification success messages fed back by all the auxiliary nodes;
if the node is an auxiliary node of the target file and the target file is copied to the target volume in the IO processing process, a bitmap modification request is sent to a master node of the target file, after a bitmap modification message sent by the master node of the target file is received, bitmap identification in the local copy bitmap is synchronously modified, and a bitmap modification success message is fed back to the master node of the target file so as to facilitate unlocking of the master node.
Corresponding to the above method embodiment, the embodiment of the present application further provides an electronic device, where an electronic device described below and a request processing method described above may be referred to correspondingly.
Referring to fig. 4, the electronic device includes:
a memory 332 for storing a computer program;
a processor 322 for implementing the steps of the request processing method of the above-described method embodiment when executing a computer program.
Specifically, referring to fig. 5, fig. 5 is a schematic diagram of a specific structure of an electronic device according to the present embodiment, where the electronic device may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 322 (e.g., one or more processors) and a memory 332, where the memory 332 stores one or more computer programs 342 or data 344. Wherein the memory 332 may be transient storage or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a series of instruction operations in the data processing apparatus. Still further, the processor 322 may be configured to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the electronic device 301.
The electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input/output interfaces 358, and/or one or more operating systems 341.
The steps in the request processing method described above may be implemented by the structure of the electronic device.
Corresponding to the above method embodiments, the embodiments of the present application further provide a readable storage medium, where a readable storage medium described below and a request processing method described above may be referred to correspondingly.
A readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the request processing method of the above-described method embodiment.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, and the like.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation should not be considered to be beyond the scope of this application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms include, comprise, or any other variation is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The principles and embodiments of the present application are described herein with specific examples, the above examples being provided only to assist in understanding the methods of the present application and their core ideas; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. A method of processing a request, comprising:
after receiving an IO request for a target file, acquiring a bitmap identifier of the target file from a local copy bitmap;
determining a locking requirement and a locking application under the condition of the locking requirement by utilizing the bitmap identification and the request type of the IO request;
applying for locking to the main node of the target file by using the locking application;
IO processing is carried out on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined;
and after IO processing is completed, updating the local copy bitmap and unlocking the target file.
2. The method of claim 1, wherein applying for locking to the master node of the target file using the locking application comprises:
if the node is the master node of the target file, adding the locking application into a locking application queue, and sequentially processing the locking application in the locking application queue;
and if the node is an auxiliary node of the target file, sending the locking application to a main node of the target file.
3. The method of claim 2, wherein sequentially processing the locking applications in the lock application queue comprises:
if the target file is unlocked, reading a first locking application from the lock application queue in sequence;
if the first locking application is a shared lock, after the shared lock is allocated, reading a second locking application from the lock application queue in sequence;
if the second locking application is a sharing lock, continuing to distribute the sharing lock;
if the second locking application is a mutual exclusion lock, after the target file is unlocked, distributing the mutual exclusion lock;
and after the mutual exclusion lock is unlocked, reading a third locking application from the lock application queue in sequence.
4. The method of claim 1, wherein determining a locking requirement and, if a locking requirement exists, a locking application using the bitmap identification and a request type of the IO request comprises:
if the request type is a source volume read request, determining that locking is not needed;
if the request type is a target volume read request and the bitmap identification is true, determining that locking is not needed;
if the request type is a target volume read request and the bitmap mark is false, determining that locking is needed and the locking application is an application sharing lock;
if the request type is a source volume write request and the bitmap identification is true, determining that locking is not needed;
if the request type is a source volume write request and the bitmap mark is false, determining that locking is needed, wherein the locking application is application mutex;
if the request type is a target volume write request and the bitmap mark is false, determining that locking is needed, wherein the locking application is application mutex;
if the request type is a target volume write request and the bitmap identification is true, determining that locking is not needed;
the target volume is a local copy backup volume for performing local copy on the target file; the bitmap is identified as true corresponding to the target file having been copied to the target volume, and the bitmap is identified as false corresponding to the target file not having been copied to the target volume.
5. The method of claim 1, wherein the IO processing the target file if a lock is obtained or it is determined that no lock is required comprises:
after the lock is acquired, acquiring the bitmap identification of the target file from the local copy bitmap again;
and carrying out IO processing on the target file according to the bitmap identification obtained at the time.
6. The method of claim 5, wherein the performing IO processing on the target file according to the bitmap identifier acquired at the time includes:
if the request type is a target volume read request and the read bitmap mark is true, directly reading the target file from the target volume;
and if the request type is a target volume read request and the read bitmap mark is false, reading the target file from the source volume corresponding to the target file in a redirection mode.
7. The method of any of claims 1 to 6, wherein updating the local copy bitmap and unlocking the target file comprises:
if the node is a master node of the target file and the target file is copied to a target volume in the IO processing process, modifying the bitmap identification in the local copy bitmap, sending a bitmap modification message to auxiliary nodes of the target file, and unlocking after receiving bitmap modification success messages fed back by all the auxiliary nodes;
if the node is an auxiliary node of the target file and the target file is copied to the target volume in the IO processing process, a bitmap modification request is sent to a main node of the target file, after a bitmap modification message sent by the main node of the target file is received, the bitmap identification in the local copy bitmap is synchronously modified, and a bitmap modification success message is fed back to the main node of the target file so that the main node is unlocked.
8. A request processing apparatus, comprising:
the bitmap identification reading module is used for acquiring the bitmap identification of the target file from the local copy bitmap after receiving the IO request aiming at the target file;
the lock demand analysis module is used for determining a locking demand and a locking application under the condition of the locking demand by utilizing the bitmap identification and the request type of the IO request;
the locking application module is used for applying for locking to the main node of the target file by utilizing the locking application;
the request processing module is used for carrying out IO processing on the target file under the condition that the lock is obtained or the condition that locking is not needed is determined;
and the unlocking processing module is used for updating the local copy bitmap and unlocking the target file after the IO processing is completed.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the request processing method according to any one of claims 1 to 7 when executing said computer program.
10. A readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the request processing method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311541223.4A CN117591013A (en) | 2023-11-17 | 2023-11-17 | Request processing method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311541223.4A CN117591013A (en) | 2023-11-17 | 2023-11-17 | Request processing method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117591013A true CN117591013A (en) | 2024-02-23 |
Family
ID=89914410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311541223.4A Pending CN117591013A (en) | 2023-11-17 | 2023-11-17 | Request processing method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117591013A (en) |
-
2023
- 2023-11-17 CN CN202311541223.4A patent/CN117591013A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018103318A1 (en) | Distributed transaction handling method and system | |
| US8495266B2 (en) | Distributed lock | |
| US6889253B2 (en) | Cluster resource action in clustered computer system incorporation prepare operation | |
| US7636868B2 (en) | Data replication in a distributed system | |
| US7814065B2 (en) | Affinity-based recovery/failover in a cluster environment | |
| US4480304A (en) | Method and means for the retention of locks across system, subsystem, and communication failures in a multiprocessing, multiprogramming, shared data environment | |
| US7698391B2 (en) | Performing a provisioning operation associated with a software application on a subset of the nodes on which the software application is to operate | |
| CN108509462B (en) | Method and device for synchronizing activity transaction table | |
| US20040236914A1 (en) | Method to provide atomic update primitives in an asymmetric heterogeneous multiprocessor environment | |
| US7823008B2 (en) | Maintaining consistency in a remote copy data storage system | |
| JP2565658B2 (en) | Resource control method and apparatus | |
| CN103703464A (en) | Method and apparatus for distributed configuration management | |
| US10698767B1 (en) | Decentralized management of multi-service workflows | |
| CN115550384B (en) | Cluster data synchronization method, device and equipment and computer readable storage medium | |
| WO2021082465A1 (en) | Method for ensuring data consistency and related device | |
| CN113791916B (en) | Object updating and reading method and device | |
| CN117591013A (en) | Request processing method, device, equipment and readable storage medium | |
| CN110659303A (en) | Read-write control method and device for database nodes | |
| CN112381650B (en) | Cross-chain interoperation transaction processing method, device, electronic equipment and storage medium | |
| CN116974983A (en) | Data processing method, device, computer readable medium and electronic equipment | |
| WO2008039618A1 (en) | Persistent locks/resources for concurrency control | |
| Bravo et al. | Reconfigurable atomic transaction commit | |
| JP7131363B2 (en) | LICENSE MANAGEMENT DEVICE, LICENSE MANAGEMENT SYSTEM, LICENSE MANAGEMENT METHOD, AND PROGRAM | |
| CN112328637A (en) | High-speed distributed data caching method and device, computer equipment and storage medium | |
| KR20130043823A (en) | Distributed storage system for maintaining data consistency based on log, and method for the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |