CN111651123B

CN111651123B - Processing method, device and equipment for concurrent IO (input/output) requests

Info

Publication number: CN111651123B
Application number: CN202010490469.3A
Authority: CN
Inventors: 陈玉强; 何抗洪; 韩盛中
Original assignee: Suning Cloud Computing Co Ltd
Current assignee: Suning Cloud Computing Co Ltd
Priority date: 2020-06-02
Filing date: 2020-06-02
Publication date: 2022-11-18
Anticipated expiration: 2040-06-02
Also published as: CN111651123A

Abstract

The embodiment of the application discloses a method, a device and equipment for processing concurrent IO requests, wherein the method comprises the following steps: receiving concurrent IO requests; acquiring preset virtual paths, and establishing a mapping relation between concurrent IO requests and the virtual paths, wherein each virtual path comprises at least two physical paths; and selecting a physical path from the virtual paths and judging the state of the physical path, and if the physical path is effective, issuing a corresponding IO request according to the physical path. According to the method and the device, the IO requests can be balanced on a plurality of physical paths, when one physical path is in fault, only the IO requests related to the physical path are influenced, any influence on other IO requests cannot be caused, and the read-write efficiency is improved.

Description

Processing method, device and equipment for concurrent IO (input/output) requests

Technical Field

The invention belongs to the field of storage, and particularly relates to a method, a device and equipment for processing concurrent IO requests.

Background

In the prior art, when a plurality of IO requests exist on a volume device, the IO requests are usually issued through only one physical path, and therefore, if the current physical path is abnormal, the IO requests of the entire volume device are affected, thereby reducing the read-write efficiency.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a method, a device and equipment for processing concurrent IO requests, which can enable the IO requests to be balanced on a plurality of physical paths, and when one physical path fails, only the IO requests related to the physical path are affected, and no influence is caused on other IO requests, so that the read-write efficiency is improved.

The embodiment of the invention provides the following specific technical scheme:

the first aspect discloses a method for processing concurrent IO requests, which includes:

receiving concurrent IO requests;

acquiring preset virtual paths, and establishing a mapping relation between the concurrent IO requests and the virtual paths, wherein each virtual path comprises at least two physical paths;

selecting a physical path from the virtual paths and judging the state of the physical path, if the physical path is effective, issuing a corresponding IO request according to the physical path.

Preferably, the physical paths in all virtual paths are different from each other.

Preferably, the obtaining a preset virtual path, and the establishing a mapping relationship between the concurrent IO request and the virtual path specifically includes:

acquiring the number of the IO requests and the number of the virtual paths;

and comparing the number of the IO requests with the number of the virtual paths, acquiring a corresponding processing rule according to a comparison result, and allocating a virtual path to each IO request according to the processing rule to establish a mapping relation between each IO request and the allocated virtual path.

Preferably, obtaining a corresponding processing rule according to the comparison result, and allocating a virtual path to each IO request according to the processing rule to establish a mapping relationship between each IO request and the allocated virtual path specifically includes:

when the number of the IO requests is equal to the number of the virtual paths, acquiring numbers corresponding to all the virtual paths, and allocating a virtual path corresponding to a different number to each IO request to establish a mapping relation between each IO request and the allocated virtual path;

when the number of the IO requests is smaller than that of the virtual paths, randomly selecting the virtual paths with the number equal to that of the IO requests from all the virtual paths, and distributing the virtual paths corresponding to different numbers to each IO request according to the numbers corresponding to the randomly selected virtual paths so as to establish a mapping relation between each IO request and the distributed virtual paths;

when the number of the IO requests is larger than that of the virtual paths, obtaining the number of the virtual paths to be distributed corresponding to each IO request according to a preset calculation rule, and distributing the virtual paths corresponding to the number to the corresponding IO requests to establish a mapping relation between each IO request and the distributed virtual paths;

and the number corresponding to each virtual path is different.

Preferably, obtaining the number of the virtual path to be allocated corresponding to each IO request according to a preset calculation rule specifically includes:

acquiring the address of the block to be processed carried in each IO request, and performing rounding calculation on the size of a preset block to be processed by using the address of the block to be processed carried in each IO request to obtain the block number of the block to be processed corresponding to each IO request;

and performing remainder calculation on the number of the preset virtual paths by using the block numbers to obtain the number of the virtual path to be distributed corresponding to each IO request.

Preferably, the selecting one physical path from the virtual paths specifically includes:

acquiring the utilization rates of all physical paths in the virtual path;

and selecting a physical path corresponding to the lowest utilization rate from the virtual paths.

Preferably, if a selected physical path in a virtual path corresponding to any IO request is invalid, the method further includes:

acquiring other physical paths different from the selected physical path from the virtual path corresponding to the IO request, and selecting one physical path from the other physical paths as a new physical path;

judging the state of the new physical path, and if the new physical path is valid, issuing the IO request according to the new physical path;

if all physical paths in the virtual path corresponding to the IO request are invalid, marking the virtual path, and randomly selecting one virtual path from other marked virtual paths different from the virtual path corresponding to the IO request as a new virtual path;

and selecting a physical path from the new virtual paths, judging the state of the selected physical path from the new virtual paths, and if the selected physical path from the new virtual paths is valid, issuing the IO request according to the selected physical path from the new virtual paths.

Preferably, the method further comprises:

judging whether the current concurrent IO request is issued completely, if so, deleting the labeled virtual path;

and acquiring an unmarked virtual path, and when receiving a new concurrent IO request, establishing a mapping relation between the new concurrent IO request and the unmarked virtual path.

In a second aspect, an apparatus for processing concurrent IO requests is disclosed, the apparatus including:

the receiving module is used for receiving concurrent IO requests;

the mapping module is used for acquiring a preset virtual path and establishing a mapping relation between the concurrent IO request and the virtual path, wherein each virtual path comprises at least two physical paths;

and the processing module is used for selecting a physical path from the virtual paths and judging the state of the physical path, and if the physical path is effective, issuing a corresponding IO request according to the physical path.

In a third aspect, a computer device is disclosed, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the processing method of the concurrent IO request according to the first aspect.

The embodiment of the invention has the following beneficial effects:

1. according to the method, the virtual paths are established, at least two physical paths are added into each virtual path, and the mapping relation between different IO requests and the virtual paths is established, so that when a certain physical path fails, only the relevant IO requests are influenced, and all IO requests on the equipment are not influenced;

2. in the application, the physical paths in all the virtual paths are different from each other, so that the condition that other IO requests are influenced due to the abnormality of a certain physical path can be completely avoided;

3. according to the method and the device, when the physical path in the virtual path is unavailable, smooth switching of the physical path can be achieved, switching is performed without waiting for all IO requests to be issued, so that the condition that IO requests are overtime due to network delay and the like can be avoided, and the reading and writing efficiency is improved;

4. in the method and the device, the utilization rate of each available physical path is obtained before the physical path is determined, so that flow balance can be realized, and the situations that a certain physical path is busy and other physical paths are idle are avoided.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a processing method for concurrent IO requests provided in embodiment 1 of the present application;

fig. 2 is a mapping relationship between an IO request on a block and a physical path provided in embodiment 1 of the present application;

fig. 3 is a mapping relationship between an IO request after path switching and a physical path provided in embodiment 1 of the present application;

fig. 4 is a schematic structural diagram of a processing apparatus for concurrent IO requests according to embodiment 2 of the present application;

fig. 5 is a schematic structural diagram of a computer device provided in embodiment 3 of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As described in the background art, in the prior art, when there are multiple IO requests on a volume device, the IO requests are usually issued through only one physical path, and therefore, if the current physical path is abnormal, the IO requests of the entire volume device are affected, thereby reducing the read-write efficiency. Based on this, the application provides a processing method for concurrent IO requests, which can balance IO requests on a plurality of physical paths, and when one physical path fails, only the IO request related to the physical path is affected, and no influence is caused on other IO requests, thereby improving the read-write efficiency.

Example 1

As shown in fig. 1, a method for processing concurrent IO requests includes the following steps:

s11, receiving concurrent IO requests;

s12, acquiring preset virtual paths, and establishing a mapping relation between concurrent IO requests and the virtual paths, wherein each virtual path comprises at least two physical paths;

specifically, since the number of IO requests may be different from the preset number of virtual paths, there may be a difference in allocating virtual paths, and the step S12 specifically includes the following steps:

s121, acquiring the number of IO requests and the number of virtual paths;

s122, comparing the number of the IO requests with the number of the virtual paths, obtaining a corresponding processing rule according to the comparison result, and allocating a virtual path to each IO request according to the processing rule to establish a mapping relation between each IO request and the allocated virtual path.

According to the method and the device, the difference condition between the number of the IO requests and the number of the virtual paths is considered, so that different processing rules are designed, and each IO request is guaranteed to be allocated with one virtual path. The step S122 may specifically include the following steps:

1. numbering the virtual paths, wherein the numbers corresponding to the virtual paths are different;

2. when the number of the IO requests is equal to that of the virtual paths, acquiring numbers corresponding to all the virtual paths, and allocating a virtual path corresponding to a different number to each IO request to establish a mapping relation between each IO request and the allocated virtual path;

3. when the number of the IO requests is smaller than that of the virtual paths, randomly selecting the virtual paths with the number equal to that of the IO requests from all the virtual paths, and distributing the virtual paths corresponding to different numbers to each IO request according to the numbers corresponding to the randomly selected virtual paths so as to establish a mapping relation between each IO request and the distributed virtual paths;

when the number of the IO requests is equal to the number of the virtual paths or the number of the IO requests is smaller than the number of the virtual paths, a virtual path can be allocated to each IO request without repeating, and each virtual path includes at least two physical paths, so that backup of the physical paths is also realized, and a guarantee is provided for smooth switching when the physical paths are abnormal.

4. When the number of the IO requests is larger than that of the virtual paths, the number of the virtual path to be distributed corresponding to each IO request is obtained according to a preset calculation rule, and the virtual path corresponding to the number is distributed to the corresponding IO request so as to establish a mapping relation between each IO request and the distributed virtual path.

When the number of IO requests is greater than the number of virtual paths, a situation may occur in which a plurality of IO requests share one virtual path, but since each virtual path includes at least two physical paths, even if different IO requests correspond to the same virtual path, there is a probability that different physical paths are allocated.

The obtaining, according to a preset calculation rule, a number of a virtual path to be allocated corresponding to each IO request specifically includes:

a: acquiring the address of the block to be processed carried in each IO request, and performing rounding calculation on the size of the preset block to be processed by using the address of the block to be processed carried in each IO request to obtain the block number of the block to be processed corresponding to each IO request;

b: and performing remainder calculation on the number of the preset virtual paths by using the block numbers to obtain the number of the virtual path to be distributed corresponding to each IO request.

The following are exemplary:

if 4 physical paths exist currently, which are p0, p1, p2, and p3, respectively, two preset virtual paths are provided, and the numbers are 0 and 1, respectively. The number 0 virtual path includes two physical paths p0 and p1, and the number 1 virtual path includes two physical paths p2 and p 3.

If three IO requests exist currently, the first IO request is as follows: reading 4k of data from the A1 byte location of the block device, the second IO request being: reading 4k data from the A2 byte position of the block device, the third IO request being: 4k of data is read from the A3 byte location of the block device, with a block size of 1GB.

And when the number of the IO requests is larger than that of the virtual paths, calculating the number of the virtual path corresponding to each IO request by adopting the calculation rule.

If the calculated block number of the block corresponding to the first IO request is 0, the block number of the block corresponding to the second IO request is 2, and the block number of the block corresponding to the third IO request is 3, then the number of the virtual path corresponding to the first IO request is 0, the number of the virtual path corresponding to the second IO request is also 0, and the number of the virtual path corresponding to the third IO request is 1, then the first IO request and the second IO request correspond to the same virtual path 0.

According to the steps, mapping between the IO request and the virtual path can be achieved.

And S13, selecting a physical path from the virtual paths, judging the state of the physical path, and if the physical path is effective, issuing a corresponding IO request according to the physical path.

Specifically, before this step, a corresponding relationship between the virtual path and the physical path needs to be established, in this scheme, the physical path is an IP address, the virtual path is equivalent to a folder, and the same physical path may be added to different virtual paths. Therefore, the IO request can be issued to a server or other receiving devices through the IP address.

In addition, the selecting of one physical path from the virtual paths may specifically include the following steps:

1. acquiring the utilization rates of all physical paths in the virtual path;

2. and selecting the physical path corresponding to the lowest utilization rate from the virtual paths.

Therefore, the flow balance can be realized, and the situations that a certain physical path is busy and other physical paths are idle are avoided.

According to the scheme, through establishing the mapping relation between different IO requests and the virtual paths, when a certain physical path fails, only the relevant IO requests are influenced, and all IO requests on the equipment cannot be influenced.

In addition, if the selected physical path in the virtual path corresponding to any IO request is invalid, the method further includes:

1. acquiring other physical paths different from the selected physical path from the virtual path corresponding to the IO request, and selecting one physical path from the other physical paths as a new physical path;

2. judging the state of the new physical path, and if the new physical path is effective, issuing an IO request according to the new physical path;

3. if all physical paths in the virtual path corresponding to the IO request are invalid, marking the virtual path, and randomly selecting one virtual path from other unmarked virtual paths different from the virtual path corresponding to the IO request as a new virtual path;

4. and selecting a physical path from the new virtual paths, judging the state of the physical path in the new virtual paths, and issuing the IO request according to the physical path in the new virtual paths if the physical path in the new virtual paths is effective.

When a new physical path is selected/a physical path is selected from the new virtual paths, the utilization rates of all the physical paths can be obtained, and the corresponding physical path with the lowest utilization rate is selected to realize flow balance.

In addition, this scheme still includes:

1. judging whether the current concurrent IO request is issued completely, if so, deleting the labeled virtual path;

2. and acquiring an unmarked virtual path, and when receiving a new concurrent IO request, establishing a mapping relation between the new concurrent IO request and the unmarked virtual path.

According to the scheme, when the physical path in the virtual path is unavailable, smooth switching of the physical path can be achieved, switching is performed without waiting for all IO requests to be issued, so that the condition that IO requests are overtime due to network delay and the like can be avoided, and the read-write efficiency is improved.

Fig. 2 is a mapping relationship diagram between concurrent IO requests and physical paths established according to the present solution, and fig. 3 is a mapping relationship diagram after path switching when a physical path is invalid; referring to fig. 2, the IO request on block 1 corresponds to virtual path 1, and virtual path 1 includes two physical paths (1, 2), the IO request on block 1 corresponds to virtual path 2, and virtual path 2 includes three physical paths (2, 3, 4), when all physical paths (1, 2) in virtual path 1 are invalid, referring to fig. 3, the IO request on block 1 is switched to block 2, and two physical paths (3, 4) in block 2 are still valid, so the IO requests on block 1 and block 2 can both be normally issued.

In addition, as a preferred embodiment, the physical paths in all the virtual paths are different. Therefore, the situation that two or more IO requests share one virtual path does not occur, and the situation that the IO requests of the whole volume are influenced by the fault of one physical path can be completely avoided.

Example 2

Based on the foregoing embodiment 1, an embodiment of the present invention further provides a device for processing concurrent IO requests, and as shown in fig. 4, the device includes:

a receiving module 41, configured to receive concurrent IO requests;

the mapping module 42 is configured to obtain preset virtual paths, and establish a mapping relationship between concurrent IO requests and the virtual paths, where each virtual path includes at least two physical paths;

and the processing module 43 is configured to select a physical path from the virtual paths and determine a state of the physical path, and if the physical path is valid, issue a corresponding IO request according to the physical path.

Preferably, the mapping module 42 is specifically configured to:

acquiring the number of IO requests and the number of virtual paths;

Preferably, the mapping module 42 is further specifically configured to:

when the number of the IO requests is equal to that of the virtual paths, acquiring numbers corresponding to all the virtual paths, and allocating a virtual path corresponding to a different number to each IO request to establish a mapping relation between each IO request and the allocated virtual path;

when the number of the IO requests is smaller than that of the virtual paths, randomly selecting the virtual paths with the number equal to that of the IO requests from all the virtual paths, and distributing the virtual paths corresponding to different numbers for each IO request according to the numbers corresponding to the randomly selected virtual paths to establish a mapping relation between each IO request and the distributed virtual paths;

and the number corresponding to each virtual path is different.

Preferably, the mapping module 42 is further specifically configured to:

acquiring an address of a block to be processed carried in each IO request, and performing rounding calculation on the size of a preset block to be processed by using the address of the block to be processed carried in each IO request to obtain a block number of the block to be processed corresponding to each IO request;

Preferably, the processing module 43 is further configured to:

acquiring the utilization rates of all physical paths in the virtual path;

Preferably, the processing module 43 is further configured to, if a selected physical path in the virtual paths corresponding to any IO request is invalid, obtain other physical paths different from the selected physical path in the virtual paths corresponding to the IO request, and select one physical path from the other physical paths as a new physical path;

judging the state of the new physical path, and if the new physical path is effective, issuing an IO request according to the new physical path;

if all physical paths in the virtual path corresponding to the IO request are invalid, marking the virtual path, and randomly selecting one virtual path from other unmarked virtual paths different from the virtual path corresponding to the IO request as a new virtual path;

and selecting a physical path from the new virtual paths, judging the state of the physical path selected from the new virtual paths, and issuing the IO request according to the physical path selected from the new virtual paths if the physical path selected from the new virtual paths is effective.

Preferably, the above apparatus further comprises:

the judging module 44 is configured to judge whether the current concurrent IO request is completely issued, and if yes, delete the labeled virtual path;

the mapping module 42 is further configured to obtain an unmarked virtual path, and when a new concurrent IO request is received, establish a mapping relationship between the new concurrent IO request and the unmarked virtual path.

It should be noted that: in the processing device for concurrent IO requests provided in this embodiment, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the processing apparatus for concurrent IO requests of this embodiment and the processing method embodiment for concurrent IO requests of embodiment 1 belong to the same concept, and specific implementation processes and beneficial effects thereof are detailed in the text recognition model training method embodiment, and are not described herein again.

Example 3

In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing all the methods described in embodiment 1 when executing the computer program.

Fig. 5 is an internal structural diagram of a computer device according to an embodiment of the present invention. The computer device may be a server, and its internal structure diagram may be as shown in fig. 5. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of processing concurrent IO requests.

Those skilled in the art will appreciate that the configuration shown in fig. 5 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing devices to which aspects of the present invention may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for processing concurrent IO requests, the method comprising:

receiving concurrent IO requests;

selecting a physical path from the virtual paths and judging the state of the physical path, if the physical path is effective, issuing a corresponding IO request according to the physical path;

the obtaining of the preset virtual path and the establishing of the mapping relationship between the concurrent IO request and the virtual path specifically include:

acquiring the number of the IO requests and the number of the virtual paths;

comparing the number of the IO requests with the number of the virtual paths, acquiring a corresponding processing rule according to a comparison result, and allocating a virtual path to each IO request according to the processing rule to establish a mapping relation between each IO request and the allocated virtual path;

the obtaining of the corresponding processing rule according to the comparison result, and allocating a virtual path to each IO request according to the processing rule to establish a mapping relationship between each IO request and the allocated virtual path specifically include:

when the number of the IO requests is larger than that of the virtual paths, obtaining the number of the virtual paths to be allocated corresponding to each IO request according to a preset calculation rule, and allocating the virtual paths corresponding to the number to the corresponding IO requests to establish a mapping relation between each IO request and the allocated virtual paths, wherein the numbers corresponding to each virtual path are different;

2. The method of claim 1, wherein the physical paths in all virtual paths are different from each other.

3. The method according to claim 1, wherein a corresponding processing rule is obtained according to the comparison result, and a virtual path is allocated to each IO request according to the processing rule to establish a mapping relationship between each IO request and the allocated virtual path, further comprising:

and when the number of the IO requests is smaller than that of the virtual paths, randomly selecting the virtual paths with the number equal to that of the IO requests from all the virtual paths, and allocating the virtual paths corresponding to different numbers to each IO request according to the numbers corresponding to the randomly selected virtual paths so as to establish the mapping relation between each IO request and the allocated virtual paths.

4. The method according to any one of claims 1 to 3, wherein the selecting one physical path among the virtual paths specifically comprises:

acquiring the utilization rates of all physical paths in the virtual path;

5. The method according to any one of claims 1 to 3, wherein if the selected physical path in the virtual path corresponding to any IO request is invalid, the method further comprises:

if all physical paths in the virtual path corresponding to the IO request are invalid, labeling the virtual path, and randomly selecting one virtual path from other unmarked virtual paths different from the virtual path corresponding to the IO request as a new virtual path;

6. The method of claim 5, further comprising:

7. An apparatus for implementing the method for processing the concurrent IO requests according to claim 1, wherein the apparatus comprises:

the receiving module is used for receiving concurrent IO requests;

and the processing module is used for selecting a physical path from the virtual paths and judging the state of the physical path, and if the physical path is valid, issuing a corresponding IO request according to the physical path.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that:

the processor, when executing the computer program, implements the method for processing the concurrent IO requests according to any one of claims 1 to 6.