CN108037894B - Disk space management method and device - Google Patents
Disk space management method and device Download PDFInfo
- Publication number
- CN108037894B CN108037894B CN201711270240.3A CN201711270240A CN108037894B CN 108037894 B CN108037894 B CN 108037894B CN 201711270240 A CN201711270240 A CN 201711270240A CN 108037894 B CN108037894 B CN 108037894B
- Authority
- CN
- China
- Prior art keywords
- volume
- logical
- groups
- capacity
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of data storage, in particular to a disk space management method and device, which are used for solving the problem that in the prior art, when a disk where a super block of a file system is located fails, all storage spaces in an LV corresponding to equipment are abnormal. The disk space management method provided by the embodiment of the application comprises the following steps: the method comprises the steps that a first device respectively creates volume groups in a plurality of preset disk groups, wherein each disk group comprises at least one disk; selecting at least two volume groups from the volume groups as selected volume groups, and respectively creating a logical volume on each selected volume group; and establishing a file system in the logical volume, and determining at least two logical volumes with the established file systems as storage areas of a second device connected with the first device.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a method and an apparatus for managing disk space.
Background
Logical Volume Manager (LVM) is a mechanism for managing disk partitions in a Linux environment, so that a user can conveniently adjust the size of each partition without stopping the computer, and a file system can span different disks.
At present, a disk space management method based on the LVM may be as shown in fig. 1, and specifically includes: a plurality of disks (Disk) configured inside a storage device are used as a Physical Volume (PV), and a plurality of Physical blocks (PE) with equal size are divided on each PV. The method comprises the steps of forming a Volume Group (VG) by combining physical volumes formed by a plurality of disks, and creating a Logical Volume (LV) on the VG according to the size of a specified storage space, wherein the LV is formed by a plurality of Logical blocks (LE) mapped by a PE, a file system can be created on the LV, the block at the starting position of each file system is called a super block, and the super block contains key information such as data structure information, control information and the like of the file system. Further, as shown in fig. 2, when allocating storage space for each device connected to the storage device and needing to store data, an LV is created for each device needing to store data on a VG formed by a physical volume composed of a plurality of disks as a storage area of the device needing to store data.
The inventor finds that, in the existing disk space management method based on the LVM, a VG is created on a physical volume formed by a plurality of disks, and then an LV is created in the created VG for each device needing to store data, where the LV corresponds to a storage space of a specified size of the plurality of disks, so that when a file system is created on the LV, the file system is likely to be created across the plurality of disks. For example, assuming that VG is composed of a disk1 and a disk2 with 1T storage space, LV1 with 0.8T storage space is created for the device 1 on VG, a file system 1 of LV1 is created on the disk1, and superblocks of the file system 1 are also located in the disk 1. Further, the creation of LV2 with 1.2T of storage space for device 2 continues on VG, where file system 2 is built on disk1 and disk2 in LV2, and the superblock of file system 2 is located in disk 1. If the disk1 is damaged or pulled out, for the device 1, the entire storage space of the LV1 is not available because the file system and the superblock on the LV1 are both failed, so that the device 1 cannot perform normal read-write operation; for the device 2, even though there is still available storage space in the disk2, due to the failure of the super block of the file system of the LV2, an exception occurs in the entire file system, so that the storage space of the entire LV2 is not available, and the device 2 cannot perform read/write operations normally. Therefore, as one device has one corresponding file system, if a disk in which a super block of the file system is located is damaged or pulled out, the super block fails, which may cause an exception to occur in the entire file system built on a plurality of disks, that is, all storage spaces in the LV corresponding to the device are abnormal, so that the device cannot perform read-write operations normally.
Based on this, in the prior art, a way of implementing disk space management based on the LVM has a problem that when a disk where a superblock of a file system is located fails, all storage spaces in the LV corresponding to a device are abnormal.
Disclosure of Invention
The embodiment of the application provides a disk space management method and device, which are used for solving the problem that in the prior art, when a disk where a super block of a file system is located fails, all storage spaces in an LV corresponding to equipment are abnormal.
The technical scheme provided by the application is as follows:
in a first aspect, a method for managing disk space includes:
the method comprises the steps that a first device respectively creates volume groups in a plurality of preset disk groups, wherein each disk group comprises at least one disk;
the first device selects at least two volume groups from the volume groups as selected volume groups and respectively creates a logical volume on each selected volume group;
the first device establishes file systems in the logical volumes respectively, and determines at least two logical volumes with the established file systems as storage areas of a second device connected with the first device.
Further, the selecting, by the first device, at least two volume groups from the volume groups as selected volume groups, and creating a logical volume on each of the selected volume groups respectively specifically includes:
sequentially selecting N volume groups from the volume groups corresponding to each disk group according to the sequence of the volume group residual capacity from large to small, wherein N is a positive integer larger than 1;
if the sum of the residual capacities of the N volume groups is smaller than the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from large to small; and the number of the first and second electrodes,
judging whether the residual capacity of the currently selected volume group is smaller than the difference value between the total capacity required by the second equipment and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
and if not, using the capacity represented by the difference value in the currently selected volume group to create a logical volume.
Further, after the first device selects N volume groups from the volume groups respectively corresponding to the disk groups, the method further includes:
if the sum of the residual capacities of the N volume groups is larger than or equal to the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from small to large; and the number of the first and second electrodes,
judging whether the remaining capacity of the currently selected volume group is smaller than the ratio of the currently required capacity of the second device to the volume group number of the logical volume not created in the N volume groups; wherein the current required capacity of the second device is a difference value between the total capacity required by the second device and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
if not, the capacity represented by the ratio in the currently selected volume group is used for creating the logical volume, and the selected volume group is continuously selected to create the logical volume until the total capacity of the created logical volume is equal to the total capacity required by any second device.
Further, after the first device determines at least two logical volumes with the established file system as storage areas of a second device connected to the first device, the method further includes:
writing files in the logical volume of which the time of the latest file writing in the storage area is within a first specified time range;
if the logical volume in the writing state is determined to be full, switching to other logical volumes which are not full in the storage area to write files; when the residual capacity of the logical volume is smaller than the specified capacity, determining that the logical volume is full;
and if the other logical volumes in the storage area are determined to be full, writing files in the designated capacity of the logical volumes in the storage area.
Further, the method further comprises:
if it is determined that the other logical volumes in the storage area are full or the designated capacity of the logical volume in the storage area is full, the first device deletes the written file in the logical volume whose time for writing the file is recently in the second designated time range, and switches to the logical volume whose time for writing the file is recently in the second designated time range to write the file.
Further, if the first device determines that the logical volume in the write state has the designated failure, the method further includes:
stopping writing files in the logical volume in the writing state; and the number of the first and second electrodes,
and switching to other unfilled logical volumes in the storage area to write files.
In a second aspect, a disk space management apparatus includes:
the volume group creation module is used for respectively creating volume groups in a plurality of preset disk groups, wherein each disk group comprises at least one disk;
the logical volume creating module is used for selecting at least two volume groups from the volume groups as selected volume groups and respectively creating logical volumes on each selected volume group;
and the storage area determining module is used for respectively establishing file systems in the logical volumes and determining at least two logical volumes with the established file systems as the storage areas of the second equipment connected with the first equipment.
Further, the logical volume creation module is specifically configured to:
sequentially selecting N volume groups from the volume groups corresponding to each disk group according to the sequence of the volume group residual capacity from large to small, wherein N is a positive integer larger than 1;
if the sum of the residual capacities of the N volume groups is smaller than the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from large to small; and the number of the first and second electrodes,
judging whether the residual capacity of the currently selected volume group is smaller than the difference value between the total capacity required by the second equipment and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
and if not, using the capacity represented by the difference value in the currently selected volume group to create a logical volume.
Further, the logical volume creation module is further configured to:
if the sum of the residual capacities of the N volume groups is larger than or equal to the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from small to large; and the number of the first and second electrodes,
judging whether the remaining capacity of the currently selected volume group is smaller than the ratio of the currently required capacity of the second device to the volume group number of the logical volume not created in the N volume groups; wherein the current required capacity of the second device is a difference value between the total capacity required by the second device and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
if not, the capacity represented by the ratio in the currently selected volume group is used for creating the logical volume, and the selected volume group is continuously selected to create the logical volume until the total capacity of the created logical volume is equal to the total capacity required by the second device.
Further, the apparatus further comprises:
a file writing module, configured to, after the storage area determining module determines that at least two logical volumes with file systems established are storage areas of a second device connected to the first device, write a file in a logical volume in which the time for writing the file in the storage area most recently is within a first specified time range;
if the logical volume in the writing state is determined to be full, switching to other logical volumes which are not full in the storage area to write files; when the residual capacity of the logical volume is smaller than the specified capacity, determining that the logical volume is full;
and if the other logical volumes in the storage area are determined to be full, writing files in the designated capacity of the logical volumes in the storage area.
Further, the file writing module is further configured to:
if it is determined that the other logical volumes in the storage area are full or the designated capacity of the logical volume in the storage area is full, the first device deletes the written file in the logical volume whose time for writing the file is recently in the second designated time range, and switches to the logical volume whose time for writing the file is recently in the second designated time range to write the file.
Further, the file writing module is further configured to:
if the logic volume in the writing state is determined to generate the designated fault, stopping writing the file in the logic volume in the writing state; and switching to other unfilled logical volumes in the storage area to write files.
In a third aspect, an electronic device includes: one or more processors; and one or more computer readable media having stored thereon a program for executing the method of disk space management, wherein the program when executed by the one or more processors implements the steps of the method as set forth in any one of the first aspects.
In a fourth aspect, one or more computer readable media having stored thereon a program for performing a method of disk space management, wherein the program, when executed by one or more processors, causes a communication device to perform the method of any of the first aspects.
In this embodiment of the present application, a first device may respectively construct volume groups in a plurality of preset volume groups, where each volume group includes at least one disk, then select at least two volume groups from the volume groups as selected volume groups, and respectively create logical volumes on each selected volume group, so as to establish a file system in the logical volumes, and determine at least two logical volumes in which the file system has been established as a storage area of a second device connected to the first device. Therefore, by adopting the manner provided by the embodiment of the application, the storage area determined for the second device can actually contain at least two logical volumes, wherein a file system is respectively established in each logical volume, each logical volume corresponds to a selected volume group, and each selected volume group is established by at least one disk, so that even if a disk where a superblock of the file system is located is damaged or pulled out, only the storage space in the logical volume corresponding to the file system is affected, and the storage spaces of other logical volumes in the storage area of the device are not affected, so that read-write operation can be continuously performed on other logical volumes, thereby greatly reducing the influence on the service of reading and writing data of the device due to damage or pulling out of the disk, and ensuring that the device can normally perform read-write operation.
Drawings
FIG. 1 is a schematic diagram of a disk space management method based on LVM in the prior art;
FIG. 2 is a schematic diagram illustrating a LVM-based storage space allocation method in the prior art;
FIG. 3 is a flowchart of a method for managing disk space according to an embodiment of the present application;
fig. 4a is a flowchart of a method for selecting VGs and creating an LV according to a first embodiment of the present disclosure;
fig. 4b is a flowchart of a method for selecting VGs and creating an LV according to the first embodiment of the present application;
FIG. 5 is a diagram illustrating allocation of a storage area for a second device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a disk space management apparatus according to a second embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without any creative effort belong to the protection scope of the present application.
In order to solve the problem that in the prior art, when a disk of a superblock of a file system is in failure, all storage spaces in an LV corresponding to a device are abnormal, the present application provides a disk space management method and apparatus, wherein volume groups are respectively configured in a plurality of preset disk groups, each disk group includes at least one disk, then at least two volume groups are selected from the volume groups as selected volume groups, and logical volumes are respectively created on each selected volume group, so that the file system can be established in the logical volumes, and the at least two logical volumes with the established file system are determined as a storage area of a second device connected to the first device. Therefore, the storage area determined for the second device can actually contain at least two logical volumes, wherein a file system is respectively established in each logical volume, each logical volume corresponds to a selected volume group, and each selected volume group is constructed by at least one disk, so that even if a disk where a super block of the file system is located is damaged or pulled out, only the storage space in the logical volume corresponding to the file system is affected, the storage space of other logical volumes in the storage area of the device is not affected, so that read-write operation can be continuously performed on other logical volumes, the influence on the service of reading and writing data of the device due to damage or pulling out of the disk is greatly reduced, and the normal read-write operation of the device is ensured.
The following describes the disk space management method proposed in the present application in detail with reference to specific embodiments.
Example one
As shown in fig. 3, a flowchart of a method for managing a disk space according to an embodiment of the present application includes the following steps:
step 301: the first device creates a volume group (hereinafter, the volume group is denoted by VG) in a preset plurality of disk groups, respectively, wherein each disk group includes at least one disk.
The first device may be any storage device having disk space management and storage functions, such as a server configured with multiple disks. And, the plurality of disks configured by the first device may be grouped in advance to obtain a plurality of disk groups. Each disk group may be a single disk, or may be at least two Disks in the form of a Redundant Array of Independent Disks (RAID). Moreover, the first device may be connected to at least one second device, and the second device may be a device having data storage requirements, for example, the second device may be a plurality of monitoring cameras, and files to be stored in the second device may be stored in a storage area previously allocated to the second device by the first device.
In an implementation, the first device may create a VG in each disk group, where the timing of creating VGs may include the following: identifying all disks during initial startup; when a RAID of a disk is created; when a newly inserted disk is identified during operation.
Further, when the first device allocates a storage space for at least one second device connected to the first device, the following steps may be performed for any one of the second devices:
step 302: the first device selects at least two of said VGs as selected VGs and creates a logical volume on each of said selected VGs, respectively (the logical volume is hereinafter denoted by LV).
In a specific implementation, in order to ensure that the LV is created for the second device in 2 or more VGs as much as possible and that the sum of the created LV capacities is equal to the total capacity required by the second device, the LV may be created for the second device in the following way:
firstly, N VGs can be sequentially selected from VGs corresponding to each disk in the descending order of the residual capacity of the VGs, where N is a positive integer greater than 1.
Here, N is a preset suggested value of the number of selected VGs, and it is preferable to try to create the LV for the second device among the N VGs. Wherein, in a scenario where multiple disks exist, the N value may be set to a positive integer greater than 1. When the number of created VGs is larger than 1, the value of N is less than or equal to the number of created VGs as much as possible under the condition that the value of N is larger than 1.
In addition, in order to ensure that the N VGs selected from the created VGs can satisfy the total capacity required by the second device as much as possible, in the embodiment of the present application, the N VGs may be sequentially selected from the created VGs in the order from large to small of the remaining capacities of the VGs. However, in the implementation, there may be a case where the remaining capacity of VGs is small, so the sum of the remaining capacities of the N VGs selected may not satisfy the total capacity required by the second device, and therefore, the following two cases are specifically described below as a process of selecting VGs and creating LVs for the second device.
(1) If the sum of the remaining capacities of the N VGs is less than the total capacity required by the second device, referring to fig. 4a, the process of selecting VGs and creating an LV specifically includes:
step 401 a: and sequentially selecting the selected VG from the N VGs according to the sequence that the residual capacity of the VG is from large to small.
Here, in the case that the sum of the remaining capacities of the N VGs is smaller than the total capacity required by the second device, preferably, the VGs may be selected in descending order of the remaining capacities of the VGs, so that not only can the requirement of creating an LV for the second device among the VGs selected, but also the LV can be created on the VG selected with a relatively abundant remaining space as preferentially as possible, so that the number of the VGs selected is not too large, and the operation flow is simplified. Of course, the embodiment of the present application is not limited to this manner, and in practical applications, the selection may also be randomly selected or set according to practical requirements, and details are not described here.
Step 402 a: it is determined whether the remaining capacity of the currently selected VG is less than the difference between the total capacity required by the second device and the total capacity of the created LV.
If yes, go to step 403 a; if not, go to step 404 a.
Step 403 a: the LV is created using the total remaining capacity in the currently selected VG. And returning to execute the steps 401a to 402a, and continuing to select the selected VG to create the LV until the judgment result is negative.
Step 404 a: the LV is created using a volume characterized by the difference between the total volume required by the second device in the currently selected VG and the total LV volume already created.
To this end, the total LV capacity created is equal to the total capacity required by the second device, and the process of allocating storage space for the second device is completed. Further, the created LV may be formatted, for example, by respectively creating a file system in the created LV. The specific formatting process can be found in the prior art and is not described in detail here.
(2) If the sum of the remaining capacities of the N VGs is greater than or equal to the total capacity required by the second device, referring to fig. 4b, the process of selecting VGs and creating an LV specifically includes:
step 401 b: and sequentially selecting and selecting VG from the N VGs according to the sequence that the residual capacity of the VG is from small to large.
Here, in order to ensure that it is possible to create an LV for the second device among a plurality of selected VGs as much as possible in the case where the sum of the remaining capacities of the N VGs is greater than or equal to the total capacity required by the second device, the selected VGs may be sequentially selected from the N VGs in order of the remaining capacities of the VGs from small to small. Of course, the embodiment of the present application is not limited to this manner, and in practical applications, the selection may also be randomly selected or set according to practical requirements, and details are not described here.
Step 402 b: and judging whether the remaining capacity of the selected VG selected currently is smaller than the ratio of the currently required capacity of the second equipment to the number of VGs which do not create the LV in the N VGs.
Wherein the second device currently required capacity is a difference between the total capacity required by the second device and the total capacity of the created LV. The ratio between the currently required capacity of the second device and the number of VGs of the N VGs that did not create a LV may actually be understood as the average of the required capacities when creating a LV on a VG.
If yes, go to step 403 b; if not, go to step 404 b;
step 403 b: the LV is created using the total remaining capacity in the currently selected VG. And then, the steps 401b to 402b can be continuously executed, and the selected VG is continuously selected to create the LV until the judgment result is negative.
Step 404 b: the LV is created using the capacity characterized by the ratio in the currently selected VG.
Further, after the LV is created by using the capacity characterized by the ratio in the currently selected VG, if the total created LV capacity is still smaller than the total capacity required by the second device, the steps 401b to 402b may be executed to continue selecting the selected VG to create the LV until the total capacity of the created LV is equal to the total capacity required by the second device.
Step 303: the first device establishes file systems in the logical volumes respectively, and determines at least two logical volumes with the established file systems as storage areas of the second device connected with the first device.
Here, a file system is created in each LV created for the second device, that is, a process of formatting each LV. Reference is made in particular to the prior art and will not be described in detail here.
Here, in implementation, the storage space of the second device shown to the user is a storage area composed of LVs, and each LV may be managed in the storage area, for example, a directory of a file system is created.
After the second device has been allocated a memory area, it is also possible to subsequently write files or read files in the memory area. Next, a process of writing a file to be stored in the second device into the storage area will be specifically described.
In a specific implementation, when a file is written in the storage area, it may be preferentially selected that the file is written in the LV within the first specified range at the time when the file is most recently written in the storage area. The first designated range can be set by a user according to requirements. Preferably, the LV with the time of the latest written file closer to the current time, that is, the LV on which the write operation is executed last, may be selected to continue to execute the write operation. For example, if 7:00 in the same day performs a write operation in LV1 and 15:00 in LV2, then if a write operation needs to be performed in 19:00, then the file can continue to be written in LV 2. Thus, for files with chronological order, such as video files, files written in the same LV can keep the continuity of the files in time as much as possible. Of course, in actual application, the LV in the storage area may be randomly selected for writing or preset according to actual requirements, which is not limited in the present application.
Further, if it is determined that the LV in the writing state is fully written, it may be switched to other LV write files not fully written in the storage area. Wherein when the remaining capacity of the LV is less than the specified capacity, it is determined that the LV is full.
Further, if it is determined that the other LVs in the storage area are full, the file is written in the specified capacity of the LV in the storage area.
Here, the designated capacity is reserved in the LV, and when the LV in the storage area is full, the file can be written by using the reserved designated capacity, so that the second device can perform normal writing operation. Moreover, for a file system established in the LV, such as an XMethods File System (XFS), a specified capacity is reserved for the file system, which may also reduce the fragmentation rate of the file system.
When a specified failure occurs in the written LV, the writing of the file in the written LV may be stopped, and the LV may be switched to another LV that is not fully written in the storage area. If all other LVs are full, the file can be written by using the specified capacity reserved in each LV, and normal writing of the file in the second device is guaranteed.
The specified fault may be a fault occurring in the LV when the disk is pulled out or damaged, or the like. The designated fault can be detected, specifically, if the LV performs a write operation and a write failure occurs, it may be determined that the LV has the designated fault, or, when a prompt instruction that the disk is pulled out is received, it may be determined that the LV corresponding to the disk has the designated fault. In addition, in specific implementation, a round-robin mechanism may be further employed to sequentially detect the states of the LVs in the storage area.
In addition, if it is determined that the other LVs in the storage area are fully written or it is determined that the designated capacity of the LV in the storage area is fully written, the written file in the LV whose time when the file was most recently written is within the second designated time range may be deleted and switched to the written file in the LV whose time when the file was most recently written is within the second designated time range.
Here, the LV where the time of the most recent write of the file is within the second specified time range may be an LV where the time of the most recent write of the file is farther from the current time, that is, an LV where the write operation has not been performed for a longer time. When it is determined that the other LVs in the storage area are full or the designated capacity of the LV in the storage area is full, part or all of the files stored in the LV on which the write operation has not been performed for a long time may be deleted, and the file may be continuously written in the LV.
Furthermore, after the unplugged disk is reinserted or the damaged disk is successfully repaired, the LV on the disk may still be used by the corresponding second device. Also, when a damaged disk is replaced, a new LV may be created on the new disk for the second device to make up for the lost storage space due to the designated failure of the LV. The specific LV creation process is referred to the above flow and will not be described in detail here.
For convenience of understanding, referring to the schematic diagram of allocating a storage area for a second device shown in fig. 5, a scheme of allocating a storage area for the second device based on an LVM to implement file storage in the embodiment of the present application is illustrated in combination with the foregoing specific embodiment.
Assuming that three disks Disk1, Disk2, Disk3 are configured in the first device, VG1 is created in Disk1, VG2 is created in Disk2, VG3 is created in Disk 3. Further, it is assumed that 3 second devices a to C having a data storage requirement are connected to the first device, respectively. Then the procedure for the first device to allocate storage areas for the second devices a and B is as follows:
for second device a, in allocating storage areas for it, LV1 may be created for second device a on VG1, LV3 may be created for second device a on VG2, where the total capacity of LV1 and LV3 equals the total capacity required by second device a. Further, files in the second device a may be written in the storage area 1 composed of LV1 and LV 3.
For second device B, in allocating storage areas for it, LV2 may be created for second device B on VG1, LV4 on VG2, and LV5 on VG3, where the total capacity of LV2, LV4, and LV5 equals the total capacity required by second device B. Further, files in the second device B may be written in storage area 2 defined by LV2, LV4 and LV 5.
In a specific implementation, if another second device is added, a storage area may be allocated to the added second device by referring to the above example and combining with the implementation manner described in the embodiment of the present application, so as to implement storage of a file in the added second device.
Therefore, by adopting the manner provided by the embodiment of the application, the storage area determined for the second device can actually contain at least two logical volumes, wherein a file system is respectively established in each logical volume, each logical volume corresponds to a selected volume group, and each selected volume group is established by at least one disk, so that even if a disk where a superblock of the file system is located is damaged or pulled out, only the storage space in the logical volume corresponding to the file system is affected, and the storage spaces of other logical volumes in the storage area of the device are not affected, so that read-write operation can be continuously performed on other logical volumes, thereby greatly reducing the influence on the service of reading and writing data of the device due to damage or pulling out of the disk, and ensuring that the device can normally perform read-write operation.
Based on the same application concept, the embodiment of the present application further provides a disk space management device corresponding to the disk space management method, and as the principle of solving the problem of the device is similar to the disk space management method in the embodiment of the present application, the implementation of the device may refer to the implementation of the method, and repeated parts are not described again.
Example two
As shown in fig. 6, a schematic structural diagram of a disk space management device according to a second embodiment of the present application includes:
a volume group creating module 61, configured to create volume groups in a preset plurality of disk groups, respectively, where each disk group includes at least one disk;
a logical volume creating module 62, configured to select at least two volume groups from the volume groups as selected volume groups, and create a logical volume on each selected volume group;
and a storage area determining module 63, configured to respectively establish a file system in the logical volumes, and determine at least two logical volumes with file systems established as storage areas of a second device connected to the first device.
Further, the logical volume creation module 62 is specifically configured to:
sequentially selecting N volume groups from the volume groups corresponding to each disk group according to the sequence of the volume group residual capacity from large to small, wherein N is a positive integer larger than 1;
if the sum of the residual capacities of the N volume groups is smaller than the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from large to small; and the number of the first and second electrodes,
judging whether the residual capacity of the currently selected volume group is smaller than the difference value between the total capacity required by the second equipment and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
and if not, using the capacity represented by the difference value in the currently selected volume group to create a logical volume.
Further, the logical volume creation module 62 is further configured to:
if the sum of the residual capacities of the N volume groups is larger than or equal to the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from small to large; and the number of the first and second electrodes,
judging whether the remaining capacity of the currently selected volume group is smaller than the ratio of the currently required capacity of the second device to the volume group number of the logical volume not created in the N volume groups; wherein the current required capacity of the second device is a difference value between the total capacity required by the second device and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
if not, the capacity represented by the ratio in the currently selected volume group is used for creating the logical volume, and the selected volume group is continuously selected to create the logical volume until the total capacity of the created logical volume is equal to the total capacity required by the second device.
Further, the apparatus further comprises:
a file writing module 64, configured to, after the storage area determining module determines that at least two logical volumes with file systems already established are storage areas of a second device connected to the first device, write a file in a logical volume in which a time of writing a file in the storage area most recently is within a first specified time range;
if the logical volume in the writing state is determined to be full, switching to other logical volumes which are not full in the storage area to write files; when the residual capacity of the logical volume is smaller than the specified capacity, determining that the logical volume is full;
and if the other logical volumes in the storage area are determined to be full, writing files in the designated capacity of the logical volumes in the storage area.
Further, the file writing module 64 is further configured to:
if it is determined that the other logical volumes in the storage area are full or the designated capacity of the logical volume in the storage area is full, the first device deletes the written file in the logical volume whose time for writing the file is recently in the second designated time range, and switches to the logical volume whose time for writing the file is recently in the second designated time range to write the file.
Further, the file writing module 64 is further configured to:
if the logic volume in the writing state is determined to generate the designated fault, stopping writing the file in the logic volume in the writing state; and switching to other unfilled logical volumes in the storage area to write files.
An embodiment of the present application further provides an electronic device, including: one or more processors; and one or more computer-readable media having stored thereon a program for executing the method for disk space management, wherein the program when executed by the one or more processors implements the steps of the method as described in embodiment one.
Embodiments of the present application also provide one or more computer-readable media having stored thereon a program for executing the disk space management method, wherein the program, when executed by one or more processors, causes a communication device to execute the method as described in embodiment one.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (12)
1. A disk space management method, characterized in that the method comprises:
the method comprises the steps that a first device respectively creates volume groups in a plurality of preset disk groups, wherein each disk group comprises at least one disk;
the first device selects at least two volume groups from the volume groups as selected volume groups and respectively creates a logical volume on each selected volume group;
the first device establishes a file system in the logical volumes, and determines at least two logical volumes with the established file systems as storage areas of a second device connected with the first device; and ensuring that the logical volume is created for the second device in 2 or more volume groups, and enabling the sum of the created logical volume capacities to be equal to the total capacity required by the second device.
2. The method of claim 1, wherein the first device selects at least two of the volume groups as selected volume groups and creates a logical volume on each of the selected volume groups, respectively, specifically comprising:
selecting N volume groups from the volume groups respectively corresponding to each disk group according to the sequence of the volume group residual capacity from large to small, wherein N is a positive integer larger than 1;
if the sum of the residual capacities of the N volume groups is smaller than the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from large to small;
judging whether the residual capacity of the currently selected volume group is smaller than the difference value between the total capacity required by the second equipment and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
and if not, using the capacity represented by the difference value in the currently selected volume group to create a logical volume.
3. The method of claim 2, wherein after the first device selects N volume groups from the volume groups respectively corresponding to the disk groups, the method further comprises:
if the sum of the residual capacities of the N volume groups is larger than or equal to the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from small to large; and the number of the first and second electrodes,
judging whether the remaining capacity of the currently selected volume group is smaller than the ratio of the currently required capacity of the second device to the volume group number of the logical volume not created in the N volume groups; wherein the current required capacity of the second device is a difference value between the total capacity required by the second device and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
if not, the capacity represented by the ratio in the currently selected volume group is used for creating the logical volume, and the selected volume group is continuously selected to create the logical volume until the total capacity of the created logical volume is equal to the total capacity required by any second device.
4. The method of claim 1, wherein the first device, after determining at least two logical volumes having an established file system as storage areas of a second device connected to the first device, further comprises:
writing files in the logical volume of which the time of the latest file writing in the storage area is within a first specified time range;
if the logical volume in the writing state is determined to be full, switching to other logical volumes which are not full in the storage area to write files; when the residual capacity of the logical volume is smaller than the specified capacity, determining that the logical volume is full;
and if the other logical volumes in the storage area are determined to be full, writing files in the designated capacity of the logical volumes in the storage area.
5. The method of claim 4, wherein the method further comprises:
if it is determined that the other logical volumes in the storage area are full or the designated capacity of the logical volume in the storage area is full, the first device deletes the written file in the logical volume whose time for writing the file is recently in the second designated time range, and switches to the logical volume whose time for writing the file is recently in the second designated time range to write the file.
6. The method according to claim 4 or 5, wherein if the first device determines that the logical volume in the write state has the designated failure, the method further comprises:
stopping writing files in the logical volume in the writing state; and the number of the first and second electrodes,
and switching to other unfilled logical volumes in the storage area to write files.
7. A disk space management apparatus, comprising:
the volume group creation module is used for respectively creating volume groups in a plurality of preset disk groups, wherein each disk group comprises at least one disk;
the logical volume creating module is used for selecting at least two volume groups from the volume groups as selected volume groups and respectively creating logical volumes on each selected volume group;
the storage area determining module is used for establishing a file system in the logical volumes and determining at least two logical volumes with the established file systems as storage areas of second equipment connected with the first equipment; and ensuring that the logical volume is created for the second device in 2 or more volume groups, and enabling the sum of the created logical volume capacities to be equal to the total capacity required by the second device.
8. The apparatus of claim 7, wherein the logical volume creation module is specifically configured to:
sequentially selecting N volume groups from the volume groups corresponding to each disk group according to the sequence of the volume group residual capacity from large to small, wherein N is a positive integer larger than 1;
if the sum of the residual capacities of the N volume groups is smaller than the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from large to small; and the number of the first and second electrodes,
judging whether the residual capacity of the currently selected volume group is smaller than the difference value between the total capacity required by the second equipment and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
and if not, using the capacity represented by the difference value in the currently selected volume group to create a logical volume.
9. The apparatus of claim 8, wherein the logical volume creation module is further to:
if the sum of the residual capacities of the N volume groups is larger than or equal to the total capacity required by the second equipment, sequentially selecting a selected volume group from the N volume groups according to the sequence of the residual capacities of the volume groups from small to large; and the number of the first and second electrodes,
judging whether the remaining capacity of the currently selected volume group is smaller than the ratio of the currently required capacity of the second device to the volume group number of the logical volume not created in the N volume groups; wherein the current required capacity of the second device is a difference value between the total capacity required by the second device and the total capacity of the created logical volume;
if so, using all residual capacity in the currently selected volume group to create a logical volume, and continuing to select the selected volume group to create the logical volume until the judgment result is negative;
if not, the capacity represented by the ratio in the currently selected volume group is used for creating the logical volume, and the selected volume group is continuously selected to create the logical volume until the total capacity of the created logical volume is equal to the total capacity required by any second device.
10. The apparatus of claim 7, wherein the apparatus further comprises:
a file writing module, configured to, after the storage area determining module determines that at least two logical volumes with file systems established are storage areas of a second device connected to the first device, write a file in a logical volume in which the time for writing the file in the storage area most recently is within a first specified time range;
if the logical volume in the writing state is determined to be full, switching to other logical volumes which are not full in the storage area to write files; when the residual capacity of the logical volume is smaller than the specified capacity, determining that the logical volume is full;
and if the other logical volumes in the storage area are determined to be full, writing files in the designated capacity of the logical volumes in the storage area.
11. The apparatus of claim 10, wherein the file writing module is further to:
if it is determined that the other logical volumes in the storage area are full or the designated capacity of the logical volume in the storage area is full, the first device deletes the written file in the logical volume whose time for writing the file is recently in the second designated time range, and switches to the logical volume whose time for writing the file is recently in the second designated time range to write the file.
12. The apparatus of claim 10 or 11, wherein the file writing module is further to:
if the logic volume in the writing state is determined to generate the designated fault, stopping writing the file in the logic volume in the writing state; and switching to other unfilled logical volumes in the storage area to write files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711270240.3A CN108037894B (en) | 2017-12-05 | 2017-12-05 | Disk space management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711270240.3A CN108037894B (en) | 2017-12-05 | 2017-12-05 | Disk space management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108037894A CN108037894A (en) | 2018-05-15 |
| CN108037894B true CN108037894B (en) | 2021-07-20 |
Family
ID=62095727
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711270240.3A Active CN108037894B (en) | 2017-12-05 | 2017-12-05 | Disk space management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108037894B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109600600B (en) * | 2018-10-31 | 2020-11-03 | 万维科研有限公司 | Encoder, encoding method, and storage method and format of three-layer expression relating to depth map conversion |
| CN110716884B (en) * | 2019-09-20 | 2023-11-17 | 深圳市网心科技有限公司 | Fragment data storage method, electronic equipment and system |
| CN110990302B (en) * | 2019-11-22 | 2021-11-02 | 北京云宽志业网络技术有限公司 | Data caching method and device, electronic equipment and storage medium |
| CN114840148B (en) * | 2022-06-30 | 2022-09-06 | 江苏博云科技股份有限公司 | Method for realizing disk acceleration based on linux kernel bcache technology in Kubernets |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101082853A (en) * | 2006-05-31 | 2007-12-05 | 株式会社日立制作所 | Flash memory storage system |
| JP2008003857A (en) * | 2006-06-22 | 2008-01-10 | Fujitsu Ltd | Method and program for expanding capacity of storage device, and storage device |
| CN102929786A (en) * | 2011-09-12 | 2013-02-13 | 微软公司 | Volatile memory representation of nonvolatile storage device set |
| CN103677663A (en) * | 2012-08-30 | 2014-03-26 | 想象力科技有限公司 | Memory address generation for digital signal processing |
| CN103885718A (en) * | 2012-12-21 | 2014-06-25 | 刘庆国 | Mirror disk increment synchronizing method based on lvm |
| CN103970486A (en) * | 2014-05-22 | 2014-08-06 | 浪潮(北京)电子信息产业有限公司 | Method and system for data storage |
| CN105068894A (en) * | 2015-09-10 | 2015-11-18 | 浪潮(北京)电子信息产业有限公司 | Method and device for achieving high availability of Linux system storage device on basis of LVM mirror image |
| CN106201357A (en) * | 2016-07-18 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The construction method of a kind of storage pool and system |
-
2017
- 2017-12-05 CN CN201711270240.3A patent/CN108037894B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101082853A (en) * | 2006-05-31 | 2007-12-05 | 株式会社日立制作所 | Flash memory storage system |
| JP2008003857A (en) * | 2006-06-22 | 2008-01-10 | Fujitsu Ltd | Method and program for expanding capacity of storage device, and storage device |
| CN102929786A (en) * | 2011-09-12 | 2013-02-13 | 微软公司 | Volatile memory representation of nonvolatile storage device set |
| CN103677663A (en) * | 2012-08-30 | 2014-03-26 | 想象力科技有限公司 | Memory address generation for digital signal processing |
| CN103885718A (en) * | 2012-12-21 | 2014-06-25 | 刘庆国 | Mirror disk increment synchronizing method based on lvm |
| CN103970486A (en) * | 2014-05-22 | 2014-08-06 | 浪潮(北京)电子信息产业有限公司 | Method and system for data storage |
| CN105068894A (en) * | 2015-09-10 | 2015-11-18 | 浪潮(北京)电子信息产业有限公司 | Method and device for achieving high availability of Linux system storage device on basis of LVM mirror image |
| CN106201357A (en) * | 2016-07-18 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The construction method of a kind of storage pool and system |
Non-Patent Citations (1)
| Title |
|---|
| 面向虚拟机的分布式块存储***设计及实现;贾博文;《中国优秀硕士学位论文全文数据库信息科技辑》;20160315(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108037894A (en) | 2018-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107807794B (en) | Data storage method and device | |
| CN108037894B (en) | Disk space management method and device | |
| US11392307B2 (en) | Data-protection-aware capacity provisioning of shared external volume | |
| CN109542342B (en) | Metadata management and data reconstruction method, equipment and storage medium | |
| US9122787B2 (en) | Method and apparatus to utilize large capacity disk drives | |
| US9798471B2 (en) | Performance of de-clustered disk array by disk grouping based on I/O statistics | |
| US11243706B2 (en) | Fragment management method and fragment management apparatus | |
| US8850152B2 (en) | Method of data migration and information storage system | |
| US10852966B1 (en) | System and method for creating mapped RAID group during expansion of extent pool | |
| US9471450B2 (en) | Reducing data loss in a computing storage environment | |
| US9710345B2 (en) | Using unused portion of the storage space of physical storage devices configured as a RAID | |
| US20160048342A1 (en) | Reducing read/write overhead in a storage array | |
| US20110145528A1 (en) | Storage apparatus and its control method | |
| US10324794B2 (en) | Method for storage management and storage device | |
| US20110107028A1 (en) | Dynamically Expanding Storage Capacity of a Storage Volume | |
| CN103500072A (en) | Data migration method and data migration device | |
| US20150293719A1 (en) | Storage Space Processing Method and Apparatus, and Non-Volatile Computer Readable Storage Medium | |
| CN103534688A (en) | Data recovery method, storage equipment and storage system | |
| US11347414B2 (en) | Using telemetry data from different storage systems to predict response time | |
| US8479040B2 (en) | Storage system and control method | |
| CN103176745A (en) | Hard disc array takeover method of storage system with double controllers | |
| CN105573862B (en) | Method and equipment for recovering file system | |
| US11860746B2 (en) | Resilient data storage system with efficient space management | |
| US11507458B2 (en) | Storage management method, device, and computer program product | |
| WO2015085802A1 (en) | Data storage method and storage apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |