CN114020214A - Storage cluster capacity expansion method and device, electronic equipment and readable storage medium - Google Patents
Storage cluster capacity expansion method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN114020214A CN114020214A CN202111271843.1A CN202111271843A CN114020214A CN 114020214 A CN114020214 A CN 114020214A CN 202111271843 A CN202111271843 A CN 202111271843A CN 114020214 A CN114020214 A CN 114020214A
- Authority
- CN
- China
- Prior art keywords
- storage cluster
- capacity expansion
- information
- node
- resource information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000015654 memory Effects 0.000 claims description 33
- 238000013468 resource allocation Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 description 8
- 238000005192 partition Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention relates to the technical field of distributed storage, and discloses a storage cluster capacity expansion method and device, electronic equipment and a readable storage medium. Wherein, the method comprises the following steps: acquiring first resource information of a storage cluster and second resource information of at least one node to be expanded; analyzing the first resource information and determining the use data of the storage cluster; and generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information. By implementing the invention, the automatic capacity expansion of the storage cluster is realized, the operation and maintenance manpower is greatly saved, the workload during the capacity expansion of the large-scale storage cluster is reduced, the capacity expansion requirement of a large number of nodes to be expanded can be met to a greater extent, the capacity expansion failure caused by the complicated manual capacity expansion operation is avoided, and the capacity expansion success rate of the storage cluster is improved.
Description
Technical Field
The invention relates to the technical field of distributed storage, in particular to a storage cluster capacity expansion method and device, electronic equipment and a readable storage medium.
Background
With the explosive growth of massive unstructured data, distributed storage gradually becomes a storage base stone for users to construct a data center architecture, and more key services are accessed to the distributed storage. However, as the traffic increases, the storage capacity of the in-use distributed storage cluster may cause resource shortage, and the capacity of the distributed storage cluster needs to be increased by node expansion to meet the traffic demand.
At present, capacity expansion of a distributed storage cluster is mainly performed by manually designating Monitor (MON) nodes and metadata server (MDS) nodes according to IP address information of nodes to be expanded, which is input by a user, manually partitioning disks, designating a use method for each disk or partition, and finally expanding the expansion nodes to the storage cluster. However, when the number of nodes to be expanded is large, the manual expansion operation is very complicated and error is easily caused, thereby causing the failure of expansion.
Disclosure of Invention
In view of this, embodiments of the present invention provide a capacity expansion method and apparatus for a storage cluster, an electronic device, and a readable storage medium, so as to solve the problem that when the number of nodes to be capacity expanded is large, the manual capacity expansion operation is very complicated and error is prone to occur, thereby causing capacity expansion failure.
According to a first aspect, an embodiment of the present invention provides a method for capacity expansion of a storage cluster, including: acquiring first resource information of a storage cluster and second resource information of at least one node to be expanded; analyzing the first resource information and determining the use data of the storage cluster; and generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
In the capacity expansion method for the storage cluster provided in the embodiment of the present invention, by acquiring the first resource information of the storage cluster and the second resource information of the at least one node to be expanded, the first resource information is analyzed to determine the usage data of the storage cluster, and then capacity expansion configuration information of the storage cluster is generated according to the usage data of the storage cluster and the second resource information, so as to perform capacity expansion operation on the storage cluster according to the capacity expansion configuration information. The method realizes the automatic capacity expansion of the storage cluster, can meet the capacity expansion requirement of a large number of nodes to be expanded to a greater extent, greatly saves operation and maintenance manpower by automatically and reasonably distributing resource information according to capacity expansion configuration information, reduces the workload during the capacity expansion of the large-scale storage cluster, avoids capacity expansion failure caused by complicated manual capacity expansion operation, and improves the capacity expansion success rate of the storage cluster.
With reference to the first aspect, in a first implementation manner of the first aspect, the analyzing the first resource information of the storage cluster and determining the usage data of the storage cluster includes: analyzing storage pool capacity information, disk writing information and service starting information corresponding to the storage cluster; and respectively determining the total capacity and the use state of each storage pool corresponding to the storage cluster, the number of times of writing the disk and the starting state of each service based on the storage pool capacity information, the disk writing information and the service starting information.
With reference to the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the generating, based on the usage data of the storage cluster and the second resource information, capacity expansion configuration information of the storage cluster includes: determining capacity expansion data corresponding to the storage cluster based on the use data of the storage cluster; and distributing the second resource information according to the expansion data to generate the expansion configuration information.
According to the capacity expansion method of the storage cluster provided by the embodiment of the invention, the total capacity and the use state of each storage pool corresponding to the storage cluster, the number of times of writing the disk and the starting state of each service are respectively determined by analyzing the capacity information, the disk writing information and the service starting information of the storage pool corresponding to the storage cluster. Based on the total capacity and the use state of each storage pool corresponding to the storage cluster, the write-in times of the disk and the starting state of each service, capacity expansion data corresponding to the storage cluster are determined, and second resource information is distributed according to the capacity expansion data to generate capacity expansion configuration information, so that the generation accuracy of the capacity expansion configuration information is ensured, and the capacity expansion success rate of the storage cluster is further improved.
With reference to the first aspect, in a third implementation manner of the first aspect, the obtaining second resource information of the at least one node to be expanded includes: responding to an input instruction of the node to be expanded to obtain the information of the at least one node to be expanded; analyzing the information of the at least one node to be expanded, and determining second resource information corresponding to the at least one node to be expanded.
According to the capacity expansion method of the storage cluster provided by the embodiment of the invention, at least one piece of information of the node to be expanded is obtained by responding to the input instruction of the node to be expanded, the obtained at least one piece of information of the node to be expanded is analyzed, and the second resource information corresponding to the at least one piece of node to be expanded is determined, so that the convenience of automatic capacity expansion is realized, and the operation and maintenance personnel are not required to perform the complicated operation of capacity expansion.
With reference to the first aspect, in a fourth implementation of the first aspect, the method further includes: deleting original resource information of the at least one node to be expanded to obtain current resource information corresponding to the at least one node to be expanded; and carrying out monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource allocation according to the expansion configuration information.
With reference to the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the allocating disk resources according to the capacity expansion configuration information includes: acquiring the use frequency and the use state of each storage pool corresponding to the storage cluster; and dynamically allocating the disk resources to the storage pools based on the use frequency and the use state of the storage pools.
According to the capacity expansion method of the storage cluster, the original resource information of at least one node to be expanded is deleted, so that the current resource information corresponding to the at least one node to be expanded is obtained, and monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource distribution are performed according to the capacity expansion configuration information, so that the situation that the performance of the current storage cluster is influenced by the fact that the original cluster service and configuration information of the node to be expanded, which is separated from other clusters, is reserved is avoided, and the performance of the storage cluster is guaranteed.
With reference to the fourth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the performing, according to the capacity expansion configuration information, a configuration of a metadata base service node includes: judging whether the storage cluster is a file storage cluster or not; and when the storage cluster is a file storage cluster, configuring a plurality of metadata base service nodes based on the capacity expansion configuration information.
In the capacity expansion method for the storage cluster provided in the embodiment of the present invention, whether the storage cluster is a file storage cluster is determined, and when the storage cluster is the file storage cluster, the plurality of metadata base service nodes are configured based on capacity expansion configuration information. According to the method, the type of the storage cluster is judged so as to configure the metadatabase service node for the storage cluster, high availability of the storage cluster is guaranteed, and the storage cluster after capacity expansion can achieve the best performance.
According to a second aspect, an embodiment of the present invention provides an apparatus for capacity expansion of a storage cluster, including: the acquisition module is used for acquiring first resource information of the storage cluster and second resource information of at least one node to be expanded; the analysis module is used for analyzing the first resource information and determining the use data of the storage cluster; and the generating module is used for generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
The capacity expansion device for a storage cluster according to the embodiment of the present invention analyzes first resource information of the storage cluster and second resource information of at least one node to be expanded to determine usage data of the storage cluster by obtaining the first resource information, and then generates capacity expansion configuration information of the storage cluster according to the usage data of the storage cluster and the second resource information, so as to perform capacity expansion operation on the storage cluster according to the capacity expansion configuration information. The device realizes the automatic capacity expansion of the storage cluster, can meet the capacity expansion requirement of a large number of nodes to be expanded to a greater extent, greatly saves operation and maintenance manpower by automatically and reasonably distributing resource information according to capacity expansion configuration information, reduces the workload during the capacity expansion of the large-scale storage cluster, avoids capacity expansion failure caused by complicated manual capacity expansion operation, and improves the capacity expansion success rate of the storage cluster.
According to a third aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method for expanding a storage cluster according to the first aspect or any embodiment of the first aspect.
According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where computer instructions are stored, and the computer instructions are configured to cause a computer to execute the method for capacity expansion of a storage cluster according to the first aspect or any implementation manner of the first aspect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a method of capacity expansion of a storage cluster according to an embodiment of the invention;
FIG. 2 is another flow diagram of a method for capacity expansion of a storage cluster according to an embodiment of the present invention;
FIG. 3 is another flow diagram of a method for capacity expansion of a storage cluster according to an embodiment of the present invention;
FIG. 4 is a block diagram of an embodiment of an apparatus for extending capacity of a storage cluster;
fig. 5 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, capacity expansion of a distributed storage cluster is mainly performed by manually designating Monitor (MON) nodes and metadata server (MDS) nodes according to IP address information of nodes to be expanded, which is input by a user, manually partitioning disks, designating a use method for each disk or partition, and the like, and finally expanding the expansion nodes to the storage cluster. However, when the number of nodes to be expanded is large, the manual expansion operation is very complicated and error is easily caused, thereby causing the failure of expansion.
Based on the technical scheme, the resource information of the node to be expanded is automatically and reasonably and automatically distributed to the current storage cluster according to the service condition of the current storage cluster and the resource information of the node to be expanded, so that the automatic expansion of the storage cluster is realized, the operation and maintenance manpower is saved, and the probability of expansion errors caused by the complicated manual expansion operation is reduced.
In accordance with an embodiment of the present invention, there is provided an embodiment of a method for capacity expansion of a storage cluster, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
In this embodiment, a storage cluster capacity expansion method is provided, which may be used in electronic devices, such as computers, servers, and the like, fig. 1 is a flowchart of a storage cluster capacity expansion method according to an embodiment of the present invention, and as shown in fig. 1, the flowchart includes the following steps:
s11, acquiring first resource information of the storage cluster and second resource information of at least one node to be expanded.
The first resource information is current resource information of the storage cluster, and the first resource information may include total capacity of each storage pool, disk information, each service status, and the like. The electronic device can monitor the use state of the storage cluster in real time, and determine corresponding first resource information according to the use state updated in real time.
The second resource information is resource information of the node to be expanded, and the second resource information may include a Central Processing Unit (CPU), a memory size, a disk type, the number of disks of various types, capacity information thereof, and the like corresponding to the node to be expanded. The electronic device may determine second resource information corresponding to the node to be expanded by analyzing the attribute information of the node to be expanded.
S12, analyzing the first resource information and determining the use data of the storage cluster.
The usage data is used to characterize usage of the storage cluster resource information. After the electronic device obtains the first resource information, the usage of the first resource information may be analyzed to determine the usage (remaining capacity) of each storage pool corresponding to the storage cluster, the number of times of writing to the disk, the on state or off state of each service, and the like.
And S13, generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
The capacity expansion configuration information is used for reasonably distributing the second resource information to the storage cluster, and the capacity expansion configuration information may include MON node configuration, MDS node configuration, disk partitioning, and the like. The electronic device may reasonably distribute the second resource information corresponding to the to-be-expanded node according to the usage data of the storage cluster, and generate expansion configuration information of the storage cluster. For example, the electronic device may determine the disk division ratio according to the frequency of the number of read and write operations of the disk, may determine the allocation of the storage space according to the usage and the usage frequency of the storage pool, and the like.
In the capacity expansion method of the storage cluster provided in this embodiment, the first resource information of the storage cluster and the second resource information of the at least one node to be subjected to capacity expansion are acquired, the first resource information is analyzed to determine the usage data of the storage cluster, and then capacity expansion configuration information of the storage cluster is generated according to the usage data of the storage cluster and the second resource information, so as to perform capacity expansion operation on the storage cluster according to the capacity expansion configuration information. The method realizes the automatic capacity expansion of the storage cluster, can meet the capacity expansion requirement of a large number of nodes to be expanded to a greater extent, greatly saves operation and maintenance manpower by automatically and reasonably distributing resource information according to capacity expansion configuration information, reduces the workload during the capacity expansion of the large-scale storage cluster, avoids capacity expansion failure caused by complicated manual capacity expansion operation, and improves the capacity expansion success rate of the storage cluster.
In this embodiment, a storage cluster capacity expansion method is provided, which may be used in electronic devices, such as computers, servers, and the like, fig. 2 is a flowchart of a storage cluster capacity expansion method according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps:
s21, acquiring first resource information of the storage cluster and second resource information of at least one node to be expanded. For a detailed description, refer to the related description of the corresponding steps in the above embodiments, which are not repeated herein.
S22, analyzing the first resource information and determining the use data of the storage cluster.
Specifically, the step S22 may include:
s221, analyzing the storage pool capacity information, the disk writing information and the service starting information corresponding to the storage cluster.
The storage pool capacity information is used for representing the use condition and the use frequency of the storage pool, the disk writing information is used for representing the use condition and the read-write frequency of the disk, and the service opening information is used for representing whether the service is opened or not, namely whether the service is in a running state or not.
S222, respectively determining the total capacity and the use state of each storage pool corresponding to the storage cluster, the number of times of disk writing and the starting state of each service based on the storage pool capacity information, the disk writing information and the service starting information.
The total capacity and the capacity use condition of each current storage pool can be determined by analyzing the storage pool capacity information corresponding to the storage cluster; the write-in times of each type of current disk can be determined by analyzing the write-in information of the disk; whether each service of the storage cluster is in an on state can be determined by analyzing the service opening information so as to determine whether each service is in a normal operation state.
And S23, generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
Specifically, the step S23 may include:
s231, capacity expansion data corresponding to the storage cluster is determined based on the use data of the storage cluster.
The capacity expansion data is the capacity expansion proportion of the resource information corresponding to the storage cluster, such as the configuration proportion of the MON node, the configuration proportion of the MDS node, the partition proportion of the disk, the allocation proportion of the storage space, and the like. After the electronic device determines the usage data of the storage cluster, it may determine corresponding expansion data according to the usage data.
And S232, distributing the second resource information according to the expansion data to generate expansion configuration information.
After the capacity expansion data is determined, the electronic device may perform corresponding allocation on the second resource information according to the determined capacity expansion data of the storage cluster to obtain capacity expansion configuration information corresponding to the storage cluster, so that the electronic device may perform reasonable allocation on the second resource information according to the capacity expansion configuration information. The expansion configuration information includes MON node configuration, MDS node configuration, Cluster Database (CTDB) configuration, disk resource allocation, and the like.
Optionally, when the capacity expansion configuration information is generated, the electronic device may display the capacity expansion configuration information through a management interface thereof, and if the user is not satisfied with the currently generated capacity expansion configuration information, the user may adjust the current capacity expansion configuration information according to an actual requirement, so that the capacity expansion configuration information can satisfy the capacity expansion requirement of the storage cluster.
In the capacity expansion method for the storage cluster provided in this embodiment, total capacity and usage status of each storage pool corresponding to the storage cluster, the number of times of writing to the disk, and the start status of each service are respectively determined by analyzing the storage pool capacity information, the disk writing information, and the service start information corresponding to the storage cluster. Based on the total capacity and the use state of each storage pool corresponding to the storage cluster, the write-in times of the disk and the starting state of each service, capacity expansion data corresponding to the storage cluster are determined, and second resource information is distributed according to the capacity expansion data to generate capacity expansion configuration information, so that the generation accuracy of the capacity expansion configuration information is ensured, and the capacity expansion success rate of the storage cluster is further improved.
In this embodiment, a storage cluster capacity expansion method is provided, which may be used in electronic devices, such as computers, servers, and the like, fig. 3 is a flowchart of a storage cluster capacity expansion method according to an embodiment of the present invention, and as shown in fig. 3, the flowchart includes the following steps:
s31, acquiring first resource information of the storage cluster and second resource information of at least one node to be expanded.
Specifically, the step S31 may include:
s311, first resource information of the storage cluster is obtained. For a detailed description of obtaining the first resource information of the storage cluster, refer to the relevant description corresponding to the above embodiment, which is not described herein again.
S312, responding to the input instruction of the node to be expanded, and obtaining information of at least one node to be expanded.
The user can input one or more nodes to be expanded through a storage cluster management interface of the electronic device, and correspondingly, the electronic device can respond to an input instruction of the user to the nodes to be expanded and acquire corresponding information of the nodes to be expanded. The node information to be expanded is information used for reflecting the uniqueness of the node to be expanded, so that the electronic device can accurately position the node to be expanded, for example, the IP address information corresponding to the node to be expanded, the physical address corresponding to the node to be expanded, and the like.
S313, analyzing the information of the at least one node to be expanded, and determining second resource information corresponding to the at least one node to be expanded.
After the electronic device obtains the information of the node to be expanded, the information of the node to be expanded can be analyzed to accurately locate the node to be expanded, and second resource information such as the corresponding CPI, the memory size, the software version, the disk type, the number of various disks, the capacity information and the like is determined. For example, the electronic device obtains the memory, the disk type, the number of disks, the disk capacity, and the like corresponding to the node to be expanded according to the input IP address information corresponding to the node to be expanded.
S32, analyzing the first resource information and determining the use data of the storage cluster. For a detailed description, refer to the corresponding related description of the above embodiments, which is not repeated herein.
And S33, generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information. For a detailed description, refer to the corresponding related description of the above embodiments, which is not repeated herein.
And S34, deleting the original resource information of at least one node to be expanded to obtain the current resource information corresponding to the at least one node to be expanded.
The original resource information is service information, configuration information and the like in an original storage cluster reserved by the node to be expanded. Since the newly added node may be separated from other storage clusters, in order to avoid that the original resource information retained by the newly added node has a bad influence on the current storage cluster, the original resource information of each node to be expanded needs to be cleared at this time, so as to obtain the node to be expanded which is cleared by the original resource information, and determine the current resource information of each node to be expanded.
And S35, performing monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource allocation according to the capacity expansion configuration information.
After the capacity expansion configuration information is determined and the removal of the original resource information of the node to be expanded is completed, the electronic equipment can perform capacity expansion configuration such as monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource distribution on the current storage cluster according to the determined capacity expansion configuration information, and automatically complete the capacity expansion of the storage cluster without human intervention.
Specifically, when configuring the MON node of the storage cluster, the system time of the current storage cluster is synchronized to each node to be expanded.
Specifically, the allocating the disk resources according to the capacity expansion configuration information includes:
(1) and acquiring the use frequency and the use state of each storage pool corresponding to the storage cluster.
The electronic device may monitor the usage of each storage pool corresponding to the storage cluster in real time, and determine the usage frequency and usage status corresponding to each storage pool. Specifically, a monitoring period may be set, and the usage status of each storage pool in the monitoring period may be calculated to calculate the usage frequency and usage status of the storage pool.
(2) And dynamically allocating the disk resources to the storage pools based on the use frequency and the use state of the storage pools.
And automatically dividing the disks according to the use frequency and the use state of each current storage pool to obtain a plurality of disk partitions. For example, the read-write performance of the SSD disk is better, and the SSD disk can be used as a cache, and the effect of fully utilizing the disk space is achieved by dividing the disk partition. After the disk partition is completed, the electronic device may automatically allocate the disk partitions to the storage cluster according to the capacity information corresponding to each disk partition and the disk type, so as to complete the capacity expansion of the storage space of the storage cluster.
Specifically, the metadata base service node configuration according to the capacity expansion configuration information includes:
(1) and judging whether the storage cluster is a file storage cluster.
When the metadata base service MDS node configuration is performed, the electronic device may analyze the type of the current Storage cluster to determine whether the Storage cluster is a file Attached Storage (NAS) cluster, and when the Storage cluster is the file Storage cluster, execute step (2), otherwise perform corresponding MDS node configuration according to the type of the current Storage cluster, for example, when the Storage cluster is a Storage Area Network (SAN) cluster or an object Storage (OBjcet) cluster, it is not necessary to configure the metadata base service node.
(2) And configuring a plurality of metadata base service nodes based on the capacity expansion configuration information.
The metadata base service node is used for providing metadata for storing data. When the storage cluster is a file storage cluster, because a plurality of nodes are required to provide a service for storing data in the NAS cluster, the electronic device may select a plurality of metadata repository service nodes from the NAS cluster according to the generated capacity expansion configuration information, so as to configure the plurality of metadata repository service nodes.
Specifically, when performing database configuration of the storage cluster, that is, CTDB configuration, the electronic device may configure the CTDB of the current storage cluster according to the CTDB configuration information of the original storage cluster, and automatically configure a virtual IP and a node IP, so as to ensure high availability of the storage cluster.
Meanwhile, the electronic equipment can also synchronously manage the soft configuration files, start each service of the nodes to be expanded, and add a starting item to complete the automatic expansion of the storage cluster.
In the capacity expansion method of the storage cluster provided by this embodiment, at least one piece of information of a node to be expanded is obtained by responding to an input instruction of the node to be expanded, the obtained at least one piece of information of the node to be expanded is analyzed, and second resource information corresponding to the at least one piece of node to be expanded is determined, so that convenience in automatic capacity expansion is achieved, and operation and maintenance personnel are not required to perform complicated operations of capacity expansion. The method comprises the steps of obtaining current resource information corresponding to at least one node to be expanded by deleting original resource information of the at least one node to be expanded, and carrying out monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource allocation according to the expansion configuration information, so that the influence on the performance of a current storage cluster caused by the fact that original cluster service and configuration information are reserved on the node to be expanded, which is separated from other clusters, is avoided, and the performance of the storage cluster is guaranteed. The type of the storage cluster is judged so as to configure the metadatabase service node for the storage cluster, thereby ensuring the high availability of the storage cluster and enabling the storage cluster after capacity expansion to achieve the best performance.
In this embodiment, a capacity expansion apparatus for a storage cluster is further provided, where the apparatus is used to implement the foregoing embodiments and preferred embodiments, and details are not repeated after the description is given. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
This embodiment provides an expansion apparatus for a storage cluster, as shown in fig. 4, including:
the obtaining module 41 is configured to obtain first resource information of the storage cluster and second resource information of at least one node to be expanded. The detailed description refers to the corresponding related description of the above method embodiments.
And the analysis module 42 is configured to analyze the first resource information and determine usage data of the storage cluster. The detailed description refers to the corresponding related description of the above method embodiments.
And a generating module 43, configured to generate capacity expansion configuration information of the storage cluster based on the usage data of the storage cluster and the second resource information. The detailed description refers to the corresponding related description of the above method embodiments.
The capacity expansion device of the storage cluster provided in this embodiment analyzes the first resource information to determine the usage data of the storage cluster by obtaining the first resource information of the storage cluster and the second resource information of at least one node to be expanded, and then generates capacity expansion configuration information of the storage cluster according to the usage data of the storage cluster and the second resource information, so as to perform capacity expansion operation on the storage cluster according to the capacity expansion configuration information. The device realizes the automatic capacity expansion of the storage cluster, can meet the capacity expansion requirement of a large number of nodes to be expanded to a greater extent, greatly saves operation and maintenance manpower by automatically and reasonably distributing resource information according to capacity expansion configuration information, reduces the workload during the capacity expansion of the large-scale storage cluster, avoids capacity expansion failure caused by complicated manual capacity expansion operation, and improves the capacity expansion success rate of the storage cluster.
The storage cluster expansion means in this embodiment is in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and a memory executing one or more software or fixed programs, and/or other devices capable of providing the above functions.
Further functional descriptions of the modules are the same as those of the corresponding embodiments, and are not repeated herein.
An embodiment of the present invention further provides an electronic device, which includes the capacity expansion device of the storage cluster shown in fig. 4.
Referring to fig. 5, fig. 5 is a schematic structural diagram of an electronic device according to an alternative embodiment of the present invention, and as shown in fig. 5, the electronic device may include: at least one processor 501, such as a CPU (Central Processing Unit), at least one communication interface 503, memory 504, and at least one communication bus 502. Wherein a communication bus 502 is used to enable connective communication between these components. The communication interface 503 may include a Display (Display) and a Keyboard (Keyboard), and the optional communication interface 503 may also include a standard wired interface and a standard wireless interface. The Memory 504 may be a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The memory 504 may optionally be at least one storage device located remotely from the processor 501. Wherein the processor 501 may be in connection with the apparatus described in fig. 4, an application program is stored in the memory 504, and the processor 501 calls the program code stored in the memory 504 for performing any of the above-mentioned method steps.
The communication bus 502 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 502 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.
The memory 504 may include a volatile memory (RAM), such as a random-access memory (RAM); the memory may also include a non-volatile memory (english: non-volatile memory), such as a flash memory (english: flash memory), a hard disk (english: hard disk drive, abbreviated: HDD) or a solid-state drive (english: SSD); the memory 504 may also comprise a combination of the above types of memory.
The processor 501 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of CPU and NP.
The processor 501 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.
Optionally, the memory 504 is also used to store program instructions. The processor 501 may call a program instruction to implement the method for capacity expansion of a storage cluster as shown in the embodiments of fig. 1 to fig. 3 in this application.
The embodiment of the invention also provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the processing method of the capacity expansion method of the storage cluster in any method embodiment. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (10)
1. A method for capacity expansion of a storage cluster, comprising:
acquiring first resource information of a storage cluster and second resource information of at least one node to be expanded;
analyzing the first resource information and determining the use data of the storage cluster;
and generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
2. The method of claim 1, wherein analyzing the first resource information of the storage cluster to determine usage data of the storage cluster comprises:
analyzing storage pool capacity information, disk writing information and service starting information corresponding to the storage cluster;
and respectively determining the total capacity and the use state of each storage pool corresponding to the storage cluster, the number of times of writing the disk and the starting state of each service based on the storage pool capacity information, the disk writing information and the service starting information.
3. The method of claim 2, wherein the generating capacity expansion configuration information of the storage cluster based on the usage data of the storage cluster and the second resource information comprises:
determining capacity expansion data corresponding to the storage cluster based on the use data of the storage cluster;
and distributing the second resource information according to the expansion data to generate the expansion configuration information.
4. The method of claim 1, wherein obtaining second resource information of the at least one node to be expanded comprises:
responding to an input instruction of the node to be expanded to obtain the information of the at least one node to be expanded;
analyzing the information of the at least one node to be expanded, and determining second resource information corresponding to the at least one node to be expanded.
5. The method of claim 1, further comprising:
deleting original resource information of the at least one node to be expanded to obtain current resource information corresponding to the at least one node to be expanded;
and carrying out monitor node configuration, metadata base service node configuration, cluster database configuration and disk resource allocation according to the expansion configuration information.
6. The method of claim 5, wherein the allocating disk resources according to the capacity expansion configuration information comprises:
acquiring the use frequency and the use state of each storage pool corresponding to the storage cluster;
and dynamically allocating the disk resources to the storage pools based on the use frequency and the use state of the storage pools.
7. The method according to claim 5, wherein performing metadata repository service node configuration according to the capacity expansion configuration information includes:
judging whether the storage cluster is a file storage cluster or not;
and when the storage cluster is a file storage cluster, configuring a plurality of metadata base service nodes based on the capacity expansion configuration information.
8. An apparatus for capacity expansion of a storage cluster, comprising:
the acquisition module is used for acquiring first resource information of the storage cluster and second resource information of at least one node to be expanded;
the analysis module is used for analyzing the first resource information and determining the use data of the storage cluster;
and the generating module is used for generating capacity expansion configuration information of the storage cluster based on the use data of the storage cluster and the second resource information.
9. An electronic device, comprising:
a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the method of storage cluster expansion of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a computer to perform the method of capacity expansion of a storage cluster of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111271843.1A CN114020214A (en) | 2021-10-29 | 2021-10-29 | Storage cluster capacity expansion method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111271843.1A CN114020214A (en) | 2021-10-29 | 2021-10-29 | Storage cluster capacity expansion method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114020214A true CN114020214A (en) | 2022-02-08 |
Family
ID=80058955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111271843.1A Pending CN114020214A (en) | 2021-10-29 | 2021-10-29 | Storage cluster capacity expansion method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020214A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115499304A (en) * | 2022-07-29 | 2022-12-20 | 天翼云科技有限公司 | Automatic deployment method, device, equipment and product of distributed storage |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111984190A (en) * | 2020-07-26 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Storage pool capacity expansion method and system for mass storage device |
| CN112463372A (en) * | 2020-11-23 | 2021-03-09 | 中国建设银行股份有限公司 | Target system capacity expansion method, device, equipment and storage medium |
| CN112486664A (en) * | 2020-10-23 | 2021-03-12 | 苏州浪潮智能科技有限公司 | Node capacity expansion method, system, terminal and storage medium |
| WO2021189695A1 (en) * | 2020-03-25 | 2021-09-30 | 平安科技（深圳）有限公司 | Distributed database dynamic expansion method and apparatus, and device and storage medium |
-
2021
- 2021-10-29 CN CN202111271843.1A patent/CN114020214A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021189695A1 (en) * | 2020-03-25 | 2021-09-30 | 平安科技（深圳）有限公司 | Distributed database dynamic expansion method and apparatus, and device and storage medium |
| CN111984190A (en) * | 2020-07-26 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Storage pool capacity expansion method and system for mass storage device |
| CN112486664A (en) * | 2020-10-23 | 2021-03-12 | 苏州浪潮智能科技有限公司 | Node capacity expansion method, system, terminal and storage medium |
| CN112463372A (en) * | 2020-11-23 | 2021-03-09 | 中国建设银行股份有限公司 | Target system capacity expansion method, device, equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115499304A (en) * | 2022-07-29 | 2022-12-20 | 天翼云科技有限公司 | Automatic deployment method, device, equipment and product of distributed storage |
| CN115499304B (en) * | 2022-07-29 | 2024-03-08 | 天翼云科技有限公司 | Automatic deployment method, device, equipment and product for distributed storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105893139B (en) | Method and device for providing storage service for tenant in cloud storage environment | |
| CN111212111B (en) | Object storage service management method and electronic equipment | |
| CN107666493B (en) | Database configuration method and equipment thereof | |
| CN114244717B (en) | Configuration method and device of virtual network card resources, computer equipment and medium | |
| WO2017050064A1 (en) | Memory management method and device for shared memory database | |
| CN105095103A (en) | Storage device management method and device used for cloud environment | |
| CN112988679B (en) | Log acquisition control method and device, storage medium and server | |
| US20240061712A1 (en) | Method, apparatus, and system for creating training task on ai training platform, and medium | |
| CN111241177A (en) | Data acquisition method, system and network equipment | |
| CN114090580A (en) | Data processing method, device, equipment, storage medium and product | |
| CN112799596A (en) | Capacity expansion control method and device for storage resources and electronic equipment | |
| CN111143039A (en) | Virtual machine scheduling method and device and computer storage medium | |
| CN113946291A (en) | Data access method, device, storage node and readable storage medium | |
| CN115048254A (en) | Simulation test method, system, equipment and readable medium of data distribution strategy | |
| CN114020214A (en) | Storage cluster capacity expansion method and device, electronic equipment and readable storage medium | |
| CN106708865B (en) | Method and device for accessing window data in stream processing system | |
| CN110990356A (en) | Real-time automatic capacity expansion method and system for logical mirror image | |
| CN110780821A (en) | Optimization method and device of distributed storage system, server and storage medium | |
| DE112021000408T5 (en) | PREDICTIVE DELIVERY OF REMOTELY STORED FILES | |
| CN111158595B (en) | Enterprise-level heterogeneous storage resource scheduling method and system | |
| CN112486664A (en) | Node capacity expansion method, system, terminal and storage medium | |
| CN109542727B (en) | Information prompting method and device | |
| CN111404828A (en) | Method and device for realizing global flow control | |
| CN115984022A (en) | Unified account checking method and device for distributed payment system | |
| CN112817742B (en) | Data migration method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |