CN106843755A - For the data balancing method and device of server cluster - Google Patents
For the data balancing method and device of server cluster Download PDFInfo
- Publication number
- CN106843755A CN106843755A CN201710003462.2A CN201710003462A CN106843755A CN 106843755 A CN106843755 A CN 106843755A CN 201710003462 A CN201710003462 A CN 201710003462A CN 106843755 A CN106843755 A CN 106843755A
- Authority
- CN
- China
- Prior art keywords
- disk
- server
- data
- source
- dilatation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses data balancing method and device for server cluster.One specific embodiment of method includes：In response to the dilatation to server cluster, it is determined that storing at least one source disk and the target disk for storing data to be migrated of data to be migrated, wherein, at least one source disk is located at least one former server of the server cluster before dilatation, and target disk is located in the newly-increased server in the server cluster after dilatation；Forbid newly-increased server to receive the read-write requests to target disk, and perform the migration operation that data to be migrated are moved to target disk from source disk；When target disk meets it is pre-conditioned when, stop perform migration operation；Wherein, it is pre-conditioned including：The residual capacity of target disk is identical with the average disk residual capacity of source disk, and average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.Realize the server cluster after dilatation and reach data balancing in a short time, save cost.
Description
Technical field
The application is related to technical field of the computer network, and in particular to distributed storage applied technical field, more particularly to
For the data balancing method and device of server cluster.
Background technology
Multiple server clusters are generally included in large-scale distributed storage system, with data volume and read-write requests
Increase each server cluster it is inevitable the need for dilatation.Server cluster dilatation is namely in the original service of server cluster
Increase a number of new demand servicing device on the basis of device.How data distribute what is solved as needs after server cluster dilatation
Problem.
Data distributing method is original will be owned according to certain strategy after a kind of server cluster dilatation of the prior art
Data acquisition system level is divided into multiple subsets, by the corresponding Data Migration of part subset to new demand servicing device, with above-mentioned part
The Data Migration of collection, the newly-increased write-in for corresponding to these subsets has also been followed on newly-increased server.Meanwhile, it is normal in order to not influence
Request of data, it is necessary to data are migrated into row control.So, the speed of Data Migration is slower, so that by number
The time that the data balancing of complete or collected works group is reached according to migration is more long.Additionally, this data distributing method is needed in each former server
Space carries out dilatation before writing completely, increased the dilatation cost of server cluster.
The content of the invention
The purpose of the application is to propose a kind of improved solve for the data balancing method and device of server cluster
The technical problem that certainly background section above is mentioned.
In a first aspect, this application provides a kind of data balancing method for server cluster, the method includes：Response
In the dilatation to server cluster, it is determined that storing at least one source disk of data to be migrated and for storing data to be migrated
Target disk, wherein, at least one source disk is located at least one former server of the server cluster before dilatation, target magnetic
Disk is located in the newly-increased server in the server cluster after dilatation；Forbidding newly-increased server to receive please to the read-write of target disk
Ask, and perform the migration operation that data to be migrated are moved to target disk from source disk；When target disk meets pre-conditioned
When, stop performing migration operation；Wherein, it is pre-conditioned including：The residual capacity of target disk and the average disk of source disk are surplus
Covolume amount is identical, and average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
In some optional implementations, the server increased newly after dilatation is forbidden is received please to the read-write of target disk
Before asking, the above method also includes：The state of target disk is set to the new addition states different from the state of source disk.
In some optional implementations, the above method also includes：When target disk meets it is pre-conditioned when, by target
The state of disk is set to the state identical normal state with source disk.
In some optional implementations, the above method also includes：If belonging to all magnetic of same newly-increased server
When the state of disk is normal state, read-write requests of the newly-increased server reception to target disk are enabled.
In some optional implementations, the corresponding data to be migrated of each source disk are different data.
In some optional implementations, above-mentioned determination stores at least one source disk of data to be migrated and for depositing
The target disk of the data to be migrated in source of bank savings disk, including：It is determined that the data in server cluster of the storage before dilatation are accounted for
Memory space；By memory space divided by total memory space of server cluster after dilatation, proportion threshold value is obtained；Calculate respectively
The scale parameter of each disk in former server in the server cluster before dilatation, scale parameter is to be stored on each disk
Data take memory space divided by the disk total memory space；When scale parameter is more than proportion threshold value, disk is made
It is source disk.
Second aspect, this application provides a kind of data balancing device for server cluster, the device includes：It is determined that
Unit, is configured in response to the dilatation to server cluster, it is determined that storing at least one source disk and use of data to be migrated
In the target disk for storing data to be migrated, wherein, at least one source disk is located at least of the server cluster before dilatation
In individual former server, target disk is located in the newly-increased server in the server cluster after dilatation；Execution unit, is configured to
Newly-increased server is forbidden to receive the read-write requests to target disk, and data to be migrated are moved to target magnetic by execution from source disk
The migration operation of disk；Stop element, is configured to, when target disk meets pre-conditioned, stop performing migration operation, presets
Condition includes：The residual capacity of target disk is identical with the average disk residual capacity of source disk, and average disk residual capacity is
Quantity of the source disk residual capacity sum divided by source disk.
In some optional implementations, said apparatus also include setting unit, and setting unit is configured to performing
Before unit forbids the server increased newly after dilatation to receive to the read-write requests of target disk, target disk is set to and source magnetic
The different new addition state of the state of disk.
In some optional implementations, setting unit is further configured to meet pre-conditioned when target disk
When, the state of target disk is set to the state identical normal state with source disk.
In some optional implementations, said apparatus also include enabling unit, and enabling unit is configured to work as and belongs to
When the state of all disks of same newly-increased server is normal state, enables the newly-increased server and receive to target disk
Read-write requests.
In some optional implementations, the corresponding data to be migrated of each source disk are different data.
In some optional implementations, determining unit is further configured to：It is determined that service of the storage before dilatation
The memory space that data in device cluster take；By memory space divided by total memory space of server cluster after dilatation, obtain
Proportion threshold value；The scale parameter of each disk in the former server in the server cluster before dilatation, ratio are calculated respectively
Parameter is the total memory space of the memory space divided by the disk of the data occupancy stored on each disk；When scale parameter be more than than
During example threshold value, using disk as source disk.
Data balancing method and device for server cluster that the application is provided, by determining data to be migrated first
At least one source disk and a target disk for storing data to be migrated, then forbid newly-increased server to receive to mesh
The read-write requests of disk are marked, and performs the migration operation that data to be migrated are moved to target disk from source disk, finally work as mesh
When mark disk meets pre-conditioned, stop performing migration operation.Realize by least one source disk to target disk
Rapid data is migrated, and then can realize that the server cluster after dilatation is rapidly achieved data balancing in a short time；Additionally, adopting
The technical scheme provided with the application can carry out dilatation when former data block server writes full, therefore reduce dilatation cost.
Brief description of the drawings
By the detailed description made to non-limiting example made with reference to the following drawings of reading, the application other
Feature, objects and advantages will become more apparent upon：
Fig. 1 is that the application can apply to exemplary system architecture figure therein；
Fig. 2 is the flow chart of one embodiment of the data balancing method for server cluster according to the application；
Fig. 3 is the data balancing method principle schematic for server cluster according to the application；
Fig. 4 is the flow chart of another embodiment of the data balancing method for server cluster according to the application
Fig. 5 is the structural representation of one embodiment of the device of the data balancing for server cluster according to the application
Figure；
Fig. 6 is adapted for the structural representation of the computer system of the server for realizing the embodiment of the present application.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that, in order to
Be easy to description, be illustrate only in accompanying drawing to about the related part of invention.
It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase
Mutually combination.Describe the application in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 shows can be using the application for the data balancing method of server cluster or for server cluster
Data balancing device embodiment exemplary system architecture 100.
As shown in figure 1, system architecture 100 can include data block server 101,102,103, master server 104 and visitor
Family end 105.Between master server 104 and client 105, data block server 101, between 102,103 and master server, and
Communicated by network between data block server 101,102,103 and client 105.Network herein can include various
Connection type, such as wired, wireless communication link or fiber optic cables etc..
Client 105 is interacted by network with master server 104, to receive or send message etc..User can be by each
Application is planted to be interacted with client.
Client 105 can be the various electronic equipments browsed with display screen and supported web page, including but not limited to
Smart mobile phone, panel computer, pocket computer on knee and desktop computer etc..Can be provided with and master in client 105
The interface routine that server and data block server are communicated.
Master server 104 can be the server for managing document storage system.Master server 104 is by network and visitor
Family end 105 communicates.To can support that client 105 carries out data write-in or the data block server 101,102,103 of reading is believed
Breath issues client 105, and is answering client 105 to set up name space, access control information, file when carrying out data block write-in
With the management information such as the positional information of the map information of data block and current data block.
Data block server 101,102,103 can be the server of data storage, such as common business computer.Number
Communicated with client 105 by network according to block server 101,102,103, receive the data block of the write-in of client 105；And to
Client 105 provides the data block that it reads.
It should be noted that the data balancing method for server cluster that the embodiment of the present application is provided is general by leading
Server 104 is performed, and correspondingly, the device for server cluster is generally positioned in master server 104.
It should be understood that the number of the data block server, master server and client in Fig. 1 is only schematical.Root
Factually now need, can have any number of data block server, master server and client.
Fig. 2 is refer to, one embodiment of the data balancing method for server cluster according to the application is shown
Flow 200.The described data balancing method for server cluster, comprises the following steps：
Step 201, in response to the dilatation to server cluster, it is determined that store data to be migrated at least one source disk and
Target disk for storing data to be migrated.
Server cluster generally includes many number of units according to block server and a master server.Wherein master server is to all numbers
It is managed according to the data stored in block server.When there is write-in data, master server carries out data according to fixed size
Piecemeal, is that each piecemeal sets up unique mark, and for each block determines storage location.Master server sets up the pipe of data block simultaneously
Reason information.The positional information of data block can be including the identification information and specific disk of the data block server where data block
Information.Wherein, the identification information of data block server can for example include the IP address information of data block server.Data block takes
Business device can be stored according to the space and position that master server is each data block distribution to each data block.It is worth what is pointed out
It is that master server is a logical concept, a master server can include many physical hosts.
When there is bottleneck in the capacity of server cluster, it is necessary to carry out dilatation, usual server cluster to server cluster
Dilatation refers to that new data block server is added in former server cluster.
In the present embodiment, the data balancing method for server cluster runs electronic equipment (such as Fig. 1 thereon
Shown master server) determine to store at least one source disk and the mesh for storing data to be migrated of data to be migrated first
Mark disk.
Specifically, master server can using dilatation after newly-increased server in any one disk as target disk.
That is above-mentioned target disk is located in the newly-increased server in the server cluster after dilatation.Newly-increased server for example can be with
Newly-increased data block server in for the server cluster after dilatation.
For above-mentioned any one target disk, master server obtains the memory space of the target disk, and then basis should
The memory space of target disk chooses data to be migrated from least one former data block server.When data to be migrated are chosen
At least one source disk can be determined from former data block server first, then is chosen from above-mentioned at least one source disk certain
The data block of quantity is used as data to be migrated.In application scenes, each data block in data block server is stored in
Size be a fixed value.The fixed value for example can be 32GB.
Generally, in distributed storage cluster, according to that can include multiple copies, different copies can be stored each number
In different data block servers.In some optional implementations of the present embodiment, that is chosen from each source disk treats
Migrating data can be different data." different data " can be the mark of the user corresponding to data block to be migrated herein
Information is different, can also be that the data content in data block to be migrated is different.When the number to be migrated chosen from each source disk
During according to for different data, each number is migrated in newly-increased server according to by only one of which copy.So can not only protect
Card data are uniformly distributed in the cluster after dilatation, it is also possible that a copy in a data is migrated to newly-increased service
During target disk on device, the number can be user's according to other copies being located in other former data block servers
Read-write requests provide service.
In the present embodiment, above-mentioned at least one source disk is located at least one former clothes of the former server cluster before dilatation
In business device.Herein, former server can be the data block server in former server cluster.
In some optional implementations of the present embodiment, master server can determine source disk according to following method：
First, it is determined that the memory space of all of data occupancy in server cluster of the storage before dilatation.Then by all of number
According to the memory space for taking divided by total memory space of server cluster after dilatation, proportion threshold value is obtained.Then calculate respectively and expand
The scale parameter of each disk in former server in the server cluster before appearance；Herein, the ratio of any one disk
Parameter can be total memory space of the memory space shared by the data stored on the disk divided by the disk.Finally, when one
When the scale parameter of individual disk is more than proportion threshold value, can be using the disk as source disk.
It should be noted that the method for above-mentioned determination source disk is not limited to above-mentioned method.Determine that the method for source disk can
To be method that each data block server in above-mentioned former server can be arbitrarily judged to source disk by other, the present embodiment is not right
The method of above-mentioned determination source disk is specifically limited.
Step 202, forbids newly-increased server to receive the read-write requests to target disk, and perform data to be migrated from source
Disk moves to the migration operation of target disk.
In the present embodiment, the data balancing method for server cluster runs electronic equipment (such as Fig. 1 thereon
Shown master server) target disk and source disk are determined in step 201 after, forbid newly-increased server to receive such as Fig. 1 institutes
Show read-write requests of the client to target disk.Then data to be migrated are moved to by target disk from source disk by network
In.
It is worthy of note that, master server can also be by data to be migrated and data storage to be migrated in source disk
Storage location information and target disk information target disk is sent to by network where newly-increased data block server.Then
Migrating data request is sent to the former data block server where source disk from increasing data block server newly, in migrating data request
Including source disk information and data message to be migrated.Migrating data request in response to increasing server newly, former data block server
By in the target disk of the data is activation to be migrated on source disk to newly-increased server.Similarly, can also be by data to be migrated
And the storage location information in source disk and target disk information of data storage to be migrated are sent to each source disk institute
Former data block server in, former data block server is by the target disk of data is activation to be migrated to newly-increased server.
Forbidding client can accelerate to perform at least one source disk to the read-write requests for increasing the target disk of server newly
To the speed of target disk Data Migration.
Step 203, when target disk meets it is pre-conditioned when, stop perform migration operation.
When used for data balancing method operation electronic equipment (such as main clothes shown in Fig. 1 thereon of server cluster
Business device) when judging that target disk meets pre-conditioned, execution can be stopped by the Data Migration to be migrated in source disk to target
The operation of disk.
In the present embodiment, the average disk of the above-mentioned pre-conditioned residual capacity and source disk that can include target disk
Residual capacity is identical, and average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
Data balancing method and device for server cluster that the application is provided, by determining data to be migrated first
At least one source disk and a target disk for storing data to be migrated, then forbid newly-increased server to receive to mesh
The read-write requests of disk are marked, and performs the migration operation that data to be migrated are moved to target disk from source disk, finally work as mesh
When mark disk meets pre-conditioned, stop performing migration operation.Realize by least one source disk to target disk
Rapid data is migrated, and then can realize that the server cluster after dilatation is rapidly achieved data balancing in a short time；Additionally, adopting
The technical scheme provided with the application can carry out dilatation when former data block server writes full, therefore reduce dilatation cost.
With continued reference to Fig. 3, Fig. 3 is the principle flow of the data balancing method for server cluster according to the application
Figure 30 0.As shown in figure 3, former data block server A, B, C and D in former server cluster before dilatation, the server after dilatation
Cluster includes former data block server A, B, C and D and newly-increased data block server E.Former data block server A, B, C and D
Include disk a, b, c, d respectively.Newly-increased data block server E includes disk e1 and disk e2.In server cluster after dilatation
Disk e1 and e2 are respectively by as target disk in newly-increased data block server E.Disk a, former number in former data block server A
The source disk of target disk e1 is determined to correspond to according to the disk b in block server B.Disk in former data block server C
Disk d in c, former data block server D is determined to correspond to the source disk of target disk e2.According to depositing for target disk e1
Data to be migrated are chosen from source disk a and source disk b to carrying out Data Migration in target disk e1 in storage space.Simultaneously according to mesh
The memory space of mark disk e2 is chosen data to be migrated from source disk c and source disk d and is moved to carrying out data in target disk e2
Move.Data balancing method shown in Fig. 3, can quickly realize the data balancing of the server cluster after dilatation in a short time.
With continued reference to Fig. 4, it illustrates the data balancing method for server cluster according to the application another
The flow 400 of embodiment.The described data balancing method for server cluster, comprises the following steps：
Step 401, in response to the dilatation to server cluster, it is determined that store data to be migrated at least one source disk and
Target disk for storing data to be migrated.
In the present embodiment, the data balancing method for server cluster runs electronic equipment (such as Fig. 1 thereon
Shown master server) using dilatation after newly-increased data block server in each disk as target disk, and obtain every
One memory space of target disk.
For each target disk, at least one former data block clothes of the master server according to its memory space from before dilatation
Determine at least one source disk in business device.And a certain amount of data block is chosen from above-mentioned at least one source disk as to be migrated
Data.
The corresponding data to be migrated of each source disk are differed.Each source disk correspondence data to be migrated herein are not
Identical can be that the data content in data block to be migrated is differed, and can also be that the mark of data block to be migrated is differed.
Step 402, the new addition states different from the state of source disk are set to by the state of target disk.
It is in the present embodiment, storage to be determined in step 401 to be migrated with embodiment illustrated in fig. 2 difference
After at least one source disk of data and a target disk for storing data to be migrated, for the number of server cluster
According to equalization methods operation electronic equipment (such as the master server shown in Fig. 1) thereon by the state of target disk be set to
The different new addition state of the state of source disk.So, can facilitate master server to the management of each target disk with
Control.
Step 403, forbids newly-increased server to receive the read-write requests to target disk, and perform data to be migrated from source
Disk moves to the migration operation of target disk.
After being set to newly to add state by the state of target disk in step 402, for the data of server cluster
Equalization methods operation electronic equipment (such as the master server shown in Fig. 1) thereon forbids newly-increased server to receive to target magnetic
The read-write requests of disk.
Master server performs the migration operation that data to be migrated are moved to target disk from source disk by network simultaneously.
Step 404, when target disk meets it is pre-conditioned when, the state of target disk is set to the state with source disk
Identical normal state, and stop performing migration operation.
When target disk meets it is pre-conditioned when, for the data balancing method operation electronics thereon of server cluster
The state of target disk is set to equipment (such as the master server shown in Fig. 1) the common shape of state identical with source disk
State.Stop performing migration operation simultaneously.
In the present embodiment, it is above-mentioned pre-conditioned for example to include：The residual capacity of target disk and putting down for source disk
Equal disk residual capacity is identical.Wherein, average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
In some optional implementations of the present embodiment, if belonging to the state of all disks of same newly-increased server
When being normal state, read-write requests of the newly-increased server reception to target disk are enabled.That is, when a newly-increased clothes
When the residual capacity of all of disk of business device is identical with the average residual capacity of source disk, no longer in the newly-increased server
Data in magnetic disk is migrated, while the newly-increased server can provide service for the read-write requests of user.Newly-increased server can be use
The read-write requests at family provide service, that is, the data throughout of server cluster increased.
In the present embodiment, except the server cluster after dilatation can be made to be rapidly achieved data balancing in a short time,
And save outside dilatation cost, by the way that before Data Migration is performed, the state to target disk is configured differently than the new of source disk
Addition state, and when disk target disk meets pre-conditioned, the state of target disk is set to the state with source disk
Identical normal state, had both been easy to identification and management of the master server to target disk, while newly-increased server can also be made in time
Read-write service is provided client, to improve the data throughout of server cluster.
With further reference to Fig. 5, as the realization to method shown in Fig. 1, server cluster to be used for this application provides one kind
Data balancing device one embodiment, the device embodiment is corresponding with the embodiment of the method shown in Fig. 1, and the device is specific
In can apply to various terminals.
As shown in figure 5, the data balancing device 500 that the present embodiment is used for server cluster includes：Determining unit 501, hold
Row unit 502 and stop element 503.Wherein it is determined that unit 501 is configured in response to the dilatation to server cluster, it is determined that
At least one source disk and the target disk for storing data to be migrated of data to be migrated are stored, wherein, at least one source
Disk is located at least one former server of the server cluster before dilatation, and target disk is located at the server cluster after dilatation
In newly-increased server in.Execution unit 502 is configured to forbid newly-increased server to receive the read-write requests to target disk, and
Data to be migrated are moved to the migration operation of target disk from source disk for execution.Stop element 503 is configured to when target magnetic
When disk meets pre-conditioned, stop perform migration operation, it is pre-conditioned including：The residual capacity of target disk and putting down for source disk
Equal disk residual capacity is identical, and average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
In the present embodiment, determining unit 501 for example can by dilatation after newly-increased server in any one disk
As target disk.Herein, newly-increased server can be the newly-increased data block server in the server cluster after dilatation.
For above-mentioned any one target disk, determining unit 501 obtains the memory space of the target disk, then basis
The memory space of the target disk chooses data to be migrated from least one former data block server.Choosing data to be migrated
When at least one source disk can be determined from former data block server first, then choose one from above-mentioned at least one source disk
The data block of fixed number amount is used as data to be migrated.
In some optional implementations of the present embodiment, it is to be migrated that determining unit 501 is chosen from each source disk
Data are different data.Herein different data can refer to the user corresponding to data block to be migrated identification information not
Together, the data content that can also refer in data block to be migrated is different.What determining unit 501 was chosen from each source disk waits to move
Shifting data are different data so that each number is migrated in newly-increased server according to only one of which copy.So not only can be with
Ensure to be uniformly distributed in cluster of the data after dilatation, it is also possible that a copy in a data is migrated to newly-increased clothes
During target disk on business device, the number can be user according to other copies being located in other former data block servers
Read-write requests provide service.
In some optional implementations of the present embodiment, determining unit 501 is further configured to：First, it is determined that depositing
Store up the memory space that all of data in the server cluster before dilatation take.Then the storage for all of data being taken
Space obtains proportion threshold value divided by total memory space of server cluster after dilatation.Then the server before dilatation is calculated respectively
The scale parameter of each disk in former server in cluster；Herein, the scale parameter of any one disk can be to be somebody's turn to do
Total memory space of the memory space shared by data stored on disk divided by the disk.Finally, when a ratio for disk
When parameter is more than proportion threshold value, can be using the disk as source disk.
In the present embodiment, execution unit 502 determines a target disk and at least one source magnetic in determining unit 501
After disk, newly-increased server is forbidden to receive read-write requests of the client to target disk.Then network is passed through by data to be migrated
In target disk being moved to from source disk.
In some optional implementations of the present embodiment, the data balancing device 500 for server cluster also includes
Setting unit (not shown), the server that setting unit is configured to be increased newly after execution unit forbids dilatation is received to mesh
Before marking the read-write requests of disk, target disk is set to the new addition states different from the state of source disk.
Stop element 503 is configured to, when above-mentioned target disk meets pre-conditioned, stop performing migration operation.At this
In embodiment, it is pre-conditioned including：The residual capacity of target disk is identical with the average disk residual capacity of source disk, average magnetic
Disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
In some optional implementations of the present embodiment, above-mentioned setting unit is further configured to when target disk is full
When foot is pre-conditioned, the state of target disk is set to the state identical normal state with source disk.
In some optional implementations of the present embodiment, the data balancing device 500 for server cluster also includes
Enabling unit (not shown).Enabling unit is configured to when the state of all disks for belonging to same newly-increased server is equal
During for normal state, read-write requests of the newly-increased server reception to target disk are enabled.
Below with reference to Fig. 6, it illustrates the computer system 600 for being suitable to the server for realizing the embodiment of the present application
Structural representation.
As shown in fig. 6, computer system 600 includes CPU (CPU) 601, it can be according to storage read-only
Program in memory (ROM) 602 or be loaded into program in random access storage device (RAM) 603 from storage part 608 and
Perform various appropriate actions and treatment.In RAM 603, the system that is also stored with 600 operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interfaces 605 are connected to lower component：Importation 606 including keyboard etc.；Including such as liquid crystal display
Etc. (LCD) and loudspeaker etc. output par, c 607；Storage part 608 including hard disk etc.；And including such as LAN card, tune
The communications portion 609 of the NIC of modulator-demodulator etc..Communications portion 609 performs mailing address via the network of such as internet
Reason.Driver 610 is also according to needing to be connected to I/O interfaces 605.Detachable media 611, such as disk, CD, magneto-optic disk, half
Conductor memory etc., as needed on driver 610, in order to the computer program that reads from it as needed
It is mounted into storage part 608.
Especially, in accordance with an embodiment of the present disclosure, the process above with reference to flow chart description may be implemented as computer
Software program.For example, embodiment of the disclosure includes a kind of computer program product, it includes being tangibly embodied in machine readable
Computer program on medium, the computer program includes the program code for the method shown in execution flow chart.At this
In the embodiment of sample, the computer program can be downloaded and installed by communications portion 609 from network, and/or from removable
Medium 611 is unloaded to be mounted.When the computer program is performed by CPU (CPU) 601, in execution the present processes
The above-mentioned functions of restriction.
Flow chart and block diagram in accompanying drawing, it is illustrated that according to the system of the various embodiments of the application, method and computer journey
The architectural framework in the cards of sequence product, function and operation.At this point, each square frame in flow chart or block diagram can generation
One part for module, program segment or code of table a, part for the module, program segment or code includes one or more
Executable instruction for realizing the logic function of regulation.It should also be noted that in some realizations as replacement, institute in square frame
The function of mark can also occur with different from the order marked in accompanying drawing.For example, two square frame reality for succeedingly representing
On can perform substantially in parallel, they can also be performed in the opposite order sometimes, and this is depending on involved function.Also
It is noted that the combination of the square frame in each square frame and block diagram and/or flow chart in block diagram and/or flow chart, Ke Yiyong
Perform the function of regulation or the special hardware based system of operation to realize, or can be referred to computer with specialized hardware
The combination of order is realized.
Being described in involved unit in the embodiment of the present application can be realized by way of software, it is also possible to by hard
The mode of part is realized.Described unit can also be set within a processor, for example, can be described as：A kind of processor bag
Include determining unit, execution unit and stop element.Wherein, the title of these units is not constituted to this under certain conditions
Unit restriction in itself, for example, determining unit is also described as " determining the unit of source disk and target disk ".
As on the other hand, present invention also provides a kind of nonvolatile computer storage media, the non-volatile calculating
Machine storage medium can be the nonvolatile computer storage media included in device described in above-described embodiment；Can also be
Individualism, without the nonvolatile computer storage media allocated into terminal.Above-mentioned nonvolatile computer storage media is deposited
One or more program is contained, when one or more of programs are performed by an equipment so that the equipment：Response
In the dilatation to server cluster, it is determined that storing at least one source disk of data to be migrated and for storing data to be migrated
Target disk, wherein, at least one source disk is located at least one former server of the server cluster before dilatation, target magnetic
Disk is located in the newly-increased server in the server cluster after dilatation；Forbidding newly-increased server to receive please to the read-write of target disk
Ask, and perform the migration operation that data to be migrated are moved to target disk from source disk；When target disk meets pre-conditioned
When, stop performing migration operation；Wherein, it is pre-conditioned including：The residual capacity of target disk and the average disk of source disk are surplus
Covolume amount is identical, and average disk residual capacity is quantity of the source disk residual capacity sum divided by source disk.
Above description is only the preferred embodiment and the explanation to institute's application technology principle of the application.People in the art
Member is it should be appreciated that involved invention scope in the application, however it is not limited to the technology of the particular combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where the inventive concept is not departed from, is carried out by above-mentioned technical characteristic or its equivalent feature
Other technical schemes for being combined and being formed.Such as features described above has similar work(with (but not limited to) disclosed herein
The technical scheme that the technical characteristic of energy is replaced mutually and formed.
Claims (12)
1. a kind of data balancing method for server cluster, it is characterised in that methods described includes：
In response to the dilatation to the server cluster, it is determined that storing at least one source disk of data to be migrated and for storing
The target disk of the data to be migrated, wherein, at least one source disk is located at the server cluster before dilatation at least
In one former server, the target disk is located in the newly-increased server in the server cluster after dilatation；
Forbid the newly-increased server to receive the read-write requests to the target disk, and perform the data to be migrated from source
Disk moves to the migration operation of the target disk；
When the target disk meets pre-conditioned, stop performing the migration operation；
Wherein, it is described it is pre-conditioned including：The average disk of the residual capacity of the target disk and the source disk is remaining to be held
Amount is identical, and the average disk residual capacity is quantity of the source disk residual capacity sum divided by the source disk.
2. method according to claim 1, it is characterised in that it is described forbid the dilatation after the server that increases newly receive
Before to the read-write requests of target disk, methods described also includes：
The state of the target disk is set to the new addition states different from the state of the source disk.
3. method according to claim 2, it is characterised in that methods described also includes：
When the target disk meets pre-conditioned, the state of the target disk is set to the state with the source disk
Identical normal state.
4. method according to claim 3, it is characterised in that methods described also includes：
If the state for belonging to all disks of same newly-increased server is normal state, enables the newly-increased server and receive
To the read-write requests of the target disk.
5. method according to claim 1, it is characterised in that the corresponding data to be migrated of each source disk are different
Data.
6. the method according to claim 1-5 any one, it is characterised in that the determination stores data to be migrated extremely
Few a source disk and the target disk for storing the data to be migrated in the source disk, including：
It is determined that the memory space that the data in server cluster of the storage before the dilatation take；
By the memory space divided by total memory space of the server cluster after dilatation, proportion threshold value is obtained；
The scale parameter of each disk in the former server in the server cluster before the dilatation, the ratio are calculated respectively
Example parameter is the total memory space of the memory space divided by the disk of the data occupancy stored on each disk；
When the scale parameter is more than the proportion threshold value, using the disk as source disk.
7. a kind of data balancing device for server cluster, it is characterised in that described device includes：
Determining unit, the determining unit is configured in response to the dilatation to the server cluster, it is determined that storage is to be migrated
At least one source disk of data and the target disk for storing the data to be migrated, wherein, at least one source magnetic
Disk is located at least one former server of the server cluster before dilatation, and the target disk is located at the server set after dilatation
In newly-increased server in group；
Execution unit, the execution unit is configured to forbid read-write of the newly-increased server reception to the target disk to ask
Ask, and perform the migration operation that the data to be migrated are moved to the target disk from source disk；
Stop element, the stop element is configured to, when the target disk meets pre-conditioned, stop performing migration behaviour
Make, it is described it is pre-conditioned including：The residual capacity of the target disk is identical with the average disk residual capacity of the source disk,
The average disk residual capacity is quantity of the source disk residual capacity sum divided by the source disk.
8. device according to claim 7, it is characterised in that described device also includes setting unit,
The server that the setting unit is configured to be increased newly after the execution unit forbids the dilatation is received to target magnetic
Before the read-write requests of disk, the target disk is set to the new addition states different from the state of the source disk.
9. device according to claim 8, it is characterised in that
The setting unit is further configured to when the target disk meets pre-conditioned, by the shape of the target disk
State is set to the state identical normal state with the source disk.
10. device according to claim 9, it is characterised in that described device also includes enabling unit, the enabling unit
It is configured to, when belonging to the state of all disks of same newly-increased server and being normal state, enable the newly-increased server
Receive the read-write requests to the target disk.
11. devices according to claim 7, it is characterised in that the corresponding data to be migrated of each source disk are difference
Data.
12. device according to claim 7-11 any one, it is characterised in that the determining unit further configures use
In：
It is determined that the memory space that the data in server cluster of the storage before the dilatation take；
By the memory space divided by total memory space of the server cluster after dilatation, proportion threshold value is obtained；
The scale parameter of each disk in the former server in the server cluster before the dilatation, the ratio are calculated respectively
Example parameter is the total memory space of the memory space divided by the disk of the data occupancy stored on each disk；
When the scale parameter is more than the proportion threshold value, using the disk as source disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710003462.2A CN106843755B (en) | 2017-01-04 | 2017-01-04 | Data balancing method and device for server cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710003462.2A CN106843755B (en) | 2017-01-04 | 2017-01-04 | Data balancing method and device for server cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106843755A true CN106843755A (en) | 2017-06-13 |
| CN106843755B CN106843755B (en) | 2019-10-11 |
Family
ID=59118428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710003462.2A Active CN106843755B (en) | 2017-01-04 | 2017-01-04 | Data balancing method and device for server cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106843755B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357896A (en) * | 2017-07-13 | 2017-11-17 | 北京小度信息科技有限公司 | Expansion method, device, system and the data base cluster system of data-base cluster |
| CN107391050A (en) * | 2017-09-14 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of data migration method, system, device and computer-readable recording medium |
| CN108197229A (en) * | 2017-12-29 | 2018-06-22 | 北京搜狐新媒体信息技术有限公司 | The balance method and system of a kind of data in magnetic disk |
| CN108388468A (en) * | 2018-01-26 | 2018-08-10 | 杨立群 | A kind of Flase File System dispatching method and device |
| WO2019033949A1 (en) * | 2017-08-18 | 2019-02-21 | 阿里巴巴集团控股有限公司 | Data migration method, apparatus and device |
| CN109426439A (en) * | 2017-09-04 | 2019-03-05 | 阿里巴巴集团控股有限公司 | The method and device of dilatation is carried out to distributed memory system |
| CN109683826A (en) * | 2018-12-26 | 2019-04-26 | 北京百度网讯科技有限公司 | Expansion method and device for distributed memory system |
| CN110333824A (en) * | 2019-06-05 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of expansion method and device of storage system |
| CN110874344A (en) * | 2018-08-10 | 2020-03-10 | 阿里巴巴集团控股有限公司 | Data migration method and device and electronic equipment |
| CN112463050A (en) * | 2020-11-26 | 2021-03-09 | 新华三技术有限公司成都分公司 | Storage system capacity expansion method, device, equipment and machine-readable storage medium |
| CN112596671A (en) * | 2020-12-16 | 2021-04-02 | 东软集团股份有限公司 | Data storage method and device, storage medium and electronic equipment |
| CN112667148A (en) * | 2020-11-30 | 2021-04-16 | 上海七牛信息技术有限公司 | Method and system for cluster storage data online migration |
| CN113127438A (en) * | 2019-12-30 | 2021-07-16 | 北京百度网讯科技有限公司 | Method, apparatus, server and medium for storing data |
| CN113867923A (en) * | 2021-12-02 | 2021-12-31 | 苏州浪潮智能科技有限公司 | Method and device for migrating container application across CPU (Central processing Unit) architecture |
| CN113986825A (en) * | 2021-12-27 | 2022-01-28 | 北京星汉未来网络科技有限公司 | System, method and device for data migration, electronic equipment and readable storage medium |
| CN115442388A (en) * | 2022-10-21 | 2022-12-06 | 四川省华存智谷科技有限责任公司 | Capacity expansion method, device and system for 100% utilization rate of distributed storage cluster |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN103067433A (en) * | 2011-10-24 | 2013-04-24 | 阿里巴巴集团控股有限公司 | Method, device and system of data migration of distributed type storage system |
| CN105260376A (en) * | 2015-08-17 | 2016-01-20 | 北京京东尚科信息技术有限公司 | Method, equipment and system used for cluster node contraction and expansion |
| CN105468473A (en) * | 2014-07-16 | 2016-04-06 | 北京奇虎科技有限公司 | Data migration method and data migration apparatus |
| CN105653630A (en) * | 2015-12-25 | 2016-06-08 | 北京奇虎科技有限公司 | Data migration method and apparatus for distributed database |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
-
2017
- 2017-01-04 CN CN201710003462.2A patent/CN106843755B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103067433A (en) * | 2011-10-24 | 2013-04-24 | 阿里巴巴集团控股有限公司 | Method, device and system of data migration of distributed type storage system |
| CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
| CN105468473A (en) * | 2014-07-16 | 2016-04-06 | 北京奇虎科技有限公司 | Data migration method and data migration apparatus |
| CN105260376A (en) * | 2015-08-17 | 2016-01-20 | 北京京东尚科信息技术有限公司 | Method, equipment and system used for cluster node contraction and expansion |
| CN105653630A (en) * | 2015-12-25 | 2016-06-08 | 北京奇虎科技有限公司 | Data migration method and apparatus for distributed database |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357896A (en) * | 2017-07-13 | 2017-11-17 | 北京小度信息科技有限公司 | Expansion method, device, system and the data base cluster system of data-base cluster |
| WO2019033949A1 (en) * | 2017-08-18 | 2019-02-21 | 阿里巴巴集团控股有限公司 | Data migration method, apparatus and device |
| CN109426439A (en) * | 2017-09-04 | 2019-03-05 | 阿里巴巴集团控股有限公司 | The method and device of dilatation is carried out to distributed memory system |
| CN107391050A (en) * | 2017-09-14 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of data migration method, system, device and computer-readable recording medium |
| CN108197229A (en) * | 2017-12-29 | 2018-06-22 | 北京搜狐新媒体信息技术有限公司 | The balance method and system of a kind of data in magnetic disk |
| CN108388468B (en) * | 2018-01-26 | 2022-02-22 | 杨立群 | Fast file system scheduling method and device |
| CN108388468A (en) * | 2018-01-26 | 2018-08-10 | 杨立群 | A kind of Flase File System dispatching method and device |
| CN110874344A (en) * | 2018-08-10 | 2020-03-10 | 阿里巴巴集团控股有限公司 | Data migration method and device and electronic equipment |
| CN110874344B (en) * | 2018-08-10 | 2023-05-09 | 阿里巴巴集团控股有限公司 | Data migration method and device and electronic equipment |
| CN109683826A (en) * | 2018-12-26 | 2019-04-26 | 北京百度网讯科技有限公司 | Expansion method and device for distributed memory system |
| CN109683826B (en) * | 2018-12-26 | 2023-08-29 | 北京百度网讯科技有限公司 | Capacity expansion method and device for distributed storage system |
| CN110333824A (en) * | 2019-06-05 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of expansion method and device of storage system |
| CN113127438A (en) * | 2019-12-30 | 2021-07-16 | 北京百度网讯科技有限公司 | Method, apparatus, server and medium for storing data |
| CN113127438B (en) * | 2019-12-30 | 2023-07-28 | 北京百度网讯科技有限公司 | Method, apparatus, server and medium for storing data |
| CN112463050A (en) * | 2020-11-26 | 2021-03-09 | 新华三技术有限公司成都分公司 | Storage system capacity expansion method, device, equipment and machine-readable storage medium |
| CN112667148A (en) * | 2020-11-30 | 2021-04-16 | 上海七牛信息技术有限公司 | Method and system for cluster storage data online migration |
| CN112596671A (en) * | 2020-12-16 | 2021-04-02 | 东软集团股份有限公司 | Data storage method and device, storage medium and electronic equipment |
| CN113867923A (en) * | 2021-12-02 | 2021-12-31 | 苏州浪潮智能科技有限公司 | Method and device for migrating container application across CPU (Central processing Unit) architecture |
| CN113867923B (en) * | 2021-12-02 | 2022-03-08 | 苏州浪潮智能科技有限公司 | Method and device for migrating container application across CPU (Central processing Unit) architecture |
| CN113986825A (en) * | 2021-12-27 | 2022-01-28 | 北京星汉未来网络科技有限公司 | System, method and device for data migration, electronic equipment and readable storage medium |
| CN115442388A (en) * | 2022-10-21 | 2022-12-06 | 四川省华存智谷科技有限责任公司 | Capacity expansion method, device and system for 100% utilization rate of distributed storage cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106843755B (en) | 2019-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106843755A (en) | For the data balancing method and device of server cluster | |
| CN108737325A (en) | A kind of multi-tenant data partition method, apparatus and system | |
| CN105516367B (en) | Distributed data-storage system, method and apparatus | |
| CN105426223B (en) | Using loading method and device | |
| CN105631035B (en) | Date storage method and device | |
| CN108055320A (en) | Processing method, terminal device and the computer readable storage medium of resource downloading | |
| CN108306956A (en) | Information-pushing method, device, server and computer-readable medium | |
| CN105550345A (en) | File operation method and apparatus | |
| CN103631869B (en) | A kind of method and apparatus for alleviating server-side database access pressure | |
| CN114244717B (en) | Configuration method and device of virtual network card resources, computer equipment and medium | |
| CN110083381A (en) | The method and device of increment upgrading | |
| CN104205780B (en) | A kind of method and apparatus of data storage | |
| CN109471725A (en) | Resource allocation methods, device and server | |
| CN106131055A (en) | Method and apparatus for batch operation authority | |
| CN106357791A (en) | Method, device and system for processing services | |
| CN109582329A (en) | Data management and method for subscribing, device, system, electronic equipment and storage medium | |
| CN106649600A (en) | Way, device and system of migrating file permissions | |
| CN107071036A (en) | A kind of method, the webserver and system for downloading driving file | |
| JP2015180991A (en) | Image forming apparatus, control method of image forming apparatus, and program | |
| CN108255989A (en) | Picture storage method, device, terminal device and computer storage media | |
| CN114924751A (en) | Method and device for distributing service access request flow | |
| CN112306993A (en) | Data reading method, device and equipment based on Redis and readable storage medium | |
| CN110019537A (en) | Local cache method for refreshing, device, computer equipment and storage medium | |
| CN108681550A (en) | Page furbishing method, the control method of device and page furbishing, device | |
| CN106326132A (en) | Storage system, storage management device, storage, hybrid storage device and storage management method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |