CN118069293A - Cluster expansion and contraction method, device, equipment, medium and program product - Google Patents
Cluster expansion and contraction method, device, equipment, medium and program product Download PDFInfo
- Publication number
- CN118069293A CN118069293A CN202410274175.5A CN202410274175A CN118069293A CN 118069293 A CN118069293 A CN 118069293A CN 202410274175 A CN202410274175 A CN 202410274175A CN 118069293 A CN118069293 A CN 118069293A
- Authority
- CN
- China
- Prior art keywords
- cluster
- metadata
- contraction
- processed
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008602 contraction Effects 0.000 title claims abstract description 131
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000012423 maintenance Methods 0.000 claims abstract description 29
- 238000012508 change request Methods 0.000 claims description 60
- 230000008859 change Effects 0.000 claims description 42
- 238000004590 computer program Methods 0.000 claims description 36
- 230000004044 response Effects 0.000 claims description 24
- 238000012217 deletion Methods 0.000 claims description 8
- 230000037430 deletion Effects 0.000 claims description 8
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a cluster capacity expansion and contraction method, a device, equipment, a medium and a program product, relates to the technical field of computers, and can be used in the technical field of finance and other related fields. The method is applicable to any member node in the cluster to be processed, and comprises the following steps: under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction; loading a dynamic configuration file, and loading the metadata subjected to capacity expansion and contraction into a memory; and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction. According to the technology provided by the application, the operation and maintenance efficiency under the condition of cluster expansion and contraction is improved, and the operation and maintenance cost and the risk of external stopping are reduced.
Description
Technical Field
The present application relates to the field of computer technology, and may be used in the field of financial technology or other related fields, and in particular, to a method, apparatus, device, medium and program product for cluster expansion and contraction.
Background
A distributed system is a loosely coupled system of multiple computing devices interconnected by communication lines, thereby serving a group of individual computing devices as a whole to a user. The distributed system has various general physical and logical resources, and can dynamically allocate tasks to improve service performance.
Distributed systems typically support clustered deployment modes to achieve a high availability architecture. In the process of providing services, there may be a case of expanding and contracting capacity as required. In the prior art, when the expansion and contraction capacity processing is performed on the distributed system, operation and maintenance personnel are required to adjust the full configuration of the computing equipment one by one, and after the adjustment, the equipment is restarted one by one, so that the operation and maintenance cost is high, the efficiency is low, and the risk of stopping the service is caused.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a cluster expansion and contraction method, apparatus, device, medium, and program product that can reduce operation and maintenance costs and risk of outages, and improve operation and maintenance efficiency.
In a first aspect, the present application provides a method for expanding and shrinking a cluster, which is applied to any member node in a cluster to be processed, and includes:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading the dynamic configuration file and loading the metadata after expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In one embodiment, the method further comprises: acquiring a member change request; the member change request comprises member information and change types of member nodes to be changed; wherein the change type comprises addition and/or deletion; and updating the member node information in the metadata according to the member information of the member node to be changed and the change type to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
In one embodiment, the obtaining the member change request includes: responding to the expansion and contraction operation of the operation and maintenance interface, and generating the member change request; or generating the member change request in response to metadata synchronization operation initiated by other member nodes in the to-be-processed cluster.
In one embodiment, the method further comprises: if the metadata of member nodes in the to-be-processed cluster is monitored to change, determining that the to-be-processed cluster has expansion and contraction processing conditions.
In one embodiment, after the generating the member change request in response to a scaling operation on the operation and maintenance interface, the method further includes: and initiating a metadata synchronization instruction to other member nodes in the to-be-processed cluster, wherein the metadata synchronization instruction is used for indicating the other member nodes to respond to the metadata synchronization instruction, generating a member change request, and updating member node information in metadata according to member information and change types of the member nodes to be changed in the member change request to obtain metadata of each member node in the to-be-processed cluster after expansion and contraction.
In one embodiment, the generating a dynamic configuration file according to metadata of each member node in the expanded and contracted to-be-processed cluster includes: generating a configuration file copy of the dynamic configuration file of the current use version; and updating the member node addresses in the configuration file copies according to the metadata of each member node in the expanded and contracted to-be-processed cluster to obtain the latest version of dynamic configuration file.
In one embodiment, the method further comprises: and in response to the node restarting operation, loading the latest version of dynamic configuration file.
In a second aspect, the present application further provides a cluster expansion and contraction device configured at any member node in a cluster to be processed, including:
The configuration file generation module is used for generating a dynamic configuration file according to metadata of each member node in the cluster to be processed after expansion and contraction under the condition of carrying out expansion and contraction processing on the cluster to be processed;
The configuration file loading module is used for loading the dynamic configuration file and loading the metadata subjected to expansion and contraction into the memory;
The main node selection module is used for selecting a cluster main node from member nodes of the expanded and contracted to-be-processed cluster according to the metadata after the expansion and contraction.
In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading the dynamic configuration file and loading the metadata after expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading the dynamic configuration file and loading the metadata after expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In a fifth aspect, the application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading the dynamic configuration file and loading the metadata after expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
According to the cluster expansion and contraction method, device, equipment, medium and program product, under the condition that expansion and contraction processing is carried out on the cluster to be processed, the dynamic configuration file is directly generated according to the metadata of each member node in the cluster to be processed after expansion and contraction, manual processing by operation and maintenance personnel is not needed, the generation efficiency of the dynamic configuration file is improved, the time cost and the labor cost of operation and maintenance are reduced, meanwhile, the situation that configuration is wrong due to human factors is avoided, and the accuracy of the generation result of the dynamic configuration file is improved. After the dynamic configuration file is generated, the dynamic configuration file is loaded, and the metadata after the expansion and contraction is loaded into the memory for subsequent use, so that file configuration does not need to be repeatedly performed before the next expansion and contraction of the cluster to be processed is performed. And selecting a cluster master node from member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction so as to start a new working cycle of the cluster and ensure normal execution of cluster service. In the process, the situation that the external service is suspended due to the fact that the number of member nodes involved in the capacity expansion and contraction processing is large is effectively avoided, and therefore the risk of stopping service caused by the capacity expansion and contraction of the cluster is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flow chart of a method for cluster expansion and contraction in one embodiment;
FIG. 2 is a flow chart of metadata acquisition steps in one embodiment;
FIG. 3 is a flow chart illustrating steps for dynamic profile generation in one embodiment;
FIG. 4 is a flow chart of a method for cluster expansion and contraction according to another embodiment;
FIG. 5 is a block diagram of a cluster expansion and contraction device according to one embodiment;
fig. 6 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The cluster expansion and contraction processing method provided by the embodiment of the application is suitable for the situation of carrying out expansion and contraction processing on member nodes of the cluster to be processed. In an exemplary embodiment, as shown in fig. 1, a cluster scaling method is provided, which can be applied to any member node in a cluster to be processed, and includes the following steps:
S110, under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction.
The dynamic configuration file is different from a static configuration file configured by member nodes in the to-be-processed cluster under the condition of first starting and carrying a cluster address of the added cluster and other unified configuration parameters, and can be understood as a starting configuration file dynamically generated under the condition that the member nodes in the to-be-processed cluster are changed.
The cluster to be processed may be any distributed cluster in the conventional technology, and the application is not limited in any way. In one particular implementation, the cluster to be processed may be a ZooKeeper cluster. Accordingly, the dynamic profile may be a zoo.cfg file.
For example, the operation and maintenance personnel can trigger the processing cluster to perform the capacity expansion and contraction processing, or any member node in the processing cluster senses whether the capacity expansion and contraction processing of the processing cluster exists through whether the member node in the processing cluster changes.
The metadata of the member nodes carries node identification information and node description information of the member nodes. The node identification information is used for uniquely characterizing the member nodes, and different member nodes can be distinguished through the information. The node description information may be understood as an attribute description of a member node, and the like. The attribute description dimension of the member node is not limited, and can be set or adjusted by a technician according to actual needs or experience.
In an alternative embodiment, if the metadata of the member nodes in the to-be-processed cluster is monitored to change, determining that the to-be-processed cluster has a capacity expansion processing condition.
For example, a monitoring thread dedicated to monitoring metadata of the participating clusters may be set in the member node. When there is a change in metadata, for example, at least one of addition of metadata of a newly added node, deletion of metadata of an already added node, modification of metadata of an already added node, and the like.
In a specific implementation, member nodes in a cluster to be processed may be subscribed to metadata of the participating clusters; when the metadata changes, metadata change information is received.
Under the condition of capacity expansion and contraction processing, metadata of each member node in the cluster to be processed after capacity expansion and contraction processing is obtained from local or other storage devices in communication connection; and generating a dynamic configuration file according to the acquired metadata. By way of example, the dynamic profile may be generated locally at the member node, as the application is not limited in this regard.
S120, loading the dynamic configuration file, and loading the metadata after expansion and contraction into the memory.
For example, the dynamic configuration file may be loaded in the member node by a hot loading manner for use in starting up the member device. And the metadata after expansion and contraction are loaded into the memory, so as to lay a foundation for normally providing service for a new working cycle of the cluster.
S130, selecting a cluster master node from member nodes of the expanded and contracted to-be-processed cluster according to the metadata after expansion and contraction.
After the metadata after expansion and contraction is loaded into the memory, a cluster selection mechanism is actively triggered, so that a cluster master node and a cluster slave node are selected again from member nodes of the cluster to be processed after expansion and contraction according to the metadata after expansion and contraction in the memory, and preparation is made for providing services to the outside.
The cluster selection mechanism adopted by the application can adopt at least one cluster selection mode in the traditional technology, and the application is not limited in any way. For example, the selection of the cluster can be participated in through a consistency algorithm based on the election port, vote information is sent to other member nodes of the cluster, and voting are performed at least one round until a cluster master node is selected, and the other member nodes in the cluster are used as cluster slave nodes.
According to the embodiment of the application, under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, the dynamic configuration file is directly generated according to the metadata of each member node in the to-be-processed cluster after capacity expansion and contraction, and the manual processing of operation and maintenance personnel is not needed, so that the generation efficiency of the dynamic configuration file is improved, the time cost and the labor cost of operation and maintenance are reduced, and meanwhile, the situation that configuration is wrong due to human factors is avoided, and the accuracy of the generation result of the dynamic configuration file is improved. After the dynamic configuration file is generated, the dynamic configuration file is loaded, and the metadata after the expansion and contraction is loaded into the memory for subsequent use, so that file configuration does not need to be repeatedly performed before the next expansion and contraction of the cluster to be processed is performed. And selecting a cluster master node from member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction so as to start a new working cycle of the cluster and ensure normal execution of cluster service. In the process, the situation that the external service is suspended due to the fact that the number of member nodes involved in the capacity expansion and contraction processing is large is effectively avoided, and therefore the risk of stopping service caused by the capacity expansion and contraction of the cluster is reduced. Based on the technical solutions of the foregoing embodiments, the present application further provides an optional embodiment, where the step of obtaining metadata of each member node in the to-be-processed cluster after expansion and contraction is refined.
Referring to the metadata acquisition step shown in fig. 2, it includes:
S210, acquiring a member change request; the member change request comprises member information of the member node to be changed and a change type.
The member node to be changed may include at least one of a member node to be added and a member node to be deleted. Wherein the member information may include node identification information. Or further, the member information may include other member node information, such as node description information. Wherein the change type comprises addition and/or deletion.
In an alternative embodiment, the member change request may be generated in response to a scaling operation on the operation-and-maintenance interface.
The operation of expanding and shrinking the operation and maintenance interface can adopt at least one operation mode in the conventional technology, and the application is not limited in any way.
Optionally, in the capacity expansion scenario, the capacity expansion operation on the operation and maintenance interface may include a node adding operation for a newly added node. The node to be newly added can be the member node itself or other member nodes. Or alternatively, in the capacity reduction scene, the capacity expansion operation on the operation and maintenance interface can comprise a node deleting operation of the node to be deleted. The node to be deleted can be the member node itself or other member nodes.
In another alternative embodiment, the member change request may be generated in response to metadata synchronization operations initiated by other member nodes in the pending cluster.
When any member node in the to-be-processed cluster receives the capacity expansion operation on the operation and maintenance interface, a member change request is generated in response to the capacity expansion operation, and the member node information in the metadata is updated according to the member information and the change type of the member node to be changed in the member change request, so that the metadata of each member node in the to-be-processed cluster after capacity expansion is obtained. In addition, after the member change request is generated, a metadata synchronization instruction is also initiated to other member nodes. Correspondingly, other member nodes respond to the metadata synchronization instruction to generate a member change request, and the member node information in the metadata is updated according to the member information and the change type of the member node to be changed in the member change request, so that the metadata of each member node in the expanded and contracted cluster to be processed is obtained.
It can be understood that the member change request is generated in different modes, so that the update processing of the metadata of the cluster to be processed is realized, and the richness and diversity of the update mode of the metadata are improved.
S220, according to the member information and the change type of the member node to be changed, the member node information in the metadata is updated, and the metadata of each member node in the expanded and contracted cluster to be processed is obtained.
If the change type is newly added, the corresponding member node to be changed is the node to be newly added, and the member node information of the node to be added is newly added in the existing metadata according to the member information of the node to be newly added. If the change type is delete, the corresponding member node to be changed is the node to be deleted, and the member node information of the node to be deleted is deleted in the existing metadata according to the member information of the node to be deleted.
According to the embodiment of the application, the metadata of each member node in the expanded and contracted cluster to be processed is obtained by newly obtaining the member change request and updating the member grounding information in the metadata according to the member information and the change type of the member node to be changed in the member change request, so that the method for obtaining the metadata of the member node after the expansion and contraction is provided, the operation is convenient and quick, the obtaining efficiency of the metadata after the expansion and contraction and the accuracy of the obtaining result are improved, and the condition that the metadata change is not timely obtained and the expansion and contraction delay occurs is avoided.
Based on the technical solutions of the above embodiments, the present application further provides an optional embodiment, in which the step of generating the dynamic configuration file is refined.
Referring to the dynamic profile generation step shown in fig. 3, it includes:
S310, generating a configuration file copy of the dynamic configuration file of the current use version.
Wherein the dynamic configuration file of the currently used version is usually the dynamic configuration file of the latest version. The version of the dynamic profile may be determined by at least one of a generation time of the dynamic profile, a file name, a storage location name, and the like.
And S320, updating member node addresses in the configuration file copies according to metadata of each member node in the expanded and contracted to-be-processed cluster, and obtaining the latest version of dynamic configuration file.
For example, metadata after expansion and contraction can be substituted for metadata in the configuration file copy to obtain the dynamic configuration file of the latest version.
Correspondingly, the dynamic configuration file is loaded, in particular the dynamic configuration file of the latest version is loaded.
In an alternative embodiment, under the condition that the node restarting requirement exists, the latest version of dynamic configuration file is loaded in response to the node restarting operation, so that the phenomenon that the member node cannot normally participate in the cluster due to the fact that the dynamic configuration file is wrongly loaded in version is avoided.
According to the embodiment of the application, the configuration file copy is generated on the basis of the dynamic configuration file of the current use version, and the member node addresses in the configuration file copy are updated according to the metadata after the expansion and contraction, so that each member node in the cluster to be processed after the expansion and contraction is ensured to be capable of acquiring the corresponding member node address in the updated metadata at the first time of the metadata change moment, the accuracy and timeliness of the member node addresses in the dynamic configuration file are improved, and the timeliness and the accuracy of the dynamic configuration file generation result are improved.
Based on the technical solutions of the above embodiments, the present application further provides an optional embodiment, in which the operation process in the cluster expansion and contraction scenario is described in detail.
Referring to fig. 4, the cluster scaling method is applied to any member node in a cluster to be processed, and includes:
and S410, adding a cluster address of a cluster to be participated in a static configuration file of the node to be newly added in a cluster capacity expansion scene, synchronizing data, and executing S420.
And S420, under the expansion and contraction scene, responding to the expansion and contraction operation of the operation and maintenance personnel on the operation and maintenance interface, and generating a member change request.
S430, updating the metadata of each member node in the locally stored to-be-processed cluster according to the change type in the member change request and the member information of the to-be-changed member node.
S440, sending member change requests to other member nodes.
The steps S430 and S440 may be performed sequentially or simultaneously, and the execution sequence of the two steps is not limited in the present application.
S450, other member nodes update the metadata of each member node in the locally stored to-be-processed cluster according to the change type in the member change request and the member information of the to-be-changed member node.
S460, each member node respectively generates a configuration file copy of the dynamic configuration file of the current use version.
And S470, updating member node addresses in the configuration file copies according to the metadata after the expansion and contraction to obtain the dynamic configuration file of the latest version.
S480, loading the dynamic configuration file, and loading the metadata after expansion and contraction into the memory.
S490, each member node in the cluster to be processed selects a cluster master node again according to the metadata after expansion and contraction.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a cluster expansion and contraction device for realizing the above-mentioned cluster expansion and contraction method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the cluster expansion and contraction device provided below may refer to the limitation of the cluster expansion and contraction method hereinabove, and will not be repeated herein.
In an exemplary embodiment, as shown in fig. 5, there is provided a cluster expansion and contraction device, including: a profile generation module 510, a profile loading module 520, and a master node selection module 530. Wherein,
The configuration file generating module 510 is configured to generate a dynamic configuration file according to metadata of each member node in the cluster to be processed after the expansion and contraction under the condition that the cluster to be processed is subjected to the expansion and contraction processing;
The configuration file loading module 520 is configured to load a dynamic configuration file and load metadata after expansion and contraction into a memory;
the master node selecting module 530 is configured to select a cluster master node from the member nodes of the expanded and contracted to-be-processed cluster according to the expanded and contracted metadata.
In one embodiment, the apparatus further includes a first obtaining module configured to obtain a member change request; the member change request comprises member information and change types of the member nodes to be changed; the change type comprises new addition and/or deletion; and the first updating module is used for updating the member node information in the metadata according to the member information and the change type of the member node to be changed to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
In one embodiment, the first acquisition module includes: the first response unit is used for responding to the expansion and contraction operation of the operation and maintenance interface and generating a member change request; or the second response unit is used for responding to the metadata synchronization instruction initiated by other member nodes in the cluster to be processed and generating a member change request.
In one embodiment, the first response unit is further configured to, after generating the member change request in response to the capacity expansion operation on the operation and maintenance interface, initiate a metadata synchronization instruction to other member nodes in the to-be-processed cluster, and instruct the other member nodes to respond to the metadata synchronization instruction, generate the member change request, and update member node information in the metadata according to member information and change types of the member nodes to be changed in the member change request, so as to obtain metadata of each member node in the to-be-processed cluster after capacity expansion.
In one embodiment, the profile generation module 510 includes: the copy generation unit is used for generating a configuration file copy of the dynamic configuration file of the current use version; and the copy updating unit is used for updating the addresses of the member nodes in the configuration file copy according to the metadata of each member node in the expanded and contracted cluster to be processed, so as to obtain the dynamic configuration file of the latest version.
In one embodiment, the apparatus further includes a restart loading module, configured to load the latest version of the dynamic configuration file in response to a node restart operation.
The modules in the cluster expansion and contraction device can be realized in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one exemplary embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store dynamic configuration files, metadata, and the like. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a cluster expansion and contraction method.
It will be appreciated by those skilled in the art that the structure shown in FIG. 6 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one exemplary embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading a dynamic configuration file, and loading the metadata subjected to capacity expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In one embodiment, the processor when executing the computer program further performs the steps of: acquiring a member change request; the member change request comprises member information and change types of the member nodes to be changed; the change type comprises new addition and/or deletion; and updating the member node information in the metadata according to the member information and the change type of the member node to be changed to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
In one embodiment, the processor when executing the computer program further performs the steps of: responding to the capacity expansion and contraction operation of the operation and maintenance interface, and generating a member change request; or generating a member change request in response to a metadata synchronization instruction initiated by other member nodes in the cluster to be processed.
In one embodiment, the processor when executing the computer program further performs the steps of: after a member change request is generated in response to the capacity expansion operation on the operation and maintenance interface, a metadata synchronization instruction is initiated to other member nodes in the cluster to be processed, the metadata synchronization instruction is used for indicating the other member nodes to respond to the metadata synchronization instruction, the member change request is generated, and the member node information in the metadata is updated according to the member information and the change type of the member nodes to be changed in the member change request, so that the metadata of each member node in the cluster to be processed after capacity expansion and contraction is obtained.
In one embodiment, the processor when executing the computer program further performs the steps of: generating a configuration file copy of the dynamic configuration file of the current use version; and updating the addresses of the member nodes in the configuration file copies according to the metadata of each member node in the expanded and contracted to-be-processed cluster, so as to obtain the latest version of dynamic configuration file.
In one embodiment, the processor when executing the computer program further performs the steps of: and in response to the node restarting operation, loading the latest version of dynamic configuration file.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading a dynamic configuration file, and loading the metadata subjected to capacity expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a member change request; the member change request comprises member information and change types of the member nodes to be changed; the change type comprises new addition and/or deletion; and updating the member node information in the metadata according to the member information and the change type of the member node to be changed to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
In one embodiment, the computer program when executed by the processor further performs the steps of: responding to the capacity expansion and contraction operation of the operation and maintenance interface, and generating a member change request; or generating a member change request in response to a metadata synchronization instruction initiated by other member nodes in the cluster to be processed.
In one embodiment, the computer program when executed by the processor further performs the steps of: after a member change request is generated in response to the capacity expansion operation on the operation and maintenance interface, a metadata synchronization instruction is initiated to other member nodes in the cluster to be processed, the metadata synchronization instruction is used for indicating the other member nodes to respond to the metadata synchronization instruction, the member change request is generated, and the member node information in the metadata is updated according to the member information and the change type of the member nodes to be changed in the member change request, so that the metadata of each member node in the cluster to be processed after capacity expansion and contraction is obtained.
In one embodiment, the computer program when executed by the processor further performs the steps of: generating a configuration file copy of the dynamic configuration file of the current use version; and updating the addresses of the member nodes in the configuration file copies according to the metadata of each member node in the expanded and contracted to-be-processed cluster, so as to obtain the latest version of dynamic configuration file.
In one embodiment, the computer program when executed by the processor further performs the steps of: and in response to the node restarting operation, loading the latest version of dynamic configuration file.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading a dynamic configuration file, and loading the metadata subjected to capacity expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a member change request; the member change request comprises member information and change types of the member nodes to be changed; the change type comprises new addition and/or deletion; and updating the member node information in the metadata according to the member information and the change type of the member node to be changed to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
In one embodiment, the computer program when executed by the processor further performs the steps of: responding to the capacity expansion and contraction operation of the operation and maintenance interface, and generating a member change request; or generating a member change request in response to a metadata synchronization instruction initiated by other member nodes in the cluster to be processed.
In one embodiment, the computer program when executed by the processor further performs the steps of: after a member change request is generated in response to the capacity expansion operation on the operation and maintenance interface, a metadata synchronization instruction is initiated to other member nodes in the cluster to be processed, the metadata synchronization instruction is used for indicating the other member nodes to respond to the metadata synchronization instruction, the member change request is generated, and the member node information in the metadata is updated according to the member information and the change type of the member nodes to be changed in the member change request, so that the metadata of each member node in the cluster to be processed after capacity expansion and contraction is obtained.
In one embodiment, the computer program when executed by the processor further performs the steps of: generating a configuration file copy of the dynamic configuration file of the current use version; and updating the addresses of the member nodes in the configuration file copies according to the metadata of each member node in the expanded and contracted to-be-processed cluster, so as to obtain the latest version of dynamic configuration file.
In one embodiment, the computer program when executed by the processor further performs the steps of: and in response to the node restarting operation, loading the latest version of dynamic configuration file.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A cluster scaling method, applied to any member node in a cluster to be processed, comprising:
Under the condition of carrying out capacity expansion and contraction processing on the to-be-processed cluster, generating a dynamic configuration file according to metadata of each member node in the to-be-processed cluster after capacity expansion and contraction;
Loading the dynamic configuration file and loading the metadata after expansion and contraction into a memory;
and selecting a cluster master node from the member nodes of the cluster to be processed after the capacity expansion and contraction according to the metadata after the capacity expansion and contraction.
2. The method according to claim 1, wherein the method further comprises:
Acquiring a member change request; the member change request comprises member information and change types of member nodes to be changed; the change type comprises addition and/or deletion;
and updating the member node information in the metadata according to the member information of the member node to be changed and the change type to obtain the metadata of each member node in the expanded and contracted cluster to be processed.
3. The method of claim 2, wherein the obtaining the member change request comprises:
responding to the expansion and contraction operation of the operation and maintenance interface, and generating the member change request; or alternatively
And generating the member change request in response to metadata synchronization instructions initiated by other member nodes in the to-be-processed cluster.
4. The method of claim 3, wherein after generating the member change request in response to a scaling operation on an operation-to-dimension interface, the method further comprises:
And initiating a metadata synchronization instruction to other member nodes in the to-be-processed cluster, wherein the metadata synchronization instruction is used for indicating the other member nodes to respond to the metadata synchronization instruction, generating a member change request, and updating member node information in metadata according to member information and change types of the member nodes to be changed in the member change request to obtain metadata of each member node in the to-be-processed cluster after expansion and contraction.
5. The method according to any one of claims 1-4, wherein generating the dynamic configuration file according to metadata of each member node in the scalable to-be-processed cluster includes:
Generating a configuration file copy of the dynamic configuration file of the current use version;
and updating the member node addresses in the configuration file copies according to the metadata of each member node in the expanded and contracted to-be-processed cluster to obtain the latest version of dynamic configuration file.
6. The method of claim 5, wherein the method further comprises:
And in response to the node restarting operation, loading the latest version of dynamic configuration file.
7. A cluster expansion and contraction device, characterized in that it is configured at any member node in a cluster to be processed, and comprises:
The configuration file generation module is used for generating a dynamic configuration file according to metadata of each member node in the cluster to be processed after expansion and contraction under the condition of carrying out expansion and contraction processing on the cluster to be processed;
The configuration file loading module is used for loading the dynamic configuration file and loading the metadata subjected to expansion and contraction into the memory;
The main node selection module is used for selecting a cluster main node from member nodes of the expanded and contracted to-be-processed cluster according to the metadata after the expansion and contraction.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1-6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410274175.5A CN118069293A (en) | 2024-03-11 | 2024-03-11 | Cluster expansion and contraction method, device, equipment, medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410274175.5A CN118069293A (en) | 2024-03-11 | 2024-03-11 | Cluster expansion and contraction method, device, equipment, medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118069293A true CN118069293A (en) | 2024-05-24 |
Family
ID=91097052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410274175.5A Pending CN118069293A (en) | 2024-03-11 | 2024-03-11 | Cluster expansion and contraction method, device, equipment, medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118069293A (en) |
-
2024
- 2024-03-11 CN CN202410274175.5A patent/CN118069293A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107357688B (en) | Distributed system and fault recovery method and device thereof | |
| US11860833B2 (en) | Data redistribution method, apparatus, and system | |
| CN112015595B (en) | Master-slave database switching method, computing device and storage medium | |
| CN111209084B (en) | FAAS distributed computing method and device | |
| CN107943615B (en) | Data processing method and system based on distributed cluster | |
| CN111414356A (en) | Data storage method and device, non-relational database system and storage medium | |
| CN116389233B (en) | Container cloud management platform active-standby switching system, method and device and computer equipment | |
| CN114363356B (en) | Data synchronization method, system, device, computer equipment and storage medium | |
| CN116048878A (en) | Business service recovery method, device and computer equipment | |
| CN118069293A (en) | Cluster expansion and contraction method, device, equipment, medium and program product | |
| CN111752892A (en) | Distributed file system, method for implementing the same, management system, device, and medium | |
| CN114416438A (en) | Data export method and device, computer equipment and scheduling service system | |
| CN113672665A (en) | Data processing method, data acquisition system, electronic device and storage medium | |
| CN112527760A (en) | Data storage method, device, server and medium | |
| CN115730016B (en) | Data synchronization method, system, device, computer equipment and storage medium | |
| CN117478299B (en) | Block chain consensus algorithm switching method, device and computer equipment | |
| CN115840756B (en) | Data updating method of main and standby modulation system, equipment and medium | |
| Eberhardt et al. | Smac: State management for geo-distributed containers | |
| US11657293B2 (en) | Asynchronous architecture for evolutionary computation techniques | |
| CN114201465A (en) | Data integration method, system, computer device and storage medium | |
| CN117290160A (en) | Backup method, storage medium and equipment for database read-write separation cluster | |
| CN116633764A (en) | System switching method, apparatus, computer device, storage medium and computer program product | |
| CN117290354A (en) | Data processing method, device, computer equipment and storage medium | |
| CN118069598A (en) | Data writing method, device, computer equipment and storage medium | |
| CN117560424A (en) | Task allocation method, device, system, apparatus, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |