CN112685055A - Cluster construction method and device - Google Patents
Cluster construction method and device Download PDFInfo
- Publication number
- CN112685055A CN112685055A CN202110013596.9A CN202110013596A CN112685055A CN 112685055 A CN112685055 A CN 112685055A CN 202110013596 A CN202110013596 A CN 202110013596A CN 112685055 A CN112685055 A CN 112685055A
- Authority
- CN
- China
- Prior art keywords
- cluster
- target
- template
- task
- target cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Stored Programmes (AREA)
Abstract
The application discloses a cluster construction method and device, and relates to the field of cloud computing. Wherein, the method comprises the following steps: acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed; acquiring an initial cluster template corresponding to the cluster parameters from a cluster template library; rendering the initial cluster template by using the cluster parameters to obtain a target cluster template; and constructing a target cluster on the cloud platform by using the target cluster template, wherein the target cluster is used for running the project task. The method and the device solve the technical problem that the complexity of constructing the cluster is high in the related technology.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a method and an apparatus for constructing a cluster.
Background
With the development of big data and AI technologies, various industries are aware of the value of big data technologies to product services of the industries, the use and the construction of big data platforms are the problems faced by many companies, a common way is to construct a physical cluster or purchase a cloud host at a cloud provider to construct a cluster, the problems faced are mostly complex component deployment and configuration, and for students and companies with a common scale, it takes time and money to purchase and maintain a big data cluster.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The application provides a cluster construction method and device, which are used for at least solving the technical problem of high complexity of cluster construction in the related technology.
According to an aspect of an embodiment of the present application, there is provided a cluster building method, including: acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed; acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library; rendering the initial cluster template by using the cluster parameters to obtain a target cluster template; and constructing a target cluster on a cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
Optionally, after building a target cluster on a cloud platform using the target cluster template, the method further comprises: judging whether the project task is operated and completed in the target cluster; and under the condition that the project task runs and is completed in the target cluster, destroying the target cluster on the cloud platform.
Optionally, the obtaining the initial cluster template corresponding to the cluster parameter from the cluster template library includes: acquiring an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, wherein a plurality of big data components are stored in the cluster template library; and constructing the initial cluster template by using the initial big data component mirror image.
Optionally, the obtaining an initial cluster template corresponding to the cluster parameter from a cluster template library includes: calling a package manager plug-in; acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in; rendering the initial cluster template using the cluster parameters to obtain a target cluster template comprises: and rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
Optionally, constructing the target cluster on the cloud platform using the target cluster template includes: constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform; and deploying the mirror image of each target big data component in a component container corresponding to the mirror image of each target big data component to obtain the target cluster.
Optionally, after building a target cluster on a cloud platform using the target cluster template, the method further comprises: judging whether the target cluster is constructed; and submitting the project task to the target cluster under the condition that the target cluster is constructed completely.
Optionally, submitting the project task to the target cluster comprises: constructing a task container in the target cluster; submitting the project task into the task container.
Optionally, the obtaining of the project task submitted by the user and the cluster parameter of the cluster to be constructed includes: receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
According to another aspect of the embodiments of the present application, there is also provided a cluster building apparatus, including: the first acquisition module is used for acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed; the second acquisition module is used for acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library; the rendering module is used for rendering the initial cluster template by using the cluster parameters to obtain a target cluster template; and the building module is used for building a target cluster on the cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
Optionally, the apparatus further comprises: the first judgment module is used for judging whether the project task is operated and completed in the target cluster after the target cluster is constructed on the cloud platform by using the target cluster template; and the destruction module is used for destroying the target cluster on the cloud platform under the condition that the project task runs and finishes in the target cluster.
Optionally, the second obtaining module includes: an obtaining unit, configured to obtain an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, where a plurality of big data components are stored in the cluster template library; and the first construction unit is used for constructing the initial cluster template by using the initial big data component mirror image.
Optionally, the second obtaining module is configured to: calling a package manager plug-in; acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in; the rendering module is to: and rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
Optionally, the building module comprises: the second construction unit is used for constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform; and the deployment unit is used for deploying the mirror image of each target big data component in the component container corresponding to the mirror image of each target big data component to obtain the target cluster.
Optionally, the apparatus further comprises: the second judgment module is used for judging whether the target cluster is constructed or not after the target cluster is constructed on the cloud platform by using the target cluster template; and the submitting module is used for submitting the project task to the target cluster under the condition that the target cluster is constructed.
Optionally, the submission module includes: a third constructing unit, configured to construct a task container in the target cluster; and the submitting unit is used for submitting the project task to the task container.
Optionally, the first obtaining module is configured to: receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.
In the embodiment of the application, project tasks submitted by a user and cluster parameters of a cluster to be constructed are acquired; acquiring an initial cluster template corresponding to the cluster parameters from a cluster template library; rendering the initial cluster template by using the cluster parameters to obtain a target cluster template; the method comprises the steps of using a target cluster template to construct a target cluster on a cloud platform, wherein the target cluster is used for acquiring a project task submitted by a user and cluster parameters of a cluster required to be constructed for the project task in a mode of operating the project task, acquiring a corresponding template from a cluster template library for rendering, and constructing the rendered target cluster template in the cloud platform, so that the purpose of deploying the cluster by one key is achieved, the technical effect of reducing the complexity of constructing the cluster is achieved, and the technical problem of high complexity of constructing the cluster in the related technology is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a schematic diagram of a hardware environment for a method of building a cluster according to an embodiment of the present application;
FIG. 2 is a flow chart of an alternative method of cluster construction according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a big data component deployment architecture, according to an alternative embodiment of the present application;
FIG. 4 is a schematic diagram of a process for constructing a big data cluster, according to an alternative embodiment of the present application;
FIG. 5 is a schematic diagram of an alternative cluster building apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of embodiments of the present application, an embodiment of a method for constructing a cluster is provided.
Optionally, in this embodiment, the above-described cluster building method may be applied to a hardware environment formed by the cluster building server 101 and the cloud platform server 103 as shown in fig. 1. As shown in fig. 1, a clustered building server 101 is connected to a cloud platform server 103 through a network, which may be used for building a cluster on the cloud platform server 103 for a user, and a database may be provided on the server or independent from the server for providing a data storage service for the building server 101, where the network includes but is not limited to: a wide area network, a metropolitan area network, or a local area network. The cluster building method according to the embodiment of the present application may be executed by the cluster building server 101.
Fig. 2 is a flowchart of an alternative cluster building method according to an embodiment of the present application, and as shown in fig. 2, the method may include the following steps:
step S202, acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed;
step S204, obtaining an initial cluster template corresponding to the cluster parameter from a cluster template library;
step S206, rendering the initial cluster template by using the cluster parameters to obtain a target cluster template;
step S208, constructing a target cluster on the cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
Through the steps S202 to S208, the project task submitted by the user and the cluster parameter of the cluster that needs to be constructed for the project task are obtained, the corresponding template is obtained from the cluster template library to be rendered, and the rendered target cluster template is constructed in the cloud platform, so that the purpose of deploying the cluster by one key is achieved, thereby achieving the technical effect of reducing the complexity of constructing the cluster, and further solving the technical problem of high complexity of constructing the cluster in the related art.
Optionally, in this embodiment, the above cluster building method may be applied, but not limited to, in a scenario where a cluster is built for a user on a cloud platform. The cloud platform may include, but is not limited to, kubernets and the like.
In the technical solution provided in step S202, the project task submitted by the user may include, but is not limited to: program code of the project task, information of the project task acquired from the program code of the project task, and the like.
Optionally, in this embodiment, the cluster parameter of the cluster to be constructed may be, but is not limited to, actively submitted by a user, or may also be automatically generated according to a project task submitted by the user.
Optionally, in this embodiment, the cluster parameters may include, but are not limited to: cluster size, resource configuration, list of installed components, etc.
In the above step S202, the project task and the cluster parameter may be obtained by, but are not limited to, the following manners:
and S11, receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
Optionally, in this embodiment, the project submission tool is configured to extract task information and cluster parameters from a code library, such as: the project submission tool may be, but is not limited to, a Jenkins tool, and the like.
Optionally, in this embodiment, the code library is used to store task codes of project tasks submitted by users. Such as: the code library can be but not limited to a gitlab, a user can submit a written task code to the gitlab by one key, and after Jenkins obtains the code submission condition, information of a project task and cluster parameters (such as resource size, scale and installation component list) of a cluster required to be created are sent to a tool or a plug-in unit for the user to construct the cluster.
In the technical solution provided in step S204, the cluster template library may include, but is not limited to, a large number of big data components (for example: kafka components and the like, and kafka components may include, but are not limited to: brooker, zk and the like) stored in the cluster template library, and the cluster template library may include, but is not limited to: a helm chat mirror source, and so on.
In the above step S204, the initial cluster template corresponding to the cluster parameter may be obtained by, but is not limited to:
s21, acquiring an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, wherein a plurality of big data components are stored in the cluster template library;
s22, constructing the initial cluster template by using the initial big data component mirror image.
Optionally, in this embodiment, the big data component may be obtained, but not limited to, by mirroring. The initial big data component image obtained may be, but is not limited to being, determined according to the cluster size and the list of installed components included in the cluster parameters.
Optionally, in this embodiment, the big data component includes tools and systems in the production of the internet big data ecosystem, such as hadoop, hbase, elastic search, and the like.
In the technical solution provided in step S206, rendering may be, but is not limited to, according to a resource configuration included in the cluster parameter.
In the step S204, the initial cluster template corresponding to the cluster parameter may be obtained from the cluster template library by, but is not limited to:
s31, calling a package manager plug-in;
s32, acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in;
in the above step S206, the initial cluster template may be rendered by, but is not limited to:
s33, rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
Optionally, in this embodiment, the process of obtaining the initial cluster template and rendering the initial cluster template may be performed, but is not limited to, by calling a package manager plug-in. The package manager plug-in may be, but is not limited to, a hell plug-in. The helm plug-in is a plug-in of kubernets and can perform configuration rendering and container management, and the helm plug-in completes functions of acquiring a corresponding big data component mirror image, building and destroying a cluster or submitting a task and the like after acquiring cluster parameters.
In the technical solution provided in step S208, the cloud platform may be, but is not limited to, a container management system (e.g., kubernets). A target cluster for running a project task may be built in a container management system using, but not limited to, containerization techniques (e.g., docker). The method can realize rapid deployment and installation of the big data assembly by using docker templated configuration and mirroring, manage the docker container by using kubernets, and perform deployment task arrangement by using plug-in helm of the kubernets.
In the above step S208, the target cluster may be built on the cloud platform by, but not limited to:
s41, constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform;
s42, deploying the mirror image of each target big data component in a component container corresponding to the mirror image of each target big data component to obtain the target cluster.
Optionally, in this embodiment, the target cluster template building may be deployed on the cloud platform, but not limited to, using a container virtualization technology. The container isolates different processes running on the host through a virtualization technology, thereby achieving the purposes of isolating processes from each other and not influencing each other.
Alternatively, in this embodiment, the component container constructed for each target big data component image may be, but is not limited to, a container type suitable for the target big data component image among a plurality of container types.
As an alternative embodiment, after the step S208, the method further includes:
s51, judging whether the project task is operated and completed in the target cluster;
s52, destroying the target cluster on the cloud platform when the project task runs and finishes in the target cluster.
Optionally, in this embodiment, after the project task is completed, the target cluster on the cloud platform is automatically destroyed, it is seen that the constructed target cluster belongs to a non-resident cluster (that is, the cluster is destroyed after being used up, and the submitted task is automatically constructed), and after the user writes the task, the cluster is automatically created by one key, the task is submitted, and after the task is executed, the cluster is automatically destroyed to release resources. And realizing the efficient reuse of resources.
Optionally, in this embodiment, the operation of destroying the target cluster may be, but is not limited to, a process of closing the target cluster and clearing the deployment directory of the target cluster, so as to release the resources occupied by the target cluster.
As an alternative embodiment, after the step S208, the method further includes:
s61, judging whether the target cluster is constructed completely;
s62, submitting the project task to the target cluster when the target cluster is constructed.
Optionally, in this embodiment, after the target cluster is constructed, the project task may be submitted to the target cluster to be executed.
Optionally, in this embodiment, the manner of submitting the project task may include, but is not limited to: a way to create a pod, a way to use the REST api interface provided by big data, etc.
As an alternative embodiment, project tasks may be submitted to the target cluster by, but are not limited to:
s71, constructing a task container in the target cluster;
s72, submitting the project task to the task container.
Optionally, in this embodiment, a project task may be a container POD of JOB type (task type) for the cloud platform, and the container is destroyed after being executed, and the task written by the user is encapsulated in the POD.
The application also provides an optional embodiment, the optional embodiment provides a method for rapid deployment and task submission of a big data component based on a container virtualization technology, fig. 3 is a schematic diagram of a big data component deployment architecture according to the optional embodiment of the application, as shown in fig. 3, a program task written by a user is submitted to a gitlab code library, Jenkins is a project submission tool and can execute some tasks, and after acquiring a code submission condition in the gitlab code library, Jenkins informs the task information, cluster parameters (resource size, scale, installation component list) of a cluster to be created, and other task information to the hellm. The helm is a plug-in of kubernets and can perform configuration rendering and container management, and after acquiring task information, the helm acquires a corresponding big data component mirror image to complete the operation of constructing and destroying a cluster or submitting a task. A project task is a container POD of JOB type for kubernets, and the project task is destroyed after being executed, and tasks written by a user are packaged in the POD.
In this optional embodiment, a process of constructing a big data cluster through the above framework is further provided, and fig. 4 is a schematic diagram of a process of constructing a big data cluster according to an optional embodiment of the present application, where as shown in fig. 4, a user writes a task program and submits the task program to a gitlab item, Jenkins senses the submission of the task in gitlab, and issues task and cluster description information to a helm, which renders a template and creates a cluster. And checking whether the cluster is created completely, and if not, continuing to wait. If so, a user task pod is created. And then judging whether the task is operated and completed in the cluster, and if not, continuing to wait. If so, the cluster container is destroyed.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling an electronic device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
According to another aspect of the embodiment of the present application, there is also provided a cluster building apparatus for implementing the cluster building method. Fig. 5 is a schematic diagram of an alternative cluster building apparatus according to an embodiment of the present application, and as shown in fig. 5, the apparatus may include:
a first obtaining module 52, configured to obtain a project task submitted by a user and a cluster parameter of a cluster to be constructed;
a second obtaining module 54, configured to obtain an initial cluster template corresponding to the cluster parameter from a cluster template library;
a rendering module 56, configured to render the initial cluster template using the cluster parameters to obtain a target cluster template;
a building module 58 configured to build a target cluster on a cloud platform using the target cluster template, wherein the target cluster is used to run the project task.
It should be noted that the first obtaining module 52 in this embodiment may be configured to execute step S202 in this embodiment, the second obtaining module 54 in this embodiment may be configured to execute step S204 in this embodiment, the rendering module 56 in this embodiment may be configured to execute step S206 in this embodiment, and the constructing module 58 in this embodiment may be configured to execute step S208 in this embodiment.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 1, and may be implemented by software or hardware.
Through the modules, the project task submitted by a user and the cluster parameters of the cluster required to be constructed for the project task are obtained, the corresponding template is obtained from the cluster template library for rendering, and the rendered target cluster template is constructed in the cloud platform, so that the purpose of deploying the cluster by one key is achieved, the technical effect of reducing the complexity of constructing the cluster is achieved, and the technical problem of high complexity of constructing the cluster in the related technology is solved.
As an alternative embodiment, the apparatus further comprises:
the first judgment module is used for judging whether the project task is operated and completed in the target cluster after the target cluster is constructed on the cloud platform by using the target cluster template;
and the destruction module is used for destroying the target cluster on the cloud platform under the condition that the project task runs and finishes in the target cluster.
As an alternative embodiment, the second obtaining module includes:
an obtaining unit, configured to obtain an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, where a plurality of big data components are stored in the cluster template library;
and the first construction unit is used for constructing the initial cluster template by using the initial big data component mirror image.
As an alternative embodiment, the second obtaining module is configured to: calling a package manager plug-in; acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in;
the rendering module is to: and rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
As an alternative embodiment, the building block comprises:
the second construction unit is used for constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform;
and the deployment unit is used for deploying the mirror image of each target big data component in the component container corresponding to the mirror image of each target big data component to obtain the target cluster.
As an alternative embodiment, the apparatus further comprises:
the second judgment module is used for judging whether the target cluster is constructed or not after the target cluster is constructed on the cloud platform by using the target cluster template;
and the submitting module is used for submitting the project task to the target cluster under the condition that the target cluster is constructed.
As an alternative embodiment, the commit module comprises:
a third constructing unit, configured to construct a task container in the target cluster;
and the submitting unit is used for submitting the project task to the task container.
As an alternative embodiment, the first obtaining module is configured to:
receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.
According to another aspect of the embodiment of the present application, there is also provided an electronic apparatus for implementing the above-mentioned cluster building method.
Fig. 6 is a block diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 6, the electronic device may include: one or more processors 601 (only one of which is shown), a memory 603, and a transmission 605. as shown in fig. 6, the electronic apparatus may further include an input-output device 607.
The memory 603 may be configured to store software programs and modules, such as program instructions/modules corresponding to the cluster building method and apparatus in the embodiment of the present application, and the processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 603, that is, implements the above-described cluster building method. The memory 603 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 603 may further include memory located remotely from the processor 601, which may be connected to the electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The above-mentioned transmission device 605 is used for receiving or sending data via a network, and may also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 605 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 605 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
Among them, the memory 603 is used to store an application program, in particular.
The processor 601 may call the application stored in the memory 603 through the transmission device 605 to perform the following steps:
acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed;
acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library;
rendering the initial cluster template by using the cluster parameters to obtain a target cluster template;
and constructing a target cluster on a cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
By adopting the embodiment of the application, a scheme for constructing the cluster is provided. The method comprises the steps of obtaining a project task submitted by a user and cluster parameters of a cluster required to be constructed for the project task, obtaining a corresponding template from a cluster template library for rendering, and constructing a rendered target cluster template in a cloud platform, so that the purpose of deploying the cluster by one key is achieved, the technical effect of reducing the complexity of constructing the cluster is achieved, and the technical problem of high complexity of constructing the cluster in the related technology is solved.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
It will be understood by those skilled in the art that the structure shown in fig. 6 is merely an illustration, and the electronic device may be a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 6 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 6, or have a different configuration than shown in FIG. 6.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program for instructing hardware associated with an electronic device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Embodiments of the present application also provide a storage medium. Alternatively, in this embodiment, the storage medium may be a program code for executing the cluster building method.
Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:
acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed;
acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library;
rendering the initial cluster template by using the cluster parameters to obtain a target cluster template;
and constructing a target cluster on a cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (18)
1. A cluster construction method is characterized by comprising the following steps:
acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed;
acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library;
rendering the initial cluster template by using the cluster parameters to obtain a target cluster template;
and constructing a target cluster on a cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
2. The method of claim 1, wherein after building a target cluster on a cloud platform using the target cluster template, the method further comprises:
judging whether the project task is operated and completed in the target cluster;
and under the condition that the project task runs and is completed in the target cluster, destroying the target cluster on the cloud platform.
3. The method of claim 1, wherein obtaining the initial cluster template corresponding to the cluster parameter from the library of cluster templates comprises:
acquiring an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, wherein a plurality of big data components are stored in the cluster template library;
and constructing the initial cluster template by using the initial big data component mirror image.
4. The method of claim 1,
acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library comprises: calling a package manager plug-in; acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in;
rendering the initial cluster template using the cluster parameters to obtain a target cluster template comprises: and rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
5. The method of claim 1, wherein building the target cluster on a cloud platform using the target cluster template comprises:
constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform;
and deploying the mirror image of each target big data component in a component container corresponding to the mirror image of each target big data component to obtain the target cluster.
6. The method of claim 1, wherein after building a target cluster on a cloud platform using the target cluster template, the method further comprises:
judging whether the target cluster is constructed;
and submitting the project task to the target cluster under the condition that the target cluster is constructed completely.
7. The method of claim 6, wherein submitting the project task to the target cluster comprises:
constructing a task container in the target cluster;
submitting the project task into the task container.
8. The method of claim 1, wherein obtaining project tasks submitted by a user and cluster parameters of a cluster to be built comprises:
receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
9. An apparatus for constructing a cluster, comprising:
the first acquisition module is used for acquiring project tasks submitted by a user and cluster parameters of a cluster to be constructed;
the second acquisition module is used for acquiring an initial cluster template corresponding to the cluster parameter from a cluster template library;
the rendering module is used for rendering the initial cluster template by using the cluster parameters to obtain a target cluster template;
and the building module is used for building a target cluster on the cloud platform by using the target cluster template, wherein the target cluster is used for running the project task.
10. The apparatus of claim 9, further comprising:
the first judgment module is used for judging whether the project task is operated and completed in the target cluster after the target cluster is constructed on the cloud platform by using the target cluster template;
and the destruction module is used for destroying the target cluster on the cloud platform under the condition that the project task runs and finishes in the target cluster.
11. The apparatus of claim 9, wherein the second obtaining module comprises:
an obtaining unit, configured to obtain an initial big data component mirror image corresponding to the cluster parameter from the cluster template library, where a plurality of big data components are stored in the cluster template library;
and the first construction unit is used for constructing the initial cluster template by using the initial big data component mirror image.
12. The apparatus of claim 9,
the second obtaining module is configured to: calling a package manager plug-in; acquiring an initial cluster template mirror image corresponding to the cluster parameter from the cluster template library through the package manager plug-in;
the rendering module is to: and rendering the initial cluster template through the package manager plug-in to obtain a target cluster template.
13. The apparatus of claim 9, wherein the building module comprises:
the second construction unit is used for constructing a component container corresponding to each target big data component mirror image included in the target cluster template on the cloud platform;
and the deployment unit is used for deploying the mirror image of each target big data component in the component container corresponding to the mirror image of each target big data component to obtain the target cluster.
14. The apparatus of claim 9, further comprising:
the second judgment module is used for judging whether the target cluster is constructed or not after the target cluster is constructed on the cloud platform by using the target cluster template;
and the submitting module is used for submitting the project task to the target cluster under the condition that the target cluster is constructed.
15. The apparatus of claim 14, wherein the commit module comprises:
a third constructing unit, configured to construct a task container in the target cluster;
and the submitting unit is used for submitting the project task to the task container.
16. The apparatus of claim 9, wherein the first obtaining module is configured to:
receiving task information and the cluster parameters of the project task sent by a project submission tool, wherein the project submission tool is used for extracting the task information and the cluster parameters from a code library, and the code library is used for storing task codes of the project task submitted by the user.
17. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 8.
18. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 1 to 8 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110013596.9A CN112685055A (en) | 2021-01-05 | 2021-01-05 | Cluster construction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110013596.9A CN112685055A (en) | 2021-01-05 | 2021-01-05 | Cluster construction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112685055A true CN112685055A (en) | 2021-04-20 |
Family
ID=75455990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110013596.9A Pending CN112685055A (en) | 2021-01-05 | 2021-01-05 | Cluster construction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685055A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114138750A (en) * | 2021-12-03 | 2022-03-04 | 无锡星凝互动科技有限公司 | AI consultation database cluster building method and system |
| CN116560722A (en) * | 2023-07-12 | 2023-08-08 | 腾讯科技（深圳）有限公司 | Operation and maintenance flow processing method and device, electronic equipment and storage medium |
| CN116643846A (en) * | 2023-06-01 | 2023-08-25 | 北京首都在线科技股份有限公司 | Timed task processing method and device based on container cluster arrangement management platform |
-
2021
- 2021-01-05 CN CN202110013596.9A patent/CN112685055A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114138750A (en) * | 2021-12-03 | 2022-03-04 | 无锡星凝互动科技有限公司 | AI consultation database cluster building method and system |
| CN114138750B (en) * | 2021-12-03 | 2022-10-18 | 无锡星凝互动科技有限公司 | AI consultation database based cluster building method and system |
| CN116643846A (en) * | 2023-06-01 | 2023-08-25 | 北京首都在线科技股份有限公司 | Timed task processing method and device based on container cluster arrangement management platform |
| CN116643846B (en) * | 2023-06-01 | 2024-02-20 | 北京首都在线科技股份有限公司 | Timed task processing method and device based on container cluster arrangement management platform |
| CN116560722A (en) * | 2023-07-12 | 2023-08-08 | 腾讯科技（深圳）有限公司 | Operation and maintenance flow processing method and device, electronic equipment and storage medium |
| CN116560722B (en) * | 2023-07-12 | 2024-01-02 | 腾讯科技（深圳）有限公司 | Operation and maintenance flow processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112685055A (en) | Cluster construction method and device | |
| US11323273B2 (en) | System and method for generating a blockchain application for different blockchain technologies | |
| CN109104467B (en) | Development environment construction method and device, platform system and storage medium | |
| US11036857B2 (en) | Protecting a machine learning model | |
| WO2022078345A1 (en) | Method for data interaction between multiple devices, and related device | |
| US20180191581A1 (en) | Method and Apparatus for Deploying Network Service | |
| US11436819B2 (en) | Consolidation and history recording of a physical display board using an online task management system | |
| CN110830546A (en) | Available domain construction method, device and equipment based on container cloud platform | |
| CN110688662A (en) | Sensitive data desensitization and inverse desensitization method and electronic equipment | |
| US20170024396A1 (en) | Determining application deployment recommendations | |
| US20170228280A1 (en) | System and Method for Error Handling Based on a Boot Profile | |
| US8024444B2 (en) | Associating telemetry data from a group of entities | |
| CN104091140A (en) | Information processing method and electronic device | |
| CN111327607A (en) | Security threat information management method, system, storage medium and terminal based on big data | |
| CN108390786B (en) | Business operation and maintenance method and device and electronic equipment | |
| CN114327374A (en) | Business process generation method and device and computer equipment | |
| JP7369229B2 (en) | How to change the skin of the mini program page, devices and electronic devices | |
| US20190124044A1 (en) | Preventing Unauthorized Access to Secure Enterprise Information Systems Using a Multi-Filtering and Randomizing Control System | |
| US10917478B2 (en) | Cloud enabling resources as a service | |
| US9503351B1 (en) | Deployment feedback for system updates to resources in private networks | |
| EP3163453B1 (en) | Securing an application by randomizing its memory layout at launch time | |
| CN112083925A (en) | Data acquisition method, device, equipment and storage medium based on H5 page development | |
| JP6721551B2 (en) | Extraction device, extraction method, and extraction program | |
| CN103051607B (en) | Access method, equipment and system | |
| CN112686391A (en) | Modeling method and device based on federal learning, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |