WO2023231572A1 - 一种容器的创建方法、装置及存储介质 - Google Patents
一种容器的创建方法、装置及存储介质 Download PDFInfo
- Publication number
- WO2023231572A1 WO2023231572A1 PCT/CN2023/087179 CN2023087179W WO2023231572A1 WO 2023231572 A1 WO2023231572 A1 WO 2023231572A1 CN 2023087179 W CN2023087179 W CN 2023087179W WO 2023231572 A1 WO2023231572 A1 WO 2023231572A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- node
- virtual machine
- container
- memory
- storage space
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 248
- 230000005012 migration Effects 0.000 claims description 91
- 238000013508 migration Methods 0.000 claims description 91
- 238000012545 processing Methods 0.000 claims description 51
- 238000013507 mapping Methods 0.000 claims description 28
- 230000002085 persistent effect Effects 0.000 claims description 14
- 238000012546 transfer Methods 0.000 claims 2
- 238000004891 communication Methods 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 120
- 238000013461 design Methods 0.000 description 32
- 230000006870 function Effects 0.000 description 27
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 18
- 230000004044 response Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 6
- 238000007726 management method Methods 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 208000000044 Amnesia Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 231100000863 loss of memory Toxicity 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
Definitions
- the present application relates to the field of computer communication technology, and in particular, to a container creation method, device and storage medium.
- Kubernetes (K8S for short) is an open source container orchestration, scheduling and management system, which is regarded as the operating system in the cloud native era.
- a cluster implemented using K8S can also be called a K8S cluster.
- a K8S cluster includes several physical machines or virtual machines (VMs).
- VMs virtual machines
- a physical machine or a virtual machine can be regarded as a node.
- K8S uses pod as the smallest unit to create containers on nodes in the cluster.
- a pod can include one or more containers. Among them, containers are generally used to provide a running environment for application processes, or can be understood as being used to run application instances.
- pods have a short life cycle. Therefore, pods will be frequently created, scheduled, and destroyed. When a pod is scheduled to a node, it will generally always run on this node. Since pods do not have self-healing capabilities, when a node does not meet the pod's survival requirements (for example, the node's resources are exhausted), in this case, the node will delete the pod. Moreover, K8S will create a new pod on another node to replace the deleted pod.
- This application provides a container creation method, device and storage medium. This method can realize the migration of containers without affecting the continuity of the container business and without losing the container business data.
- this application provides a method for creating a container, which method is applied to the first node.
- the method includes: obtaining a first creation request for instructing creation of a target container; creating a first virtual machine based on the first creation request; and running the target container in the first virtual machine.
- the target container is a secure container
- the first creation request includes resource parameters of the first virtual machine running the target container
- the physical resource of the memory of the first virtual machine is the first physical storage space in the first storage node.
- the secure container created based on the method provided in this application needs to be migrated from the first node to another node, the first physical storage space only needs to be used as the memory physical resource of the virtual machine for running the migrated secure container.
- the secure container can be migrated without affecting the business continuity of the container or losing the container business data.
- the above-mentioned first node accesses the first physical storage space through remote direct memory access (RDMA).
- RDMA remote direct memory access
- the first virtual machine created in the first node can access the first physical storage space of the first storage node as memory, that is, the first physical storage space is used as the memory of the first virtual machine. Purpose.
- the above-mentioned first physical storage space is a persistent memory medium.
- the storage performance of the persistent memory medium is between traditional DRAM and flash memory (such as NAND flash memory). Therefore, the first physical storage space as the first virtual machine memory has higher access speed and lower access delay than flash memory, and has larger storage capacity than traditional DRAM, and data is not lost when power is turned off. The cost of persistent storage is relatively low.
- the above method before the first virtual machine is created based on the first creation request, the above method also includes: determining the memory size of the first virtual machine according to the first creation request; applying to the first storage node for the first physical storage space according to the memory size of the first virtual machine, and determining the first physical storage space as the first physical storage space.
- the physical resource of a virtual machine's memory wherein, the first physical storage space is greater than or equal to the memory size of the first virtual machine.
- the above method also includes: determining the second physical storage space of the data processing unit DPU in the first node as a cached physical resource of the first virtual machine; instructing the DPU to establish the second physical storage The mapping relationship between the space and the first physical storage space.
- the DPU is used to access the first physical storage space through RDMA, and to write data in the second physical storage space into the first physical storage space.
- the first virtual machine in the first node is writing data.
- the first virtual machine in the first node can determine that the data writing task of the process has been completed. Subsequently, the DPU, as an independent unit in the first node, will write the data written into the second physical storage space into the first physical storage space through RDMA operation.
- the first virtual machine in the first node does not need to determine that the data writing task of the process has been completed after receiving the response message of the RDMA operation returned by the first storage node.
- the second physical storage space of the DPU is used as a cache for the first node to write data into the first physical storage space.
- the related process resources for writing data in the first node can be quickly released, thereby saving the resources required for running the process for writing data on the first node.
- the above-mentioned DPU is also used to read the hot data of the target container from the first physical storage space.
- the DPU when the DPU pre-reads the data that the application process in the target container has accessed more frequently in the recent period (that is, hot data) from the first physical storage space to the second physical storage space, thereby , when the application process in the target container accesses the hot data again, the DPU can directly return the data to the application process, thereby improving the efficiency of the application process accessing data.
- the above-mentioned first node is any node in the container cluster.
- the container cluster also includes a second node.
- the target container is run on the second node.
- the physical resource of the memory of the second virtual machine is the first physical storage space.
- the above-mentioned first node is any node in the container cluster.
- the container cluster also includes a third node.
- the third node is used to create a backup container of the target container.
- the third node runs The third virtual machine of the backup container is in a suspended state, and the physical resource of the memory of the third virtual machine is the first physical storage space.
- the third virtual machine when the first virtual machine is in a fault or suspended state, the third virtual machine is in a running state.
- the purpose of migrating the target container from the first node to the third node is achieved, and since the first physical storage space serves as a memory physical resource for the third virtual machine running the migrated secure container, it is achieved This ensures the migration of the secure container without affecting the business continuity of the container or losing the container's business data.
- the above method further includes: after the target container is migrated to another node in the container cluster, updating the network routing information for accessing the target container.
- the above method also includes: after the target container is migrated to another node in the container cluster, sending a routing update message to other nodes in the container cluster except the first node, so that other nodes Node update access Network routing information for the target container.
- the purpose of subsequent normal access of the user device to the target container can be achieved, and the user is unaware of the target container migration process.
- the present application provides a device for creating a container.
- the creation device is used to perform any of the methods provided in the first aspect.
- This application can divide the creation device into functional modules according to any of the methods provided in the first aspect.
- each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
- this application can divide the creation device into an acquisition unit and a processing unit according to functions.
- the creation device includes: a memory, a network interface, and one or more processors.
- the one or more processors receive or send data through the network interface.
- the one or more processors are It is configured to read the program instructions stored in the memory, so that the creation device executes any method provided by the first aspect and any possible design manner thereof.
- this application provides a method for creating a container, which method is applied to the second node.
- the method includes: receiving a container migration request, the container migration request indicating migrating a target container running a first virtual machine in a first node to a second node, and instructing the second node to create a second virtual machine for running the target container.
- the target container is a secure container.
- the first physical storage space in the first storage node is determined as a physical resource of the memory of the second virtual machine, and the first physical storage space is a physical resource of the memory of the first virtual machine. Based on the physical resources of the memory of the second virtual machine, a second virtual machine is created, and the target container is run in the second virtual machine.
- the physical resources of the memory of the target container are the same before and after migration.
- the memory data of the target container running in the second virtual machine of the second node after migration is the memory data of the target container of the first virtual machine running on the first node before migration. Therefore, when the memory data of the target container before and after migration is the same, it indicates that the business status of the target container before and after migration is the same. That is, the method of the present application can achieve migration of containers without affecting the continuity of container services and without losing the container's business data.
- the container migration request includes the identification number ID of the target container, and the first physical storage space in the first storage node is determined as the physical resource of the memory of the second virtual machine. It includes: sending a memory request message including the ID of the target container to the first storage node, where the memory request message is used to request to obtain the address of the first physical storage space that has a corresponding relationship with the ID of the target container. The address of the first physical storage space returned by the first storage node is received, and the first physical storage space indicated by the address is determined as a physical resource of the memory of the second virtual machine.
- This possible design is for the second node to obtain the address of the first physical storage space (that is, the physical resource of the memory of the first virtual machine), thereby using the first physical storage space as a physical resource of the memory of the second virtual machine. Way.
- the container migration request includes the identification number ID of the first virtual machine, and the first physical storage space in the first storage node is determined as the physical storage space of the memory of the second virtual machine.
- the resource includes: sending a memory request message including the ID of the first virtual machine to the first storage node, where the memory request message is used to request to obtain an address of the first physical storage space that has a corresponding relationship with the ID of the first virtual machine.
- the address of the first physical storage space returned by the first storage node is received, and the first physical storage space indicated by the address is determined as a physical resource of the memory of the second virtual machine.
- This possible design is for the second node to obtain the address of the first physical storage space (that is, the physical resource of the memory of the first virtual machine), thereby using the first physical storage space as another physical resource of the memory of the second virtual machine.
- the present application provides a container creation device.
- the creation device is used to perform any of the methods provided in the above third aspect.
- This application Please divide the creation device into functional modules according to any of the methods provided in the third aspect above.
- each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
- this application can divide the creation device into a receiving unit and a processing unit according to functions.
- the creation device includes: a memory, a network interface, and one or more processors.
- the one or more processors receive or send data through the network interface.
- the one or more processors are It is configured to read the program instructions stored in the memory, so that the creation device executes any method provided by the third aspect and any possible design manner thereof.
- this application provides a container cluster, which includes a first node and a second node.
- the first node is used to execute any method provided by the first aspect and any possible design method thereof to create the target container.
- the second node is configured to execute any method as provided in the third aspect and any possible design manner thereof, so as to migrate the target container running on the first node to the second node.
- the present application provides a computer-readable storage medium, which includes program instructions.
- the program instructions When the program instructions are run on a computer or processor, the computer or processor causes the computer or processor to execute the first aspect or the second aspect. Any method provided by any possible implementation in the two aspects.
- this application provides a computer program product that, when run on a container creation device, causes any method provided by any possible implementation in the first aspect to be executed.
- this application provides a chip system, including: a processor, configured to call and run a computer program stored in the memory from a memory, and execute any method provided by the implementation in the first aspect.
- any of the devices, computer storage media, computer program products or chip systems provided above can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the corresponding methods. The beneficial effects of the method will not be repeated here.
- the name of the container creation device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and its equivalent technologies.
- Figure 1 is a schematic architectural diagram of a container cluster provided by an embodiment of the present application.
- Figure 2 is a schematic diagram of the hardware structure of a first node provided by an embodiment of the present application.
- Figure 3 is a schematic flowchart of a container creation method provided by an embodiment of the present application.
- Figure 4 is a schematic diagram of the data reading and writing process of a target container when processing services provided by the embodiment of the present application;
- Figure 5 is a schematic flowchart of another container creation method provided by an embodiment of the present application.
- Figure 6 is a schematic diagram of a shared memory provided by an embodiment of the present application.
- Figure 7 is a schematic flowchart of yet another container creation method provided by an embodiment of the present application.
- Figure 8 is a schematic diagram of the data reading and writing process of another target container when processing services provided by the embodiment of the present application.
- Figure 9 is a schematic flowchart of yet another container creation method provided by an embodiment of the present application.
- Figure 10 is a schematic flow chart of yet another container creation method provided by an embodiment of the present application.
- Figure 11 is a schematic structural diagram of a container creation device provided by an embodiment of the present application.
- Figure 12 is a schematic structural diagram of another container creation device provided by an embodiment of the present application.
- Figure 13 is a schematic diagram of a chip system provided by an embodiment of the present application.
- Containers can be used to provide a running environment for application processes, or can be understood as being used to run application instances.
- containers can be divided into ordinary containers and safe containers.
- ordinary containers can run directly in nodes, while secure containers run in virtual machines created on nodes in the cluster.
- lightweight virtual machines are configured with independent operating system kernels and are securely isolated at the virtualization layer. That is to say, containers running on lightweight virtual machines will not share the operating system kernel of the host node (that is, the node (including physical nodes and virtual nodes) used to run the lightweight virtual machine). Therefore, secure containers are more secure than ordinary containers.
- lightweight virtual machines are only used to run containers, lightweight virtual machines only need to be configured with functional modules that support secure container operation. That is, lightweight virtual machines start and run faster than ordinary virtual machines.
- first and second do not indicate a sequential relationship, but are used to distinguish different objects.
- the first, second, etc. mentioned in the following documents are also used to distinguish different reports. documents, etc., and cannot be understood as indicating or implying the relative importance or implicitly indicating the quantity of the technical features indicated.
- the size of the sequence number of each process does not mean the order of execution.
- the execution order of each process should be determined by its function and internal logic, and should not be determined by the execution order of the embodiments of the present application.
- the implementation process constitutes no limitation.
- the migration of the secure container can be achieved by migrating the virtual machine used to run the secure container.
- the primary virtual machine normally processes the business submitted by the user, and passes the business submitted by the user to the standby virtual machine, so that the standby virtual machine can process the business submitted by the user, so that the standby virtual machine can receive the same processing as the primary virtual machine. result.
- the primary virtual machine determines that the standby virtual machine has completed processing the user business, it outputs the processing result obtained after processing the user business to the user. All external output of the standby virtual machine is blocked, that is, the standby virtual machine does not output its own processing results to the outside.
- the node running the primary virtual machine only needs to notify the node running the standby virtual machine to unblock the output of the standby virtual machine, so that instant switching between the active and standby virtual machines can be achieved without interrupting the business.
- the purpose of migrating the virtual machine is achieved, and then the purpose of migrating the secure container running in the virtual machine is achieved.
- embodiments of the present application provide a method for creating a container, in which the physical resource of the memory of the first virtual machine created when creating a secure container is set as the first physical storage space in the first storage node.
- the physical resource of the memory of the first virtual machine created when creating a secure container is set as the first physical storage space in the first storage node.
- Embodiments of the present application also provide a container cluster, which may include a control node and multiple nodes (or worker nodes) for deploying containers.
- the control node is used to manage the containers running on the multiple working nodes, for example, to instruct the working nodes to create containers, or to monitor the running status of the containers on the working nodes.
- the number of control nodes can be multiple.
- the above-mentioned multiple working nodes may be physical nodes or virtual nodes, and there is no limitation on this.
- the above-mentioned control node can be any node device with computing processing capabilities.
- the node device can be a physical node or a virtual node, which is not limited.
- the physical node may be, for example, a general-purpose computer, a laptop or other computing device, and the virtual node may be, for example, a virtual machine, etc., but is not limited thereto.
- control node can be an independent node device.
- function of the control node can also be implemented by any one of the above-mentioned multiple working nodes, that is, the control node is a functional module in any one of the working nodes. There is no limit to this. For simple description, the embodiment of the present application will be described below assuming that the control node is an independent node device.
- the container cluster can also include one or more storage nodes.
- One or more storage nodes in the container cluster can be used to provide shared memory for working nodes in the container cluster where secure containers are deployed.
- the shared memory is, for example, a memory unit composed of multiple persistent memory media (such as XL-Flash, etc.). Because the storage performance of persistent memory media is between traditional dynamic random access memory (DRAM) and flash memory (such as NAND flash memory). Therefore, this shared memory has higher access speed and lower access delay than flash memory, and has larger storage capacity than traditional DRAM, and the data is not lost when power is turned off. It can be stored persistently and the cost is relatively low. .
- DRAM dynamic random access memory
- flash memory such as NAND flash memory
- one or more storage nodes in the container cluster can also be used to provide storage space for image files of containers running on the working nodes in the container cluster.
- the storage space is, for example, a storage unit composed of multiple non-volatile storage media, such as a mechanical hard disk, a solid-state hard disk, etc.
- the storage performance of this storage space is weaker than that of persistent memory media, but the capacity is larger and the cost is lower.
- the image file of the container (or container image) stores the program code of the container, as well as the data that needs to be stored persistently for the container, etc., which will not be described in detail.
- the storage node that provides shared memory for the nodes where secure containers are deployed in the container cluster and the storage node that provides storage space for the image files of the containers run by the nodes in the container cluster, can be the same storage nodes in the container cluster. , of course, it can also be different storage nodes in the container cluster, and there is no limit to this.
- the container cluster in the embodiment of this application may not include storage nodes.
- data storage services can be provided for the container cluster through storage nodes in an independent storage system.
- the storage nodes in the storage system can be used to provide shared memory storage services for nodes where secure containers are deployed in the container cluster, and/or to provide storage space for image files of containers running on nodes in the container cluster. , there is no limit to this.
- the embodiment of the present application will be described below by taking the container cluster provided by the embodiment of the present application as including one or more storage nodes as an example.
- FIG. 1 shows a schematic architectural diagram of a container cluster 100 provided by an embodiment of the present application.
- the container cluster 100 includes a control node 101, a control node 102, and n working nodes for deploying containers.
- the n working nodes are respectively a working node 111, a working node 112, a working node 113,..., and Worker node 11n.
- n is an integer greater than 3.
- the control node 102 is a backup control node of the control node 101.
- the container cluster 100 also includes two storage nodes, namely storage node 121 and storage node 122.
- the storage node 121 and the storage node 122 may have communication connections with the above-mentioned n working nodes respectively.
- the n worker nodes can
- the storage node 121 and the storage node 122 are accessed through remote direct memory access (RDMA).
- RDMA remote direct memory access
- the storage node that provides shared memory for the working nodes where secure containers are deployed in the container cluster 100 and the storage node that provides storage space for the image files of the containers running in the above n working nodes can both be the storage node 121, or Both are storage nodes 122, or respectively storage nodes 121 and storage nodes 122.
- control node 101 there may also be communication connections between the control node 101, the control node 102 and the storage node 121 (and/or the storage node 122), so that the control node 101, the control node 102 can access the storage node 121 (and/or storage node 122).
- the working nodes 111, 112, 113, ..., and the working nodes 11n may communicate with each other.
- An embodiment of the present application also provides a container creation device, which is applied to the above-mentioned container cluster.
- the device may be any working node for deploying containers in the above container cluster (hereinafter referred to as the first node), or the device may be a functional module in the first node, which is not limited.
- the first node may be the working node 111, the working node 112, the working node 113, etc. in the container cluster 100 shown in Figure 1, which is not limited.
- the first node 20 includes a processor 201 , a memory 202 , a network interface 203 and a bus 204 .
- the processor 201, the memory 202, and the network interface 203 are connected through a bus 204.
- the processor 201 is the control center of the first node 20, and may be a general central processing unit (CPU).
- the processor 201 may also be other general processors, digital signal processing (DSP), Application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, graphics processors (graphics processors) processing unit (GPU), neural network processing unit (NPU), tensor processing unit (TPU) or artificial intelligence (artificial intelligent) chip, data processing unit (DPU), etc.
- DSP digital signal processing
- ASIC Application-specific integrated circuit
- FPGA field-programmable gate array
- processor 201 includes one or more CPUs, such as CPU 0 and CPU 1 shown in FIG. 2 .
- this application does not limit the number of processor cores in each processor.
- the memory 202 is used to store program instructions or data to be accessed by the application process.
- the processor 201 can execute the program instructions in the memory 202 to implement the container creation method provided by the embodiment of the present application.
- Memory 202 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
- volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable memory Except electrically programmable read-only memory (EPROM, EEPROM) or flash memory.
- Volatile memory can be random access memory (RAM), which is used as an external cache.
- RAM static random access memory
- DRAM dynamic random access memory
- SDRAM synchronous dynamic random access memory
- DDR SDRAM double data rate synchronous dynamic random access memory
- Access memory double data date SDRAM, DDR SDRAM
- enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
- synchronous link dynamic random access memory synchronization RAM, SLDRAM
- direct memory bus random access memory direct rambus RAM, DR RAM
- Non-volatile memory can be storage class memory (SCM), solid state drive (SSD), mechanical hard disk (hard disk drive, HDD), etc.
- SCM storage class memory
- SSD solid state drive
- HDD hard disk drive
- storage-level memory can be, for example, non-volatile Non-volatile memory (NVM), phase-change memory (PCM), persistent memory, etc.
- NVM non-volatile Non-volatile memory
- PCM phase-change memory
- persistent memory etc.
- the memory 202 exists independently of the processor 201 .
- the memory 202 is connected to the processor 201 through the bus 204 and is used to store data, instructions or program codes.
- the processor 201 calls and executes the instructions or program codes stored in the memory 202, it can implement the container creation method provided by the embodiment of the present application.
- the memory 202 and the processor 201 are integrated together.
- Network interface 203 is used to connect the first node 20 with other devices (storage nodes in the container cluster 100 as shown in Figure 1, etc.) through a communication network.
- the communication network can be Ethernet, wireless access network (radio access network) network, RAN), wireless local area networks (WLAN), etc.
- the network interface 203 includes a receiving unit for receiving data/messages, and a sending unit for sending data/messages.
- the network interface 203 may be a network interface card (NIC) that supports RDMA, and the NIC that supports RDMA may be abbreviated as RNIC.
- NIC network interface card
- RDMA is a transport layer network protocol used to solve data processing delays in network transmission.
- RDMA can reduce the copy operation of data within the host device used to transmit data during the data transmission process, and does not require the intervention of the operating system of the host device used to transmit data. Therefore, RDMA can reduce the CPU load of the host device used to transmit data. .
- the bus 204 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, a high-speed serial computer expansion bus (peripheral component interconnect express, PCIe), or a computing express link (compute express link, CXL) or extended industry standard architecture (extended industry standard architecture, EISA) bus, etc.
- ISA industry standard architecture
- PCIe peripheral component interconnect express
- CXL compute express link
- EISA extended industry standard architecture
- the bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 2, but it does not mean that there is only one bus or one type of bus.
- the above-mentioned first node 20 can also be configured with an independent DPU 205.
- the DPU 205 communicates with the processor 201, the memory 202, and the network interface 203 through the bus 204.
- DPU 205 includes storage module and RNIC driver module.
- the hardware implementation of DPU 205 can be ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. that can implement storage functions and RNIC functions.
- the embodiments of this application do not specifically limit this. . It can be understood that in practice, the DPU 205 can be presented as a circuit board/card with storage functions and RNIC functions.
- the storage module of DPU 205 may include DRAM.
- the physical storage space in the storage module can be used as a cache for the virtual machine running on the first node 20 and supports memory access semantics and a direct access (Direct Access, DAX) access mechanism.
- DAX direct access
- the storage module of the DPU 205 can be presented as a DAX-enabled persistent memory device on the first node 20 .
- the DAX access mechanism is a mechanism that supports user-mode software to directly access data stored in persistent memory. In this way, when the first node 20 accesses the data in the storage module of the DPU 205, it is like accessing its own memory without first performing a page cache (page cache) process of copying the data to the operating system.
- the RNIC driver module of DPU 205 is used to communicate with storage nodes (such as storage node 121 or storage node 122 in Figure 1), and access the physical storage space used as shared memory in the storage node through RDMA.
- the above-mentioned network interface 203 may or may not support RDMA, which is not limited in the embodiment of the present application.
- first node 20 does not constitute a limitation on the first node 20.
- the first node 20 may also include more or less components than those shown in Figure 2 components, or combinations of certain components, or different component arrangements.
- FIG. 3 shows a schematic flowchart of a container creation method provided by an embodiment of the present application.
- this method is applied to the container cluster 100 shown in Figure 1, and can be executed by the first node having the hardware structure shown in Figure 2.
- the method may include the following steps:
- the first node obtains a first creation request for instructing the creation of a target container.
- the target container is a secure container.
- the description of the security container can be found above and will not be repeated here.
- the first creation request may carry first indication information, and the first indication information is used to instruct the first node to create a target container that supports the dynamic migration feature.
- the target container that supports the dynamic migration feature is the container created by the first node based on the method described in the embodiment of this application.
- the feature of supporting dynamic migration refers to the feature of completing the migration of normally running containers without interrupting business.
- the first creation request is also used to instruct the first node to create a lightweight virtual machine for running the target container (hereinafter, the lightweight virtual machine is the first virtual machine as an example), and instructs the first node to Run the target container in the first created virtual machine.
- the lightweight virtual machine is the first virtual machine as an example
- the first creation request includes the identity document (ID) of the target container (or the ID of the pod to which the target container belongs), the image ID of the target container, and resource parameters including the resources required to run the target container.
- the resource parameters include the CPU resource size and memory resource size required to run the target container.
- the image ID of the target container is used to identify the image file of the target container, and the first node can access the image file of the target container based on the image ID of the target container.
- the image file of the target container is pre-stored on the second storage node.
- the second storage node may be the same storage node as the first storage node mentioned above, or may be a different storage node, which is not limited.
- the second storage node has pre-stored image files of one or more containers.
- the image files of the containers store the program code of the container and the data that needs to be persistently stored for the container. This will not be described in detail.
- the program code of the container pre-stored in the second storage node can be the program code of the container pre-designed and developed by the designer, and the container image files including different program codes can be identified by different IDs, which is not limited.
- the first creation request may also include the ID of the first virtual machine.
- the ID of the first virtual machine may also be the ID of the first virtual machine generated by the first node based on the target container ID in the first creation request or the ID of the pod to which the target container belongs.
- the target container ID (or the ID of the pod to which the target container belongs) has a corresponding relationship with the ID of the first virtual machine.
- the first node may receive a first creation request sent by a control node of the container cluster (for example, the control node 10 shown in Figure 1).
- the container cluster 100 shown in Figure 1 as a K8S cluster with a K8S system installed as an example.
- the control node 10 and n working nodes in the container cluster 100 are all installed with the K8S container management module.
- the K8S container management module of the control node 10 may generate the first creation request based on the user input instructions.
- the K8S container management module in the control node 10 can receive user input instructions through the input interface (such as keyboard, mouse, touch screen, etc.) of the control node 10, and then generate a first creation request, wherein the user input instructions For example, it includes the ID of the target container, the image ID of the target container, etc.
- the ID of the container may be, for example, the name, number, etc. of the container, which is not limited.
- the image ID can be, for example, the name, number, etc. of the container image file, and is not limited to this.
- the control node 10 determines that the node used to deploy the target container is the first node in the container cluster 100 based on the predetermined selection policy or the received user instruction, and sends the generated first creation request to the first node.
- the first node may receive the first creation request.
- the embodiments of the present application do not limit the predetermined selection strategy.
- the predetermined selection strategy may be, for example, a load balancing strategy.
- the first node may generate the first creation request based on instructions input by the user.
- the first node may receive user input instructions through an input interface (such as keyboard, mouse, touch screen, etc.), and then generate a first creation request.
- an input interface such as keyboard, mouse, touch screen, etc.
- the first node determines the physical resources of the memory of the first virtual machine according to the first creation request. Among them, the first virtual machine is used to run the target container.
- the process of the first node determining the physical resources of the memory of the first virtual machine according to the first creation request can be implemented through steps S1021-S1022:
- the first node determines the memory size of the first virtual machine according to the first creation request.
- the target container is a secure container
- the virtual machine running the secure container is generally a lightweight virtual machine only used to run the secure container. It can be understood that if the first creation request only indicates the creation of one target container, the resources required to run the target container can be regarded as the resources required to create the first virtual machine. If the first creation request indicates the creation of multiple containers including the target container, the resources required to run the multiple containers may be regarded as the resources required to create the first virtual machine. In this case, the first creation request includes resource parameters of resources required to run the multiple containers, and the resources required to run the multiple containers include resources required to run the target container.
- the first node may determine the resource parameters of the resources required to create the first virtual machine based on the resource parameters of the resources required to run the target container included in the first creation request. For example, the first node determines the CPU resource size and memory size required to create the first virtual machine based on the resource parameters of the resources required to run the target container included in the first creation request.
- the memory of the first virtual machine can be understood as the physical memory of the first virtual machine, and the physical memory of the first virtual machine is actually a section of virtual memory of the virtual machine process.
- the first node applies to the first storage node for physical resources of the memory of the first virtual machine according to the memory size of the first virtual machine.
- the first node may apply for a section of physical storage space from the first storage node according to the memory size of the first virtual machine as a physical resource of the memory of the first virtual machine.
- the first node may generate a first memory request message according to the memory size of the first virtual machine, and send the first memory request message to the first storage node.
- the first node may send a first memory request message to the first storage node through its own network interface (such as the network interface 203 shown in FIG. 2).
- the first memory request message is used to instruct the first storage node to allocate a section of physical storage space to the first virtual machine.
- the first memory request message carries the memory size of the first virtual machine.
- the first memory request message may also carry the ID of the target container (or the ID of the pod to which the target container belongs) and/or the ID of the first virtual machine.
- the description of the ID of the first virtual machine can be referred to the above and will not be described again.
- the first storage node may be a storage node in the container cluster where the first node is located (for example, the storage node 121 shown in Figure 1), or it may be a storage node in an independent storage system that the container cluster can access, which is not limited.
- the first storage node allocates a segment of physical storage space to the first virtual machine and returns the address (physical address or logical address) of the "segment of physical storage space" to the first node.
- the "section of physical storage space" allocated by the first storage node to the first virtual machine is called the first physical storage space.
- the first physical storage space may be a memory unit composed of multiple persistent memory media (such as XL-Flash, etc.). Since the storage performance of persistent memory media is between traditional DRAM and flash memory, the first physical storage space has higher access speed and lower access latency than flash memory. Compared with traditional DRAM, it has a larger storage capacity, data is not lost when power is turned off, it can be stored persistently, and the cost is relatively low.
- the size of the first physical storage space allocated by the first storage node to the first virtual machine may be equal to the memory size of the first virtual machine, or may be greater than the memory size of the first virtual machine, and is not limited to this.
- the first storage node may randomly allocate a segment of physical storage space that is currently free in its own memory and is greater than or equal to the memory size of the first virtual machine as the first physical storage space to the first virtual machine.
- the first storage node may allocate a segment of physical storage space that is currently free in its own memory and is greater than or equal to the memory size of the first virtual machine as the first physical storage space to the first virtual machine based on a preset rule.
- the embodiment of the present application does not limit the preset rules.
- the first storage node in order to facilitate the management of local storage space, after the first storage node allocates the first physical storage space to the first virtual machine, and when the first memory request message carries the ID of the first virtual machine, the first storage node establishes Correspondence between the address of the first physical storage space and the first virtual machine ID.
- the first memory request message carries the ID of the target container (or the ID of the pod to which the target container belongs), and establishes a relationship between the address of the first physical storage space and the ID of the target container (or the ID of the pod to which the target container belongs). Correspondence.
- the first memory request message carries the ID of the target container (or the ID of the pod to which the target container belongs) and the ID of the first virtual machine, and establishes the address of the first physical storage space, the ID of the target container (or the ID of the pod to which the target container belongs). pod ID) and the first virtual machine ID.
- the first node In response to the address of the first physical storage space returned by the first storage node to the first node, the first node obtains the address of the first physical storage space, thus completing the application for the memory of the first virtual machine from the first storage node.
- the process of physical resources In response to the address of the first physical storage space returned by the first storage node to the first node, the first node obtains the address of the first physical storage space, thus completing the application for the memory of the first virtual machine from the first storage node. The process of physical resources.
- the first node can access the first physical storage space of the first storage node through RDMA operation.
- the first node may register a new DAX storage device according to the address of the first physical storage space returned by the first storage node.
- the DAX storage device is a virtual storage device.
- the DAX storage device has no actual physical storage space on the first node.
- the DAX storage device is used to indicate the address correspondence of the first physical storage space returned by the first storage node. of storage space. Among them, the relevant description of DAX can be referred to above and will not be described in detail here.
- the first physical storage space mentioned above can also be used to store CPU status information of the first virtual machine as part of the storage space in the first physical storage space.
- the size of the first physical storage space is at least equal to: the sum of the memory size of the first virtual machine and the size of the CPU status information of the first virtual machine.
- the CPU status information includes register information (such as the identifier and current value of the register), interrupt information, etc., which is not limited.
- the first physical storage space includes a first space and a second space. The first space is used to store memory data of the first virtual machine, and the second space is used to store CPU status information of the first virtual machine.
- the first node creates the first virtual machine based on the first creation request and the determined physical resources.
- the first node may first create a virtual machine process for creating the first virtual machine based on the first creation request.
- the virtual machine process is used to create (or manage/monitor) the first virtual machine, that is, the first virtual machine is equivalent to one or more threads in the virtual machine process.
- the K8S container management module in the first node is sent to a virtual machine monitor (virtual machine monitor, VMM) of the first node, and the VMM instructs the first node to create a virtual machine process for creating the first virtual machine.
- VMM virtual machine monitor
- the first node can create the first virtual machine through the created virtual machine process.
- the first node can first address a section of virtual memory through the virtual machine process to serve as the physical memory of the first virtual machine (hereinafter referred to as the first physical memory). Then, the first node establishes a mapping relationship between the first physical memory and the first physical storage space through the virtual machine process, so as to use the first physical storage space as a physical resource of the memory of the first virtual machine.
- the first node may establish a mapping relationship between the first physical memory and the first virtual memory of the virtual machine process through the virtual machine process.
- the first virtual memory is a section of memory in the virtual memory of the virtual machine process, and the address of the first virtual memory corresponds to the address of the first physical memory one-to-one.
- the first node also establishes a mapping relationship between the first virtual memory and the first physical storage space returned by the first storage node through the virtual machine process, thereby realizing using the first physical storage space as a physical resource of the memory of the first virtual machine. Purpose.
- the size of the first virtual memory (or the first physical memory) is less than or equal to the size of the first physical storage space, and the first physical storage space includes the same address as the first virtual memory (or the first physical memory). corresponding address.
- the address of a section or all of the storage space in the first physical storage space corresponds to the address of the first virtual memory (or the first physical memory) in a one-to-one manner.
- the first node when the first node establishes the mapping relationship between the first virtual memory and the first physical storage space through the virtual machine process, the first node can open the DAX storage registered on the first node based on the address of the first physical storage space. device, and calls the mmap interface through the virtual machine process to establish a mapping relationship between the first virtual memory and the first physical storage space.
- the first node also applies for the CPU resources of the first node for the first virtual machine through the virtual machine process based on the determined CPU resource size required by the first virtual machine, which will not be described in detail here.
- the first node can create the first virtual machine through the virtual machine process.
- the first node can read the operating system code of the virtual machine in a locally pre-stored virtual machine image file, and run the operating system code in a determined resource through the virtual machine process to complete the creation of the first virtual machine.
- the operating system code in the virtual machine image file is the operating system code of the virtual machine.
- the working nodes in the container cluster have virtual machine image files pre-stored.
- the virtual machine image file includes the operating system code of the virtual machine, and the virtual machine image file can be used to store data that needs to be stored persistently during the running of the virtual machine. It can be understood that the virtual machine image file can be regarded as the hard disk of the virtual machine.
- the first node can write the CPU status information of the first virtual machine into the second space in the first physical storage space through the virtual machine process.
- the second space and CPU status information please refer to the relevant description in S1022, which will not be described again here.
- the first node may first write the CPU status information of the first virtual machine into the first virtual memory through the virtual machine process. Since the actual physical storage space corresponding to the first virtual memory is the second physical memory of the first node (that is, a section of local physical memory of the first node), the first node actually writes the CPU status information to the first node through the virtual machine process. Second physical memory, and further writes the CPU status information into the memory of the RNIC of the first node. In this way, the RNIC can write the CPU status information into the second space in the first physical space of the first storage node through an RDMA operation (such as a write operation/send operation), thereby realizing the first storage node's first physical space. Recording of virtual machine CPU status information.
- RDMA operation such as a write operation/send operation
- the first node when the first node monitors the CPU status of the first virtual machine to change through the virtual machine process, it will use the process described above to obtain the current CPU status information of the first virtual machine.
- the first node creates the target container in the first virtual machine.
- the first node may first access the image file identified by the image ID in the second storage node based on the image ID in the first creation request, and read the program code in the image file. Furthermore, the first node runs the program in the first virtual machine program code to complete the creation of the target container.
- the first node sets the state of the first virtual machine to the running state through the virtual machine process.
- the first node accesses the image file identified by the image ID in the second storage node based on the image ID, it also sends the ID of the target container to the second storage node.
- the second storage node can establish a corresponding relationship between the target container ID and the above-mentioned image ID. In this way, when the target container is migrated to another node, the other node can use the ID of the target container to find the image file corresponding to the target container ID in the second storage node, and read the program in the image file. Code to create the target container to achieve migration of the target container.
- the target container can run normally on the first virtual machine in the first node.
- the first node creates and runs a virtual machine process
- the virtual machine process creates a first virtual machine. That is, the first virtual machine runs in the virtual machine process as one or more threads of the virtual machine process.
- the target container runs in the first virtual machine, and application process 1, application process 2 and application process 3 run in the target container.
- the first situation the user device accesses the application process 1 in the target container and writes data 1 to the first virtual machine through the application process 1.
- the first node first writes data 1 into the first physical memory (that is, the physical memory of the first virtual machine) through application process 1. Since the first physical memory corresponds to the first virtual memory of the virtual machine process, that is, the first node first writes data 1 into the first virtual memory corresponding to the first physical memory.
- the first node When the first node writes data to the first virtual memory, the first node will allocate a section of actual physical memory to the first virtual memory (i.e., a section of local physical memory of the first node, as shown in Figure 4 for the second physical memory ). Therefore, the first node writes data 1 into the first virtual memory through application process 1, which actually writes data 1 into the second physical memory.
- the first node can further write the data 1 in the second physical memory into the memory of the RNIC of the first node.
- RNIC can write data 1 into the first space in the first physical storage space of the first storage node through RDMA operation (such as write operation/send operation), thereby realizing the data in the first physical storage space and the first Data synchronization in physical memory.
- RDMA operation such as write operation/send operation
- the RNIC of the first node can write the data 1 written into the RNIC memory into the first space in real time through an RDMA operation.
- the RNIC can also periodically write the data written into the RNIC memory into the first space through RDMA operations.
- the RNIC can also write the data into the first space through an RDMA operation, which is not limited in the embodiments of the present application.
- the second case the user device accesses the application process 1 in the target container and reads the data 2 of the first virtual machine through the application process 1.
- the first virtual machine in the first node queries the second physical memory corresponding to the first physical memory (ie, a section of local physical memory of the first node) through application process 1.
- the operating system of the first node triggers a page fault exception event.
- the first node instructs the RNIC to generate an RDMA read request for reading data 2 to read data 2 from the first space in the first physical storage space of the first storage node, and further returns data 2 to the application process.
- control node in the container cluster where the first node is located (such as the control node 10 in the container cluster 100 shown in Figure 1) monitors that the above target container needs to be migrated, the control node removes the container cluster from the container cluster.
- the first node Send a container migration request to another node to instruct the target container to be migrated to the other node.
- Figure 5 shows a schematic flowchart of another container creation method provided by an embodiment of the present application.
- this method can be applied to the container cluster shown in Figure 1. This method includes the following steps:
- the control node determines the second node in the container cluster.
- the control node determines that the above-mentioned target container created by the first node through the method shown in Figure 3 needs to be migrated, it determines the second node in the container cluster for migrating the target container.
- the second node is any working node in the container cluster except the first node.
- the first node is the working node 111 in the container cluster 100 shown in Figure 1
- the second node can be any one of the working node 112, the working node 113 or the working node 11n in the container cluster 100, and no further changes will be made to this. limited.
- control node monitors that the resources required by the target container in the first node are about to be exhausted, or monitors that the resources of the first virtual machine running the target container in the first node are about to be exhausted, or monitors that the resources of the first virtual machine running the target container in the first node are about to be exhausted, or it monitors that the The resources of a node are about to be exhausted, or the first node (or the first virtual machine, or the target container) is monitored to be faulty, or the first node (or the first virtual machine, or the target container) is monitored to be suspended for other reasons. , or the control node receives the user's instruction to migrate the target container, etc., the control node determines the second node in the container cluster.
- control node can randomly determine the second node in the container cluster.
- control node may determine the second node in the container cluster based on a preset policy.
- the preset policy is, for example, a load balancing policy, which is not limited.
- the control node determines the second node in the container cluster based on the ID of the second node carried in the obtained user indication.
- the control node sends a container migration request to the second node to instruct the second node to create a second virtual machine for running the target container.
- the target container running on the first virtual machine in the first node is the container to be migrated.
- the container migration request is used to instruct the second node to create a lightweight virtual machine for running the target container (hereinafter, the lightweight virtual machine is the second virtual machine as an example), and instructs the second node to create a lightweight virtual machine. Run the target container in the completed second virtual machine.
- the container migration request can include the ID of the target container (or the ID of the pod to which the target container belongs) and the resource parameters of the resources required to run the target container.
- the resource parameters of the resources required to run the target container include the CPU resource size and memory resource size required to run the target container.
- the container migration request may also include the ID of the second virtual machine and the ID of the first virtual machine.
- the ID of the second virtual machine may also be the ID of the second virtual machine generated by the second node based on the target container ID in the container migration request or the ID of the pod to which the target container belongs. This is not a limitation.
- the ID of the second virtual machine and the ID of the first virtual machine may be the same or different, and the comparison is not limited. It can be understood that when the ID of the second virtual machine is the same as the ID of the second virtual machine, the container migration request may only include the ID of one virtual machine.
- the second node obtains the address of the first physical storage space according to the received container migration request.
- the second node can use the ID of the first virtual machine and/or the ID of the target container (or the ID of the pod to which the target container belongs) in the container migration request. , obtain the address of the above-mentioned first physical storage space (that is, the physical resource of the memory of the first virtual machine), so as to determine the first physical storage space as the physical resource of the memory of the second virtual machine. In this way, the memory data of the first virtual machine can be used as the memory data of the second virtual machine.
- the first physical storage space that is, the physical resource of the memory of the first virtual machine
- the second node may generate a memory request message (for example, a second memory request message) according to the container migration request, and send the second memory request message to the first storage node.
- the second memory request message is used to request to obtain the address of the first physical storage space allocated by the first storage node to the first virtual machine.
- the second node may send a second memory request message to the first storage node through its own network interface.
- the first memory request message sent to the first storage node includes the ID of the first virtual machine, that is, indicating that The first storage node records the corresponding relationship between the address of the first physical storage space and the ID of the first virtual machine.
- the second memory request message includes the ID of the first virtual machine. Furthermore, in response to receiving the second memory request message, the first storage node searches and determines the first physical storage space that has a corresponding relationship with the ID of the first virtual machine in the recorded corresponding information according to the ID of the first virtual machine. address and returns the address to the second node. In this way, the second node obtains the address of the first physical storage space.
- the first memory request message sent to the first storage node includes the ID of the target container (or the ID of the target container).
- the ID of the pod to which the pod belongs which indicates the correspondence between the address of the first physical storage space recorded in the first storage node and the ID of the target container (or the ID of the pod to which the target container belongs).
- the second memory request message includes the ID of the target container (or the ID of the pod to which the target container belongs). Furthermore, in response to receiving the second memory request message, the first storage node searches and determines the ID of the target container (or the ID of the pod to which the target container belongs) in the recorded corresponding information. ID of the pod to which it belongs) has a corresponding address of the first physical storage space, and returns the address to the second node. In this way, the second node obtains the address of the first physical storage space.
- the first memory request message sent to the first storage node includes the ID of the target container (or the ID of the target container).
- the ID of the pod to which the target container belongs) and the ID of the first virtual machine that is, the address of the first physical storage space, the ID of the target container (or the ID of the pod to which the target container belongs), and the ID of the first virtual machine are recorded in the first storage node. Correspondence between IDs.
- the second memory request message includes at least one of the ID of the target container (or the ID of the pod to which the target container belongs) or the ID of the first virtual machine.
- the first storage node searches and determines the address of the first physical storage space that has a corresponding relationship with the at least one ID in the recorded corresponding information according to the at least one ID, and determines the address of the first physical storage space. This address returns the second node. In this way, the second node obtains the address of the first physical storage space.
- the second node may register a new DAX storage device based on the obtained address of the first physical storage space.
- a new DAX storage device based on the obtained address of the first physical storage space.
- the second node creates a second virtual machine according to the address of the first physical storage space.
- the second node can determine the resource parameters of the resources required to create the second virtual machine based on the resource parameters of the target container running in the container migration request, for example, determine the CPU resource size and memory size required to create the second virtual machine.
- the second node first creates a virtual machine process for creating the second virtual machine, and establishes a mapping relationship between the physical memory of the second virtual machine and the first physical storage space through the virtual machine process.
- the purpose of the first physical storage space being used as a physical resource of the memory of the second virtual machine is achieved.
- S103 for a detailed description of the mapping relationship between the physical memory of the second virtual machine and the first physical storage space established by the second node through the virtual machine process, please refer to S103 in which the first node establishes the first physical memory through the virtual machine process (i.e. The description of the mapping relationship between the physical memory of the first virtual machine and the first physical storage space will not be described again.
- the second node also applies for the second node's CPU resources for the second virtual machine through the virtual machine process, which will not be described in detail here.
- the second node can also use virtual machines process, and accesses the first physical storage space through RDMA operations (such as read operations) to read the CPU status information recorded in the second space of the first physical storage space, and set the second virtual storage space based on the read CPU status information.
- the status of the CPU resources requested by the machine For descriptions of the second space and CPU status information, please refer to the relevant descriptions in S1022 and will not be described again.
- the second node after determining the memory physical resources and CPU resources of the second virtual machine, the second node creates the second virtual machine through the virtual machine process.
- the second node can read the operating system code of the virtual machine in a locally pre-stored virtual machine image file, and run the operating system code in a determined resource through the virtual machine process to complete the creation of the second virtual machine.
- the relevant description of the virtual machine image file can be referred to the above description and will not be repeated.
- the second node when the second node monitors that the CPU status of the second virtual machine changes during the running of the second virtual machine, the second node will change the CPU state of the second virtual machine through the virtual machine process.
- the current CPU status information of the second virtual machine is written into the second space in the first physical space of the first storage node to update the CPU status information recorded in the second space, thereby ensuring that the latest CPU status information of the second virtual machine is Recorded in the second space.
- the description of the second node writing the CPU status information of the second virtual machine into the second space in the first physical storage space can be referred to the above description of the first node writing the current CPU status information of the first virtual machine into the first physical storage space.
- the description of the second space in the first physical space of the storage node will not be described again.
- the second node creates the target container in the second virtual machine.
- the second node may first search for the image file of the target container in the second storage node based on the target container ID in the container migration request and the correspondence between the target container ID and the image ID pre-stored in the second storage node. , and read the program code in the image file. Furthermore, the second node runs the program code in the second virtual machine, thereby completing the creation of the target container.
- control node when the control node instructs the second node to migrate the target container, it may also send an instruction to migrate the target container to the first node running the target container before migration.
- the second node after creating the second virtual machine and creating the target container in the second virtual machine, sets the state of the second virtual machine to the running state.
- the second node may set the state of the second virtual machine to the running state through the virtual machine process.
- the control node monitors that the target container created by the second node has started running, and then sends an instruction to delete the target container to the first node.
- the first node deletes the local target container.
- the first node may delete the local target container when the first preset time period arrives after receiving the instruction to migrate the target container sent by the control node. The first preset time period is used to ensure that the first node writes all the local memory data of the first virtual machine into the first physical storage space.
- the resource parameters required by the second virtual machine for running the migrated target container and the first virtual machine used for running the pre-migrated target container are the same (both are based on the resources required for running the target container). Parameters determined), the CPU state of the second virtual machine is the same as the CPU state of the first virtual machine. Therefore, the resource configuration and CPU state of the second virtual machine and the first virtual machine are the same.
- the physical resources of the memory of the second virtual machine are the same as the physical resources of the memory of the first virtual machine, that is, the second virtual machine and the first virtual machine share the physical resources of the memory, that is, the first physical storage space is used for running Shared memory of the security container's virtual machine before and after migration.
- the image file of the target container running in the second virtual machine is the same as the image file of the target container running in the first virtual machine, that is, the target container running in the second virtual machine after migration is the same as the image file running in the first virtual machine before migration.
- the file data of the target container is also the same. It can be seen that when the target container is migrated based on the method provided by the embodiment of this application, the file data of the target container will not be lost.
- the second node when the second node completes the creation of the second virtual machine and the creation of the target container in the second virtual machine, and has received an instruction message sent by the first node to start the target container, then Set the state of the second virtual machine to the running state.
- the second node When the second node has completed the creation of the second virtual machine and the creation of the target container in the second virtual machine, but has not received the instruction message sent by the first node to open the target container, it first changes the status of the second virtual machine. Set to the paused state, and when receiving an instruction message sent by the first node to open the target container, set the state of the second virtual machine to the running state to open the target container.
- the second node may set the state of the second virtual machine to a paused state or a running state through a virtual machine process.
- the first node after receiving the instruction of the migration target container sent by the control node, the first node queries and determines the second physical memory corresponding to the first physical memory (that is, the physical memory of the first virtual machine) in response. Whether all the memory data in the physical memory (that is, a section of local physical memory of the first node) has been written to the first physical storage space. It should be understood that the memory data in the second physical memory is the memory data of the first virtual machine. For description of the correspondence between the first physical memory and the second physical memory, please refer to the above description and will not be described again.
- the first node determines that all the memory data in the second physical memory has been written into the first physical storage space, it deletes the local target container and sends an instruction message to open the target container to the second node to instruct the second node to complete the process.
- the state of the second virtual machine is set to the running state.
- the first node determines that all the memory data in the second physical memory has not been written into the first physical storage space, it calculates the size of the remaining memory data in the second physical memory that has not yet been written into the first physical storage space, and calculates the size of the remaining memory data in the second physical memory that has not been written into the first physical storage space based on the predetermined Assume that the algorithm determines the estimated time required to write all the remaining memory data into the first physical storage space.
- the embodiment of the present application does not specifically limit the preset algorithm.
- the preset algorithm can be any algorithm based on the current network rate and the remaining memory data size.
- the first node determines whether the estimated duration is less than the second preset duration. If the first node determines that the estimated duration is less than or equal to the second preset duration, it suspends running the local target container, and after the estimated duration from the current moment, the first node again determines whether the memory data in the second physical memory is All have been written into the first physical storage space.
- the embodiment of the present application does not limit the specific value of the second preset duration.
- the first node determines that all the memory data in the second physical memory has been written to the first physical storage space after the estimated time from the current moment, it deletes the local target container and sends a message to the second node to open the target container.
- An instruction message is provided to instruct the second node to set the status of the second virtual machine to the running status after completing the creation of the second virtual machine and the creation of the target container in the second virtual machine.
- the first node determines that the memory data in the second physical memory has not all been written into the first physical storage space after the estimated time period from the current time, it will determine again after the second preset time period from the current time. Whether all memory data in the second physical memory has been written into the first physical storage space. If the first node determines that all the memory data in the second physical memory has been written to the first physical storage space after the second preset time period from the current moment, the local target container is deleted and an open message is sent to the second node. An instruction message for the target container to instruct the second node to set the status of the second virtual machine to the running status after completing the creation of the second virtual machine and the creation of the target container in the second virtual machine.
- the second virtual machine used to run the migrated target container and the first virtual machine used to run the pre-migrated target container The memory data and process status of the virtual machine are the same, and the file data of the target container running on the second virtual machine after migration and the target container running on the first virtual machine before migration are also the same.
- the resource configuration and CPU status of the second virtual machine used to run the target container after migration and the first virtual machine used to run the target container before migration are the same. Therefore, when migrating the target container based on the method of this application, no memory data is lost, so the user's business will not be interrupted, and the user will not be aware of it.
- the second node updates the network routing information of the target container.
- the second node may update the network routing information used to indicate access to the target container in the routing information stored by itself. For example, the second node modifies the address of the node to which the container belongs in the original network routing information for accessing the target container from the address of the first node to the address of the second node.
- the second node can send a routing update message to some or all working nodes in the container cluster except the second node to instruct the working node that receives the routing update message to update its own stored routing information to indicate access to the target container. network routing information.
- the second node may update the network routing information used to indicate access to the target container in the routing information stored by itself.
- the control node monitors that the target container has been migrated to the second node (for example, the control node monitors that the second virtual machine in the second node has started to run normally), it sends routes to some or all working nodes in the container cluster.
- Update message to instruct the working node that receives the routing update message to update the routing information stored by itself to indicate the network routing information for accessing the target container.
- the first node updates its stored routing information to indicate the network routing information for accessing the target container, and sends routes to some or all working nodes in the container cluster except the first node.
- Update message to instruct the working node that receives the routing update message to update the routing information stored by itself to indicate the network routing information for accessing the target container.
- the embodiments of the present application do not limit this.
- the physical resource of the memory of the first virtual machine of the first node used to run the target container before migration (that is, the first physical storage space on the first storage node) is determined as the first virtual machine used to run the target container after migration.
- the physical resources of the memory of the second virtual machine of the second node that is to say, the first physical storage space on the first storage node not only serves as the memory of the target container running on the first virtual machine before migration, but also serves as the memory of the target container running on the second virtual machine after migration.
- the target container running on the first virtual machine before migration and the target container running on the second virtual machine after migration share memory, as shown in Figure 6 . Furthermore, the image files of the target container running on the first virtual machine before migration and the target container running on the second virtual machine after migration are also the same. Therefore, through the method described in the first possible implementation, it is possible to migrate containers without losing memory data and file data, without interrupting user services, and without users being aware of it.
- FIG. 7 shows a schematic flowchart of yet another container creation method provided by an embodiment of the present application.
- this method is applied to the container cluster 100 shown in Figure 1, and can be executed by the first node having the hardware structure shown in Figure 2.
- the first node may first perform S101 and S1021 in Embodiment 1, and then the first node may perform steps S303-S305.
- the first node determines the second physical storage space of the DPU in the first node as the cached physical resource of the first virtual machine according to the memory size of the first virtual machine.
- the first node is configured with a DPU.
- the second physical storage space is a section of memory space in the storage module of the DPU.
- the second physical storage space can be an ordinary memory space, such as a memory unit composed of DRAM. It can be seen that the second physical storage space is also a storage space local to the first node.
- the first node determines the first resource parameter based on the resource parameters required to run the target container in the first creation request.
- a section of physical storage space ie, the second physical storage space
- the embodiment of the present application does not limit the size of the second physical storage space.
- the size of the second physical storage space may be greater than or equal to the memory size of the first virtual machine, or may be less than or equal to the memory size of the first virtual machine. .
- the DPU in the first node applies to the first storage node for physical resources of the memory of the first virtual machine according to the memory size of the first virtual machine.
- the DPU may apply for a section of physical storage space (ie, the first physical storage space) from the first storage node according to the memory size of the first virtual machine as a physical resource of the memory of the first virtual machine.
- a section of physical storage space ie, the first physical storage space
- the DPU in the first node applies to the first storage node for the physical resources of the memory of the first virtual machine according to the memory size of the first virtual machine.
- the first node determines the memory size of the first virtual machine based on the memory size of the first virtual machine. , the description of the process of applying for physical resources of the memory of the first virtual machine from the first storage node will not be described again.
- the execution subject of S1022 is the first node.
- S1022 can be executed by the CPU and network card of the first node, while the execution subject of S303 is an independent server in the first node.
- Unit DPU For detailed description of the DPU, please refer to the relevant description of the DPU in the first node shown in Figure 2 above, and will not be described again here.
- the DPU of the first node receives the address of the first physical storage space returned by the first storage node, it stores the address of the first physical storage space.
- the size of the first physical storage space may be greater than or equal to the size of the second physical storage space of the DPU, or may be less than or equal to the size of the second physical storage space of the DPU, which is not limited.
- the first node creates the first virtual machine based on the first creation request and the determined physical resources.
- the first node may create a virtual machine process for creating the first virtual machine based on the first creation request.
- relevant instructions on creating a virtual machine process on the first node please refer to the relevant description in S103 above, and will not be described again.
- the first node can establish a mapping relationship between the first physical memory and the second physical storage space through the virtual machine process, and establish a mapping relationship between the second physical storage space and the first physical storage space through the DPU, thereby
- the first physical storage space is used as a physical resource of the memory of the first virtual machine
- the second physical storage space is used as a physical resource of the cache of the first virtual machine.
- the first node may establish a mapping relationship between the first physical memory and the first virtual memory of the virtual machine process through the virtual machine process.
- the first node also establishes a mapping relationship between the first virtual memory and a section of virtual space of the DPU (hereinafter referred to as the DPU virtual space) through the virtual machine process.
- the second physical storage space is the physical storage space of the DPU virtual space.
- the DPU in the first node establishes a mapping relationship between the second physical storage space and the first physical storage space.
- the first physical storage space serves as a physical resource of the memory of the first virtual machine
- the second physical storage space serves as a physical resource of the cache of the first virtual machine.
- the first virtual memory is a section of memory in the virtual memory of the virtual machine process, and the address of the first virtual memory corresponds to the address of the first physical memory one-to-one.
- the DPU virtual space includes an address that corresponds one-to-one to the address of the first virtual memory, that is, the address of the first virtual memory corresponds one-to-one to the addresses of a section of space or all spaces in the DPU virtual space.
- the first physical storage space includes an address that has a one-to-one correspondence with the address of the DPU virtual space, that is, the address of the DPU virtual space has a one-to-one correspondence with the addresses of a section of space or all of the space in the first physical storage space.
- the first node can call the mmap interface through the virtual machine process to establish the mapping relationship between the first virtual memory and the DPU virtual space, and the DPU can write data to the DPU virtual space when the first virtual machine is created. Allocate a section of free physical storage space in the second physical storage space for storing data.
- the DPU of the first node receives and stores the address of the first physical storage space returned by the first storage node, which is equivalent to establishing a mapping relationship between the second physical storage space and the first physical storage space.
- the first node also applies for the CPU resources of the first node for the first virtual machine through the virtual machine process based on the determined CPU resource size required by the first virtual machine, which will not be described in detail.
- the first node can create the first virtual machine through the virtual machine process.
- the first node can read the operating system code of the virtual machine in a locally pre-stored virtual machine image file, and run the operating system code in a determined resource through the virtual machine process to complete the creation of the first virtual machine.
- the relevant description of the virtual machine image file can be referred to the above description and will not be repeated.
- the first node can write the CPU status information of the first virtual machine into the second space in the first physical storage space through the second physical storage space through the virtual machine process.
- the second space and CPU status information please refer to the relevant description in S1022, which will not be described again here.
- the first node may first write the CPU status information of the first virtual machine into the first virtual memory through the virtual machine process. Because the actual physical storage space corresponding to the first virtual memory is the second physical memory (ie, a section of local physical memory of the first node). Therefore, the first node actually writes the CPU status information into the second physical memory through the virtual machine process, and further writes the CPU status information into the second physical storage space in the DPU of the first node. In this way, the DPU writes the CPU status information in the second storage space into the second space in the first physical space of the first storage node through RDMA operations (such as write operations/send operations) based on its own configured RNIC driver module. Thus, recording of CPU status information of the first virtual machine is achieved.
- RDMA operations such as write operations/send operations
- the first node when the first node monitors the CPU status of the first virtual machine to change through the virtual machine process, it will use the process described above to obtain the current CPU status information of the first virtual machine.
- the first node executes S104 to complete the creation of the target container and run the target container.
- the first node runs a virtual machine process
- the virtual machine process creates a first virtual machine, that is, the first virtual machine runs in the virtual machine process as one or more threads of the virtual machine process.
- the target container runs in the first virtual machine, and application process 1, application process 2 and application process 3 run in the target container.
- the first situation the user device accesses the application process 1 in the target container and writes data 1 to the first virtual machine through the application process 1.
- the first node first writes data 1 into the first physical memory (that is, the physical memory of the first virtual machine) through application process 1. Since the first physical memory is the first virtual memory of the virtual machine process, that is, the first node first writes data 1 into the first virtual memory corresponding to the first physical memory.
- the first node When the first node writes data to the first virtual memory, the first node allocates a section of actual physical memory to the first virtual memory (i.e., a section of local physical memory of the first node, as shown in Figure 8 for the second physical memory ). Therefore, the first node writes data 1 into the first virtual memory through application process 1, which actually writes data 1 into the second physical memory.
- the first node writes the data 1 of the second physical memory into the DPU virtual space through the application process 1, and then the DPU can be the DPU in the second physical storage space.
- Virtual space allocates a section of address space to store data1.
- the DPU writes data 1 in the second physical storage space into the first space in the first physical storage space of the first storage node through RDMA operations (such as write operations/send operations),
- RDMA operations such as write operations/send operations
- the DPU can write the data 1 written into the second physical storage space into the first space through RDMA operation in real time.
- the DPU may also periodically write data written into the second physical storage space into the first space through RDMA operations.
- the DPU can also write the data into the first space through an RDMA operation, which is not limited in the embodiment of the present application.
- the first virtual machine in the first node can determine that the data writing task of the process has been completed. Subsequently, the DPU, as an independent unit in the first node, will write the data 1 written into the second physical storage space into the first physical storage space through RDMA operation.
- the first virtual machine in the first node does not need to determine that the data writing task of the process has been completed after receiving the response message of the RDMA operation returned by the first storage node.
- the second physical storage space of the DPU is used as a cache for the first node to write data into the first physical storage space.
- the related process resources for writing data in the first node can be quickly released, thereby saving the resources required for running the process for writing data on the first node.
- the second case the user device accesses the application process 1 in the target container and reads the data 2 of the first virtual machine through the application process 1.
- the first virtual machine in the first node queries the second physical memory corresponding to the first physical memory (ie, a section of local physical memory of the first node) through application process 1.
- the operating system of the first node triggers a page fault exception event.
- the first virtual machine in the first node further queries whether the second physical storage space of the DPU includes data 2.
- the second physical storage space includes data 2
- data 2 is returned to application process 2.
- the RNIC driver module in the DPU generates an RDMA read request for reading data 2 to read from the first space in the first physical storage space of the first storage node. Get data 2 and further return data 2 to application process 1.
- hot data algorithms can be preset in the DPU.
- DPU can determine the data that the application process in the target container has accessed more frequently in the recent period (called hot data). Furthermore, the DPU can pre-read the hot data from the first physical storage space to the second physical storage space through RDMA operations. Therefore, when the application process in the target container accesses the hot data again, the DPU can directly return the data to the application process, thereby improving the efficiency of the application process accessing data.
- control node in the container cluster where the first node is located monitors that the above target container needs to be migrated
- the control node removes the container cluster from the container cluster.
- Another node other than the first node sends a container migration request to instruct the target container to be migrated to the other node.
- FIG. 9 shows a schematic flowchart of yet another container creation method provided by an embodiment of the present application.
- this method can be applied to the container cluster shown in Figure 1.
- This method can first perform S201-S202, and then perform the following steps:
- the second node determines the third physical storage space of the DPU in the second node as a cached physical resource of the second virtual machine according to the container migration request.
- DPU DPU Description of DPU can be referred to above and will not be repeated.
- the second node determines the third physical storage space of the DPU in the second node as the cached physical resource of the second virtual machine according to the container migration request.
- the determination by the first node according to the first creation request in S303 please refer to the determination by the first node according to the first creation request in S303.
- the memory size of the first virtual machine determines the second physical storage space of the DPU in the first node as the cached physical resource description of the first virtual machine, which will not be described again.
- the DPU of the second node obtains the address of the first physical storage space according to the container migration request.
- the execution subject of S203 is the second node.
- S1022 can be executed by the CPU and network card of the second node, while the execution subject of S404 is the independent unit DPU in the second node.
- DPU For detailed description of the DPU, please refer to the relevant description of the DPU in the first node shown in Figure 2 above, and will not be described again here.
- the second node creates a second virtual machine based on the addresses of the third physical storage space and the first physical storage space.
- the second node may first create a virtual machine process for creating the second virtual machine according to the container migration request, and establish a mapping relationship between the physical memory of the second virtual machine and the third physical storage space through the virtual machine process, and, The mapping relationship between the third physical storage space and the first physical storage space is established through the DPU of the second node, so that the first physical storage space is used as the physical resource of the memory of the second virtual machine, and the third physical storage space of the DPU is used as The cached physical resources of the second virtual machine.
- the second node establishes the mapping relationship between the physical memory of the second virtual machine and the third physical storage space
- the detailed description of establishing the mapping relationship between the third physical storage space and the first physical storage space can be referred to S305.
- the description of how a node establishes the mapping relationship between the first physical memory and the second physical storage space and the mapping relationship between the second physical storage space and the first physical storage space will not be described again.
- the second node may also determine the resource parameters of the resources required to create the second virtual machine based on the resource parameters of the target container running in the container migration request, for example, determine the size of the CPU resources required to create the second virtual machine. It should be noted that the second node can also instruct the DPU to access the first physical storage space through an RDMA operation (such as a read operation) to read the CPU status information in the second space of the first physical storage space, and based on the read CPU The status information sets the status of the CPU resources applied for by the second virtual machine.
- RDMA operation such as a read operation
- the second node can create the second virtual machine through the virtual machine process based on the physical resources of the memory of the second virtual machine, the physical resources of the cache of the second virtual machine, and the CPU resources.
- the second node can read the operating system code of the virtual machine in a locally pre-stored virtual machine image file, and run the operating system code in a determined resource through the virtual machine process to complete the creation of the second virtual machine.
- the relevant description of the virtual machine image file can be referred to the above description and will not be repeated.
- the second node when the second node monitors that the CPU status of the second virtual machine changes during the running of the second virtual machine, the second node will change the CPU state of the second virtual machine through the virtual machine process.
- the current CPU status information of the second virtual machine is written into the second space in the first physical space of the first storage node through the third physical storage space to update the CPU status information recorded in the second space, thereby ensuring that the second virtual machine The latest CPU status information is recorded in the second space.
- the second node creates the target container in the second virtual machine.
- the second node creates a target container in the second virtual machine to complete the migration of the target container.
- S406 details Please refer to the description of S205 above.
- the first node queries and determines whether the memory data of the first virtual machine in the second physical storage space of the DPU has been completely written into the first physical storage space to determine whether to send an open message to the second node. Instruction message for the target.
- the first node queries and determines whether the memory data of the first virtual machine in the second physical storage space of the DPU has been completely written into the first physical storage space to determine whether to send the details of the instruction message to open the target container to the second node.
- the first node can query and determine whether all the memory data in the second physical memory has been written to the first physical storage space to determine whether to send the description of the instruction message for opening the target container to the second node. No more Repeat.
- the second node executes S206, thereby completing the migration of the target container.
- the first physical storage space on the first storage node is determined as the first virtual machine used to run the target container after migration.
- the physical resources of the memory of the second virtual machine of the second node that is to say, the first physical storage space on the first storage node not only serves as the memory of the target container running on the first virtual machine before migration, but also serves as the memory of the target container running on the second virtual machine after migration.
- the target container running on the first virtual machine before migration and the target container running on the second virtual machine after migration share memory, as shown in Figure 6 . Furthermore, the image files of the target container running on the first virtual machine before migration and the target container running on the second virtual machine after migration are also the same. Therefore, through the method described in the second possible implementation, it is possible to migrate the container without losing memory data and file data, without interrupting the user's business, and without the user being aware of it.
- the DPU since the hot data of the target container can be pre-cached in the DPU of the first node and the second node, when the application process in the target container accesses the hot data, the DPU can directly The application process returns these data, thereby improving the efficiency of the application process in accessing data.
- FIG. 10 shows a schematic flowchart of yet another container creation method provided by an embodiment of the present application.
- this method is applied to the container cluster 100 shown in Figure 1.
- This method can continue to perform the following steps on the basis of performing the method of S101-S104 shown in Figure 3 or Figure 7:
- the third node obtains a second creation request for instructing the creation of the target container.
- the target container is the target container created by the first node based on the method shown in Figure 3 or Figure 7.
- the third node is any working node other than the first node in the container cluster where the first node is located.
- the first node is the working node 111 in the container cluster 100 shown in FIG. 1
- the third node may be the working node 113 in the container cluster 100 shown in FIG. 1 .
- the third node obtains the description of the second creation request for instructing the creation of the target container.
- the description of the second creation request can refer to the first creation request in S101 above. The description will not be repeated.
- the third node creates the target container based on the second creation request.
- the third node may first create a third virtual machine for running the target container based on the second creation request, and create the target container in the third virtual machine.
- the physical resource of the memory of the third virtual machine is the first physical storage space, that is, the third node is used for
- the physical resources of the memory of the third virtual machine running the target container are the same as the physical resources of the memory of the first virtual machine used to run the target container in the first node.
- the third virtual machine used to run the target container in the third node and the first virtual machine used to run the target container in the first node share the same physical memory resource.
- the third node After the third node completes creating the target container, it sets the status of the third virtual machine running the target container to the paused state.
- the target container running on the third virtual machine on the third node can be used as a backup container for the target container running on the first virtual machine on the first node. That is, the target container running on the first virtual machine in the first node is the main target container, and the target container running on the third virtual machine in the third node is the backup target container.
- the embodiment of the present application does not specifically limit the order in which the first node creates the primary target container and the third node creates the backup target container.
- the first node can first create a primary target container based on the method described in Figure 3 or Figure 7 , and then the third node can create a backup target container based on the method described in Figure 10 .
- the third node creates the backup target container based on the method described in Figure 10. Not limited to this.
- the embodiment of this application has completed the creation of the primary and backup target containers.
- control node monitors that the target container in the first node needs to be migrated:
- control node sends an instruction message for opening the target container to the third node.
- third node sets the state of the third virtual machine used to run the target container to the running state.
- the target container in the third virtual machine serving as the backup target container starts processing services.
- the third node may also read the latest CPU status of the first virtual machine from the second space of the first physical storage space through an RDMA operation.
- Information for updating the CPU status of the third virtual machine For example, the third node can directly read the latest CPU status information of the first virtual machine from the second space of the first physical storage space through RDMA operation, or the DPU of the third node can read the latest CPU status information of the first virtual machine from the second space of the first physical storage space through RDMA operation.
- the latest CPU status information of the first virtual machine is read from the second space, which is not limited.
- control node also sends an instruction message to pause/delete the target container to the first node.
- first node deletes the local target container.
- control node sends an instruction message for migrating the target container to the first node.
- the first node queries and determines whether all the locally stored memory data of the first virtual machine has been written into the first physical storage space, and determines whether to send an instruction message for opening the target container to the third node.
- For instructions on how the first node determines whether all the locally stored memory data of the first virtual machine has been written into the first physical storage space refer to S205 for the first node to query and determine whether all the memory data in the second physical memory has been written. The description of writing into the first physical storage space will not be described again.
- the third node After receiving the instruction message sent by the first node to open the target container, the third node sets the state of the third virtual machine to the running state. As a result, the target container in the third virtual machine serving as the backup target container starts processing services.
- the third node may also read the latest CPU status of the first virtual machine from the second space of the first physical storage space through an RDMA operation. Information for updating the CPU status of the third virtual machine.
- the third node and the first node update the network routing information for accessing the target container to complete the migration of the target container from the first node to the third node.
- the first node and the third node update network routing information for accessing the target container to complete migrating the target container from the first node to the third node.
- the target container running on the third virtual machine in the third node is upgraded to the main target container and is used to run application processes to process user services. .
- any working node other than the third node (or other than the third node and the first node) in the container cluster to which the third node belongs can obtain a creation request indicating the creation of a target container to create a backup of the main target container. target container. This will not be elaborated upon.
- the embodiment of the present application can switch the running status of the primary and backup target containers, thereby quickly realizing the situation without losing memory data, without interrupting user services, and without user awareness. Complete the container migration below.
- the containers created by the container creation method provided by the embodiments of the present application are migrated, the containers before and after migration share a section of physical storage space in the storage node, so that memory data can be lost without interrupting user services. , and the container migration is completed without the user being aware of it.
- Figure 11 shows a schematic structural diagram of a container creation device 110 provided by an embodiment of the present application.
- the container creation device 110 is applied to the first node in the container cluster shown in Figure 1.
- the container creation device 110 can be used to execute the above-mentioned method of creating a container, for example, to execute Figures 3, 5, 7, and 9 Or the part of the method shown in Figure 10 that is executed by the first node.
- the container creation device 110 may include an acquisition unit 111 and a processing unit 112 .
- the obtaining unit 111 is configured to obtain the first creation request indicating the creation of the target container.
- the processing unit 112 is configured to create a first virtual machine based on the first creation request, and run the target container in the first virtual machine.
- the target container is a secure container
- the first creation request includes resource parameters of a first virtual machine running the target container
- the physical resource of the memory of the first virtual machine is the first physical storage space in the first storage node.
- the acquisition unit 111 may be used to perform S101, and the processing unit 112 may be used to perform S103 and S104.
- the above-mentioned first node accesses the first physical storage space through RDMA.
- the above-mentioned first physical storage space is a persistent memory medium.
- the processing unit 112 is also configured to: before creating the first virtual machine based on the first creation request: determine the memory size of the first virtual machine based on the first creation request; The node applies for a first physical storage space and determines the first physical storage space as a physical resource of the memory of the first virtual machine. Wherein, the first physical storage space is greater than or equal to the memory size of the first virtual machine.
- the processing unit 112 may be used to perform S102.
- the processing unit 112 is also configured to: determine the second physical storage space of the DPU in the first node as a cached physical resource of the first virtual machine; instruct the DPU to establish the second physical storage space and the first physical storage space. mapping relationship between them.
- the DPU in the first node is used to access the first physical storage space through RDMA, and is used to write data in the second physical storage space into the first physical storage space.
- the processing unit 112 may be used to perform S303.
- the DPU in the first node is also used to read the hot data of the target container from the first physical storage space.
- the first node is any node in the above container cluster.
- the container cluster also includes a second node.
- the second virtual machine running the target container in the second node
- the physical resource of memory is the first physical storage space.
- the first node is any node in the above container cluster.
- the container cluster also includes a third node.
- the third node is used to create a backup container of the target container.
- the third node runs a third virtual machine of the backup container. The machine is in a suspended state, and the physical resource of the memory of the third virtual machine is the first physical storage space.
- the third virtual machine when the first virtual machine is in a fault or suspended state, the third virtual machine is in a running state.
- processing unit 112 is also configured to update network routing information for accessing the target container after the target container is migrated to another node in the container cluster.
- the container creation device 110 also includes: a sending unit 113, configured to send a routing update message to other nodes in the container cluster except the first node after the target container is migrated to another node in the container cluster, To enable other nodes to update the network routing information for accessing the target container.
- a sending unit 113 configured to send a routing update message to other nodes in the container cluster except the first node after the target container is migrated to another node in the container cluster, To enable other nodes to update the network routing information for accessing the target container.
- the functions implemented by the acquisition unit 111 and the sending unit 113 in the container creation device 110 can be implemented through the network interface 203 in Figure 2, or the functions implemented by the acquisition unit 111 can be implemented through the network interface 203 in Figure 2. It shows that the input interface of the first node (such as keyboard, touch screen, mouse, etc. input interface) is implemented, and the functions implemented by the processing unit 112 can be implemented by the processor 201 in FIG. 2 .
- the input interface of the first node such as keyboard, touch screen, mouse, etc. input interface
- FIG. 12 shows a schematic structural diagram of a container creation device 120 provided by an embodiment of the present application.
- the container creation device 120 is applied to the second node in the container cluster shown in Figure 1.
- the container creation device 120 can be used to perform the above-mentioned method of creating a container, for example, to perform the method shown in Figure 5 or Figure 9. The part executed by the two nodes.
- the container creation device 120 may include a receiving unit 121 and a processing unit 122 .
- the receiving unit 121 is configured to receive a container migration request, which instructs the second node to create a second container for running the target container.
- Virtual machine the target container is a secure container.
- the processing unit 122 is configured to determine the first physical storage space in the first storage node as a physical resource of the memory of the second virtual machine; create a second virtual machine based on the physical resource of the memory of the second virtual machine; and, Run the target container in the second virtual machine.
- the first physical storage space is a physical resource of the memory of the first virtual machine.
- the receiving unit 121 may be used to perform S202, and the processing unit 122 may be used to perform S203, S204 and S205.
- the container migration request includes the ID of the target container
- the container creation device 120 further includes: a sending unit 123, configured to send a memory request message including the ID of the target container to the first storage node.
- the memory request message Used to request to obtain the address of the first physical storage space corresponding to the ID of the target container.
- the receiving unit 121 is also configured to receive the address of the first physical storage space returned by the first storage node.
- the processing unit 123 is specifically configured to determine the first physical storage space indicated by the received address as the physical resource of the memory of the second virtual machine.
- the container creation device 120 further includes: a sending unit 123, configured to send a memory request message including the ID of the first virtual machine to the first storage node, The memory request message is used to request to obtain the address of the first physical storage space that has a corresponding relationship with the ID of the first virtual machine.
- the receiving unit 122 is configured to receive the address of the first physical storage space returned by the first storage node.
- the processing unit 123 is specifically configured to determine the first physical storage space indicated by the received address as the physical resource of the memory of the second virtual machine.
- the receiving unit 121 and the sending unit 123 in the container creation device 120 implement The functions can be implemented through the network interface 203 in Figure 2, and the functions implemented by the processing unit 122 can be implemented through the processor 201 in Figure 2.
- An embodiment of the present application also provides a container cluster, which includes a first node and a second node.
- the first node is used to execute the part executed by the first node in the method described above, for example, to execute the method shown in Figure 3 to create the target container.
- the second node is used to execute the part of the method described above that is executed by the second node, for example, execute the method shown in Figure 5 to migrate the target container running on the first node to the second node.
- the embodiment of the present application also provides a chip system 130.
- the chip system 130 includes at least one processor and at least one.
- the processor may be the processor 131 shown in the solid line box in Figure 13 (or the processor 131 shown in the dotted line box)
- the one interface circuit may be the interface circuit 132 shown in the solid line box in FIG. 13 (or the interface circuit 132 shown in the dotted line box).
- the two processors include the processor 131 shown in the solid line box and the processor 131 shown in the dotted line box in Figure 13.
- the two interfaces The circuit includes the interface circuit 132 shown in the solid line box and the interface circuit 132 shown in the dotted line box in Figure 13 . There is no limit to this.
- the processor 131 and the interface circuit 132 may be interconnected via lines.
- the interface circuit 132 may be configured to receive a signal (eg, obtain the above-mentioned first creation request or container migration request, etc.).
- interface circuit 132 may be used to send signals to other modules (eg, processor 131).
- the interface circuit 132 can read instructions stored in the memory and send the instructions to the processor 131 .
- the container creation device can be caused to perform each step in the above embodiment.
- the chip system 130 may also include other discrete devices, which are not specifically limited in the embodiments of the present application.
- Embodiments of the present application also provide a computer program product and a computer-readable storage medium for storing the computer program product.
- the computer program product may include one or more program instructions that, when executed by one or more processors, may provide what is described above with respect to Figure 3, Figure 5, Figure 7, Figure 9 or Figure 10 function or part of a function. Therefore, for example, one or more features referred to in S101 to S104 in FIG. 3 may be performed by one or more instructions in the computer program product.
- a container creation apparatus such as that described with respect to FIG. 3, FIG. 5, FIG. 7, FIG. 9, or FIG. 10, may be configured to, in response to one or more program instructions stored in a computer-readable storage medium, Provide various operations, functions, or actions.
- the computer program product includes one or more computer instructions.
- the computer executes the instructions, the processes or functions according to the embodiments of the present application are generated in whole or in part.
- the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
- Computer instructions may be stored in a computer-readable storage medium or may be obtained from a computer-readable storage medium.
- Computer instructions can be transmitted from a website, computer, server, or data center over wires (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or Wireless (such as infrared, wireless, microwave, etc.) transmission to another website, computer, server or data center.
- Computer-readable storage media can be any available media that can be accessed by a computer or include one or more data storage devices such as servers and data centers that can be integrated with the media. Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请公开了一种容器的创建方法、装置及存储介质,涉及计算机通信技术领域。该方法由容器集群中的第一节点执行,该方法包括:获取用于指示创建安全容器的第一创建请求,基于第一创建请求创建第一虚拟机,并在第一虚拟机中运行该安全容器。其中,第一虚拟机的内存的物理资源为第一存储节点中的第一物理存储空间。这样,当需要将该安全容器从第一节点迁移至容器集群中的另一节点时,只需将第一物理存储空间作为用于运行迁移后的安全容器的虚拟机的内存物理资源,就能实现在不影响容器业务连续性、不丢失容器业务数据情况下对该安全容器的迁移。
Description
本申请涉及计算机通信技术领域,尤其涉及一种容器的创建方法、装置及存储介质。
Kubernetes(简称为K8S)是一个开源容器编排、调度和管理***,被视为云原生时代的操作***。采用K8S实现的集群又可以称为K8S集群。一个K8S集群包括若干个物理机或虚拟机(virtual machine,VM),一个物理机或者一个虚拟机可以视为一个节点(node)。K8S以pod为最小单元在集群内的节点上创建容器。一个pod可以包括一个或多个容器。其中,容器一般用于为应用进程提供运行环境,或者可以理解为用于运行应用实例。
通常,pod的生命周期短暂。因此,pod会被频繁的创建、调度以及销毁。当pod被调度到某个节点后,一般会一直在这个节点中运行。由于pod不具备自愈能力,因此当节点不满足pod的存活要求时(例如节点的资源耗尽),这种情况下,该节点会删除该pod。并且,K8S会在另一个节点上新建一个pod来代替被删除的pod。
然而,K8S删除pod、以及在另一个节点新建pod的过程会导致pod中的容器所运行的应用实例被中断,从而破坏了业务的连续性并导致业务数据丢失。因此,如何创建在迁移时不会影响业务连续性、不丢失业务数据的容器,是当前亟待解决的技术问题。
发明内容
本申请提供了一种容器的创建方法、装置及存储介质,该方法能够在不影响容器业务连续性、不丢失容器业务数据情况下实现对容器的迁移。
为达上述目的,本申请提供如下技术方案:
第一方面,本申请提供了一种容器的创建方法,该方法应用于第一节点。该方法包括:获取用于指示创建目标容器的第一创建请求;基于第一创建请求创建第一虚拟机;在第一虚拟机中运行目标容器。其中,目标容器为安全容器,第一创建请求包括运行目标容器的第一虚拟机的资源参数,第一虚拟机的内存的物理资源为第一存储节点中的第一物理存储空间。
这样,当需要将基于本申请所提供方法创建的安全容器从第一节点迁移至另一节点时,只需将第一物理存储空间作为用于运行迁移后的安全容器的虚拟机的内存物理资源,就能实现在不影响容器业务连续性、不丢失容器业务数据情况下对该安全容器的迁移。
在一种可能的设计方式中,上述的第一节点通过远程直接内存访问RDMA访问第一物理存储空间。
通过该可能的设计,第一节点中创建的第一虚拟机可以将第一存储节点的第一物理存储空间当做内存来访问,即实现了将第一物理存储空间作为第一虚拟机的内存的目的。
在另一种可能的设计方式中,上述的第一物理存储空间为持久化内存介质。
通过该可能的设计,由于持久化内存介质的存储性能介于传统DRAM与闪存存储器(例如NAND闪存)之间。因此作为第一虚拟机内存的第一物理存储空间相比闪存存储器拥有更高的访问速度、更低的访问延迟,而相比传统DRAM拥有更大的存储容量,且数据断电不丢失,可以持久化存储,成本也相对较低。
在另一种可能的设计方式中,在上述的基于第一创建请求创建第一虚拟机之前,上述方
法还包括:根据第一创建请求,确定第一虚拟机的内存大小;根据第一虚拟机的内存大小,向第一存储节点申请第一物理存储空间,并将第一物理存储空间确定为第一虚拟机的内存的物理资源。其中,第一物理存储空间大于或等于第一虚拟机的内存大小。
在另一种可能的设计方式中,上述方法还包括:将第一节点中的数据处理单元DPU的第二物理存储空间确定为第一虚拟机的缓存的物理资源;指示DPU建立第二物理存储空间和第一物理存储空间之间的映射关系。其中,DPU用于通过RDMA访问第一物理存储空间,并用于将第二物理存储空间的数据写入第一物理存储空间。
通过该可能的设计,当第一节点创建第一虚拟机后,在第一虚拟机将数据写入第一物理存储空间的过程中,对于第一节点中的第一虚拟机为将数据写入第一物理存储空间而发起的进程而言,当数据被写入DPU的第二物理存储空间后,第一节点中的第一虚拟机即可确定该进程的数据写入任务已完成。后续,DPU作为第一节点中的独立单元,会通过RDMA操作将写入第二物理存储空间的数据写入第一物理存储空间。
也就是说,第一节点中的第一虚拟机无需在接收到第一存储节点返回的RDMA操作的响应报文后,才确定进程的数据写入任务已完成。换言之,DPU的第二物理存储空间是作为第一节点将数据写入第一物理存储空间的缓存来用的。对于第一节点而言,当确定数据已写入第二物理存储空间,即确定数据已写入第一物理存储空间。这样的话,第一节点中的写入数据的相关进程资源即可被快速释放,从而节省了第一节点的运行用于写数据的进程所需的资源。
在另一种可能的设计方式中,上述DPU还用于从第一物理存储空间读取目标容器的热数据。
这样,通过该可能的设计,当DPU预先将目标容器中的应用进程在近一段时间内访问次数较多的数据(即热数据)从第一物理存储空间读取到第二物理存储空间,从而,当目标容器中的应用进程再次访问这些热数据时,DPU可以直接向应用进程返回这些数据,从而提高了应用进程访问数据的效率。
在另一种可能的设计方式中,上述第一节点是容器集群中的任一个节点,该容器集群中还包括第二节点,当目标容器被迁移至第二节点,第二节点中运行目标容器的第二虚拟机的内存的物理资源为第一物理存储空间。
通过该可能的设计,在目标容器被迁移至第二节点,且将第一物理存储空间作为第二节点中第二虚拟机的内存物理资源,就能实现在不影响容器业务连续性、不丢失容器业务数据情况下对该安全容器的迁移。
在另一种可能的设计方式中,上述第一节点是容器集群中的任一个节点,该容器集群中还包括第三节点,第三节点用于创建目标容器的备份容器,第三节点上运行备份容器的第三虚拟机为暂停状态,第三虚拟机的内存的物理资源为第一物理存储空间。
在另一种可能的设计方式中,当上述第一虚拟机为故障或暂停状态,则上述第三虚拟机为运行状态。
通过该可能的设计实现了将目标容器从第一节点迁移至第三节点的目的,且由于第一物理存储空间作为用于运行迁移后的安全容器的第三虚拟机的内存物理资源,从而实现了在不影响容器业务连续性、不丢失容器业务数据情况下对该安全容器的迁移。
在另一种可能的设计方式中,上述方法还包括:当目标容器被迁移至容器集群中的另一节点后,更新访问目标容器的网络路由信息。
在另一种可能的设计方式中,上述方法还包括:当目标容器被迁移至容器集群中的另一节点后,向容器集群中除第一节点外的其他节点发送路由更新消息,以使其他节点更新访问
目标容器的网络路由信息。
通过上述两种可能的设计方式,能够实现用户设备后续正常访问目标容器的目的,且用户对目标容器迁移的过程是无感知的。
第二方面,本申请提供了一种容器的创建装置。
在一种可能的设计方式中,该创建装置用于执行上述第一方面提供的任一种方法。本申请可以根据上述第一方面提供的任一种方法,对该创建装置进行功能模块的划分。例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能集成在一个处理模块中。示例性的,本申请可以按照功能将该创建装置划分为获取单元和处理单元等。上述划分的各个功能模块执行的可能的技术方案和有益效果的描述均可以参考上述第一方面或其相应的可能的设计提供的技术方案,此处不再赘述。
在另一种可能的设计中,该创建装置包括:存储器、网络接口、以及一个或多个处理器,该一个或多个处理器通过网络接口接收或发送数据,该一个或多个处理器被配置为读取存储在存储器中的程序指令,以使得创建装置执行如第一方面及其任一种可能的设计方式提供的任一种方法。
第三方面,本申请提供了一种容器的创建方法,该方法应用于第二节点。该方法包括:接收容器迁移请求,该容器迁移请求指示将运行于第一节点中第一虚拟机的目标容器迁移至第二节点,以及指示第二节点创建用于运行目标容器的第二虚拟机,该目标容器为安全容器。将第一存储节点中的第一物理存储空间确定为第二虚拟机的内存的物理资源,该第一物理存储空间为第一虚拟机的内存的物理资源。基于第二虚拟机的内存的物理资源,创建第二虚拟机,并在第二虚拟机中运行目标容器。
基于本申请提供的方法,目标容器在迁移前后的内存的物理资源是一样的。这样,迁移后运行于第二节点的第二虚拟机中的目标容器的内存数据,即为迁移前运行于第一节点的第一虚拟机的目标容器的内存数据。因此,当迁移前后的目标容器的内存数据相同,这表明目标容器在迁移前后的业务状态是相同的。即,本申请方法能够实现在迁移容器时不影响容器业务的连续性、且不丢失容器的业务数据。
在另一种可能的设计中,上述的容器迁移请求中包括目标容器的身份标识号ID,则上述的将第一存储节点中的第一物理存储空间确定为第二虚拟机的内存的物理资源包括:向第一存储节点发送包括目标容器的ID的内存请求消息,该内存请求消息用于请求获取与目标容器的ID具有对应关系的第一物理存储空间的地址。接收第一存储节点返回的第一物理存储空间的地址,并将该地址指示的第一物理存储空间确定为第二虚拟机的内存的物理资源。
该可能的设计为第二节点获取第一物理存储空间(即第一虚拟机的内存的物理资源)的地址,从而将第一物理存储空间作为第二虚拟机的内存的物理资源的一种实现方式。
在另一种可能的设计中,上述容器迁移请求中包括第一虚拟机的身份标识号ID,则上述的将第一存储节点中的第一物理存储空间确定为第二虚拟机的内存的物理资源包括:向第一存储节点发送包括第一虚拟机的ID的内存请求消息,该内存请求消息用于请求获取与第一虚拟机的ID具有对应关系的第一物理存储空间的地址。接收第一存储节点返回的第一物理存储空间的地址,并将该地址指示的第一物理存储空间确定为第二虚拟机的内存的物理资源。
该可能的设计为第二节点获取第一物理存储空间(即第一虚拟机的内存的物理资源)的地址,从而将第一物理存储空间作为第二虚拟机的内存的物理资源的另一种实现方式。
第四方面,本申请提供了一种容器的创建装置。
在一种可能的设计方式中,该创建装置用于执行上述第三方面提供的任一种方法。本申
请可以根据上述第三方面提供的任一种方法,对该创建装置进行功能模块的划分。例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能集成在一个处理模块中。示例性的,本申请可以按照功能将该创建装置划分为接收单元和处理单元等。上述划分的各个功能模块执行的可能的技术方案和有益效果的描述均可以参考上述第三方面或其相应的可能的设计提供的技术方案,此处不再赘述。
在另一种可能的设计中,该创建装置包括:存储器、网络接口、以及一个或多个处理器,该一个或多个处理器通过网络接口接收或发送数据,该一个或多个处理器被配置为读取存储在存储器中的程序指令,以使得创建装置执行如第三方面及其任一种可能的设计方式提供的任一种方法。
第五方面,本申请提供了一种容器集群,该容器集群包括第一节点和第二节点。其中,第一节点用于执行如第一方面及其任一种可能的设计方式提供的任一种方法,以创建目标容器。第二节点用于执行如第三方面及其任一种可能的设计方式提供的任一种方法,以实现将运行于第一节点的目标容器迁移至所述第二节点。
第六方面,本申请提供了一种计算机可读存储介质,该计算机可读存储介质包括程序指令,当程序指令在计算机或处理器上运行时,使得计算机或处理器执行第一方面中或第二方面中的任一种可能的实现方式提供的任一种方法。
第七方面,本申请提供了一种计算机程序产品,当其在容器的创建装置上运行时,使得第一方面中的任一种可能的实现方式提供的任一种方法被执行。
第八方面,本申请提供了一种芯片***,包括:处理器,处理器用于从存储器中调用并运行该存储器中存储的计算机程序,执行第一方面中的实现方式提供的任一种方法。
可以理解的是,上述提供的任一种装置、计算机存储介质、计算机程序产品或芯片***等均可以应用于上文所提供的对应的方法,因此,其所能达到的有益效果可参考对应的方法中的有益效果,此处不再赘述。
在本申请中,上述容器的创建装置的名字对设备或功能模块本身不构成限定,在实际实现中,这些设备或功能模块可以以其他名称出现。只要各个设备或功能模块的功能和本申请类似,属于本申请权利要求及其等同技术的范围之内。
图1为本申请实施例提供的一种容器集群的架构示意图;
图2为本申请实施例提供的一种第一节点的硬件结构示意图;
图3为本申请实施例提供的一种容器的创建方法的流程示意图;
图4为本申请实施例提供的一种目标容器在处理业务时的数据读写流程示意图;
图5为本申请实施例提供的另一种容器的创建方法的流程示意图;
图6为本申请实施例提供的一种共享内存的示意图;
图7为本申请实施例提供的又一种容器的创建方法的流程示意图;
图8为本申请实施例提供的另一种目标容器在处理业务时的数据读写流程示意图;
图9为本申请实施例提供的又一种容器的创建方法的流程示意图;
图10为本申请实施例提供的又一种容器的创建方法的流程示意图;
图11为本申请实施例提供的一种容器的创建装置的结构示意图;
图12为本申请实施例提供的另一种容器的创建装置的结构示意图;
图13为本申请实施例提供的一种芯片***的示意图。
为了更清楚的理解本申请实施例,下面对本申请实施例中涉及的部分术语或技术进行说明:
1)、容器
容器可以用于为应用进程提供运行环境,或者可以理解为用于运行应用实例。
一般的,容器可以分为普通容器和安全容器。其中,普通容器可以直接运行于节点中,安全容器则运行于集群中的节点上所创建的虚拟机中。
相比普通容器,运行于虚拟机中的安全容器具有更高的安全性。
这是由于用于运行容器的虚拟机通常仅用于运行容器,这种虚拟机一般被称为轻量化虚拟机。轻量化虚拟机和普通虚拟机一样配置有独立的操作***内核,且在虚拟化层被安全隔离。也就是说,运行于轻量化虚拟机的容器不会共享宿主节点(即用于运行该轻量化虚拟机的节点(包括物理节点和虚拟节点))的操作***内核。因此,安全容器比普通容器更具安全性。
另外,由于轻量化虚拟机仅用于运行容器,因此轻量化虚拟机仅需配置支持安全容器运行的功能模块即可,即轻量化虚拟机比普通虚拟机的启动和运行都更快。
2)、其他术语
在本申请的实施例中,术语“第一”、“第二”并不是表示顺序关系,而是为了区别不同的对象,以下文件中提到的第一、第二等也是为了区别不同的报文等,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。
还应理解,在本申请的各个实施例中,各个过程的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。
在实际中,对于安全容器,当需要迁移安全容器时,可以通过迁移用于运行安全容器的虚拟机来实现安全容器的迁移。
在一种可能的实现方式中,可以通过同时在两个节点上创建两个相同的虚拟机,并将其中一个虚拟机作为主虚拟机,另一个虚拟机作为备虚拟机。其中,主虚拟机正常处理用户提交的业务,并将用户提交的业务传递至备虚拟机,以使备虚拟机对用户提交的业务进行处理,从而备虚拟机可以得到和主虚拟机相同的处理结果。当主虚拟机确定备虚拟机已处理完成用户业务,则将自己处理用户业务后得到的处理结果输出给用户。而备虚拟机的所有对外输出被屏蔽,即备虚拟机不向外输出自身得到的处理结果。当主虚拟机失效/故障时,运行主虚拟机的节点只需通知运行备虚拟机的节点解除对备虚拟机的输出屏蔽,即可实现主、备虚拟机的即时切换,从而达到在不中断业务的情况下实现迁移虚拟机的目的,进而实现迁移运行于虚拟机中的安全容器的目的。
然而,通过该方式实现虚拟机中容器的迁移时,主、备虚拟机中的容器所运行的应用实例需要都完成对用户业务的处理后,主虚拟机才会向用户输出业务的处理结果,这会造成用户面业务的响应延迟,从而降低了用户的体验。
基于此,本申请实施例提供了一种容器的创建方法,该方法在创建安全容器时所创建的第一虚拟机的内存的物理资源被设置为第一存储节点中的第一物理存储空间。这样,基于该方法创建的安全容器在迁移时,只需将用于运行迁移后的安全容器的第二虚拟机的内存的物理资源也设置为第一物理存储空间,也即第一物理存储空间作为用于运行迁移前后的安全容
器的虚拟机的共享内存。通过这种方式迁移安全容器,能够实现不丢失业务数据、不中断用户业务、且用户无感知。
本申请实施例还提供了一种容器集群,该容器集群可以包括控制节点和用于部署容器的多个节点(或称为工作节点(worker node))。其中,控制节点用于管理该多个工作节点上运行的容器,例如用于指示工作节点创建容器、或监控工作节点上容器的运行情况等。可选的,为实现容灾备份,控制节点的数量可以是多个。
可选的,上述的多个工作节点可以是物理节点,也可以是虚拟节点,对此不作限定。上述的控制节点可以是任意具有计算处理能力的节点设备,该节点设备可以是物理节点,也可以是虚拟节点,对此不作限定。其中,物理节点例如可以是通用计算机、笔记本电脑等计算设备,虚拟节点例如可以是虚拟机等,不限于此。
可以理解,上述控制节点可以是独立的节点设备,当然,该控制节点的功能也可以由上述多个工作节点中的任一个工作节点实现,即控制节点是该任一个工作节点中的功能模块,对此不作限定。为简单描述,本申请实施例在下文中以控制节点是独立的节点设备进行说明。
可选的,该容器集群还可以包括一个或多个存储节点。
本申请实施例提供的容器集群中的一个或多个存储节点,可以用于为容器集群中部署有安全容器的工作节点提供共享内存。该共享内存例如是由多个持久化内存介质(如XL-Flash等)组成的内存单元。由于持久化内存介质的存储性能介于传统动态随机存取存储器(dynamic random access memory,DRAM)与闪存存储器(flash memory)(例如NAND闪存)之间。因此该共享内存相比闪存存储器拥有更高的访问速度、更低的访问延迟,而相比传统DRAM拥有更大的存储容量,且数据断电不丢失,可以持久化存储,成本也相对较低。
可选的,本申请实施例所提供的容器集群中的一个或多个存储节点,还可以用于为该容器集群中的工作节点上所运行的容器的镜像文件提供存储空间。该存储空间例如是由多个非易失性存储介质组成的存储单元,如机械硬盘、固态硬盘等。该存储空间的存储性能弱于持久化内存介质,但容量更大,成本更低。其中,容器的镜像文件(或者称为容器镜像(container image))中存储有该容器的程序代码,以及用于存储该容器需要持久化存储的数据等,对此不作详述。
需要说明,为容器集群中部署有安全容器的节点提供共享内存的存储节点,以及为容器集群中的节点所运行的容器的镜像文件提供存储空间的存储节点,可以是容器集群中相同的存储节点,当然也可以是容器集群中不同的存储节点,对此不作限定。
当然,本申请实施例中的容器集群中也可以不包括存储节点。这种情况下,可以通过独立存储***中的存储节点来为该容器集群提供数据存储服务。例如该存储***中的存储节点可以用于为该容器集群中部署有安全容器的节点提供共享内存的存储服务,和/或用于为该容器集群中节点上运行的容器的镜像文件提供存储空间,对此不作限定。
为简单描述,本申请实施例在下文中以本申请实施例提供的容器集群中包括一个或多个存储节点为例进行说明。
作为示例,参考图1,图1示出了本申请实施例提供的一种容器集群100的架构示意图。
如图1所示,容器集群100包括控制节点101、控制节点102以及用于部署容器的n个工作节点,该n个工作节点分别为工作节点111、工作节点112、工作节点113、…、以及工作节点11n。其中n是大于3的整数。其中,控制节点102为控制节点101的备份控制节点。
容器集群100还包括2个存储节点,分别为存储节点121和存储节点122。存储节点121和存储节点122可以分别和上述n个工作节点之间具有通信连接。这样,该n个工作节点可
以通过远程直接内存访问(remote direct memory access,RDMA)来访问存储节点121和存储节点122。其中,为容器集群100中部署有安全容器的工作节点提供共享内存的存储节点,以及为上述n个工作节点中运行的容器的镜像文件提供存储空间的存储节点,可以均为存储节点121,或者均为存储节点122,或者分别是存储节点121和存储节点122。
可以理解,尽管图1中未示出,控制节点101、控制节点102和存储节点121(和/或存储节点122)之间也可以具有通信连接,以便控制节点101、控制节点102可以访问存储节点121(和/或存储节点122)。
还应理解,尽管图1中未示出,工作节点111、工作节点112、工作节点113、…、以及工作节点11n之间可以互相通信。
本申请实施例还提供了一种容器的创建装置,该装置应用于上述的容器集群。可选的,该装置可以是上述容器集群中的任一个用于部署容器的工作节点(下文中称为第一节点),或者,该装置是第一节点中的功能模块,对此不作限定。作为示例,第一节点可以是图1所示容器集群100中的工作节点111、工作节点112或工作节点113等等,对此不作限定。
以本申请实施例提供的容器的创建装置是容器集群中的第一节点为例,参考图2,图2示出了第一节点的硬件结构示意图。如图2所示,第一节点20包括处理器201、存储器202、网络接口203以及总线204。处理器201、存储器202、网络接口203之间通过总线204连接。
处理器201是第一节点20的控制中心,可以是一个通用中央处理器(central processing unit,CPU),处理器201还可能是其他通用处理器、数字信号处理器(digital signal processing,DSP)、专用集成电路(application-specific integrated circuit,ASIC)、现场可编程门阵列(field-programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件、图形处理器(graphics processing unit,GPU)、神经网络处理单元(neural processing unit,NPU)、张量处理器(tensor processing unit,TPU)或人工智能(artificial intelligent)芯片、数据处理器(data processing unit,DPU)等。
作为一个示例,处理器201包括一个或多个CPU,例如图2中所示的CPU 0和CPU 1。此外,本申请并不限定每个处理器中处理器核的个数。
存储器202用于存储程序指令或应用进程所要访问的数据,处理器201可以通过执行存储器202中的程序指令,以实现本申请实施例提供的容器的创建方法。
存储器202包括易失性存储器或非易失性存储器,或可包括易失性和非易失性存储器两者。
其中,易失性存储器可以是只读存储器(read-only memory,ROM)、可编程只读存储器(programmable ROM,PROM)、可擦除可编程只读存储器(erasable PROM,EPROM)、电可擦除可编程只读存储器(electrically EPROM,EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory,RAM),其用作外部高速缓存。通过示例性但不是限制性说明,许多形式的RAM可用,例如静态随机存取存储器(static RAM,SRAM)、DRAM、同步动态随机存取存储器(synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(double data date SDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM,SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM,DR RAM)。
非易失性存储器可以是储存级内存(storage class memory,SCM)、固态硬盘(solid state drive,SSD)、机械硬盘(hard disk drive,HDD)等。其中,储存级内存例如可以是非易失
性内存(non-volatile memory,NVM)、相变化内存(phase-change memory,PCM)、持久化内存等。
在一种可能的实现方式中,存储器202独立于处理器201存在。存储器202通过总线204与处理器201相连接,用于存储数据、指令或者程序代码。处理器201调用并执行存储器202中存储的指令或程序代码时,能够实现本申请实施例提供的容器的创建方法。
在另一种可能的实现方式中,存储器202和处理器201集成在一起。
网络接口203,用于第一节点20与其他设备(如图1中所示容器集群100中的存储节点等)通过通信网络连接,所述通信网络可以是以太网,无线接入网(radio access network,RAN),无线局域网(wireless local area networks,WLAN)等。网络接口203包括用于接收数据/报文的接收单元,以及用于发送数据/报文的发送单元。
在本申请实施例中,网络接口203可以是支持RDMA的网络接口卡(network interface card,NIC),支持RDMA的NIC可以简写为RNIC。
其中,RDMA是一种用于解决网络传输中数据处理延迟的传输层网络协议。RDMA能够减少数据传输过程中在用于传输数据的主机设备内部对数据的复制操作,并且无需用于传输数据的主机设备的操作***介入,因此RDMA能够降低用于传输数据的主机设备的CPU负载。
总线204,可以是工业标准体系结构(industry standard architecture,ISA)总线、外部设备互连(peripheral component interconnect,PCI)总线、高速串行计算机扩展总线(peripheral component interconnect express,PCIe)、计算快速链路(compute express link,CXL)或扩展工业标准体系结构(extended industry standard architecture,EISA)总线等。该总线可以分为地址总线、数据总线、控制总线等。为便于表示,图2中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。
在一些实施例中,可选的,上述的第一节点20还可以配置有独立的DPU 205。其中,DPU 205通过总线204与处理器201、存储器202、网络接口203通信。
DPU 205包括存储模块和RNIC驱动模块。可选的,DPU 205的硬件实现可以是能够实现存储功能和RNIC功能的ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等,本申请实施例对此不作具体限定。可以理解,在实际中,DPU 205可以呈现为一张具有存储功能和RNIC功能的电路板/卡。
其中,DPU 205的存储模块可以包括DRAM。
可选的,该存储模块中的物理存储空间可以作为第一节点20上所运行的虚拟机的缓存,并支持内存访问语义和直接访问(Direct Access,DAX)的访问机制。这样,DPU 205的存储模块可以在第一节点20上呈现为支持DAX的持久化内存设备。其中,DAX访问机制是一种支持用户态软件直接访问存储于持久化内存中数据的机制。这样,第一节点20在访问DPU 205的存储模块中的数据时,就如同访问自身的内存,而无需先执行将数据拷贝到操作***的页高速缓存(page cache)过程。
DPU 205的RNIC驱动模块,用于和存储节点(例如图1中的存储节点121或存储节点122)进行通信,并通过RDMA访问存储节点中用作共享内存的物理存储空间。
需要说明的是,当第一节点20配置了DPU 205时,上述的网络接口203可以支持RDMA,也可以不支持RDMA,本申请实施例对此不作限定。
需要指出的是,图2中示出的结构并不构成对第一节点20的限定,除图2所示部件之外,该第一节点20还可以包括比图2所示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
下面结合附图,对本申请实施例提供的容器的创建方法进行说明。
在第一种可能的实现方式中,参考图3,图3示出了本申请实施例提供的一种容器的创建方法的流程示意图。可选的,该方法应用于图1所示的容器集群100,并可以由具有图2所示硬件结构的第一节点执行。该方法可以包括以下步骤:
S101、第一节点获取用于指示创建目标容器的第一创建请求。
其中,目标容器为安全容器。安全容器的说明可以参考上文,这里不作赘述。
可选的,在具体实现时,第一创建请求中可以携带第一指示信息,第一指示信息用于指示第一节点创建支持动态迁移特性的目标容器。
可以理解,支持动态迁移特性的目标容器即为第一节点基于本申请实施例所述的方法创建的容器。其中,支持动态迁移特性是指在不中断业务的情况下完成正常运行的容器的迁移的特性。
具体的,第一创建请求还用于指示第一节点创建用于运行目标容器的轻量化虚拟机(下文中以该轻量化虚拟机是第一虚拟机为例进行说明),并指示第一节点在创建完成的第一虚拟机中运行目标容器。
第一创建请求中包括目标容器的身份标识号(identity document,ID)(或目标容器所属pod的ID)、目标容器的镜像ID以及包括运行目标容器所需资源的资源参数。该资源参数包括运行目标容器所需的CPU资源大小以及内存资源大小等。
其中,目标容器的镜像ID用于标识目标容器的镜像文件,第一节点可以基于目标容器的镜像ID访问该目标容器的镜像文件。
可选的,目标容器的镜像文件预存于第二存储节点。第二存储节点可以和上文所述的第一存储节点是同一个存储节点,也可以是不同的存储节点,对此不作限定。第二存储节点中预存有一个或多个容器的镜像文件,容器的镜像文件中存储有该容器的程序代码,以及用于存储该容器需要持久化存储的数据等,对此不作详述。可以理解,第二存储节点中预存的容器的程序代码可以是设计人员预先设计开发的容器的程序代码,包括不同程序代码的容器镜像文件可以通过不同的ID标识,对此不作限定。
可选的,第一创建请求中还可以包括第一虚拟机的ID。可以理解,第一虚拟机的ID也可以是第一节点根据第一创建请求中的目标容器ID或目标容器所属pod的ID,生成的用于运行目标容器的第一虚拟机的ID,对此不作限定。其中,目标容器ID(或目标容器所属pod的ID),与第一虚拟机的ID具有对应关系。
在一种可能的实现方式中,第一节点可以接收容器集群的控制节点(例如图1所示的控制节点10)发送的第一创建请求。
具体实现时,以图1所示的容器集群100是安装有K8S***的K8S集群为例,这种情况下,容器集群100中的控制节点10和n个工作节点均安装有K8S容器管理模块。其中,控制节点10的K8S容器管理模块可以基于用户输入指示生成第一创建请求。
示例性的,控制节点10中的K8S容器管理模块可以通过控制节点10的输入接口(例如键盘、鼠标、触摸屏等)接收到用户输入的指示,进而生成第一创建请求,其中,用户输入的指示例如包括目标容器的ID、目标容器的镜像ID等。可以理解,容器的ID例如可以是容器的名称、编号等,对此不作限定。镜像ID例如可以是容器镜像文件的名称、编号等,对此不作限定。
然后,控制节点10基于预定的选择策略或接收到的用户的指示,确定用于部署目标容器的节点为容器集群100中的第一节点,并向第一节点发送生成的第一创建请求。作为响应,
第一节点可以接收到该第一创建请求。其中,本申请实施例对预定的选择策略不作限定,该预定的选择策略例如可以是负载均衡策略等。
在另一种可能的实现方式中,第一节点可以基于用户输入的指示生成第一创建请求。示例性的,第一节点可以通过输入接口(例如键盘、鼠标、触摸屏等)接收到用户输入的指示,进而生成第一创建请求。本申请实施例对此不作具体限定。
S102、第一节点根据第一创建请求确定第一虚拟机的内存的物理资源。其中,第一虚拟机用于运行目标容器。
具体的,第一节点根据第一创建请求确定第一虚拟机的内存的物理资源的过程可以通过步骤S1021-S1022实现:
S1021、第一节点根据第一创建请求,确定第一虚拟机的内存大小。
由于目标容器为安全容器,且运行安全容器的虚拟机一般为仅用于运行安全容器的轻量化虚拟机。可以理解的是,若第一创建请求仅指示创建一个目标容器,则运行目标容器所需的资源,可以看作是创建第一虚拟机所需的资源。若第一创建请求指示创建包括目标容器在内的多个容器,则运行该多个容器所需的资源,可以看作是创建第一虚拟机所需的资源。这种情况下,第一创建请求中包括运行该多个容器所需资源的资源参数,运行该多个容器所需的资源包括运行目标容器所需的资源。
以第一创建请求仅指示创建一个目标容器为例,第一节点可以根据第一创建请求中包括的运行目标容器所需资源的资源参数,确定创建第一虚拟机所需资源的资源参数。例如,第一节点根据第一创建请求中包括的运行目标容器所需资源的资源参数,确定创建第一虚拟机所需的CPU资源大小和内存大小等。
应理解,第一虚拟机的内存可以理解为第一虚拟机的物理内存,第一虚拟机的物理内存实际为虚拟机进程的一段虚拟内存。
S1022、第一节点根据第一虚拟机的内存大小,向第一存储节点申请第一虚拟机的内存的物理资源。
具体的,第一节点可以根据第一虚拟机的内存大小,向第一存储节点申请一段物理存储空间,以作为第一虚拟机的内存的物理资源。
可选的,第一节点可以根据第一虚拟机的内存大小,生成第一内存请求消息,并向第一存储节点发送第一内存请求消息。
例如,第一节点可以通过自身的网络接口(例如图2所示的网络接口203),向第一存储节点发送第一内存请求消息。其中,第一内存请求消息用于指示第一存储节点为第一虚拟机分配一段物理存储空间。第一内存请求消息中携带第一虚拟机的内存大小,第一内存请求消息中还可以携带有目标容器的ID(或目标容器所属pod的ID)、和/或第一虚拟机的ID。这里,第一虚拟机的ID的说明可以参考上文,不再赘述。第一存储节点可以是第一节点所在容器集群中的存储节点(例如图1所示的存储节点121),也可以是该容器集群能够访问的独立存储***中的存储节点,对此不作限定。
响应于接收到的第一内存请求消息,第一存储节点为第一虚拟机分配一段物理存储空间,并将该“一段物理存储空间”的地址(物理地址或逻辑地址)返回第一节点。
为便于描述,在本申请实施例中,将第一存储节点为第一虚拟机分配的该“一段物理存储空间”称为第一物理存储空间。可选的,第一物理存储空间可以是由多个持久化内存介质(如XL-Flash等)组成的内存单元。由于持久化内存介质的存储性能介于传统DRAM与闪存存储器之间,因此第一物理存储空间相比闪存存储器拥有更高的访问速度、更低的访问延迟,
而相比传统DRAM拥有更大的存储容量,且数据断电不丢失,可以持久化存储,成本也相对较低。
可选的,第一存储节点为第一虚拟机分配的第一物理存储空间的大小,可以等于第一虚拟机的内存大小,也可以大于第一虚拟机的内存大小,对此不作限定。
可选的,第一存储节点可以随机的将自身内存中当前空闲、且大于或等于第一虚拟机内存大小的一段物理存储空间,作为第一物理存储空间分配给第一虚拟机。或者,第一存储节点可以基于预设规则将自身内存中当前空闲、且大于或等于第一虚拟机内存大小的一段物理存储空间,作为第一物理存储空间分配给第一虚拟机。本申请实施例对该预设规则不作限定。
可选的,为便于管理本地的存储空间,第一存储节点在为第一虚拟机分配第一物理存储空间后,还在上述第一内存请求消息中携带有第一虚拟机的ID时,建立第一物理存储空间的地址和第一虚拟机ID之间的对应关系。或者,在第一内存请求消息中携带有目标容器的ID(或目标容器所属pod的ID),建立第一物理存储空间的地址和目标容器的ID(或目标容器所属pod的ID)之间的对应关系。或者,在第一内存请求消息中携带有目标容器的ID(或目标容器所属pod的ID)和第一虚拟机的ID,建立第一物理存储空间的地址、目标容器的ID(或目标容器所属pod的ID)、以及第一虚拟机ID之间的对应关系。
响应于第一存储节点向第一节点返回的第一物理存储空间的地址,第一节点即获取到第一物理存储空间的地址,从而完成了向第一存储节点申请第一虚拟机的内存的物理资源的过程。
需要说明,第一节点可以通过RDMA操作访问第一存储节点的第一物理存储空间。
在一些示例中,第一节点可以根据第一存储节点返回的第一物理存储空间的地址,注册一个新的DAX存储设备。
其中,该DAX存储设备是虚拟的存储设备,该DAX存储设备在第一节点上并无实际的物理存储空间,该DAX存储设备用于指示第一存储节点返回的第一物理存储空间的地址对应的存储空间。其中,DAX的相关说明可以参考上文,这里不作赘述。
在一些实施例中,上述的第一物理存储空间除作为第一虚拟机的内存的物理资源外,第一物理存储空间中的一部分存储空间,还可以用于存储第一虚拟机的CPU状态信息。
这种情况下,第一物理存储空间的大小至少等于:第一虚拟机的内存大小和第一虚拟机的CPU状态信息的大小之和。其中,CPU状态信息包括寄存器信息(例如寄存器的标识和当前值)、中断信息等,对此不作限定。这种情况下,第一物理存储空间包括第一空间和第二空间。其中,第一空间用于存储第一虚拟机的内存数据,第二空间用于存储第一虚拟机的CPU状态信息。
S103、第一节点基于第一创建请求和确定的物理资源,创建第一虚拟机。
具体的,第一节点可以基于第一创建请求,先创建用于创建第一虚拟机的虚拟机进程。可以理解,该虚拟机进程用于创建(或管理/监控)第一虚拟机,即第一虚拟机相当于是该虚拟机进程中的一个或多个线程。
作为示例,以图1所示的容器集群100是安装有K8S***的K8S集群为例,则容器集群100中的第一节点在获取第一创建请求后,第一节点中的K8S容器管理模块则将第一创建请求发送至第一节点的虚拟机监控器(virtual machine monito,VMM),并由VMM指示第一节点创建用于创建第一虚拟机的虚拟机进程。
这样,基于上述确定出的创建第一虚拟机所需资源的资源参数,第一节点可以通过上述创建的虚拟机进程创建第一虚拟机。
具体的,基于第一虚拟机的内存大小,第一节点可以先通过虚拟机进程编址一段虚拟内存,以作为第一虚拟机的物理内存(下文中称为第一物理内存)。然后,第一节点通过虚拟机进程建立第一物理内存和第一物理存储空间之间的映射关系,以实现将第一物理存储空间作为第一虚拟机的内存的物理资源。
第一节点可以通过虚拟机进程建立第一物理内存和该虚拟机进程的第一虚拟内存之间的映射关系。其中,第一虚拟内存是该虚拟机进程的虚拟内存中的一段内存,且第一虚拟内存的地址与第一物理内存的地址一一对应。第一节点还通过虚拟机进程建立第一虚拟内存和第一存储节点返回的第一物理存储空间之间的映射关系,从而实现将第一物理存储空间作为第一虚拟机的内存的物理资源的目的。
其中,第一虚拟内存(或第一物理内存)的大小,小于或等于第一物理存储空间大小,且第一物理存储空间中包括与第一虚拟内存(或第一物理内存)的地址一一对应的地址。换句话说,第一物理存储空间中的一段或全部存储空间的地址与第一虚拟内存(或第一物理内存)的地址一一对应。
作为示例,当第一节点通过虚拟机进程建立第一虚拟内存和第一物理存储空间之间的映射关系时,第一节点可以打开基于第一物理存储空间的地址在第一节点注册的DAX存储设备,并通过虚拟机进程调用mmap接口来建立第一虚拟内存和第一物理存储空间之间的映射关系。
另外,第一节点还基于确定出的第一虚拟机所需的CPU资源大小,并通过虚拟机进程为第一虚拟机申请第一节点的CPU资源,这里不作详述。
这样,在确定第一虚拟机的内存的物理资源和第一虚拟机的CPU资源后,第一节点可以通过虚拟机进程创建第一虚拟机。例如,第一节点可以在本地预存的虚拟机镜像文件中读取虚拟机的操作***代码,并通过虚拟机进程在确定的资源中运行该操作***代码,以完成第一虚拟机的创建。其中,虚拟机镜像文件中的操作***代码为虚拟机的操作***代码。
需要说明,在本申请实施例中,容器集群中的工作节点预存有虚拟机镜像文件。虚拟机镜像文件中包括虚拟机的操作***代码,并且,虚拟机镜像文件可以用于存储虚拟机运行过程中需要持久化存储的数据。可以理解,虚拟机镜像文件可以看作是虚拟机的硬盘。
还需要说明,在第一虚拟机创建后,第一节点可以通过虚拟机进程将第一虚拟机的CPU状态信息写入第一物理存储空间中的第二空间。其中,第二空间和CPU状态信息的说明可以参考S1022中的相关描述,这里不作赘述。
具体的,第一节点可以通过虚拟机进程先将第一虚拟机的CPU状态信息写入第一虚拟内存。由于第一虚拟内存对应的实际物理存储空间为第一节点的第二物理内存(即第一节点的一段本地物理内存),因此,第一节点通过虚拟机进程实际是将CPU状态信息写入第二物理内存,并进一步将该CPU状态信息写入第一节点的RNIC的内存中。这样,RNIC即可将该CPU状态信息通过RDMA操作(例如写(write)操作/发送(send)操作)写入第一存储节点的第一物理空间中的第二空间中,从而实现对第一虚拟机的CPU状态信息的记录。
还需说明,在第一虚拟机运行的过程中,当第一节点通过虚拟机进程监控到第一虚拟机的CPU状态发生变化,则通过上述描述的过程将第一虚拟机当前的CPU状态信息写入第一存储节点的第一物理空间中的第二空间中,以更新第二空间中记录的CPU状态信息,从而保证第一虚拟机的最新CPU状态信息被记录于第二空间中。
S104、第一节点在第一虚拟机中创建目标容器。
第一节点可以先基于第一创建请求中的镜像ID,访问第二存储节点中的该镜像ID所标识的镜像文件,并读取该镜像文件中的程序代码。进而,第一节点在第一虚拟机中运行该程
序代码,从而完成目标容器的创建。
然后,第一节点通过虚拟机进程将第一虚拟机的状态设置为运行状态。
应理解,第一节点在基于镜像ID访问第二存储节点中的该镜像ID所标识的镜像文件时,还向第二存储节点发送目标容器的ID。
进而,第二存储节点可以建立目标容器ID和上述镜像ID之间的对应关系。这样,当将目标容器迁移至其他节点时,该其他节点即可通过目标容器的ID,在第二存储节点中查找到与目标容器ID对应的镜像文件,并读取将该镜像文件中的程序代码,以创建目标容器,从而实现目标容器的迁移。
至此,目标容器可以在第一节点中的第一虚拟机上正常运行。
下面参考图4,对基于图3所述方法创建的目标容器在处理业务时的数据读写流程进行说明。
如图4所示,第一节点创建并运行有虚拟机进程,虚拟机进程创建第一虚拟机,即第一虚拟机作为虚拟机进程的一个或多个线程运行于虚拟机进程中。进一步的,目标容器运行于第一虚拟机中,目标容器中运行有应用进程1、应用进程2以及应用进程3。
第一种情况:用户设备访问目标容器中的应用进程1,并通过应用进程1向第一虚拟机写入数据1。
这种情况下,第一节点通过应用进程1首先将数据1写入第一物理内存(即第一虚拟机的物理内存)。由于第一物理内存与虚拟机进程的第一虚拟内存对应,即第一节点首先将数据1写入与第一物理内存对应第一虚拟内存。
当第一节点向第一虚拟内存写入数据时,第一节点会为第一虚拟内存分配一段实际的物理内存(即第一节点的一段本地物理内存,如图4所示的第二物理内存)。因此,第一节点通过应用进程1将数据1写入第一虚拟内存,实际为将数据1写入第二物理内存。
基于第一虚拟内存和第一物理存储空间之间的映射关系,第一节点可以进一步将第二物理内存中的数据1写入第一节点的RNIC的内存中。这样,RNIC即可将数据1通过RDMA操作(例如write操作/send操作)写入第一存储节点的第一物理存储空间中的第一空间,从而实现第一物理存储空间中的数据和第一物理内存中的数据同步。其中,第一空间的说明可以参考上文描述。
其中,可选的,第一节点的RNIC可以实时的将写入RNIC内存的数据1通过RDMA操作写入第一空间。
或者,RNIC也可以周期性的将写入RNIC内存的数据通过RDMA操作写入第一空间。当然,RNIC还可以在写入RNIC内存的数据大小超过阈值时,通过RDMA操作将这些数据写入第一空间,本申请实施例对此不作限定。
第二种情况:用户设备访问目标容器中的应用进程1,并通过应用进程1读取第一虚拟机的数据2。
这种情况下,第一节点中的第一虚拟机通过应用进程1查询与第一物理内存对应的第二物理内存(即第一节点的一段本地物理内存)。当第一节点确定第二物理内存中不存在数据2,则第一节点的操作***触发缺页异常事件。
然后,第一节点指示RNIC生成用于读取数据2的RDMA读请求,以从第一存储节点的第一物理存储空间中的第一空间读取到数据2,并进一步将数据2返回应用进程1。
在一些实施例中,当第一节点所在的容器集群中的控制节点(例如图1所示容器集群100中的控制节点10)监控到上述目标容器需要迁移时,控制节点向该容器集群中除第一节点之
外的另一个节点发送容器迁移请求,以指示将该目标容器迁移至该另一个节点。
下面对目标容器的迁移过程进行说明。参考图5,图5示出了本申请实施例提供的另一种容器的创建方法的流程示意图。可选的,该方法可以应用于图1所示的容器集群,该方法包括以下步骤:
S201、控制节点在容器集群中确定第二节点。
具体的,控制节点在确定第一节点通过图3所示方法在上创建的上述目标容器需要迁移时,则在容器集群中确定用于迁移该目标容器的第二节点。其中,第二节点是该容器集群中除第一节点之外的任一个工作节点。例如,假设第一节点是图1所示容器集群100中的工作节点111,则第二节点可以是容器集群100中的工作节点112、工作节点113或工作节点11n中的任一个,对此不作限定。
作为示例,当控制节点监控到第一节点中的目标容器所需的资源即将耗尽,或者监控到第一节点中运行该目标容器的上述第一虚拟机的资源即将耗尽,或者监控到第一节点的资源即将耗尽,或者监控到第一节点(或第一虚拟机,亦或目标容器)故障,或者监控到第一节点(或第一虚拟机,亦或目标容器)因其他原因暂停、又或者控制节点接收到用户迁移目标容器的指示,等等,控制节点在容器集群中确定第二节点。
可选的,控制节点可以随机的在容器集群中确定第二节点。或者,控制节点可以基于预设策略在容器集群中确定第二节点,该预设策略例如是负载均衡策略,对此不作限定。又或者,控制节点基于获取的用户指示中携带的第二节点的ID,在容器集群中确定第二节点。
S202、控制节点向第二节点发送容器迁移请求,以指示第二节点创建用于运行目标容器的第二虚拟机。
其中,运行于第一节点中第一虚拟机的目标容器即为待迁移容器。
具体的,容器迁移请求用于指示第二节点创建用于运行目标容器的轻量化虚拟机(下文中以该轻量化虚拟机是第二虚拟机为例进行说明),并指示第二节点在创建完成的第二虚拟机中运行目标容器。容器迁移请求中可以包括目标容器的ID(或目标容器所属pod的ID)、运行目标容器所需资源的资源参数。其中,运行目标容器所需资源的资源参数包括运行该目标容器所需的CPU资源大小以及内存资源大小等。
可选的,容器迁移请求中还可以包括第二虚拟机的ID和第一虚拟机的ID。可以理解的是,第二虚拟机的ID也可以是第二节点根据容器迁移请求中的目标容器ID或目标容器所属pod的ID,生成的用于运行目标容器的第二虚拟机的ID,对此不作限定。
需要说明,第二虚拟机的ID和第一虚拟机的ID可以相同,也可以不同,对比不作限定。可以理解,当第二虚拟机的ID和第二虚拟机的ID相同时,容器迁移请求中可以只包括一个虚拟机的ID即可。
S203、第二节点根据接收到的容器迁移请求,获取第一物理存储空间的地址。
需要说明,为实现在不中断业务的情况下实现目标容器的迁移,第二节点可以根据容器迁移请求中的第一虚拟机的ID和/或目标容器的ID(或目标容器所属pod的ID),获取上文所述的第一物理存储空间(即第一虚拟机的内存的物理资源)的地址,以便将第一物理存储空间确定为第二虚拟机的内存的物理资源。这样,第一虚拟机的内存数据即可作为第二虚拟机的内存数据。其中,第一物理存储空间的详细说明可以参考上文,这里不予赘述。
具体的,第二节点可以在接收到容器迁移请求后,根据容器迁移请求生成内存请求消息(例如第二内存请求消息),并向第一存储节点发送第二内存请求消息。其中,第二内存请求消息用于请求获取第一存储节点为第一虚拟机分配的第一物理存储空间的地址。作为示例,
第二节点可以通过自身的网络接口向第一存储节点发送第二内存请求消息。
在一种可能的实现方式中,当第一节点在为第一虚拟机申请第一物理存储空间时,向第一存储节点发送的第一内存请求消息中包括第一虚拟机的ID,即表明第一存储节点中记录有第一物理存储空间的地址和第一虚拟机的ID之间的对应关系。
这样的话,第二内存请求消息中包括第一虚拟机的ID。进而,作为接收到第二内存请求消息的响应,第一存储节点根据第一虚拟机的ID在记录的对应信息中,查找并确定与第一虚拟机的ID具有对应关系的第一物理存储空间的地址,并将该地址返回第二节点。这样,第二节点即获取到第一物理存储空间的地址。
在另一种可能的实现方式中,当第一节点在为第一虚拟机申请第一物理存储空间时,向第一存储节点发送的第一内存请求消息中包括目标容器的ID(或目标容器所属pod的ID),即表明第一存储节点中记录有第一物理存储空间的地址和目标容器的ID(或目标容器所属pod的ID)之间的对应关系。
这样的话,第二内存请求消息中包括目标容器的ID(或目标容器所属pod的ID)。进而,作为接收到第二内存请求消息的响应,第一存储节点根据目标容器的ID(或目标容器所属pod的ID)在记录的对应信息中,查找并确定与目标容器的ID(或目标容器所属pod的ID)具有对应关系的第一物理存储空间的地址,并将该地址返回第二节点。这样,第二节点即获取到第一物理存储空间的地址。
在又一种可能的实现方式中,当第一节点在为第一虚拟机申请第一物理存储空间时,向第一存储节点发送的第一内存请求消息中包括目标容器的ID(或目标容器所属pod的ID)和第一虚拟机的ID,即表明第一存储节点中记录有第一物理存储空间的地址、目标容器的ID(或目标容器所属pod的ID)、以及第一虚拟机的ID之间的对应关系。
这样的话,第二内存请求消息中包括目标容器的ID(或目标容器所属pod的ID)或第一虚拟机的ID中的至少一个ID。进而,作为接收到第二内存请求消息的响应,第一存储节点根据该至少一个ID在记录的对应信息中查找并确定与该至少一个ID具有对应关系的第一物理存储空间的地址,并将该地址返回第二节点。这样,第二节点即获取到第一物理存储空间的地址。
在一些示例中,第二节点可以根据获取的第一物理存储空间的地址,注册一个新的DAX存储设备。第二节点根据获取的第一物理存储空间的地址,注册一个新的DAX存储设备的说明,可以参考上文第一节点根据第一存储节点返回的第一物理存储空间的地址,注册一个新的DAX存储设备的描述,不再赘述。
S204、第二节点根据第一物理存储空间的地址,创建第二虚拟机。
具体的,第二节点可以根据容器迁移请求中运行目标容器的资源参数,确定创建第二虚拟机所需资源的资源参数,例如确定创建第二虚拟机所需的CPU资源大小和内存大小。
进一步的,第二节点根据容器迁移请求,先创建用于创建第二虚拟机的虚拟机进程,并通过虚拟机进程建立第二虚拟机的物理内存和第一物理存储空间之间的映射关系,从而实现第一物理存储空间作为第二虚拟机的内存的物理资源的目的。其中,第二节点通过虚拟机进程建立第二虚拟机的物理内存和第一物理存储空间之间的映射关系的详细说明,可以参考S103中第一节点通过虚拟机进程建立第一物理内存(即第一虚拟机的物理内存)和第一物理存储空间之间的映射关系的描述,不再赘述。
另外,基于确定出的创建第二虚拟机的CPU资源大小,第二节点还通过虚拟机进程为第二虚拟机申请第二节点的CPU资源,这里不作详述。需要说明,第二节点还可以利用虚拟机
进程,并通过RDMA操作(例如读操作)访问第一物理存储空间,以读取第一物理存储空间的第二空间中记录的CPU状态信息,并基于读取到的CPU状态信息设置第二虚拟机所申请的CPU资源的状态。其中,第二空间和CPU状态信息的说明可以参考S1022中的相关描述,不再赘述。
这样,在确定第二虚拟机的内存的物理资源和CPU资源后,第二节点通过虚拟机进程创建第二虚拟机。例如,第二节点可以在本地预存的虚拟机镜像文件中读取虚拟机的操作***代码,并通过虚拟机进程在确定的资源中运行该操作***代码,以完成第二虚拟机的创建。其中,虚拟机镜像文件的相关说明可以参考上文描述,不再赘述。
此外,第二节点在通过虚拟机进程创建第二虚拟机后,在第二虚拟机运行的过程中,当第二节点通过虚拟机进程监控到第二虚拟机的CPU状态发生变化,则将第二虚拟机当前的CPU状态信息写入第一存储节点的第一物理空间中的第二空间中,以更新第二空间中记录的CPU状态信息,从而保证第二虚拟机的最新CPU状态信息被记录于第二空间中。其中,第二节点将第二虚拟机的CPU状态信息写入第一物理存储空间中的第二空间的说明,可以参考上文中第一节点将第一虚拟机当前的CPU状态信息写入第一存储节点的第一物理空间中的第二空间中的描述,不作赘述。
S205、第二节点在第二虚拟机中创建目标容器。
第二节点可以先基于上述容器迁移请求中的目标容器ID、以及第二存储节点中预存的目标容器ID和镜像ID之间的对应关系,在第二存储节点中的查找到目标容器的镜像文件,并读取该镜像文件中的程序代码。进而,第二节点在第二虚拟机中运行该程序代码,从而完成目标容器的创建。
其中,第二存储节点中预存的目标容器ID和镜像ID之间的对应关系,可以参考上文S104中的相关描述,不再赘述。
可以理解的是,控制节点在指示第二节点迁移目标容器时,还可以向运行迁移前的目标容器的第一节点发送迁移目标容器的指示。
这样的话,在一种可能的情况中,第二节点在创建第二虚拟机、以及在第二虚拟机中创建目标容器后,将第二虚拟机的状态设置为运行状态。例如,第二节点可以通过虚拟机进程将第二虚拟机的状态设置为运行状态。
这种情况下,可选的,控制节点监控到第二节点创建的目标容器已经开始运行,则向第一节点发送删除目标容器的指示。作为响应,第一节点删除本地的目标容器。或者,第一节点可以在接收到控制节点发送的迁移目标容器的指示后的第一预设时长到来时,删除本地的目标容器。其中,第一预设时长用于保证第一节点将第一虚拟机的本地内存数据全部写入第一物理存储空间。
结合上文可知,用于运行迁移后的目标容器的第二虚拟机和用于运行迁移前的目标容器的第一虚拟机所需资源的资源参数相同(均基于运行目标容器所需资源的资源参数确定),第二虚拟机的CPU状态和第一虚拟机的CPU状态相同,因此,第二虚拟机和第一虚拟机的资源配置和CPU状态均相同。并且,第二虚拟机的内存的物理资源和第一虚拟机的内存的物理资源相同,即第二虚拟机与第一虚拟机共享内存的物理资源,也即第一物理存储空间作为用于运行迁移前后的安全容器的虚拟机的共享内存。这样的话,当第二节点将第二虚拟机的状态设置为运行状态时,第一虚拟机在第一节点的本地内存中的内存数据已全部通过RDMA操作写入第一物理存储空间,则表示第二虚拟机和第一虚拟机的内存数据、进程状态也都是相同的。这样,基于本申请方法迁移目标容器时没有造成内存数据丢失,从而不会中断用户业务,且
用户无感知。
此外,第二虚拟机中运行的目标容器的镜像文件和第一虚拟机中运行的目标容器的镜像文件相同,即迁移后运行于第二虚拟机的目标容器和迁移前运行于第一虚拟机的目标容器的文件数据也是相同的。可见基于本申请实施例提供的方法迁移目标容器时也不会造成目标容器的文件数据丢失。
在另一种可能的情况中,当第二节点在完成创建第二虚拟机、以及完成在第二虚拟机中创建目标容器,并且已接收到第一节点发送的开启目标容器的指示消息,则将第二虚拟机的状态设置为运行状态。当第二节点在完成创建第二虚拟机、以及完成在第二虚拟机中创建目标容器后,还未接收到第一节点发送的开启目标容器的指示消息,则先将第二虚拟机的状态设置为暂停状态,并在接收到第一节点发送的开启目标容器的指示消息时,将第二虚拟机的状态设置为运行状态,以开启目标容器。作为示例,第二节点可以通过虚拟机进程将第二虚拟机的状态设置为暂停状态或运行状态。
这种情况下,第一节点在接收到控制节点发送的迁移目标容器的指示后,作为响应,第一节点查询并确定与第一物理内存(即第一虚拟机的物理内存)对应的第二物理内存(即第一节点的一段本地物理内存)中的内存数据是否已全部写入第一物理存储空间。应理解,第二物理内存中的内存数据为第一虚拟机的内存数据。其中,第一物理内存和第二物理内存的对应关系的说明可以参考上文描述,不再赘述。
如果第一节点确定第二物理内存中的内存数据已全部写入第一物理存储空间,则删除本地的目标容器,并向第二节点发送开启目标容器的指示消息,以指示第二节点在完成创建完成第二虚拟机、并完成在第二虚拟机中创建目标容器后,将第二虚拟机的状态设置为运行状态。
如果第一节点确定第二物理内存中的内存数据还未全部写入第一物理存储空间,则计算第二物理内存中还未写入第一物理存储空间的剩余内存数据的大小,并基于预设算法确定将该剩余内存数据全部写入第一物理存储空间需要的预估时长。其中,本申请实施例对该预设算法不作具体限定,例如该预设算法可以是基于当前网络速率和剩余内存数据大小的任意算法。
进一步的,第一节点确定预估时长是否小于第二预设时长。如果第一节点确定预估时长小于或等于第二预设时长,则暂停运行本地的目标容器,并在距离当前时刻的预估时长后,第一节点再次确定第二物理内存中的内存数据是否已全部写入第一物理存储空间。其中,本申请实施例对第二预设时长的具体取值不作限定。
如果在距离当前时刻的预估时长后,第一节点确定第二物理内存中的内存数据已全部写入第一物理存储空间,则删除本地的目标容器,并向第二节点发送开启目标容器的指示消息,以指示第二节点在完成创建完成第二虚拟机、并完成在第二虚拟机中创建目标容器后,将第二虚拟机的状态设置为运行状态。
如果在距离当前时刻的预估时长后,第一节点确定第二物理内存中的内存数据还未全部写入第一物理存储空间,则在距离上述当前时刻的第二预设时长后,再次确定第二物理内存中的内存数据是否已全部写入第一物理存储空间。若在距离上述当前时刻的第二预设时长后,第一节点确定第二物理内存中的内存数据已全部写入第一物理存储空间,则删除本地的目标容器,并向第二节点发送开启目标容器的指示消息,以指示第二节点在完成创建完成第二虚拟机、并完成在第二虚拟机中创建目标容器后,将第二虚拟机的状态设置为运行状态。
这样,用于运行迁移后的目标容器的第二虚拟机和用于运行迁移前的目标容器的第一虚
拟机的内存数据和进程状态均相同,且迁移后运行于第二虚拟机的目标容器和迁移前运行于第一虚拟机的目标容器的文件数据也是相同的。结合上文可知,用于运行迁移后的目标容器的第二虚拟机和用于运行迁移前的目标容器的第一虚拟机的资源配置、CPU状态相同。因此,基于本申请方法迁移目标容器时没有造成内存数据丢失,从而不会中断用户业务,且用户无感知。
S206、第二节点更新目标容器的网络路由信息。
可选的,第二节点可以在将第二虚拟机的状态设置为运行状态后,对自身存储的路由信息中用于指示访问目标容器的网络路由信息进行更新。例如,第二节点将原本的访问目标容器的网络路由信息中容器所属节点的地址由第一节点的地址修改为第二节点的地址。
可选的,第二节点可以向容器集群内除第二节点外的部分或全部工作节点发送路由更新消息,以指示接收到该路由更新消息的工作节点更新自身存储的路由信息中指示访问目标容器的网络路由信息。
可选的,第二节点也可以在接收到第一节点或控制节点发送的路由更新消息后,更新自身存储的路由信息中用于指示访问目标容器的网络路由信息。
可以理解,当控制节点在监控到目标容器已迁移至第二节点(例如控制节点监控到第二节点中的第二虚拟机开始正常运行),则向容器集群内的部分或全部工作节点发送路由更新消息,以指示接收到该路由更新消息的工作节点更新自身存储的路由信息中指示访问目标容器的网络路由信息。或者,第一节点在删除本地的目标容器后,更新自身存储的路由信息中用于指示访问目标容器的网络路由信息,以及,向容器集群内除第一节点外的部分或全部工作节点发送路由更新消息,以指示接收到该路由更新消息的工作节点更新自身存储的路由信息中指示访问目标容器的网络路由信息。本申请实施例对此不作限定。
至此,当通过第一种可能的实现方式中所述的方法在第一节点上创建目标容器后,在将该目标容器迁移至容器集群中的另一节点(例如第二节点)时,只需将用于运行迁移前的目标容器的第一节点的第一虚拟机的内存的物理资源(即第一存储节点上的第一物理存储空间),确定为用于运行迁移后的目标容器的第二节点的第二虚拟机的内存的物理资源。也就是说,第一存储节点上的第一物理存储空间不仅作为迁移前运行于第一虚拟机的目标容器的内存,还作为迁移后运行于第二虚拟机的目标容器的内存。也即,迁移前运行于第一虚拟机的目标容器和迁移后运行于第二虚拟机的目标容器共享内存,具体如图6所示。并且,迁移前运行于第一虚拟机的目标容器和迁移后运行于第二虚拟机的目标容器的镜像文件也是相同的。因此,通过第一种可能的实现方式中所述的方法能够实现在迁移容器时不丢失内存数据、文件数据,不中断用户业务、且用户无感知。
在第二种可能的实现方式中,参考图7,图7示出了本申请实施例提供的又一种容器的创建方法的流程示意图。可选的,该方法应用于图1所示的容器集群100,并可以由具有图2所示硬件结构的第一节点执行。在该方法中,第一节点可以先执行实施例一中的S101和S1021,接着,第一节点可以执行步骤S303-S305。
S303、第一节点根据第一虚拟机的内存大小,将第一节点中DPU的第二物理存储空间确定为第一虚拟机的缓存的物理资源。
其中,第一节点中配置有DPU,DPU的说明可以参考上文描述。第二物理存储空间为DPU的存储模块中的一段内存空间,第二物理存储空间可以是普通的内存空间,例如DRAM构成的内存单元。可以看出,第二物理存储空间也是第一节点本地的一段存储空间。
具体的,第一节点在基于第一创建请求中运行目标容器所需资源的资源参数确定出第一
虚拟机的内存大小后,基于第一虚拟机的内存大小,将DPU中的一段物理存储空间(即第二物理存储空间)确定为第一虚拟机的缓存的物理资源。其中,本申请实施例对第二物理存储空间的大小不作限定,例如,第二物理存储空间的大小可以大于或等于第一虚拟机的内存大小,也可以小于或等于第一虚拟机的内存大小。
S304、第一节点中的DPU根据第一虚拟机的内存大小,向第一存储节点申请第一虚拟机的内存的物理资源。
具体的,DPU可以根据第一虚拟机的内存大小,向第一存储节点申请一段物理存储空间(即第一物理存储空间),以作为第一虚拟机的内存的物理资源。
其中,第一节点中的DPU根据第一虚拟机的内存大小,向第一存储节点申请第一虚拟机的内存的物理资源的详细说明,可以参考S1022第一节点根据第一虚拟机的内存大小,向第一存储节点申请第一虚拟机的内存的物理资源的过程描述,不再赘述。
可以理解的是,S1022和S303的区别在于执行主体不同,S1022的执行主体为第一节点,例如S1022可以由第一节点的CPU和网卡配合执行,而S303的执行主体为第一节点中的独立单元DPU。DPU的详细说明可以参考上文对图2所示第一节点中的DPU的相关描述,这里不再赘述。
需要说明,当第一节点的DPU接收第一存储节点返回的第一物理存储空间的地址后,存储第一物理存储空间的地址。
还需说明,第一物理存储空间的大小可以大于或等于DPU的第二物理存储空间的大小,也可以小于或等于DPU的第二物理存储空间的大小,对此不作限定。
S305、第一节点基于第一创建请求和确定的物理资源,创建第一虚拟机。
具体的,第一节点可以基于第一创建请求,创建用于创建第一虚拟机的虚拟机进程。其中第一节点创建虚拟机进程的相关说明,可以参考上文S103的相关描述,不再赘述。
进一步的,第一节点可以通过虚拟机进程建立第一物理内存和第二物理存储空间之间的映射关系,以及通过DPU建立第二物理存储空间和第一物理存储空间之间的映射关系,从而实现将第一物理存储空间作为第一虚拟机的内存的物理资源,将第二物理存储空间作为第一虚拟机的缓存的物理资源。其中,第一物理内存、第一物理存储空间以及第二物理存储空间的详细说明可以参考上文,不再赘述。
具体的,第一节点可以通过虚拟机进程建立第一物理内存和该虚拟机进程的第一虚拟内存之间的映射关系。第一节点还通过虚拟机进程建立第一虚拟内存和DPU的一段虚拟空间(以下称为DPU虚拟空间)之间的映射关系,这里,第二物理存储空间即作为DPU虚拟空间的物理存储空间,从而实现了建立第一物理内存和第二物理存储空间之间的映射关系的目的。进而,第一节点中的DPU建立第二物理存储空间和第一物理存储空间之间的映射关系。从而,第一物理存储空间即作为第一虚拟机的内存的物理资源,而第二物理存储空间则作为第一虚拟机的缓存的物理资源。其中,第一虚拟内存是该虚拟机进程的虚拟内存中的一段内存,且第一虚拟内存的地址与第一物理内存的地址一一对应。DPU虚拟空间中包括与第一虚拟内存的地址一一对应的地址,即第一虚拟内存的地址与DPU虚拟空间中的一段空间或全部空间的地址一一对应。第一物理存储空间中包括与DPU虚拟空间的地址一一对应的地址,即DPU虚拟空间的地址和第一物理存储空间中的一段空间或全部空间的地址一一对应。
作为示例,第一节点可以通过虚拟机进程调用mmap接口来建立第一虚拟内存和DPU虚拟空间之间的映射关系,而DPU可以在创建完成的第一虚拟机向DPU虚拟空间写入数据时,为其在第二物理存储空间中分配一段空闲的物理存储空间,以用于存储数据。
可以理解,第一节点的DPU接收并存储第一存储节点返回的第一物理存储空间的地址,即相当于建立了第二物理存储空间和第一物理存储空间之间的映射关系。
另外,第一节点还基于确定出的第一虚拟机所需的CPU资源大小,并通过虚拟机进程为第一虚拟机申请第一节点的CPU资源,不作详述。
这样,在确定第一虚拟机的内存的物理资源、缓存的物理资源以及第一虚拟机的CPU资源后,第一节点可以通过虚拟机进程创建第一虚拟机。例如,第一节点可以在本地预存的虚拟机镜像文件中读取虚拟机的操作***代码,并通过虚拟机进程在确定的资源中运行该操作***代码,以完成第一虚拟机的创建。其中,虚拟机镜像文件的相关说明可以参考上文描述,不再赘述。
需要说明,在第一虚拟机创建后,第一节点可以通过虚拟机进程将第一虚拟机的CPU状态信息经第二物理存储空间写入第一物理存储空间中的第二空间。其中,第二空间和CPU状态信息的说明可以参考S1022中的相关描述,这里不作赘述。
具体的,第一节点可以通过虚拟机进程先将第一虚拟机的CPU状态信息写入第一虚拟内存。由于第一虚拟内存对应的实际物理存储空间为第二物理内存(即第一节点的一段本地物理内存)。因此,第一节点通过虚拟机进程实际是将CPU状态信息写入第二物理内存,并进一步将该CPU状态信息写入第一节点的DPU中第二物理存储空间中。这样,DPU基于自身配置的RNIC驱动模块,将第二存储空间中的CPU状态信息通过RDMA操作(例如写操作/发送操作)写入第一存储节点的第一物理空间中的第二空间中,从而实现第一虚拟机的CPU状态信息的记录。
还需说明,在第一虚拟机运行的过程中,当第一节点通过虚拟机进程监控到第一虚拟机的CPU状态发生变化,则通过上述描述的过程将第一虚拟机当前的CPU状态信息经第二物理存储空间写入第一存储节点的第一物理空间中的第二空间中,以更新第二空间中记录的CPU状态信息,从而保证第一虚拟机的最新CPU状态信息被记录于第二空间中。
然后,第一节点执行S104,以完成目标容器的创建,并运行目标容器。
下面参考图8对基于图7所述方法创建的目标容器在处理业务时的数据读写进行说明。如图8所示,第一节点运行有虚拟机进程,虚拟机进程创建第一虚拟机,即第一虚拟机作为虚拟机进程的一个或多个线程运行于虚拟机进程中。进一步的,目标容器运行于第一虚拟机中,目标容器中运行有应用进程1、应用进程2以及应用进程3。
第一种情况:用户设备访问目标容器中的应用进程1,并通过应用进程1向第一虚拟机写入数据1。
这种情况下,第一节点通过应用进程1首先将数据1写入第一物理内存(即第一虚拟机的物理内存)。由于第一物理内存即为虚拟机进程的第一虚拟内存,即第一节点首先将数据1写入与第一物理内存对应第一虚拟内存。
当第一节点向第一虚拟内存写入数据时,第一节点会为第一虚拟内存分配一段实际的物理内存(即第一节点的一段本地物理内存,如图8所示的第二物理内存)。因此,第一节点通过应用进程1将数据1写入第一虚拟内存,实际为将数据1写入第二物理内存。
进一步,基于第一虚拟内存和DPU虚拟空间之间的映射关系,第一节点通过应用进程1将第二物理内存的数据1写入DPU虚拟空间,进而DPU可以在第二物理存储空间中为DPU虚拟空间分配一段地址空间以存储数据1。
然后,DPU基于自身配置的RNIC驱动模块,将第二物理存储空间中的数据1通过RDMA操作(例如write操作/send操作)写入第一存储节点的第一物理存储空间中的第一空间,
从而实现第一物理存储空间中的数据和第一物理内存中的数据同步。其中,第一空间的说明可以参考上文描述。
其中,可选的,DPU可以实时的将写入第二物理存储空间的数据1通过RDMA操作写入第一空间。
或者,DPU也可以周期性的将写入第二物理存储空间的数据通过RDMA操作写入第一空间。当然,DPU还可以在写入第二物理存储空间的数据大小超过阈值时,通过RDMA操作将这些数据写入第一空间,本申请实施例对此不作限定。
应理解,在上述的第一节点将数据1写入第一物理存储空间的过程中,对于第一节点中的第一虚拟机为将数据1写入第一物理存储空间而发起的进程而言,当数据1被写入DPU的第二物理存储空间后,第一节点中的第一虚拟机即可确定该进程的数据写入任务已完成。后续,DPU作为第一节点中的独立单元,会通过RDMA操作将写入第二物理存储空间的数据1写入第一物理存储空间。
也就是说,第一节点中的第一虚拟机无需在接收到第一存储节点返回的RDMA操作的响应报文后,才确定进程的数据写入任务已完成。换言之,DPU的第二物理存储空间是作为第一节点将数据写入第一物理存储空间的缓存来用的。对于第一节点而言,当确定数据1已写入第二物理存储空间,即确定数据1已写入第一物理存储空间。这样的话,第一节点中的写入数据的相关进程资源即可被快速释放,从而节省了第一节点的运行用于写数据的进程所需的资源。
第二种情况:用户设备访问目标容器中的应用进程1,并通过应用进程1读取第一虚拟机的数据2。
这种情况下,第一节点中的第一虚拟机通过应用进程1查询与第一物理内存对应的第二物理内存(即第一节点的一段本地物理内存)。当第一节点确定第二物理内存中不存在数据2,第一节点的操作***触发缺页异常事件。
然后,第一节点中的第一虚拟机进一步查询DPU的第二物理存储空间中是否包括数据2。当第二物理存储空间中包括数据2,则向应用进程2返回数据2。当第二物理存储空间中也不包括数据2,则DPU中的RNIC驱动模块生成用于读取数据2的RDMA读请求,以从第一存储节点的第一物理存储空间中的第一空间读取到数据2,并进一步将数据2返回应用进程1。
在一些可能的情况中,DPU中可以预置热数据算法。这样,DPU基于热数据算法,可以确定出目标容器中的应用进程在近一段时间内访问次数较多的数据(称为热数据)。进而,DPU可以通过RDMA操作将这些热数据预先从第一物理存储空间读取到第二物理存储空间。从而,当目标容器中的应用进程再次访问这些热数据时,DPU可以直接向应用进程返回这些数据,从而提高了应用进程访问数据的效率。
在一些实施例中,当第一节点所在的容器集群中的控制节点(例如图1所示容器集群100中的控制节点10)监控到上述目标容器需要迁移时,控制节点向该容器集群中除第一节点之外的另一个节点发容器送迁移请求,以指示将该目标容器迁移至该另一个节点。
参考图9,图9示出了本申请实施例提供的又一种容器的创建方法的流程示意图。可选的,该方法可以应用于图1所示的容器集群,该方法可以先执行S201-S202,然后执行以下步骤:
S403、第二节点根据容器迁移请求,将第二节点中DPU的第三物理存储空间确定为第二虚拟机的缓存的物理资源。
其中,DPU的说明可以参考上文,不再赘述。
其中,第二节点根据容器迁移请求,将第二节点中DPU的第三物理存储空间确定为第二虚拟机的缓存的物理资源的详细说明,可以参考S303中第一节点根据第一创建请求确定的第一虚拟机的内存大小,将第一节点中DPU的第二物理存储空间确定为第一虚拟机的缓存的物理资源描述,不再赘述。
S404、第二节点的DPU根据容器迁移请求,获取第一物理存储空间的地址。
具体的,第二节点的DPU根据容器迁移请求,获取第一物理存储空间的地址的详细说明,可以参考S203中第二节点根据接收到的容器迁移请求,获取第一物理存储空间的地址的描述,不再赘述。
可以理解,S404和S203的区别在于执行主体不同,S203的执行主体为第二节点,例如S1022可以由第二节点的CPU和网卡配合执行,而S404的执行主体为第二节点中的独立单元DPU。DPU的详细说明可以参考上文对图2所示第一节点中的DPU的相关描述,这里不再赘述。
S405、第二节点根据第三物理存储空间以及第一物理存储空间的地址,创建第二虚拟机。
第二节点可以根据容器迁移请求,先创建用于创建第二虚拟机的虚拟机进程,并通过虚拟机进程建立第二虚拟机的物理内存和第三物理存储空间之间的映射关系,以及,通过第二节点的DPU建立第三物理存储空间和第一物理存储空间的映射关系,从而将第一物理存储空间作为第二虚拟机的内存的物理资源,以及将DPU的第三物理存储空间作为第二虚拟机的缓存的物理资源。
其中,第二节点建立第二虚拟机的物理内存和第三物理存储空间之间的映射关系,以及建立第三物理存储空间和第一物理存储空间的映射关系的详细说明,可以参考S305中第一节点建立第一物理内存和第二物理存储空间之间的映射关系,以及建立第二物理存储空间和第一物理存储空间之间的映射关系的描述,不再赘述。
具体的,第二节点可以还根据容器迁移请求中运行目标容器的资源参数,确定创建第二虚拟机所需资源的资源参数,例如确定创建第二虚拟机所需的CPU资源大小。需要说明,第二节点还可以指示DPU通过RDMA操作(例如读操作)访问第一物理存储空间,以读取第一物理存储空间的第二空间中的CPU状态信息,并基于读取到的CPU状态信息设置第二虚拟机所申请的CPU资源的状态。其中,第二空间和CPU状态信息的说明可以参考S1022中的相关描述,不再赘述。
这样,第二节点可以根据第二虚拟机的内存的物理资源、第二虚拟机的缓存的物理资源、CPU资源后,第二节点通过虚拟机进程创建第二虚拟机。例如,第二节点可以在本地预存的虚拟机镜像文件中读取虚拟机的操作***代码,并通过虚拟机进程在确定的资源中运行该操作***代码,以完成第二虚拟机的创建。其中,虚拟机镜像文件的相关说明可以参考上文描述,不再赘述。
此外,第二节点在通过虚拟机进程创建第二虚拟机后,在第二虚拟机运行的过程中,当第二节点通过虚拟机进程监控到第二虚拟机的CPU状态发生变化,则将第二虚拟机当前的CPU状态信息经第三物理存储空间写入第一存储节点的第一物理空间中的第二空间中,以更新第二空间中记录的CPU状态信息,从而保证第二虚拟机的最新CPU状态信息被记录于第二空间中。其中,第二节点将第二虚拟机的CPU状态信息经第三物理存储空间写入第一物理存储空间中的第二空间的说明,可以参考上文S305中第一节点将第一虚拟机当前的CPU状态信息经第二物理存储空间写入第一存储节点的第一物理空间中的第二空间中的相关描述,不作赘述。
S406、第二节点在第二虚拟机中创建目标容器。
第二节点在第二虚拟机中创建目标容器,以完成目标容器的迁移。其中,S406的详细说
明可以参考上文S205的描述。
需要说明的是,在S406中,第一节点查询并确定DPU的第二物理存储空间中第一虚拟机的内存数据是否已全部写入第一物理存储空间,以确定是否向第二节点发送开启目标的指示消息。
其中,第一节点查询并确定DPU的第二物理存储空间中第一虚拟机的内存数据是否已全部写入第一物理存储空间,以确定是否向第二节点发送开启目标容器的指示消息的详细说明,可以参考S205中第一节点查询并确定第二物理内存中的内存数据是否已全部写入第一物理存储空间,以确定是否向第二节点发送开启目标容器的指示消息的描述,不再赘述。
然后,第二节点执行S206,从而完成了目标容器的迁移。
至此,当通过第二种可能的实现方式中所述的方法在第一节点上创建目标容器后,在将该目标容器迁移至容器集群中的另一节点(例如第二节点)时,只需将用于运行迁移前的目标容器的第一节点的第一虚拟机的内存的物理资源(即第一存储节点上的第一物理存储空间),确定为用于运行迁移后的目标容器的第二节点的第二虚拟机的内存的物理资源。也就是说,第一存储节点上的第一物理存储空间不仅作为迁移前运行于第一虚拟机的目标容器的内存,还作为迁移后运行于第二虚拟机的目标容器的内存。也即,迁移前运行于第一虚拟机的目标容器和迁移后运行于第二虚拟机的目标容器共享内存,具体如图6所示。并且,迁移前运行于第一虚拟机的目标容器和迁移后运行于第二虚拟机的目标容器的镜像文件也是相同的。因此,通过第二种可能的实现方式中所述的方法能够实现在迁移容器时不丢失内存数据、文件数据,不中断用户业务、且用户无感知。
此外,在第二种可能的实现方式中,由于第一节点和第二节点的DPU中可以预先缓存目标容器的热数据,因此当目标容器中的应用进程访问这些热数据时,DPU可以直接向应用进程返回这些数据,从而提高了应用进程访问数据的效率。
在第三种可能的实现方式中,为进一步提高容器迁移的效率,参考图10,图10示出了本申请实施例提供的又一种容器的创建方法的流程示意图。可选的,该方法应用于图1所示的容器集群100。该方法可以在执行图3或图7所示S101-S104的方法的基础上,继续执行以下步骤:
S501、第三节点获取用于指示创建目标容器第二创建请求。
其中,该目标容器即为第一节点基于图3或图7所示方法创建的目标容器。第三节点是第一节点所在容器集群中除第一节点外的任一个工作节点。作为示例,假设第一节点是图1所示容器集群100中的工作节点111,第三节点可以是图1所示容器集群100中的工作节点113。
第三节点获取用于指示创建目标容器第二创建请求的说明,可以参考上文S101中第一节点获取第一创建请求的描述,第二创建请求的说明可以参考上文S101中第一创建请求的描述,不再赘述。
S502、第三节点基于第二创建请求,创建目标容器。
其中,第三节点可以先基于第二创建请求创建用于运行目标容器的第三虚拟机,并在第三虚拟机中创建目标容器。
具体的,第三节点基于第二创建请求创建目标容器的详细说明,可以参考图3所示方法中第一节点根据第一创建请求创建目标容器的描述,或者参考图7所示方法中第一节点根据第一创建请求创建目标容器的描述,不再赘述。
需要说明的是,第三虚拟机的内存的物理资源为第一物理存储空间,即第三节点中用于
运行目标容器的第三虚拟机的内存的物理资源,和第一节点中用于运行目标容器的第一虚拟机的内存的物理资源相同。或者可以理解为,第三节点中用于运行目标容器的第三虚拟机和第一节点中用于运行目标容器的第一虚拟机共享相同的内存物理资源。
还需要说明,第三节点在创建完成目标容器后,将运行目标容器的第三虚拟机的状态设置为暂停状态。这样的话,第三节点中运行于第三虚拟机的目标容器,即可作为第一节点中运行于第一虚拟机的目标容器的备份容器。也即,第一节点中运行于第一虚拟机的目标容器为主目标容器,第三节点中运行于第三虚拟机的目标容器备目标容器。
还需说明,本申请实施例对第一节点创建主目标容器和第三节点创建备目标容器的先后顺序不作具体限定。例如,第一节点可以先基于图3或图7所述的方法创建主目标容器,第三节点再基于图10所述的方法创建备目标容器。或者,第一节点基于图3或图7所述的方法创建主目标容器的同时,第三节点基于图10所述的方法创建备目标容器。不限于此。
至此,本申请实施例即完成了创建主、备目标容器。
在一些实施例中,当控制节点监控到第一节点中的目标容器需要迁移时:
一种可能的实现方式中,控制节点向第三节点发送开启目标容器的指示消息。作为响应,第三节点将用于运行目标容器的第三虚拟机的状态设置为运行状态。从而,作为备目标容器的第三虚拟机中的目标容器开始处理业务。
可选的,在第三节点将第三虚拟机的状态设置为运行状态之前,第三节点还可以通过RDMA操作从第一物理存储空间的第二空间中读取第一虚拟机最新的CPU状态信息,以用于更新第三虚拟机的CPU状态。例如,第三节点可以直接通过RDMA操作从第一物理存储空间的第二空间中读取第一虚拟机最新的CPU状态信息,或者,第三节点的DPU通过RDMA操作从第一物理存储空间的第二空间中读取第一虚拟机最新的CPU状态信息,对此不作限定。
可选的,控制节点还向第一节点发送暂停/删除目标容器的指示消息。作为响应,第一节点删除本地的目标容器。
另一种可能的实现方式中,控制节点向第一节点发送迁移目标容器的指示消息。作为响应,第一节点查询并确定本地存储的第一虚拟机的内存数据是否已全部写入第一物理存储空间,以确定是否向第三节点发送开启目标容器的指示消息。其中,第一节点确定本地存储的第一虚拟机的内存数据是否已全部写入第一物理存储空间的说明,可以参考S205中第一节点查询并确定第二物理内存中的内存数据是否已全部写入第一物理存储空间的描述,不再赘述。
当第三节点接收到第一节点发送的开启目标容器的指示消息后,则将第三虚拟机的状态设置为运行状态。从而,作为备目标容器的第三虚拟机中的目标容器开始处理业务。
可选的,在第三节点将第三虚拟机的状态设置为运行状态之前,第三节点还可以通过RDMA操作从第一物理存储空间的第二空间中读取第一虚拟机最新的CPU状态信息,以用于更新第三虚拟机的CPU状态。
进一步的,在第三节点将第三虚拟机的状态设置为运行状态之后,第三节点和第一节点更新访问目标容器的网络路由信息,以完成将目标容器从第一节点迁移至第三节点。或者,在第一节点向第三节点发送开启目标容器的指示消息后,第一节点和第三节点更新访问目标容器的网络路由信息,以完成将目标容器从第一节点迁移至第三节点。
其中,第三节点、第一节点更新访问目标容器的网络路由信息的说明,可以参考S206中第二节点更新目标容器的网络路由信息的描述,不再赘述。
可以看出,通过该实施例提供的方法进行容器迁移时,无需在控制节点指示迁移容器时,再新建用于运行目标容器的虚拟机,而是直接启用已预先创建好的用于运行备份容器的虚拟
机,从而快速的实现了在不丢失内存数据、不中断用户业务、且用户无感知的情况下高效地完成容器的迁移。
此外,在该实施例中,当目标容器从第一节点迁移至第三节点,第三节点中运行于第三虚拟机的目标容器即升级为主目标容器,并用于运行应用进程以处理用户业务。然后,第三节点所属容器集群中除第三节点外(或者除第三节点和第一节点外)的任一个工作节点,可以获取指示创建目标容器的创建请求,以创建该主目标容器的备目标容器。对此不作详述。
至此,当通过第三种可能的实现方式中所述的方法在第一节点上创建主目标容器、且在第三节点上创建备目标容器后,由于主、备目标容器共享内存(即共享第一存储节点的第一物理存储空间),因此本申请实施例可以通过切换主、备目标容器的运行状态,从而能够快速的实现在不丢失内存数据、不中断用户业务、且用户无感知的情况下完成容器的迁移。
综上,通过本申请实施例提供的容器创建方法所创建的容器在迁移时,通过令迁移前后的容器共享存储节点中的一段物理存储空间,从而可以实现在不丢失内存数据、不中断用户业务、且用户无感知的情况下完成容器的迁移。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。
为了实现上述功能,如图11所示,图11示出了本申请实施例提供的一种容器的创建装置110的结构示意图。容器的创建装置110应用于图1所示容器集群中的第一节点,容器的创建装置110可以用于执行上述的创建容器的方法,例如用于执行图3、图5、图7、图9或图10所示方法中由第一节点执行的部分。
其中,容器的创建装置110可以包括获取单元111和处理单元112。
获取单元111,用于获取用于指示创建目标容器的第一创建请求。处理单元112,用于基于第一创建请求创建第一虚拟机,以及,在第一虚拟机中运行目标容器。其中,目标容器为安全容器,第一创建请求包括运行该目标容器的第一虚拟机的资源参数,第一虚拟机的内存的物理资源为第一存储节点中的第一物理存储空间。
作为示例,结合图3,获取单元111可以用于执行S101,处理单元112可以用于执行S103和S104。
可选的,上述的第一节点通过RDMA访问第一物理存储空间。
可选的,上述的第一物理存储空间为持久化内存介质。
可选的,处理单元112还用于在基于第一创建请求创建第一虚拟机之前:根据第一创建请求确定第一虚拟机的内存大小;根据第一虚拟机的内存大小,向第一存储节点申请第一物理存储空间,并将第一物理存储空间确定为第一虚拟机的内存的物理资源。其中,第一物理存储空间大于或等于第一虚拟机的内存大小。
作为示例,结合图3,处理单元112可以用于执行S102。
可选的,处理单元112还用于:将第一节点中的DPU的第二物理存储空间确定为第一虚拟机的缓存的物理资源;指示DPU建立第二物理存储空间和第一物理存储空间之间的映射关系。其中,第一节点中的DPU用于通过RDMA访问第一物理存储空间,并用于将第二物理存储空间的数据写入第一物理存储空间。
作为示例,结合图7,处理单元112可以用于执行S303。
可选的,第一节点中的DPU还用于从第一物理存储空间读取目标容器的热数据。
可选的,第一节点是上述容器集群中的任一个节点,该容器集群中还包括第二节点,当目标容器被迁移至第二节点,第二节点中运行目标容器的第二虚拟机的内存的物理资源为第一物理存储空间。
可选的,第一节点是上述容器集群中的任一个节点,该容器集群中还包括第三节点,第三节点用于创建目标容器的备份容器,第三节点上运行备份容器的第三虚拟机为暂停状态,第三虚拟机的内存的物理资源为第一物理存储空间。
可选的,当第一虚拟机为故障或暂停状态,上述第三虚拟机为运行状态。
可选的,处理单元112,还用于当目标容器被迁移至容器集群中的另一节点后,更新访问目标容器的网络路由信息。
可选的,容器的创建装置110还包括:发送单元113,用于当目标容器被迁移至容器集群中的另一节点后,向容器集群中除第一节点外的其他节点发送路由更新消息,以使其他节点更新访问目标容器的网络路由信息。
关于上述可选方式的具体描述可以参见前述的方法实施例,此处不再赘述。此外,上述提供的任一种容器的创建装置110的解释以及有益效果的描述均可参考上述对应的方法实施例,不再赘述。
作为示例,结合图2,容器的创建装置110中的获取单元111和发送单元113所实现的功能可以通过图2中的网络接口203实现,或者,获取单元111所实现的功能可以通过图2所示第一节点的输入接口(例如键盘、触摸屏、鼠标等输入接口)实现,处理单元112实现的功能可以通过图2中的处理器201实现。
如图12所示,图12示出了本申请实施例提供的一种容器的创建装置120的结构示意图。容器的创建装置120应用于图1所示容器集群中的第二节点,容器的创建装置120可以用于执行上述的创建容器的方法,例如用于执行图5或图9所示方法中由第二节点执行的部分。
其中,容器的创建装置120可以包括接收单元121和处理单元122。
接收单元121,用于接收容器迁移请求,该容器迁移请求指示将运行于第一节点中第一虚拟机的目标容器迁移至第二节点,以及指示第二节点创建用于运行目标容器的第二虚拟机,该目标容器为安全容器。处理单元122,用于将第一存储节点中的第一物理存储空间确定为第二虚拟机的内存的物理资源;基于第二虚拟机的内存的物理资源,创建第二虚拟机;以及,在第二虚拟机中运行目标容器。其中,第一物理存储空间为第一虚拟机的内存的物理资源。
作为示例,结合图5,接收单元121可以用于执行S202,处理单元122可以用于执行S203、S204和S205。
可选的,上述容器迁移请求中包括目标容器的ID,则容器的创建装置120还包括:发送单元123,用于向第一存储节点发送包括目标容器的ID的内存请求消息,该内存请求消息用于请求获取与目标容器的ID具有对应关系的第一物理存储空间的地址。接收单元121,还用于接收第一存储节点返回的第一物理存储空间的地址。处理单元123,具体用于将接收到的地址所指示的第一物理存储空间确定为第二虚拟机的内存的物理资源。
可选的,上述容器迁移请求中包括第一虚拟机的ID,则容器的创建装置120还包括:发送单元123,用于向第一存储节点发送包括第一虚拟机的ID的内存请求消息,该内存请求消息用于请求获取与第一虚拟机的ID具有对应关系的第一物理存储空间的地址。接收单元122,用于接收第一存储节点返回的第一物理存储空间的地址。处理单元123,具体用于将接收到的地址所指示的第一物理存储空间确定为第二虚拟机的内存的物理资源。
关于上述可选方式的具体描述可以参见前述的方法实施例,此处不再赘述。此外,上述提供的任一种容器的创建装置120的解释以及有益效果的描述均可参考上述对应的方法实施例,不再赘述。
作为示例,结合图2,容器的创建装置120中的接收单元121和发送单元123所实现的
功能可以通过图2中的网络接口203实现,处理单元122实现的功能可以通过图2中的处理器201实现。
本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
需要说明的是,图11或图12中对模块的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。例如,还可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。
本申请实施例还提供一种容器集群,该容器集群包括第一节点和第二节点。其中,第一节点用于执行上文所述方法中由第一节点执行的部分,例如用于执行图3所示方法以创建目标容器。第二节点用于执行上文所述方法中由第二节点执行的部分,例如执行图5所示方法,以实现将运行于第一节点的目标容器迁移至第二节点。
本申请实施例还提供一种芯片***130,如图13所示,该芯片***130包括至少一个处理器和至少一个。作为示例,当该芯片***130包括一个处理器和一个接口电路时,则该一个处理器可以是图13中实线框所示的处理器131(或者是虚线框所示的处理器131),该一个接口电路可以是图13中实线框所示的接口电路132(或者是虚线框所示的接口电路132)。当该芯片***130包括两个处理器和两个接口电路时,则该两个处理器包括图13中实线框所示的处理器131和虚线框所示的处理器131,该两个接口电路包括图13中实线框所示的接口电路132和虚线框所示的接口电路132。对此不作限定。
处理器131和接口电路132可通过线路互联。例如,接口电路132可用于接收信号(例如获取上文所述的第一创建请求或容器迁移请求等)。又例如,接口电路132可用于向其它模块(例如处理器131)发送信号。示例性的,接口电路132可读取存储器中存储的指令,并将该指令发送给处理器131。当该指令被处理器131执行时,可使得容器的创建装置执行上述实施例中的各个步骤。当然,该芯片***130还可以包含其他分立器件,本申请实施例对此不作具体限定。
本申请实施例还提供一种计算机程序产品,以及用于存储该计算机程序产品的计算机可读存储介质。
该计算机程序产品可以包括一个或多个程序指令,当该一个或多个程序指令被一个或多个处理器运行时可以提供以上针对图3、图5、图7、图9或图10描述的功能或者部分功能。因此,例如,参考图3中S101~S104的一个或多个特征可以由该计算机程序产品中的一个或多个指令来承担。
在一些示例中,诸如针对图3、图5、图7、图9或图10描述的容器的创建装置可以被配置为,响应于通过计算机可读存储介质中存储的一个或多个程序指令,提供各种操作、功能、或者动作。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件程序实现时,可以全部或部分地以计算机程序产品的形式来实现。该计算机程序产品包括一个或多个计算机指令。在计算机上和执行计算机执行指令时,全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存
储介质向另一个计算机可读存储介质传输,例如,计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line,DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如,软盘、硬盘、磁带),光介质(例如,DVD)、或者半导体介质(例如固态硬盘(solid state disk,SSD))等。
以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应以所述权利要求的保护范围为准。
Claims (31)
- 一种容器的创建方法,其特征在于,应用于第一节点;所述方法包括:获取用于指示创建目标容器的第一创建请求,所述目标容器为安全容器,所述第一创建请求包括运行所述目标容器的第一虚拟机的资源参数;基于所述第一创建请求创建所述第一虚拟机,所述第一虚拟机的内存的物理资源为第一存储节点中的第一物理存储空间;在所述第一虚拟机中运行所述目标容器。
- 根据权利要求1所述的方法,其特征在于,所述第一节点通过远程直接内存访问RDMA访问所述第一物理存储空间。
- 根据权利要求1或2所述的方法,其特征在于,第一物理存储空间为持久化内存介质。
- 根据权利要求1-3中任一项所述的方法,其特征在于,在所述基于所述第一创建请求创建所述第一虚拟机之前,所述方法还包括:根据所述第一创建请求,确定所述第一虚拟机的内存大小;根据所述第一虚拟机的内存大小,向所述第一存储节点申请所述第一物理存储空间,所述第一物理存储空间大于或等于所述第一虚拟机的内存大小;将所述第一物理存储空间确定为所述第一虚拟机的内存的物理资源。
- 根据权利要求4所述的方法,其特征在于,所述方法还包括:将所述第一节点中的数据处理单元DPU的第二物理存储空间确定为所述第一虚拟机的缓存的物理资源;指示所述DPU建立所述第二物理存储空间和所述第一物理存储空间之间的映射关系;所述DPU用于通过RDMA访问所述第一物理存储空间,并用于将所述第二物理存储空间的数据写入所述第一物理存储空间。
- 根据权利要求5所述的方法,其特征在于,所述DPU还用于从所述第一物理存储空间读取所述目标容器的热数据。
- 根据权利要求1-6中任一项所述的方法,其特征在于,所述第一节点是容器集群中的任一个节点,所述容器集群中还包括第二节点,则当所述目标容器被迁移至所述第二节点,所述第二节点中运行所述目标容器的第二虚拟机的内存的物理资源为所述第一物理存储空间。
- 根据权利要求1-6中任一项所述的方法,其特征在于,所述第一节点是容器集群中的任一个节点,所述容器集群中还包括第三节点,所述第三节点用于创建所述目标容器的备份容器,所述第三节点上运行所述备份容器的第三虚拟机为暂停状态,所述第三虚拟机的内存的物理资源为所述第一物理存储空间。
- 根据权利要求8所述的方法,其特征在于,当所述第一虚拟机为故障或暂停状态,所述第三虚拟机为运行状态。
- 根据权利要求1-9中任一项所述的方法,其特征在于,所述方法还包括:当所述目标容器被迁移至所述容器集群中的另一节点后,更新访问所述目标容器的网络路由信息。
- 根据权利要求1-10中任一项所述的方法,其特征在于,所述方法还包括:当所述目标容器被迁移至所述容器集群中的另一节点后,向所述容器集群中的其他节点发送路由更新消息,以使所述其他节点更新访问所述目标容器的网络路由信息。
- 一种容器的创建方法,其特征在于,应用于第二节点;所述方法包括:接收容器迁移请求;所述容器迁移请求指示将运行于第一节点中第一虚拟机的目标容器迁移至所述第二节点,以及指示所述第二节点创建用于运行所述目标容器的第二虚拟机,所述目标容器为安全容器;将第一存储节点中的第一物理存储空间确定为所述第二虚拟机的内存的物理资源;所述第一物理存储空间为所述第一虚拟机的内存的物理资源;基于所述第二虚拟机的内存的物理资源,创建所述第二虚拟机;在所述第二虚拟机中运行所述目标容器。
- 根据权利要求12所述的方法,其特征在于,所述容器迁移请求中包括所述目标容器的身份标识号ID,则所述将第一存储节点中的第一物理存储空间确定为所述第二虚拟机的内存的物理资源包括:向所述第一存储节点发送内存请求消息,所述内存请求消息包括所述目标容器的ID,所述内存请求消息用于请求获取与所述目标容器的ID具有对应关系的所述第一物理存储空间的地址;接收所述第一存储节点返回的所述第一物理存储空间的地址;将所述地址指示的所述第一物理存储空间确定为所述第二虚拟机的内存的物理资源。
- 根据权利要求12所述的方法,其特征在于,所述容器迁移请求中包括所述第一虚拟机的身份标识号ID,则所述将第一存储节点中的第一物理存储空间确定为所述第二虚拟机的内存的物理资源包括:向所述第一存储节点发送内存请求消息,所述内存请求消息包括所述第一虚拟机的ID,所述内存请求消息用于请求获取与所述第一虚拟机的ID具有对应关系的所述第一物理存储空间的地址;接收所述第一存储节点返回的所述第一物理存储空间的地址;将所述地址指示的所述第一物理存储空间确定为所述第二虚拟机的内存的物理资源。
- 一种容器的创建装置,其特征在于,应用于第一节点;所述装置包括:获取单元,用于获取用于指示创建目标容器的第一创建请求,所述目标容器为安全容器,所述第一创建请求包括运行所述目标容器的第一虚拟机的资源参数;处理单元,用于基于所述第一创建请求创建所述第一虚拟机,所述第一虚拟机的内存的物理资源为第一存储节点中的第一物理存储空间;以及,在所述第一虚拟机中运行所述目标容器。
- 根据权利要求15所述的装置,其特征在于,所述第一节点通过远程直接内存访问RDMA访问所述第一物理存储空间。
- 根据权利要求15或16所述的装置,其特征在于,第一物理存储空间为持久化内存介质。
- 根据权利要求15-17中任一项所述的装置,其特征在于,所述处理单元还用于在基于所述第一创建请求创建所述第一虚拟机之前,根据所述第一创建请求,确定所述第一虚拟机的内存大小;根据所述第一虚拟机的内存大小,向所述第一存储节点申请所述第一物理存储空间,所述第一物理存储空间大于或等于所述第一虚拟机的内存大小;将所述第一物理存储空间确定为所述第一虚拟机的内存的物理资源。
- 根据权利要求18所述的装置,其特征在于,所述处理单元还用于,将所述第一节点中的数据处理单元DPU的第二物理存储空间确定为所述第一虚拟机的缓 存的物理资源;指示所述DPU建立所述第二物理存储空间和所述第一物理存储空间之间的映射关系;所述DPU用于通过RDMA访问所述第一物理存储空间,并用于将所述第二物理存储空间的数据写入所述第一物理存储空间。
- 根据权利要求19所述的装置,其特征在于,所述DPU还用于从所述第一物理存储空间读取所述目标容器的热数据。
- 根据权利要求15-20中任一项所述的装置,其特征在于,所述第一节点是容器集群中的任一个节点,所述容器集群中还包括第二节点,则当所述目标容器被迁移至所述第二节点,所述第二节点中运行所述目标容器的第二虚拟机的内存的物理资源为所述第一物理存储空间。
- 根据权利要求15-20中任一项所述的装置,其特征在于,所述第一节点是容器集群中的任一个节点,所述容器集群中还包括第三节点,所述第三节点用于创建所述目标容器的备份容器,所述第三节点上运行所述备份容器的第三虚拟机为暂停状态,所述第三虚拟机的内存的物理资源为所述第一物理存储空间。
- 根据权利要求22所述的装置,其特征在于,当所述第一虚拟机为故障或暂停状态,所述第三虚拟机为运行状态。
- 根据权利要求15-23中任一项所述的装置,其特征在于,所述处理单元,还用于当所述目标容器被迁移至所述容器集群中的另一节点后,更新访问所述目标容器的网络路由信息。
- 根据权利要求15-24中任一项所述的装置,其特征在于,所述装置还包括:发送单元,用于当所述目标容器被迁移至所述容器集群中的另一节点后,向所述容器集群中的其他节点发送路由更新消息,以使所述其他节点更新访问所述目标容器的网络路由信息。
- 一种容器的创建装置,其特征在于,应用于第二节点;所述装置包括:接收单元,用于接收容器迁移请求;所述容器迁移请求指示将运行于第一节点中第一虚拟机的目标容器迁移至所述第二节点,以及指示所述第二节点创建用于运行所述目标容器的第二虚拟机,所述目标容器为安全容器;处理单元,用于将第一存储节点中的第一物理存储空间确定为所述第二虚拟机的内存的物理资源;基于所述第二虚拟机的内存的物理资源,创建所述第二虚拟机;以及,在所述第二虚拟机中运行所述目标容器;其中,所述第一物理存储空间为所述第一虚拟机的内存的物理资源。
- 根据权利要求26所述的装置,其特征在于,所述容器迁移请求中包括所述目标容器的身份标识号ID,则所述装置还包括:发送单元,用于向所述第一存储节点发送内存请求消息,所述内存请求消息包括所述目标容器的ID,所述内存请求消息用于请求获取与所述目标容器的ID具有对应关系的所述第一物理存储空间的地址;所述接收单元,还用于接收所述第一存储节点返回的所述第一物理存储空间的地址;所述处理单元,具体用于将所述地址指示的所述第一物理存储空间确定为所述第二虚拟机的内存的物理资源。
- 根据权利要求26所述的装置,其特征在于,所述容器迁移请求中包括所述第一虚拟机的身份标识号ID,则所述装置还包括:发送单元,用于向所述第一存储节点发送内存请求消息,所述内存请求消息包括所述第一虚拟机的ID,所述内存请求消息用于请求获取与所述第一虚拟机的ID具有对应关系的所述第一物理存储空间的地址;所述接收单元,用于接收所述第一存储节点返回的所述第一物理存储空间的地址;所述处理单元,具体用于将所述地址指示的所述第一物理存储空间确定为所述第二虚拟机的内存的物理资源。
- 一种容器的创建装置,其特征在于,包括:存储器、网络接口、以及一个或多个处理器,所述一个或多个处理器通过所述网络接口接收或发送数据,所述一个或多个处理器被配置为读取存储在所述存储器中的程序指令,以执行如权利要求1-11或12-14中任一项所述的方法。
- 一种容器集群,其特征在于,所述容器集群包括第一节点和第二节点,所述第一节点用于执行如权利要求1-11中任一项所述的方法,以创建目标容器;所述第二节点用于执行如权利要求12-14中任一项所述的方法,以将运行于所述第一节点的所述目标容器迁移至所述第二节点。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质包括程序指令,当所述程序指令在计算机或处理器上运行时,使得所述计算机或所述处理器执行权利要求1-11或12-14中任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210622548.4A CN117215712A (zh) | 2022-06-02 | 2022-06-02 | 一种容器的创建方法、装置及存储介质 |
CN202210622548.4 | 2022-06-02 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023231572A1 true WO2023231572A1 (zh) | 2023-12-07 |
Family
ID=89026859
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/087179 WO2023231572A1 (zh) | 2022-06-02 | 2023-04-08 | 一种容器的创建方法、装置及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN117215712A (zh) |
WO (1) | WO2023231572A1 (zh) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105354076A (zh) * | 2015-10-23 | 2016-02-24 | 深圳前海达闼云端智能科技有限公司 | 一种应用部署方法及装置 |
CN109885377A (zh) * | 2018-11-23 | 2019-06-14 | ***股份有限公司 | 统一资源调度协调器及其创建虚拟机和/或容器的方法、统一资源调度*** |
CN113900773A (zh) * | 2021-10-25 | 2022-01-07 | 北京计算机技术及应用研究所 | 一种基于国产平台虚拟机和容器的超融合调度方法 |
CN114185641A (zh) * | 2021-11-11 | 2022-03-15 | 北京百度网讯科技有限公司 | 虚拟机冷迁移方法、装置、电子设备及存储介质 |
-
2022
- 2022-06-02 CN CN202210622548.4A patent/CN117215712A/zh active Pending
-
2023
- 2023-04-08 WO PCT/CN2023/087179 patent/WO2023231572A1/zh unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105354076A (zh) * | 2015-10-23 | 2016-02-24 | 深圳前海达闼云端智能科技有限公司 | 一种应用部署方法及装置 |
CN109885377A (zh) * | 2018-11-23 | 2019-06-14 | ***股份有限公司 | 统一资源调度协调器及其创建虚拟机和/或容器的方法、统一资源调度*** |
CN113900773A (zh) * | 2021-10-25 | 2022-01-07 | 北京计算机技术及应用研究所 | 一种基于国产平台虚拟机和容器的超融合调度方法 |
CN114185641A (zh) * | 2021-11-11 | 2022-03-15 | 北京百度网讯科技有限公司 | 虚拟机冷迁移方法、装置、电子设备及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN117215712A (zh) | 2023-12-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230237102A1 (en) | Transparent referrals for distributed file servers | |
US20210247973A1 (en) | Virtualized file server user views | |
US9811276B1 (en) | Archiving memory in memory centric architecture | |
US11144399B1 (en) | Managing storage device errors during processing of inflight input/output requests | |
US10838829B2 (en) | Method and apparatus for loading data from a mirror server and a non-transitory computer readable storage medium | |
US8473692B2 (en) | Operating system image management | |
EP3748926A1 (en) | Communication method and apparatus | |
US9959074B1 (en) | Asynchronous in-memory data backup system | |
US11947985B2 (en) | Data processing method and apparatus, and server for ensuring consistency of data processing processes of a plurality of containers | |
US20210326270A1 (en) | Address translation at a target network interface device | |
WO2019148841A1 (zh) | 一种分布式存储***、数据处理方法和存储节点 | |
US11151686B2 (en) | GPU based server in a distributed file system | |
US10620871B1 (en) | Storage scheme for a distributed storage system | |
JP6421470B2 (ja) | 仮想マシンマイグレーションプログラム、仮想マシンマイグレーションシステムおよび仮想マシンマイグレーション方法 | |
US11461123B1 (en) | Dynamic pre-copy and post-copy determination for live migration between cloud regions and edge locations | |
US20240211435A1 (en) | Information processing method and apparatus | |
US20230281141A1 (en) | Method for order-preserving execution of write request and network device | |
US8621260B1 (en) | Site-level sub-cluster dependencies | |
WO2024051292A1 (zh) | 数据处理***、内存镜像方法、装置和计算设备 | |
WO2023155694A1 (zh) | 内存换页方法、***及存储介质 | |
WO2023231572A1 (zh) | 一种容器的创建方法、装置及存储介质 | |
CN116594551A (zh) | 一种数据存储方法及装置 | |
CN115794368A (zh) | 业务***、内存管理方法及装置 | |
WO2024060710A1 (zh) | 一种页面换入方法以及装置 | |
CN117519908B (zh) | 一种虚拟机热迁移方法、计算机设备及介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23814768 Country of ref document: EP Kind code of ref document: A1 |