CN116450328A - Memory allocation method, memory allocation device, computer equipment and storage medium - Google Patents

Memory allocation method, memory allocation device, computer equipment and storage medium Download PDF

Info

Publication number
CN116450328A
CN116450328A CN202210013752.6A CN202210013752A CN116450328A CN 116450328 A CN116450328 A CN 116450328A CN 202210013752 A CN202210013752 A CN 202210013752A CN 116450328 A CN116450328 A CN 116450328A
Authority
CN
China
Prior art keywords
memory
task
priority
service quality
scheduling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210013752.6A
Other languages
Chinese (zh)
Inventor
蒋彪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210013752.6A priority Critical patent/CN116450328A/en
Publication of CN116450328A publication Critical patent/CN116450328A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022Mechanisms to release resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/48Indexing scheme relating to G06F9/48
    • G06F2209/484Precedence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5021Priority
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a memory allocation method, a memory allocation device, computer equipment and a storage medium. The method can be applied to an application scene for performing memory allocation on a vehicle-mounted terminal, a cloud server or other equipment, and comprises the following steps: receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, preferentially scheduling the processes in the accelerated scheduling queue so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory block to the target task. By adopting the method, the memory service quality of the task can be effectively ensured.

Description

Memory allocation method, memory allocation device, computer equipment and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a memory allocation method, apparatus, computer device, and storage medium.
Background
With the development of computer technology and internet technology, in a cloud computing scene, the task has obvious peak-valley characteristics during running, and in general, the utilization rate of host hardware resources is higher during the peak period of the task, and the utilization rate of resources is reduced during the low peak period of the task.
However, in the current memory allocation manner, different types of tasks are generally deployed on the same cloud host in a mixed manner, and because some types of tasks need to consume hardware resources continuously, serious interference is generated on other tasks, so that other tasks generate larger delay, timeliness cannot be guaranteed, and therefore, the memory service quality (QoS, quality of Service) of the tasks cannot be guaranteed.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a memory allocation method, apparatus, computer device, and storage medium that can effectively guarantee the quality of service of a memory for a task.
A memory allocation method, the method comprising: receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, preferentially scheduling the processes in the accelerated scheduling queue so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory block to the target task.
A memory allocation apparatus, the apparatus comprising: the receiving module is used for receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task; the searching module is used for searching a process corresponding to a task with the service quality priority not higher than the first memory when the residual capacity of the system memory is smaller than the memory space required by the target task; the adding module is used for adding the process corresponding to the task into an acceleration scheduling queue; the scheduling module is used for preferentially scheduling the process in the accelerated scheduling queue when the scheduling condition is met, so that the scheduled process stops running and releases the occupied memory block; and the distribution module is used for distributing the released memory blocks to the target task.
In one embodiment, the apparatus further comprises: the selecting module is used for selecting memory blocks from the rest memory of the system memory according to the memory space required by the target task when the residual capacity of the system memory is greater than or equal to the memory space required by the target task; the allocation module is also used for allocating the selected memory blocks to the target task.
In one embodiment, the apparatus further comprises: the determining module is used for determining the second memory service quality priority of the currently running task; the comparison module is used for comparing the second memory service quality priority with the first memory service quality priority; and the searching module is also used for searching the process corresponding to the task which is not higher than the first memory service quality priority based on the comparison result in the process corresponding to the task which is currently running.
In one embodiment, the determining module is further configured to determine that the task corresponding to the second memory quality of service priority is an offline task when the second memory quality of service priority meets an offline task priority; the selecting module is also used for preferentially selecting the process of the offline task in the processes corresponding to the currently running task; and the memory service quality priority of the offline task is not higher than the first memory service quality priority.
In one embodiment, the apparatus further comprises: the acquisition module is used for acquiring the process of the offline task from the process corresponding to the currently running task when the first memory service quality priority accords with the offline task priority; the selecting module is further configured to preferentially select, in the processes of the offline task, a process with a memory occupation value greater than an occupation threshold.
In one embodiment, the selecting module is further configured to preferentially select, in a process corresponding to a task lower than the first priority, a process corresponding to a task with the second priority meeting the first priority condition; if the process corresponding to the task meeting the first priority condition is not found, selecting the process corresponding to the task with the second memory service quality priority meeting the second priority condition; and if the process corresponding to the task meeting the second priority condition is not found, selecting a process with a memory occupation value larger than an occupation threshold value from the processes corresponding to the tasks with the priority equal to the first memory service quality.
In one embodiment, the apparatus further comprises: the checking module is used for checking whether the acceleration scheduling queue is an empty queue or not; the scheduling module is further configured to schedule a process from the accelerated scheduling queue if the accelerated scheduling queue is not an empty queue, and take the scheduled process as a process executed by a processor core next, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core; the accelerated scheduling queue is used for storing the process of the memory block to be released so that the stored process is scheduled preferentially.
In one embodiment, the apparatus further comprises: the monitoring module is used for monitoring the scheduling event; the scheduling event comprises a user mode returned by a clock interrupt or a kernel mode; the checking module is further configured to, when the scheduling event is monitored, preferentially check whether the accelerated scheduling queue is an empty queue.
In one embodiment, the apparatus further comprises: a sending module, configured to send a termination signal to the scheduled process, and wake up the scheduled process based on the termination signal; taking the scheduled process as the next process executed in the processor core; and the execution module is used for executing the scheduled process through the processor core when the processor core is not preempted, so that the scheduled process processes the termination signal until the relevant logic of the termination signal is processed, and then exits, and the occupied memory block is released.
In one embodiment, the apparatus further comprises: the receiving module is also used for receiving the memory service quality priority parameters of each task through the priority setting interface; the storage module is used for storing the memory service quality priority parameters into data sets corresponding to the processes of the tasks; the acquisition module is further used for acquiring a second memory service quality priority of each task in a data set corresponding to the process of each task; the selecting module is further configured to select, in the processes of each task, a process corresponding to a task with a priority of the second memory service quality not higher than that of the first memory service quality.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of: receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, preferentially scheduling the processes in the accelerated scheduling queue so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory block to the target task.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of: receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, preferentially scheduling the processes in the accelerated scheduling queue so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory block to the target task.
The memory allocation method, the memory allocation device, the computer equipment and the storage medium are used for applying for the memory of the target task by receiving a memory application request; the memory application request carries the first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, the processes in the accelerating scheduling queue are scheduled preferentially, so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory blocks to the target task. Because different types of tasks carry different levels of memory service quality priority information, when the system memory is insufficient, the cloud host can select and recycle the memory occupied by processes corresponding to the tasks at different levels according to the high-low order of the memory service quality priority, and meanwhile, the processes to be recycled are added into the accelerated scheduling queue, so that the cloud host can schedule the processes to be recycled in the accelerated scheduling queue preferentially, namely, the processes to be recycled can be scheduled at the first time by accelerating the scheduled processes of the processes to be recycled, so that the purpose of improving the memory recycling efficiency is achieved, the recycled memory can be redistributed to the target task, and the memory service quality of the task is effectively ensured.
Drawings
FIG. 1 is an application environment diagram of a memory allocation method in one embodiment;
FIG. 2 is a flow chart illustrating a memory allocation method according to an embodiment;
FIG. 3 is a flowchart illustrating steps corresponding to searching tasks with priority not higher than the first memory quality of service in one embodiment;
FIG. 4 is a schematic diagram of a product application for a hybrid deployment of online and offline tasks in one embodiment;
FIG. 5 is a schematic diagram of an application framework for a hybrid deployment of online and offline tasks in one embodiment;
FIG. 6 is a schematic diagram of an implementation architecture of a method for guaranteeing quality of service of a memory in a hybrid deployment scenario in an embodiment;
FIG. 7 is a schematic diagram illustrating a logic process of the priority memory reclamation module according to one embodiment;
FIG. 8 is a schematic diagram illustrating a logic process of the memory reclamation module according to one embodiment;
FIG. 9 is a schematic diagram of the logic processing of the add-on dispatch module in one embodiment;
FIG. 10 is a block diagram illustrating an embodiment of a memory allocation apparatus;
FIG. 11 is an internal block diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
Before describing embodiments of the present application, specific terms or techniques related to the present application will be described as follows:
cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside.
At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage position information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage position information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of Independent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.
The memory allocation method provided by the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The server 104 receives a memory application request for applying memory for a target task; the memory application request carries the first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, the server 104 searches the process corresponding to the task with the service quality priority not higher than the first memory, and adds the process corresponding to the task into the acceleration scheduling queue; when the scheduling condition is met, the server 104 preferentially schedules the process in the acceleration scheduling queue, so that the scheduled process terminates running and releases the occupied memory block; server 104 allocates the freed memory blocks to the target task.
The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, intelligent voice interaction devices, intelligent home appliances, vehicle terminals, and other internet of vehicles devices.
The server 104 may be implemented by a stand-alone server or a server cluster formed by a plurality of servers, and it may be appreciated that the server 104 provided in the embodiment of the present application may also be a service node in a blockchain system, where a Peer-To-Peer (P2P) network is formed between service nodes in the blockchain system, and the P2P protocol is an application layer protocol running on top of a transmission control protocol (TCP, transmission Control Protocol) protocol.
The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.
In one embodiment, as shown in fig. 2, a memory allocation method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
step 202, receiving a memory application request for applying a memory for a target task; the first memory service quality priority of the target task is carried in the memory application request.
The target tasks refer to tasks corresponding to different service types in the cloud computing scene, namely, the tasks corresponding to the services can be divided into multiple types of tasks based on different angles. For example, the tasks corresponding to the service may be divided into online tasks and offline tasks, i.e., the target tasks may include offline tasks and online tasks. For example, the target task may be an online task corresponding to a certain service type, where the online task belongs to a delay-sensitive task, that is, the online task is sensitive to delay, but the overall resource occupancy of the online task is low, for example, a typical service scenario of the online task may include: online shopping carts, payment services, etc. The target task may also be an offline task corresponding to a certain service type, where the offline task belongs to a CPU (central processing unit central processing unit, abbreviated as CPU) intensive task. Namely, the offline task is insensitive to delay and can tolerate starvation, but the offline task needs to continuously consume hardware resources, the overall resource occupancy rate of the offline task is high, and typical business scenarios can include: AI (Artificial Intelligence) training, function computation, etc. It can be understood that the application scenario in the embodiment of the present application may be that different types of tasks are deployed in the same cloud host in a mixed manner, and the different types of tasks share the memory hardware resources of the cloud host, and since the memory is an incompressible resource, reasonable allocation is required. For example, the online task and the offline task are mixed and deployed on the same cloud host, and the online task and the offline task can share memory hardware resources of the cloud host.
The memory application request refers to a request for applying for a task for a memory, for example, based on a memory size required by a task set by a user, the memory size required by the task is applied to a cloud host, so that the task can be normally operated and scheduled.
The memory quality of service (Quality of Service, qoS) refers to the quality of service of a memory, and in order to achieve efficient scheduling and allocation of resources and improve the utilization of resources, the quality of service management may be performed through the memory quality of service.
The priority of the memory service quality refers to priority parameters of the memory service quality corresponding to different tasks or task groups set by a user, for example, the setting range of the priority parameters of the predefined memory service quality is 0-7, 0-7 represents the level parameters of 8 priorities respectively, and 0 represents the lowest priority, which can be used by offline tasks; 7 represents the highest priority available for online tasks.
The first priority of memory service quality refers to a priority parameter of memory service quality corresponding to the target task, for example, the priority parameter of memory service quality corresponding to the target task is 7, that is, the priority of the first memory service quality is 7.
Specifically, a cloud host is taken as a Linux operating system for illustration. The Linux operating system refers to an open-source operating system. It is understood that the cloud host in this embodiment includes, but is not limited to, linux operating systems, and may also be other operating systems, for example, the operating system of the cloud host may also be Windows, macOS, chrome OS, iOS, android, ipad OS, and the like. Cloud hosts using different operating systems can all manage the service quality through the service quality of the memory.
The cloud host, i.e. the server, may receive a memory application request for applying for a memory for a target task, where the memory application request carries a first quality of service priority of the first memory of the target task. For example, in a mixed scene, when the cloud host receives memory application requests of different tasks or task groups, the cloud host needs to allocate memory to the different tasks as required, that is, when the cloud host needs to allocate corresponding new memory to the different tasks, the cloud host can detect whether the residual memory of the current system is insufficient, and if the cloud host detects that the residual memory size of the current system is insufficient to meet the new memory allocation requests, the cloud host can determine that the residual memory of the current system is insufficient; if the cloud host detects that the residual memory size of the current system is enough to meet the new memory allocation request, the cloud host can normally allocate the memory to different types of tasks, for example, the cloud host can normally allocate the memory to online tasks and offline tasks.
For example, assuming that the memory size required by the online task a is 64MB, when the cloud host receives a memory application request for applying for the memory for the online task a, the cloud host may detect whether the remaining memory of the current system is insufficient based on the memory size required by the target task, that is, when the cloud host detects that the remaining memory size of the current system is 30MB, it indicates that the new memory allocation request is insufficient, that is, the cloud host detects that the remaining memory size of the current system is insufficient to the memory size required by the online task a, and the cloud host may determine that the remaining memory of the current system is insufficient. When the cloud host detects that the current residual memory size of the system is 100MB, the cloud host can normally allocate the memory according to the memory size required by the online task A.
And 204, searching a process corresponding to the task with the service quality priority not higher than the first memory when the residual capacity of the system memory is smaller than the memory space required by the target task.
The system memory refers to the size of physical memory that the cloud host can occupy, i.e. all installed physical memory is managed by the operating system of the cloud host, and the operating system determines how much physical memory the system can occupy and how much physical memory the application program can allocate according to the total size of the installed physical memory, and dynamically adjusts between the physical memory and the cache on the hard disk.
A Process (Process) refers to a running activity of a program in a computer on a certain data set, and is a basic unit of resource allocation and scheduling by a system, and is a basis of an operating system structure. In early process-oriented computer architecture, a process is the basic execution entity of a program; in contemporary thread-oriented computer architectures, the process is a container for the threads. A program is a description of instructions, data, and their organization, and a process is an entity of a program. In this embodiment, the operating system of the cloud host is illustrated by taking a Linux operating system as an example. In a Linux operating system, a task may correspond to a process, and a process may be used to manage one or more threads, i.e., threads are the units of actual work.
Specifically, after the cloud host receives a memory application request for applying for a memory for a target task, an operating system of the cloud host can detect whether the remaining memory of the current system is insufficient; when the operating system determines that the residual capacity of the system memory is smaller than the memory space required by the target task, the operating system searches the process corresponding to the task with the priority of the service quality not higher than the first memory service quality, namely the operating system traverses and searches all processes or processes in the process group to be recovered, wherein the process is based on the priority of the service quality of the first memory service quality of the target task carried in the memory application request.
In the searching process, the operating system can preferentially search the process with the lowest memory service quality priority in all processes or process groups based on a preset strategy, and if the process with the lowest memory service quality priority exists and is lower than the first memory service quality priority of the current target task, the operating system can take the process with the lowest memory service quality priority as the process to be recovered. It is understood that the searching method for the process to be recycled in the present application includes, but is not limited to, the above method, and other searching methods are also possible.
For example, assuming that the size of the memory required by the online task a is 64MB, and the priority of the memory service quality of the online task a is 7, when the cloud host receives a memory application request for applying for the memory for the online task a, the cloud host may detect whether the remaining memory of the current system is insufficient, that is, when the cloud host detects that the size of the remaining memory of the current system is 30MB, it indicates that the new memory allocation request is insufficient, that is, when the cloud host detects that the remaining capacity of the current system memory is smaller than the memory space required by the target task a, the cloud host may search for a process corresponding to a task not higher than the priority of the memory service quality 7 based on the priority 7 of the memory service quality of the target task a, and assume that the priority of the memory service quality of the process 1 is 5, and the priority of the memory service quality of the process 2 is 7, that the priority of the memory service quality of the process 3 is 0, then the cloud host may preferentially select all processes or processes with the lowest priority of the memory service quality of the process group, and the priority of the process 3 is 0, and the priority of the process 3 is selected as the priority of the target task to be recovered.
And 206, adding the process corresponding to the task into an acceleration scheduling queue.
The accelerated scheduling queue is different from the conventional scheduling queue, that is, the accelerated scheduling queue is a new queue additionally created. For example, a normal scheduling queue in the cloud host is M1, an accelerated scheduling queue is A1, and when the system operates, an operating system of the cloud host preferentially selects a scheduling object from the accelerated scheduling queue A1 to perform kernel operation.
Specifically, when the remaining capacity of the system memory is smaller than the memory space required by the target task, after the cloud host searches the process corresponding to the task with the priority not higher than the first memory service quality, the cloud host can add the process to be recovered corresponding to the searched task to the accelerated scheduling queue, so that the process to be recovered can be scheduled preferentially. If the task corresponding to the process to be recovered is a low-priority task or an offline task, if the cloud host does not add the process to be recovered to the accelerated scheduling queue, because the task corresponding to the process is the low-priority task or the offline task, the scheduling priority of the offline task is lower, the process needs to be scheduled, and can not be scheduled for a long time, namely, can not be scheduled in the first time, so that the process can not process a termination signal for a long time, the occupied memory of the process can not be released for a long time, finally, the memory allocation of the target task has larger delay, and the timeliness of memory recovery can not be effectively ensured. Therefore, in the embodiment of the application, by improving the scheduling priority of the object to be recycled, namely the process to be recycled, the scheduled flow of the low-priority task or the offline task can be accelerated.
For example, assuming that the priority of the memory service quality of the online task a is 7 and the scheduling priority is 7, the priority of the memory service quality of the offline task B is 0 and the scheduling priority is 0, the priority of the memory service quality of the process 3 corresponding to the offline task B is 0, when the remaining capacity of the system memory is smaller than the memory space required by the online task a, the cloud host may traverse all the processes to find the process corresponding to the task not higher than the priority of the memory service quality 7, and preferentially select the process with the lowest priority of the memory service quality in all the processes or the process group, and since the priority of the memory service quality of the process 3 is 0,0 represents the lowest priority level, the cloud host may select the process 3 as the process to be recycled. Further, the cloud host can add the process 3 corresponding to the offline task B to the accelerated scheduling queue, so that the process 3 can be scheduled preferentially, that is, the scheduled flow of the process 3 is accelerated, and therefore timeliness of memory reclamation is guaranteed.
And step 208, when the scheduling condition is met, the process in the accelerating scheduling queue is scheduled preferentially, so that the scheduled process terminates running and releases the occupied memory block.
The scheduling conditions refer to the scheduling conditions of the kernel of the operating system, and different operating systems correspond to different scheduling conditions, for example, in a Linux operating system, the scheduling conditions may be scheduling occasions, and the common scheduling occasions include clock interruption, user mode returned by kernel mode, and the like.
The termination operation refers to the process flow of actively exiting the process and terminating the operation after the process processes the related logic of the termination signal.
The memory block refers to a memory area formed by a plurality of memory units which can be allocated, a plurality of memory blocks used for meeting memory application requests can exist in a memory pool, all the memory blocks form a memory block linked list, and the memory blocks are used for storing programs and data when a process runs.
Specifically, after the cloud host adds the process corresponding to the task to the acceleration scheduling queue, when the scheduling condition is met, the cloud host preferentially schedules the process in the acceleration scheduling queue, so that the scheduled process terminates running and releases the occupied memory block. Wherein different operating systems correspond to different scheduling conditions.
For example, a cloud host is taken as a Linux operating system for illustration. In the Linux operating system, the scheduling condition may be scheduling opportunity, and the commonly used scheduling opportunity includes clock interruption, user mode returned by kernel mode, and the like. The cloud host adds the process 3 corresponding to the offline task B to the acceleration scheduling queue, and then the next scheduling opportunity is used for temporarily scheduling the process 3 in the acceleration scheduling queue, so that the scheduled process 3 can timely process a termination signal and terminate operation, and simultaneously, the occupied memory block is released.
And step 210, distributing the released memory block to the target task.
Specifically, when the scheduling condition is met, the cloud host preferentially schedules the process in the acceleration scheduling queue, so that the scheduled process terminates operation and releases the occupied memory block, and further, the cloud host can allocate the memory block released by the process which terminates operation to the target task, so as to ensure the memory service quality of the target task.
For example, a cloud host is taken as a Linux operating system for illustration. After the cloud host adds the process 3 corresponding to the offline task B to the acceleration scheduling queue, the cloud host schedules the process 3 in the acceleration scheduling queue preferentially when the next scheduling opportunity comes, so that the scheduled process 3 can process a termination signal in time and terminate operation, and simultaneously releases the occupied memory block 3. Furthermore, the cloud host can quickly recycle the memory blocks 3 released by the process 3 which terminates the operation, and allocate the recycled memory blocks 3 to the target task A, so that the target task A can be quickly allocated to the required memory, and the memory service quality of the task A is effectively ensured.
In the memory allocation method, a memory application request for applying a memory for a target task is received; the memory application request carries the first memory service quality priority of the target task; when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory; adding the process corresponding to the task into an acceleration scheduling queue; when the scheduling conditions are met, the processes in the accelerating scheduling queue are scheduled preferentially, so that the scheduled processes terminate running and the occupied memory blocks are released; and distributing the released memory blocks to the target task. Because different types of tasks carry different levels of memory service quality priority information, when the system memory is insufficient, the cloud host can select and recycle the memory occupied by processes corresponding to the tasks at different levels according to the high-low order of the memory service quality priority, and meanwhile, the processes to be recycled are added into the accelerated scheduling queue, so that the cloud host can schedule the processes to be recycled in the accelerated scheduling queue preferentially, namely, the processes to be recycled can be scheduled at the first time by accelerating the scheduled processes of the processes to be recycled, so that the purpose of improving the memory recycling efficiency is achieved, the recycled memory can be redistributed to the target task, and the memory service quality of the target task is effectively ensured.
In one embodiment, after receiving the memory application request for applying for the memory for the target task, the method further includes:
when the residual capacity of the system memory is larger than or equal to the memory space required by the target task, selecting a memory block from the residual memory of the system memory according to the memory space required by the target task;
and distributing the selected memory block to the target task.
Specifically, after the cloud host receives a memory application request for applying for a memory for a target task, the cloud host may detect whether the remaining memory of the current system is insufficient based on the memory size required by the target task, that is, when the cloud host detects that the remaining capacity of the system memory is greater than or equal to the memory space required by the target task, which indicates that the cloud host is sufficient to satisfy the new memory allocation request, the cloud host selects a memory block in the remaining memory of the system memory according to the memory space required by the target task, and allocates the selected memory block to the target task.
For example, assuming that the memory size required by the online task a is 64MB, when the cloud host receives a memory application request for applying for the online task a for the memory, the cloud host may detect whether the remaining memory of the current system is insufficient based on the memory size required by the target task, that is, when the cloud host detects that the remaining memory size of the current system is 100MB, it indicates that the current system is sufficient to satisfy the new memory allocation request, the cloud host may select the memory block 1 according to the memory space required by the online task a, and allocate the selected memory block 1 to the online task a. Therefore, by judging whether the residual memory of the system can meet the new memory allocation requirement, the low-priority task can fully utilize the idle memory and can effectively ensure the memory service quality of the high-priority task when the memory is insufficient.
In one embodiment, as shown in fig. 3, the step of searching for a process corresponding to a task having a priority not higher than the first memory quality of service includes:
step 302, determining a second memory quality of service priority of the currently running task.
Step 304, comparing the second memory quality of service priority with the first memory quality of service priority.
And 306, searching a process corresponding to the task with the service quality priority not higher than the first memory based on the comparison result in the process corresponding to the currently running task.
The second memory quality of service priority refers to a memory quality of service priority parameter of a task other than the target task, and is different from the first memory quality of service priority of the target task, that is, the memory quality of service priority corresponding to different tasks is indicated. For example, the first memory quality of service priority of the target task a is 3, and the second memory quality of service priorities corresponding to the other tasks B and C are 4 and 6, respectively.
After the cloud host receives a memory application request for applying for a memory for a target task, when the cloud host detects that the residual capacity of the system memory is smaller than the memory space required by the target task, the cloud host searches a process corresponding to the task with the service quality priority not higher than that of the first memory. Specifically, the cloud host may determine a second memory quality of service priority of all tasks currently running, and compare the second memory quality of service priority with the first memory quality of service priority. And searching the process corresponding to the task with the priority not higher than the service quality of the first memory based on the comparison result in the process corresponding to the task which is currently operated.
For example, assuming that the size of the memory required by the online task a is 64MB, and the priority of the memory service quality of the online task a is 7, when the cloud host receives a memory application request for applying for the memory for the online task a, the cloud host can detect whether the remaining memory of the current system is insufficient based on the size of the memory required by the online task a, that is, when the cloud host detects that the size of the remaining memory of the current system is 30MB, the cloud host indicates that the current memory allocation request is insufficient, that is, when the remaining capacity of the memory of the current system is detected to be smaller than the memory space required by the target task a, the cloud host can search for the second memory service quality priority of all tasks currently running based on the memory service quality priority 7 of the target task a, and assume that the priority of the memory service quality of the memory service of all tasks currently running is 5, and the memory service quality priority of task 2 is 7, that is 0, that the cloud host can determine that the second memory service quality priority of all tasks currently running is 0, that is respectively 0, 7, that is more than 7, that is, and that is, the priority of the corresponding to the first task is not the priority and the priority is 0, and the priority is not lower than the priority is set to be found by the first process, and the priority is compared with the priority of the first process, and the first process is not corresponding to the priority and the priority is set to the priority and the priority is 0.
If the cloud host traverses all the tasks currently running, the memory service quality priority of the task 1 is found to be 5, the memory service quality priority of the task 2 is found to be 7, that is, the cloud host can determine that the second memory service quality priorities of all the tasks currently running are 5 and 7 respectively, and the cloud host can compare the second memory service quality priorities with the first memory service quality priorities, because 5 is smaller than 7, that is, in the processes corresponding to the tasks currently running, the cloud host can preferentially select the processes corresponding to the task 1 lower than the first memory service quality priority 7 based on the comparison result, that is, the cloud host can take the processes corresponding to the task 1 as the processes to be recovered.
If the cloud host traverses all the tasks currently running, the memory service quality priority of the task 1 is found to be 7, and the memory service quality priority of the task 2 is found to be 7, namely the cloud host can determine that the second memory service quality priorities of all the tasks currently running are 7 and 7 respectively, and the cloud host can compare the second memory service quality priorities with the first memory service quality priorities, and because 7 is equal to 7, namely in the processes corresponding to the tasks currently running, the cloud host searches for the processes corresponding to the tasks not higher than the first memory service quality priority 7 to be the processes corresponding to the task 1 and the task 2 based on the comparison result, and the cloud host can take the processes corresponding to the task 1 and the task 2 as the processes to be recycled. In addition, the cloud host can further select a process with the largest memory occupation amount from the task 1 and the task 2 as a process to be recycled based on a preset strategy, and the memory occupation amount of the process 1 corresponding to the task 1 is assumed to be 40MB; and if the memory occupation amount of the process 2 corresponding to the task 2 is 45MB, the cloud host can select the process 2 corresponding to the task 2 as a process to be recovered.
In this embodiment, when the system memory is insufficient, memory recovery can be automatically performed according to the order of the priority of the memory service quality, so as to automatically recover the memory occupied by the task with low priority of the memory service quality, thereby effectively guaranteeing the memory service quality of the task with high priority.
In one embodiment, the step of searching for a process corresponding to a task not higher than the first memory quality of service priority based on the comparison result includes:
when the second memory service quality priority accords with the offline task priority, determining that the task corresponding to the second memory service quality priority is an offline task;
preferentially selecting the process of the offline task from the processes corresponding to the currently running task; the memory quality of service priority of the offline task is not higher than the first memory quality of service priority.
In this embodiment, a scenario of mixed deployment of an online task and an offline task is taken as an example for illustration. The online task and the offline task share the memory hardware resources of the cloud host, and the memory is an incompressible resource, so that when the memory of the cloud host is insufficient and the cloud host needs to be allocated with new memory, certain memory must be recovered through a certain mechanism to ensure that the cloud host and the key task can normally operate. The online task and the offline task respectively apply for the memory required by themselves as required. When the memory resources are sufficient, the cloud host does not interfere with each other when the memory is allocated; when the memory resources are insufficient, the cloud host needs to perform memory recycling processing, namely, the new memory application request can be met by recycling the memory used by part of tasks. Specifically, when the memory resources are insufficient, the cloud host may determine the second memory quality of service priority of all the tasks currently running, and compare the second memory quality of service priority with the first memory quality of service priority. When the cloud host detects that the second memory service quality priority accords with the offline task priority, determining that the task corresponding to the second memory service quality priority is the offline task. Further, in the processes corresponding to the currently running tasks, the cloud host can preferentially select the process of the offline task as the process to be recovered, and the memory service quality priority of the offline task is not higher than the first memory service quality priority.
For example, assuming that the predefined priority of the memory service quality of the online task is 7, the priority of the memory service quality of the offline task is 0, the size of the memory required by the task a is 64MB, and the priority of the memory service quality of the task a is 7, it indicates that the task a is an online task, and when the cloud host receives a memory application request for applying for the memory for the online task a, the cloud host may detect whether the remaining memory of the current system is insufficient based on the size of the memory required by the online task a. When the memory resources are insufficient, the cloud host can search the second memory service quality priorities of all the tasks currently running in a traversing manner, and supposing that the cloud host searches the memory service quality priorities of the tasks 1 to be 5 and the memory service quality priorities of the tasks 2 to be 7, the memory service quality priorities of the tasks 3 to be 0, namely the cloud host can determine that the second memory service quality priorities of all the tasks currently running are 0, 5 and 7 respectively, and when the cloud host detects that the second memory service quality priorities 0 accord with the offline task priorities, the task 3 corresponding to the second memory service quality priorities is determined to be the offline task. Further, in the process corresponding to the currently running task, the cloud host may preferentially select the process 3 of the offline task 3 as the process to be recovered, where the priority 0 of the memory quality of service of the offline task is necessarily lower than the priority 7 of the first memory quality of service of the online task a. Therefore, when the system memory is insufficient, the memory recovery can be automatically carried out according to the high-low order of the priority of the memory service quality, the memory occupied by the offline task is recovered preferentially, and the memory service quality of the online task is ensured.
In one embodiment, in a process corresponding to a currently running task, a step of preferentially selecting a process of an offline task includes:
when the first memory service quality priority accords with the offline task priority, acquiring a process of the offline task from a process corresponding to the currently running task;
and in the process of the offline task, preferentially selecting a process of which the memory occupation value is larger than the occupation threshold value.
Specifically, when the memory resources are insufficient and when the cloud host detects that the first memory service quality priority of the target task accords with the offline task priority, the cloud host can acquire the processes of all the offline tasks in the processes corresponding to the currently running tasks. Further, the cloud host can preferentially select a process with the memory occupation value larger than the occupation threshold value from all the processes of the offline tasks. In the scene of mixed deployment of the offline task and the online task, if the target task is the offline task, the cloud host preferentially selects the process of the offline task with the maximum memory occupation value as the process to be recovered.
For example, assuming that the predefined priority of the memory service quality of the online task is 7, the priority of the memory service quality of the offline task is 0, the size of the memory required by the task B is 60MB, and the priority of the memory service quality of the online task B is 0, it indicates that the task B is an offline task, and when the cloud host receives a memory application request for applying for the memory for the offline task B, the cloud host may detect whether the remaining memory of the current system is insufficient based on the size of the memory required by the task B. When the memory resources are insufficient, the memory service quality priority 0 of the task B accords with the offline task priority, so that the cloud host can acquire the processes of all offline tasks in the processes corresponding to the currently running task, and preferentially select the processes with memory occupation values larger than the occupation threshold value as the processes to be recovered in the processes of all offline tasks. Therefore, when the system memory is insufficient, the memory recovery can be automatically carried out according to the high-low order of the priority of the memory service quality, the memory occupied by the offline task is recovered preferentially, the serious interference of the offline task on the online service is effectively avoided, and the memory service quality of the online task is ensured.
In one embodiment, the method further comprises:
preferentially selecting a process corresponding to the task with the second memory service quality priority meeting the first priority condition from processes corresponding to the task with the lower first memory service quality priority;
if the process corresponding to the task meeting the first priority condition is not found, selecting the process corresponding to the task with the second memory service quality priority meeting the second priority condition;
if the process corresponding to the task meeting the second priority condition is not found, selecting a process with the memory occupation value larger than the occupation threshold value from the processes corresponding to the tasks with the priority equal to the service quality of the first memory.
The first priority condition and the second priority condition refer to different preset memory service quality priority conditions. For example, the first priority condition may be a low priority in the range of 0-2; the second priority condition may be a medium priority in the range of 3-5.
Specifically, when the memory resources are insufficient, the cloud host searches for a process corresponding to a task with a priority not higher than the first memory service quality based on a comparison result in a process corresponding to a currently running task, and the cloud host can preferentially select a process corresponding to a task with a priority meeting the first priority condition from a process corresponding to a task with a priority lower than the first memory service quality; if the cloud host does not find the process corresponding to the task meeting the first priority condition, the cloud host selects the process corresponding to the task with the second memory service quality priority meeting the second priority condition; if the cloud host does not find the process corresponding to the task meeting the second priority condition, selecting a process with the memory occupation value larger than the occupation threshold value from the processes corresponding to the tasks with the priority equal to the service quality of the first memory.
For example, assume that the predefined first priority condition is a low priority in the range of 0-2; the second priority condition is a medium priority in the range of 3-5, the memory size required by the task B is 60MB, the memory service quality priority of the task B is 6, and when the cloud host receives a memory application request for applying for the memory for the task B, the cloud host can detect whether the residual memory of the current system is insufficient or not based on the memory size required by the task B.
When the memory resources are insufficient, the cloud host can search for the second memory service quality priority of all the tasks currently running based on the memory service quality priority 6 of the target task B, and if the cloud host searches for all the tasks currently running, the memory service quality priority of the task 1 is found to be 5, the memory service quality priority of the task 2 is found to be 7, and the memory service quality priority of the task 3 is found to be 0, i.e., the cloud host can determine that the second memory service quality priorities of all the tasks currently running are respectively 0, 5 and 7, and as the memory service quality priority of the task 3 is 0 and accords with the first priority condition, the cloud host can preferentially select the process 3 corresponding to the task 3, of which the second memory service quality priority accords with the first priority condition, as a process to be recovered in the process corresponding to the task lower than the first memory service quality priority condition 6.
When the memory resources are insufficient, the cloud host traverses and searches the second memory service quality priority of all the tasks currently running based on the memory service quality priority 6 of the target task B, and the cloud host is assumed to traverse all the tasks currently running and find the memory service quality priority of the task 1 to be 5, and the memory service quality priority of the task 2 to be 7, namely, the cloud host can determine that the second memory service quality priorities of all the tasks currently running are respectively 5 and 7, and as no task conforming to the first priority condition is found, namely, the cloud host does not find the process corresponding to the task conforming to the first priority condition, the cloud host selects the process corresponding to the task of which the second memory service quality priority conforms to the second priority condition. Because the priority of the memory service quality of the task 1 is 5 and meets the second priority condition, the cloud host can select the process 1 corresponding to the task 1 with the second memory service quality priority meeting the second priority condition as the process to be recycled.
When the memory resources are insufficient, the cloud host traverses and searches the second memory service quality priority of all the tasks currently running based on the memory service quality priority 6 of the target task B, and if the cloud host traverses all the tasks currently running, the memory service quality priority of the task 1 is found to be 6, the memory value occupied by the process 1 corresponding to the task 1 is 60MB, the memory service quality priority of the task 2 is 6, the memory value occupied by the process 2 corresponding to the task 2 is 100MB, namely the cloud host can determine that the second memory service quality priorities of all the tasks currently running are 6 and 6 respectively, and as no task meeting the first priority condition and the second priority condition is found by the cloud host, namely the process corresponding to the task meeting the first priority condition and the second priority condition is not found by the cloud host, and the process with the memory occupation value larger than the occupied threshold is selected as the process to be recovered in the processes corresponding to the tasks equal to the first memory service quality priority 6. Assuming that the occupation threshold is 80MB, as 100MB is larger than 80MB, the cloud host selects a process 2 with the memory occupation value larger than the occupation threshold as a process to be recycled in a process corresponding to a task equal to the first memory service quality priority 6.
In this embodiment, when the system memory is insufficient, memory recovery can be automatically performed according to the order of the priority of the memory service quality, so as to automatically recover the memory occupied by the task with low priority of the memory service quality, thereby ensuring the memory service quality of the task with high priority.
In one embodiment, the step of preferentially scheduling the processes in the accelerated scheduling queue such that the scheduled processes terminate operation and free up occupied memory blocks comprises:
preferentially checking whether the acceleration scheduling queue is an empty queue;
if the acceleration scheduling queue is not an empty queue, scheduling a process from the acceleration scheduling queue, and taking the scheduled process as the next process executed in the processor core, so that the process terminates operation and releases occupied memory blocks in the execution process of the processor core; the accelerated dispatch queue is used for storing the process of the memory block to be released so that the stored process is dispatched preferentially.
Specifically, when the scheduling condition is satisfied, the cloud host preferentially checks whether the acceleration scheduling queue is an empty queue. If the cloud host detects that the acceleration scheduling queue is not an empty queue, the cloud host schedules a process from the acceleration scheduling queue, and takes the scheduled process as a next process executed in the processor core, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core; the accelerated dispatch queue is used for storing the process of the memory block to be released so that the stored process is dispatched preferentially.
For example, assuming that the normal scheduling queue in the cloud host is M1, the accelerated scheduling queue is A1, and the accelerated scheduling queue A1 includes a process 1 and a process 2, when the scheduling condition is satisfied, the cloud host preferentially checks whether the accelerated scheduling queue A1 is an empty queue. If the cloud host detects that the acceleration scheduling queue A1 is not an empty queue, the cloud host schedules the process 1 from the acceleration scheduling queue A1, and takes the scheduled process 1 as the next process executed in the processor core, so that the process 1 terminates running and releases occupied memory blocks in the execution process of the processor core. When the scheduling condition is met next time, the cloud host continues to check preferentially whether the acceleration scheduling queue A1 is an empty queue. If the cloud host detects that the acceleration scheduling queue A1 is not an empty queue, the cloud host schedules the process 2 from the acceleration scheduling queue A1, and takes the scheduled process 2 as the next process executed in the processor core, so that the process 2 terminates running and releases occupied memory blocks in the execution process of the processor core. It can be understood that when a process in the acceleration queue A1 is invoked, the cloud host may schedule the corresponding process in sequence according to the first-in first-out principle until the acceleration queue A1 is empty, and schedule the process in the normal scheduling queue M1 according to the first-in first-out principle. Therefore, the process to be recycled is added into the acceleration scheduling queue, so that the operating system can schedule the process of the memory to be recycled in the acceleration scheduling queue preferentially, namely, the purpose of improving the memory recycling efficiency is achieved by accelerating the priority of the process to be recycled, and the memory service quality of the target task can be effectively guaranteed.
In one embodiment, the step of preferentially checking whether the accelerated dispatch queue is an empty queue includes:
monitoring a scheduling event; the scheduling event comprises a user mode returned by a clock interrupt or a kernel mode;
when a scheduling event is monitored, a priority check is made to see if the expedited scheduling queue is an empty queue.
Specifically, the cloud host may monitor the scheduling event in real time, where the scheduling event includes a clock interrupt or a user mode returned by the kernel mode. When the cloud host monitors a scheduling event, the cloud host preferentially checks whether the acceleration scheduling queue is an empty queue. It will be appreciated that the scheduling events corresponding to different operating systems are different. In the Linux operating system, the scheduling event may be a scheduling opportunity, where the scheduling opportunity has a clock interrupt, a user mode returned by a kernel mode, and the like. When the scheduling opportunity comes, the cloud host preferentially checks whether the acceleration scheduling queue is an empty queue, if the cloud host detects that the acceleration scheduling queue is not the empty queue, the cloud host schedules a process from the acceleration scheduling queue, and takes the scheduled process as the next process executed in the processor core, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core. During the execution of the processor core, the cloud host can close the preemption function, i.e. prevent the processor core from being preempted by other processes. For example, in an iOS operating system, the signal is not allowed to be preempted by other processes while being processed in the kernel state. Therefore, the scheduling opportunity of the low-priority task can be effectively ensured in time, the memory recovery efficiency of the low-priority task in the mixed scene is improved, and the timeliness of memory recovery is ensured, so that the memory service quality of the high-priority task is effectively ensured.
In one embodiment, the method further comprises:
sending a termination signal to the scheduled process, and waking the scheduled process based on the termination signal;
taking the scheduled process as the next process executed in the processor core, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core, comprising:
taking the scheduled process as the next process executed in the processor core;
when the processor core is not preempted, executing the scheduled process through the processor core, so that the scheduled process processes the termination signal, exits after the relevant logic of the termination signal is processed, and releases the occupied memory block.
Where the termination signal refers to a signal for terminating the execution of a process, for example, the termination signal may be a SIGKILL signal, which is a signal sent to a process to cause it to terminate immediately on a POSIX-compliant platform.
Specifically, the cloud host may send a termination signal to the scheduled process, and the cloud host wakes up the scheduled process based on the termination signal, and uses the scheduled process as a next process executed in the processor core. Further, when the processor core is not preempted, the cloud host executes the scheduled process through the processor core, so that the scheduled process processes the termination signal, exits after the relevant logic of the termination signal is processed, and releases the memory block occupied by the process itself.
For example, the processing step of the cloud host for accelerating memory reclamation may include: the cloud host searches an object to be recovered through the priority memory recovery module, the object to be recovered can be a process or a process group, the cloud host sends a SIGKILL signal to the object to be recovered, and wakes up the object to be recovered at the same time, and the object to be recovered is used as a next process executed in the processor core. Further, when the processor core is not preempted, the cloud host executes the scheduled object to be reclaimed through the processor core, so that the scheduled object to be reclaimed processes the SIGKILL signal, automatically exits until relevant logic of the SIGKILL signal is processed, and releases the occupied memory block. The KILL refers to a computer running a process that is immediately terminated, and since memory reclamation is usually performed by dropping the process to be reclaimed through the KILL, the purpose of reclaiming the memory occupied by the process is achieved. When executing a KILL pending reclaim process, the SIGKILL signal needs to be processed in the context of the KILL process, thereby exiting the process and reclaiming memory.
Because the processing of the SIGKILL signal needs to be performed in the context of the KILL process, reclaiming memory depends on the process to be reclaimed being scheduled in time, and if the process cannot be scheduled, the SIGKILL signal cannot be processed, and the memory occupied by the process cannot be released in time. Particularly in a mixed part scene, the scheduling priority of a low-priority process is very low and is suppressed to be severe, so that when a high-priority task is relatively busy, the low-priority task is difficult to schedule and run, and the situation of slow memory recovery can occur. In this embodiment, by improving the scheduling priority of the object to be recycled, that is, the process to be recycled, the scheduling opportunity of the low-priority task can be effectively ensured to be timely obtained, the memory recycling efficiency of the low-priority task in the mixed scene is improved, and the timeliness of memory recycling is ensured, so that the memory service quality of the high-priority task is effectively ensured.
In one embodiment, before receiving the memory application request for applying for the memory for the target task, the method further includes:
receiving the memory service quality priority parameters of each task through a priority setting interface;
storing the priority parameters of the service quality of the memory into the data sets corresponding to the processes of the tasks;
searching the process corresponding to the task with the service quality priority not higher than the first memory comprises the following steps:
acquiring a second memory service quality priority of each task in a data set corresponding to the progress of each task;
and selecting a process corresponding to the task with the second memory service quality priority not higher than the first memory service quality priority from the processes of the tasks.
The data set refers to a data set corresponding to a process or a process group and storing data, for example, the data set may be task blocks, and each task block stores data corresponding to each task.
Specifically, before the cloud host receives a memory application request for applying for a memory for a target task, the cloud host may receive, through a priority setting interface, a memory quality of service priority parameter of each task set by a user, and the cloud host may store the memory quality of service priority parameter in a data set corresponding to a process of each task. When the cloud host searches the process corresponding to the task with the priority not higher than the first memory service quality, the cloud host can acquire the priority of the second memory service quality of each task in the data set corresponding to the process of each task, and select the process corresponding to the task with the priority of the second memory service quality not higher than the priority of the first memory service quality.
For example, a cloud host is taken as a Linux operating system for illustration. In the Linux operating system, the priority setting interface may include a proc file system interface and a cgroup interface, where the proc file system is a standard interface of a user mode and a kernel in the Linux system, and is an implementation manner for the Linux system. The user can realize the setting of the memory service quality priority of the process through the proc file system interface; or the user can also realize the setting of the priority of the service quality of the memory of the process group through the cgroup interface, and it can be understood that the priority setting interface in the embodiment can be defined as other interfaces in an abstract way. For example, the priority setting range of the service quality of the memory is 0-7, which respectively represents 8 priority levels, and 0 represents the lowest priority for offline tasks; 7 represents the highest priority for use by the online task. When the user needs to set the priority parameters of the memory service quality of the appointed process or the process group, the user writes the appointed priority parameters of the memory service quality into the relevant interfaces. After the priority parameters of the service quality of the memory are written in, the corresponding priority parameters are stored in a data structure corresponding to the process or the process group in the kernel of the operating system. And then, when the operating system carries out memory recovery, the priority parameters of the stored memory service quality in the process or the process group can be directly read. Therefore, different memory service quality priorities are predefined for different types of tasks, so that in a mixed scene, when the system memory is insufficient, the cloud host can select and recover the memory occupied by processes corresponding to the tasks with different levels according to the high-low sequence of the memory service quality priorities, thereby achieving the purpose of improving the memory recovery efficiency, and further effectively guaranteeing the memory service quality of the high-priority tasks.
The method provided by the embodiment of the application can be applied to cloud product application scenes. Taking a cloud computing scenario of hybrid deployment as an example, the memory allocation method provided by the embodiment of the application is described, and includes the following steps:
in the traditional cloud computing scenario, the service types are various, and there are CPU intensive offline tasks and delay sensitive online tasks. The service has obvious peak-valley characteristic during operation, and the host hardware resource utilization rate is higher during the service peak period, and the resource utilization rate is reduced during the service low peak period. In order to avoid service jitter and abnormality caused by host resource competing, the service capacity planning can be generally performed only according to the maximum value of the service volume in the service peak period, and a certain margin is reserved to cope with the emergency, so that more idle hardware resources can not be utilized in the service low peak period, and the resources are idle, so that the cost is high.
Meanwhile, in a cloud computing scene of mixed deployment, different types of services are deployed on the same cloud host in a mixed manner, for example, in a scene of mixed deployment of online services and offline services, because the offline services need to consume hardware resources continuously, serious interference can be generated on the online services, so that larger delay is generated on the online services, or abnormal services are caused. Therefore, in the scenario of offline service and online service deployment, the key core is the underlying resource isolation technology. The underlying resources typically include: the CPU (central processing unit, abbreviated as CPU), memory, IO (Input/Output) and network all require respective isolation.
Wherein, the memory resource is incompressible resource, the multiplexing difficulty is relatively higher, and the difficulty of the memory isolation technology is also relatively higher. The core of the memory isolation is to ensure low delay of memory allocation of a high priority task, which means that when a high priority task needs memory, a cloud host is required to quickly recycle the memory occupied by the low priority task, whereas in the existing cloud host operating system, for example, in a Linux operating system, timeliness of memory recycling cannot be ensured generally, on one hand, because the memory recycling needs to execute a SIGKILL signal in the context of a recycled process, that is, the recycled process is depended on timely scheduling; on the other hand, in the mixed part scene, the scheduling priority of the low-priority task is lower, when the high-priority task runs continuously, the low-priority task is difficult to obtain scheduling opportunities, or the running time is insufficient, so that the memory recovery time of the low-priority task in the mixed part scene is easily uncontrollable, timeliness cannot be ensured, and the memory service quality of the high-priority task cannot be ensured.
Therefore, in order to solve the above-mentioned problem, that is, to solve the problem that the service quality of the high-priority task memory in the mixed portion scene cannot be guaranteed, a new method for accelerating recovery of the memory occupied by the low-priority task is provided in this embodiment, so as to ensure the service quality of the memory of the high-priority task in the offline and online task mixed portion environments of the cloud host, and ensure the service quality of the memory of the high-priority task when the memory is tense while fully utilizing the idle memory of the low-priority task. It can be appreciated that the method provided in this embodiment is applicable to a container blend scene, and may be extended to a virtual machine blend scene. The core content comprises:
1. And setting the priority of the service quality of the internal memory of the task. Different memory quality of service priorities are defined for different types of tasks, for example, in the context of an offline and online task mix, the online and offline tasks are set with different memory quality of service priorities.
2. And recovering the memory according to the priority of the service quality of the memory when the memory is insufficient. When the memory is insufficient, the cloud host triggers a memory recycling process, and the memory occupied by tasks of different levels is recycled from low to high according to the high-low sequence of the priority of the service quality of the memory in the memory recycling process.
3. And the memory recovery is accelerated. When the memory occupied by the task with the specified memory service quality priority is recovered, the cloud host usually recovers the memory occupied by the task with low priority preferentially, namely the cloud host achieves the aim of improving the memory recovery efficiency by accelerating the process of scheduling and executing the task to be recovered, so that the memory service quality of the task with high priority is ensured.
In addition, taking a cloud host as a Linux operating system as an example, in the existing Linux operating system, when the system memory is exhausted, OOM (Out Of Memory) is triggered, and basic logic of the standard ook includes:
1. the operating system scores all the processes according to the occupied amount of the memory and other characteristics, the process with the highest score is selected as the process needing to be recovered, the operating system scores according to the occupied memory size of the process, and the more the occupied memory is, the higher the score is;
2. The operating system sends a SIGKILL signal to the process to be recovered;
3. processing the SIGKILL signal depends on the process to be reclaimed being scheduled, after which the relevant signal can be processed in the context of the process.
4. When the process is scheduled, the process immediately terminates operation after processing the SIGKILL signal, and releases the memory occupied by the process itself, i.e. the operating system recovers all the memory occupied by the process.
The above prior art has the following disadvantages:
1. when the memory is recovered, that is, when the operating system triggers the execution of the OOM flow, and when the process to be recovered is selected, the operating system mainly selects the process to be recovered according to the memory occupation amount of the process, and does not consider the priority of the memory service quality of the process, so that the memory of the low-priority task is not recovered preferentially, and the high-priority task is possibly recovered, so that the memory service quality of the high-priority task cannot be ensured.
2. Since the scheduling priority of the low-priority task is lowest in the mixed scenario and is suppressed by the high-priority task, when the high-priority task is loaded higher or runs continuously, the low-priority task cannot be scheduled for a long time, for example, since the priority of the high-priority task is much higher than that of the low-priority task, assuming 100:1, i.e. after the high priority task runs 100 time slices, the low priority task can only acquire 1 time slice. Therefore, when the operating system executes the OOM flow, the low-priority task cannot process the SIGKILL signal for a long time, so that memory recovery is blocked, and the memory service quality of the high-priority task is difficult to guarantee.
3. The process to be recycled may be blocked due to other reasons, such as waiting for the lock to be released in the ook process, and cannot be scheduled, and the time when the process to be recycled is actually executed cannot be controlled, so that the timeliness of memory recycling cannot be guaranteed.
Therefore, in this embodiment, a method for guaranteeing the service quality of a memory in a hybrid deployment scenario is provided, which specifically includes:
on the product side, in a cloud product application scenario, as shown in fig. 4, a product application schematic diagram for mixed deployment of an online task and an offline task is shown. I.e., in the cloud host, different types of tasks are deployed in a hybrid, such as online tasks and offline tasks. The on-line task and the off-line task share the memory hardware resource of the host, and the memory is an incompressible resource, which means that when the memory of the host is insufficient and the host needs to allocate new memory, certain memory must be recovered through a certain mechanism to ensure that the host and the key task can normally operate.
As shown in FIG. 5, an application framework diagram for a hybrid deployment of online and offline tasks is shown. That is, from a task classification perspective, the application framework is a hybrid deployment of online and offline tasks on the host. The online task and the offline task respectively allocate the needed memory according to the needs. When the memory resources are sufficient, the memory resources are not interfered with each other; when the memory resources are insufficient, the cloud host needs to recover the memory, and the memory used by part of tasks is recovered, so that a new memory application request can be met. The most critical techniques in this embodiment have two points:
1. When memory is recovered under tension, how to select the recovered task.
2. How to ensure the quick recovery of the memory, thereby ensuring the new memory application request in time.
On the technical side, the overall implementation flow of the scheme in this embodiment is as follows:
1. the kernel of the operating system is modified, the memory service quality guarantee capability is enhanced, and the following functions are mainly realized:
1) Defining the memory QoS priority of different tasks, and providing a memory QoS priority setting interface for a user;
2) When the memory is recovered under the condition of insufficient memory, the cloud host recovers the memory occupied by the appointed process according to the sequence from low to high of the QoS priority of the memory;
3) And the memory occupied by the task of designating the QoS priority of the memory is quickly recovered.
2. After the host operating system and the virtual machine are started, setting an interface provided by the operating system:
1) Setting the priority of the related process of the offline task as low priority;
2) The priority of the online task related process is set to be high.
3. Memory is allocated as needed for different tasks. When the cloud host detects that the system has insufficient memory, the memory recovery process is triggered;
4. when the memory is recovered, the cloud host preferentially selects the related process of the offline task as a recovery object according to the QoS priority of the memory.
5. Triggering and accelerating the memory recovery flow, and rapidly recovering the memory occupied by the low-priority task, namely the offline task.
6. After the memory is quickly recovered, the cloud host distributes the recovered memory to the high-priority task, so that the high-priority task, namely the online task, can be quickly distributed to the required memory, and the memory service quality of the online task is ensured.
It can be understood that, in this embodiment of the present application, when the cloud host allocates memory to a task with high priority to have insufficient memory, or when the cloud host allocates memory to a task with low priority to have insufficient memory, the cloud host triggers the operating system to execute the memory reclamation process, and after triggering, the cloud host selects a process to be reclaimed that is lower than or equal to the process memory QoS priority of the process memory that triggers the memory reclamation process, and reclaims the memory occupied by the process to be reclaimed that is lower than or equal to the process memory QoS priority.
Fig. 6 is a schematic diagram of an implementation architecture of a method for guaranteeing the quality of service of a memory in a hybrid deployment scenario. The dashed box in fig. 6 is a functional module to be implemented in this embodiment, and specifically includes the following three modules:
1. memory QoS priority module
And the memory QoS priority module is used for defining and managing the memory QoS priority of the process or the process group. The priority range may be set as desired. And simultaneously, providing a setting interface of the memory QoS priority to the user for setting the memory QoS priority of the process and the process group.
2. Priority memory reclamation module
And the priority memory recycling module is used for selecting the recycled object according to the QoS priority of the memory. Specifically, when the cloud host allocates memory to a task and the system memory is insufficient, the cloud host can select the recovered objects, i.e. processes or process groups, according to the priorities of the QoS of the memories of different tasks or task groups in order from low to high.
3. Accelerating memory recovery module
And the accelerating memory recycling module is used for accelerating the memory recycling speed of the low-priority task, especially under the condition that the CPU is busy, so that the memory service quality of the high-priority task is ensured.
The following problems exist due to the general memory reclamation mechanism in the conventional manner: when the CPU is busy, when the offline task is selected as the recovered object, the SIGKILL signal is usually sent to the offline task, and due to the lower scheduling priority of the offline task, the offline task may not be scheduled for a long time, so that the offline task cannot process the SIGKILL signal for a long time, and thus the occupied memory cannot be released for a long time, and finally, the memory allocation of the online service is delayed greatly.
In order to solve the problem, the accelerated memory reclamation module is optimized together with a memory management module and a process scheduling module in the kernel of the operating system, and adds the reclaimed object into a newly built accelerated scheduling queue while sending a SIGKILL signal to an offline task in a memory reclamation process, so as to improve the scheduling priority of the reclaimed object, namely the offline task, accelerate the scheduled process of the task to be reclaimed, and ensure that the task to be reclaimed can be scheduled at the first time, thereby ensuring the timeliness of memory reclamation.
The main processing logic of the memory QoS priority module is realized as follows:
1) Setting of QoS priority of a process memory is achieved through a proc file system interface; or the QoS priority of the process group memory is set through the cgroup interface.
2) The priority setting range is 0-7, 8 levels of priority are represented respectively, and 0 represents the lowest priority for offline tasks; 7 represents the highest priority for use by the online task; it will be appreciated that each of the 0-7 stages may be used for different tasks or task groups, with the processing logic being the same between each stage, except that in the offline, online task mix scenario of this example, only two priority parameters are set.
3) When a user needs to set the QoS priority of the memory of a designated process or a process group, the designated priority parameter is written into the related interface;
4) After writing, the corresponding memory QoS priority information is stored in a data structure corresponding to a process or a process group in the kernel of the operating system;
5) And subsequently, when the memory is recovered, the operating system reads the stored memory QoS priority information in the process or the process group.
Fig. 7 is a schematic diagram of logic processing of the priority memory reclamation module. The main processing logic of the priority memory reclamation module is realized as follows:
1) In the normal running process of the system, normally distributing memories for online tasks and offline tasks;
2) Gradually reducing the residual memory of the system to a memory tension state;
3) When the online task needs to allocate new memory, the operating system enters a memory allocation flow;
4) Judging that the residual memory of the current system is insufficient, and enabling the operating system to enter a memory recovery process;
5) The operating system traverses all processes or process groups according to the memory QoS priority, and preferentially selects the process with the lowest memory QoS priority as the recovered object. If the recovered object meeting the minimum condition of the QoS priority of the memory is not found, sequentially searching the object to be recovered from the high priority;
6) If no process or process group with lower memory QoS priority than the current process, namely the online process is found, selecting the process with the highest memory occupation value from the memory QoS priorities of the same level as the current process as the object to be recovered.
FIG. 8 is a schematic diagram illustrating a logic process of the accelerated memory reclamation module. The specific implementation logic is as follows:
1) Finding a suitable recovered object, such as a low-priority task or task group, through a priority memory recovery module;
2) Sending a SIGKILL signal to the recovered object and waking up the recovered object;
3) Placing the recovered object into an acceleration scheduling queue;
4) Triggering an acceleration scheduling module to process related logic;
5) After the processing of the accelerated scheduling module is completed, the memory occupied by the recovered object is successfully recovered, and the operating system can allocate the recovered memory to the high-priority process so as to meet the memory allocation request of the high-priority process.
Fig. 9 is a schematic diagram of a logic process of the accelerated dispatch module. The specific implementation logic is as follows:
1) The recovered object is put into an acceleration recovery queue;
2) The next scheduling opportunity is temporary, such as a clock interrupt, and the accelerated scheduling module preferentially selects a scheduled object from the accelerated recovery queue;
3) When the acceleration scheduling queue is empty, maintaining the original scheduling logic;
4) When the accelerated scheduling queue is not empty, taking out an object from the accelerated scheduling queue as a task which runs on the CPU immediately;
5) The selected object is put on the core to run until the process processes logic such as SIGKILL signals and the like, and then actively exits, an operating system can close a preemption function in the SIGKILL signal processing flow to prevent preemption, namely other tasks cannot preempt a CPU during SIGKILL signal processing;
6) Every time the scheduling opportunity comes, the operating system needs to check whether the acceleration scheduling queue is empty or not preferentially so as to ensure that the recovered object is scheduled preferentially.
In addition, a new mechanism is proposed in the Linux kernel of the current new version: the oom repeater is mainly used for solving the problem of livelock or deadlock which easily occurs under high CPU and memory pressure in the original oom mechanism, and can also improve the memory recovery speed.
The basic principle of the oom repeater is as follows: when the process to be kill meets the preset condition, the selected kill process will not return to user state to execute, and the anonymous memory occupied by the selected kill process will not be accessed, so that the operating system can directly recover the anonymous memory occupied by the kill process in the context of the thread by creating a special kernel thread under the action of the oom repeater mechanism, and the aim of recovering the anonymous memory occupied by the kill process quickly is fulfilled without depending on the fact that the kill process waits to be scheduled.
However, the oom repeater mechanism has the following disadvantages compared with the methods provided in the embodiments of the present application:
1. since the oom repeater mechanism depends on other kernel threads, the execution of other kernel threads also depends on timely scheduling, and scheduling delay may exist, the timeliness of memory reclamation cannot be guaranteed basically.
2. The process of executing reclamation by the oom reflector is that in the context of a single thread, i.e. a kernel thread, when a large amount of concurrent reclamation memory is needed, i.e. when offline tasks are relatively many, the reclamation can only be performed by traversing all tasks, and thus, bottlenecks in performance and efficiency exist.
3. The oom repeater can only accelerate anonymous memory recovery of a kill process, can not accelerate the exit flow of the kill process, and can not accelerate release of locks when other flows are blocked due to locking by the kill process, so that the problem of similar deadlock or livelock can not be solved.
4. There are also limitations in the oom reader execution process, such as the need to rely on global locks, which are discarded if not taken by multiple attempts. This means that the oom repeater cannot ensure certain success, nor recovery timeliness.
5. The oom repeater can only improve the memory recovery speed to a certain extent, and is not a complete memory service quality guarantee scheme aiming at mixed scenes.
In the embodiment of the application, because the different types of tasks preset the memory service quality priority information of different levels, when the system memory is insufficient, the cloud host can select and recover the memory occupied by the processes corresponding to the tasks of different levels according to the high-low sequence of the memory service quality priority, and meanwhile, by adding the processes to be recovered into the acceleration scheduling queue, the operating system can schedule the processes to be recovered in the acceleration scheduling queue preferentially, and the scheduled flow of the acceleration low-priority task or the offline task can be realized.
It should be understood that, although the steps in the flowcharts of fig. 1-9 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in FIGS. 1-9 may include multiple steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the steps or stages in other steps or other steps.
In one embodiment, as shown in fig. 10, there is provided a memory allocation apparatus, which may use a software module or a hardware module, or a combination of both, as a part of a computer device, and the apparatus specifically includes: a receiving module 1002, a searching module 1004, a joining module 1006, a scheduling module 1008, and an assigning module 1010, wherein:
a receiving module 1002, configured to receive a memory application request for applying a memory for a target task; the first memory service quality priority of the target task is carried in the memory application request.
And a searching module 1004, configured to search a process corresponding to a task that is not higher than the first memory quality of service priority when the remaining capacity of the system memory is smaller than the memory space required by the target task.
And a joining module 1006, configured to join the process corresponding to the task into the accelerated scheduling queue.
And the scheduling module 1008 is used for preferentially scheduling the processes in the accelerated scheduling queue when the scheduling conditions are met, so that the scheduled processes terminate running and the occupied memory blocks are released.
And the allocation module 1010 is configured to allocate the released memory block to the target task.
In one embodiment, the apparatus further comprises: the selecting module is used for selecting memory blocks from the rest memory of the system memory according to the memory space required by the target task when the residual capacity of the system memory is greater than or equal to the memory space required by the target task; the allocation module is also used for allocating the selected memory blocks to the target task.
In one embodiment, the apparatus further comprises: the determining module is used for determining the second memory service quality priority of the currently running task; the comparison module is used for comparing the second memory service quality priority with the first memory service quality priority; and the searching module is also used for searching the process corresponding to the task which is not higher than the first memory service quality priority based on the comparison result in the process corresponding to the task which is currently running.
In one embodiment, the determining module is further configured to determine that the task corresponding to the second memory quality of service priority is an offline task when the second memory quality of service priority meets an offline task priority; the selecting module is also used for preferentially selecting the process of the offline task in the processes corresponding to the currently running task; and the memory service quality priority of the offline task is not higher than the first memory service quality priority.
In one embodiment, the apparatus further comprises: the acquisition module is used for acquiring the process of the offline task from the process corresponding to the currently running task when the first memory service quality priority accords with the offline task priority; the selecting module is further configured to preferentially select, in the processes of the offline task, a process with a memory occupation value greater than an occupation threshold.
In one embodiment, the selecting module is further configured to preferentially select, in a process corresponding to a task lower than the first priority, a process corresponding to a task with the second priority meeting the first priority condition; if the process corresponding to the task meeting the first priority condition is not found, selecting the process corresponding to the task with the second memory service quality priority meeting the second priority condition; and if the process corresponding to the task meeting the second priority condition is not found, selecting a process with a memory occupation value larger than an occupation threshold value from the processes corresponding to the tasks with the priority equal to the first memory service quality.
In one embodiment, the apparatus further comprises: the checking module is used for checking whether the acceleration scheduling queue is an empty queue or not; the scheduling module is further configured to schedule a process from the accelerated scheduling queue if the accelerated scheduling queue is not an empty queue, and take the scheduled process as a process executed by a processor core next, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core; the accelerated scheduling queue is used for storing the process of the memory block to be released so that the stored process is scheduled preferentially.
In one embodiment, the apparatus further comprises: the monitoring module is used for monitoring the scheduling event; the scheduling event comprises a user mode returned by a clock interrupt or a kernel mode; the checking module is further configured to, when the scheduling event is monitored, preferentially check whether the accelerated scheduling queue is an empty queue.
In one embodiment, the apparatus further comprises: a sending module, configured to send a termination signal to the scheduled process, and wake up the scheduled process based on the termination signal; taking the scheduled process as the next process executed in the processor core; and the execution module is used for executing the scheduled process through the processor core when the processor core is not preempted, so that the scheduled process processes the termination signal until the relevant logic of the termination signal is processed, and then exits, and the occupied memory block is released.
In one embodiment, the apparatus further comprises: the receiving module is also used for receiving the memory service quality priority parameters of each task through the priority setting interface; the storage module is used for storing the memory service quality priority parameters into data sets corresponding to the processes of the tasks; the acquisition module is further used for acquiring a second memory service quality priority of each task in a data set corresponding to the process of each task; the selecting module is further configured to select, in the processes of each task, a process corresponding to a task with a priority of the second memory service quality not higher than that of the first memory service quality.
For specific limitations of the memory allocation apparatus, reference may be made to the above limitation of the memory allocation method, and no further description is given here. The modules in the memory allocation device may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 11. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing memory allocation data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a memory allocation method.
It will be appreciated by those skilled in the art that the structure shown in fig. 11 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-described method embodiments.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.

Claims (15)

1. The memory allocation method is characterized by comprising the following steps:
receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task;
when the residual capacity of the system memory is smaller than the memory space required by the target task, searching a process corresponding to the task with the service quality priority not higher than the first memory;
Adding the process corresponding to the task into an acceleration scheduling queue;
when the scheduling conditions are met, preferentially scheduling the processes in the accelerated scheduling queue so that the scheduled processes terminate running and the occupied memory blocks are released;
and distributing the released memory block to the target task.
2. The method of claim 1, wherein after receiving a memory application request for applying for memory for a target task, the method further comprises:
when the residual capacity of the system memory is larger than or equal to the memory space required by the target task, selecting a memory block from the residual memory of the system memory according to the memory space required by the target task;
and distributing the selected memory block to the target task.
3. The method of claim 1, wherein the searching for a process corresponding to a task of no higher priority than the first memory quality of service comprises:
determining a second memory service quality priority of the currently running task;
comparing the second memory quality of service priority with the first memory quality of service priority;
and searching a process corresponding to the task with the service quality priority not higher than the first memory based on the comparison result in the process corresponding to the task which is currently operated.
4. The method of claim 3, wherein the searching for a process corresponding to a task that is not higher than the first quality of service priority based on the comparison result comprises:
when the second memory service quality priority accords with the offline task priority, determining that the task corresponding to the second memory service quality priority is an offline task;
preferentially selecting the process of the offline task from the processes corresponding to the currently running task; and the memory service quality priority of the offline task is not higher than the first memory service quality priority.
5. The method according to claim 4, wherein the preferentially selecting the process of the offline task from the processes corresponding to the currently running task includes:
when the first memory service quality priority accords with the offline task priority, acquiring a process of the offline task from a process corresponding to the currently running task;
and preferentially selecting a process with the memory occupation value larger than the occupation threshold value from the processes of the offline task.
6. A method according to claim 3, characterized in that the method further comprises:
Preferentially selecting the process corresponding to the task with the second memory service quality priority meeting the first priority condition from the processes corresponding to the tasks with the lower priority than the first memory service quality priority;
if the process corresponding to the task meeting the first priority condition is not found, selecting the process corresponding to the task with the second memory service quality priority meeting the second priority condition;
and if the process corresponding to the task meeting the second priority condition is not found, selecting a process with a memory occupation value larger than an occupation threshold value from the processes corresponding to the tasks with the priority equal to the first memory service quality.
7. The method of claim 1, wherein the preferentially scheduling the process in the accelerated scheduling queue to cause the scheduled process to terminate running and free up occupied memory blocks, comprises:
preferentially checking whether the acceleration scheduling queue is an empty queue;
if the acceleration scheduling queue is not an empty queue, scheduling a process from the acceleration scheduling queue, and taking the scheduled process as a next process executed in a processor core, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core; the accelerated scheduling queue is used for storing the process of the memory block to be released so that the stored process is scheduled preferentially.
8. The method of claim 7, wherein the preferentially checking whether the accelerated dispatch queue is an empty queue comprises:
monitoring a scheduling event; the scheduling event comprises a user mode returned by a clock interrupt or a kernel mode;
when the scheduling event is monitored, preferentially checking whether the accelerated scheduling queue is an empty queue.
9. The method of claim 7, wherein the method further comprises:
sending a termination signal to the scheduled process, and waking the scheduled process based on the termination signal;
the step of taking the scheduled process as the next process executed in the processor core, so that the process terminates running and releases occupied memory blocks in the execution process of the processor core, includes:
taking the scheduled process as the next process executed in the processor core;
and when the processor core is not preempted, executing the scheduled process through the processor core, so that the scheduled process processes the termination signal, exits after the relevant logic of the termination signal is processed, and releases the occupied memory block.
10. The method according to any one of claims 1 to 9, wherein prior to receiving a memory application request for applying for memory for a target task, the method further comprises:
Receiving the memory service quality priority parameters of each task through a priority setting interface;
storing the memory service quality priority parameters into data sets corresponding to the processes of the tasks;
the process corresponding to the task for searching the priority of the service quality of the first memory is not higher than the priority of the service quality of the first memory, which comprises the following steps:
acquiring a second memory service quality priority of each task in a data set corresponding to the process of each task;
and selecting a process corresponding to the task with the second memory service quality priority not higher than the first memory service quality priority from the processes of the tasks.
11. A memory allocation apparatus, the apparatus comprising:
the receiving module is used for receiving a memory application request for applying a memory for a target task; the memory application request carries a first memory service quality priority of the target task;
the searching module is used for searching a process corresponding to a task with the service quality priority not higher than the first memory when the residual capacity of the system memory is smaller than the memory space required by the target task;
the adding module is used for adding the process corresponding to the task into an acceleration scheduling queue;
The scheduling module is used for preferentially scheduling the process in the accelerated scheduling queue when the scheduling condition is met, so that the scheduled process stops running and releases the occupied memory block;
and the distribution module is used for distributing the released memory blocks to the target task.
12. The memory allocation device of claim 11, wherein said device further comprises:
the selecting module is used for selecting memory blocks from the rest memory of the system memory according to the memory space required by the target task when the residual capacity of the system memory is greater than or equal to the memory space required by the target task; the allocation module is also used for allocating the selected memory blocks to the target task.
13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 10 when the computer program is executed.
14. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 10.
15. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 10.
CN202210013752.6A 2022-01-06 2022-01-06 Memory allocation method, memory allocation device, computer equipment and storage medium Pending CN116450328A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210013752.6A CN116450328A (en) 2022-01-06 2022-01-06 Memory allocation method, memory allocation device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210013752.6A CN116450328A (en) 2022-01-06 2022-01-06 Memory allocation method, memory allocation device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116450328A true CN116450328A (en) 2023-07-18

Family

ID=87126053

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210013752.6A Pending CN116450328A (en) 2022-01-06 2022-01-06 Memory allocation method, memory allocation device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116450328A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116643892A (en) * 2023-07-25 2023-08-25 北京芯驰半导体科技有限公司 Memory management method, device, chip and traffic equipment
CN117076346A (en) * 2023-07-24 2023-11-17 龙芯中科(成都)技术有限公司 Application program data processing method and device and electronic equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117076346A (en) * 2023-07-24 2023-11-17 龙芯中科(成都)技术有限公司 Application program data processing method and device and electronic equipment
CN116643892A (en) * 2023-07-25 2023-08-25 北京芯驰半导体科技有限公司 Memory management method, device, chip and traffic equipment
CN116643892B (en) * 2023-07-25 2023-10-27 北京芯驰半导体科技有限公司 Memory management method, device, chip and traffic equipment

Similar Documents

Publication Publication Date Title
CN107038069B (en) Dynamic label matching DLMS scheduling method under Hadoop platform
CN110647394B (en) Resource allocation method, device and equipment
US8458712B2 (en) System and method for multi-level preemption scheduling in high performance processing
US7945913B2 (en) Method, system and computer program product for optimizing allocation of resources on partitions of a data processing system
US9319281B2 (en) Resource management method, resource management device, and program product
US8959515B2 (en) Task scheduling policy for limited memory systems
US8583756B2 (en) Dynamic configuration and self-tuning of inter-nodal communication resources in a database management system
US8424007B1 (en) Prioritizing tasks from virtual machines
CN108334396B (en) Data processing method and device, and resource group creation method and device
CN112527489B (en) Task scheduling method, device, equipment and computer readable storage medium
CA2780231A1 (en) Goal oriented performance management of workload utilizing accelerators
CN116450328A (en) Memory allocation method, memory allocation device, computer equipment and storage medium
US10884667B2 (en) Storage controller and IO request processing method
CN107273200B (en) Task scheduling method for heterogeneous storage
CN107515781B (en) Deterministic task scheduling and load balancing system based on multiple processors
CN112799817A (en) Micro-service resource scheduling system and method
US20190286582A1 (en) Method for processing client requests in a cluster system, a method and an apparatus for processing i/o according to the client requests
CN114546587A (en) Capacity expansion and reduction method of online image recognition service and related device
CN109150759B (en) Progressive non-blocking opportunity resource reservation method and system
CN112073532B (en) Resource allocation method and device
CN110569112B (en) Log data writing method and object storage daemon device
CN116700901A (en) Container construction and operation system and method based on microkernel
CN110659303A (en) Read-write control method and device for database nodes
CN116954816A (en) Container cluster control method, device, equipment and computer storage medium
CN115756726A (en) Container local storage intelligent scheduling and distributing method applied to cloud platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination