WO2024103927A1 - Job scheduling method and apparatus in hybrid deployment scenario, and electronic device - Google Patents

Job scheduling method and apparatus in hybrid deployment scenario, and electronic device Download PDF

Info

Publication number
WO2024103927A1
WO2024103927A1 PCT/CN2023/117543 CN2023117543W WO2024103927A1 WO 2024103927 A1 WO2024103927 A1 WO 2024103927A1 CN 2023117543 W CN2023117543 W CN 2023117543W WO 2024103927 A1 WO2024103927 A1 WO 2024103927A1
Authority
WO
WIPO (PCT)
Prior art keywords
job
offline
queue
online
ready
Prior art date
Application number
PCT/CN2023/117543
Other languages
French (fr)
Chinese (zh)
Inventor
张伟
赵小强
Original Assignee
京东科技信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东科技信息技术有限公司 filed Critical 京东科技信息技术有限公司
Publication of WO2024103927A1 publication Critical patent/WO2024103927A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present disclosure relates to the field of data processing technology, and in particular to a method, device and electronic device for scheduling jobs in a hybrid deployment scenario.
  • the present disclosure aims to solve one of the technical problems in the related art at least to some extent.
  • the present disclosure proposes a job scheduling method, device and electronic device in a hybrid deployment scenario, so as to realize the priority execution of online jobs in the ready job queue of the processor, and execute offline jobs after the online jobs are completed, so as to avoid the interference of offline jobs with online jobs, and ensure the operation efficiency and service quality of online jobs while improving resource utilization.
  • the first aspect of the present disclosure provides a method for scheduling jobs in a hybrid deployment scenario, which is applied to a CPU processor in the hybrid deployment scenario.
  • the method includes:
  • the to-be-processed job is an offline job and there is an online job in the ready job queue
  • the to-be-processed job is pushed into the restricted queue, and a to-be-processed job is reselected until the selected to-be-processed job is an online job;
  • the online job process in the selected online job is scheduled to execute the online job.
  • the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of the offline job with the online job, and ensuring the operation efficiency and service quality of the online job while improving resource utilization.
  • a second aspect of the present disclosure provides a job scheduling device in a hybrid deployment scenario, which is applied to a CPU processor in the hybrid deployment scenario.
  • the device includes:
  • a selection module configured to select a job to be processed from a ready job queue of the processor
  • a processing module is used to: when the to-be-processed job is an offline job and there is an online job in the ready job queue When the job is processed, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job;
  • the scheduling module is used to schedule the online job process selected in the online job to execute the online job.
  • the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
  • An embodiment of the third aspect of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the job scheduling method in a hybrid deployment scenario proposed in the embodiment of the first aspect of the present disclosure.
  • the fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the job scheduling method in a hybrid deployment scenario proposed by the first aspect embodiment of the present disclosure.
  • the fifth aspect embodiment of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the job scheduling method in the hybrid deployment scenario described in the first aspect embodiment of the present disclosure.
  • FIG1 is a schematic diagram of a process of a job scheduling method in a hybrid deployment scenario provided by a first embodiment of the present disclosure
  • FIG2 is a schematic diagram of a ready job queue
  • FIG3 is a schematic diagram showing how offline jobs in a restricted queue are returned to the ready job queue after online jobs in the ready job queue are completed;
  • FIG4 is a schematic diagram of a flow chart of a method for scheduling jobs in a hybrid deployment scenario provided by Embodiment 2 of the present disclosure
  • FIG5 is a schematic diagram of the structure of a job scheduling device in a hybrid deployment scenario provided by Embodiment 3 of the present disclosure
  • FIG. 6 shows a block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure.
  • the present disclosure mainly aims at the technical problems of low operating efficiency and poor service quality of online jobs in hybrid deployment scenarios in related technologies, and proposes a job scheduling method, device and electronic device in hybrid deployment scenarios.
  • the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
  • FIG1 is a flow chart of a method for scheduling jobs in a hybrid deployment scenario according to a first embodiment of the present disclosure.
  • the embodiment of the present disclosure takes the job scheduling method in the hybrid deployment scenario as an example in which the job scheduling device in the hybrid deployment scenario is configured.
  • the job scheduling device in the hybrid deployment scenario can be applied to any electronic device, or to a CPU processor in the electronic device, or to a Completely Fair Scheduler (CFS) in the CPU processor, so that the electronic device can perform the job scheduling function in the hybrid deployment scenario.
  • CFS Completely Fair Scheduler
  • the electronic device may be a personal computer (PC), a cloud device, a mobile device, etc.
  • the mobile device may be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a vehicle-mounted device, or other hardware device with various operating systems, touch screens, and/or display screens.
  • the CPU processor in the hybrid deployment scenario is used as an example for explanation.
  • the job scheduling method in the hybrid deployment scenario may include the following steps:
  • Step 101 Select a job to be processed from a ready job queue of a processor.
  • the jobs in the ready job queue are jobs to be processed or executed by the processor.
  • the CPU processor can update the ready job queue, for example, by obtaining the offline job process that has been created or awakened; when the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
  • the CPU processor can allocate resources for the online job process to run; while after an offline job process is created or awakened, the CPU processor does not allocate resources for the offline job process to run, but updates the offline job process to the offline job to which the process belongs in the ready job queue. This avoids the creation or awakening of the offline job process from affecting the running online job process.
  • the CPU processor may create a container for each job for executing the job. For example, a container is created for each online job; a container is created for each offline job. The container is used to use the resources of the CPU processor to run each job process in the corresponding job.
  • the ready job queue may be expressed in the form of a red-black tree, for example.
  • the red-black tree is a self-balancing binary search tree, a data structure used in computer science, and is typically used to implement an associative array.
  • FIG2 a schematic diagram of a ready job queue is shown.
  • the ready job queue may include online jobs and offline jobs.
  • the offline cfs-rq represents an offline job, which includes multiple job processes of the offline job; the offline job processes included therein are shown as the offline se in FIG2 .
  • the online cfs-rq represents an online job, which includes multiple job processes of the online job; the online job processes included therein are shown as the online se in FIG2 .
  • the cfs pick next class can be used to select jobs to be processed from the ready job queue according to a certain selection strategy.
  • Step 102 When the job to be processed is an offline job and there is an online job in the ready job queue, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job.
  • the characteristics of online jobs include at least one of the following characteristics: long running time, latency sensitivity, and resource tidal phenomenon.
  • the characteristics of offline jobs include at least one of the following characteristics: short running time, large computing requirements, high fault tolerance, latency insensitivity, and re-running allowed.
  • offline jobs include, for example, MapReduce jobs and Spark jobs in the Hadoop ecosystem.
  • the ready job queue is provided with a mark, and the mark indicates whether there is an online job in the ready job queue.
  • the mark indicates that there is an online job in the ready job queue; when the mark is a value other than the first value, it indicates that there is no online job in the ready job queue.
  • Step 103 Schedule the online job process in the selected online job to execute the online job.
  • the online job selected for scheduling may include multiple online job processes.
  • the CPU processor may schedule each online job process in the online job in sequence, and allocate CPU resources to the container where the online job is located for the online job process, so that the container uses the CPU resources to run the online job process. After the multiple online job processes included in the online job are completed, it is determined that the online job execution is completed.
  • the method may further include the following steps: after the online jobs in the ready job queue are executed, the offline jobs in the restricted queue are put back into the ready job queue.
  • FIG3 it is a schematic diagram of returning offline jobs in the restricted queue to the ready job queue after the online jobs in the ready job queue are executed.
  • the ready job queue in FIG3 when all online job processes in the online jobs are executed, the offline jobs in the restricted queue are put back to the ready job queue.
  • the action of returning offline jobs in the restricted queue to the ready job queue can be, for example, unthrottle.
  • the method also includes: when the job to be processed is an offline job and there is no online job in the ready job queue, scheduling the offline job process in the selected offline job to execute the offline job; or, when the job to be processed is an online job, scheduling the online job process in the selected online job to execute the online job.
  • the pending job is an online job.
  • the first case is that there is no offline job in the ready job queue.
  • the second case is that there is an offline job in the ready job queue. In the above two cases, there may be or may not be an offline job in the restricted queue.
  • the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of the offline job with the online job, and ensuring the operation efficiency and service quality of the online job while improving resource utilization.
  • FIG4 is a flow chart of a method for scheduling jobs in a hybrid deployment scenario provided in Embodiment 2 of the present disclosure.
  • the job scheduling method in the hybrid deployment scenario may further include the following steps:
  • Step 401 Select a job to be processed from the ready job queue of the processor.
  • Step 402 When the job to be processed is an offline job and there is an online job in the ready job queue, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job.
  • Step 403 Schedule the online job process in the selected online job to execute the online job.
  • Step 404 acquiring the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold.
  • the restriction duration of the offline job can be timed to obtain the timed restriction duration; when the restriction duration is greater than a preset restriction duration threshold, the offline job is taken as the first offline job.
  • Step 405 adding a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue.
  • the sleep condition may include, for example, a sleep trigger position, wherein the sleep trigger position may be a specific position of the offline job process. When the offline job process is in operation and runs to the specific position, the sleep condition is satisfied.
  • Step 406 put the processed first offline job back into the ready job queue.
  • the processor selects a job to be processed from the ready job queue, it selects the first offline job and can schedule the offline job process from the first offline job; after scheduling the offline job process and running the offline job process, if it runs to the sleep trigger position in the offline job process, the sleep condition is met and the first offline job is transferred to the sleep queue.
  • the sleep condition may also include the required sleep duration.
  • the method may also include the following steps: when there is a second offline job in the sleep queue, the second offline job is transferred to the ready job queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration.
  • the sleep queue includes multiple first offline jobs, and the multiple first offline jobs include the second offline job.
  • the sleep duration is measured; when the actual sleep duration of the first offline job is greater than the required sleep duration, the first offline job is transferred to the ready job queue as the second offline job, and the ready job queue is updated.
  • the sleep condition in the second offline job may be deleted.
  • the transfer processing of offline jobs in the restricted queue can shorten the length of the restricted queue, shorten the time spent on putting offline jobs in the restricted queue back into the ready job queue, and further improve the operating efficiency and service quality of offline jobs.
  • steps 401 to 403 may refer to the detailed description of steps 101 to 103 in the embodiment shown in FIG. 1 , and will not be described in detail here.
  • the job scheduling method in a hybrid deployment scenario of the embodiment of the present disclosure is as follows: the CPU processor in the hybrid deployment scenario selects a to-be-processed job from the ready job queue of the processor; when the to-be-processed job is an offline job and there is an online job in the ready job queue, the to-be-processed job is pushed into the restricted queue, and the to-be-processed job is reselected until the selected to-be-processed job is an online job; the online job process in the selected online job is scheduled to execute the online job; according to the restricted duration of the offline job in the restricted queue, the first offline job in the restricted queue is obtained; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold; a sleep condition is added to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue; the processed first offline job is put back into the ready job queue; thereby achieving priority execution of the online
  • FIG5 is a schematic diagram of the structure of a job scheduling device in a hybrid deployment scenario provided in Embodiment 3 of the present disclosure.
  • the job scheduling device 500 in the hybrid deployment scenario is applied to a CPU processor in the hybrid deployment scenario and may include: a selection module 510 , a processing module 520 and a scheduling module 530 .
  • the selection module 510 is used to select a job to be processed from the ready job queue of the processor
  • the processing module 520 is used for, when the to-be-processed job is an offline job and there is an online job in the ready job queue, pushing the to-be-processed job into the restricted queue and reselecting the to-be-processed job until the selected to-be-processed job is an online job;
  • the scheduling module 530 is used to schedule the online job process in the selected online job to execute the online job.
  • the processing module 520 is further configured to put the offline jobs in the restricted queue back into the ready job queue after the online jobs in the ready job queue are executed.
  • the processing module 520 is also used to, when the job to be processed is an offline job and the online job does not exist in the ready job queue, schedule the offline job process selected in the offline job to execute the offline job; or, when the job to be processed is an online job, schedule the online job process selected in the online job to execute the online job.
  • the device also includes: an acquisition module; the acquisition module is used to acquire the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold; the processing module 520 is also used to add a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue; and the processed first offline job is put back into the ready job queue.
  • an acquisition module is used to acquire the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold
  • the processing module 520 is also used to add a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue; and the processed first offline job is put back into the ready job queue.
  • the sleeping condition includes a required sleep duration
  • the processing module 520 is also used to transfer a second offline job to a ready job queue when there is a second offline job in the sleep queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration.
  • the processing module 520 is also used to obtain an offline job process that has been created or awakened; when the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
  • the ready job queue is provided with a flag, and the flag indicates whether there is an online job in the ready job queue.
  • the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
  • the present disclosure also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the job scheduling method in the hybrid deployment scenario proposed in the aforementioned embodiments of the present disclosure.
  • the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the hybrid deployment scenario proposed in the above embodiments of the present disclosure.
  • the following job scheduling method The following job scheduling method.
  • the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the job scheduling method in the hybrid deployment scenario provided in the aforementioned embodiments of the present disclosure.
  • Fig. 6 shows a block diagram of an exemplary computer device suitable for implementing the embodiments of the present disclosure.
  • the computer device 12 shown in Fig. 6 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
  • computer device 12 is in the form of a general purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 connecting various system components (including system memory 28 and processing unit 16).
  • Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures.
  • these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.
  • ISA Industry Standard Architecture
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnection
  • the computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.
  • the memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter referred to as: RAM) 30 and/or cache memory 32.
  • RAM random access memory
  • the computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 6, usually referred to as a "hard drive").
  • each drive may be connected to the bus 18 via one or more data medium interfaces.
  • the memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present disclosure.
  • a program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
  • the program modules 42 generally perform the functions and/or methods of the embodiments described in the present disclosure.
  • the computer device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), and may also communicate with one or more devices that enable a user to interact with the computer device 12, and/or Or communicate with any device (such as a network card, a modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication can be performed through an input/output (I/O) interface 22.
  • I/O input/output
  • the computer device 12 can also communicate with one or more networks (such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN) and/or a public network, such as the Internet) through a network adapter 20.
  • networks such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN) and/or a public network, such as the Internet) through a network adapter 20.
  • the network adapter 20 communicates with other modules of the computer device 12 through the bus 18.
  • other hardware and/or software modules can be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
  • the processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.
  • first and second are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
  • a feature defined as “first” or “second” may explicitly or implicitly include at least one of the features.
  • “plurality” means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
  • Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.
  • the logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses.
  • "computer-readable medium” can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses.
  • Computer-readable media include the following: an electrical connection portion with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).
  • the computer-readable medium may even be a computer-readable medium.
  • the paper or other suitable medium on which the program is printed may be obtained electronically, for example by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner as necessary, and then stored in a computer memory.
  • any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
  • PGA programmable gate array
  • FPGA field programmable gate array
  • each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module.
  • the above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Power Sources (AREA)

Abstract

A job scheduling method in a hybrid deployment scenario, comprising: selecting, from a ready job queue of a processor, a job to be processed; when the job to be processed is an offline job and an online job exists in the ready job queue, pushing the job to be processed into a limiting queue, and re-selecting a job to be processed until the selected job to be processed is an online job; and scheduling an online job process in the selected online job to execute the online job.

Description

混合部署场景下的作业调度方法、装置及电子设备Job scheduling method, device and electronic equipment in hybrid deployment scenario 技术领域Technical Field
本公开涉及数据处理技术领域，尤其涉及一种混合部署场景下的作业调度方法、装置及电子设备。The present disclosure relates to the field of data processing technology, and in particular to a method, device and electronic device for scheduling jobs in a hybrid deployment scenario.
背景技术Background technique
目前，互联网企业都拥有海量的服务器，其中大部分服务器只运行交互类延时敏感的在线业务，CPU利用率非常低。为提高服务器CPU的利用率，需要在运行中在线业务的服务器上，混合部署一些CPU消耗高且延时不敏感的离线业务。Currently, Internet companies have a large number of servers, most of which only run interactive delay-sensitive online services, and the CPU utilization rate is very low. In order to improve the utilization rate of server CPUs, it is necessary to deploy some offline services with high CPU consumption and insensitive to delays on the servers running online services.
相关技术中，在在线业务和离线业务混合部署的场景下，在线业务中在线作业的运行效率低，服务质量差。In the related art, in the scenario where online services and offline services are deployed in a mixed manner, the operation efficiency of online operations in online services is low and the service quality is poor.
发明内容Summary of the invention
本公开旨在至少在一定程度上解决相关技术中的技术问题之一。The present disclosure aims to solve one of the technical problems in the related art at least to some extent.
本公开提出一种混合部署场景下的作业调度方法、装置及电子设备，以实现优先执行处理器的就绪作业队列中的在线作业，在在线作业执行完成后，再执行离线作业，避免离线作业对在线作业的干扰，在提高资源利率用的情况下，确保在线作业的运行效率和服务质量。The present disclosure proposes a job scheduling method, device and electronic device in a hybrid deployment scenario, so as to realize the priority execution of online jobs in the ready job queue of the processor, and execute offline jobs after the online jobs are completed, so as to avoid the interference of offline jobs with online jobs, and ensure the operation efficiency and service quality of online jobs while improving resource utilization.
本公开第一方面实施例提出了一种混合部署场景下的作业调度方法，应用于混合部署场景中的CPU处理器，所述方法包括：The first aspect of the present disclosure provides a method for scheduling jobs in a hybrid deployment scenario, which is applied to a CPU processor in the hybrid deployment scenario. The method includes:
从所述处理器的就绪作业队列中选择待处理作业；Selecting a job to be processed from a ready job queue of the processor;
在所述待处理作业为离线作业，且所述就绪作业队列中存在在线作业时，将所述待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；When the to-be-processed job is an offline job and there is an online job in the ready job queue, the to-be-processed job is pushed into the restricted queue, and a to-be-processed job is reselected until the selected to-be-processed job is an online job;
调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。The online job process in the selected online job is scheduled to execute the online job.
本公开实施例的混合部署场景下的作业调度方法，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业，从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，在提高资源利率用的情况下，确保在线作业的运行效率和服务质量。According to the job scheduling method in a hybrid deployment scenario of the embodiment of the present disclosure, the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of the offline job with the online job, and ensuring the operation efficiency and service quality of the online job while improving resource utilization.
本公开第二方面实施例提出了一种混合部署场景下的作业调度装置，应用于混合部署场景中的CPU处理器，所述装置包括：A second aspect of the present disclosure provides a job scheduling device in a hybrid deployment scenario, which is applied to a CPU processor in the hybrid deployment scenario. The device includes:
选择模块，用于从所述处理器的就绪作业队列中选择待处理作业；A selection module, configured to select a job to be processed from a ready job queue of the processor;
处理模块，用于在所述待处理作业为离线作业，且所述就绪作业队列中存在在线 作业时，将所述待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；A processing module is used to: when the to-be-processed job is an offline job and there is an online job in the ready job queue When the job is processed, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job;
调度模块，用于调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。The scheduling module is used to schedule the online job process selected in the online job to execute the online job.
本公开实施例的混合部署场景下的作业调度装置，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业，从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，在提高资源利率用的情况下，确保在线作业的运行效率和服务质量。In the job scheduling device under the hybrid deployment scenario of the embodiment of the present disclosure, the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
本公开第三方面实施例提出了一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本公开第一方面实施例提出的混合部署场景下的作业调度方法。An embodiment of the third aspect of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the job scheduling method in a hybrid deployment scenario proposed in the embodiment of the first aspect of the present disclosure.
本公开第四方面实施例提出了一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行本公开第一方面实施例提出的混合部署场景下的作业调度方法。The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the job scheduling method in a hybrid deployment scenario proposed by the first aspect embodiment of the present disclosure.
本公开第五方面实施例提出了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现本公开第一方面实施例所述的混合部署场景下的作业调度方法。The fifth aspect embodiment of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the job scheduling method in the hybrid deployment scenario described in the first aspect embodiment of the present disclosure.
本公开附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本公开的实践了解到。Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
本公开上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：The above and/or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
图1为本公开实施例一所提供的混合部署场景下的作业调度方法的流程示意图；FIG1 is a schematic diagram of a process of a job scheduling method in a hybrid deployment scenario provided by a first embodiment of the present disclosure;
图2为就绪作业队列的示意图；FIG2 is a schematic diagram of a ready job queue;
图3为就绪作业队列中的在线作业执行完成后，限制队列中离线作业返回就绪作业队列的示意图；FIG3 is a schematic diagram showing how offline jobs in a restricted queue are returned to the ready job queue after online jobs in the ready job queue are completed;
图4为本公开实施例二所提供的混合部署场景下的作业调度方法的流程示意图；FIG4 is a schematic diagram of a flow chart of a method for scheduling jobs in a hybrid deployment scenario provided by Embodiment 2 of the present disclosure;
图5为本公开实施例三所提供的混合部署场景下的作业调度装置的结构示意图；FIG5 is a schematic diagram of the structure of a job scheduling device in a hybrid deployment scenario provided by Embodiment 3 of the present disclosure;
图6示出了适于用来实现本公开实施方式的示例性计算机设备的框图。FIG. 6 shows a block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure.
具体实施方式Detailed ways
下面详细描述本公开的实施例，所述实施例的示例在附图中示出，其中自始至终 相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本公开，而不能理解为对本公开的限制。Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein The same or similar reference numerals represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present disclosure, but should not be construed as limiting the present disclosure.
目前，互联网企业都拥有海量的服务器，其中大部分服务器只运行交互类延时敏感的在线业务，CPU利用率非常低。为提高服务器CPU的利用率，需要在运行中在线业务的服务器上，混合部署一些CPU消耗高且延时不敏感的离线业务。Currently, Internet companies have a large number of servers, most of which only run interactive delay-sensitive online services, and the CPU utilization rate is very low. In order to improve the utilization rate of server CPUs, it is necessary to deploy some offline services with high CPU consumption and insensitive to delays on the servers running online services.
相关技术中，在在线业务和离线业务混合部署的场景下，在线业务中在线作业的运行效率低，服务质量差。In the related art, in the scenario where online services and offline services are deployed in a mixed manner, the operation efficiency of online operations in online services is low and the service quality is poor.
因此，本公开主要针对相关技术中混合部署场景下在线作业的运行效率低，服务质量差的技术问题，提出一种混合部署场景下的作业调度方法、装置及电子设备。Therefore, the present disclosure mainly aims at the technical problems of low operating efficiency and poor service quality of online jobs in hybrid deployment scenarios in related technologies, and proposes a job scheduling method, device and electronic device in hybrid deployment scenarios.
本公开实施例的混合部署场景下的作业调度方法，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业，从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，在提高资源利率用的情况下，确保在线作业的运行效率和服务质量。According to the job scheduling method in a hybrid deployment scenario of the embodiment of the present disclosure, the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
下面参考附图描述本公开实施例的混合部署场景下的作业调度方法、装置及电子设备。The following describes a job scheduling method, device, and electronic device in a hybrid deployment scenario according to an embodiment of the present disclosure with reference to the accompanying drawings.
图1为本公开实施例一所提供的混合部署场景下的作业调度方法的流程示意图。FIG1 is a flow chart of a method for scheduling jobs in a hybrid deployment scenario according to a first embodiment of the present disclosure.
本公开实施例以该混合部署场景下的作业调度方法被配置于混合部署场景下的作业调度装置中来举例说明，该混合部署场景下的作业调度装置可以应用于任一电子设备中，或者，应用于电子设备中的CPU处理器中，或者应用于CPU处理器中的完全公平调度器(Completely Fair Scheduler，CFS)，以使该电子设备可以执行混合部署场景下的作业调度功能。The embodiment of the present disclosure takes the job scheduling method in the hybrid deployment scenario as an example in which the job scheduling device in the hybrid deployment scenario is configured. The job scheduling device in the hybrid deployment scenario can be applied to any electronic device, or to a CPU processor in the electronic device, or to a Completely Fair Scheduler (CFS) in the CPU processor, so that the electronic device can perform the job scheduling function in the hybrid deployment scenario.
其中，电子设备可以为个人电脑(Personal Computer，简称PC)、云端设备、移动设备等，移动设备例如可以为手机、平板电脑、个人数字助理、穿戴式设备、车载设备等具有各种操作***、触摸屏和/或显示屏的硬件设备。其中，以下实施例中，以混合部署场景下的作业调度装置为混合部署场景中的CPU处理器为例进行说明。The electronic device may be a personal computer (PC), a cloud device, a mobile device, etc. The mobile device may be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a vehicle-mounted device, or other hardware device with various operating systems, touch screens, and/or display screens. In the following embodiments, the CPU processor in the hybrid deployment scenario is used as an example for explanation.
如图1所示，该混合部署场景下的作业调度方法可以包括以下步骤：As shown in FIG1 , the job scheduling method in the hybrid deployment scenario may include the following steps:
步骤101，从处理器的就绪作业队列中选择待处理作业。Step 101 : Select a job to be processed from a ready job queue of a processor.
本公开实施例中，就绪作业队列中的作业为处理器待处理或者待执行的作业。相关技术中，不管离线作业进程还是在线作业进程，在进程创建或者唤醒后，会占用CPU处理器的部分资源以运行，有可能会造成当前正在运行的进程的中断。为了避免离线作业进程对在线作业进程的上述影响，本公开实施例中，CPU处理器对就绪作业队列的更新过程例如可以为，获取已创建或者已唤醒的离线作业进程；在离线作业进程所属的离线作业位于就绪作业队列中时，将离线作业进程更新到就绪作业队列中所属的离线作业中。 In the embodiment of the present disclosure, the jobs in the ready job queue are jobs to be processed or executed by the processor. In the related art, no matter it is an offline job process or an online job process, after the process is created or awakened, it will occupy part of the resources of the CPU processor to run, which may cause the interruption of the currently running process. In order to avoid the above-mentioned influence of the offline job process on the online job process, in the embodiment of the present disclosure, the CPU processor can update the ready job queue, for example, by obtaining the offline job process that has been created or awakened; when the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
也就是说，在本公开实施例中，在线作业进程在创建或者唤醒后，CPU处理器可以为该在线作业进程分配资源以运行；而离线作业进程在创建或者唤醒后，CPU处理器不为该离线作业进程分配资源以运行，而是将该离线作业进程更新到就绪作业队列中该进程所属的离线作业中。从而避免离线作业进程的创建或者唤醒，对正在运行的在线作业进程的影响。That is, in the embodiment of the present disclosure, after an online job process is created or awakened, the CPU processor can allocate resources for the online job process to run; while after an offline job process is created or awakened, the CPU processor does not allocate resources for the offline job process to run, but updates the offline job process to the offline job to which the process belongs in the ready job queue. This avoids the creation or awakening of the offline job process from affecting the running online job process.
本公开实施例中，CPU处理器可以针对每个作业，创建一个容器，用于该作业的执行。例如，针对每个在线作业，创建一个容器；针对每个离线作业，创建一个容器。其中，容器用于利用CPU处理器的资源运行对应作业中的各个作业进程。In the disclosed embodiment, the CPU processor may create a container for each job for executing the job. For example, a container is created for each online job; a container is created for each offline job. The container is used to use the resources of the CPU processor to run each job process in the corresponding job.
在本公开实施例中，就绪作业队列的表现形式例如可以为红黑树(Red Black Tree)。其中，红黑树是一种自平衡二叉查找树，是在计算机科学中用到的一种数据结构，典型的用途是实现关联数组。如图2所示，为就绪作业队列的示意图。在图2中，就绪作业队列可以包括在线作业和离线作业。离线cfs-rq表示离线作业，其中包括离线作业的多个作业进程；其中包括的离线作业进程如图2中的离线se所示。在线cfs-rq表示在线作业，其中包括在线作业的多个作业进程；其中包括的在线作业进程如图2中的在线se所示。In the disclosed embodiment, the ready job queue may be expressed in the form of a red-black tree, for example. The red-black tree is a self-balancing binary search tree, a data structure used in computer science, and is typically used to implement an associative array. As shown in FIG2 , a schematic diagram of a ready job queue is shown. In FIG2 , the ready job queue may include online jobs and offline jobs. The offline cfs-rq represents an offline job, which includes multiple job processes of the offline job; the offline job processes included therein are shown as the offline se in FIG2 . The online cfs-rq represents an online job, which includes multiple job processes of the online job; the online job processes included therein are shown as the online se in FIG2 .
本公开实施例中，可以通过cfs pick next类按照一定的选择策略从就绪作业队列中选择待处理作业。In the disclosed embodiment, the cfs pick next class can be used to select jobs to be processed from the ready job queue according to a certain selection strategy.
步骤102，在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业。Step 102: When the job to be processed is an offline job and there is an online job in the ready job queue, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job.
本公开实施例中，在线作业的特征包括以下特征中的至少一种：运行时间长、时延敏感、存在资源潮汐现象。离线作业的特征包括以下特征中的至少一种：运行时间短、计算需求大、容错率高，时延不敏感、允许重运行。其中，离线作业例如，Hadoop生态下的MapReduce作业、Spark作业等。In the disclosed embodiment, the characteristics of online jobs include at least one of the following characteristics: long running time, latency sensitivity, and resource tidal phenomenon. The characteristics of offline jobs include at least one of the following characteristics: short running time, large computing requirements, high fault tolerance, latency insensitivity, and re-running allowed. Among them, offline jobs include, for example, MapReduce jobs and Spark jobs in the Hadoop ecosystem.
其中，需要说明的是，就绪作业队列设置有标记，标记表征所述就绪作业队列中是否存在在线作业。其中，在标记为第一数值时，表征就绪作业队列中存在在线作业；在标记为除第一数值之外的其他数值时，表征就绪作业队列中不存在在线作业。It should be noted that the ready job queue is provided with a mark, and the mark indicates whether there is an online job in the ready job queue. When the mark is a first value, it indicates that there is an online job in the ready job queue; when the mark is a value other than the first value, it indicates that there is no online job in the ready job queue.
其中，以就绪作业队列为图2中所示为例，在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，例如，将待处理作业通过throttle动作放入percpu throttle list中。在图2中，percpu throttle list表示限制队列。Among them, taking the ready job queue as shown in Figure 2 as an example, when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, for example, the pending job is put into the percpu throttle list through the throttle action. In Figure 2, percpu throttle list represents the restricted queue.
步骤103，调度选择的在线作业中的在线作业进程，以执行在线作业。Step 103: Schedule the online job process in the selected online job to execute the online job.
本公开实施例中，调度选择的在线作业中可以包括多个在线作业进程。CPU处理器可以依次调度该在线作业中的各个在线作业进程，针对在线作业进程为在线作业所在的容器分配CPU资源，以使容器利用CPU资源运行在线作业进程。在在线作业中包括的多个在线作业进程运行完成后，确定在线作业执行完成。In the embodiment of the present disclosure, the online job selected for scheduling may include multiple online job processes. The CPU processor may schedule each online job process in the online job in sequence, and allocate CPU resources to the container where the online job is located for the online job process, so that the container uses the CPU resources to run the online job process. After the multiple online job processes included in the online job are completed, it is determined that the online job execution is completed.
本公开实施例中，在步骤103之后，所述的方法还可以包括以下步骤：在所述就绪作业队列中的在线作业执行完成后，将限制队列中的离线作业放回至就绪作业队列中。 In the embodiment of the present disclosure, after step 103, the method may further include the following steps: after the online jobs in the ready job queue are executed, the offline jobs in the restricted queue are put back into the ready job queue.
其中，如图3所示，为就绪作业队列中的在线作业执行完成后，限制队列中离线作业返回就绪作业队列的示意图。在图3中的就绪作业队列中，在线作业中的所有在线作业进程全部执行完成，则将限制队列中的离线作业放回至就绪作业队列中。其中，限制队列中离线作业返回就绪作业队列的动作例如可以为unthrottle。As shown in FIG3, it is a schematic diagram of returning offline jobs in the restricted queue to the ready job queue after the online jobs in the ready job queue are executed. In the ready job queue in FIG3, when all online job processes in the online jobs are executed, the offline jobs in the restricted queue are put back to the ready job queue. The action of returning offline jobs in the restricted queue to the ready job queue can be, for example, unthrottle.
本公开实施例中，所述方法还包括：在待处理作业为离线作业且就绪作业队列中未存在在线作业时，调度选择的离线作业中的离线作业进程，以执行所述离线作业；或者，在待处理作业为在线作业时，调度选择的在线作业中的在线作业进程，以执行在线作业。In an embodiment of the present disclosure, the method also includes: when the job to be processed is an offline job and there is no online job in the ready job queue, scheduling the offline job process in the selected offline job to execute the offline job; or, when the job to be processed is an online job, scheduling the online job process in the selected online job to execute the online job.
其中，待处理作业为在线作业，有两种情况。第一种情况为就绪作业队列中未存在离线作业。第二种情况为就绪作业队列中存在离线作业。上述两种情况中，限制队列中可以存在离线作业或者不存在离线作业。There are two cases where the pending job is an online job. The first case is that there is no offline job in the ready job queue. The second case is that there is an offline job in the ready job queue. In the above two cases, there may be or may not be an offline job in the restricted queue.
本公开实施例的混合部署场景下的作业调度方法，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业，从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，在提高资源利用率的情况下，确保在线作业的运行效率和服务质量。According to the job scheduling method in a hybrid deployment scenario of the embodiment of the present disclosure, the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of the offline job with the online job, and ensuring the operation efficiency and service quality of the online job while improving resource utilization.
图4为本公开实施例二所提供的混合部署场景下的作业调度方法的流程示意图。FIG4 is a flow chart of a method for scheduling jobs in a hybrid deployment scenario provided in Embodiment 2 of the present disclosure.
如图4所示，在图1所示实施例的基础上，该混合部署场景下的作业调度方法还可以包括以下步骤：As shown in FIG4 , based on the embodiment shown in FIG1 , the job scheduling method in the hybrid deployment scenario may further include the following steps:
步骤401，从处理器的就绪作业队列中选择待处理作业。Step 401: Select a job to be processed from the ready job queue of the processor.
步骤402，在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业。Step 402: When the job to be processed is an offline job and there is an online job in the ready job queue, the job to be processed is pushed into the restricted queue, and the job to be processed is reselected until the selected job to be processed is an online job.
步骤403，调度选择的在线作业中的在线作业进程，以执行在线作业。Step 403: Schedule the online job process in the selected online job to execute the online job.
步骤404，根据限制队列中离线作业的限制时长，获取限制队列中的第一离线作业；其中，第一离线作业的限制时长大于预设限制时长阈值。Step 404: acquiring the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold.
本公开实施例中，在将离线作业压入限制队列时，可以对离线作业的限制时长进行计时，获取计时得到的限制时长；在限制时长大于预设限制时长阈值时，将该离线作业作为第一离线作业。In the embodiment of the present disclosure, when an offline job is pushed into a restriction queue, the restriction duration of the offline job can be timed to obtain the timed restriction duration; when the restriction duration is greater than a preset restriction duration threshold, the offline job is taken as the first offline job.
步骤405，在第一离线作业的离线作业进程中添加睡眠条件，得到处理后的第一离线作业；其中，睡眠条件满足时，第一离线作业转移至睡眠队列中。Step 405, adding a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue.
其中，睡眠条件例如可以包括：睡眠触发位置，其中，睡眠触发位置可以为离线作业进程的特定位置。当离线作业进程处于运行中，且运行至该特定位置时，睡眠条件满足。The sleep condition may include, for example, a sleep trigger position, wherein the sleep trigger position may be a specific position of the offline job process. When the offline job process is in operation and runs to the specific position, the sleep condition is satisfied.
步骤406，将处理后的第一离线作业放回至就绪作业队列中。Step 406: put the processed first offline job back into the ready job queue.
本公开实施例中，将处理后的第一离线作业放回至就绪作业队列中后，若CPU处 理器从就绪作业队列中选择待处理作业时，选择到该第一离线作业，可以从该第一离线作业中调度离线作业进程；调度离线作业进程并运行该离线作业进程后，若运行至该离线作业进程中的睡眠触发位置，则满足睡眠条件，将该第一离线作业转移至睡眠队列中。In the embodiment of the present disclosure, after the processed first offline job is placed back into the ready job queue, if the CPU is When the processor selects a job to be processed from the ready job queue, it selects the first offline job and can schedule the offline job process from the first offline job; after scheduling the offline job process and running the offline job process, if it runs to the sleep trigger position in the offline job process, the sleep condition is met and the first offline job is transferred to the sleep queue.
在本公开实施例中，睡眠条件中还可以包括需求睡眠时长。对应的，所述方法还可以包括以下步骤：在睡眠队列中存在第二离线作业时，将第二离线作业转移至就绪作业队列中；其中，第二离线作业的实际睡眠时长大于或者等于需求睡眠时长。其中，睡眠队列中包括多个第一离线作业，多个第一离线作业中包括所述第二离线作业。In the embodiment of the present disclosure, the sleep condition may also include the required sleep duration. Correspondingly, the method may also include the following steps: when there is a second offline job in the sleep queue, the second offline job is transferred to the ready job queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration. The sleep queue includes multiple first offline jobs, and the multiple first offline jobs include the second offline job.
其中，针对睡眠队列中的第一离线作业，在该第一离线作业转移至睡眠队列中时，进行睡眠时长计时；在该第一离线作业的实际睡眠时长大于需求睡眠时长时，将该第一离线作业作为第二离线作业，转移至就绪作业队列中，对就绪作业队列的更新处理。Among them, for the first offline job in the sleep queue, when the first offline job is transferred to the sleep queue, the sleep duration is measured; when the actual sleep duration of the first offline job is greater than the required sleep duration, the first offline job is transferred to the ready job queue as the second offline job, and the ready job queue is updated.
其中，需要说明的是，在将第二离线作业转移至就绪作业队列中时，可以删除第二离线作业中的睡眠条件。It should be noted that when the second offline job is transferred to the ready job queue, the sleep condition in the second offline job may be deleted.
其中，对限制队列中离线作业的转移处理，可以缩短限制队列的长度，缩短将限制队列中离线作业放回至就绪作业队列中的耗时，进一步提高离线作业的运行效率和服务质量。Among them, the transfer processing of offline jobs in the restricted queue can shorten the length of the restricted queue, shorten the time spent on putting offline jobs in the restricted queue back into the ready job queue, and further improve the operating efficiency and service quality of offline jobs.
其中，步骤401至步骤403的描述，可以参考图1所示实施例中步骤101至步骤103的详细描述，此处不再进行详细说明。The description of steps 401 to 403 may refer to the detailed description of steps 101 to 103 in the embodiment shown in FIG. 1 , and will not be described in detail here.
本公开实施例的混合部署场景下的作业调度方法，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业；根据限制队列中离线作业的限制时长，获取限制队列中的第一离线作业；其中，第一离线作业的限制时长大于预设限制时长阈值；在第一离线作业的离线作业进程中添加睡眠条件，得到处理后的第一离线作业；其中，睡眠条件满足时，第一离线作业转移至睡眠队列中；将处理后的第一离线作业放回至就绪作业队列中；从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，提高在线作业的运行效率和服务质量，且确保离线作业的运行效率和服务质量。The job scheduling method in a hybrid deployment scenario of the embodiment of the present disclosure is as follows: the CPU processor in the hybrid deployment scenario selects a to-be-processed job from the ready job queue of the processor; when the to-be-processed job is an offline job and there is an online job in the ready job queue, the to-be-processed job is pushed into the restricted queue, and the to-be-processed job is reselected until the selected to-be-processed job is an online job; the online job process in the selected online job is scheduled to execute the online job; according to the restricted duration of the offline job in the restricted queue, the first offline job in the restricted queue is obtained; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold; a sleep condition is added to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue; the processed first offline job is put back into the ready job queue; thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of the offline job with the online job, improving the operating efficiency and service quality of the online job, and ensuring the operating efficiency and service quality of the offline job.
图5为本公开实施例三所提供的混合部署场景下的作业调度装置的结构示意图。FIG5 is a schematic diagram of the structure of a job scheduling device in a hybrid deployment scenario provided in Embodiment 3 of the present disclosure.
如图5所示，该混合部署场景下的作业调度装置500，应用于混合部署场景中的CPU处理器，可以包括：选择模块510、处理模块520和调度模块530。As shown in FIG. 5 , the job scheduling device 500 in the hybrid deployment scenario is applied to a CPU processor in the hybrid deployment scenario and may include: a selection module 510 , a processing module 520 and a scheduling module 530 .
其中，选择模块510，用于从所述处理器的就绪作业队列中选择待处理作业；The selection module 510 is used to select a job to be processed from the ready job queue of the processor;
处理模块520，用于在所述待处理作业为离线作业，且所述就绪作业队列中存在在线作业时，将所述待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；The processing module 520 is used for, when the to-be-processed job is an offline job and there is an online job in the ready job queue, pushing the to-be-processed job into the restricted queue and reselecting the to-be-processed job until the selected to-be-processed job is an online job;
调度模块530，用于调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。 The scheduling module 530 is used to schedule the online job process in the selected online job to execute the online job.
进一步地，在本公开实施例的一种可能的实现方式中，所述处理模块520还用于，在所述就绪作业队列中的在线作业执行完成后，将所述限制队列中的离线作业放回至所述就绪作业队列中。Furthermore, in a possible implementation of the embodiment of the present disclosure, the processing module 520 is further configured to put the offline jobs in the restricted queue back into the ready job queue after the online jobs in the ready job queue are executed.
进一步地，在本公开实施例的一种可能的实现方式中，所述处理模块520还用于，在所述待处理作业为离线作业且所述就绪作业队列中未存在所述在线作业时，调度选择的所述离线作业中的离线作业进程，以执行所述离线作业；或者，在所述待处理作业为在线作业时，调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。Furthermore, in a possible implementation of the embodiment of the present disclosure, the processing module 520 is also used to, when the job to be processed is an offline job and the online job does not exist in the ready job queue, schedule the offline job process selected in the offline job to execute the offline job; or, when the job to be processed is an online job, schedule the online job process selected in the online job to execute the online job.
进一步地，在本公开实施例的一种可能的实现方式中，所述装置还包括：获取模块；所述获取模块，用于根据所述限制队列中离线作业的限制时长，获取所述限制队列中的第一离线作业；其中，所述第一离线作业的限制时长大于预设限制时长阈值；所述处理模块520，还用于在所述第一离线作业的离线作业进程中添加睡眠条件，得到处理后的第一离线作业；其中，所述睡眠条件满足时，所述第一离线作业转移至睡眠队列中；将所述处理后的第一离线作业放回至所述就绪作业队列中。Furthermore, in a possible implementation of the embodiment of the present disclosure, the device also includes: an acquisition module; the acquisition module is used to acquire the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold; the processing module 520 is also used to add a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue; and the processed first offline job is put back into the ready job queue.
进一步地，在本公开实施例的一种可能的实现方式中，所述睡眠条件中包括需求睡眠时长，所述处理模块520还用于，在所述睡眠队列中存在第二离线作业时，将所述第二离线作业转移至就绪作业队列中；其中，所述第二离线作业的实际睡眠时长大于或者等于所述需求睡眠时长。Furthermore, in a possible implementation of the embodiment of the present disclosure, the sleeping condition includes a required sleep duration, and the processing module 520 is also used to transfer a second offline job to a ready job queue when there is a second offline job in the sleep queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration.
进一步地，在本公开实施例的一种可能的实现方式中，所述处理模块520还用于，获取已创建或者已唤醒的离线作业进程；在所述离线作业进程所属的离线作业位于所述就绪作业队列中时，将所述离线作业进程更新到所述就绪作业队列中所属的离线作业中。Furthermore, in a possible implementation of the embodiment of the present disclosure, the processing module 520 is also used to obtain an offline job process that has been created or awakened; when the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
进一步地，在本公开实施例的一种可能的实现方式中，所述就绪作业队列设置有标记，所述标记表征所述就绪作业队列中是否存在在线作业。Furthermore, in a possible implementation manner of the embodiment of the present disclosure, the ready job queue is provided with a flag, and the flag indicates whether there is an online job in the ready job queue.
需要说明的是，前述实施例一中的解释说明也适用于该实施例的混合部署场景下的作业调度装置，此处不再赘述。It should be noted that the explanations in the aforementioned embodiment 1 are also applicable to the job scheduling device in the hybrid deployment scenario of this embodiment, and will not be repeated here.
本公开实施例的混合部署场景下的作业调度装置，混合部署场景中的CPU处理器通过从处理器的就绪作业队列中选择待处理作业；在待处理作业为离线作业，且就绪作业队列中存在在线作业时，将待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；调度选择的在线作业中的在线作业进程，以执行在线作业，从而实现优先执行处理器的就绪作业队列中的在线作业，避免离线作业对在线作业的干扰，在提高资源利率用的情况下，确保在线作业的运行效率和服务质量。In the job scheduling device under the hybrid deployment scenario of the embodiment of the present disclosure, the CPU processor in the hybrid deployment scenario selects a pending job from the ready job queue of the processor; when the pending job is an offline job and there is an online job in the ready job queue, the pending job is pushed into the restricted queue, and the pending job is reselected until the selected pending job is an online job; the online job process in the selected online job is scheduled to execute the online job, thereby achieving priority execution of the online job in the ready job queue of the processor, avoiding interference of offline jobs with online jobs, and ensuring the operating efficiency and service quality of online jobs while improving resource utilization.
为了实现上述实施例，本公开还提出一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本公开前述实施例提出的混合部署场景下的作业调度方法。In order to implement the above embodiments, the present disclosure also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the job scheduling method in the hybrid deployment scenario proposed in the aforementioned embodiments of the present disclosure.
为了实现上述实施例，本公开还提出存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行如本公开前述实施例提出的混合部署场景 下的作业调度方法。In order to implement the above embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the hybrid deployment scenario proposed in the above embodiments of the present disclosure. The following job scheduling method.
本公开实施例还提出一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现本公开前述实施例提出的混合部署场景下的作业调度方法。The embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the job scheduling method in the hybrid deployment scenario provided in the aforementioned embodiments of the present disclosure.
图6示出了适于用来实现本公开实施方式的示例性计算机设备的框图。图6显示的计算机设备12仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。Fig. 6 shows a block diagram of an exemplary computer device suitable for implementing the embodiments of the present disclosure. The computer device 12 shown in Fig. 6 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
如图6所示，计算机设备12以通用计算设备的形式表现。计算机设备12的组件可以包括但不限于：一个或者多个处理器或者处理单元16，***存储器28，连接不同***组件(包括***存储器28和处理单元16)的总线18。6, computer device 12 is in the form of a general purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 connecting various system components (including system memory 28 and processing unit 16).
总线18表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，***总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(Industry Standard Architecture；以下简称：ISA)总线，微通道体系结构(Micro Channel Architecture；以下简称：MAC)总线，增强型ISA总线、视频电子标准协会(Video Electronics Standards Association；以下简称：VESA)局域总线以及***组件互连(Peripheral Component Interconnection；以下简称：PCI)总线。Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.
计算机设备12典型地包括多种计算机***可读介质。这些介质可以是任何能够被计算机设备12访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.
存储器28可以包括易失性存储器形式的计算机***可读介质，例如随机存取存储器(Random Access Memory；以下简称：RAM)30和/或高速缓存存储器32。计算机设备12可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机***存储介质。仅作为举例，存储***34可以用于读写不可移动的、非易失性磁介质(图6未显示，通常称为“硬盘驱动器”)。尽管图6中未示出，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如：光盘只读存储器(Compact Disc Read Only Memory；以下简称：CD-ROM)、数字多功能只读光盘(Digital Video Disc Read Only Memory；以下简称：DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线18相连。存储器28可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本公开各实施例的功能。The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter referred to as: RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 6, usually referred to as a "hard drive"). Although not shown in FIG. 6, a disk drive for reading and writing removable non-volatile disks (such as "floppy disks"), and an optical disk drive for reading and writing removable non-volatile optical disks (such as compact disc read only memory (hereinafter referred to as: CD-ROM), digital versatile disc read only memory (hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present disclosure.
具有一组(至少一个)程序模块42的程序/实用工具40，可以存储在例如存储器28中，这样的程序模块42包括但不限于操作***、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块42通常执行本公开所描述的实施例中的功能和/或方法。A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and/or methods of the embodiments described in the present disclosure.
计算机设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信，还可与一个或者多个使得用户能与该计算机设备12交互的设备通信，和/ 或与使得该计算机设备12能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且，计算机设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network；以下简称：LAN)，广域网(Wide Area Network；以下简称：WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器20通过总线18与计算机设备12的其它模块通信。应当明白，尽管图中未示出，可以结合计算机设备12使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID***、磁带驱动器以及数据备份存储***等。The computer device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), and may also communicate with one or more devices that enable a user to interact with the computer device 12, and/or Or communicate with any device (such as a network card, a modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication can be performed through an input/output (I/O) interface 22. In addition, the computer device 12 can also communicate with one or more networks (such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN) and/or a public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although not shown in the figure, other hardware and/or software modules can be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
处理单元16通过运行存储在***存储器28中的程序，从而执行各种功能应用以及数据处理，例如实现前述实施例中提及的方法。The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本公开的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本公开的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本公开的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本公开的实施例所属技术领域的技术人员所理解。Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行***、装置或设备(如基于计算机的***、包括处理器的***或其他可以从指令执行***、装置或设备取指令并执行指令的***)使用，或结合这些指令执行***、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行***、装置或设备或结合这些指令执行***、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是 可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。The logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a computer-readable medium. The paper or other suitable medium on which the program is printed may be obtained electronically, for example by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner as necessary, and then stored in a computer memory.
应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行***执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
此外，在本公开各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本公开的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本公开的限制，本领域的普通技术人员在本公开的范围内可以对上述实施例进行变化、修改、替换和变型。 The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims (17)

  1. 一种混合部署场景下的作业调度方法，应用于混合部署场景中的CPU处理器，所述方法包括：A job scheduling method in a hybrid deployment scenario is applied to a CPU processor in the hybrid deployment scenario, the method comprising:
    从所述处理器的就绪作业队列中选择待处理作业；Selecting a job to be processed from a ready job queue of the processor;
    在所述待处理作业为离线作业，且所述就绪作业队列中存在在线作业时，将所述待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；When the to-be-processed job is an offline job and there is an online job in the ready job queue, the to-be-processed job is pushed into the restricted queue, and a to-be-processed job is reselected until the selected to-be-processed job is an online job;
    调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。The online job process in the selected online job is scheduled to execute the online job.
  2. 根据权利要求1所述的方法，还包括：The method according to claim 1, further comprising:
    在所述就绪作业队列中的在线作业执行完成后，将所述限制队列中的离线作业放回至所述就绪作业队列中。After the online jobs in the ready job queue are executed, the offline jobs in the restricted queue are put back into the ready job queue.
  3. 根据权利要求1或2所述的方法，还包括：The method according to claim 1 or 2, further comprising:
    在所述待处理作业为离线作业且所述就绪作业队列中未存在所述在线作业时，调度选择的所述离线作业中的离线作业进程，以执行所述离线作业；When the to-be-processed job is an offline job and the online job does not exist in the ready job queue, scheduling an offline job process in the selected offline job to execute the offline job;
    或者，or,
    在所述待处理作业为在线作业时，调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。When the job to be processed is an online job, an online job process selected from the online job is scheduled to execute the online job.
  4. 根据权利要求1至3中任一项所述的方法，还包括：The method according to any one of claims 1 to 3, further comprising:
    根据所述限制队列中离线作业的限制时长，获取所述限制队列中的第一离线作业；其中，所述第一离线作业的限制时长大于预设限制时长阈值；According to the limited duration of the offline jobs in the limited queue, obtaining the first offline job in the limited queue; wherein the limited duration of the first offline job is greater than a preset limited duration threshold;
    在所述第一离线作业的离线作业进程中添加睡眠条件，得到处理后的第一离线作业；其中，所述睡眠条件满足时，所述第一离线作业转移至睡眠队列中；Adding a sleep condition to the offline job process of the first offline job to obtain a processed first offline job; wherein when the sleep condition is met, the first offline job is transferred to the sleep queue;
    将所述处理后的第一离线作业放回至所述就绪作业队列中。Put the processed first offline job back into the ready job queue.
  5. 根据权利要求4所述的方法，其中，所述睡眠条件中包括需求睡眠时长，所述方法还包括：The method according to claim 4, wherein the sleep condition includes a required sleep duration, and the method further comprises:
    在所述睡眠队列中存在第二离线作业时，将所述第二离线作业转移至所述就绪作业队列中；其中，所述第二离线作业的实际睡眠时长大于或者等于所述需求睡眠时长。When there is a second offline job in the sleep queue, the second offline job is transferred to the ready job queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration.
  6. 根据权利要求1至5中任一项所述的方法，还包括：The method according to any one of claims 1 to 5, further comprising:
    获取已创建或者已唤醒的离线作业进程；Get the created or awakened offline job process;
    在所述离线作业进程所属的离线作业位于所述就绪作业队列中时，将所述离线作业进程更新到所述就绪作业队列中所属的离线作业中。 When the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
  7. 根据权利要求1至6中任一项所述的方法，其中，所述就绪作业队列设置有标记，所述标记表征所述就绪作业队列中是否存在在线作业。The method according to any one of claims 1 to 6, wherein the ready job queue is provided with a mark, and the mark indicates whether there is an online job in the ready job queue.
  8. 一种混合部署场景下的作业调度装置，应用于混合部署场景中的CPU处理器，所述装置包括：A job scheduling device in a hybrid deployment scenario is applied to a CPU processor in the hybrid deployment scenario, and the device includes:
    选择模块，用于从所述处理器的就绪作业队列中选择待处理作业；A selection module, configured to select a job to be processed from a ready job queue of the processor;
    处理模块，用于在所述待处理作业为离线作业，且所述就绪作业队列中存在在线作业时，将所述待处理作业压入限制队列中，并重新选择待处理作业，直至选择的待处理作业为在线作业；A processing module, configured to, when the to-be-processed job is an offline job and there is an online job in the ready job queue, push the to-be-processed job into the restricted queue, and reselect a to-be-processed job until the selected to-be-processed job is an online job;
    调度模块，用于调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。The scheduling module is used to schedule the online job process selected in the online job to execute the online job.
  9. 根据权利要求8所述的装置，其中，所述处理模块还用于，在所述就绪作业队列中的在线作业执行完成后，将所述限制队列中的离线作业放回至所述就绪作业队列中。The device according to claim 8, wherein the processing module is further used to put the offline jobs in the restricted queue back into the ready job queue after the online jobs in the ready job queue are executed.
  10. 根据权利要求8或9所述的装置，其中，所述处理模块还用于，The device according to claim 8 or 9, wherein the processing module is further configured to:
    在所述待处理作业为离线作业且所述就绪作业队列中未存在所述在线作业时，调度选择的所述离线作业中的离线作业进程，以执行所述离线作业；When the to-be-processed job is an offline job and the online job does not exist in the ready job queue, scheduling an offline job process in the selected offline job to execute the offline job;
    或者，or,
    在所述待处理作业为在线作业时，调度选择的所述在线作业中的在线作业进程，以执行所述在线作业。When the job to be processed is an online job, an online job process selected from the online job is scheduled to execute the online job.
  11. 根据权利要求8至10中任一项所述的装置，还包括：获取模块；The device according to any one of claims 8 to 10, further comprising: an acquisition module;
    所述获取模块，用于根据所述限制队列中离线作业的限制时长，获取所述限制队列中的第一离线作业；其中，所述第一离线作业的限制时长大于预设限制时长阈值；The acquisition module is used to acquire the first offline job in the restricted queue according to the restricted duration of the offline jobs in the restricted queue; wherein the restricted duration of the first offline job is greater than a preset restricted duration threshold;
    所述处理模块，还用于在所述第一离线作业的离线作业进程中添加睡眠条件，得到处理后的第一离线作业；其中，所述睡眠条件满足时，所述第一离线作业转移至睡眠队列中；将所述处理后的第一离线作业放回至所述就绪作业队列中。The processing module is also used to add a sleep condition to the offline job process of the first offline job to obtain the processed first offline job; wherein, when the sleep condition is met, the first offline job is transferred to the sleep queue; and the processed first offline job is put back into the ready job queue.
  12. 根据权利要求11所述的装置，其中，所述睡眠条件中包括需求睡眠时长，所述处理模块还用于，在所述睡眠队列中存在第二离线作业时，将所述第二离线作业转移至所述就绪作业队列中；其中，所述第二离线作业的实际睡眠时长大于或者等于所述需求睡眠时长。 According to the device according to claim 11, wherein the sleeping condition includes a required sleep duration, and the processing module is also used to transfer the second offline job to the ready job queue when there is a second offline job in the sleep queue; wherein the actual sleep duration of the second offline job is greater than or equal to the required sleep duration.
  13. 根据权利要求8至12中任一项所述的装置，其中，所述处理模块还用于，The device according to any one of claims 8 to 12, wherein the processing module is further configured to:
    获取已创建或者已唤醒的离线作业进程；Get the created or awakened offline job process;
    在所述离线作业进程所属的离线作业位于所述就绪作业队列中时，将所述离线作业进程更新到所述就绪作业队列中所属的离线作业中。When the offline job to which the offline job process belongs is in the ready job queue, the offline job process is updated to the offline job in the ready job queue.
  14. 根据权利要求8至13中任一项所述的装置，其中，所述就绪作业队列设置有标记，所述标记表征所述就绪作业队列中是否存在在线作业。The device according to any one of claims 8 to 13, wherein the ready job queue is provided with a mark, and the mark indicates whether there is an online job in the ready job queue.
  15. 一种电子设备，包括：An electronic device, comprising:
    至少一个处理器；以及at least one processor; and
    与所述至少一个处理器通信连接的存储器；其中，a memory communicatively connected to the at least one processor; wherein,
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-7中任一项所述的方法。The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.
  16. 一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行权利要求1-7中任一项所述的方法。A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.
  17. 一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据权利要求1-7中任一项所述的方法。 A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.
PCT/CN2023/117543 2022-11-14 2023-09-07 Job scheduling method and apparatus in hybrid deployment scenario, and electronic device WO2024103927A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211422091.9 2022-11-14
CN202211422091.9A CN115809126A (en) 2022-11-14 2022-11-14 Job scheduling method and device in mixed deployment scene and electronic equipment

Publications (1)

Publication Number Publication Date
WO2024103927A1 true WO2024103927A1 (en) 2024-05-23

Family

ID=85483184

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/117543 WO2024103927A1 (en) 2022-11-14 2023-09-07 Job scheduling method and apparatus in hybrid deployment scenario, and electronic device

Country Status (2)

Country Link
CN (1) CN115809126A (en)
WO (1) WO2024103927A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115809126A (en) * 2022-11-14 2023-03-17 京东科技信息技术有限公司 Job scheduling method and device in mixed deployment scene and electronic equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120096468A1 (en) * 2010-10-13 2012-04-19 Microsoft Corporation Compute cluster with balanced resources
CN108549939A (en) * 2018-03-09 2018-09-18 中国科学院软件研究所 A kind of real-time rule-based reasoning method and apparatus of distribution towards magnanimity stream data
US20190130327A1 (en) * 2017-10-31 2019-05-02 Dell Products L. P. Applying machine learning to dynamically scale computing resources to satisfy a service level agreement (sla)
CN110503318A (en) * 2019-08-02 2019-11-26 中国航空无线电电子研究所 Comprehensively modularized aviation electronics processing system resource distribution and Simulation Evaluation platform
CN115809126A (en) * 2022-11-14 2023-03-17 京东科技信息技术有限公司 Job scheduling method and device in mixed deployment scene and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120096468A1 (en) * 2010-10-13 2012-04-19 Microsoft Corporation Compute cluster with balanced resources
US20190130327A1 (en) * 2017-10-31 2019-05-02 Dell Products L. P. Applying machine learning to dynamically scale computing resources to satisfy a service level agreement (sla)
CN108549939A (en) * 2018-03-09 2018-09-18 中国科学院软件研究所 A kind of real-time rule-based reasoning method and apparatus of distribution towards magnanimity stream data
CN110503318A (en) * 2019-08-02 2019-11-26 中国航空无线电电子研究所 Comprehensively modularized aviation electronics processing system resource distribution and Simulation Evaluation platform
CN115809126A (en) * 2022-11-14 2023-03-17 京东科技信息技术有限公司 Job scheduling method and device in mixed deployment scene and electronic equipment

Also Published As

Publication number Publication date
CN115809126A (en) 2023-03-17

Similar Documents

Publication Publication Date Title
US11372688B2 (en) Resource scheduling method, scheduling server, cloud computing system, and storage medium
JP2020537784A (en) Machine learning runtime library for neural network acceleration
WO2019090993A1 (en) Deployment method and apparatus for monitoring system, computer device and storage medium
US8572614B2 (en) Processing workloads using a processor hierarchy system
US10402223B1 (en) Scheduling hardware resources for offloading functions in a heterogeneous computing system
CN105159841B (en) A kind of internal memory migration method and device
WO2024103927A1 (en) Job scheduling method and apparatus in hybrid deployment scenario, and electronic device
US20220229701A1 (en) Dynamic allocation of computing resources
WO2022148086A1 (en) Information processing method and apparatus, and device and storage medium
EP4052126A1 (en) Management of multiple physical function non-volatile memory devices
JP2009087282A (en) Parallel computation system and parallel computation method
US7539992B2 (en) Scheduling method, program product for use in such method, and task scheduling apparatus
CN114827048B (en) Dynamic configurable high-performance queue scheduling method, system, processor and protocol
CN109284108A (en) Date storage method, device, electronic equipment and storage medium
US20200371827A1 (en) Method, Apparatus, Device and Medium for Processing Data
WO2024119930A1 (en) Scheduling method and apparatus, and computer device and storage medium
WO2024066380A1 (en) Pod scheduling method and apparatus, and computer device
CN113424152A (en) Workflow-based scheduling and batching in a multi-tenant distributed system
US20150220361A1 (en) Parallel computer system, control method of parallel computer system, and computer-readable storage medium
US10198784B2 (en) Capturing commands in a multi-engine graphics processing unit
CN115878333A (en) Method, device and equipment for judging consistency between process groups
US11954534B2 (en) Scheduling in a container orchestration system utilizing hardware topology hints
CN115858114A (en) Scheduling method and device of idle task, electronic equipment and readable storage medium
CN114675954A (en) Task scheduling method and device
WO2018188416A1 (en) Data search method and apparatus, and related devices

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23890364

Country of ref document: EP

Kind code of ref document: A1