CN111625326A - Task pipeline execution method and device and electronic equipment - Google Patents

Task pipeline execution method and device and electronic equipment Download PDF

Info

Publication number
CN111625326A
CN111625326A CN202010401618.4A CN202010401618A CN111625326A CN 111625326 A CN111625326 A CN 111625326A CN 202010401618 A CN202010401618 A CN 202010401618A CN 111625326 A CN111625326 A CN 111625326A
Authority
CN
China
Prior art keywords
task
sub
container
executed
processes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010401618.4A
Other languages
Chinese (zh)
Other versions
CN111625326B (en
Inventor
白戈
王长虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN202010401618.4A priority Critical patent/CN111625326B/en
Publication of CN111625326A publication Critical patent/CN111625326A/en
Application granted granted Critical
Publication of CN111625326B publication Critical patent/CN111625326B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/4401—Bootstrapping
    • G06F9/4418—Suspend and resume; Hibernate and awake
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
    • G06F9/4451—User profiles; Roaming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022—Mechanisms to release resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45583—Memory management, e.g. access or allocation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Stored Programmes (AREA)

Abstract

The embodiment of the disclosure provides a task pipeline execution method, a task pipeline execution device and electronic equipment, which belong to data scheduling, and the method comprises the following steps: setting a plurality of sub-processes which are isolated from the main process in a task container, wherein each sub-process is used for executing a pipeline task; after a task to be executed related to a main process is acquired, selecting a target sub-process from the plurality of sub-processes, wherein the target sub-process is used for running the task to be executed; after the task to be executed in the target sub-process is executed, executing the task in the associated sub-process related to the target sub-process according to a preset strategy; and after the tasks in all the sub-processes in the container are executed, releasing the system resources occupied by all the sub-processes in the process of executing the tasks. By the processing scheme, the execution efficiency of the task pipeline can be improved.

Description

Task pipeline execution method and device and electronic equipment
Technical Field
The present disclosure relates to data scheduling, and in particular, to a method and an apparatus for executing a task pipeline, and an electronic device.
Background
There are two different ways to implement a task pipeline, one loosely coupled and the other tightly coupled. Loosely coupled means that each task runs in a separate container, and when one task of the task pipeline is completed, the container is killed, and then another container is started to run the next task. Tightly coupled means that all tasks are scheduled by the same host process. The tight coupling mode has the advantages that the task pipeline execution efficiency is high, all tasks run in the same container, and the overhead of frequently starting the container is avoided. The problem with the tightly coupled approach is that the python script has a memory leak problem and the modules of python cannot be unloaded cleanly. If two tasks have duplicate modules with different functions (such as different versions of the same tool), the tasks cannot be executed normally.
Disclosure of Invention
In view of the above, embodiments of the present disclosure provide a task pipeline execution method, a task pipeline execution device, and an electronic device to at least partially solve the problems in the prior art.
In a first aspect, an embodiment of the present disclosure provides a task pipeline execution method, including:
setting a plurality of sub-processes which are isolated from the main process in a task container, wherein each sub-process is used for executing a pipeline task;
after a task to be executed related to a main process is acquired, selecting a target sub-process from the plurality of sub-processes, wherein the target sub-process is used for running the task to be executed;
after the task to be executed in the target sub-process is executed, executing the task in the associated sub-process related to the target sub-process according to a preset strategy;
and after the tasks in all the sub-processes in the container are executed, releasing the system resources occupied by all the sub-processes in the process of executing the tasks.
According to a specific implementation manner of the embodiment of the present disclosure, before setting a plurality of sub processes separately set from a main process in a task container, the method includes:
after the main process is awakened, judging whether a container configuration file exists or not;
if yes, further judging whether the task container exists or not based on the container system configuration file;
if not, the main process creates a container setup sub-process, and the task container is created by using the container setup sub-process;
and if the starting container exists, the main process sends the packaged starting message to the task starting subprocess, the task starting subprocess unpacks the sent message to obtain a starting container message entity, and the task container is started.
According to a specific implementation manner of the embodiment of the present disclosure, the setting of the plurality of sub processes isolated from the main process in the task container includes:
setting kernel resource isolation parameters of the task container, and updating a configuration file;
starting a resource isolation mechanism according to the updated configuration file, and setting parent-child process isolation attributes;
and creating a task container subprocess, wherein the subprocess is used for loading the task process entity into the subprocess.
According to a specific implementation manner of the embodiment of the present disclosure, after the task to be executed related to the main process is acquired, selecting a target sub-process from the plurality of sub-processes includes:
acquiring a task list required to be executed by the main process;
determining the execution sequence of each task in the task list;
determining an execution order of the plurality of sub-processes based on the execution order;
and determining a target sub-process matched with the current task based on the execution sequence of the sub-processes.
According to a specific implementation manner of the embodiment of the present disclosure, after the task to be executed related to the main process is acquired, selecting a target sub-process from the plurality of sub-processes includes:
acquiring a task ID of a current task to be executed;
and selecting a sub-process matched with the task ID from the plurality of sub-processes as the target sub-process.
According to a specific implementation manner of the embodiment of the present disclosure, before executing the task in the associated sub-process related to the target sub-process according to the preset policy, the method further includes:
pre-loading a program module corresponding to the associated subprocess in the target subprocess;
and binding the loaded program module and the associated sub-process.
According to a specific implementation manner of the embodiment of the present disclosure, the executing a task in an associated sub-process related to the target sub-process according to a preset policy includes:
monitoring the execution state of the target subprocess;
judging whether a task completion mark of the target subprocess exists or not;
if the program module exists, the program module which is loaded in advance in the associated subprocess is further started and executed.
According to a specific implementation manner of the embodiment of the present disclosure, the releasing system resources occupied by all the sub-processes in the process of executing the task includes:
acquiring memory address space occupied by all sub-processes in the task execution process;
and executing release operation on the data in the memory address space.
In a second aspect, an embodiment of the present disclosure provides a task pipeline execution device, including:
the device comprises a setting module, a task container and a control module, wherein the setting module is used for setting a plurality of sub-processes which are arranged in an isolation way with a main process in the task container, and each sub-process is used for executing a pipeline task;
the selection module is used for selecting a target subprocess from the plurality of subprocesses after the task to be executed related to the main process is obtained, wherein the target subprocess is used for running the task to be executed;
the execution module is used for executing the tasks in the associated sub-processes related to the target sub-process according to a preset strategy after the tasks to be executed in the target sub-process are executed;
and the releasing module is used for releasing the system resources occupied by all the subprocesses in the task executing process after the tasks in all the subprocesses in the container are executed.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of task pipeline execution of the first aspect or any implementation of the first aspect.
In a fourth aspect, the disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the task pipeline execution method in the first aspect or any implementation manner of the first aspect.
In a fifth aspect, the embodiments of the present disclosure also provide a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the task pipeline execution method in the foregoing first aspect or any implementation manner of the first aspect.
The task pipeline execution scheme in the embodiment of the disclosure comprises a plurality of sub-processes which are arranged in a task container in an isolation way with a main process, wherein each sub-process is used for executing a pipeline task; after a task to be executed related to a main process is acquired, selecting a target sub-process from the plurality of sub-processes, wherein the target sub-process is used for running the task to be executed; after the task to be executed in the target sub-process is executed, executing the task in the associated sub-process related to the target sub-process according to a preset strategy; and after the tasks in all the sub-processes in the container are executed, releasing the system resources occupied by all the sub-processes in the process of executing the tasks. By the processing scheme, the execution efficiency of the task pipeline can be improved, and system resources are saved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of a task pipeline execution method according to an embodiment of the present disclosure;
FIG. 2 is a schematic configuration diagram of a task pipeline execution method according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of another task pipeline execution method provided by the embodiments of the present disclosure;
FIG. 4 is a flow chart of another task pipeline execution method provided by the embodiments of the present disclosure;
FIG. 5 is a schematic diagram of a task pipeline execution device according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
The embodiment of the disclosure provides a task pipeline execution method. The task pipeline execution method provided by the embodiment can be executed by a computing device, the computing device can be implemented as software, or implemented as a combination of software and hardware, and the computing device can be integrated in a server, a client and the like.
Referring to fig. 1, a task pipeline execution method in an embodiment of the present disclosure may include the following steps:
s101, a plurality of sub processes which are arranged in an isolation mode with the main process are arranged in the task container, and each sub process is used for executing a pipeline task.
A task container (or container) provides the environment in which the components run, and the container itself can provide a set of services that the components associated with performing the task utilize in a standard manner.
There are two different ways to implement a task pipeline, one loosely coupled and the other tightly coupled. Loosely coupled means that each task runs in a separate container, and when one task of the task pipeline is completed, the task container is killed, and then another container is started to run the next task. Tightly coupled means that all tasks are scheduled by the same host process. The tight coupling mode has the advantages that the task pipeline execution efficiency is high, all tasks run in the same container, and the overhead of frequently starting the container is avoided. The problem with the tightly coupled approach is that program scripts (e.g., python) have memory leaks and program modules cannot be unloaded cleanly. If two tasks have duplicate modules with different functions (such as different versions of the same tool), the tasks cannot be executed normally.
To this end, referring to fig. 2, the solution of the present disclosure sets a plurality of sub-processes in a task container, the task container corresponding to the main process, the sub-processes being set separately from the main process, and a pipeline task is executed by each sub-process by importing a program module. The number of the sub-processes can be set according to actual needs, and the number of the sub-processes is not limited herein.
In the process of setting the sub-process, the sub-process needs to be set in an isolated manner from the main process, specifically, kernel resource isolation parameters of the task container can be set, the kernel resource isolation parameters are updated into a configuration file of the container, and the sub-process and the main process are quickly and effectively isolated by updating the configuration file. In the process of executing the isolation operation, a resource isolation mechanism is started according to the updated configuration file, isolation attributes of parent and child processes (namely, a main process and a child process) are set, and finally, a task container child process is created and is used for loading task process entities (such as program modules) into the child process.
S102, after a task to be executed related to a main process is obtained, selecting a target sub-process from the plurality of sub-processes, wherein the target sub-process is used for running the task to be executed.
The main process in the task container may correspond to a plurality of to-be-executed tasks that need to be executed, and for this reason, the to-be-executed task that needs to be executed at present may be acquired, a target sub-process may be determined in a manner of matching the current to-be-executed task with one sub-process of the plurality of sub-processes, and the to-be-executed task may be executed by the target sub-process.
As one mode, a task list required to be executed by a main process may be obtained through a query, an execution order of each task in the task list is determined through the task list, an execution order of the plurality of sub-processes is determined through the execution order, and finally, a target sub-process matched with a current task is determined based on the execution order of the sub-processes.
Further, a task ID of a current task to be executed may also be acquired, and a sub-process matched with the task ID is selected from the plurality of sub-processes as the target sub-process.
S103, after the task to be executed in the target sub-process is executed, executing the task in the associated sub-process related to the target sub-process according to a preset strategy.
In the process of executing the pipeline task in the target sub-process, the execution condition of the task can be monitored, and whether the execution of the task in the target sub-process is completed or not can be judged by monitoring the execution condition of the task, so that the next execution strategy can be further determined.
Specifically, the execution state of the target subprocess may be monitored, whether a task completion flag of the target subprocess exists is determined, and when the task completion flag of the subprocess exists, the preloaded program module in the associated subprocess is further started and executed. The associated sub-process may be a sub-process that needs to be executed next after the task in the target sub-process is executed according to a preset policy.
S104, after the execution of the tasks in all the sub-processes in the container is finished, releasing the system resources occupied by all the sub-processes in the process of executing the tasks.
In order to save system resources, the execution conditions of all the subprocesses in the container can be further monitored, and after the tasks in all the subprocesses are completed, the system resources occupied by all the subprocesses in executing the tasks are released. For example, the memory resources occupied by all the sub-processes in the process of executing the task may be monitored, and the memory resources may be released after the tasks of all the sub-processes are completed. Of course, the processing memory resource may be other types of system resources, and the type and content of the system resource are not limited herein.
Through the scheme in the steps S101-S104, the main process and the plurality of sub-processes which are arranged in an isolated mode with the main process can be arranged in the container, all tasks can be guaranteed to run in a closed and safe running environment through the mode, the sub-processes and the main process are isolated, all memories in the sub-processes are all recycled by the operating system along with the process exit, and memory leakage and environment pollution do not exist. Tasks are run in a completely clean software environment and are not contaminated by previously run tasks.
Referring to fig. 3, according to a specific implementation manner of the embodiment of the present disclosure, before setting a plurality of sub processes in a task container, the sub processes being set in isolation from a main process, the method includes:
s301, after the main wakening process is waken up, whether a container configuration file exists is judged.
By means of the configuration file, a specific type of task container can be generated, and therefore, whether the configuration file corresponding to the main process exists or not can be judged after the main process is awakened. The configuration file may have a preset storage address, and whether the container configuration file exists may be determined by inquiring whether the configuration file exists at the storage address.
S302, if yes, further based on the container system configuration file, judging whether the task container exists.
By reading the content in the configuration file, whether the related task container is generated or not can be further judged, so that a foundation is laid for further executing the related operation of the task container.
S303, if the task container does not exist, the main process creates a container setting sub-process, and the task container is created by using the container setting sub-process.
When the task container is found not to be created through the configuration file, the task container can be created by setting a sub-process through the set container in a mode that the sub-process is set by the main process.
S304, if the starting container exists, the main process sends the packaged starting message to the task starting subprocess, the task starting subprocess unpacks the sent message to obtain a starting container message entity, and the task container is started.
After finding through the configuration file that the task container has been created, the already existing task container can be started through a task start subprocess set by the main process.
Through the embodiment, the task container can be started and set quickly.
According to a specific implementation manner of the embodiment of the present disclosure, the setting of the plurality of sub processes isolated from the main process in the task container includes: setting kernel resource isolation parameters of the task container, and updating a configuration file; starting a resource isolation mechanism according to the updated configuration file, and setting parent-child process isolation attributes; and creating a task container subprocess, wherein the subprocess is used for loading the task process entity into the subprocess.
Referring to fig. 4, according to a specific implementation manner of the embodiment of the present disclosure, after acquiring a task to be executed related to a main process, selecting a target sub-process from the plurality of sub-processes includes:
s401, acquiring a task list required to be executed by the main process.
The task list required to be executed by the main process can be obtained through a query mode, and the task list can be stored in a specific storage position through a preset mode.
S402, determining the execution sequence of each task in the task list.
By parsing the tasks in the task list, the execution order of each task in the task list can be determined.
S403, determining the execution sequence of the plurality of sub-processes based on the execution sequence.
After the execution sequence of the tasks is determined, the task types of the sub-processes and the execution sequence of the tasks can be matched according to the task types allocated to each sub-process, so that the execution sequence of the sub-processes is finally determined.
S404, determining a target sub-process matched with the current task based on the execution sequence of the sub-processes.
After the execution order of the subprogram is determined to be completed, by analyzing the current task, the subprocess required for executing the current task can be determined and set as the target subprocess through which the current task is executed.
By the method in the embodiment, one target sub-process for executing the current task can be quickly selected from the plurality of sub-processes.
According to a specific implementation manner of the embodiment of the present disclosure, after the task to be executed related to the main process is acquired, selecting a target sub-process from the plurality of sub-processes includes: acquiring a task ID of a current task to be executed; and selecting a sub-process matched with the task ID from the plurality of sub-processes as the target sub-process.
According to a specific implementation manner of the embodiment of the present disclosure, before executing the task in the associated sub-process related to the target sub-process according to the preset policy, the method further includes: pre-loading a program module corresponding to the associated subprocess in the target subprocess; and binding the loaded program module and the associated sub-process.
According to a specific implementation manner of the embodiment of the present disclosure, the executing a task in an associated sub-process related to the target sub-process according to a preset policy includes: monitoring the execution state of the target subprocess; judging whether a task completion mark of the target subprocess exists or not; if the program module exists, the program module which is loaded in advance in the associated subprocess is further started and executed.
According to a specific implementation manner of the embodiment of the present disclosure, the releasing system resources occupied by all the sub-processes in the process of executing the task includes:
acquiring memory address space occupied by all sub-processes in the task execution process; and executing release operation on the data in the memory address space.
Corresponding to the above method embodiment, referring to fig. 5, the disclosed embodiment further provides a task pipeline execution device 50, including:
a setting module 501, configured to set, in a task container, a plurality of sub-processes that are set separately from a main process, where each sub-process is configured to execute a pipeline task;
a selecting module 502, configured to select a target sub-process from the multiple sub-processes after acquiring a to-be-executed task related to a main process, where the target sub-process is used to run the to-be-executed task;
an executing module 503, configured to execute, according to a preset policy, a task in an associated sub-process related to the target sub-process after the task to be executed in the target sub-process is executed;
a releasing module 504, configured to release, after the tasks in all the sub-processes in the container are executed, the system resources occupied by all the sub-processes in the process of executing the tasks.
For parts not described in detail in this embodiment, reference is made to the contents described in the above method embodiments, which are not described again here.
Referring to fig. 6, an embodiment of the present disclosure also provides an electronic device 60, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of task pipeline execution of the method embodiments described above.
The disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the task pipeline execution method in the aforementioned method embodiments.
The disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the task pipeline execution method of the aforementioned method embodiments.
Referring now to FIG. 6, a schematic diagram of an electronic device 60 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device 60 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 60 are also stored. The processing device 601, the ROM602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 60 to communicate with other devices wirelessly or by wire to exchange data. While the figures illustrate an electronic device 60 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects the internet protocol addresses from the at least two internet protocol addresses and returns the internet protocol addresses; receiving an internet protocol address returned by the node evaluation equipment; wherein the obtained internet protocol address indicates an edge node in the content distribution network.
Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present disclosure should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims (11)

1. A method for task pipeline execution, comprising:
setting a plurality of sub-processes which are isolated from the main process in a task container, wherein each sub-process is used for executing a pipeline task;
after a task to be executed related to a main process is acquired, selecting a target sub-process from the plurality of sub-processes, wherein the target sub-process is used for running the task to be executed;
after the task to be executed in the target sub-process is executed, executing the task in the associated sub-process related to the target sub-process according to a preset strategy;
and after the tasks in all the sub-processes in the container are executed, releasing the system resources occupied by all the sub-processes in the process of executing the tasks.
2. The method of claim 1, wherein before the plurality of sub-processes are set in the task container in isolation from the main process, the method comprises:
after the main process is awakened, judging whether a container configuration file exists or not;
if yes, further judging whether the task container exists or not based on the container system configuration file;
if not, the main process creates a container setup sub-process, and the task container is created by using the container setup sub-process;
and if the starting container exists, the main process sends the packaged starting message to the task starting subprocess, the task starting subprocess unpacks the sent message to obtain a starting container message entity, and the task container is started.
3. The method of claim 1, wherein setting a plurality of sub-processes in a task container that are set separately from a main process comprises:
setting kernel resource isolation parameters of the task container, and updating a configuration file;
starting a resource isolation mechanism according to the updated configuration file, and setting parent-child process isolation attributes;
and creating a task container subprocess, wherein the subprocess is used for loading the task process entity into the subprocess.
4. The method according to claim 1, wherein the selecting a target sub-process from the plurality of sub-processes after acquiring the to-be-executed task related to the main process comprises:
acquiring a task list required to be executed by the main process;
determining the execution sequence of each task in the task list;
determining an execution order of the plurality of sub-processes based on the execution order;
and determining a target sub-process matched with the current task based on the execution sequence of the sub-processes.
5. The method according to claim 1, wherein the selecting a target sub-process from the plurality of sub-processes after acquiring the to-be-executed task related to the main process comprises:
acquiring a task ID of a current task to be executed;
and selecting a sub-process matched with the task ID from the plurality of sub-processes as the target sub-process.
6. The method of claim 1, wherein before executing the task in the associated sub-process related to the target sub-process according to the preset policy, the method further comprises:
pre-loading a program module corresponding to the associated subprocess in the target subprocess;
and binding the loaded program module and the associated sub-process.
7. The method according to claim 6, wherein the executing the task in the associated sub-process related to the target sub-process according to the preset policy comprises:
monitoring the execution state of the target subprocess;
judging whether a task completion mark of the target subprocess exists or not;
if the program module exists, the program module which is loaded in advance in the associated subprocess is further started and executed.
8. The method according to claim 1, wherein said releasing system resources occupied by all the sub-processes in the process of executing tasks comprises:
acquiring memory address space occupied by all sub-processes in the task execution process;
and executing release operation on the data in the memory address space.
9. A task pipeline execution device, comprising:
the device comprises a setting module, a task container and a control module, wherein the setting module is used for setting a plurality of sub-processes which are arranged in an isolation way with a main process in the task container, and each sub-process is used for executing a pipeline task;
the selection module is used for selecting a target subprocess from the plurality of subprocesses after the task to be executed related to the main process is obtained, wherein the target subprocess is used for running the task to be executed;
the execution module is used for executing the tasks in the associated sub-processes related to the target sub-process according to a preset strategy after the tasks to be executed in the target sub-process are executed;
and the releasing module is used for releasing the system resources occupied by all the subprocesses in the task executing process after the tasks in all the subprocesses in the container are executed.
10. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the task pipeline execution method of any of the preceding claims 1-8.
11. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the task pipeline execution method of any of the preceding claims 1-8.
CN202010401618.4A 2020-05-13 2020-05-13 Task pipeline execution method and device and electronic equipment Active CN111625326B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010401618.4A CN111625326B (en) 2020-05-13 2020-05-13 Task pipeline execution method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010401618.4A CN111625326B (en) 2020-05-13 2020-05-13 Task pipeline execution method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN111625326A true CN111625326A (en) 2020-09-04
CN111625326B CN111625326B (en) 2023-06-27

Family

ID=72258075

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010401618.4A Active CN111625326B (en) 2020-05-13 2020-05-13 Task pipeline execution method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN111625326B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114356446A (en) * 2021-12-27 2022-04-15 湖北天融信网络安全技术有限公司 Method, device and equipment for processing inter-process event and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030046681A1 (en) * 2001-08-30 2003-03-06 International Business Machines Corporation Integrated system and method for the management of a complete end-to-end software delivery process
US20080320490A1 (en) * 2007-06-20 2008-12-25 Nokia Corporation Method, apparatus and computer program product for providing sub-process resource management
US20130339079A1 (en) * 2012-06-15 2013-12-19 International Business Machines Corporation Configurable resource policies
CN106095530A (en) * 2016-06-08 2016-11-09 电子科技大学 The container of a kind of many android system automatically creates and startup method
CN107133086A (en) * 2016-02-29 2017-09-05 阿里巴巴集团控股有限公司 Task processing method, device and system based on distributed system
US9792163B1 (en) * 2016-06-28 2017-10-17 Vmware, Inc. Decentralized control plane for a computing system
CN110413386A (en) * 2019-06-27 2019-11-05 深圳市富途网络科技有限公司 Multiprocessing method, apparatus, terminal device and computer readable storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030046681A1 (en) * 2001-08-30 2003-03-06 International Business Machines Corporation Integrated system and method for the management of a complete end-to-end software delivery process
US20080320490A1 (en) * 2007-06-20 2008-12-25 Nokia Corporation Method, apparatus and computer program product for providing sub-process resource management
US20130339079A1 (en) * 2012-06-15 2013-12-19 International Business Machines Corporation Configurable resource policies
CN107133086A (en) * 2016-02-29 2017-09-05 阿里巴巴集团控股有限公司 Task processing method, device and system based on distributed system
CN106095530A (en) * 2016-06-08 2016-11-09 电子科技大学 The container of a kind of many android system automatically creates and startup method
US9792163B1 (en) * 2016-06-28 2017-10-17 Vmware, Inc. Decentralized control plane for a computing system
CN110413386A (en) * 2019-06-27 2019-11-05 深圳市富途网络科技有限公司 Multiprocessing method, apparatus, terminal device and computer readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114356446A (en) * 2021-12-27 2022-04-15 湖北天融信网络安全技术有限公司 Method, device and equipment for processing inter-process event and storage medium
CN114356446B (en) * 2021-12-27 2023-08-22 湖北天融信网络安全技术有限公司 Processing method, device, equipment and storage medium for inter-process event

Also Published As

Publication number Publication date
CN111625326B (en) 2023-06-27

Similar Documents

Publication Publication Date Title
CN109582310B (en) Data processing method and device, electronic equipment and computer readable storage medium
CN109857506B (en) Method and device for realizing guide information, electronic equipment and storage medium
CN111177112A (en) Database blocking method and device based on operation and maintenance management system and electronic equipment
CN107644075B (en) Method and device for collecting page information
CN111079125A (en) Method and device for calling third-party library dynamic lifting authority by application program
CN112905220B (en) Thermal restoration method, device, equipment and storage medium
CN111625291B (en) Automatic iteration method and device for data processing model and electronic equipment
CN111460432B (en) On-line document authority control method, device, equipment and computer readable medium
CN111625326B (en) Task pipeline execution method and device and electronic equipment
CN111382058B (en) Service testing method and device, server and storage medium
CN111309406A (en) Event processing method and device of application program and electronic equipment
CN116302271A (en) Page display method and device and electronic equipment
CN110674050A (en) Memory out-of-range detection method and device, electronic equipment and computer storage medium
CN113448585B (en) Compiling method and device of thread pool, electronic equipment and storage medium
CN111382057B (en) Test case generation method, test method and device, server and storage medium
CN113032046A (en) Method, device and equipment for repairing so file and storage medium
CN112506781A (en) Test monitoring method, test monitoring device, electronic device, storage medium, and program product
CN111949528A (en) Memory leak detection method and device, electronic equipment and storage medium
CN110908711A (en) Control development method and device of application program and electronic equipment
CN111641692B (en) Session data processing method and device and electronic equipment
CN112688863B (en) Gateway data processing method and device and electronic equipment
CN112732269B (en) Access method and device applied to physical address of AVS2 hardware encoder
CN112559394B (en) System library access method and device and electronic equipment
CN111026571B (en) Processor down-conversion processing method and device and electronic equipment
CN113448550B (en) Method and device for realizing collection management of classes, electronic equipment and computer medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant