CN113760482A - Task processing method, device and system - Google Patents

Task processing method, device and system Download PDF

Info

Publication number
CN113760482A
CN113760482A CN202010574916.3A CN202010574916A CN113760482A CN 113760482 A CN113760482 A CN 113760482A CN 202010574916 A CN202010574916 A CN 202010574916A CN 113760482 A CN113760482 A CN 113760482A
Authority
CN
China
Prior art keywords
task
code
target
processed
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010574916.3A
Other languages
Chinese (zh)
Inventor
杨恒占
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202010574916.3A priority Critical patent/CN113760482A/en
Publication of CN113760482A publication Critical patent/CN113760482A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/484—Precedence

Abstract

The invention discloses a task processing method, a task processing device and a task processing system, and relates to the technical field of computers. One embodiment of the method comprises: acquiring a target task to be processed according to the task state of the target task, wherein a plurality of target tasks are stored in a task pool and comprise code related files; and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed. The implementation mode can realize flexible scheduling of the code packing device under specific conditions, so that the distribution of the tasks to be processed is more reasonable, the dynamic distribution of the tasks in the task processing queue can be realized, and the task processing efficiency is improved.

Description

Task processing method, device and system
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, and a system for task processing.
Background
With the rapid development of the internet technology, how to efficiently and reasonably process a large number of tasks is an urgent problem to be solved. For example, in the process of performing persistent integration by Jenkins (which is an open source software project platform and is a persistent integration tool developed based on Java), after a plurality of code tasks are completed, the code tasks can be packaged by a plurality of code packaging devices. The specific process is as follows: after a plurality of tasks are created, one task is matched for each code packaging device, the rest tasks directly enter a waiting queue of one code packaging device to wait for processing, and other code packaging devices may have an idle state.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
1. under specific conditions (for example, a plurality of tasks need to be processed, and the processing time of each task is long), the code packaging device cannot be flexibly scheduled, and the tasks to be processed are unreasonably distributed;
2. the dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low.
Disclosure of Invention
In view of this, embodiments of the present invention provide a task processing method and apparatus, which can implement flexible scheduling of a code packing apparatus under specific conditions, so that to-be-processed tasks are more reasonably allocated, and dynamic allocation of tasks in a task processing queue can also be implemented, thereby improving task processing efficiency.
To achieve the above object, according to a first aspect of an embodiment of the present invention, there is provided a task processing method including:
acquiring a target task to be processed according to the task state of the target task, wherein a plurality of target tasks are stored in a task pool and comprise code related files;
and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed.
Further, before the step of allocating the code packing device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device, the task processing method further includes: the processing state of the code packaging apparatus is monitored.
Further, the task state of the target task is obtained by monitoring the target task according to the task monitoring period determined by the historical task processing time length.
Further, the task processing queue corresponding to the code packaging device is determined according to the code packaging device information, wherein the code packaging device information includes at least one of the following information: the method comprises the steps of Internet protocol address information, host address information, device number information and task number threshold values corresponding to task processing queues.
Further, the queue information includes one or more of the following: the task number, the task type and the processing priority corresponding to the target task to be processed in the current queue.
Further, the step of allocating the code packing device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device includes: and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device and the task information of the target task to be processed.
Further, after the step of performing, by the code packing device, packing processing on the target task to be processed, the task processing method further includes: and adjusting the task state of the processed target task.
According to a second aspect of the embodiments of the present invention, there is provided a task processing apparatus including:
the system comprises an acquisition module, a task pool and a processing module, wherein the acquisition module is used for acquiring a target task to be processed according to the task state of the target task, a plurality of target tasks are stored in the task pool, and the target task comprises a code related file;
and the task processing module is used for allocating the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed.
According to a third aspect of the embodiments of the present invention, there is provided a task processing system including:
the task creating device is used for creating a target task and sending the target task to the task pool;
the task pool is used for acquiring a target task to be processed according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and each target task comprises a code related file; distributing a code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device;
and the code packing device is used for receiving the target task to be processed and packing the target task to be processed.
According to a fourth aspect of embodiments of the present invention, there is provided an electronic apparatus, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement any of the task processing methods described above.
According to a fifth aspect of embodiments of the present invention, there is provided a computer-readable medium on which a computer program is stored, the program, when executed by a processor, implementing any one of the task processing methods described above.
One embodiment of the above invention has the following advantages or benefits: the target tasks to be processed are obtained according to the task states of the target tasks, wherein a plurality of target tasks are stored in a task pool and comprise code related files; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram of a main flow of a task processing method provided according to a first embodiment of the present invention;
fig. 2 is a schematic diagram of a main flow of a task processing method provided according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of the main modules of a task processing device provided according to an embodiment of the present invention;
FIG. 4 is a block diagram of a task processing system provided in accordance with an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a task processing method provided according to a first embodiment of the present invention; as shown in fig. 1, a task processing method provided in an embodiment of the present invention mainly includes:
step S101, a target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in a task pool, and the target task comprises a code related file.
Through the arrangement, the target task is stored by utilizing the task pool, and when the task needs to be processed, the target task to be processed is obtained from the task pool according to the task state of the target task, so that the code packaging device is favorable for being subsequently distributed to the target task to be processed, and the problems that in the prior art, the task distribution is unreasonable and the code packaging device cannot be flexibly scheduled due to the fact that the target task to be processed is directly sent to the code packaging device from the task creating device are solved. On the other hand, a plurality of (here, more than or equal to two) target tasks are stored in the task pool, so that the whole processing state of the target tasks is conveniently recorded, and the follow-up query is conveniently carried out.
Specifically, according to the embodiment of the present invention, the code-related file includes source code, executable code, a related configuration file, a resource file, and the like.
According to an embodiment of the present invention, before the step of allocating a code packing device to a target task to be processed according to queue information of a task processing queue corresponding to the code packing device, the task processing method further includes: monitoring the processing state of the code packaging device and confirming that the code packaging device is not in a rest state.
Wherein, the processing state of the code packaging device comprises: in-process, idle, and rest (i.e., rest state, which means that the code packaging apparatus does not perform related task processing due to problems such as failure and system setup). The code packaging device in the rest state does not distribute the target task to be processed.
Further, according to the embodiment of the present invention, the task state of the target task is obtained by monitoring the target task according to the task monitoring period determined by the historical task processing time length.
Specifically, a task monitoring period is determined according to the historical task processing duration, and the task state of each target task is obtained according to the task monitoring period. The step can be executed by a task pool, and the task pool can perform real-time monitoring (namely the task monitoring period is short enough) when being executed; the method can be executed by constructing a task state monitoring device, the task state monitoring device is used for acquiring the task state of each target task and then sending the task state of each target task to the task pool, the monitoring frequency can be reduced when the polling device is executed, and the pressure of frequent monitoring operation on the performance of the computer is avoided.
And S102, distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed.
Through the arrangement, the task platform for creating the tasks is decoupled from the code packaging device for processing the tasks, and the target tasks to be processed are distributed through the task pool, so that the code packaging device is flexibly scheduled under specific conditions, and the task processing distribution is more reasonable.
Specifically, according to the embodiment of the present invention, the task processing queue corresponding to the code packaging apparatus is determined according to the code packaging apparatus information, where the code packaging apparatus information includes at least one of the following information: IP (Internet Protocol Address) Address information, host Address information, device number information, and a task number threshold corresponding to the task processing queue.
According to an embodiment of the present invention, the queue information includes one or more of the following information: the task number, the task type and the processing priority corresponding to the target task to be processed in the current queue.
Further, according to an embodiment of the present invention, the step of allocating a code packing device to a target task to be processed according to queue information of a task processing queue corresponding to the code packing device includes: and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device and the task information of the target task to be processed.
According to a specific implementation manner of the embodiment of the present invention, the task information of the target task to be processed indicates a task type and a processing priority corresponding to the target task to be processed.
According to a specific implementation manner of the embodiment of the invention, after allocating codes to the target tasks to be processed and packaging the codes according to the queue information of the task processing queue corresponding to the code packaging device and the task information of the target tasks to be processed, when the processing state of the code packaging device obtained by monitoring is idle, the target tasks to be processed can be directly selected to be sent to the code packaging device for packaging; when the processing state of the code packaging device obtained by monitoring is in processing, the target task to be processed can be sent to the task processing queue corresponding to the code packaging device, and when the code packaging device is idle, the target task to be processed is obtained from the corresponding task processing queue for packaging.
Further, according to a specific implementation manner of the embodiment of the present invention, the queue information indicates a processing priority of an existing target task to be processed in the task processing queue, and the task information indicates a processing priority of a currently allocated target task to be processed, and for a target task to be processed that needs to be processed urgently (i.e., the processing priority is high), the target task to be processed in the task processing queue may be packed before the target task to be processed.
Further, according to an embodiment of the present invention, after the step of performing, by the code packing device, packing processing on the target task to be processed, the task processing method further includes: and adjusting the task state of the processed target task.
According to the technical scheme of the embodiment of the invention, the target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
Fig. 2 is a schematic diagram of a main flow of a task processing method provided according to a second embodiment of the present invention; as shown in fig. 2, an application scenario of the embodiment of the present invention is to perform packing processing on a code task created by a Jenkins (a persistent integration tool developed based on Java) platform, and a task processing method provided by the embodiment of the present invention mainly includes:
step S201, monitoring the target task according to the task monitoring period determined by the historical task processing time length to obtain the task state of the target task.
According to the embodiment of the present invention, before the step of monitoring the target task in the task monitoring period determined according to the historical task processing duration to obtain the task state of the target task, the task processing method further includes: and receiving a target task created by a task creation device (Jenkins platform).
Through the arrangement, the task platform for creating the target task is decoupled from the code packaging device for processing the task, and each target task to be processed is distributed through the task pool, so that the code packaging device is flexibly scheduled under a specific condition, and the task processing distribution is more reasonable.
Further, according to the embodiment of the present invention, the target task is sent to the task pool in a certain data analysis form, which includes data analysis forms such as a character string, xml (Extensible Markup Language), list (a class in a class library, which refers to a list, and a detail table), and is preferably in a json (JavaScript Object notification, a lightweight data exchange format). The json is a general data analysis mode, and various platforms have a corresponding analysis mode with the json data, so that the json data is not limited due to different language forms when being transmitted and used among the platforms, and the universality and the usability in the data transmission process are improved.
Step S202, a target task to be processed is obtained according to the task state of the target task.
Specifically, according to the embodiment of the present invention, a task state monitoring device is used to determine a task monitoring period according to the historical task processing duration, the task state of each target task is obtained according to the task monitoring period, and the task pool receives the task state of each target task sent by the task state monitoring device.
Through the setting, the pressure on the performance of the computer caused by frequent monitoring operation is avoided by setting the task monitoring period.
Specifically, according to a specific implementation manner of the embodiment of the present invention, firstly, a map (mapping a key to an object of a value) may be initialized for each code packaging machine by using a task state monitoring device (or a task pool may perform the following steps), so as to record a time node of the code packaging machine, obtain time nodes of 7 historical target tasks from a processed target task stack, then calculate packaging processing durations of 6 target tasks, take one fourth of an average packaging processing duration of the 6 target tasks as a duration corresponding to a monitoring period, and obtain a task state of each target task and a processing state of each code packaging device according to the task monitoring period. With the progress of the packaging Processing process, the task monitoring period can be re-determined in a recursive manner, so that the triggering frequency of the task state monitoring device is reduced, and the pressure of a Central Processing Unit (cpu) is reduced. It should be noted that the above numerical values are only examples in the embodiment, and do not represent limitations to the present invention, and the numerical values may be adjusted according to actual situations in the specific implementation process.
Step S203, monitoring the processing state of the code packaging apparatus, and determining that the code packaging apparatus is in a non-rest state.
According to the embodiment of the invention, the monitoring of the code packaging device can be performed by the polling device, and can also be directly performed by the task pool. Wherein, the processing state of the code packaging device comprises: in-process, idle, and rest (i.e., rest state, which means that the code packaging apparatus does not perform related task processing due to problems such as failure and system setup). The code packaging device in the rest state does not distribute the target task to be processed.
And step S204, distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device.
Through the arrangement, the Jenkins platform and the code packing device are decoupled, and the target tasks to be processed are distributed through the task pool, so that the code packing device is flexibly dispatched under a specific condition, and the task processing and distribution are more reasonable.
Specifically, according to the embodiment of the present invention, the task processing queue corresponding to the code packaging apparatus is determined according to the code packaging apparatus information, where the code packaging apparatus information includes at least one of the following information: IP (Internet Protocol Address) Address information, host Address information, device number information, and a task number threshold corresponding to the task processing queue.
According to an embodiment of the present invention, the queue information includes one or more of the following information: the task number, the task type and the processing priority corresponding to the target task to be processed in the current queue.
And step S205, when the processing state corresponding to the code packaging device is idle, sending the target task to be processed to the code packaging device for packaging processing.
Further, according to a specific implementation manner of the embodiment of the present invention, the queue information indicates a processing priority of an existing target task to be processed in the task processing queue, and the task information indicates a processing priority of a currently allocated target task to be processed, and for a target task to be processed that needs to be processed urgently (i.e., the processing priority is high), the target task to be processed in the task processing queue may be packed before the target task to be processed.
Step S206, when the processing state corresponding to the code packaging device is in process, sending the target task to be processed to the task processing queue corresponding to the code packaging device, so that the target task to be processed is obtained from the corresponding task processing queue for packaging when the code packaging device is in an idle state.
Further, according to an embodiment of the present invention, after the step of performing, by the code packing device, packing processing on the target task to be processed, the task processing method further includes: and adjusting the task state of the processed target task.
Through the setting, the page display platform can acquire the latest state of the target task to perform corresponding page display, so that a user can acquire the execution state of the target task. According to the embodiment of the invention, the page presentation platform can be presented through html5 (core language of the world wide web, an application hypertext markup language under standard general markup language).
According to the technical scheme of the embodiment of the invention, the target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
FIG. 3 is a schematic diagram of the main modules of a task processing device provided according to an embodiment of the present invention; as shown in fig. 3, a task processing device 300 according to an embodiment of the present invention mainly includes:
an obtaining module 301, configured to obtain a target task to be processed according to a task state of the target task, where a plurality of target tasks are stored in a task pool, and each target task includes a code-related file.
Through the arrangement, the target task is stored by utilizing the task pool, and when the task needs to be processed, the target task to be processed is obtained from the task pool according to the task state of the target task, so that the code packaging device is favorable for being subsequently distributed to the target task to be processed, and the problems that in the prior art, the task distribution is unreasonable and the code packaging device cannot be flexibly scheduled due to the fact that the target task to be processed is directly sent to the code packaging device from the task creating device are solved. On the other hand, a plurality of (here, more than or equal to two) target tasks are stored in the task pool, so that the whole processing state of the target tasks is conveniently recorded, and the follow-up query is conveniently carried out.
Specifically, according to the embodiment of the present invention, the code-related file includes source code, executable code, a related configuration file, a resource file, and the like.
According to an embodiment of the present invention, the task processing device 300 further includes a processing state monitoring device, and before the step of allocating a code packing device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device, the processing state monitoring device is configured to: monitoring the processing state of the code packaging device and confirming that the code packaging device is not in a rest state.
Wherein, the processing state of the code packaging device comprises: in-process, idle, and rest (i.e., rest state, which means that the code packaging apparatus does not perform related task processing due to problems such as failure and system setup). The code packaging device in the rest state does not distribute the target task to be processed.
Further, according to the embodiment of the present invention, the task state of the target task is obtained by monitoring the target task according to the task monitoring period determined by the historical task processing time length.
Specifically, a task monitoring period is determined according to the historical task processing duration, and the task state of each target task is obtained according to the task monitoring period. The step can be executed by a task pool, and the task pool can perform real-time monitoring (namely the task monitoring period is short enough) when being executed; the method can be executed by constructing a task state monitoring device, the task state monitoring device is used for acquiring the task state of each target task and then sending the task state of each target task to the task pool, the monitoring frequency can be reduced when the task state monitoring device is executed, and the pressure of frequent monitoring operation on the performance of the computer is avoided.
The task processing module 302 is configured to allocate a code packing device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device, and send the target task to be processed to the code packing device, so that the code packing device performs packing processing on the target task to be processed.
Through the arrangement, the task platform for creating the tasks is decoupled from the code packaging device for processing the tasks, and the target tasks to be processed are distributed through the task pool, so that the code packaging device is flexibly scheduled under specific conditions, and the task processing distribution is more reasonable.
Specifically, according to the embodiment of the present invention, the task processing queue corresponding to the code packaging apparatus is determined according to the code packaging apparatus information, where the code packaging apparatus information includes at least one of the following information: IP (Internet Protocol Address) Address information, host Address information, device number information, and a task number threshold corresponding to the task processing queue.
According to an embodiment of the present invention, the queue information includes one or more of the following information: the task number, the task type and the processing priority corresponding to the target task to be processed in the current queue.
Further, according to an embodiment of the present invention, the task processing module 302 is further configured to: and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device and the task information of the target task to be processed.
According to a specific implementation manner of the embodiment of the present invention, the task information of the target task to be processed indicates a task type and a processing priority corresponding to the target task to be processed.
According to a specific implementation manner of the embodiment of the invention, after allocating codes to the target tasks to be processed and packaging the codes according to the queue information of the task processing queue corresponding to the code packaging device and the task information of the target tasks to be processed, when the processing state of the code packaging device obtained by monitoring is idle, the target tasks to be processed can be directly selected to be sent to the code packaging device for packaging; when the processing state of the code packaging device obtained by monitoring is in processing, the target task to be processed can be sent to the task processing queue corresponding to the code packaging device, and when the code packaging device is idle, the target task to be processed is obtained from the corresponding task processing queue for packaging.
Further, according to a specific implementation manner of the embodiment of the present invention, the queue information indicates a processing priority of an existing target task to be processed in the task processing queue, and the task information indicates a processing priority of a currently allocated target task to be processed, and for a target task to be processed that needs to be processed urgently (i.e., the processing priority is high), the target task to be processed in the task processing queue may be packed before the target task to be processed.
Further, according to an embodiment of the present invention, the task processing device 300 further includes an adjusting module, and after the step of performing the packing processing on the target task to be processed by the code packing device, the adjusting module is configured to: and adjusting the task state of the processed target task.
According to the technical scheme of the embodiment of the invention, the target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
FIG. 4 is a block diagram of a task processing system provided in accordance with an embodiment of the present invention; as shown in fig. 4, a task processing system 400 provided in the embodiment of the present invention mainly includes:
and the task creating device 401 is configured to create a target task and send the target task to the task pool.
According to the embodiment of the present invention, the task creating device 401 is further configured to build a code packaging device (the 3 code packaging devices shown in fig. 4 are only examples, and the specific number of the code packaging devices may be set according to actual situations).
The task processing device is built through the task platform, so that the specific processing method indicated by the task processing device is matched with the target task to be processed, and the task processing process can be completed more efficiently. Preferably, the task processing device constructed herein may be plural, and the task processing device may be a packaging machine.
Further, according to the embodiment of the present invention, the task creating device 401 is further configured to: and converting the data analysis form of the task processing device information and the plurality of target tasks and then sending the converted data to the task pool.
Specifically, according to the embodiment of the present invention, the task processing device information and the plurality of target tasks are converted into a json form. Since json is a general data analysis mode and various platforms have a corresponding analysis mode with json data, json data is not limited due to different language forms when being transmitted and used among the platforms, and generality and usability in the data transmission process are improved.
Wherein the task processing device information includes IP (Internet Protocol Address) Address information corresponding to the task processing device) and at least one of the following information: host address information, task processing device number information, and task processing queue information corresponding to each task processing device. The node allocation means may be a task scheduling pool or a task matching pool.
A task pool 402, configured to obtain a target task to be processed according to a task state of the target task, where multiple target tasks are stored in the task pool, and each target task includes a code-related file; and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device.
According to an embodiment of the present invention, the task pool 402 is further configured to: and performing task processing queue distribution on each target task to be processed according to the code packing device information.
With the above arrangement, one task processing queue is allocated to each code packaging device based on the IP address information corresponding to the code packaging device indicated in the code packaging device information, the host address information, the number of code packaging devices information, the task processing queue information corresponding to each code packaging device, and the like.
According to an embodiment of the present invention, before the step of performing task processing queue allocation on each target task according to the task processing device information by using the node allocation device, the task processing method further includes: the json string (including the task processing device information and a plurality of target tasks) transmitted by the task creating device is parsed by the task pool 402.
Further, according to an embodiment of the present invention, the task pool 402 is further configured to determine a task monitoring period according to the historical processing time, and obtain a task state of each target task and a processing state of each code packaging apparatus according to the task monitoring period. According to a specific implementation manner of the embodiment of the present invention, the above steps can also be performed by constructing a task monitoring device.
According to a specific implementation manner of the embodiment of the present invention, when the processing state of a certain code packing device is idle, a target task to be processed may be determined from a task processing queue corresponding to the code packing device to be matched with the code packing device, so that the code packing device packs the target task to be processed.
According to another specific implementation manner of the embodiment of the present invention, when the processing state of a certain code packing device is idle, one target task to be processed may be determined from all target tasks to be processed to match with the code packing device, so that the code packing device performs packing processing on the target task to be processed, thereby implementing dynamic allocation of tasks in a task processing queue, and improving the technical effect of task processing efficiency.
And a code packing device 403, configured to receive the target task to be processed, and perform packing processing on the target task to be processed.
According to the embodiment of the present invention, after the code packaging apparatus 403 finishes performing the step of packaging the target task to be processed, the task state of the processed target task may be adjusted through the task pool 402.
The page display platform 404 obtains the latest state (adjusted state) of the target task to perform corresponding page display, so that the user can obtain the execution status of the target task. According to the embodiment of the invention, the page presentation platform can be presented through html5 (core language of the world wide web, an application hypertext markup language under standard general markup language).
According to the technical scheme of the embodiment of the invention, the target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
Fig. 5 shows an exemplary system architecture 500 of a task processing device or a task processing method to which an embodiment of the present invention can be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505 (this architecture is merely an example, and the components included in a particular architecture may be adapted according to application specific circumstances). The network 504 serves to provide a medium for communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 501, 502, 503. The background management server may analyze and perform other processing on the received data such as the target task to be processed and the queue information, and feed back a processing result (for example, the target task to be processed is allocated to a corresponding code packaging device — only an example) to the terminal device.
It should be noted that the task processing method provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the task processing device is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an acquisition module and a task processing module. The names of the modules do not limit the modules themselves in some cases, and for example, the acquiring module may also be described as a "module for acquiring a target task to be processed according to a task state of the target task".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring a target task to be processed according to the task state of the target task, wherein a plurality of target tasks are stored in a task pool and comprise code related files; and distributing the code packaging device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed.
According to the technical scheme of the embodiment of the invention, the target task to be processed is obtained according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; according to the queue information of the task processing queue corresponding to the code packing device, the code packing device is allocated to the target task to be processed, and the target task to be processed is sent to the code packing device, so that the code packing device can perform packing processing on the target task to be processed, the technical problems that the code packing device cannot be flexibly scheduled under a specific condition in the conventional task processing method, the code packing device is unreasonably allocated, dynamic allocation of tasks in the task processing queue cannot be realized, and the task processing efficiency is low are solved, and the code packing device can be flexibly scheduled under the specific condition, so that the allocation of the task to be processed is more reasonable, the dynamic allocation of the tasks in the task processing queue can be realized, and the technical effect of improving the task processing efficiency is achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (11)

1. A task processing method, comprising:
acquiring a target task to be processed according to a task state of the target task, wherein a plurality of target tasks are stored in a task pool and comprise code related files;
and distributing the code packing device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device, and sending the target task to be processed to the code packing device, so that the code packing device packs the target task to be processed.
2. The task processing method according to claim 1, wherein before the step of allocating the code packing device to the target task to be processed according to queue information of a task processing queue corresponding to the code packing device, the task processing method further comprises: and monitoring the processing state of the code packaging device.
3. The task processing method according to claim 1, wherein the task state of the target task is obtained by monitoring the target task according to a task monitoring period determined by a historical task processing duration.
4. The task processing method according to claim 1, wherein the task processing queue corresponding to the code packaging device is determined according to code packaging device information, wherein the code packaging device information includes at least one of the following information: the method comprises the steps of Internet protocol address information, host address information, device number information and task number threshold values corresponding to task processing queues.
5. The task processing method according to claim 1, wherein the queue information includes one or more of the following information: the task number, the task type and the processing priority corresponding to the target task to be processed in the current queue.
6. The task processing method according to claim 1, wherein the step of allocating the code packing device to the target task to be processed according to queue information of a task processing queue corresponding to the code packing device comprises: and distributing the code packing device for the target task to be processed according to the queue information of the task processing queue corresponding to the code packing device and the task information of the target task to be processed.
7. The task processing method according to claim 1, wherein after the step of performing, by the code packaging apparatus, packaging processing on the target task to be processed, the task processing method further comprises: and adjusting the task state of the processed target task.
8. A task processing apparatus, comprising:
the system comprises an acquisition module, a task processing module and a task processing module, wherein the acquisition module is used for acquiring a target task to be processed according to a task state of the target task, a plurality of target tasks are stored in a task pool, and the target task comprises a code related file;
and the task processing module is used for allocating the code packaging device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device and sending the target task to be processed to the code packaging device, so that the code packaging device packages the target task to be processed.
9. A task processing system, comprising:
the task creating device is used for creating a target task and sending the target task to a task pool;
the task pool is used for acquiring a target task to be processed according to the task state of the target task, wherein a plurality of target tasks are stored in the task pool, and the target task comprises a code related file; distributing the code packaging device to the target task to be processed according to the queue information of the task processing queue corresponding to the code packaging device, and sending the target task to be processed to the code packaging device;
and the code packing device is used for receiving the target task to be processed and packing the target task to be processed.
10. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
11. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
CN202010574916.3A 2020-06-22 2020-06-22 Task processing method, device and system Pending CN113760482A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010574916.3A CN113760482A (en) 2020-06-22 2020-06-22 Task processing method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010574916.3A CN113760482A (en) 2020-06-22 2020-06-22 Task processing method, device and system

Publications (1)

Publication Number Publication Date
CN113760482A true CN113760482A (en) 2021-12-07

Family

ID=78785580

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010574916.3A Pending CN113760482A (en) 2020-06-22 2020-06-22 Task processing method, device and system

Country Status (1)

Country Link
CN (1) CN113760482A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114500514A (en) * 2022-02-14 2022-05-13 京东科技信息技术有限公司 File transmission method and device, electronic equipment and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017167070A1 (en) * 2016-03-30 2017-10-05 阿里巴巴集团控股有限公司 Method for copying clustered data, and method and device for determining priority
CN108182111A (en) * 2018-01-23 2018-06-19 百度在线网络技术（北京）有限公司 Task scheduling system, method and apparatus
US20190163524A1 (en) * 2017-11-30 2019-05-30 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for processing task in smart device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017167070A1 (en) * 2016-03-30 2017-10-05 阿里巴巴集团控股有限公司 Method for copying clustered data, and method and device for determining priority
US20190163524A1 (en) * 2017-11-30 2019-05-30 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for processing task in smart device
CN108182111A (en) * 2018-01-23 2018-06-19 百度在线网络技术（北京）有限公司 Task scheduling system, method and apparatus

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SHU-CHENG CHANG: "Applying express-queue-based approach to Software Reliability and cost analysis", IEEE, 24 November 2016 (2016-11-24) *
于琨;张正本;海本斋;: "基于多目标全局约束的任务分配和调度算法", 计算机工程与应用, no. 08, 16 May 2017 (2017-05-16) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114500514A (en) * 2022-02-14 2022-05-13 京东科技信息技术有限公司 File transmission method and device, electronic equipment and computer readable storage medium
CN114500514B (en) * 2022-02-14 2023-12-12 京东科技信息技术有限公司 File transmission method and device for cloud storage, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN107590001B (en) Load balancing method and device, storage medium and electronic equipment
CN109274731B (en) Method and device for deploying and calling web service based on multi-tenant technology
CN109995801B (en) Message transmission method and device
CN109254854A (en) Asynchronous invoking method, computer installation and storage medium
CN112612467A (en) Method and device for processing micro front-end architecture based on qiankun
CN109428926B (en) Method and device for scheduling task nodes
CN110928905A (en) Data processing method and device
CN113821352A (en) Remote service calling method and device
CN110888639A (en) Business code compiling and packaging method and device
CN112104679B (en) Method, apparatus, device and medium for processing hypertext transfer protocol request
CN111427899A (en) Method, device, equipment and computer readable medium for storing file
CN111381899A (en) Method and apparatus for developing target system
CN112084042A (en) Message processing method and device
CN113760264A (en) Method and device for multiplexing components among multiple applications
CN113742389A (en) Service processing method and device
CN113760482A (en) Task processing method, device and system
CN112948138A (en) Method and device for processing message
CN113778499A (en) Method, device, equipment and computer readable medium for publishing service
CN113127225A (en) Method, device and system for scheduling data processing tasks
CN111767126A (en) System and method for distributed batch processing
CN111831503A (en) Monitoring method based on monitoring agent and monitoring agent device
CN113779122B (en) Method and device for exporting data
CN113127561B (en) Method and device for generating service single number, electronic equipment and storage medium
CN112688982B (en) User request processing method and device
CN113064620A (en) Method and device for processing system data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination