CN109815087B - Task processing process monitoring method and device, computer equipment and storage medium - Google Patents
Task processing process monitoring method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109815087B CN109815087B CN201910012587.0A CN201910012587A CN109815087B CN 109815087 B CN109815087 B CN 109815087B CN 201910012587 A CN201910012587 A CN 201910012587A CN 109815087 B CN109815087 B CN 109815087B
- Authority
- CN
- China
- Prior art keywords
- task
- semaphore
- processed
- permission
- license
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The application relates to a method and a device for monitoring a task processing process, computer equipment and a storage medium. Relates to the field of anomaly monitoring, and the method comprises the following steps: generating a task to be processed; adding a task to be processed into a task processing queue of a semaphore; when the task to be processed obtains the semaphore permission, obtaining a first semaphore operation, adding the task to be processed to an entry node of a thread, and obtaining task information of the task to be processed at the entry node; generating a first semaphore operation record according to the first semaphore operation and the task information; executing a task to be processed entering a thread, and acquiring execution result information when the task processing is completed; releasing semaphore permission and acquiring a second semaphore operation; generating a second semaphore operation record according to the execution result information and the second semaphore operation; and generating a task execution log according to the first semaphore operation record and the second semaphore operation record. By adopting the method, the process monitoring can be more comprehensive, and the abnormal positioning can be more accurate and rapid.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for monitoring a task processing process, a computer device, and a storage medium.
Background
During the running process of the program, the program instance loaded into the memory for execution is called a process (process). A process is an execution activity of a program on a computer and is a dynamic execution process of the program. It is an independent unit that can be executed concurrently for resource allocation and scheduling by the system, or it can be said that a process is a running instance of an application program and is a unit for resource allocation by the operating system. When a program is run, a process is started.
Under the scenes of large data volume synchronization, multitasking and the like, in order to improve the processing efficiency, a multiprocess processing program is generally developed to meet the requirement of a service function. When a multi-process processes a service synchronously, monitoring of the service processing status becomes especially difficult. The traditional task monitoring method based on the task execution log needs to manually decode huge and obscure logs, and even if the single task log comprises each link of task processing, the data volume is huge, so that the monitoring effect presented by the traditional mode of monitoring the task processing process based on the log of the task processing dimension is poor, and therefore the inventor thinks that the task process monitoring method which is more comprehensive in task processing process monitoring and more accurate and rapid in abnormal positioning is particularly necessary and urgent.
Disclosure of Invention
Therefore, it is necessary to provide a task processing process monitoring method, device, computer device and storage medium, which can make process monitoring more comprehensive and locate an exception more accurately and quickly.
A method of monitoring a task processing process, the method comprising:
generating a task to be processed;
judging the task type of the task to be processed, and acquiring a semaphore corresponding to the task type;
adding the task to be processed into a task processing queue waiting for the semaphore;
when the task to be processed acquires the semaphore permission of the semaphore, acquiring a first semaphore operation corresponding to the semaphore permission at present, adding the task to be processed to an entry node of a thread, and acquiring task information of the task to be processed at the entry node;
generating a first semaphore operation record according to the first semaphore operation and the task information;
executing the task to be processed entering the thread, when the task processing is completed, releasing the semaphore permission by the thread, acquiring a second semaphore operation corresponding to the released semaphore permission, and acquiring execution result information from an output node of the thread;
generating a second semaphore operation record according to the execution result information and the second semaphore operation;
and generating a task execution log according to the first semaphore operation record and the second semaphore operation record, and monitoring a task processing process according to the task execution log.
In one embodiment, after adding the task to be processed to the task processing queue waiting for the semaphore, the method further includes:
if the semaphore license is full, obtaining a license borrowing configuration of the semaphore, wherein the license borrowing configuration is used for describing a borrowing attribute and a borrowed attribute of the semaphore;
if the borrowing attribute described by the license borrowing configuration is allowed borrowing, searching a target semaphore, wherein the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed;
obtaining a borrowed license from the target semaphore and assigning the borrowed license to the pending task;
when the task to be processed acquires the semaphore permission of the semaphore, acquiring a first semaphore operation corresponding to the semaphore permission currently, including: and distributing the borrowed license to the task to be processed, and acquiring a first semaphore operation corresponding to the borrowed license.
In an embodiment, the adding the task to be processed to an entry node of a thread and acquiring task information of the task to be processed includes:
adding the task to be processed to an entry node of a thread, and calling a first data extraction function to extract task information from the input node;
when the task to be processed entering the thread is executed and the task processing is completed, the method for obtaining the execution result information at the output node of the thread comprises the following steps:
and executing the to-be-processed task entering the thread until the task processing is finished and the to-be-processed task enters the output node of the thread, and calling a second data extraction function to extract execution result information from the output node.
In one embodiment, after obtaining the license borrowing configuration for the semaphore if the semaphore license is full, the method further comprises:
if the borrowing attribute described by the permission borrowing configuration is not allowed to borrow, marking the task to be processed as a blocking waiting permission;
when the number of tasks marked as blocking waiting permission exceeds a set threshold and/or the waiting time exceeds a set threshold, adjusting the initial permission configuration of the corresponding semaphore.
In one embodiment, after the generating a first semaphore operation record according to the first semaphore operation and the task information, the method includes:
generating a monitoring task, wherein the monitoring task is provided with a task identifier in the task information;
monitoring the second semaphore operation record corresponding to the task identifier;
and if the second semaphore operation record is not monitored after the set time is exceeded, generating early warning information.
A task processing progress monitoring apparatus, the apparatus comprising:
the task generating module is used for generating a task to be processed;
the semaphore acquisition module is used for judging the task type of the task to be processed and acquiring a semaphore corresponding to the task type;
the sub-queue module is used for adding the tasks to be processed into the task processing queue of the semaphore;
the first semaphore operation acquisition module is used for acquiring a first semaphore operation corresponding to the semaphore permission currently when the task to be processed acquires the semaphore permission;
the task information extraction module is used for adding the task to be processed to an entry node of a thread and acquiring task information of the task to be processed at the entry node;
the first semaphore operation record generation module is used for generating a first semaphore operation record according to the first semaphore operation and the task information;
a second semaphore operation acquisition module, configured to execute the to-be-processed task entering a thread, where when the task processing is completed, the thread releases the semaphore permission and acquires a second semaphore operation corresponding to the released semaphore permission;
the execution result extraction module is used for acquiring execution result information from the output node of the thread;
a second semaphore operation record generation module, configured to generate a second semaphore operation record according to the execution result information and the second semaphore operation;
and the log generation module is used for generating a task execution log according to the first semaphore operation record and the second semaphore operation record and monitoring a task processing process according to the task execution log.
In one embodiment, the apparatus further comprises:
a license borrowing module for obtaining a license borrowing configuration for the semaphore if the semaphore license is full, wherein the license borrowing configuration is used for describing a borrowing attribute and a borrowed attribute of the semaphore; if the borrowing attribute described by the license borrowing configuration is allowed borrowing, searching a target semaphore, wherein the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed; obtaining a borrowed license from the target semaphore and assigning the borrowed license to the pending task;
the first semaphore operation acquiring module is further configured to acquire a first semaphore operation corresponding to the borrowed license, where the first semaphore operation at least includes a license type and a target semaphore identifier of the borrowed license.
In one embodiment, the task information extraction module is further configured to add the task to be processed to an entry node of a thread, and call a first data extraction function to extract task information from the input node;
the execution result extraction module is further configured to execute the to-be-processed task entering the thread until the task processing is completed and the to-be-processed task enters the output node of the thread, and invoke a second data extraction function to extract execution result information from the output node.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method described above.
According to the task processing process monitoring method, the device, the computer equipment and the storage medium, the ordered execution of the batch tasks to be processed is controlled through the semaphore, the task information and the semaphore operation information are recorded in a correlation mode while the semaphore controls the task flow through permission, the semaphore operation record is generated, monitoring is carried out based on the semaphore operation record, information of two dimensionalities of business and semaphore operation control can be comprehensively considered, the business processing process monitoring not only comprises a task processing aspect, but also deepens the flow direction of process resources, the task processing process monitoring is more comprehensive, and abnormal positioning is more accurate and rapid.
Drawings
FIG. 1 is a diagram illustrating an exemplary implementation of a method for monitoring a task process;
FIG. 2 is a flowchart illustrating a task processing process monitoring method according to an embodiment;
FIG. 3 is a schematic diagram of signal flow control task flow in one embodiment;
FIG. 4 is a flow diagram illustrating the flow involved in semaphore borrowing license in one embodiment;
FIG. 5 is a flow diagram that illustrates the flow involved in generating a semaphore operation record in one embodiment;
FIG. 6 is a block diagram showing the construction of a task process monitoring apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The task processing process method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The method comprises the steps that a terminal 102 initiates a task handling request, a server generates a task to be processed based on the task handling request, the server controls the execution time of the task to be processed through a semaphore, when the task to be processed is executed to obtain an execution permission and the execution permission is released, the operation of the semaphore on the execution permission is associated with task information to generate a semaphore operation record with the task information, and the task processing progress can be accurately tracked based on the semaphore operation record with the task information. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 is a single-point application server, i.e., a stand-alone server.
In one embodiment, as shown in fig. 2, a task processing process monitoring method is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
The server is used to provide services to the terminal, and in order to respond to various operations of the terminal user, the server needs to perform various data processing and operations. The data processing and calculation performed by the server become tasks required to be performed by the server.
It should be noted that the present embodiment is based on a task to be executed sent by a single point application server (i.e., an application server).
And 204, judging the task type corresponding to the task to be processed, and acquiring the semaphore corresponding to the task type.
In this embodiment, all tasks that the single-point application server can provide are divided into different task types in advance, and the granularity of the divided task types may be determined according to specific needs. When the task type is divided, one task may be divided into a plurality of task types, or a plurality of tasks may be defined as the same task type. The method includes dividing a task into a plurality of task types, for example, for a task of inquiring credit report, defining the task from a first inquiry main body set and being an online inquiry as a first task type, defining the task from the first inquiry main body set and being an offline inquiry as a second type, defining the task from the second inquiry main body set and being an online inquiry as a third type, and defining the task from the second inquiry main body set and being an offline inquiry as a fourth type. The tasks can be divided into important tasks and common tasks, all the important tasks are defined as the same task type, and the task type is defined with higher priority, so that the semaphore with more flow is defined for the task type through the semaphore, and the critical tasks can be processed in time preferentially.
Semaphores, also known as semaphores, are resource-oriented signals used in a multi-threaded environment that are responsible for coordinating the use of computing resources by various threads. The semaphore in this embodiment is resource guidance information that enables dynamic guidance of resources. The dynamic behavior of the semaphore embodies the dynamic flow of resources between multiple semaphores through semaphore operations implemented by the semaphore. In this embodiment, the predefined semaphore for the management task includes a static attribute parameter, i.e. an initial number of permissions. The initial grant number of semaphores determines the maximum length of the thread to which the semaphore corresponds, with each semaphore grant corresponding to a thread. In this embodiment, based on the divided task types, a semaphore corresponding to the task type is defined. One semaphore for each task type.
As shown in fig. 3, the job of the unicast server is divided into 5 job types in advance. Task 1, task 2, and task 3 are of a first task type, and task 4, task 5, and task 6 are of a second task type …. Corresponding semaphores, respectively semaphore 1, semaphore 2, semaphore 3, semaphore 4 and semaphore 5, are defined for the 5 task types. The initial permission number of semaphore 1 is 4, and the maximum length of the thread corresponding to semaphore 1 is 4. In fig. 3, all permissions of semaphore 1 have been filled by pending tasks, and there are currently no idle permissions, i.e. no idle threads. The initial grant number of semaphore 3 is 5 and the maximum length of the thread for semaphore 3 is 5. In fig. 3, 3 threads of semaphore 3 are occupied by tasks and there are currently 2 idle grants.
The server generates a task to be processed based on a request of the terminal, and determines a task type corresponding to the task to be processed according to a preset task type definition. And determining the semaphore corresponding to the task to be processed according to the corresponding relation between the task type and the semaphore, and adding the task to be processed into a task processing queue waiting for the semaphore.
With continued reference to fig. 3, a new pending task 7, the pending task 7 belonging to the third task type and the third task type corresponding to the semaphore 3, therefore, the pending task 7 is added to the semaphore-granted task processing queue waiting for the semaphore 3. All tasks in the task processing queue wait for the semaphore grant given by semaphore 3. The previous tasks in the semaphore's 3 task processing queue, task 7, task 8, and task 9, have acquired threads and are in the processing state.
And step 208, when the to-be-processed task acquires the semaphore permission, acquiring a first semaphore operation corresponding to the current semaphore permission, adding the to-be-processed task to an entry node of the thread, and acquiring task information of the to-be-processed task.
And after the task to be processed is added into the task processing queue of the semaphore, waiting for obtaining the semaphore permission. The permission released by the semaphore processing completed task is dispatched to the tasks to be processed in the task processing queue one by one. When the task to be processed acquires the semaphore permission, the task to be processed can acquire a thread, and the thread is used for processing the task to be processed which acquires the semaphore permission.
And when the task to be processed acquires the semaphore permission, adding the task to be processed to an entry node of the thread, calling a data extraction function by the server, and extracting task information from the task to be processed. Wherein, the extracted task information may include: task identification, order number or task number, requester information, etc. The extracted task information can be configured according to specific monitoring requirements, a corresponding data extraction function is defined according to the data extraction requirements, and the data extraction function can extract the required task information from the tasks to be processed.
Meanwhile, when the to-be-processed task acquires the semaphore permission, the server acquires the semaphore from the semaphore handle and operates the semaphore in real time according to the semaphore permission of the to-be-processed task. This semaphore operation is named a first semaphore operation for distinguishing a second semaphore operation hereinafter.
The semaphore handle records each step of the semaphore's operation and the real-time state of the semaphore. Such as semaphore use idle grant, semaphore reclaim grant, number of semaphore idle grants, etc. After extracting task information from a task to be processed entering a node, the server acquires semaphore operation from the handle and generates a first semaphore operation record corresponding to the task to be processed by combining the semaphore operation and the task information.
In one embodiment, when the task to be processed acquires the semaphore permission of its corresponding semaphore, the server acquires the semaphore operation corresponding to the semaphore permission from the semaphore handle. The semaphore operation to which the semaphore grant corresponds includes the type of semaphore grant, the source of the grant, and the destination. Such as a semaphore operation may be an idle grant derived from a self semaphore. The source of the semaphore operation is a self semaphore, the destination is also the self semaphore, and the type of the grant is an idle grant. As shown in fig. 3, the pending task 7 enters the task processing queue waiting for the semaphore grant and gets an idle grant for semaphore 3. The idle grant originates from semaphore 3 and is assigned to the pending task 7 corresponding to semaphore 3. For another example, when a new task 13 comes, the semaphore corresponding to the task 13 is semaphore 5, and all permissions of semaphore 5 are occupied, at this time, task 13 will be in a wait permission state until semaphore 5 acquires an available permission. The executing task 13 executes the complete task, and releases the semaphore permission of the occupied semaphore 5, and the semaphore 5 acquires the release permission, at this time, the semaphore 5 distributes the release permission to the task 13 in idle permission, and the task 13 acquires the semaphore permission, and the semaphore permission corresponding to the semaphore permission operates as follows: and obtaining the release permission of the self semaphore. The first semaphore operation corresponding to the semaphore grant is therefore: the release grant is derived from the self-semaphore and removed from the self-semaphore.
And step 210, generating a first semaphore operation record according to the first semaphore operation and the task information.
The first semaphore operation record comprises task information, semaphore operation, task acquisition permission time and first semaphore operation record generation time. Since the first semaphore operation record is generated for the to-be-processed task once the to-be-processed task acquires the semaphore permission, the record generation time is very close to the task acquisition permission time, and the task acquisition permission time can be used as the record generation time.
Once the to-be-processed task acquires the semaphore permission, the to-be-processed task acquires the thread through an entry node of the thread. Therefore, the to-be-processed task execution time is very close to the task acquisition permission time, and the task permission acquisition time can be taken as the to-be-executed task execution time.
After extracting task information from the task to be processed, the task to be processed enters a thread, the thread processes the task, the task enters an output node of the thread after the task is executed, and execution result information of the task is obtained from the output node of the thread. After the task execution is completed, the thread will automatically release the permission.
It should be noted that the thread entry node and the thread output node are virtual nodes, and the thread entry node may be a set time interval before the task to be executed is actually added to the thread. Accordingly, the thread output node is a set time interval before the thread is released (when the thread is released, the semaphore grant is also released) after the task execution is completed. The execution result information may include a task identification, an order number or task number, an output result (normal or abnormal), a processing time, and the like.
And the second semaphore operation corresponding to the task to be processed is the semaphore operation which is released and allowed to correspond to the task after the task is processed. The semaphore operation may also be retrieved from the semaphore handle corresponding to the semaphore. The second semaphore operation also includes the type of license, the origin of the license. The type of the grant of the second semaphore operation is a release grant (or the grant is a borrow grant, and the type of the grant is a borrow grant), and the sources of the grants are all self semaphores. As shown in fig. 3, after the task 7 is completed, the semaphore grant is released, the source of the semaphore grant is the semaphore 3, and the destination of the grant may be an idle grant as the semaphore 3, may be another task to be executed allocated to the semaphore 3, or may be borrowed to another semaphore, so that the destination of the grant may be the self-semaphore, the self-semaphore task, or another semaphore.
And step 214, generating a second semaphore operation record according to the execution result information and the second semaphore operation.
The second semaphore operation record comprises task information, semaphore operation, task release permission time and second semaphore operation record generation time.
And step 216, generating a task execution log according to the first semaphore operation record and the second semaphore operation record, and monitoring a task processing process according to the task execution log.
In the embodiment, the ordered execution of the batch tasks to be processed is controlled through the semaphore, the task information and the semaphore operation information are recorded in an associated mode while the semaphore controls the task flow through permission, the semaphore operation record is generated, monitoring is carried out based on the semaphore operation record, information of two dimensionalities of service and semaphore control operation can be comprehensively considered, the monitoring of a service processing process not only comprises a task processing aspect, but also deepens the flow direction of process resources, the task processing process is more comprehensive, and abnormal positioning is more accurate and rapid.
In addition, since the semaphore is defined by classification according to the grouping situation of the task, the corresponding operation record is also a classification record. The type of the abnormal task can be quickly defined by inquiring the semaphore operation record, and the abnormal task can be positioned in a fine-grained mode according to the recorded task information and the task processing result information.
In one embodiment, as shown in fig. 4, after adding the task to be processed to the task processing queue waiting for the semaphore at step 206, the method further includes:
if the semaphore grant is full, a grant borrowing configuration of the semaphore is obtained, step 402, wherein the grant borrowing configuration is used for describing a borrowing attribute and a borrowed attribute of the semaphore.
The borrowing attribute and the borrowed attribute of the semaphore represent the dynamic property of the semaphore. When a server receives a task to be processed, firstly, a task type is determined according to task data of the task to be processed, and then, a semaphore corresponding to the task to be processed is obtained according to a predefined corresponding relation between the task type and the semaphore.
The server acquires the current license state corresponding to the semaphore, wherein the current license state of the semaphore comprises the remaining license number, the lent license number and the borrowed license number. Wherein the remaining number of licenses is the initial number of licenses of the semaphore minus the number of occupied licenses. If the initial license number of the semaphore is fully occupied, i.e. the remaining license number is 0, the license for the semaphore is full.
The borrowing attribute in the semaphore grant borrowing configuration describes whether or not a grant for other semaphores is allowed to be borrowed. The borrowed attribute describes whether other semaphores are allowed to borrow a license.
In step 404, if the borrowing attribute described by the license borrowing configuration is allowed borrowing, a target semaphore is searched, and the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed.
At step 406, borrowing grants are obtained from the target semaphore and may be distributed to pending tasks.
And if the permission of the semaphore corresponding to the task to be processed is full, obtaining the permission borrowing configuration of the semaphore. If the borrowing attribute of the semaphore is to allow borrowing, the borrowing permission is allowed to be obtained from other semaphores. The server looks up if there is a target semaphore for which licenses can be borrowed. The target semaphore needs to have two conditions, one is that the borrowed attribute of the target semaphore is allowed to be borrowed, and the other two target semaphores have spare permission. If the target semaphore meeting the above condition is found, the idle permission is borrowed from the target semaphore. The free licenses borrowed from the target semaphore are named borrowed licenses.
After the semaphore corresponding to the task to be processed obtains the borrowed license from the target semaphore, the borrowed license is distributed to the task to be processed, so that the state of the task to be processed is the obtained license.
In this embodiment, when there is no available grant for the semaphore corresponding to the to-be-processed task, according to the grant borrowing configuration defined for each semaphore, obtaining a borrowing grant from other semaphores, and allocating the borrowing grant to the to-be-processed task, the to-be-processed task obtains the semaphore grant, then step 208 is performed, and when the to-be-processed task obtains the semaphore grant, the to-be-processed task is added to the entry node of the thread, and task information of the to-be-processed task is obtained.
In one embodiment, in step 208, the operation of obtaining the first semaphore corresponding to the current semaphore permission is: and acquiring a first semaphore operation corresponding to the semaphore permission obtained by the task to be processed from the semaphore handle. Wherein the first semaphore operates to borrow a license, the source of the license is the borrowed semaphore, and the destination of the license is the self semaphore.
Correspondingly, in step 212, the thread releases the semaphore permission, and the server obtains a second semaphore operation corresponding to the current semaphore permission, which is: when the execution of the task is completed, the thread releases the semaphore permission. In one embodiment, according to the semaphore license borrowing rule, the semaphore will return the semaphore license to the borrowed semaphore or continue to be allocated to its own task to be executed, and the like. The semaphore will handle the borrowing permission according to predefined semaphore borrowing rules. The semaphore handle will record where the released semaphore has been granted. And the server acquires the semaphore operation corresponding to the semaphore from the semaphore handle. The second semaphore operation, as corresponding to the released semaphore grant, may be: returning the borrowed license to the borrowed semaphore,
in this embodiment, by defining the license borrowing configuration for each semaphore, resources between semaphores can be flexibly allocated. In addition, the semaphore operation record generated based on the semaphore operation and the task information corresponding to the semaphore operation further comprises the borrowing relation between the semaphores. Monitoring is carried out based on the semaphore operation record, and the traffic borrowing relation among the service, the semaphore control and the semaphore can be comprehensively considered, so that the service processing process is monitored more comprehensively, and abnormal positioning is more accurate and rapid.
In addition, since the semaphore is defined by classification according to the grouping situation of the task, the corresponding operation record is also a classification record. The type of the abnormal task can be quickly defined by inquiring the semaphore operation record, and other abnormal tasks with borrowing relations can be quickly found.
In one embodiment, in step 208, when the to-be-processed task obtains the semaphore permission, the to-be-processed task is added to an entry node of the thread, and task information of the to-be-processed task is obtained, which includes: and when the task to be processed acquires the semaphore permission, adding the task to be processed to an entry node of the thread, and calling a first data extraction function to extract task information from the task to be processed.
As shown in fig. 5, the horizontal axis represents the time axis of the handle semaphore operation, such as the operation time axis of semaphore 3, at time t1, a task to be processed enters the entry node of the thread, at this time, the first data extraction function is called to extract the task information, and the semaphore operation corresponding to time t1 is extracted from the semaphore operations recorded in the handle, and the semaphore operation and the task information constitute one operation record of the semaphore, such as borrowing permission from semaphore 4. Accordingly, at time t2, the task processing is complete and proceeds to the output node of the thread, at which point a second data extraction function is invoked to extract the task processing results and extract from the handle the semaphore operation generated by the semaphore at time t2, the semaphore operation and the task processing results forming another record of the semaphore operation.
In this embodiment, a data extraction function is predefined, different data extraction functions are defined for the entry node and the output node, different data extraction rules are defined by the different data extraction functions, and the different data extraction rules can be defined as needed, so that the task information carried in the semaphore-based operation record can more efficiently and accurately locate the abnormality.
In one embodiment, at step 402, if the semaphore license is full, obtaining a license borrowing configuration for the semaphore, wherein the license borrowing configuration is used for describing borrowing attributes and borrowed attributes of the semaphore, further comprising: if the borrowing attribute described by the license borrowing configuration is that borrowing is not allowed, the pending task is marked as a block wait license. When the number of tasks marked as blocking waiting permission exceeds a set threshold and/or the waiting time exceeds a set threshold, the initial permission configuration of the corresponding semaphore is adjusted.
And if the permission of the semaphore corresponding to the task to be processed is full and the borrowing attribute of the semaphore corresponding to the task to be processed is that the semaphore is not allowed to be borrowed, namely the semaphore can not borrow the permission from other semaphores, marking the task to be processed as a blocking waiting permission. Accordingly, the pending tasks generated thereafter are also marked as blocking wait permissions.
Monitoring the number of the tasks to be processed of the blocking wait permission and the time of the blocking wait (only monitoring the time of waiting for the first task to be processed of the blocking wait permission), if the number of the tasks to be processed of the blocking wait permission exceeds a set threshold, or the waiting time of the initial task to be processed exceeds a set threshold, or both the number of the tasks to be processed and the waiting time exceed corresponding set values, the server adjusts the permission configuration of the semaphore corresponding to the queue of the tasks to be processed, wherein the permission configuration comprises an initial permission number and permission borrowing configuration.
For example, if the number of pending tasks for blocking the pending grant exceeds a set threshold, or the waiting time of the first pending task exceeds a set threshold, or both the number of pending tasks and the waiting time exceed corresponding set values, the initial grant number of the corresponding semaphore is increased, or the configuration of the semaphore to be borrowed from the disallowed grant is adjusted to the configuration of the allowed borrowing grant, or the upper limit of the allowed borrowing grant of the corresponding semaphore is increased.
And correspondingly, adjusting the semaphore configuration corresponding to the unsaturated semaphore. If the remaining allowable number of semaphores is at a high value for a long time, the semaphore is an unsaturated semaphore. The initial number of licenses for unsaturated semaphores and the license borrowing configuration are adjusted. Specifically, the initial number of licenses for the unsaturated semaphore is reduced, the required borrowing of the unsaturated semaphore is configured not to allow borrowing, and the disallowed borrowing is configured to allow borrowing.
In this embodiment, when the semaphore dynamically allocates the multi-channel traffic, the real-time task processing state of each channel is monitored, and the configuration of the semaphore is dynamically adjusted according to the real-time task processing state, so that the traffic control of the semaphore is more flexible.
In one embodiment, after an entry node of a thread generates a first semaphore operation record, a record monitoring task is generated, the monitoring task is associated with a task identifier and is used for monitoring a second semaphore operation record generated at an output node, whether the generated second semaphore operation record corresponds to the task identifier is judged, and if the second semaphore operation record corresponding to the corresponding task identifier is not monitored for a long time, early warning information is generated according to the monitoring task, wherein the early warning information comprises the first semaphore operation record, namely information such as thread entry time, task identifier, semaphore identifier, permission type and permission source.
In the embodiment, the automatic monitoring of the task processing abnormity is realized.
It should be understood that although the steps in the flowcharts of fig. 2 and 4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 and 4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 6, there is provided a task processing progress monitoring apparatus including:
and a task generating module 602, configured to generate a task to be processed.
A semaphore obtaining module 604, configured to determine a task type of the task to be processed, and obtain a semaphore corresponding to the task type.
A queue splitting module 606, configured to add the to-be-processed task to a task processing queue of the semaphore.
A first semaphore operation obtaining module 608, configured to, when the task to be processed obtains a semaphore permission, obtain a first semaphore operation corresponding to the semaphore permission currently.
A task information extraction module 610, configured to add the to-be-processed task to an entry node of a thread, and obtain task information of the to-be-processed task at the entry node;
a first semaphore operation record generating module 612, configured to generate a first semaphore operation record according to the first semaphore operation and the task information.
A second semaphore operation acquiring module 614, configured to execute the to-be-processed task entering a thread, where when the task processing is completed, the thread releases the semaphore permission, and acquires a second semaphore operation corresponding to the released semaphore permission.
And an execution result extraction module 616, configured to obtain execution result information from an output node of the thread.
A second semaphore operation record generating module 618, configured to generate a second semaphore operation record according to the execution result information and the second semaphore operation.
And a log generating module 620, configured to generate a task execution log according to the first semaphore operation record and the second semaphore operation record, and monitor a task processing process according to the task execution log.
In one embodiment, the task processing progress monitoring apparatus further includes:
a license borrowing module for obtaining a license borrowing configuration for the semaphore if the semaphore license is full, wherein the license borrowing configuration is used to describe a borrowed attribute and a borrowed attribute of the semaphore; if the borrowing attribute described by the license borrowing configuration is allowed borrowing, searching a target semaphore, wherein the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed; obtaining a borrowed license from the target semaphore and assigning the borrowed license to the pending task.
The first semaphore operation acquisition module 608 is further configured to acquire a first semaphore operation corresponding to the borrowed license, where the first semaphore operation comprises at least a license type and a target semaphore identification for the borrowed license.
In one embodiment, the task information extraction module 610 is further configured to add the task to be processed to an entry node of a thread, and call a first data extraction function to extract task information from the entry node.
The execution result extraction module 616 is further configured to execute the to-be-processed task entering the thread until the task processing is completed and enters the output node of the thread, and invoke a second data extraction function to extract execution result information from the output node.
In one embodiment, the task processing progress monitoring apparatus further includes: a permission configuration adjusting module, configured to mark the to-be-processed task as a blocking waiting permission if the borrowing attribute described in the permission borrowing configuration is that borrowing is not allowed; and when the number of tasks marked as blocking waiting permission exceeds a set threshold and/or the waiting time exceeds a set threshold, adjusting the initial permission configuration of the corresponding semaphore.
In one embodiment, the task processing progress monitoring apparatus further includes: the monitoring module is used for generating a monitoring task, and the monitoring task is provided with a task identifier in the task information; monitoring the second semaphore operation record corresponding to the task identifier; and if the second semaphore operation record is not monitored after the set time is exceeded, generating early warning information.
For specific limitations of the task process monitoring device, reference may be made to the above limitations of the task process monitoring method, which are not described herein again. The modules in the task processing progress monitoring device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the semaphore. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of task process monitoring.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
A computer device comprising one or more processors and a memory having stored therein computer readable instructions which, when executed by the processors, implement the steps of a method of task process progress monitoring as provided in any one of the embodiments of the present application.
One or more non-transitory storage media storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to implement the steps of the task processing process monitoring method provided in any one of the embodiments of the present application.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus (Rambus) direct RAM (RDRAM), direct bused dynamic RAM (DRDRAM), and bused dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of monitoring a task processing process, the method comprising:
generating a task to be processed;
judging the task type of the task to be processed, and acquiring a semaphore corresponding to the task type;
adding the task to be processed into a task processing queue waiting for the semaphore;
when the task to be processed acquires the semaphore permission of the semaphore, acquiring a first semaphore operation corresponding to the semaphore permission at present, adding the task to be processed to an entry node of a thread, and acquiring task information of the task to be processed at the entry node;
generating a first semaphore operation record according to the first semaphore operation and the task information;
executing the task to be processed entering the thread, when the task processing is completed, releasing the semaphore permission by the thread, acquiring a second semaphore operation corresponding to the released semaphore permission, and acquiring execution result information from an output node of the thread;
generating a second semaphore operation record according to the execution result information and the second semaphore operation;
and generating a task execution log according to the first semaphore operation record and the second semaphore operation record, and monitoring a task processing process according to the task execution log.
2. The method of claim 1, after adding the task to be processed to a task processing queue waiting for the semaphore, further comprising:
if the semaphore license is full, obtaining a license borrowing configuration of the semaphore, wherein the license borrowing configuration is used for describing a borrowing attribute and a borrowed attribute of the semaphore;
if the borrowing attribute described by the license borrowing configuration is allowed borrowing, then a target semaphore is searched, and the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed;
obtaining a borrowed license from the target semaphore and assigning the borrowed license to the pending task;
when the task to be processed acquires the semaphore permission of the semaphore, acquiring a first semaphore operation corresponding to the semaphore permission currently, including:
and distributing the borrowed license to the task to be processed, and acquiring a first semaphore operation corresponding to the borrowed license.
3. The method according to claim 1, wherein the adding the task to be processed to an entry node of a thread and acquiring task information of the task to be processed comprises:
adding the task to be processed to an entry node of a thread, and calling a first data extraction function to extract task information from the task to be processed;
when the task to be processed entering the thread is executed and the task processing is completed, the method for obtaining the execution result information at the output node of the thread comprises the following steps:
and executing the to-be-processed task entering the thread until the task processing is finished and the to-be-processed task enters the output node of the thread, and calling a second data extraction function to extract execution result information from the output node.
4. The method of claim 2, further comprising, after obtaining a license borrowing configuration for the semaphore if the semaphore license is full, further comprising:
if the borrowing attribute described by the permission borrowing configuration is not allowed to borrow, marking the task to be processed as a blocking waiting permission;
when the number of tasks marked as blocking waiting permission exceeds a set threshold and/or the waiting time exceeds a set threshold, adjusting the initial permission configuration of the corresponding semaphore.
5. The method according to any of claims 1-4, comprising, after said generating a first semaphore operation record based on said first semaphore operation and said task information:
generating a monitoring task, wherein the monitoring task is provided with a task identifier in the task information;
monitoring the second semaphore operation record corresponding to the task identifier;
and if the second semaphore operation record is not monitored after the set time is exceeded, generating early warning information.
6. A task processing progress monitoring apparatus, characterized in that the apparatus comprises:
the task generating module is used for generating a task to be processed;
the semaphore acquisition module is used for judging the task type of the task to be processed and acquiring a semaphore corresponding to the task type;
the sub-queue module is used for adding the tasks to be processed into a task processing queue waiting for the semaphore;
a first semaphore operation acquisition module, configured to acquire, when the to-be-processed task acquires a semaphore permission of the semaphore, a first semaphore operation corresponding to the semaphore permission currently;
the task information extraction module is used for adding the task to be processed to an entry node of a thread and acquiring task information of the task to be processed at the entry node;
the first semaphore operation record generation module is used for generating a first semaphore operation record according to the first semaphore operation and the task information;
a second semaphore operation acquisition module, configured to execute the to-be-processed task entering a thread, where when task processing is completed, the thread releases the semaphore permission, and acquires a second semaphore operation corresponding to the released semaphore permission;
the execution result extraction module is used for acquiring execution result information from the output node of the thread;
a second semaphore operation record generation module, configured to generate a second semaphore operation record according to the execution result information and the second semaphore operation;
and the log generation module is used for generating a task execution log according to the first semaphore operation record and the second semaphore operation record and monitoring a task processing process according to the task execution log.
7. The apparatus of claim 6, further comprising:
a license borrowing module for obtaining a license borrowing configuration for the semaphore if the semaphore license is full, wherein the license borrowing configuration is used for describing a borrowing attribute and a borrowed attribute of the semaphore; if the borrowing attribute described by the license borrowing configuration is allowed borrowing, searching a target semaphore, wherein the borrowed attribute described by the license borrowing configuration of the target semaphore is allowed to be borrowed; obtaining a borrowed license from the target semaphore and assigning the borrowed license to the pending task;
the first semaphore operation acquisition module is further configured to allocate the borrowed license to the to-be-processed task, and acquire a first semaphore operation corresponding to the borrowed license.
8. The device according to claim 6, wherein the task information extraction module is further configured to add the task to be processed to an entry node of a thread, and call a first data extraction function to extract task information from the task to be processed;
the execution result extraction module is further configured to execute the to-be-processed task entering the thread until the task processing is completed and the to-be-processed task enters the output node of the thread, and invoke a second data extraction function to extract execution result information from the output node.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910012587.0A CN109815087B (en) | 2019-01-07 | 2019-01-07 | Task processing process monitoring method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910012587.0A CN109815087B (en) | 2019-01-07 | 2019-01-07 | Task processing process monitoring method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109815087A CN109815087A (en) | 2019-05-28 |
| CN109815087B true CN109815087B (en) | 2022-05-20 |
Family
ID=66604035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910012587.0A Active CN109815087B (en) | 2019-01-07 | 2019-01-07 | Task processing process monitoring method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109815087B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111741080B (en) * | 2020-06-02 | 2023-09-29 | 拉卡拉支付股份有限公司 | Network file distribution method and device |
| CN113672364B (en) * | 2021-08-02 | 2023-09-01 | 北京奇艺世纪科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043667A (en) * | 2010-11-25 | 2011-05-04 | 深圳市科陆电子科技股份有限公司 | Task scheduling method for embedded operating system |
| JP2011232956A (en) * | 2010-04-27 | 2011-11-17 | Clarion Co Ltd | Computer system and program |
| CN102981904A (en) * | 2011-09-02 | 2013-03-20 | 阿里巴巴集团控股有限公司 | Task scheduling method and system |
| CN104572277A (en) * | 2014-12-17 | 2015-04-29 | 大唐移动通信设备有限公司 | Thread flow control method and thread flow control device |
| CN106874083A (en) * | 2017-01-03 | 2017-06-20 | 杭州医学院 | A kind of data actuation man-machine interface method for scheduling task |
| CN108319495A (en) * | 2017-01-16 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Task processing method and device |
-
2019
- 2019-01-07 CN CN201910012587.0A patent/CN109815087B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011232956A (en) * | 2010-04-27 | 2011-11-17 | Clarion Co Ltd | Computer system and program |
| CN102043667A (en) * | 2010-11-25 | 2011-05-04 | 深圳市科陆电子科技股份有限公司 | Task scheduling method for embedded operating system |
| CN102981904A (en) * | 2011-09-02 | 2013-03-20 | 阿里巴巴集团控股有限公司 | Task scheduling method and system |
| CN104572277A (en) * | 2014-12-17 | 2015-04-29 | 大唐移动通信设备有限公司 | Thread flow control method and thread flow control device |
| CN106874083A (en) * | 2017-01-03 | 2017-06-20 | 杭州医学院 | A kind of data actuation man-machine interface method for scheduling task |
| CN108319495A (en) * | 2017-01-16 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Task processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 实时多任务测试执行控制机制的实现;田孟光;《中国优秀硕士学位论文全文数据库》;20131215;第I138-421页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109815087A (en) | 2019-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108845884B (en) | Physical resource allocation method, device, computer equipment and storage medium | |
| CN110297711B (en) | Batch data processing method, device, computer equipment and storage medium | |
| US11216324B2 (en) | Query watchdog | |
| CN110297701B (en) | Data processing job scheduling method, device, computer equipment and storage medium | |
| EP0272836B1 (en) | Controlled dynamic load balancing for a multiprocessor system | |
| CN111708627B (en) | Task scheduling method and device based on distributed scheduling framework | |
| CN108319495B (en) | Task processing method and device | |
| CN109815087B (en) | Task processing process monitoring method and device, computer equipment and storage medium | |
| CN110597858A (en) | Task data processing method and device, computer equipment and storage medium | |
| CN110704177B (en) | Computing task processing method and device, computer equipment and storage medium | |
| CN108205469B (en) | MapReduce-based resource allocation method and server | |
| CN110971591A (en) | Method and system for multi-process access to trusted application | |
| CN104239147A (en) | Method and system for processing deadlock cycle | |
| US20110219378A1 (en) | Iterative data parallel opportunistic work stealing scheduler | |
| CN106528065B (en) | A kind of thread acquisition methods and equipment | |
| CN113010286A (en) | Parallel task scheduling method and device, computer equipment and storage medium | |
| CN114625507A (en) | Task scheduling method, system, equipment and storage medium based on directed acyclic graph | |
| WO2019200762A1 (en) | Insurance platform data processing method, electronic device and computer-readable storage medium | |
| CN116450328A (en) | Memory allocation method, memory allocation device, computer equipment and storage medium | |
| CN110298549B (en) | Project task processing method, device, equipment and medium for airport construction engineering | |
| CN110275777B (en) | Resource scheduling system | |
| CN111273962B (en) | Configuration management method, device, computer readable storage medium and computer equipment | |
| CN116881003A (en) | Resource allocation method, device, service equipment and storage medium | |
| CN111858020A (en) | User resource limiting method, device and computer storage medium | |
| CN113159657A (en) | Execution resource allocation method, device and storage medium for procedures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |