CN113835958B - Distributed task execution progress monitoring method, device, medium and equipment - Google Patents
Distributed task execution progress monitoring method, device, medium and equipment Download PDFInfo
- Publication number
- CN113835958B CN113835958B CN202111105334.1A CN202111105334A CN113835958B CN 113835958 B CN113835958 B CN 113835958B CN 202111105334 A CN202111105334 A CN 202111105334A CN 113835958 B CN113835958 B CN 113835958B
- Authority
- CN
- China
- Prior art keywords
- thread
- target task
- progress
- execution
- processing data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 104
- 238000012544 monitoring process Methods 0.000 title claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 110
- 230000008569 process Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000012806 monitoring device Methods 0.000 description 4
- 241000189662 Calla Species 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a method, a device, a medium and equipment for monitoring the execution progress of a distributed task. Wherein the method comprises the following steps: acquiring processing data of each thread of at least one target task of an access program through a software development kit deployed in a client; determining the execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task; reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display. According to the technical scheme, visual progress display service and timely alarm service can be provided for the access program, so that operators can conveniently observe the execution condition of the access program in time, monitor the availability of the service in time and protect navigation for normal operation of the service.
Description
Technical Field
The embodiment of the application relates to the technical field of electronic information, in particular to a method, a device, a medium and equipment for monitoring the execution progress of a distributed task.
Background
In a distributed task system, there are a large number of access procedures. The task type, the number of threads, the thread type and the like of the access program are complex. In the execution process of a large number of access programs, the data volume is large and the execution time is long.
In the prior art, for abnormal tasks, the distributed task system only sends out an alarm prompt when the running of the access program is finished, and operators check and process the access program through the alarm prompt, so that the access program is re-executed after the program fault is solved.
In the prior art, the task progress of the access program is invisible, operators cannot find the program faults in time, a great amount of time is wasted by re-executing the program after the program faults are solved, and the service cannot be normally operated easily, so that the economic benefit of enterprises is influenced.
Disclosure of Invention
The embodiment of the application provides a monitoring method, a device, a medium and equipment for the execution progress of distributed tasks, which can determine the execution progress of each thread for executing each target task by acquiring the processing data of each thread of the target task, provide a visual progress display and timely alarm service for an access program, facilitate operators to observe the execution condition of the access program in time, monitor the availability of the service in time and protect navigation for normal operation of the service.
In a first aspect, an embodiment of the present application provides a method for monitoring a distributed task execution progress, where the method includes:
acquiring processing data of each thread of at least one target task of an access program through a software development kit deployed in a client;
determining the execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task;
reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
In a second aspect, an embodiment of the present application provides a monitoring device for a distributed task execution progress, where the device includes:
the processing data acquisition module is used for acquiring processing data of each thread of at least one target task of the access program through a software development kit deployed in the client;
the execution progress determining module is used for determining the execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task;
and the execution progress reporting module is used for reporting the execution progress of each thread for executing each target task so as to be summarized and counted by the central processing service, determining the execution progress of the distributed task and feeding back to the monitoring end for display.
In a third aspect, embodiments of the present application provide a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements a method for monitoring the progress of distributed task execution according to embodiments of the present application.
In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of being executed by the processor, where the processor executes the computer program to implement a method for monitoring a distributed task execution progress according to an embodiment of the present application.
According to the technical scheme provided by the embodiment of the application, the processing data of each thread of at least one target task of an access program is obtained through a software development kit deployed in a client; determining the execution progress of each thread for executing each target task according to the processing data of each thread of at least one target task; reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to the monitoring end for display. According to the technical scheme, the execution progress of each thread for executing each target task can be determined by acquiring the processing data of each thread of the target task, a visual progress display and a timely alarm service are provided for the access program, an operator can observe the execution condition of the access program in time, the availability of the service is monitored in time, and the normal operation of the service is protected.
Drawings
FIG. 1 is a flowchart of a method for monitoring the progress of distributed task execution according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a distributed task execution progress monitoring service structure according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for monitoring the execution progress of a distributed task according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a monitoring device for distributed task execution progress according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.
Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts steps as a sequential process, many of the steps may be implemented in parallel, concurrently, or with other steps. Furthermore, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example 1
Fig. 1 is a flowchart of a method for monitoring a distributed task execution progress according to an embodiment of the present application, where the embodiment is applicable to any distributed task execution progress monitoring scenario, and the method may be performed by a device for monitoring a distributed task execution progress according to the embodiment of the present application, where the device may be implemented by software and/or hardware, and may be integrated in an electronic device.
As shown in fig. 1, the method for monitoring the execution progress of the distributed task includes:
s110, processing data of each thread of at least one target task of the access program is obtained through a software development kit deployed in the client.
In the actual software development process, a large number of calculation tasks and batch running tasks are often required to be run, and a single client is difficult to complete all tasks. Thus, the distributed system task management center may distribute the central task to a plurality of different clients for execution to achieve large-scale program operation. In addition, in the operation process of the application software, the application software running on different clients needs to interact with the application server, and at this time, the distributed tasks of the clients need to be managed. The monitoring method of the distributed task execution progress can facilitate the development and management of the software program by operators.
The scheme can be executed by a client, and the client can be a computer, a server, a tablet personal computer, a smart phone and the like. A software development kit (Software Development Kit, SDK) is deployed on the client, which may be a collection of development tools that a software developer uses for application software created for a particular software package, software framework, hardware platform, operating system, etc. The software development tools may be programming languages, i.e. different software development tools, for example in the Java programming language, the software development tool package of which has a specific name called Java development tool package (Java Development Kit, JDK). The software development kit can be a software development kit in client system software, such as an SDK in an Android system, or a software development kit in client application software, such as an SDK in a payment treasures application. The software development kit can include a basic class library, a development component and the like. The software development kit may have an application program interface (Application Programming Interface, API) that may enable interaction of the software development kit with the access program. Taking a shopping application and a payment application as an example, assume that when a user places an order for payment in the shopping application, the shopping application can call a software development kit in the payment application through a payment instruction to complete an order payment task.
The software development tool package can acquire processing data of each thread of a target task of an access program through an application program interface, wherein the access program can be application software installed in a client operating system. Fig. 2 is a schematic diagram of a distributed task execution progress monitoring service structure provided by an embodiment of the present invention, and fig. 2 illustrates two access procedures in a distributed system, where an access procedure a and an access procedure B may be application software on two different clients. The access procedure may include one or more target tasks, each of which may include one or more threads. Taking the access procedure a in fig. 2 as an example, the access procedure a takes 3 target tasks as examples, namely a target task a, a target task B and a target task C, wherein 3 threads as examples are taken as 3 threads in the 3 target tasks, namely a thread 1, a thread 2 and a thread 3. The thread is the minimum unit of program execution, one thread is a single-order control flow in a target task, a plurality of threads can be concurrent in one target task, and each thread executes different subtasks. For example, assuming that target task A needs to complete a task that records the sum of two input data, thread 1 may be the read input data, thread 2 may be the sum of two input data calculated, and thread 3 may be the write data. The software development kit in the client may obtain, through the application program interface, processing data for each thread of at least one target task of the access program.
In this solution, optionally, the acquiring, by a software development kit deployed in a client, processing data of each thread of at least one target task of an access program includes:
maintaining a counter through a software development kit deployed in the client to count the calling times of an update progress method called in the execution process of each thread of the target task;
and providing the counting result to a local statistical thread of the software development kit, and determining processing data of each thread of at least one target task of the access program by the local statistical thread.
The software development kit deployed at the client may include a counter, which may be a special object for the client operating system to issue performance data for certain applications and components, and may also support other applications to acquire and analyze the issued performance data. The operator can determine program bottlenecks and system performance using the data provided by the counters.
In order to acquire the execution progress of the target task, each thread of the target task can continuously call an update progress method in the execution process, so that an operating system or application software of the client can acquire the execution progress of the target task in real time. In the execution process of each thread of the target task, the call of the update progress method can be a call with a certain interval time, or can be a call at the node moment in a mode of establishing a thread node. The counter can count the calling times of the updating progress method which is called in the execution process of each thread of the target task.
The counter may provide the acquired count to the local statistics thread. The local statistical thread may run in client system software or in application software installed on the client, such as the local statistical thread shown in fig. 2. The local statistical thread can determine the processing data of each thread of the target task according to the counting result, and further determine the execution progress of each thread of each target task.
According to the method, the calling times of the updating progress method which is called in the execution process of each thread of the target task can be counted through the counter, the counting result can be provided for the local counting thread of the software development kit, an effective path is provided for determining the processing data of each thread of the target task, meanwhile, the counting result can accurately measure the processing data, and the execution progress of each thread of the target task can be further determined.
S120, determining the execution progress of each thread for executing each target task according to the processing data of each thread of at least one target task.
The client may further determine, according to the processing data of each thread of the target task, an execution progress of each thread that executes each target task. The processing data may be the current latest processing data or the total amount of the currently processed data. Under the condition that the processing data of different time periods are differentiated, the client can determine the execution progress of the thread according to the current latest processing data. The client may also be a statistical processing data amount, and determine the execution progress of the thread according to the processing data amount. The execution progress may be the amount of data processed by each thread of each target task, or may be the ratio of the total amount of data processed by each thread of each target task to the total amount of data of the task. The execution schedule of each thread of each target task may be different due to the different subtasks each thread executes.
In this embodiment, optionally, the determining, according to the processing data of each thread of the at least one target task, an execution progress of each thread for executing each target task includes:
according to the processing data of each thread of the at least one target task, summarizing and determining the total processing data amount and the planned processing data amount of the at least one target task;
and determining the execution progress of all threads for executing each target task according to the total processing data amount and the planned processing data amount of the at least one target task.
In the scheme, the client can summarize the total processing data amount and the planned processing data amount of a target task according to the processing data of each thread of the target task. The total amount of processing data may include an amount of data that has been processed and an amount of data being processed by the target task, and the planned amount of processing data may include an amount of data that has been processed, an amount of data being processed, and an amount of unprocessed data in the task list of the target task.
The client may determine execution progress of all threads executing each target task according to the total amount of processing data of the target task and the amount of scheduled processing data. The client may calculate the ratio of the total amount of processing data and the amount of scheduled processing data for a target task to determine the execution progress of all threads for the target task. For example, assuming that the total amount of processing data is 100 and the amount of scheduled processing data is 500, the execution progress of all threads of the target task may be 20%.
According to the method, the execution progress of all threads of each target task can be quantitatively determined by summarizing the total processing data amount and the planned processing data amount of the target task, and a scientific and effective execution progress determination method is provided.
And S130, reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
It will be appreciated that the central processing service may be a client local system or an application server operating application software deployed on the client. As shown in fig. 2, the client may report the execution progress of each thread of each target task to the central processing service, and the central processing service may collect and count the execution progress of each thread of a certain target task, determine the execution progress of a distributed task, and display the execution progress on the monitoring end. The monitoring end can be a display, a smart phone screen and the like.
On the basis of the foregoing embodiment, optionally, after the central processing service performs summary statistics to determine the distributed task execution progress, the method further includes:
if the execution progress of the target task is not updated within the first preset time, generating alarm information of the target task.
It may be appreciated that the first preset duration may be a preset duration of time according to the actual properties of the access program and the operating environment. If the execution progress of the target task is not updated within the period of time, the situation that a fault possibly exists in the running process of the program is indicated, and at the moment, the client can generate alarm information of the target task to inform an operator that a fault exists in the execution process of a certain target task of the access program. Further, the alarm information may include a cause of the program failure, a current program failure location, a failure occurrence time, and the like.
The scheme can guide operators to locate program faults in time and remind the operators to remove the program faults, so that the normal operation of the service is protected.
In a possible embodiment, optionally, the method further comprises:
if a configuration updating event of the software development kit is detected, the configuration of the software development kit is updated in real time according to the updating content.
In a practical scenario, a configuration update event may occur in a software development kit. If a configuration update event of the software development kit is detected, the client can update the configuration of the software development kit in real time according to the content required to be updated. Through updating the software development kit, the client can increase or decrease the target tasks and can increase or decrease each thread of one or more target tasks, so that flexible configuration of the target tasks is realized. According to the scheme, the targeted configuration updating can be carried out on the software development kit according to the updating content, so that the target task can be flexibly configured, the monitoring service requirements of different scenes can be met, and further more flexible and effective monitoring can be provided for the execution progress of the distributed task.
In another possible embodiment, optionally, the method further comprises:
if the heartbeat packet sent by the local statistical thread of the software development kit is not received within the second preset time, determining that the local statistical thread of the software development kit is not available, and sending alarm information.
On the basis of the above embodiment, the length of time set by the second preset time period may be a fixed value, for example, 10 seconds, 15 seconds, or even 30 seconds. If the client side does not receive the heartbeat packet sent by the local statistical thread of the software development kit in a longer time, the central processing service can determine that the local statistical thread of the software development kit is not available and send out alarm information. The method and the system can monitor whether the local statistical thread of the software development kit is available or not in real time, and the local statistical thread can report the execution progress of each thread of the target task to the central processing service, so that the method and the system further ensure the reliability of monitoring the distributed task execution progress, facilitate operators to discover the working state of the local statistical thread in time and eliminate faults.
According to the technical scheme provided by the embodiment of the application, the processing data of each thread of at least one target task of an access program is obtained through a software development kit deployed in a client; determining the execution progress of each thread for executing each target task according to the processing data of each thread of at least one target task; reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to the monitoring end for display. According to the technical scheme, the execution progress of each thread for executing each target task can be determined by acquiring the processing data of each thread of the target task, a visual progress display and a timely alarm service are provided for the access program, an operator can observe the execution condition of the access program in time, the availability of the service is monitored in time, and the normal operation of the service is protected.
Example two
Fig. 3 is a flowchart of a method for monitoring the execution progress of a distributed task according to a second embodiment of the present invention, which is optimized based on the above-mentioned embodiment.
As shown in fig. 3, the method of this embodiment specifically includes the following steps:
s210, maintaining a counter through a software development kit deployed in the client to count the calling times of the updating progress method called in the execution process of each thread of the target task.
The counter of the software development kit can count the calling times of the updating progress method which is called in the execution process of each thread of the target task. The counter may provide the acquired count to the local statistics thread.
In this scheme, optionally, the update progress method is obtained by calling through a ruable interface or a Callable interface in the execution process of each thread of the target task.
The Ruable interface and the Calla interface may be ways to implement multithreading in Java. The method specified by the Ruable interface is run (), and the method specified by the Callicable interface is call (). The tasks of the Callable interface can return values after execution, while the tasks of the ruable interface cannot return values after execution. In addition, the call () method may throw an exception, while the run () method may not throw an exception. Running the Callable task can take a Future object representing the result of the asynchronous computation. It provides a method of checking whether a calculation is completed, waiting for completion of the calculation, and retrieving the result of the calculation. The status of task execution can be known through the Future object, the execution of the task can be canceled, and the execution result can be obtained.
According to the scheme, the update progress method can be called by calling the Ruable interface or the Calla interface, so that the update progress method can be called more easily, popularization and use of the distributed task execution progress monitoring method are facilitated, and further software development work of operators is facilitated.
And S220, providing the counting result to a local statistical thread of the software development kit, and determining processing data of each thread of at least one target task of the access program by the local statistical thread.
S230, according to the processing data of each thread of the at least one target task, the total processing data amount and the planned processing data amount of the at least one target task are determined in a summarized mode.
S240, determining the execution progress of all threads for executing each target task according to the processing data total amount and the planned processing data amount of the at least one target task.
S250, reporting the execution progress of each thread for executing each target task, collecting statistics by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
According to the technical scheme provided by the embodiment of the application, the counter is maintained through the software development kit deployed in the client so as to count the calling times of the updating progress method called in the execution process of each thread of the target task; providing the counting result to a local counting thread of a software development kit, and determining processing data of each thread of at least one target task of an access program by the local counting thread; according to the processing data of each thread of at least one target task, summarizing and determining the total processing data amount and the planned processing data amount of at least one target task; determining the execution progress of all threads for executing each target task according to the total processing data amount and the planned processing data amount of at least one target task; reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to the monitoring end for display. According to the technical scheme, the execution progress of each thread for executing each target task can be determined by acquiring the processing data of each thread of the target task, a visual progress display and a timely alarm service are provided for the access program, an operator can observe the execution condition of the access program in time, the availability of the service is monitored in time, and the normal operation of the service is protected.
Example III
Fig. 4 is a block diagram of a distributed task execution progress monitoring device according to a third embodiment of the present invention, where the device may execute the distributed task execution progress monitoring method according to any embodiment of the present invention, and the device has functional modules and beneficial effects corresponding to the execution method. The device is configured at a blockchain node, wherein the blockchain node comprises a master node, a slave node and a proxy node; the proxy node is connected with the master node and the slave node; as shown in fig. 4, the apparatus may include:
a processing data obtaining module 310, configured to obtain, by using a software development kit deployed in a client, processing data of each thread of at least one target task of an access program;
an execution progress determining module 320, configured to determine an execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task;
and the execution progress reporting module 330 is configured to report the execution progress of each thread executing each target task, so that the central processing service performs summary statistics, determines the execution progress of the distributed task, and feeds back the execution progress to the monitoring end for display.
In this embodiment, optionally, the processing data acquisition module 310 is specifically configured to:
maintaining a counter through a software development kit deployed in the client to count the calling times of an update progress method called in the execution process of each thread of the target task;
and providing the counting result to a local statistical thread of the software development kit, and determining processing data of each thread of at least one target task of the access program by the local statistical thread.
On the basis of the embodiment, optionally, the update progress method is obtained by calling through a Ruable interface or a Calla interface in the execution process of each thread of the target task.
In a possible embodiment, optionally, the execution progress determining module 320 is specifically configured to:
according to the processing data of each thread of the at least one target task, summarizing and determining the total processing data amount and the planned processing data amount of the at least one target task;
and determining the execution progress of all threads for executing each target task according to the total processing data amount and the planned processing data amount of the at least one target task.
In another possible embodiment, optionally, the execution progress reporting module 330 is further configured to:
if the execution progress of the target task is not updated within the first preset time, generating alarm information of the target task.
In a preferred embodiment, optionally, the apparatus further comprises:
and the configuration updating module is used for updating the configuration of the software development kit in real time according to the updating content if the configuration updating event of the software development kit is detected.
In another preferred embodiment, optionally, the apparatus further comprises:
and the alarm information sending module is used for determining that the local statistical thread of the software development kit is unavailable and sending alarm information if the heartbeat packet sent by the local statistical thread of the software development kit is not received within the second preset time.
The product can execute the monitoring method of the distributed task execution progress provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
A fourth embodiment of the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for monitoring a distributed task execution progress as provided in all embodiments of the present invention:
acquiring processing data of each thread of at least one target task of an access program through a software development kit deployed in a client;
determining the execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task;
reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Example five
The fifth embodiment of the application provides an electronic device. Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application. As shown in fig. 5, the present embodiment provides an electronic device 400, which includes: one or more processors 420; a storage device 410, configured to store one or more programs, where the one or more programs are executed by the one or more processors 420, and cause the one or more processors 420 to implement a method for monitoring a distributed task execution progress provided by an embodiment of the present application, where the method includes:
acquiring processing data of each thread of at least one target task of an access program through a software development kit deployed in a client;
determining the execution progress of each thread for executing each target task according to the processing data of each thread of the at least one target task;
reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
Of course, those skilled in the art will appreciate that the processor 420 further implements the technical solution of the method for monitoring the execution progress of the distributed task provided in any embodiment of the present application.
The electronic device 400 shown in fig. 5 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 5, the electronic device 400 includes a processor 420, a storage device 410, an input device 430, and an output device 440; the number of processors 420 in the electronic device may be one or more, one processor 420 being taken as an example in fig. 5; the processor 420, the storage device 410, the input device 430, and the output device 440 in the electronic device may be connected by a bus or other means, for example by a bus 450 in fig. 5.
The storage device 410 is used as a computer readable storage medium, and can be used to store a software program, a computer executable program, and a module unit, such as program instructions corresponding to a method for monitoring a distributed task execution progress in the embodiment of the present application.
The storage device 410 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, the storage 410 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, storage device 410 may further include memory located remotely from processor 420, which may be connected via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 430 may be used to receive input numeric, character information, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. The output device 440 may include an electronic device such as a display screen, a speaker, etc.
The electronic equipment provided by the embodiment of the application can determine the execution progress of each thread for executing each target task by acquiring the processing data of each thread of the target task, provides a visual progress display and a timely alarm service for an access program, is convenient for operators to observe the execution condition of the access program in time, monitors the availability of the service in time, and protects the navigation for normal operation of the service.
The monitoring device, medium and equipment for the execution progress of the distributed task provided in the above embodiment can execute the monitoring method for the execution progress of the distributed task provided in any embodiment of the present application, and have the corresponding functional modules and beneficial effects of executing the method. Technical details not described in detail in the foregoing embodiments may be referred to a method for monitoring the execution progress of a distributed task provided in any embodiment of the present application.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (8)
1. A method for monitoring the execution progress of a distributed task, the method comprising:
maintaining a counter through a software development kit deployed in the client to count the calling times of an update progress method called in the execution process of each thread of the target task;
providing the counting result to a local statistical thread of a software development kit, and determining processing data of each thread of at least one target task of an access program by the local statistical thread;
according to the processing data of each thread of the at least one target task, summarizing and determining the total processing data amount and the planned processing data amount of the at least one target task;
determining the execution progress of all threads for executing each target task according to the total processing data amount and the planned processing data amount of the at least one target task;
reporting the execution progress of each thread for executing each target task, collecting and counting by the central processing service, determining the distributed task execution progress, and feeding back to a monitoring end for display.
2. The method according to claim 1, wherein the update progress method is invoked by a ruable interface or a Callable interface during execution of each thread of the target task.
3. The method of claim 1, wherein after the aggregate statistics for the central processing service determine the distributed task execution progress, the method further comprises:
if the execution progress of the target task is not updated within the first preset time, generating alarm information of the target task.
4. The method according to claim 1, wherein the method further comprises:
if a configuration updating event of the software development kit is detected, the configuration of the software development kit is updated in real time according to the updating content.
5. The method according to claim 1, wherein the method further comprises:
if the heartbeat packet sent by the local statistical thread of the software development kit is not received within the second preset time, determining that the local statistical thread of the software development kit is not available, and sending alarm information.
6. A device for monitoring progress of distributed task execution, the device comprising:
the processing data acquisition module is used for maintaining a counter through a software development kit deployed in the client so as to count the calling times of an updating progress method called in the execution process of each thread of the target task; providing the counting result to a local statistical thread of a software development kit, and determining processing data of each thread of at least one target task of an access program by the local statistical thread;
the execution progress determining module is used for summarizing and determining the total processing data amount and the planned processing data amount of the at least one target task according to the processing data of each thread of the at least one target task; determining the execution progress of all threads for executing each target task according to the total processing data amount and the planned processing data amount of the at least one target task;
and the execution progress reporting module is used for reporting the execution progress of each thread for executing each target task so as to be summarized and counted by the central processing service, determining the execution progress of the distributed task and feeding back to the monitoring end for display.
7. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a method of monitoring the progress of distributed task execution according to any of claims 1-5.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of monitoring the progress of distributed task execution as claimed in any one of claims 1 to 5 when the computer program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111105334.1A CN113835958B (en) | 2021-09-22 | 2021-09-22 | Distributed task execution progress monitoring method, device, medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111105334.1A CN113835958B (en) | 2021-09-22 | 2021-09-22 | Distributed task execution progress monitoring method, device, medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113835958A CN113835958A (en) | 2021-12-24 |
| CN113835958B true CN113835958B (en) | 2024-02-23 |
Family
ID=78960209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111105334.1A Active CN113835958B (en) | 2021-09-22 | 2021-09-22 | Distributed task execution progress monitoring method, device, medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113835958B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3056970A1 (en) * | 2015-02-13 | 2016-08-17 | Xiaomi Inc. | Method and device for controlling power consumption |
| CN106325992A (en) * | 2016-08-20 | 2017-01-11 | 国云科技股份有限公司 | Method for calculating progress of long tasks in distributed type environment |
| CN107690077A (en) * | 2017-09-05 | 2018-02-13 | 北京京东尚科信息技术有限公司 | The method and apparatus for monitoring net cast interim card |
| CN109032775A (en) * | 2018-06-29 | 2018-12-18 | 武汉斗鱼网络科技有限公司 | A kind of method, terminal device and storage medium obtaining Task Progress |
| CN112286905A (en) * | 2020-10-15 | 2021-01-29 | 北京沃东天骏信息技术有限公司 | Data migration method and device, storage medium and electronic equipment |
| CN112306633A (en) * | 2020-10-27 | 2021-02-02 | 广州锦行网络科技有限公司 | System and method for acquiring different virtualized resources |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9280407B2 (en) * | 2014-04-16 | 2016-03-08 | International Business Machines Corporation | Program development in a distributed server environment |
-
2021
- 2021-09-22 CN CN202111105334.1A patent/CN113835958B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3056970A1 (en) * | 2015-02-13 | 2016-08-17 | Xiaomi Inc. | Method and device for controlling power consumption |
| CN106325992A (en) * | 2016-08-20 | 2017-01-11 | 国云科技股份有限公司 | Method for calculating progress of long tasks in distributed type environment |
| CN107690077A (en) * | 2017-09-05 | 2018-02-13 | 北京京东尚科信息技术有限公司 | The method and apparatus for monitoring net cast interim card |
| CN109032775A (en) * | 2018-06-29 | 2018-12-18 | 武汉斗鱼网络科技有限公司 | A kind of method, terminal device and storage medium obtaining Task Progress |
| CN112286905A (en) * | 2020-10-15 | 2021-01-29 | 北京沃东天骏信息技术有限公司 | Data migration method and device, storage medium and electronic equipment |
| CN112306633A (en) * | 2020-10-27 | 2021-02-02 | 广州锦行网络科技有限公司 | System and method for acquiring different virtualized resources |
Non-Patent Citations (1)
| Title |
|---|
| "如何利用Prometheus监控你的应用（此列子是对于golang sdk进行运用）";Awindow;《https://www.cnblogs.com/lixinsi/p/15065165.html》;第1-6页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113835958A (en) | 2021-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111831420B (en) | Method for task scheduling, related device and computer program product | |
| US9389936B2 (en) | Monitoring the responsiveness of a user interface | |
| JP5474177B2 (en) | Distributed application monitoring | |
| US10116534B2 (en) | Systems and methods for WebSphere MQ performance metrics analysis | |
| CN108595316B (en) | Lifecycle management method, manager, device, and medium for distributed application | |
| US20080155350A1 (en) | Enabling tracing operations in clusters of servers | |
| CN110515808B (en) | Database monitoring method and device, computer equipment and storage medium | |
| WO2016188100A1 (en) | Information system fault scenario information collection method and system | |
| US9015731B2 (en) | Event handling system and method | |
| CN112099935B (en) | Task processing method and device | |
| US20160321906A1 (en) | Alert management within a network based virtual collaborative space | |
| CN114785690B (en) | Monitoring method based on service grid and related equipment | |
| CN111917814B (en) | Data publishing method, data subscribing method, data publishing device, data subscribing system and readable storage medium | |
| CN115629933A (en) | Business system monitoring method, device, equipment and storage medium | |
| CN113495820A (en) | Method and device for collecting and processing abnormal information and abnormal monitoring system | |
| CN113238815B (en) | Interface access control method, device, equipment and storage medium | |
| US10305762B2 (en) | Techniques for determining queue backlogs, active counts, and external system interactions in asynchronous systems | |
| CN110635956A (en) | Equipment management method, platform, system, equipment and storage medium | |
| CN112153146B (en) | Operation notification method and apparatus, storage medium, and electronic apparatus | |
| CN113656102A (en) | Data point burying method and device, storage medium and electronic equipment | |
| CN113835958B (en) | Distributed task execution progress monitoring method, device, medium and equipment | |
| CN112685247A (en) | Alarm suppression method based on Zabbix monitoring system and monitoring system | |
| CN117370053A (en) | Information system service operation-oriented panoramic monitoring method and system | |
| KR20170122874A (en) | Apparatus for managing log of application based on data distribution service | |
| Alekseev et al. | The BigPanDA self-monitoring alarm system for ATLAS |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |