CN109933485B - Scheduling method for algorithm execution and monitoring - Google Patents
Scheduling method for algorithm execution and monitoring Download PDFInfo
- Publication number
- CN109933485B CN109933485B CN201711364615.2A CN201711364615A CN109933485B CN 109933485 B CN109933485 B CN 109933485B CN 201711364615 A CN201711364615 A CN 201711364615A CN 109933485 B CN109933485 B CN 109933485B
- Authority
- CN
- China
- Prior art keywords
- algorithm
- scheduling
- executed
- execution
- scheduled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a scheduling method for algorithm execution and monitoring. The method comprises the following steps: (1) the method divides the algorithm to be scheduled and executed into three categories and provides different categories of algorithm definitions; (2) providing different algorithm scheduling execution frames, packaging algorithms according to different algorithm classifications, providing different scheduling methods for different algorithms, monitoring states in the feedback algorithm execution process, (3) dividing the algorithms to be scheduled and executed according to the definition of the method, and setting algorithm scheduling configuration through a corresponding algorithm scheduling configuration table in a database, (4) performing algorithm scheduling execution and monitoring according to the algorithm setting in the database, and providing a mutual exclusion guarantee mechanism according to different types of algorithms to ensure that the algorithms cannot be repeatedly scheduled and executed under the incomplete condition. The method can simplify algorithm classification, improve algorithm scheduling and executing efficiency and monitor algorithm executing conditions.
Description
Technical Field
The invention relates to the field of computer application technology and method, in particular to a scheduling technical scheme for algorithm execution and monitoring.
Background
Since the new century, scientific development and maturation of information technology have accelerated the development of various data-based analysis methods, and well overcome the problems of difficult modeling and the like commonly existing in a complex process system. However, due to the fact that data analysis algorithms are various and use scenes are different, the number of executed algorithms is still artificially controlled, and an automatic control scheduling method is lacked. And lacks a means for monitoring and recording the overall performance of the algorithm. Especially, the problem that a large number of algorithms are used for scheduling and monitoring in the current big data processing and other scenes is solved, and due to the fact that the data are various and the types and the number of the algorithms are large, the data analysis and the control scheduling of the mining algorithm are carried out manually or semi-automatically, the overall efficiency is affected, and the labor consumption is increased.
Disclosure of Invention
In view of the above technical deficiencies, the present invention provides a scheduling method for algorithm execution and monitoring. The method can provide automatic and customizable algorithm scheduling and monitoring on a complex algorithm platform. By dividing the scheduled algorithms into three categories: the real-time algorithm, the timing algorithm and the equal interval algorithm simplify the algorithm types and improve the algorithm scheduling efficiency. The configuration information of the scheduled algorithm, including the classification identifier, the execution time identifier, whether the execution identifier is executed or not, is stored through the database, and the whole configuration process is simple and clear by means of the characteristics of intuition and easy operation of the database. By packaging different scheduling and monitoring frames for different types of algorithms, the safety of each type of algorithm when being scheduled is ensured, and a mutual exclusion mechanism is added to ensure that each algorithm cannot be repeatedly executed in the scheduling execution process.
The technical scheme adopted by the invention for solving the technical problems is as follows: a scheduling method for algorithm execution and monitoring, comprising the steps of:
(1) classifying the algorithm;
(2) aiming at different algorithms, different frame packaging, scheduling modes and feedback algorithm execution state information are provided and stored in a database and a log;
(3) setting different algorithm type identifications and execution time identifications by changing an algorithm configuration table in a database;
(4) and a mutual exclusion mechanism is provided for different algorithms, and the execution and monitoring of the algorithms are realized.
The algorithm classification includes:
and (3) real-time algorithm: algorithms that are always executed in the background;
timing algorithm: an algorithm that is executed once per day at a fixed time;
the interval algorithm comprises the following steps: the algorithm is executed once every set time, and the interval time is as follows: dividing the whole time.
The algorithm configuration table comprises an algorithm name, an algorithm type identification bit, algorithm execution time setting and whether the algorithm is executed or not.
Step (2), for the timing algorithm:
firstly, acquiring whether state information of an algorithm to be scheduled exists in an algorithm state table, and if so, proving that the algorithm has been scheduled; if the existing status information identification bit is 0, the algorithm fails to be scheduled and executed again; if the existing status information identification bit is not 0, the scheduling does not need to be carried out;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 represents the execution, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and printing failure information to a log if the algorithm fails to execute.
Step (2), for the real-time algorithm:
firstly, acquiring whether state information of an algorithm to be scheduled exists in an algorithm state table, and if so, proving that the algorithm has been scheduled; if the existing status information identification bit is 0, the algorithm fails to be scheduled and executed again; if the existing status information identification bit is not 0, the scheduling does not need to be carried out;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 represents the execution, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and printing failure information to a log if the algorithm fails to execute.
Step (2) for the interval algorithm:
firstly, acquiring the latest state information of an algorithm to be scheduled in an algorithm state table, and if the existing state information identification bit is not 2, namely no algorithm in execution currently exists, the algorithm can be scheduled and executed; if the existing status information identification bit is 2, the scheduling is not needed;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 represents the execution, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and printing failure information to a log if the algorithm fails to execute.
Providing a mutual exclusion mechanism for different kinds of algorithms comprises the steps of:
after the algorithm to be scheduled is obtained, firstly, mutual exclusion judgment is carried out, namely whether the algorithm is already executed or not is judged; if yes, the algorithm is not executed, otherwise, the algorithm is scheduled to be executed; and meanwhile, judging whether the real-time algorithm and the timing algorithm fail, and if so, rescheduling and executing the algorithm.
The invention has the following beneficial effects and advantages:
1. the method of the invention classifies all algorithms into three categories, and simplifies the complexity of scheduling control of the algorithms.
2. The method of the invention provides different scheduling modes aiming at different algorithms, and ensures the scheduling accuracy and safety of each algorithm.
3. The configuration data of the method of the invention is stored in the database, and the configuration data comprises the category identification of the scheduled algorithm, the execution time identification, whether the execution identification is executed or not and the like, thereby being convenient for changing and setting.
4. The method feeds back the state of the scheduled algorithm, including start-stop time, time consumed by the algorithm, algorithm state (failure, success, execution) and other information, and stores the information in a database table and a log.
5. The method of the invention has a mutual exclusion mechanism, and ensures that different algorithms can not be repeatedly scheduled during execution.
6. The method can realize automatic control without manual control.
Drawings
FIG. 1 is an algorithm scheduling overall architecture diagram;
FIG. 2 is a scheduling method architecture diagram;
FIG. 3 is a timing algorithm block flow diagram;
FIG. 4 is a real-time algorithm block flow diagram;
FIG. 5 is a flow chart of an equal interval algorithm module.
Detailed Description
The present invention will be described in further detail with reference to examples.
The invention relates to a scheduling method for algorithm execution and monitoring. The method comprises the following steps: (1) the method divides algorithms to be scheduled and executed into three categories, namely a real-time algorithm, a timing algorithm and an equal interval algorithm, and provides different category algorithm definitions; (2) providing different algorithm scheduling execution frames, packaging algorithms according to different algorithm classifications, providing different scheduling methods for different algorithms, monitoring states including success, failure and execution medium states in the execution process of the feedback algorithm, storing various information including start-stop time, algorithm time consumption, algorithm execution states and the like of the algorithm execution conditions in a database table, and storing failure information in a log; (3) according to the definition of the method, the algorithm to be scheduled and executed is divided, algorithm scheduling configuration is set through a corresponding algorithm scheduling configuration table in a database, and the algorithm scheduling configuration comprises algorithm execution time identification, whether the identification is executed or not, and identification bit setting of which type of algorithm belongs; (4) the scheduling method carries out algorithm scheduling execution and monitoring according to algorithm settings in the database, provides a mutual exclusion guarantee mechanism according to different types of algorithms, and ensures that the algorithms cannot be repeatedly scheduled and executed under the incomplete condition. The method can simplify algorithm classification, improve algorithm scheduling and executing efficiency and monitor algorithm executing conditions.
(1) The scheduling method divides data analysis and mining algorithms which need to be scheduled, executed and monitored into three categories, including: real-time algorithms, timing algorithms, and equal interval algorithms.
And (3) real-time algorithm: algorithms that are executed all the time in the background.
Timing algorithm: an algorithm that is executed once a day at a fixed time.
The equal interval algorithm: the algorithm is executed once at intervals of a fixed duration, which may be minute granularity and hour granularity, and does not support a mixed granularity of 1 hour and 15 minutes.
(2) A simple flowchart of the scheduling method is shown in fig. 1.
According to the definition of algorithm classification, firstly, the classification of the algorithm to be scheduled is determined, and relevant classification information, algorithm information and the like are stored in an algorithm configuration table (table 1) in a database.
The algorithm configuration table structure is as follows:
TABLE 1
Description of fields:
algorithm: algorithm storage location and name;
timeingtask: a timing algorithm flag bit, wherein 1 represents that the algorithm is a timing algorithm, and 0 represents that the algorithm is not a timing algorithm;
realtimeask: a real-time algorithm flag bit, wherein 1 represents that the algorithm is a real-time algorithm, and 0 represents not;
intervaltask: an equally spaced time algorithm, 1 representing that the algorithm is an equally spaced algorithm and 0 representing not;
(Note: timing, real-time, and equally spaced algorithms are not repeatable, i.e., one algorithm can only be one of them.)
time: algorithm execution time, representing different for three different algorithms
a. Timing algorithm: 09:00:00 represents 9 o 'clock 0 o' clock per day to execute the algorithm;
b. and (3) real-time algorithm: 00:00:00 is the default write-down, and the real-time algorithm is executed from the start of the whole control system;
c. the equal interval algorithm: 00:30:00 represents execution every 30 minutes, and modification can be achieved by modifying the corresponding field, for example: 01:00:00, representing once per hour, requiring hours, minutes field allows only one to have a value, no setting can occur that is performed every 1 hour for 15 minutes, and the equal interval algorithm does not support settings to the second level.
d. runorn: whether the algorithm needs to execute the flag bit or not, wherein 1 represents that the algorithm needs to be executed, and 0 represents that the algorithm does not need to be executed, and the algorithm module which runs on the day is set according to the requirement;
according to the algorithm configuration table, the algorithms to be executed can be configured in a customized manner, the execution conditions of all the algorithms can be manually controlled, and the diversity, the practicability and the usability of the scheduling method are improved. And the algorithm configuration condition is stored through the database, so that the access is convenient and the modification is easy.
(3) After the configuration is completed, the scheduling method scans the algorithm configuration table every minute, obtains the algorithm which is in accordance with the current time for scheduling and executing, and the flow chart of the scheduling method is shown in fig. 2.
After the scheduling method obtains the algorithm to be scheduled which meets the conditions, the mutual exclusion judgment is firstly carried out, whether the algorithm is already in execution is judged, algorithm execution protection is carried out, and the algorithm confusion caused by repeated scheduling algorithm is prevented. For the executable algorithms, different scheduling and monitoring methods are adopted according to different algorithm classifications, which are described below.
(4) Before introducing the scheduling and monitoring methods of the three types of algorithms, an algorithm state table, namely a data table for storing algorithm execution conditions, is introduced.
The algorithm state table structure is as in table 2:
TABLE 2
Description of fields:
and (3) date: date of algorithm execution
algorithm: description of the equivalent Algorithm configuration section
starting time: algorithm execution start time, accurate to seconds
finish time: the algorithm execution is finished, and when the algorithm execution is not finished, default filling 2020-01-0100: 00 is performed
runtime: the algorithm takes time in seconds
status: the algorithm executes the status flag, 0 for failure, 1 for success, and 2 for in-execution.
The scheduling method can feed back the algorithm state to be stored in the table in the process of scheduling execution and monitoring the algorithm, algorithm maintenance and monitoring are carried out, algorithm general information is provided for users, and for a failure algorithm, failure information can be stored in a log, so that positioning and analysis are facilitated.
(5) The corresponding scheduling and monitoring methods are introduced in three categories.
A. The timing algorithm flow chart is shown in fig. 3.
Firstly, whether the state information of the algorithm to be scheduled exists in an algorithm state table or not is obtained, if the state information exists, the algorithm is proved to be scheduled, if the existing state information identification bit is 0 (namely failure), the algorithm needs to be scheduled and executed again, and if the existing state information identification bit is not 0, the scheduling does not need to be carried out because the timing algorithm only needs to be executed once a day. For the algorithm needing scheduling, the status information identification bit is written/updated to be 2 (in execution), the algorithm is executed, the algorithm execution condition is finally judged, the status identification bit in the algorithm status table is updated to be a corresponding status (failure is 0, success is 1), and if the algorithm execution fails, failure information needs to be printed to a log.
B. The real-time algorithm flow chart is as shown in FIG. 4:
the real-time algorithm scheduling and monitoring process is similar to the timing algorithm. The difference is that the real-time algorithm is executed in the background all the time, no success state exists, and only the failure state needs to be judged.
C. The equal interval algorithm flow chart is as shown in FIG. 5:
and repeating the mutual exclusion judgment once by acquiring the state information, namely determining the algorithm which is not in execution. The safety and the stability of the algorithm are ensured. Since the equidistant algorithm is executed at intervals, it is not necessary to determine whether there is failure status information.
Through the whole scheduling method flow and the introduction of the algorithm scheduling and monitoring method flows aiming at different categories, the scheduling method simplifies algorithm classification, improves algorithm scheduling and executing efficiency and realizes algorithm executing condition monitoring.
Claims (2)
1. A scheduling method for algorithm execution and monitoring, comprising the steps of:
(1) classifying the algorithm;
the algorithm comprises the following steps:
and (3) real-time algorithm: algorithms that are always executed in the background;
timing algorithm: algorithms that need only be executed once a day;
the equal interval algorithm: the algorithm is executed once every set time, and the interval time is as follows: dividing and timing;
(2) aiming at different algorithms, different frame packaging, scheduling modes and feedback algorithm execution state information are provided and stored in a database and a log;
for the timing algorithm:
firstly, acquiring whether state information of an algorithm to be scheduled exists in an algorithm state table, and if so, proving that the algorithm has been scheduled; if the existing status information identification bit is 0, the algorithm fails to be scheduled and executed again; if the existing status information identification bit is not 0, the scheduling does not need to be carried out;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 to indicate that the algorithm is executed, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and if the algorithm is failed to execute, printing failure information to a log;
for a real-time algorithm:
firstly, acquiring whether state information of an algorithm to be scheduled exists in an algorithm state table, and if so, proving that the algorithm has been scheduled; if the existing status information identification bit is 0, the algorithm fails to be scheduled and executed again; if the existing status information identification bit is not 0, the scheduling does not need to be carried out;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 to indicate that the algorithm is executed, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and if the algorithm is failed to execute, printing failure information to a log;
for the equal interval algorithm:
firstly, acquiring the latest state information of an algorithm to be scheduled in an algorithm state table, and if the existing state information identification bit is not 2, namely no algorithm in execution currently exists, the algorithm can be scheduled and executed; if the existing status information identification bit is 2, the scheduling is not needed;
for the algorithm needing scheduling, writing or updating the state information identification bit to be 2 to indicate that the algorithm is executed, executing the algorithm, finally judging the execution condition of the algorithm, updating the state identification bit in the algorithm state table to be a corresponding state, and if the algorithm is failed to execute, printing failure information to a log;
(3) setting different algorithm type identifications and execution time identifications by changing an algorithm configuration table in a database;
(4) providing a mutual exclusion mechanism for different algorithms to realize the execution and monitoring of the algorithms;
the providing a mutual exclusion mechanism for heterogeneous algorithms comprises the steps of:
after the algorithm to be scheduled is obtained, firstly, mutual exclusion judgment is carried out, namely whether the algorithm is already executed or not is judged; if yes, the algorithm is not executed, otherwise, the algorithm is scheduled to be executed; and meanwhile, judging whether the real-time algorithm and the timing algorithm fail, and if so, rescheduling and executing the algorithm.
2. The scheduling method for algorithm execution and monitoring as claimed in claim 1, wherein the algorithm configuration table comprises an algorithm name, an algorithm class flag, an algorithm execution time setting, and whether the algorithm is executed or not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711364615.2A CN109933485B (en) | 2017-12-18 | 2017-12-18 | Scheduling method for algorithm execution and monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711364615.2A CN109933485B (en) | 2017-12-18 | 2017-12-18 | Scheduling method for algorithm execution and monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109933485A CN109933485A (en) | 2019-06-25 |
| CN109933485B true CN109933485B (en) | 2021-06-15 |
Family
ID=66982595
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711364615.2A Active CN109933485B (en) | 2017-12-18 | 2017-12-18 | Scheduling method for algorithm execution and monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109933485B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255672B (en) * | 2020-03-26 | 2023-06-02 | 深圳怡化电脑股份有限公司 | Paper money identification method and device and terminal equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6243736B1 (en) * | 1998-12-17 | 2001-06-05 | Agere Systems Guardian Corp. | Context controller having status-based background functional task resource allocation capability and processor employing the same |
| CN101060463A (en) * | 2007-06-04 | 2007-10-24 | 杭州华三通信技术有限公司 | Asynchronous transmission mode dispatching method and device |
| CN105468500A (en) * | 2015-11-16 | 2016-04-06 | 中国建设银行股份有限公司 | Timing task monitoring method and device |
| CN105653365A (en) * | 2016-02-22 | 2016-06-08 | 青岛海尔智能家电科技有限公司 | Task processing method and device |
-
2017
- 2017-12-18 CN CN201711364615.2A patent/CN109933485B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6243736B1 (en) * | 1998-12-17 | 2001-06-05 | Agere Systems Guardian Corp. | Context controller having status-based background functional task resource allocation capability and processor employing the same |
| CN101060463A (en) * | 2007-06-04 | 2007-10-24 | 杭州华三通信技术有限公司 | Asynchronous transmission mode dispatching method and device |
| CN105468500A (en) * | 2015-11-16 | 2016-04-06 | 中国建设银行股份有限公司 | Timing task monitoring method and device |
| CN105653365A (en) * | 2016-02-22 | 2016-06-08 | 青岛海尔智能家电科技有限公司 | Task processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109933485A (en) | 2019-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20170109206A1 (en) | Task scheduling method, apparatus, and device | |
| CN110647387B (en) | Education cloud big data task scheduling method and system | |
| US10997637B2 (en) | Method and system for determining quality of application based on user behaviors of application management | |
| CN107577579B (en) | Log recording method and device | |
| CN108681801B (en) | Method and system for monitoring operation index in production process | |
| CN110909006A (en) | Data synchronization method and device, computer equipment and storage medium | |
| CN105719112A (en) | Determination method and device for distribution abnormal state, and server | |
| CN107241205A (en) | abnormality monitoring method and device | |
| CN110187980A (en) | Message queue monitoring method, device, electronic equipment and readable storage medium storing program for executing | |
| CN109634589A (en) | A kind of processing method for the automatic Realtime Alerts task of sewage treatment process | |
| CN106575254B (en) | Log analysis device, Log Analysis System, log analysis method and storage medium | |
| CN109933485B (en) | Scheduling method for algorithm execution and monitoring | |
| CN103903312A (en) | Method and device for executing automatic inspection | |
| CN111324426A (en) | ORACLE database task job management scheduling system and method | |
| AU2023200227A1 (en) | A model management system | |
| CN113031991B (en) | Remote self-adaptive upgrading method and device for embedded system | |
| CN114862368A (en) | Workflow instance operation method and device, electronic equipment and storage medium | |
| CN111190817B (en) | Method and device for processing software defects | |
| CN111737233A (en) | Data monitoring method and device | |
| CN112559641B (en) | Pull chain table processing method and device, readable storage medium and electronic equipment | |
| CN106096123A (en) | A kind of VMM for passive label chip verifies system and method | |
| GB2507874A (en) | Comparing man-hours for manual and automated testing | |
| CN112836869A (en) | KPI prediction method, KPI prediction device and storage medium | |
| JP2023537956A (en) | Anticipatory pre-execution of data queries | |
| CN112965793A (en) | Data warehouse task scheduling method and system oriented to identification analysis data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |