CN117873768A - Batch processing task processing method, device, equipment and readable storage medium - Google Patents

Batch processing task processing method, device, equipment and readable storage medium Download PDF

Info

Publication number
CN117873768A
CN117873768A CN202410017247.8A CN202410017247A CN117873768A CN 117873768 A CN117873768 A CN 117873768A CN 202410017247 A CN202410017247 A CN 202410017247A CN 117873768 A CN117873768 A CN 117873768A
Authority
CN
China
Prior art keywords
task
abnormal
batch
processing
tasks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410017247.8A
Other languages
Chinese (zh)
Inventor
陈志超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Property and Casualty Insurance Company of China Ltd
Original Assignee
Ping An Property and Casualty Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Property and Casualty Insurance Company of China Ltd filed Critical Ping An Property and Casualty Insurance Company of China Ltd
Priority to CN202410017247.8A priority Critical patent/CN117873768A/en
Publication of CN117873768A publication Critical patent/CN117873768A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/079—Root cause analysis, i.e. error or fault diagnosis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9024—Graphs; Linked lists

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application belongs to the technical field of data processing, and provides a method, a device, equipment and a readable storage medium for processing batch processing tasks, wherein the method can be applied to the financial field and comprises the following steps: acquiring a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow; under the condition that abnormal tasks occur in the process of executing each batch of processing task flows, tracing back front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph; and asynchronously intervening in the preposed abnormal task so as to finish orderly execution of the preposed abnormal task and the abnormal task. The method and the device greatly improve the convenience and efficiency of solving the problem of batch processing abnormality, and reduce the operation and maintenance cost of a financial system and a financial business in the financial field.

Description

Batch processing task processing method, device, equipment and readable storage medium
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for processing a batch processing task.
Background
In the financial field, along with the continuous expansion of financial services, the service volumes correspondingly carried by a plurality of service systems serving the financial services are also continuously increasing, for example, the insurance system carries a huge amount of insurance transaction services, and the banking system carries a huge amount of financial transaction services. In the case of a large number of service requests, in order to improve processing efficiency, each service system performs batch processing on the service requests in a batch processing manner. For a single business system, each task of batch processing can be processed simultaneously, and some tasks can be processed after completing processing of some tasks. Meanwhile, because of the close correlation between the business of any business system and the business of other business systems, the business of any business system also has to be processed after the completion of certain task processing of other business systems.
At present, when an abnormality occurs in a certain batch, an operation and maintenance person needs to spend a great deal of time and effort to comb the tasks depending on the upstream and downstream of the abnormal batch, locate the tasks causing the abnormality in the batch, and then inform the operation and maintenance person of the corresponding service system to solve the tasks. The method for locating the abnormality by manual investigation is inconvenient, particularly inconvenient if the abnormality occurs at night, low in efficiency, extremely wasteful to manpower and increases operation and maintenance cost.
Disclosure of Invention
The main purpose of the application is to provide a method, a device, equipment and a readable storage medium for processing batch processing tasks, which aim to solve the technical problems of low convenience and efficiency and high operation and maintenance cost of the existing manual mode for solving batch processing abnormality.
In a first aspect, the present application provides a method for processing a batch task, where the method includes:
acquiring a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow;
under the condition that abnormal tasks occur in the process of executing each batch processing task flow, tracing front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph;
and asynchronously intervening the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
In a second aspect, the present application further provides an apparatus for processing a batch task, where the apparatus includes:
the acquisition module is used for acquiring a first dependency relationship topological graph among the batch task flows and a second dependency relationship topological graph among the single tasks in each batch task flow;
the tracing module is used for tracing the front abnormal task which causes the abnormal task according to the first dependency relationship topological graph and the second dependency relationship topological graph under the condition that the abnormal task occurs in the process of executing each batch processing task flow;
and the intervention module is used for asynchronously intervening the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program when executed by the processor implements the steps of the method for processing batch processing tasks as described above.
In a fourth aspect, the present application further provides a computer readable storage medium, where a computer program is stored on the computer readable storage medium, where the computer program, when executed by a processor, implements a method for processing a batch task as described above.
The application discloses a processing method, a device, equipment and a readable storage medium for batch processing tasks, wherein the processing method for batch processing tasks obtains a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow; under the condition that abnormal tasks occur in the process of executing each batch of processing task flows, tracing back front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph; and asynchronously intervening in the preposed abnormal task so as to finish orderly execution of the preposed abnormal task and the abnormal task. Thus, the preposed abnormal tasks causing batch processing abnormality can be rapidly and accurately positioned through the first dependency topological graph and the second dependency topological graph, the investigation speed of batch processing abnormality is greatly improved, abnormal tasks in batch processing can be rapidly recovered and processed through asynchronous intervention of the preposed abnormal tasks, manpower is saved, convenience and efficiency for solving the batch processing abnormality problem are greatly improved, and operation and maintenance costs of financial systems and financial services in the financial field are reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an embodiment of a method for processing a batch task according to the present application;
FIG. 2 is an exemplary diagram of a second dependency topology among individual tasks in a batch task flow according to an embodiment of a method for processing batch tasks of the present application;
FIG. 3 is a schematic block diagram of an apparatus for processing batch processing tasks according to an embodiment of the present application;
fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present application.
The realization, functional characteristics and advantages of the present application will be further described with reference to the embodiments, referring to the attached drawings.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.
It is to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and effect. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
The embodiment of the application provides a method, a device, equipment and a readable storage medium for processing batch processing tasks. According to the processing method of the batch processing task, the preposed abnormal task causing batch processing abnormality can be rapidly and accurately located through the first dependency relationship topological graph and the second dependency relationship topological graph, the investigation speed of the batch processing abnormality is greatly improved, the preposed abnormal task is asynchronously interfered, the abnormal task in the batch processing can be rapidly restored to be processed, manpower is saved, convenience and efficiency for solving the batch processing abnormality problem are greatly improved, and operation and maintenance costs of financial systems and financial services in the financial field are reduced.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic diagram of a processing method of a batch processing task according to an embodiment of the present application, where the processing method of the batch processing task is mainly applied to a processing device of the batch processing task, and the processing device of the batch processing task may be a terminal device with a data processing function, such as a server.
The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CoNteNt Delivery Network, CDN), and basic cloud computing services such as big data and data analysis platforms.
As shown in fig. 1, the processing method of the batch processing task includes steps S101 to S103.
Step S101, a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow are obtained.
The type of batch processing is, for example, financial transaction batch settlement, financial statement batch processing, policy data batch synchronization, and the like.
First, a first dependency relationship topological graph among all batch processing task flows is obtained, wherein the first dependency relationship topological graph comprises blood relationship (also called as association relationship) existing among all batch processing task flows, and belongs to dependence among business systems.
For example, taking a banking system as an example, each of the branch business systems correspondingly executes one batch task flow, that is, each business system and each batch task flow are in a one-to-one correspondence, for example, the blood-margin relationship between the batch task flow a corresponding to the first branch business system and the batch task flow B corresponding to the second branch business system is that the starting condition of the task B1 in the batch task flow B is that the task A2 in the batch task flow a is completed, and the associated flow relationship flow between the task of the batch task flow a and the task of the batch task flow B can be created according to the blood-margin relationship between the batch task flows a and B, so as to represent the blood-margin relationship existing between the batch task flows, which indicates that failure of any one task in one batch task flow affects the completion progress of the other batch task flow.
And a second dependency relationship topological graph among the single tasks in each batch task flow is also obtained, wherein the second dependency relationship topological graph comprises blood-edge relationships (also called as circulation relationships) existing among all the single tasks in each batch task flow, and belongs to the dependence in the business system.
For example, in a same business system, the execution of an A2 task must depend on the completion of an A1 task, the execution of an A3 task must depend on the completion of an A2 task, if the execution of an A1 task fails, the A2 task and the A3 task cannot be executed, if the execution of an A2 task fails, the A3 task cannot be executed, and the A1, the A2 task and the A3 task can be placed in the same batch task flow a according to the blood relationship between them, that is, a task flow relationship flow of the batch task flow a is created, which is used for indicating the blood relationship existing between the single tasks, so that any one of the task failures affects the completion progress of the whole batch task flow.
Step S102, under the condition that abnormal tasks occur in the process of executing each batch of processing task flows, the front abnormal tasks causing the abnormal tasks are traced back according to the first dependency relationship topological graph and the second dependency relationship topological graph.
Therefore, under the condition that the task is abnormal in the process of executing each batch processing task flow, the front abnormal task causing the abnormal task can be traced quickly and accurately according to the first dependency relationship topological graph among the batch processing task flows and the second dependency relationship topological graph among the single tasks in each batch processing task flow, so that the batch processing abnormal investigation speed is greatly improved.
Step S103, asynchronously intervening in the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
After the preposed abnormal task causing the abnormal task is traced back, the preposed abnormal task is asynchronously interfered, so that the preposed abnormal task is successfully executed again, and the abnormal task can be successfully executed again, and therefore, the abnormal task in batch processing can be quickly restored.
The method for processing batch processing tasks provided in the above embodiment obtains a first dependency relationship topological graph between each batch processing task stream and a second dependency relationship topological graph between single tasks in each batch processing task stream; under the condition that abnormal tasks occur in the process of executing each batch of processing task flows, tracing back front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph; and asynchronously intervening in the preposed abnormal task so as to finish orderly execution of the preposed abnormal task and the abnormal task. Thus, the preposed abnormal tasks causing batch processing abnormality can be rapidly and accurately positioned through the first dependency topological graph and the second dependency topological graph, the investigation speed of batch processing abnormality is greatly improved, abnormal tasks in batch processing can be rapidly recovered and processed through asynchronous intervention of the preposed abnormal tasks, manpower is saved, convenience and efficiency for solving the batch processing abnormality problem are greatly improved, and operation and maintenance costs of financial systems and financial services in the financial field are reduced.
In some embodiments, step S101 may be to generate a first dependency topology map in response to first relationship information configured by a user on a configuration page for dependency relationships between batch task flows; and generating a second dependency relationship topological graph according to second relationship information configured by the user aiming at the dependency relationship among the single tasks in each batch task flow in the configuration page.
Users such as operation staff and maintenance staff can configure first relation information among various batch task flows and second relation information among single tasks in each batch task flow on a configuration page of processing equipment of batch tasks.
The first relationship information includes names, batch types, developers, operation and maintenance personnel, scheduling units (i.e. the service systems to which the batch task flows belong), processing states, exception information logs, blood-edge relationships among the batch task flows, and the like. The second relationship information includes names, execution states, exception information logs corresponding to the individual tasks in each batch, blood relationship among the individual tasks, and the like. The first relation information and the second relation information can be stored in the appointed table of the preset database, the first dependency relation table and the second dependency relation table are obtained, and users such as operation and maintenance personnel can conveniently maintain and update in time.
And generating a first dependency relationship topological graph according to the first relationship information configured by the user on the configuration page aiming at the dependency relationship among the batch task flows. Specifically, a blank topological graph can be newly established, a plurality of nodes are added for the blank topological graph according to the first relation information, each node corresponds to a batch task flow, and the link relation among the plurality of nodes is adjusted; different colors can be adopted to represent the processing state of the batch processing task flow corresponding to each node, for example, red represents an abnormal processing state, green represents a normal processing state and blue represents a to-be-processed state; and identifying the name, the scheduling unit and the like of the corresponding batch processing task flow in each node, thereby obtaining a first dependency topology graph. Therefore, users such as operation and maintenance personnel can intuitively see the association relation flow among the batch task flows and the specific information of each batch task flow in the first dependency relation topological graph.
And generating a two-dependency topological graph according to second relation information configured by the user aiming at the dependency relation among the single tasks in each batch task flow in the configuration page. Specifically, for a single batch task flow, a blank topological graph can be newly established, a plurality of nodes are added for the blank topological graph according to second relation information, each node corresponds to one task in the same batch task flow, and the circulation relation among the plurality of nodes is adjusted; the execution state of the single task corresponding to each node may be represented by different colors, for example, red represents an abnormal execution state, green represents a normal execution state, and blue represents a to-be-executed state, or may be represented by a specific symbol, for example, v represents a normal execution state, x represents an abnormal execution state, and Δ represents a to-be-executed state; a name or the like corresponding to the single task is identified in each node, thereby obtaining a second dependency topology. Thus, as shown in fig. 2, users such as operation staff and maintenance staff can also intuitively see the flow relation flow among the tasks in the same batch of processing task flows in the second dependency relation topological graph.
Therefore, based on the relation information configured by the user such as operation and maintenance personnel on the configuration page, a corresponding dependency relationship topological graph is generated, and a reliable basis is provided for the follow-up tracing of the front-end abnormal task causing batch processing abnormality.
In some embodiments, according to the first dependency relationship topological graph and the second dependency relationship topological graph, tracing the front-end abnormal task causing the abnormal task may be tracing the front-end abnormal task causing the abnormal task from the batch task flow corresponding to the abnormal task according to the second dependency relationship topological graph; and under the condition that the front-end abnormal task is not traced from the batch processing task flow corresponding to the abnormal task, tracing the front-end abnormal task from the associated batch processing task flow of the batch processing task flow corresponding to the abnormal task according to the first dependency relationship topological graph.
Specifically, firstly tracing the front abnormal task causing the abnormal task from the batch processing task stream where the abnormal task is located according to the second dependency relationship topological graph, if all the front abnormal tasks before the abnormal task are successfully executed in the batch processing task stream where the abnormal task is located, the front abnormal task causing the abnormal task cannot be traced from the batch processing task stream where the abnormal task is located, and in this case, the front abnormal task causing the abnormal task is traced from the associated batch processing task stream of the batch processing task stream corresponding to the abnormal task according to the first dependency relationship topological graph.
In some embodiments, according to the second dependency topology graph, a front-end abnormal task that causes an abnormal task is traced from a batch task flow corresponding to the abnormal task, which may be that whether the front-end task of the abnormal task is abnormal is judged according to the second dependency topology graph; and under the condition that the previous front-end task is not abnormal, judging whether the two front-end tasks of the abnormal task are abnormal or not until all the front-end tasks of the abnormal task are not abnormal, and determining that the front-end abnormal task is not traced from the batch processing task flow corresponding to the abnormal task.
Specifically, according to the second dependency topology graph, whether the front-end task of the abnormal task is abnormal or not is judged, if the front-end task is not abnormal, whether the front-end tasks of the abnormal task are abnormal or not is judged, if the front-end tasks are not abnormal, whether the front-end tasks of the abnormal task are abnormal or not is judged, and the like until all the front-end tasks of the abnormal task are not abnormal is judged, and it can be determined that the front-end abnormal task causing the abnormal task is not traced back from the batch processing task flow corresponding to the abnormal task.
In some embodiments, according to the first dependency topology graph, whether the previous task of the abnormal task is abnormal or not is judged, or may be that according to the first dependency topology graph, the previous task of the abnormal task is queried, and the execution state of the previous task is obtained; and determining whether the previous task is abnormal or not according to the execution state of the previous task.
Specifically, according to the first dependency topology, a previous task of the abnormal task is queried first, and then according to the color of a node corresponding to the previous task in the first dependency topology, the execution state of the previous task is determined, for example, green, so that the previous task is indicated that no abnormality occurs.
In some embodiments, according to the first dependency relationship topological graph, a front-end abnormal task is traced from an associated batch task flow of a batch task flow corresponding to the abnormal task, which may be that according to the first dependency relationship topological graph, an associated batch task flow with a dependency relationship with the batch task flow corresponding to the abnormal task is queried; and tracing the front-end abnormal task from the associated batch processing task flow.
Specifically, according to a first dependency relationship topological graph, an associated batch processing task flow with a dependency relationship of a batch processing task flow corresponding to an abnormal task is queried, and then a front abnormal task causing the abnormal task is traced from the associated batch processing task flow.
Therefore, based on the second dependency topological graph and the first dependency topological graph, the preposed abnormal tasks causing the abnormal tasks are orderly eliminated, and the rationality and the rigor of batch abnormal investigation are improved.
In some embodiments, step S103 may be to obtain an identifier of the pre-exception task and an exception information log; generating alarm information according to the identification and the abnormal information log, and sending the alarm information to a service system corresponding to the front-end abnormal task so that the service system can intervene in the front-end abnormal task, and re-executing the abnormal task until the execution is completed under the condition that the front-end abnormal task is completed.
Specifically, the name of the pre-abnormal task is firstly obtained as an identification, an information log of the pre-abnormal task is also obtained, alarm information is generated according to the identification of the pre-abnormal task and the abnormal information log, and the alarm information is sent to a service system corresponding to the pre-abnormal task according to a scheduling unit of the pre-abnormal task, so that the service system intervenes the pre-abnormal task, the pre-abnormal task is completed, and the abnormal task is re-executed until the pre-abnormal task is completed. For example, the cause of the abnormality is extracted from the abnormality information log, so as to generate an alarm message, such as "cause task A1 to task abnormality due to XX cause, and cause task B2 to task abnormality, please process task A1 in time, so as to ensure normal execution of task B2", and send the alarm message to the service system corresponding to task A1.
Therefore, the related party service system with batch processing abnormality can be timely notified, so that the related party service system can process the preposed abnormal task as soon as possible, and the efficiency of solving the problem of batch processing abnormality is greatly improved.
For better understanding of the above embodiments, an example application scenario is as follows:
for example, under the scene of batch settlement of financial transactions among various branches of banking institutions in the financial field, a first dependency relationship topological graph among batch processing task flows corresponding to the various branches and a second dependency relationship topological graph among single tasks in the batch processing task flows corresponding to each branch are firstly obtained; when the B1 task abnormality occurs in the process of executing the batch processing task flow B of the branch business system B, firstly tracing the front abnormal task causing the B1 task abnormality from the batch processing task flow B according to the second dependency relationship topological graph; under the condition that a front-end abnormal task causing B1 task abnormality cannot be traced from the batch task flow B, a batch task flow A corresponding to a branch business system a associated with the batch task flow B is found according to a first dependency topological graph, so that a front-end abnormal task causing B1 task abnormality is traced from the batch task flow A to an A2 task; and finally, asynchronously intervening in the A2 task, so that the score service system a processes the A2 task in time, and then re-executing the B1 task until the execution is completed under the condition that the A2 task is completed. Therefore, the A2 task causing the abnormality of the B1 task can be rapidly and accurately positioned through the first dependency topological graph and the second dependency topological graph, the investigation speed of batch processing abnormality is greatly improved, the A2 task is asynchronously interfered, the B1 task can be rapidly restored to be processed, the manpower is saved, the convenience and the efficiency for solving the problem of batch processing abnormality are greatly improved, and the operation and maintenance cost of financial systems and financial services in the financial field is reduced.
Referring to fig. 3, fig. 3 is a schematic block diagram of a processing apparatus for generating a batch task by using a regular expression of contract terms according to an embodiment of the present application.
As shown in fig. 3, the apparatus 300 includes: an acquisition module 301, a traceback module 302 and an intervention module 303.
The obtaining module 301 is configured to obtain a first dependency topology map between each batch task flow and a second dependency topology map between individual tasks in each batch task flow;
the tracing module 302 is configured to trace back, when an abnormal task occurs in the process of executing each batch task flow, a front abnormal task that causes the abnormal task according to the first dependency relationship topological graph and the second dependency relationship topological graph;
and the intervention module 303 is configured to asynchronously intervene in the pre-abnormal task, so that the pre-abnormal task and the abnormal task are orderly executed.
It should be noted that, for convenience and brevity of description, specific working processes of the above-described apparatus and each module and unit may refer to corresponding processes in the foregoing embodiment of the batch processing task processing method, which are not described herein again.
The apparatus provided by the above embodiments may be implemented in the form of a computer program which may be run on a computer device as shown in fig. 4.
Referring to fig. 4, fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device may be a personal computer (persoNal computer, PC), a server, or the like having a data processing function.
As shown in fig. 4, the computer device includes a processor, a memory, and a network interface connected by a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any of a number of batch processing methods.
The processor is used to provide computing and control capabilities to support the operation of the entire computer device.
The internal memory provides an environment for the execution of a computer program in a non-volatile storage medium that, when executed by a processor, causes the processor to perform any of a number of processing methods for batch processing tasks.
The network interface is used for network communication such as transmitting assigned tasks and the like. Those skilled in the art will appreciate that the structures shown in FIG. 4 are block diagrams only and do not constitute a limitation of the computer device on which the present aspects apply, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.
It should be appreciated that the processor may be a central processing UNit (CeNtral ProcessiNg UNit, CPU), but may also be other general purpose processors, digital signal processors (Digital SigNal Processor, DSP), application specific integrated circuits (ApplicatioN Specific INtegrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein in one embodiment the processor is configured to run a computer program stored in the memory to implement the steps of:
acquiring a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow;
under the condition that abnormal tasks occur in the process of executing each batch processing task flow, tracing front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph;
and asynchronously intervening the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
In some embodiments, when the processor implements the pre-exception task that causes the exception task according to the first dependency topology map and the second dependency topology map, the processor is configured to implement:
according to the second dependency relationship topological graph, tracing the front abnormal task from the batch processing task flow corresponding to the abnormal task;
and under the condition that the front abnormal task is not traced from the batch processing task flow corresponding to the abnormal task, tracing the front abnormal task from the associated batch processing task flow of the batch processing task flow corresponding to the abnormal task according to the first dependency relationship topological graph.
In some embodiments, when the processor implements the tracing back the pre-exception task from the batch task flow corresponding to the exception task according to the second dependency topology map, the processor is configured to implement:
judging whether the front-end task of the abnormal task is abnormal or not according to the two-dependency topological graph;
and under the condition that the previous preposed task is not abnormal, judging whether the previous two preposed tasks of the abnormal task are abnormal or not until all the preposed tasks of the abnormal task are not abnormal, and determining that the preposed abnormal task is not traced from the batch processing task flow corresponding to the abnormal task.
In some embodiments, when the processor implements tracing the pre-exception task from the associated batch task flow of the batch task flow corresponding to the exception task according to the first dependency topology map, the processor is configured to implement:
inquiring an associated batch task flow with a dependency relationship with the batch task flow corresponding to the abnormal task according to the first dependency relationship topological graph;
and tracing the preposed abnormal task from the associated batch task flow.
In some embodiments, when the processor implements the obtaining a first dependency topology between each batch task stream and a second dependency topology between individual tasks in each batch task stream, the processor is configured to implement:
responding to first relation information configured by a user for the dependency relation among the batch task flows in a configuration page, and generating a first dependency relation topological graph;
and generating a second dependency relationship topological graph according to second relationship information configured by the user aiming at the dependency relationship among the single tasks in each batch task flow in the configuration page.
In some embodiments, the processor is configured to implement, when determining whether an exception occurs in a preceding task of the exception task according to the first dependency topology map, to:
inquiring a front-end task of the abnormal task according to the first dependency topological graph, and acquiring an execution state of the front-end task;
and determining whether the previous task is abnormal or not according to the execution state of the previous task.
In some embodiments, the processor implements the asynchronously intervening the pre-exception tasks such that, when the pre-exception tasks and the exception tasks are executed in order, the processor is configured to implement:
acquiring the identification and the abnormal information log of the preposed abnormal task;
generating alarm information according to the identification and the abnormal information log, and sending the alarm information to a service system corresponding to the preposed abnormal task so that the service system can intervene in the preposed abnormal task, and re-executing the abnormal task until the execution is completed under the condition that the preposed abnormal task is completed.
Embodiments of the present application also provide a computer readable storage medium having a computer program stored thereon, where the computer program includes program instructions, where a method implemented when the program instructions are executed may refer to various embodiments of a method for processing a batch task of the present application.
The computer readable storage medium may be an internal storage unit of the computer device according to the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided on the computer device.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created from the use of blockchain nodes, and the like.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments. While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method of processing a batch processing task, the method comprising the steps of:
acquiring a first dependency relationship topological graph among various batch processing task flows and a second dependency relationship topological graph among single tasks in each batch processing task flow;
under the condition that abnormal tasks occur in the process of executing each batch processing task flow, tracing front abnormal tasks which cause the abnormal tasks according to the first dependency relationship topological graph and the second dependency relationship topological graph;
and asynchronously intervening the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
2. The method for processing a batch task according to claim 1, wherein tracing back a pre-exception task that causes the exception task according to the first dependency topology and the second dependency topology, comprises:
according to the second dependency relationship topological graph, tracing the front abnormal task from the batch processing task flow corresponding to the abnormal task;
and under the condition that the front abnormal task is not traced from the batch processing task flow corresponding to the abnormal task, tracing the front abnormal task from the associated batch processing task flow of the batch processing task flow corresponding to the abnormal task according to the first dependency relationship topological graph.
3. The method for processing a batch task according to claim 2, wherein tracing back the pre-exception task from the batch task flow corresponding to the exception task according to the second dependency topology graph includes:
judging whether the front-end task of the abnormal task is abnormal or not according to the two-dependency topological graph;
and under the condition that the previous preposed task is not abnormal, judging whether the previous two preposed tasks of the abnormal task are abnormal or not until all the preposed tasks of the abnormal task are not abnormal, and determining that the preposed abnormal task is not traced from the batch processing task flow corresponding to the abnormal task.
4. The method for processing a batch task according to claim 2, wherein tracing back the pre-exception task from the associated batch task flow of the batch task flow corresponding to the exception task according to the first dependency topology graph includes:
inquiring an associated batch task flow with a dependency relationship with the batch task flow corresponding to the abnormal task according to the first dependency relationship topological graph;
and tracing the preposed abnormal task from the associated batch task flow.
5. A method for processing batch tasks according to claim 1, wherein the obtaining a first dependency topology between each batch task flow and a second dependency topology between individual tasks in each batch task flow comprises:
responding to first relation information configured by a user for the dependency relation among the batch task flows in a configuration page, and generating a first dependency relation topological graph;
and generating a second dependency relationship topological graph according to second relationship information configured by the user aiming at the dependency relationship among the single tasks in each batch task flow in the configuration page.
6. A method for processing a batch task according to claim 3, wherein the determining whether an exception occurs in a preceding task of the exception task according to the first dependency topology comprises:
inquiring a front-end task of the abnormal task according to the first dependency topological graph, and acquiring an execution state of the front-end task;
and determining whether the previous task is abnormal or not according to the execution state of the previous task.
7. The method according to claim 1, wherein the asynchronously intervening in the pre-exception task to complete the orderly execution of the pre-exception task and the exception task comprises:
acquiring the identification and the abnormal information log of the preposed abnormal task;
generating alarm information according to the identification and the abnormal information log, and sending the alarm information to a service system corresponding to the preposed abnormal task so that the service system can intervene in the preposed abnormal task, and re-executing the abnormal task until the execution is completed under the condition that the preposed abnormal task is completed.
8. A batch processing task processing device, characterized in that the batch processing task processing device comprises:
the acquisition module is used for acquiring a first dependency relationship topological graph among the batch task flows and a second dependency relationship topological graph among the single tasks in each batch task flow;
the tracing module is used for tracing the front abnormal task which causes the abnormal task according to the first dependency relationship topological graph and the second dependency relationship topological graph under the condition that the abnormal task occurs in the process of executing each batch processing task flow;
and the intervention module is used for asynchronously intervening the preposed abnormal task so as to enable the preposed abnormal task and the abnormal task to be orderly executed.
9. Computer device, characterized in that it comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when being executed by the processor, realizes the steps of the method for processing batch processing tasks according to any of claims 1 to 7.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program, wherein the computer program, when being executed by a processor, realizes the steps of the method for processing batch processing tasks according to any of claims 1 to 7.
CN202410017247.8A 2024-01-04 2024-01-04 Batch processing task processing method, device, equipment and readable storage medium Pending CN117873768A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410017247.8A CN117873768A (en) 2024-01-04 2024-01-04 Batch processing task processing method, device, equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410017247.8A CN117873768A (en) 2024-01-04 2024-01-04 Batch processing task processing method, device, equipment and readable storage medium

Publications (1)

Publication Number Publication Date
CN117873768A true CN117873768A (en) 2024-04-12

Family

ID=90591458

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410017247.8A Pending CN117873768A (en) 2024-01-04 2024-01-04 Batch processing task processing method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN117873768A (en)

Similar Documents

Publication Publication Date Title
US9020949B2 (en) Method and system for centralized issue tracking
CN109492013B (en) Data processing method, device and system applied to database cluster
CN111694592A (en) Management method and system for project version release
CN107909486B (en) Commission management method, application server and computer readable storage medium
CN111538757B (en) Data storage method, query method, device, server and medium
CN110688828A (en) File processing method and device, file processing system and computer equipment
WO2021139778A1 (en) System scheduling workflow generation method, system, apparatus, and computer readable storage medium
CN113570479B (en) Block chain transmission method, system and storage medium for real estate transaction data
CN114153862A (en) Service data processing method, device, equipment and storage medium
CN111899018B (en) Order data processing method, device, server and storage medium
CN113626218A (en) Data processing method, data processing device, storage medium and computer equipment
CN110908801B (en) Data processing method and device based on block chain, computer equipment and storage medium
CN110059097B (en) Data processing method and device
CN106708869B (en) Group data processing method and device
CN115147031B (en) Clearing workflow execution method, device, equipment and medium
CN117873768A (en) Batch processing task processing method, device, equipment and readable storage medium
CN109377391B (en) Information tracking method, storage medium and server
CN116701053A (en) Method, device, equipment and medium for restoring data backup of production environment database
CN105335362A (en) Real-time data processing method and system, and instant processing system
CN112862245B (en) Data exchange method and device and electronic equipment
CN112039893B (en) Private transaction processing method and device, electronic equipment and readable storage medium
CN116997895A (en) Reducing transaction aborts in an execution ordering validation blockchain model
CN109901933B (en) Operation method and device of business system, storage medium and electronic device
CN114416560A (en) Program crash analysis aggregation method and system
CN113612832A (en) Streaming data distribution method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination