CN113010310B

CN113010310B - Method, device and server for processing job data

Info

Publication number: CN113010310B
Application number: CN202110233003.XA
Authority: CN
Inventors: 唐一帆; 李湘玲; 聂冬琴; 王伟
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2021-03-03
Filing date: 2021-03-03
Publication date: 2024-06-28
Anticipated expiration: 2041-03-03
Also published as: CN113010310A

Abstract

The specification provides a processing method, a processing device and a server for job data. Based on the method, the server can firstly determine the target operation requiring priority processing in the next time period in the current time period, and acquire the identification information and the priority control parameters of the target operation; then, by inquiring the first data table, determining the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; further, a plurality of job processing paths taking the target job as an end job can be determined according to the identification information of the target job, the identification information of the first previous job and the second data table, so as to construct and obtain a corresponding third data table; and then, adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table and the priority control parameters of the target jobs so as to process the related jobs according to the priority parameters adjusted in the current time period in the next time period, thereby preferentially completing the target jobs and meeting the service processing requirements.

Description

Method, device and server for processing job data

Technical Field

The specification belongs to the technical field of big data, and particularly relates to a processing method, a processing device and a processing server of job data.

Background

In a data processing scenario involving large data (e.g., a job data processing scenario for banking), a server responsible for job data processing is faced with a large amount of job data to be processed every day. In order to meet specific business processing requirements, the server is required to perform priority processing on different job data every day. And complex and tedious dependency relationship exists among the operation data. For example, the a job needs to start processing after the B job processing is completed, and so on.

Based on the existing processing method of the operation data, the priority of the operation data is manually configured by combining the complex dependency relationship based on experience knowledge of technicians, so as to realize the operation processing meeting the operation processing requirement. Based on the method, the treatment process is complex and complicated, the treatment difficulty is relatively high, and the treatment cost is relatively high.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The specification provides a method, a device and a server for processing job data, so that the priority adjustment process of the job is simplified, and the processing difficulty and the processing cost of the job data processing process are reduced.

The specification provides a processing method of job data, comprising the following steps:

determining a target job, and acquiring identification information and priority control parameters of the target job; the target job comprises job data which is required to be processed preferentially in the next time period;

Inquiring a first data table according to the identification information of the target job to determine the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job;

Determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table, so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path;

And adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs.

In one embodiment, adjusting the priority parameter of the related job according to the preset processing rule, the third data table, and the identification information of the target job includes:

counting the number of first previous operations of the target operation according to a preset processing rule;

comparing the number of the first previous operations with a preset number threshold value to obtain a comparison result;

According to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path with time consumption meeting preset requirements is screened out of job processing paths taking target jobs as end jobs by inquiring the third data table and is used as a target job processing path;

And according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the job on the target job processing path.

In one embodiment, after comparing the number of the first previous jobs with a preset number threshold, the method further includes:

According to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path taking the target job as an end job is determined as the target job processing path by inquiring the third data table;

In one embodiment, the first data table, the second data table, and the third data table are hash tables.

In one embodiment, before determining the target job and acquiring the identification information and the priority control parameter of the target job, the method further includes:

Acquiring job reference data of a plurality of jobs; wherein the job reference data includes at least: the identification information of the job, the operation frequency of the job and the previous dependency relationship of the job;

according to the job reference data, respectively determining the identification information of a plurality of previous jobs with a previous dependency relationship with each job in the plurality of jobs, and constructing an identification information set of the previous jobs corresponding to the identification information of the jobs;

Determining a first key value according to the identification information of each operation; and determining a first value corresponding to the first key value according to the identification information set of the previous operation corresponding to the identification information of the operation so as to construct a corresponding hash table as the first data table.

In one embodiment, the preceding dependencies of the job are obtained as follows:

acquiring SQL instruction sentences associated with the jobs according to the identification information of the jobs;

and determining the previous dependency relationship corresponding to the job by analyzing the SQL instruction statement.

acquiring a running log record of a job;

according to the operation log record of the job, calculating the average processing time length of each job;

Determining a second key value according to the identification information of each operation; and determining a second value corresponding to the second key value according to the average processing time length of the job so as to construct a corresponding hash table as the second data table.

In one embodiment, determining a plurality of job processing paths using the target job as an endpoint job according to the identification information of the target job, the identification information of the plurality of first previous jobs, and the second data table, so as to construct and obtain a third data table, including:

Inquiring a second data table according to the identification information of the plurality of first preceding jobs and the identification information of the target job to determine the average processing time length of the first preceding jobs and the average processing time length of the target job;

inquiring a first data table according to the identification information of the first previous operation to determine the identification information of a plurality of second previous operations with previous dependency relations with the first previous operation;

Determining a plurality of job processing sub-paths taking the first previous job as an end job and the processing time consumption of the job processing sub-paths by using a recursion algorithm according to the identification information of the first previous job, the identification information of the plurality of second previous jobs and the second data table;

Splicing the target job and the plurality of job processing sub-paths to obtain a plurality of job processing paths taking the target job as an end job; according to the average processing time length of the target job and the processing time consumption of the job processing sub-path, counting the processing time consumption of the job processing path;

Determining a third key value according to the identification information of the starting point operation in the operation processing path; and determining a third value corresponding to the third key value according to the identification information sequence of the job in the job processing path where the starting job is located and the processing time consumption of the job processing path, so as to construct a corresponding hash table as the third data table.

In one embodiment, the determining the target job includes:

receiving a priority adjustment request; the priority adjustment request carries identification information of the to-be-adjusted job of which the priority is to be adjusted and priority control parameters of the to-be-adjusted job; the priority control parameter at least comprises a priority frequency parameter;

acquiring time information of a current time period;

And screening out the to-be-adjusted job requiring priority processing in the next time period after the current time period from the plurality of to-be-adjusted jobs as the target job according to the time information of the current time period and the priority frequency parameter of the to-be-adjusted job.

In one embodiment, the priority control parameter further comprises a priority level parameter;

correspondingly, adjusting the priority parameters of the related jobs comprises:

And adjusting the priority parameters of the related jobs to the priority parameters matched with the priority level parameters.

In one embodiment, after adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table, and the priority control parameters of the target job, the method further includes:

And according to the adjusted priority parameter, the related operation is processed preferentially.

The specification also provides a processing device for job data, comprising:

The first determining module is used for determining a target job and acquiring identification information and priority control parameters of the target job; the target job comprises job data which is required to be processed preferentially in the next time period;

The second determining module is used for inquiring the first data table according to the identification information of the target job so as to determine the identification information of a plurality of first previous jobs with the previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job;

The construction module is used for determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path;

And the processing module is used for adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs.

The present specification also provides a server comprising a processor and a memory for storing processor-executable instructions which, when executed by the processor, implement the steps associated with the method of processing job data as described above.

The present specification also provides a computer-readable storage medium having stored thereon computer instructions which, when executed, implement the steps associated with the method of processing job data as described above.

According to the job data processing method, device and server, the server can automatically determine the target job requiring priority processing in the next time period in the current time period, and acquire the identification information and priority control parameters of the target job; then, by inquiring the first data table, determining the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; further, according to the identification information of the target job, the identification information of the first previous job and the second data table, determining a plurality of job processing paths taking the target job as an end job so as to construct and obtain a corresponding third data table; and then adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table and the priority control parameters of the target jobs, so that the processing of the target jobs can be completed preferentially in the next time period by processing the related jobs according to the priority parameters adjusted in the previous time period. Therefore, the complex job processing requirements can be met, the priority parameters of related jobs related to the priority processing of the target job are automatically and accurately determined and adjusted in the current time period, so that the priority processing of the target job can be accurately completed in the next time period, the manual configuration of the priority parameters is not needed, the priority adjustment process of the job is simplified, and the processing difficulty and the processing cost in the process of job data processing are reduced.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure, the drawings that are required for the embodiments will be briefly described below, in which the drawings are only some of the embodiments described in the present disclosure, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of one embodiment of the structural composition of a system to which the job data processing method provided by the embodiments of the present specification is applied;

FIG. 2 is a flow chart of a method for processing job data according to one embodiment of the present disclosure;

FIG. 3 is a schematic diagram of the structural composition of a server according to one embodiment of the present disclosure;

Fig. 4 is a schematic structural diagram of a processing apparatus for job data provided in one embodiment of the present specification;

Fig. 5 is a schematic diagram of an embodiment of a processing method of job data to which the embodiments of the present specification are applied, in one example of a scenario.

Detailed Description

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

Considering that the existing job data-based processing method often needs to rely on the experience knowledge of technicians, the dependency relationship among the job data is analyzed, and then the priority of the job data is manually configured, so that a plurality of job data requiring priority processing can be processed in a follow-up mode, and specific service processing requirements can be met. Based on the method, in a data processing scene related to big data, because a technician is required to analyze and process complex relations among a large amount of operation data, and manual parameter configuration is also required to be carried out on the large amount of operation data, the processing process of the operation data becomes complex and tedious, the processing cost and the processing difficulty of operation data processing are increased, and the overall processing efficiency is also influenced.

For the root cause of the above problem, the present specification considers that before implementation, a first data table storing the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job may be respectively constructed and obtained according to the related information of all the accessed job data (including the historically processed job, the currently processed job, and the unprocessed job); and a second data table storing identification information of the job and an average processing time length corresponding to the identification information of the job. In the implementation, the target operation requiring priority processing in the next time period can be determined in the current time period, and the data such as the identification information, the priority control parameter and the like of the target operation can be obtained; then, by inquiring the first data table and the second data table, a plurality of job processing paths taking the target job as an end job are determined, the processing time consumption of each job processing path is counted, and a third data table which stores the identification information of the starting job in the job processing paths and the processing time consumption of the job processing paths taking the target job as the end job and corresponding to the identification information of the starting job can be constructed; finally, according to the third data table, the preset processing rule and the priority control parameter of the target job, determining the related job which is related to the priority processing of the target job and also needs to adjust the priority parameter in the next time period; and the priority parameters of the related jobs are correspondingly adjusted in advance in the current time period, so that the jobs can be processed according to the adjusted priority parameters in the next time period, and the target jobs can be processed in a priority mode. Therefore, the complex job processing requirements can be met, the priority parameters of related jobs are automatically and accurately determined and adjusted in the current time period, the priority adjustment process of the jobs is simplified, and the processing difficulty and processing cost in the process of job data processing are reduced.

The embodiment of the specification provides a processing method of job data, which can be particularly applied to a system comprising a server and terminal equipment. Reference may be made in particular to fig. 1. The server and the terminal device may be connected by wired or wireless means to perform specific data interactions.

In this embodiment, the server may specifically include a background server applied to a side of a service processing platform and capable of implementing functions such as data transmission and data processing. Specifically, the server may be, for example, an electronic device having data operation, storage function and network interaction function. Or the server may be a software program running in the electronic device that provides support for data processing, storage, and network interactions. In the present embodiment, the number of servers is not particularly limited. The server may be one server, several servers, or a server cluster formed by several servers.

In this embodiment, the terminal device may specifically include a front-end electronic device applied to a side of a worker (or a technician) and capable of implementing functions such as data acquisition and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a notebook computer, a smart phone, etc. Or the terminal device may be a software application capable of running in the electronic device described above. For example, it may be a client application installed and running on a desktop computer.

In this embodiment, before implementation, the server may first obtain reference data (including identification information of the job, operation frequency of the job, previous dependency relationship of the job, etc.) of all jobs accessed by the platform, and log records of the jobs.

The server may construct a first data table storing identification information of the job and an identification information set of a preceding job corresponding to the identification information of the job, based on the reference data of the job. Meanwhile, the server can construct a second data table storing the identification information of the job according to the operation log record of the job so as to correspond to the average processing time length of the identification information of the job.

The staff can generate and send a priority adjustment request to the server through the terminal equipment according to specific situations and business data processing requirements. The priority adjustment request may specifically carry identification information of the job to be adjusted of the priority to be adjusted, and a priority control parameter of the job to be adjusted. The priority control parameter may at least include a priority frequency parameter.

The server can trigger the target operation requiring priority processing in the next time period according to the priority control parameters every time period; and the priority parameters of related jobs related to the processing of the target jobs are adjusted in advance in the current time period, so that the target jobs can be accurately and smoothly processed in priority in the next time period.

Specifically, taking the current time period as an example, firstly, the server may obtain time information of the current time period, and determine, according to the time information of the current time period, the priority frequency parameter of the job to be adjusted, and the operation frequency of the job to be adjusted, a target job requiring priority processing in a next time period after the current time period from the job to be adjusted, and obtain identification information and priority control parameters of the target job.

The method comprises the steps that a server can obtain identification information of a plurality of first previous jobs with previous dependency relationship with a target job by inquiring a first data table according to the identification information of the target job; and determining a plurality of job processing paths taking the target job as an end job and the processing time of the job processing paths through a recursion algorithm according to the identification information of the target job, the identification information of a plurality of first previous jobs and the second data table so as to construct a third data table storing the identification information of the start job in the job processing paths and the processing time of the job processing paths taking the target job as the end job and the corresponding identification information of the start job.

Specifically, the server may query the second data table according to the identification information of the first previous job and the identification information of the target job, so as to determine an average processing duration of the first previous job and an average processing duration of the target job; inquiring a first data table according to the identification information of the first previous operation to determine the identification information of a plurality of second previous operations with previous dependency relationship with the first previous operation; determining a plurality of job processing sub-paths taking the first previous job as an end job and the processing time consumption of the job processing sub-paths by using a recursion algorithm according to the identification information of the first previous job, the identification information of the plurality of second previous jobs and the second data table; splicing the target job and the plurality of job processing sub-paths to obtain a plurality of job processing paths taking the target job as an end job; and counting the processing time consumption of the job processing path according to the average processing time length of the target job and the processing time consumption of the job processing sub-path so as to establish a corresponding third data table.

Furthermore, the server can determine the related jobs associated with the processing of the target job in the current time period according to the preset processing rule and the third data table, and the related jobs also need to be adjusted in priority; and according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the related jobs.

Thus, by the next time period, the server can process the related jobs according to the pre-adjusted priority parameters, so that the target jobs can be ensured to be accurately and smoothly finished preferentially in the time period.

By the above embodiment, the staff only needs to generate and provide the server with the identification information carrying the job to be tuned and the priority adjustment request of the priority control parameter of the job to be tuned through the terminal device, and the server can trigger and automatically determine the target job requiring the priority processing in the next time period according to the priority adjustment request in each time period, and accurately adjust the priority parameter of the related job related to the priority processing of the target job in the time period, so that the related job can be processed according to the pre-adjusted priority parameter in the next time period to finish the processing of the target job preferentially. Therefore, the complex job processing requirements can be met, the priority adjustment process of the job is simplified, and the processing difficulty and processing cost in the process of job data processing are reduced.

Referring to fig. 2, an embodiment of the present disclosure provides a method for processing job data. The method is particularly applied to the server side. In particular implementations, the method may include the following.

S201: determining a target job, and acquiring identification information and priority control parameters of the target job; wherein the target job includes job data for which priority processing is required for the next time period.

S202: inquiring a first data table according to the identification information of the target job to determine the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; the first data table stores identification information of a job and an identification information set of a previous job corresponding to the identification information of the job.

S203: determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table, so as to construct and obtain a third data table; the second data table stores identification information of the job and average processing time length corresponding to the identification information of the job. The third data table stores identification information of a starting job, and a job processing path and a processing time consumption of the job processing path which are corresponding to the identification information of the starting job and take a target job as an end job.

S204: and adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs so as to finish the processing of the target jobs preferentially in the next time period.

By the embodiment, the priority parameters of related jobs related to the priority processing of the target jobs are automatically and accurately determined and adjusted in the current time period without relying on manual configuration of the priority parameters by users, so that the target jobs can be accurately and preferentially processed and completed in the next time period, the priority adjustment process of the jobs is simplified, and the processing difficulty and processing cost of the job data processing are reduced.

In one embodiment, the target job may be specifically understood as job data that is required to be processed preferentially in a next period according to a service processing requirement. For example, financial statements requiring priority auditing on the tomorrow (end of month), and the like.

In one embodiment, the determining the target job, when implemented, may include: receiving a priority adjustment request; the priority adjustment request carries identification information of the to-be-adjusted job of which the priority is to be adjusted and priority control parameters of the to-be-adjusted job; the priority control parameter at least comprises a priority frequency parameter; acquiring time information of a current time period; and screening out the to-be-adjusted job requiring priority processing in the next time period after the current time period from the plurality of to-be-adjusted jobs as the target job according to the time information of the current time period and the priority frequency parameter of the to-be-adjusted job.

In this embodiment, a user (for example, related staff, technicians, etc.) only needs to determine, in advance, identification information of a job to be adjusted, which needs to adjust the priority, and priority control parameters related to a priority adjustment requirement of the job to be adjusted according to specific situations and service processing requirements, and may generate and send a corresponding priority adjustment request to a server through a terminal device, and the server automatically adjusts the priority parameters of the job according to the priority adjustment request, so as to perform priority processing on a target job requiring priority processing with a higher importance level in a certain period of time.

Through the embodiment, the operation on the user side can be simplified, the user can automatically realize priority configuration by only setting and generating the dependent priority adjustment request, the operation complexity is reduced, and the use experience of the user is improved.

In one embodiment, the identification information may be specifically understood as identification data corresponding to a job and used to distinguish other jobs. Specifically, the identification information may be a name of the job, a number of the job, a storage address of the job, or the like. Of course, the identification information of the above-listed jobs is only one schematic illustration. The specification is not limited to the specific content of the identification information of the job.

The priority control parameter may be specifically understood as parameter data corresponding to the job to be tuned, for describing the priority desired requirement of the job to be tuned. Specifically, the priority control parameter may at least include a priority expected frequency. The expected frequency of the priority may be specifically understood as a frequency parameter for the task to be tuned that needs to be prioritized.

Specifically, for example, for the job to be tuned Q, it is generally required to process 1 time per day (i.e., the running frequency), but priority is not required for each process. According to the service processing requirement, the processing of the job Q to be tuned on the last day of each month is required to be processed preferentially. Therefore, the server only needs to ensure that the job Q to be tuned is processed preferentially on the last day of each month, and the job Q to be tuned can be processed normally in other time periods. Accordingly, the priority frequency parameter may be expressed as: 1 time/month.

Based on the above, when specifically determining the target data, the server may determine that the expected time of the priority of the job Q to be tuned is the last day of the month (for example, 31 days of 1 month) according to the operation frequency and the priority frequency parameter; meanwhile, time information (for example, 1 month and 30 days) of the current time period is acquired; and further, the job to be tuned Q can be determined to be the job requiring priority processing in the next time period after the current time period. Therefore, the job to be tuned Q described above can be determined as one target job.

In addition, the priority control parameter may further include other parameter data such as a priority level parameter, a priority condition parameter, etc., in addition to the priority frequency parameter, so that the server may more finely configure the corresponding priority.

In one embodiment, the user may also set identification information of a plurality of jobs to be adjusted requiring priority processing in different time periods in one priority adjustment request at the same time. After receiving the identification information of the plurality of jobs to be adjusted, the server can find the operation frequency of the jobs to be adjusted according to the identification information of the jobs to be adjusted (the parameters will be described later); and combining the operation frequency of the job to be tuned with the priority frequency of the job to be tuned to determine whether the next time period (for example, tomorrow) in the current time period requires priority processing of the job to be tuned. If it is determined that the next time period requires priority to process the job to be tuned, the job to be tuned may be determined as the target job. In contrast, if it is determined that the next time period does not require priority to process the job to be tuned, the tuning job need not be determined to be the target job. Thus, the target job requiring preferential processing in the next time period can be accurately found in the current time period.

In specific implementation, the server may trigger the execution of the above processing procedure every time interval, and timely determine the target job that requires priority processing in the next time interval.

In one embodiment, after determining the target job, the method may further include: determining other jobs to be processed in the next time period; and initializing and setting the priority parameters of the target job and other jobs in the next time period to the same priority parameters for subsequent priority parameter adjustment.

In one embodiment, the first data table may be specifically understood as a data table that is built in advance and stores identification information of each job and an identification information set of a preceding job corresponding to the identification information of the job.

The second data table may be specifically understood as a data table which is built in advance and stores identification information of each job and average processing time length corresponding to the identification information of the job.

The third data table is specifically a data table constructed based on the job processing paths having the target job as the destination job, and storing the identification information of the start job in each of the job processing paths, the sequence of the job identification information in the job processing path corresponding to the identification information of the start job, and the processing time consumption of the job processing path.

In one embodiment, the first data table, the second data table, and the third data table may be a hash table. Specifically, it can also be respectively noted as: h1, h2, h3.

By the above embodiment, the first data table, the second data table and the third data table are constructed by using the data structure of the hash table, so that the related data query, data retrieval and other operations can be performed by using the first data table, the second data table and the third data table later, which is more convenient and efficient, and the overall data processing efficiency can be improved.

In one embodiment, the method may further construct a corresponding first data table in advance before determining the target job and acquiring the identification information and the priority control parameter of the target job. The first data table may be constructed specifically as follows.

S1: acquiring job reference data of a plurality of jobs; wherein the job reference data includes at least: the identification information of the job, the operation frequency of the job and the previous dependency relationship of the job;

s2: according to the job reference data, respectively determining the identification information of a plurality of previous jobs with a previous dependency relationship with each job in the plurality of jobs, and constructing an identification information set of the previous jobs corresponding to the identification information of the jobs;

s3: determining a first key value according to the identification information of each operation; and determining a first value corresponding to the first key value according to the identification information set of the previous operation corresponding to the identification information of the operation so as to construct a corresponding hash table as the first data table.

In this embodiment, the plurality of jobs may specifically include all jobs accessed by a platform or a server. Specifically, it may include: jobs that have been processed by the server, jobs that are being processed by the server, and jobs that have not been processed by the server.

Through the embodiment, the first data table which is relatively comprehensive and complete in coverage, relatively high in reference value and convenient for subsequent operations such as searching and inquiring can be constructed in advance.

In one embodiment, the operation frequency of the job may be specifically understood as an operation frequency of the job. For example, the operation frequency of the job a is 1 time/day.

The foregoing dependency of the job may be understood as, in particular, relationship data representing the dependency between a job to be processed and other jobs (which may be denoted as a preceding job) that are required to be processed before the job is processed based on processing logic. For example, before processing the a job, the B job and the C job are required to be executed first, and the B job and the C job may be noted as the preceding jobs of the a job.

In one embodiment, the previous dependency relationship of the job may be obtained specifically in the following manner: acquiring SQL instruction sentences associated with the jobs according to the identification information of the jobs; and determining the previous dependency relationship corresponding to the job by analyzing the SQL instruction statement.

Through the embodiment, the previous dependency relationship of each operation can be accurately obtained through analysis.

In this embodiment, the above-mentioned SQL (Structured Query Language ) may specifically refer to a programming language related to database query and programming, for accessing data, and querying, updating, and managing a relational database system, etc. Typically, when processing certain job data, the SQL instruction statement is used to query the processing status of the previous job of the job data to determine whether the job data can be processed. Therefore, based on the SQL instruction statement, the previous job queried when processing the job data can be found, so as to acquire the previous dependency relationship corresponding to the job.

In one embodiment, the job reference data may further include other types of reference data such as a start time and an end time of the job, and a service type to which the job belongs, in addition to the identification information of the job, the operation frequency of the job, and the previous dependency relationship of the job.

In one embodiment, when the first data table is specifically constructed, the first key value and the first value corresponding to each other may be combined into a key value pair, and the key value pair is stored in the hash table in the form of structured data, so as to obtain the first data table.

In one embodiment, the method may further construct a corresponding second data table in advance before determining the target job and acquiring the identification information and the priority control parameter of the target job. The second data table may be constructed specifically as follows.

S1: acquiring a running log record of a job;

s2: according to the operation log record of the job, calculating the average processing time length of each job;

S3: determining a second key value according to the identification information of each operation; and determining a second value corresponding to the second key value according to the average processing time length of the job so as to construct a corresponding hash table as the second data table.

Through the embodiment, the second data table with higher reference value and convenient subsequent operations of searching, inquiring and the like can be constructed in advance.

In one embodiment, after obtaining the log record of the job, before counting the average processing duration of each job, the method further includes: performing anomaly detection on the operation log record of the job; and eliminating abnormal operation records in the operation log records of the job.

Specifically, the server may detect, one by one, the operation records recorded in the operation log record of the job according to the job reference data such as the operation frequency of the job, and find out an operation record job abnormal operation record that does not match with the operation reference data such as the operation frequency of the job. Furthermore, the average processing time length of the job can be calculated according to the operation log records of the job which is relatively normal and has higher reference value and is removed from the abnormal operation records, so that the accuracy of the determined average processing time length can be improved, and errors introduced by the abnormal operation records can be reduced.

In one embodiment, the determining the plurality of job processing paths using the target job as the destination job according to the identification information of the target job, the identification information of the plurality of first previous jobs, and the second data table to construct the third data table may include the following when implemented.

S1: inquiring a second data table according to the identification information of the plurality of first preceding jobs and the identification information of the target job to determine the average processing time length of the first preceding jobs and the average processing time length of the target job;

S2: inquiring a first data table according to the identification information of the first previous operation to determine the identification information of a plurality of second previous operations with previous dependency relations with the first previous operation;

s3: determining a plurality of job processing sub-paths taking the first previous job as an end job and the processing time consumption of the job processing sub-paths by using a recursion algorithm according to the identification information of the first previous job, the identification information of the plurality of second previous jobs and the second data table;

S4: splicing the target job and the plurality of job processing sub-paths to obtain a plurality of job processing paths taking the target job as an end job; according to the average processing time length of the target job and the processing time consumption of the job processing sub-path, counting the processing time consumption of the job processing path;

S5: determining a third key value according to the identification information of the starting point operation in the operation processing path; and determining a third value corresponding to the third key value according to the identification information sequence of the job in the job processing path where the starting job is located and the processing time consumption of the job processing path, so as to construct a corresponding hash table as the third data table.

Through the embodiment, the third data table with higher reference value and convenient subsequent operations such as searching, inquiring and the like can be quickly constructed.

In one embodiment, determining, by using a recursive algorithm, a plurality of job processing sub-paths using the first previous job as an endpoint job according to the identification information of the first previous job, the identification information of the plurality of second previous jobs, and the second data table, and when the processing of the job processing sub-paths is time-consuming, may include: querying a second data table according to the identification information of the second previous jobs by using a recursion algorithm to determine the average processing time length of the second previous jobs; and according to the identification information of the second previous operation, inquiring a first data table to determine the identification information of a plurality of third previous operations with the second previous operation, wherein the third previous operations have the previous dependency relationship with the second previous operation. And similarly, determining the corresponding fourth and fifth previous operations until the Nth previous operation. When it is determined that the nth previous job is found, the first data table is queried, and if it is found that the n+1th previous job having a previous dependency relationship with the nth previous job does not exist, the search for the job processing sub-path is stopped, and the nth previous job is used as a starting point (may also be referred to as a source job or the like).

Further, the jobs between the start job (i.e., the nth previous job, where N is 1 or more) and the end job (i.e., the target job) may be arranged in order, and the corresponding job processing path may be obtained. Therefore, a plurality of job processing paths taking the target job as the end job can be found, and omission or errors are avoided.

Of course, in the specific implementation, the server may also gradually splice the job processing paths of the starting job and the first previous job through a recursive algorithm, so as to obtain a job processing sub-path with the first previous job as the end job. For example, a job sub-path is obtained as follows: { jobN- > job (N-1) - > … … job2- > job1}. Wherein jobN is the identification information of the starting job in the job sub-path, job1 is the identification information of a certain first previous job of the target job.

Further, the job processing sub-path may be spliced with the target job to obtain a job processing path having the target job as an end job. Specifically, the expression can be represented as follows: { jobN- > job (N-1) - > … … job2- > job1- > finaljob }. Wherein finaljob is the identification information of the target job.

Then, according to the average processing duration of each job on the job processing path queried by the second data table, the processing time consumption of the job processing path can be counted. Meanwhile, according to the job processing path, the identification information of the jobs on the job processing path is acquired and sequenced (for example, the sequence from the start job to the end job) to obtain the identification information sequence of the jobs in the job processing path. For example, the process time may be represented as follows: tN1+t (N-1) +t+ … … +t21+t1+ tfianl. Wherein tN1 is the average processing time length of the starting job, t1 is the average processing time length of the first preceding job of the target job, and tfinal is the average processing time length of the target job.

And finally, taking the identification information of the starting point job of the job processing path as a third key value, taking the identification information sequence of the job processing path where the starting point job is located (corresponding to a job processing path pointing to the target job from the starting point job) and the processing time of the job processing path as a third value corresponding to the third key value, constructing a corresponding key value pair, and storing the data in the form of structured data to obtain a third data table.

In this embodiment, the job processing path may specifically represent the following job processing flow: based on the processing logic, processing is started from the start job, and the respective jobs are sequentially processed in order until the target job is finally processed.

In one embodiment, it is also possible to find a plurality of (including two) job processing paths with the same target job as the destination job and with the same job (for example, the mth previous job) as the start job through the above embodiment, where the service may further merge the plurality of job processing paths with the same start job for subsequent processing.

Specifically, the processing of a plurality of operation processing paths with the same starting point operation is time-consuming, and the price is compared to obtain a comparison result; according to the comparison result, only the identification information sequence and the processing time of the job of the longest processing time consuming one of the plurality of job processing paths with the same starting point job are reserved in the third data table.

In one embodiment, the priority control parameter further comprises a priority level parameter; correspondingly, adjusting the priority parameters of the related jobs may specifically include: and adjusting the priority parameters of the related jobs to the priority parameters matched with the priority level parameters.

By the above-described embodiments, the priority parameters of the related jobs associated with the priority processing of the target job can be more finely adjusted appropriately.

Specifically, for example, when the priority level parameter of the target job is the highest level, the server may uniformly adjust the priority parameter in the next period of time of the relevant job associated with the target job processing to the highest level.

In one embodiment, the adjusting the priority parameter of the related job according to the preset processing rule, the third data table, and the identification information of the target job may include the following when the implementation is performed: counting the number of first previous operations of the target operation according to a preset processing rule; comparing the number of the first previous operations with a preset number threshold value to obtain a comparison result; according to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path with time consumption meeting preset requirements is screened out of job processing paths taking target jobs as end jobs by inquiring the third data table and is used as a target job processing path; and according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the job on the target job processing path.

In this embodiment, the job processing path whose processing time consumption meets the preset requirement may be specifically the job processing path whose processing time consumption is the longest.

In this embodiment, when the number of the first previous jobs of the target job is determined to be large, for example, greater than a preset number threshold, in this case, if all the first previous jobs are regarded as related jobs associated with the target job processing, the statistics is performed to adjust the priority parameters, which would occupy a large amount of processing resources and cause an overload on the overall data processing of the server; meanwhile, a certain job processing path with larger influence on the target job processing cannot be focused, and the priority processing effect of the target job in the next time period is influenced.

Therefore, in the case that the first preceding job of the target job is more, only the priority parameters of the next time period of the jobs (including the start job, the end job, and other jobs located between the start job and the end job along the target job processing path) included in the target job processing path with the longest processing time can be correspondingly adjusted, so that the priority parameters of the jobs of the next time period can be adjusted more reasonably and pertinently, and a relatively better priority processing effect on the target jobs can be obtained.

In one embodiment, after comparing the number of the first previous jobs with the preset number threshold to obtain a comparison result, the method may further include the following when implemented: according to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path taking the target job as an end job is determined as the target job processing path by inquiring the third data table; and according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the job on the target job processing path.

By the above embodiment, for the case that the first previous jobs of the target job are fewer, the first previous jobs of the target job can be used as related jobs related to the target job processing, and then priority parameters of the jobs can be uniformly adjusted, so that a large amount of processing resources are not required to be occupied, and the data processing of the whole server is not overburdened, and a better priority processing effect on the target job can be obtained.

In one embodiment, after adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table and the priority control parameters of the target job, the method may further include, when implemented: and according to the adjusted priority parameter, the related operation is processed preferentially.

By the above embodiment, by the next time period, the server can process the related job associated with the priority processing of the target job according to the priority parameter adjusted in advance in the previous time period, so that the effect of processing the target job preferentially in the time period can be achieved.

As can be seen from the above, in the job data processing method provided in the embodiments of the present disclosure, the server may automatically determine, in the current time period, a target job that needs to be processed preferentially in the next time period, and obtain identification information and priority control parameters of the target job; then, by inquiring the first data table, determining the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; further, according to the identification information of the target job, the identification information of the first previous job and the second data table, determining a plurality of job processing paths taking the target job as an end job so as to construct and obtain a corresponding third data table; and then adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table and the priority control parameters of the target jobs so as to process the related jobs according to the priority parameters adjusted in the current time period in the next time period, thereby preferentially completing the processing of the target jobs. Therefore, the complex job processing requirements can be met, and the priority parameters of related jobs are automatically and accurately determined and adjusted in the current time period, so that the target jobs can be accurately and preferentially processed in the next time period, the priority adjustment process of the jobs is simplified, and the processing difficulty and the processing cost of the job data processing process are reduced.

The embodiment of the specification also provides a server, which comprises a processor and a memory for storing instructions executable by the processor, wherein the processor can execute the following steps according to the instructions when being implemented: determining a target job, and acquiring identification information and priority control parameters of the target job; the target job comprises job data which is required to be processed preferentially in the next time period; inquiring a first data table according to the identification information of the target job to determine the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job; determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table, so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path; and adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs.

In order to more accurately complete the above instructions, referring to fig. 3, another specific server is further provided in this embodiment of the present disclosure, where the server includes a network communication port 301, a processor 302, and a memory 303, and the above structures are connected by an internal cable, so that each structure may perform specific data interaction.

Wherein, the network communication port 301 may be specifically configured to receive a priority adjustment request; the priority adjustment request carries identification information of the to-be-adjusted job of which the priority is to be adjusted and priority control parameters of the to-be-adjusted job; the priority control parameter at least comprises a priority frequency parameter.

The processor 302 may be specifically configured to determine a target job according to the priority adjustment request, and obtain identification information and a priority control parameter of the target job; the target job comprises job data which is required to be processed preferentially in the next time period; inquiring a first data table according to the identification information of the target job to determine the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job; determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table, so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path; and adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs.

The memory 303 may be used for storing a corresponding program of instructions.

In this embodiment, the network communication port 301 may be a virtual port that binds with different communication protocols, so that different data may be sent or received. For example, the network communication port may be a port responsible for performing web data communication, a port responsible for performing FTP data communication, or a port responsible for performing mail data communication. The network communication port may also be an entity's communication interface or a communication chip. For example, it may be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it may also be a Wifi chip; it may also be a bluetooth chip.

In this embodiment, the processor 302 may be implemented in any suitable manner. For example, a processor may take the form of, for example, a microprocessor or processor, and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, among others. The description is not intended to be limiting.

In this embodiment, the memory 303 may include a plurality of layers, and in a digital system, the memory may be any memory as long as it can hold binary data; in an integrated circuit, a circuit with a memory function without a physical form is also called a memory, such as a RAM, a FIFO, etc.; in the system, the storage device in physical form is also called a memory, such as a memory bank, a TF card, and the like.

The embodiments of the present specification also provide a computer storage medium storing computer program instructions that when executed implement a method for processing job data as described above: determining a target job, and acquiring identification information and priority control parameters of the target job; the target job comprises job data which is required to be processed preferentially in the next time period; inquiring a first data table according to the identification information of the target job to determine the identification information of a plurality of first previous jobs with previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job; determining a plurality of job processing paths taking the target job as an end job according to the identification information of the target job, the identification information of the plurality of first previous jobs and the second data table, so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path; and adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs.

In the present embodiment, the storage medium includes, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read-Only Memory (ROM), a Cache (Cache), a hard disk (HARD DISK DRIVE, HDD), or a Memory Card (Memory Card). The memory may be used to store computer program instructions. The network communication unit may be an interface for performing network connection communication, which is set in accordance with a standard prescribed by a communication protocol.

In this embodiment, the functions and effects of the program instructions stored in the computer storage medium may be explained in comparison with other embodiments, and are not described herein.

Referring to fig. 4, on a software level, the embodiment of the present disclosure further provides a processing device for job data, where the device may specifically include the following structural modules.

The first determining module 401 may be specifically configured to determine a target job, and obtain identification information and a priority control parameter of the target job; the target job comprises job data which is required to be processed preferentially in the next time period;

The second determining module 402 may specifically be configured to query a first data table according to the identification information of the target job, so as to determine identification information of a plurality of first previous jobs that have a previous dependency relationship with the target job; the first data table stores the identification information of the job and the identification information set of the previous job corresponding to the identification information of the job;

The construction module 403 may be specifically configured to determine, according to the identification information of the target job, the identification information of the plurality of first previous jobs, and the second data table, a plurality of job processing paths that use the target job as an endpoint job, so as to construct and obtain a third data table; the second data table stores the identification information of the job and the average processing time length corresponding to the identification information of the job; the third data table stores identification information of a starting point job, a job processing path which corresponds to the identification information of the starting point job and takes a target job as an end point job, and processing time consumption of the job processing path;

The processing module 404 may be specifically configured to adjust the priority parameter of the related job according to a preset processing rule, the third data table, and the priority control parameter of the target job.

It should be noted that, the units, devices, or modules described in the above embodiments may be implemented by a computer chip or entity, or may be implemented by a product having a certain function. For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, when the present description is implemented, the functions of each module may be implemented in the same piece or pieces of software and/or hardware, or a module that implements the same function may be implemented by a plurality of sub-modules or a combination of sub-units, or the like. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

From the above, the processing device for job data provided in the embodiments of the present disclosure can meet the complex job processing requirement, and automatically and accurately determine and adjust the priority parameter of the related job in the current time period, so as to ensure that the target job can be accurately and preferentially processed in the next time period, simplify the priority adjustment process of the job, and reduce the processing difficulty and processing cost of the job data processing process.

In a specific scenario example, the job data processing method provided in the present specification may be applied to complete automatic configuration of job priorities for a complex batch scheduling system. For a specific implementation, reference may be made to the following.

Referring to fig. 5, the implementation may include the following steps.

S1: acquiring all job status information (for example, reference data of a job) in a scheduling system (or a server), and initializing and setting the job status (for example, priority parameters of the job);

s2: acquiring expected state parameters of a target job (for example, priority control parameters of the target job) required to improve timeliness;

S3: preprocessing according to the combination of the acquired expected state parameters and the current batch state parameters;

S4: acquiring information of all jobs on which a target job depends, judging and selecting a strategy for adjusting the priority to improve the priority on the longest path (for example, a target job processing path with the longest processing time) according to the dependent job information; whether to increase the priority of all the preceding jobs (e.g., the first preceding job) of the destination job;

S5: for a job group needing to acquire the longest path job, extracting and calculating the job on the longest path according to the job state information characteristics in the scheduling system;

s6: and updating the job group subjected to screening according to the expected priority state parameters.

Based on the above steps, further refinement may include the following when embodied.

The first step: the scheduling system acquires the state information of all the jobs and performs initialization setting, including: and acquiring the frequency of operation of all the jobs, the previous dependency relationship of the jobs, and the starting time and the ending time of operation of each job in a period of time. And initializing all the jobs, namely initializing the priority level of all the jobs to the lowest priority level.

And a second step of: the obtaining of the expected state parameters of the target operation required to improve the aging comprises the following steps: the name of the destination job, the level parameter of the desired priority, the desired frequency parameter of the required set priority.

And a third step of: and (5) pretreatment. The preprocessing module comprises the following steps: and obtaining the latest date meeting the frequency requirement according to the date frequency of the job specific needing to be set with the priority in the expected state parameter, comparing the expected date with the currently running batch date parameter, and updating the priority of the target job in the current batch if the current batch date is the day before the expected date. The expected date frequency is not an essential parameter, and for a destination job not setting the parameter, priority control is required for the job in the daily lot.

Fourth step: the method comprises the steps of obtaining the job name related to the longest path of a target job, and carrying out hash storage on the dependency relationship of the global job of a dispatching system and the effective average time length required by single job operation. And carrying out forward dependent operation acquisition by using the target operation as an end operation through a recursion algorithm. The method comprises the steps of taking a target job as an end point, taking a source data loading job as a starting point, taking a plurality of links passing in the middle, taking the job of a current node as a key value, obtaining the key value as the average time required by the dependent job and the dependent job in a hash table stored in advance, accumulating the obtained average time of the jobs of each node, finally sequencing the total duration of each link, and selecting a job group on the link with the longest time consumption as the longest path.

The longest path acquisition may specifically include the following:

1. each job in the dispatching system is used as a unique key value in the hash table h1, all job name sets of the previous dependences are used as an array, and the array is used as a value corresponding to the array to be stored, so that the hash table h1 is formed.

2. Extracting the average time length of effective operation in one month of each operation, removing operation log records which do not meet the operation frequency requirement, taking the operation name participating in statistics as a key value of a hash table h2, and simultaneously storing the statistical result of the average time length corresponding to the operation name as a value to form the hash table h2.

3. And searching and acquiring a corresponding value in h1 by taking the end-point job finaljob as a key, namely, an array formed by the dependent jobs as a first-layer dependent job group.

4. The first element job1 in the acquired first-layer dependent job group, namely the name of the first dependent job, is used as a key value to retrieve and obtain the average running duration t1 of the first element job1 in the hash table h 2. With the first element name job1 as key, the path { job1- > finaljob } of the endpoint job to the dependent job and the time consumption t1 of the path as value, a hash table h3 is formed. Then, each element in the first layer is operated on as above.

5. And taking each element in the first-layer dependent operation group as a key to continuously acquire a corresponding dependent operation array in h1 and taking the element as a second-layer dependent operation array, so that the number of the second-layer dependent operation groups is the same as the number of elements in the first-layer dependent operation groups. The operation of the fourth step above is performed for each second-layer dependent job group. When the element of the second-layer dependent operation group also acquires the corresponding average duration t21 in the hash table h2, firstly judging whether the operation name of the element exists in the key in h3, if so, modifying the path { job21- > job1- > finaljob } in the value and the time consumption as t1+t21, and if not, newly adding the key and the value.

6. And continuously acquiring a corresponding dependent operation array by taking each element in the second layer dependent operation group as a key in h1 to serve as a third layer dependent operation array. Taking a third layer dependency operation array as an example, taking a first element job31 as an example for an average duration t31 corresponding to each element in the array through a hash table h2 acquirer, if the job31 does not have a same name key in h3, adding the job31 as a key, and taking { job31- > job21- > finaljob } and t31+t21+t1 as values. If the same name key exists in h3, comparing the accumulated time of t31+t21+t1 with the time consumption in the original value, and if the former time consumption is longer, reassigning the value with a new path and total time consumption.

7. The elements in the third layer dependency job set are sequentially subjected to the operation of step6 above, and thus the recursive operation is performed to the last layer dependency.

8. And ordering the finally formed hash table h3 from large to small by taking the length of time consumption as an element, and extracting the path in the record with the longest time consumption as the final longest path.

Fifth step: all job information on which the destination job depends is acquired and a priority setting policy is selected to include. All the previous dependent job information of the current job as the destination job is obtained through global dependent configuration information and a recursive algorithm in the dispatching system. Because the priority of all the previous jobs is uniformly set under the condition of too many previous dependent jobs, the actual completion time of the target job cannot be advanced fundamentally. Therefore, in the invention, a threshold value of the duty ratio of the number of the previous dependent jobs of the destination job and the number of the global jobs in the dispatching system is set, when the threshold value is larger than the threshold value, the longest path acquisition module is automatically entered, otherwise, the information of all the previous dependent jobs of the destination job is stored for subsequent operation.

Sixth step: and updating the initialized job list according to the job group of the longest path obtained by processing or the job group obtained by de-duplication of all dependent jobs and the priority level in the expected state parameters.

Through the scene example, the timeliness of key data processing can be effectively improved; by setting and utilizing the priority frequency control parameters, the system resources can be utilized maximally; the staff only needs to maintain the relevant expected parameters of the target operation, does not need to control the parameters of the relevant dependent operations except the target operation, and greatly reduces the operation and maintenance cost and difficulty.

Although the present description provides method operational steps as described in the examples or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented by an apparatus or client product in practice, the methods illustrated in the embodiments or figures may be performed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment). The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element. The terms first, second, etc. are used to denote a name, but not any particular order.

Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller can be regarded as a hardware component, and means for implementing various functions included therein can also be regarded as a structure within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of embodiments, it will be apparent to those skilled in the art that the present description may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the present specification may be embodied essentially in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and include several instructions to cause a computer device (which may be a personal computer, a mobile terminal, a server, or a network device, etc.) to perform the methods described in the various embodiments or portions of the embodiments of the present specification.

Various embodiments in this specification are described in a progressive manner, and identical or similar parts are all provided for each embodiment, each embodiment focusing on differences from other embodiments. The specification is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

Although the present specification has been described by way of example, it will be appreciated by those skilled in the art that there are many variations and modifications to the specification without departing from the spirit of the specification, and it is intended that the appended claims encompass such variations and modifications as do not depart from the spirit of the specification.

Claims

1. A method of processing job data, comprising:

Adjusting priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs;

The adjusting the priority parameter of the related job according to the preset processing rule, the third data table and the identification information of the target job comprises the following steps: counting the number of first previous operations of the target operation according to a preset processing rule; comparing the number of the first previous operations with a preset number threshold value to obtain a comparison result; according to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path with time consumption meeting preset requirements is screened out of job processing paths taking target jobs as end jobs by inquiring the third data table and is used as a target job processing path; and according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the job on the target job processing path.

2. The method of claim 1, wherein after comparing the number of the first preceding jobs to a preset number threshold, the method further comprises:

3. The method of claim 1, wherein the first data table, the second data table, and the third data table are hash tables.

4. A method according to claim 3, wherein before determining the target job and acquiring the identification information and the priority control parameters of the target job, the method further comprises:

5. The method of claim 4, wherein the preceding dependency of the job is obtained as follows:

6. A method according to claim 3, wherein before determining the target job and acquiring the identification information and the priority control parameters of the target job, the method further comprises:

acquiring a running log record of a job;

7. The method of claim 3, wherein determining a plurality of job processing paths having the target job as an end job based on the identification information of the target job, the identification information of the plurality of first preceding jobs, and the second data table to construct a third data table comprises:

8. The method of claim 4, wherein the determining the target job comprises:

acquiring time information of a current time period;

9. The method of claim 8, wherein the priority control parameters further comprise priority level parameters;

10. The method of claim 9, wherein after adjusting the priority parameters of the related jobs according to the preset processing rules, the third data table, the priority control parameters of the target jobs, the method further comprises:

11. A job data processing apparatus, comprising:

The processing module is used for adjusting the priority parameters of related jobs according to preset processing rules, the third data table and the priority control parameters of the target jobs;

The processing module is specifically used for counting the number of first previous jobs of the target job according to a preset processing rule; comparing the number of the first previous operations with a preset number threshold value to obtain a comparison result; according to the comparison result, under the condition that the number of the first previous jobs is determined to be larger than the preset number threshold, a job processing path with time consumption meeting preset requirements is screened out of job processing paths taking target jobs as end jobs by inquiring the third data table and is used as a target job processing path; and according to the priority control parameters of the target job, carrying out matched adjustment on the priority parameters of the job on the target job processing path.

12. A server comprising a processor and a memory for storing processor-executable instructions, which when executed by the processor implement the steps of the method of any one of claims 1 to 10.

13. A computer readable storage medium, having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of claims 1 to 10.