CN116401061A

CN116401061A - Method and device for processing resource data, computer equipment and storage medium

Info

Publication number: CN116401061A
Application number: CN202310389694.1A
Authority: CN
Inventors: 陈朝亮
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2023-04-03
Filing date: 2023-04-03
Publication date: 2023-07-07

Abstract

The embodiment of the application belongs to the field of big data, and relates to a method for processing resource data, which comprises the following steps: acquiring the resource consumption time length of each job from the job log; collecting the dependency relationship of each job, dividing the job into a public job and a downstream dependent job, and determining a target job from the public job; calculating the actual resource cost and the resource consumption weight of the target job based on the resource consumption time length; acquiring the relied times of the target operation; calculating the allocation cost of the target job based on the number of depended times, the actual resource cost and the resource consumption weight; the target actual resource cost of the downstream dependent job is calculated based on the apportionment cost. The application also provides a processing device of the resource data, a computer device and a storage medium. In addition, the present application relates to blockchain technology, where the actual resource costs may be stored. The method and the device improve the processing efficiency of generating the actual resource cost of the downstream dependent operation, and ensure the accuracy of the target actual resource cost.

Description

Method and device for processing resource data, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of big data technologies, and in particular, to a method and an apparatus for processing resource data, a computer device, and a storage medium.

Background

With the increasing size and business of financial enterprises, most of the existing financial enterprises use data warehouses to schedule job resources, and currently, there is a general business requirement for calculating resource costs for downstream jobs in the data warehouses. In the existing processing flow for calculating the resource cost of the downstream job in the data warehouse, a manual statistics mode is generally adopted, a calculator can roughly estimate the actual calculation resource of the downstream job according to the number of times that the downstream job depends on the public job, the processing mode needs to consume more manpower resources, and the calculation resource result obtained by statistics can have larger deviation, so that the accuracy of the calculation resource result can not be ensured.

Disclosure of Invention

An objective of the embodiments of the present application is to provide a method, an apparatus, a computer device, and a storage medium for processing resource data, so as to solve the technical problems that in the existing processing flow for performing resource cost calculation on a downstream job in a data warehouse, a manual statistics manner is generally adopted, a calculator can roughly estimate actual computing resources of the downstream job according to the number of times that the downstream job depends on a public job, such processing manner needs to consume more manpower resources, and the computing resource result obtained by statistics may have larger deviation, and the accuracy of the computing resource result cannot be guaranteed.

In order to solve the above technical problems, the embodiments of the present application provide a method for processing resource data, which adopts the following technical schemes:

acquiring the current resource consumption time length of each job from a prestored job log;

collecting the dependency relationship of each job, dividing the jobs into public jobs and downstream dependent jobs based on the dependency relationship, and determining a target job from the public jobs; wherein the number of target jobs includes a plurality;

calculating the actual resource cost of the target job based on the resource consumption duration, and calculating the resource consumption weight of the target job based on the actual resource cost;

acquiring the relied times of the target operation;

calculating the allocation cost of the target job based on the number of dependences, the actual resource cost and the resource consumption weight;

based on the apportionment costs, a target actual resource cost of the downstream dependent job is calculated.

Further, the step of calculating the actual resource cost of the target job based on the resource consumption duration specifically includes:

acquiring the appointed resource consumption duration of all jobs with upstream dependency relationship with the first target job based on the resource consumption duration; wherein the first target job is any one job among all the target jobs;

Calculating a first sum of all the specified resource consumption time periods;

and taking the first sum value as a first actual resource cost of the first target job.

Further, the step of calculating the resource consumption weight of the target job based on the actual resource cost specifically includes:

acquiring a second actual resource cost of a second target job; wherein the second target job is any one job among all the target jobs;

calculating a second sum of the actual resource costs of all the target jobs to obtain the total resource cost of the job links corresponding to all the jobs;

calculating a quotient of the second actual resource cost and the total resource cost;

and taking the quotient as a first resource consumption weight of the second target job.

Further, the step of calculating the allocation cost of the target job based on the dependent times, the actual resource cost and the resource consumption weight specifically includes:

acquiring a second resource consumption weight of a third target job; wherein the third target job is any one job among all the target jobs;

acquiring summarized actual resource cost corresponding to the public job;

Calculating the product of the summarized actual resource cost and the second resource consumption weight;

acquiring the number of times of target dependence corresponding to the third target operation;

and calculating the first allocation cost of the third target operation based on the number of times the target is relied and the product.

Further, the step of calculating the target actual resource cost of the downstream dependent job based on the apportionment cost specifically includes:

acquiring self resource cost of the appointed downstream dependent operation; wherein the specified downstream dependent job is any one of all the downstream dependent jobs;

determining a fourth target job having a dependency relationship with the specified downstream dependent job;

acquiring a second allocation cost corresponding to the fourth target job;

and calculating the target actual resource cost corresponding to the specified downstream dependent job based on the self resource cost and the second allocation cost.

Further, after the step of calculating the target actual resource cost of the downstream dependent job based on the apportionment cost, the method further includes:

judging whether the owner information corresponding to the appointed operation exists or not; wherein the specified job is any one of all the jobs;

If the owner information corresponding to the appointed operation exists, acquiring the appointed actual resource cost corresponding to the appointed operation;

summarizing the appointed actual resource cost to an owner corresponding to the owner information;

and if the owner information corresponding to the appointed operation does not exist, the appointed actual resource cost is distributed to a preset group.

acquiring a target data type corresponding to the target actual resource cost;

determining a data storage mode corresponding to the target data type;

and storing the target actual resource cost based on the data storage mode.

In order to solve the above technical problems, the embodiments of the present application further provide a device for processing resource data, which adopts the following technical scheme:

the first acquisition module is used for acquiring the current resource consumption duration of each job from the prestored job log;

the first processing module is used for collecting the dependency relationship of each job, dividing the jobs into public jobs and downstream dependent jobs based on the dependency relationship, and determining a target job from the public jobs; wherein the number of target jobs includes a plurality;

A first calculation module, configured to calculate an actual resource cost of the target job based on the resource consumption duration, and calculate a resource consumption weight of the target job based on the actual resource cost;

the second acquisition module is used for acquiring the relied times of the target operation;

the second calculation module is used for calculating the allocation cost of the target job based on the relied times, the actual resource cost and the resource consumption weight;

and a third calculation module for calculating a target actual resource cost of the downstream dependent job based on the apportionment cost.

In order to solve the above technical problems, the embodiments of the present application further provide a computer device, which adopts the following technical schemes:

Acquiring the relied times of the target operation;

In order to solve the above technical problems, embodiments of the present application further provide a computer readable storage medium, which adopts the following technical solutions:

acquiring the relied times of the target operation;

Compared with the prior art, the embodiment of the application has the following main beneficial effects:

firstly, acquiring the current resource consumption duration of each job from a prestored job log; then collecting the dependency relationship of each job, dividing the jobs into public jobs and downstream dependent jobs based on the dependency relationship, and determining a target job from the public jobs; wherein the number of target jobs includes a plurality; calculating the actual resource cost of the target job based on the resource consumption time length, and calculating the resource consumption weight of the target job based on the actual resource cost; subsequently obtaining the depended times of the target operation; calculating the allocation cost of the target job based on the relied times, the actual resource cost and the resource consumption weight; and finally, calculating the target actual resource cost of the downstream dependent job based on the apportionment cost. According to the method and the device for calculating the target actual resource cost, the obtained resource consumption time of each job and the dependency relationship of each job are subjected to corresponding calculation processing, so that the target actual resource cost corresponding to the downstream dependent job can be automatically, quickly and accurately calculated, the processing efficiency of generating the target actual resource cost corresponding to the downstream dependent job is effectively improved, and the accuracy of the generated target actual resource cost corresponding to the downstream dependent job is ensured.

Drawings

For a clearer description of the solution in the present application, a brief description will be given below of the drawings that are needed in the description of the embodiments of the present application, it being obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of a method of processing resource data according to the present application;

FIG. 3 is a schematic diagram of one embodiment of a processing apparatus for resource data according to the present application;

FIG. 4 is a schematic structural diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

In order to better understand the technical solutions of the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings.

As shown in fig. 1, a system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the

terminal devices

101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the

terminal devices

101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the

terminal devices

101, 102, 103.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the

terminal devices

101, 102, 103.

It should be noted that, the processing method of the resource data provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the processing apparatus of the resource data is generally disposed in the server/terminal device.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow chart of one embodiment of a method of processing resource data according to the present application is shown. The processing method of the resource data comprises the following steps:

Step S201, the current resource consumption duration of each job is obtained from the pre-stored job log.

In this embodiment, the electronic device (for example, the server/terminal device shown in fig. 1) on which the processing method of the resource data operates may acquire the above-mentioned resource consumption duration through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection may include, but is not limited to, 3G/4G/5G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection. The pre-stored job log is a job log of a data warehouse (e.g., hadoop). The resource consumption duration of each job may refer to the CPU consumption duration of the job, and the unit may be hours. The resource cost of the 1, hadoop calculation job is mainly CPU, so CPU consumption time of the current job is collected from the job log, if one job has a plurality of SQL, CPU consumption time of a plurality of SQL is collected and summed to obtain CPU consumption time of the job.

Step S202, collecting the dependency relationship of each job, dividing the job into a public job and a downstream dependent job based on the dependency relationship, and determining a target job from the public job; wherein the number of target jobs includes a plurality.

In the present embodiment, the dependency relationship of each of the jobs may be collected by analyzing the job information for each of the jobs currently included in the job log. Wherein the common job may include a job that is dependent on a downstream dependent job, i.e., the target job. Downstream dependent jobs may also be referred to as downstream jobs.

Step S203, calculating an actual resource cost of the target job based on the resource consumption time period, and calculating a resource consumption weight of the target job based on the actual resource cost.

In this embodiment, the specific implementation process of calculating the actual resource cost of the target job based on the resource consumption duration and calculating the resource consumption weight of the target job based on the actual resource cost is described in further detail in the following specific embodiments, which will not be described herein.

Step S204, obtaining the number of times the target job is depended on.

In the present embodiment, the number of times the target job is relied on with respect to the downstream dependent job can be obtained by a dependency relationship between the target job and the downstream dependent job.

Step S205, calculating an allocation cost of the target job based on the number of dependences, the actual resource cost, and the resource consumption weight.

In this embodiment, the foregoing specific implementation process of calculating the allocation cost of the target job based on the number of depended times, the actual resource cost and the resource consumption weight will be described in further detail in the following specific embodiments, which will not be described herein.

Step S206, calculating the target actual resource cost of the downstream dependent job based on the apportionment cost.

In this embodiment, the foregoing specific implementation process of calculating the target actual resource cost of the downstream dependent job based on the allocation cost will be described in further detail in the following specific embodiments, which will not be described herein.

Firstly, acquiring the current resource consumption duration of each job from a prestored job log; then collecting the dependency relationship of each job, dividing the jobs into public jobs and downstream dependent jobs based on the dependency relationship, and determining a target job from the public jobs; wherein the number of target jobs includes a plurality; calculating the actual resource cost of the target job based on the resource consumption time length, and calculating the resource consumption weight of the target job based on the actual resource cost; subsequently obtaining the depended times of the target operation; calculating the allocation cost of the target job based on the relied times, the actual resource cost and the resource consumption weight; and finally, calculating the target actual resource cost of the downstream dependent job based on the apportionment cost. According to the method and the device for calculating the target actual resource cost, the obtained dependence relationship between the resource consumption duration of each job and each job is subjected to corresponding calculation processing, so that the target actual resource cost corresponding to the downstream dependence job can be automatically, quickly and accurately calculated, the processing efficiency of generating the target actual resource cost corresponding to the downstream dependence job is effectively improved, and the accuracy of the generated target actual resource cost corresponding to the downstream dependence job is ensured.

In some alternative implementations, calculating the actual resource cost of the target job based on the resource consumption duration in step S203 includes the steps of:

acquiring the appointed resource consumption duration of all jobs with upstream dependency relationship with the first target job based on the resource consumption duration; the first target job is any one job among all the target jobs.

In this embodiment, for example, if the first target job is a dws_ply job, and all jobs having an upstream dependency relationship with the dws_ply job are dwd _ply job, dwd _his job, ods_save job, ods_ply job, and ods_his job, respectively, dm_ply job and dm_his job are downstream dependency jobs, dwd _ply job, dwd _his job are relied on by the downstream dependency job, so that dwd _ply job, dwd _his job belong to the target job, i.e., the link end table of the common job, dwd _ply job has a relied number of times 2 corresponding to the downstream dependency job, and dwd_his job has a relied number of times 2 corresponding to the downstream dependency job. And the resource consumption time periods of dws_ply job, dwd _ply job, dwd _his job, ods_sale job, ods_ply job, and ods_his job are 20, 15, 15, 10, 10, 10 (units are hours), respectively. In the operation link, dwd _ply operation comprises ods_save operation and ods_ply operation, namely parent tasks of the ods_save operation and the ods_ply operation; the dwd _his job includes the ods_ply job and the ods_his job, namely parent tasks of the ods_ply job and the ods_his job. For a dws_ply job, the specified resource consumption duration for all jobs having an upstream dependency with the dws_ply job includes: 20, 15, 15, 10, 10, 10. Referring to a determination manner of a specified resource consumption duration of the dws_ply job, the specified resource consumption durations of all jobs available to have an upstream dependency relationship with the dwd _his job include: 15, 10, 10.

And calculating a first sum of all the specified resource consumption time periods.

In this embodiment, for example, for a dws_ply job, the specified resource consumption duration of all jobs having an upstream dependency relationship with the dws_ply job includes: 20 15, 15, 10, 10, 10, then the first real resource cost of dws_ply job = 20+15+15+10+10+10=80. The actual resource cost of dwd _his=15+10+10=35 can be found by referring to the calculation method of the first actual resource cost of the first target job.

Acquiring appointed resource consumption duration of all jobs with upstream dependency relationship with a first target job based on the resource consumption duration; wherein the first target job is any one job among all the target jobs; then calculating a first sum of all the specified resource consumption time periods; and further taking the first sum value as a first actual resource cost of the first target job. According to the method and the device, the obtained appointed resource consumption time length of all the jobs with the upstream dependency relationship with the first target job is summed, so that the actual resource cost of the target job can be rapidly and accurately calculated, the processing efficiency and accuracy of generating the actual resource cost of the target job are ensured, and the resource consumption weight of the target job can be rapidly calculated based on the obtained actual resource cost.

In some optional implementations of the present embodiment, calculating the resource consumption weight of the target job based on the actual resource cost in step S203 includes the steps of:

acquiring a second actual resource cost of a second target job; the second target job is any one job among all the target jobs.

In this embodiment, the specific process of calculating the second actual resource cost of the second target job may refer to the foregoing process of calculating the first actual resource cost of the first target job, which is not described herein.

And calculating a second sum of the actual resource costs of all the target jobs to obtain the total resource cost of the job links corresponding to all the jobs.

In the present embodiment, according to the foregoing example, if the actual resource cost of the target job dwd _ply job is 80 and the actual resource cost of the dwd _his job is 35, the total resource cost of the job links corresponding to all the jobs=80+35=115.

And calculating the quotient of the second actual resource cost and the total resource cost.

In this embodiment, for dwd _ply job, the resource consumption weight=80/115=69.6% of the dwd _ply job can be calculated. For dwd _his jobs, the resource consumption weight=35/115=30.4% of the dwd _his job can be calculated.

The method comprises the steps of obtaining a second actual resource cost of a second target job; then calculating a second sum of the actual resource costs of all the target jobs to obtain the total resource cost of the job links corresponding to all the jobs; and further calculating a quotient of the second actual resource cost and the total resource cost, and taking the quotient as a first resource consumption weight of the second target job. According to the method and the device, the obtained second actual resource cost of the second target job and the total resource cost of the job links corresponding to all the jobs are calculated, so that the resource consumption weight of the target job can be quickly and accurately calculated, the processing efficiency and the accuracy of generating the resource consumption weight of the target job are ensured, and the subsequent calculation of the allocation cost of the target job based on the obtained actual resource cost is facilitated.

In some alternative implementations, step S205 includes the steps of:

acquiring a second resource consumption weight of a third target job; wherein the third target job is any one job among all the target jobs.

In this embodiment, the specific process of calculating the second resource consumption weight of the third target job may refer to the foregoing process of calculating the first resource consumption weight of the second target job, which will not be described herein.

And acquiring summarized actual resource cost corresponding to the public job.

In this embodiment, the aggregate actual resource cost includes a sum of resource consumption durations of all jobs. Referring to the example in step S203, a total actual resource cost=20+15+15+10+10+10=80 corresponding to the common job can be calculated.

And calculating the product of the summarized actual resource cost and the second resource consumption weight.

And acquiring the number of times that the target corresponding to the third target job is relied on.

In this embodiment, the target dependent count corresponding to the third target job refers to the dependent count of the third target job with respect to the downstream dependent job.

In this embodiment, the following calculation method may be used: the product of the actual resource cost and the second resource consumption weight/the target dependent number is summarized to calculate the first apportionment cost of the third target job. For example, if the total actual resource cost corresponding to the common job is 80, the second resource consumption weight of the third target job is 69.6%, and the number of times the target corresponding to the third target job is relied on is 2, the first allocation cost of the third target job=80×69.6%/2=27.8.

The method comprises the steps of obtaining a second resource consumption weight of a third target job; acquiring summarized actual resource cost corresponding to the public operation; then calculating the product of the summarized actual resource cost and the second resource consumption weight; subsequently obtaining the dependent times of the target corresponding to the third target operation; and finally, calculating to obtain the first allocation cost of the third target operation based on the number of times the target is relied and the product. According to the method and the device, the obtained resource consumption weight of the target job and the summarized actual resource cost corresponding to the public job are calculated, so that the allocation cost of the target job can be quickly and accurately calculated, the processing efficiency and the accuracy of generating the allocation cost of the target job are ensured, and the target actual resource cost of the downstream dependent job can be quickly calculated based on the obtained allocation cost.

In some alternative implementations, step S206 includes the steps of:

acquiring self resource cost of the appointed downstream dependent operation; wherein the specified downstream dependent job is any one of all the downstream dependent jobs.

In this embodiment, the self-resource cost of the downstream dependent job may be queried from the pre-stored job log, and the self-resource cost of the downstream dependent job may also refer to the resource consumption duration of the downstream dependent job.

A fourth target job having a dependency relationship with the specified downstream dependent job is determined.

In this embodiment, the links of all jobs may be analyzed to determine a fourth target job having a dependency relationship with the specified downstream dependent job

And acquiring a second allocation cost corresponding to the fourth target job.

In this embodiment, the specific process of calculating the second split cost of the fourth target job may refer to the foregoing process of calculating the first split cost of the third target job, which will not be described herein.

In this embodiment, the own resource cost and the second allocation cost may be summed, and the resulting sum value may be used as the target actual resource cost corresponding to the specified downstream dependent job.

The method and the device acquire the self resource cost of the appointed downstream dependent operation; then determining a fourth target job having a dependency relationship with the specified downstream dependent job; then obtaining a second allocation cost corresponding to the fourth target operation; and calculating the target actual resource cost corresponding to the specified downstream dependent job based on the self resource cost and the second allocation cost. According to the method and the device for calculating the target actual resource cost of the downstream dependent operation, the obtained self resource cost of the downstream dependent operation and the apportioned cost of the fourth target operation with the dependent relation of the appointed downstream dependent operation are calculated, so that the target actual resource cost corresponding to the downstream dependent operation can be calculated rapidly and accurately, and the processing efficiency and the accuracy of generating the target actual resource cost corresponding to the downstream dependent operation are guaranteed.

In some optional implementations of this embodiment, after step S206, the electronic device may further perform the following steps:

judging whether the owner information corresponding to the appointed operation exists or not; wherein the specified job is any one of all the jobs.

In this embodiment, by calling a preset data mapping table, the job information of the specified job is used to query the data mapping table, so as to determine whether the data mapping table has the same target job information as the job information; if so, the owner information corresponding to the target job information is obtained from the data mapping table. The data mapping table is a database which is constructed in advance according to actual service use requirements and stores a plurality of pieces of operation information and owner information corresponding to the operation information one by one. The owner information may include I T packets/items/services, etc.

And if the owner information corresponding to the specified job exists, acquiring the specified actual resource cost corresponding to the specified job.

And summarizing the appointed actual resource cost to the owner corresponding to the owner information.

In this embodiment, the preset packet may specifically be a I T packet. When the owner information corresponding to the specified job does not exist, the specified actual resource cost of the specified job is intelligently allocated to I T groups, and then the allocation is further arranged by each group.

Judging whether the owner information corresponding to the appointed operation exists or not; if the owner information corresponding to the appointed operation exists, acquiring appointed actual resource cost corresponding to the appointed operation, and summarizing the appointed actual resource cost to an owner corresponding to the owner information; and if the owner information corresponding to the appointed operation does not exist, the appointed actual resource cost is distributed to a preset group. According to the method and the device, the actual resource cost of the specified operation is collected based on the owner information of the specified operation, so that the actual resource cost of the specified operation can be accurately collected into the corresponding business operation work, and the method and the device are beneficial to analyzing input and output more clearly and accurately, and therefore business operation and decision making are better supported.

and acquiring a target data type corresponding to the target actual resource cost.

In this embodiment, the target data type is a resource cost type.

And determining a data storage mode corresponding to the target data type.

In this embodiment, a preset importance database may be accessed first, and a target data importance corresponding to a target data type may be queried from the importance database, so as to determine a data storage mode corresponding to the target data type based on a correspondence between the data importance and the storage mode. The importance database is a database which is built in advance and stores various data types and data importance corresponding to the various data types one by one. In addition, the correspondence between the importance of the data and the storage mode may include: the storage mode corresponding to the low data importance degree is local storage, the storage mode corresponding to the medium data importance degree is cloud storage, and the storage mode corresponding to the low data importance degree is blockchain storage.

And storing the target actual resource cost based on the data storage mode.

The method comprises the steps of obtaining a target data type corresponding to the target actual resource cost; then determining a data storage mode corresponding to the target data type; and further storing the target actual resource cost based on the data storage mode. According to the method and the device, the target actual resource cost is stored by correspondingly selecting the data storage mode corresponding to the target data type according to the target data type corresponding to the target actual resource cost, so that the storage standardization and the storage intelligence of the target actual resource cost of the downstream dependent operation are effectively improved.

It is emphasized that, to further guarantee the privacy and security of the target actual resource costs, the target actual resource costs may also be stored in a node of a blockchain.

The blockchain referred to in the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by computer readable instructions stored in a computer readable storage medium that, when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a device for processing resource data, where an embodiment of the device corresponds to the embodiment of the method shown in fig. 2, and the device may be specifically applied to various electronic devices.

As shown in fig. 3, the processing apparatus 300 for resource data according to the present embodiment includes: a first acquisition module 301, a first processing module 302, a first calculation module 303, a second acquisition module 304, a second calculation module 305, and a third calculation module 306. Wherein:

A first obtaining module 301, configured to obtain a current resource consumption duration of each job from a pre-stored job log;

a first processing module 302, configured to collect a dependency relationship of each job, divide the job into a public job and a downstream dependency job based on the dependency relationship, and determine a target job from the public job; wherein the number of target jobs includes a plurality;

a first calculation module 303, configured to calculate an actual resource cost of the target job based on the resource consumption duration, and calculate a resource consumption weight of the target job based on the actual resource cost;

a second obtaining module 304, configured to obtain the number of depended times of the target job;

a second calculation module 305, configured to calculate an allocation cost of the target job based on the number of dependences, the actual resource cost, and the resource consumption weight;

a third calculation module 306 is configured to calculate a target actual resource cost of the downstream dependent job based on the apportionment cost.

In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the method for processing resource data in the foregoing embodiment one by one, which is not described herein again.

In some alternative implementations of the present embodiment, the first computing module 303 includes:

the first obtaining submodule is used for obtaining the appointed resource consumption duration of all jobs with upstream dependency relationship with the first target job based on the resource consumption duration; wherein the first target job is any one job among all the target jobs;

a first calculation sub-module, configured to calculate a first sum of all the specified resource consumption durations;

and the first determination submodule is used for taking the first sum value as a first actual resource cost of the first target job.

the second obtaining submodule is used for obtaining a second actual resource cost of a second target job; wherein the second target job is any one job among all the target jobs;

a second calculation sub-module, configured to calculate a second sum of actual resource costs of all the target jobs, to obtain a total resource cost of job links corresponding to all the jobs;

A third calculation sub-module, configured to calculate a quotient of the second actual resource cost and the total resource cost;

and the second determining submodule is used for taking the quotient as the first resource consumption weight of the second target job.

In some alternative implementations of the present embodiment, the second computing module 305 includes:

the third acquisition sub-module is used for acquiring a second resource consumption weight of a third target job; wherein the third target job is any one job among all the target jobs;

a fourth obtaining sub-module, configured to obtain summarized actual resource costs corresponding to the public jobs;

a fourth calculation sub-module, configured to calculate a product of the summarized actual resource cost and the second resource consumption weight;

a fifth obtaining sub-module, configured to obtain a target depended number corresponding to the third target job;

and a sixth calculation sub-module, configured to calculate, based on the number of times the target is relied and the product, a first allocation cost of the third target job.

In some alternative implementations of the present embodiment, the third computing module 306 includes:

a sixth acquisition sub-module, configured to acquire own resource costs of the specified downstream dependent jobs; wherein the specified downstream dependent job is any one of all the downstream dependent jobs;

a third determination submodule for determining a fourth target job having a dependency relationship with the specified downstream dependent job;

a seventh obtaining sub-module, configured to obtain a second allocation cost corresponding to the fourth target job;

and a seventh calculation sub-module, configured to calculate, based on the own resource cost and the second allocation cost, a target actual resource cost corresponding to the specified downstream dependent job.

In some optional implementations of this embodiment, the processing apparatus of resource data further includes:

The judging module is used for judging whether the owner information corresponding to the appointed operation exists or not; wherein the specified job is any one of all the jobs;

a third obtaining module, configured to obtain a specified actual resource cost corresponding to the specified job if there is owner information corresponding to the specified job;

the second processing module is used for summarizing the appointed actual resource cost to the owner corresponding to the owner information;

and the third processing module is used for distributing the appointed actual resource cost to a preset group if the owner information corresponding to the appointed operation does not exist.

a fourth obtaining module, configured to obtain a target data type corresponding to the target actual resource cost;

the determining module is used for determining a data storage mode corresponding to the target data type;

and the storage module is used for storing the target actual resource cost based on the data storage mode.

In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It should be noted that only computer device 4 having components 41-43 is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.

The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.

The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used to store an operating system installed on the computer device 4 and various application software, such as computer readable instructions of a processing method of resource data, and the like. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or process data, such as computer readable instructions for executing a processing method of the resource data.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.

in the embodiment of the application, firstly, the current resource consumption duration of each job is obtained from a pre-stored job log; then collecting the dependency relationship of each job, dividing the jobs into public jobs and downstream dependent jobs based on the dependency relationship, and determining a target job from the public jobs; wherein the number of target jobs includes a plurality; calculating the actual resource cost of the target job based on the resource consumption time length, and calculating the resource consumption weight of the target job based on the actual resource cost; subsequently obtaining the depended times of the target operation; calculating the allocation cost of the target job based on the relied times, the actual resource cost and the resource consumption weight; and finally, calculating the target actual resource cost of the downstream dependent job based on the apportionment cost. According to the method and the device for calculating the target actual resource cost, the obtained resource consumption time of each job and the dependency relationship of each job are subjected to corresponding calculation processing, so that the target actual resource cost corresponding to the downstream dependent job can be automatically, quickly and accurately calculated, the processing efficiency of generating the target actual resource cost corresponding to the downstream dependent job is effectively improved, and the accuracy of the generated target actual resource cost corresponding to the downstream dependent job is ensured.

The present application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of a method for processing resource data as described above.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.

It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.

Claims

1. A method for processing resource data, comprising the steps of:

acquiring the relied times of the target operation;

2. The method for processing resource data according to claim 1, wherein the step of calculating the actual resource cost of the target job based on the resource consumption time length specifically comprises:

Calculating a first sum of all the specified resource consumption time periods;

3. The method for processing resource data according to claim 1, wherein the step of calculating the resource consumption weight of the target job based on the actual resource cost specifically comprises:

4. The method according to claim 1, wherein the step of calculating the allocation cost of the target job based on the number of dependences, the actual resource cost, and the resource consumption weight, comprises:

Acquiring summarized actual resource cost corresponding to the public job;

5. The method for processing resource data according to claim 1, wherein the step of calculating the target actual resource cost of the downstream dependent job based on the allocation cost specifically comprises:

acquiring a second allocation cost corresponding to the fourth target job;

6. The method of processing resource data according to claim 1, further comprising, after the step of calculating the target actual resource cost of the downstream dependent job based on the apportionment cost:

7. The method of processing resource data according to claim 1, further comprising, after the step of calculating the target actual resource cost of the downstream dependent job based on the apportionment cost:

acquiring a target data type corresponding to the target actual resource cost;

determining a data storage mode corresponding to the target data type;

and storing the target actual resource cost based on the data storage mode.

8. A processing apparatus for resource data, comprising:

9. A computer device comprising a memory having stored therein computer readable instructions which when executed implement the steps of the method of processing resource data as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, wherein computer-readable instructions are stored on the computer-readable storage medium, which when executed by a processor, implement the steps of the method of processing resource data according to any one of claims 1 to 7.