CN111045825A - Batch processing performance optimization method and device, computer equipment and storage medium - Google Patents
Batch processing performance optimization method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111045825A CN111045825A CN201911274496.0A CN201911274496A CN111045825A CN 111045825 A CN111045825 A CN 111045825A CN 201911274496 A CN201911274496 A CN 201911274496A CN 111045825 A CN111045825 A CN 111045825A
- Authority
- CN
- China
- Prior art keywords
- task
- judged
- critical value
- smaller
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000005457 optimization Methods 0.000 title claims abstract description 30
- 238000004590 computer program Methods 0.000 claims description 19
- 238000010923 batch production Methods 0.000 claims description 11
- 238000004806 packaging method and process Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005538 encapsulation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention relates to a batch processing performance optimization method, a device, computer equipment and a storage medium, wherein the method comprises the steps of defining a task for storing a business data object to obtain a task to be judged; judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not; if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object; and assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value. The invention controls the thread number to achieve the best efficiency, realizes the optimization of batch processing performance and improves the efficiency of multithread processing data.
Description
Technical Field
The present invention relates to computer, and is especially batch processing performance optimizing method, apparatus, computer device and storage medium.
Background
The enterprise information system has important data batch processing services, for example, account checking is a typical batch processing service processing scene, and post-credit service data batch processing, that is, there are many tasks in the system to perform batch processing. With the increasing of service development data, the performance of task batch processing needs to be optimized urgently, the batch processing based on the spring batch framework provides parallel processing in a multi-thread mode, and the performance is improved.
The multithreading of the spring batch framework is actually to split each task job of batch processing, the split task job is to assemble a plurality of sub-steps corresponding to one execution context, the assembly process is to query all batch processing data of the corresponding task job and then assemble the batch processing data to the execution context, and then the multi-sub-steps are processed in a multithreading concurrent mode, but each task job needs to be developed by splitting logic codes, the encapsulation performance is not good enough, and the splitting and assembling of the job and the execution context also needs a certain processing time, so that the efficiency is low.
Therefore, it is necessary to design a new method to optimize the batch processing performance and improve the efficiency of multithread processing data.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a batch processing performance optimization method, a batch processing performance optimization device, a computer device and a storage medium.
In order to achieve the purpose, the invention adopts the following technical scheme: the batch processing performance optimization method comprises the following steps:
defining a task for storing a business data object to obtain a task to be judged;
judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not;
if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object;
and assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
The further technical scheme is as follows: after judging whether the content capacity stored in the task to be judged is smaller than a preset critical value, the method further comprises the following steps:
and if the capacity of the array to be judged is smaller than a preset critical value, executing the content stored in the task to be judged.
The further technical scheme is as follows: the content stored in the task to be judged comprises a service data object or a service data sub-object.
The further technical scheme is as follows: the defining the task of storing the business data object to obtain the task to be judged comprises the following steps:
customizing a task;
and encapsulating the task to be judged, wherein the generic object array is used for storing the business data object to obtain the task to be judged.
The further technical scheme is as follows: the preset critical value is set by the number of the business data objects and the processing time.
The invention also provides a batch processing performance optimization device, which comprises:
the defining unit is used for defining the task of the business data object to be stored so as to obtain the task to be judged;
the judging unit is used for judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not;
the segmentation unit is used for segmenting the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object if the content capacity stored in the task to be judged is not smaller than a preset critical value;
and the assembling unit is used for assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
The further technical scheme is as follows: further comprising:
and the execution unit is used for executing the content stored in the task to be judged if the capacity of the array to be judged is smaller than a preset critical value.
The further technical scheme is as follows: the definition unit includes:
the self-defining subunit is used for self-defining the task;
and the packaging subunit is used for packaging the task and storing the generic object array of the business data object to obtain the task to be judged.
The invention also provides computer equipment which comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the method when executing the computer program.
The invention also provides a storage medium storing a computer program which, when executed by a processor, is operable to carry out the method as described above.
Compared with the prior art, the invention has the beneficial effects that: the invention judges the size of the business data object in the task to be judged by setting the task to be judged and the preset critical value, when the size is not less than the preset critical value, the object is split and assembled into a plurality of tasks to form a plurality of tasks to be judged, and the plurality of tasks to be judged are processed in parallel, so that the optimal efficiency is achieved by controlling the thread number, the batch processing performance is optimized, and the efficiency of processing data by multiple threads is improved.
The invention is further described below with reference to the accompanying drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of an application scenario of a batch processing performance optimization method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating a batch process performance optimization method according to an embodiment of the present invention;
FIG. 3 is a sub-flow diagram of a batch process performance optimization method according to an embodiment of the present invention;
FIG. 4 is a schematic block diagram of a batch process performance optimization apparatus provided by an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a defining unit of a batch process performance optimizing apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a batch process performance optimization method according to an embodiment of the present invention. FIG. 2 is a schematic flow chart of a batch process performance optimization method according to an embodiment of the present invention. The batch processing performance optimization method is applied to a server. The server performs data interaction with a plurality of actuators, and the actuators perform tasks after the server performs batch processing.
FIG. 2 is a flowchart illustrating a batch process performance optimization method according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S150.
And S110, defining a task for storing the business data object to obtain a task to be judged.
In this embodiment, the task to be determined refers to a task for performing business data object processing.
In an embodiment, referring to fig. 3, the step S110 may include steps S111 to S112.
And S111, customizing the task.
In this embodiment, the custom task is for storing business data objects.
And S112, encapsulating the task and storing the generic object array of the business data object to obtain the task to be judged.
The Task to be judged is to realize a user-defined Task and encapsulate the Task into a generic object array to support different business data objects without complex encapsulation and excessive code development.
And S120, judging whether the content capacity stored in the task to be judged is smaller than a preset critical value.
In this embodiment, the preset threshold is set by the number of the service data objects and the processing time.
The preset critical value is used for defining whether the current task belongs to a large task and whether the current task needs to be further split so as to achieve the optimal efficiency of the control thread number.
In this embodiment, a large task is split into a plurality of small tasks, and when the large task is not split, that is, the capacity of the small task has reached a preset critical value, at this time, the split small tasks can be processed in parallel, and then the processing results are summarized, so that the recursion of the tasks is realized, and the purpose of controlling the thread number to achieve the optimal efficiency is achieved.
S130, if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object.
In this embodiment, the content stored in the task to be determined includes a service data object or a service data sub-object.
Judging that the content capacity stored by the task to be judged is smaller than a preset critical value, if the content capacity stored by the task to be judged is smaller than the preset critical value, executing business logic, if the content capacity stored by the task to be judged is not smaller than the preset critical value, splitting the task to be judged into 2 or more tasks, packaging the split subtasks into the 2 tasks to be judged or the multiple tasks to be judged, and then executing all newly-built tasks to be judged, namely, the task to be judged at the moment is not a single task, but at least two tasks to be judged are executed in parallel.
S140, assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and judging whether the content capacity stored in the task to be judged is smaller than a preset critical value.
After the split task to be judged is executed, capacity judgment is needed in each execution process, the actuator is driven to execute the content stored in the task to be judged only under the condition that the content capacity stored in the current task to be judged is smaller than a preset critical value, and otherwise, the next round of task splitting is carried out. When the tasks to be judged are processed in parallel, all the tasks to be judged are processed in a multithread manner, CPU resources are fully utilized, the performance is greatly improved, and the effect is more obvious under the condition of a multi-core server, so that the batch processing performance is optimized, and the data processing efficiency of the multithread is improved.
S150, if the capacity of the array to be judged is smaller than a preset critical value, executing the content stored in the task to be judged.
The preset critical value is self-defined, the preset critical value is a judgment basis for judging whether the task to be judged is split, for example, the critical value is 1000, the method is executed from the task to be judged, whether the service data object, namely the object array, processed in batch by the task to be judged is smaller than 1000, if the service data object processed in batch by the task to be judged is not smaller than 1000, the condition is met, the service logic is executed, and the executor executes the task to be judged.
When the service data object to be batched under the task to be judged is not less than 1000, the service data object is split into a plurality of service data sub-objects and then respectively assembled into a plurality of tasks to be judged, the plurality of tasks to be judged are executed concurrently, when each task to be judged is executed, the service data sub-object corresponding to the task to be judged is judged with a critical value, recursive processing is performed, the optimization of a JDK-based toolkit ForkJoinTask on batch processing is realized, the reusability is good, all batch processing task operations can be directly called, only a few lines of codes need to be written, and multithreading is performed by extra operation on multithreading of a spring batch frame, which relates to the assembly of some tables of the spring batch; the method is realized based on forkJoin, multithreading processing data is directly carried out without extra operation, and the performance is high.
In this embodiment, the threshold value is freely configured according to the data size and processing time of the batch processing, so as to control the thread number to achieve the best efficiency.
According to the batch processing performance optimization method, the size of the business data object in the task to be judged is judged by setting the task to be judged and the preset critical value, when the size is not smaller than the preset critical value, the object is split and assembled into a plurality of tasks to form a plurality of tasks to be judged, the plurality of tasks to be judged are processed in parallel, the optimal efficiency is achieved by controlling the thread number, the batch processing performance is optimized, and the data multithreading efficiency is improved.
Fig. 4 is a schematic block diagram of a batch performance optimization apparatus 300 according to an embodiment of the present invention. As shown in fig. 4, the present invention also provides a batch process performance optimization apparatus 300 corresponding to the above batch process performance optimization method. The batch performance optimizing apparatus 300 includes a unit for performing the above-described batch performance optimizing method, and the apparatus may be configured in a server. Specifically, referring to fig. 4, the batch process performance optimization apparatus 300 includes a defining unit 301, a determining unit 302, a dividing unit 303, and an assembling unit 304.
A defining unit 301, configured to define a task for storing a service data object, so as to obtain a task to be determined; a judging unit 302, configured to judge whether content capacity stored in the task to be determined is smaller than a preset critical value; a dividing unit 303, configured to divide the content stored in the task to be determined into a plurality of service data sub-objects to obtain a target data object if the content capacity stored in the task to be determined is not less than a preset critical value; an assembling unit 304, configured to assemble a target data object into a plurality of tasks, update the task to be determined as the task, and execute the determination of whether content capacity stored in the task to be determined is smaller than a preset critical value.
In one embodiment, as shown in fig. 4, the apparatus further comprises:
an executing unit 305, configured to execute the content stored in the task to be determined if the capacity of the array to be determined is smaller than a preset threshold.
In one embodiment, as shown in fig. 5, the definition unit 301 includes a self-defining subunit 3011 and an encapsulating subunit 3012.
A custom subunit 3011 for custom tasks; and the encapsulating subunit 3012 is configured to encapsulate the task, and is configured to store a generic object array of the business data object, so as to obtain a task to be determined.
It should be noted that, as can be clearly understood by those skilled in the art, the detailed implementation process of the batch performance optimization apparatus 300 and each unit may refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, no further description is provided herein.
The batch property optimizing device 300 may be implemented in the form of a computer program that can be run on a computer apparatus as shown in fig. 6.
Referring to fig. 6, fig. 6 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 is a server, among other things. The server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 6, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 comprises program instructions that, when executed, cause the processor 502 to perform a batch performance optimization method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be caused to perform a batch performance optimization method.
The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the configuration shown in fig. 6 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation of the computer device 500 to which the present application may be applied, and that a particular computer device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:
defining a task for storing a business data object to obtain a task to be judged; judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not; if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object; and assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
The content stored in the task to be judged comprises a service data object or a service data sub-object.
The preset critical value is set by the number of the business data objects and the processing time.
In an embodiment, after the step of determining whether the content capacity stored in the task to be determined is smaller than the preset threshold value is implemented, the processor 502 further implements the following steps:
and if the capacity of the array to be judged is smaller than a preset critical value, executing the content stored in the task to be judged.
In an embodiment, when the processor 502 implements the task of defining the storage service data object to obtain the task step to be determined, the following steps are specifically implemented:
customizing a task; and encapsulating the task to be judged, wherein the generic object array is used for storing the business data object to obtain the task to be judged.
It should be understood that, in the embodiment of the present Application, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be understood by those skilled in the art that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program instructing associated hardware. The computer program includes program instructions, and the computer program may be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the steps of:
defining a task for storing a business data object to obtain a task to be judged; judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not; if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object; and assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
The content stored in the task to be judged comprises a service data object or a service data sub-object.
The preset critical value is set by the number of the business data objects and the processing time.
In an embodiment, after the step of determining whether the content capacity stored in the task to be determined is smaller than a preset threshold value is implemented by executing the computer program, the processor further implements the following steps:
and if the capacity of the array to be judged is smaller than a preset critical value, executing the content stored in the task to be judged.
In an embodiment, when the processor executes the computer program to implement the task of defining the storage service data object to obtain the task to be determined, the following steps are specifically implemented:
customizing a task; and encapsulating the task to be judged, wherein the generic object array is used for storing the business data object to obtain the task to be judged.
The storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, which can store various computer readable storage media.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be merged, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. The batch processing performance optimization method is characterized by comprising the following steps:
defining a task for storing a business data object to obtain a task to be judged;
judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not;
if the content capacity stored in the task to be judged is not smaller than a preset critical value, dividing the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object;
and assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
2. The batch processing performance optimizing method according to claim 1, wherein after determining whether the content capacity stored in the task to be determined is smaller than a preset critical value, the method further comprises:
and if the capacity of the array to be judged is smaller than a preset critical value, executing the content stored in the task to be judged.
3. The batch performance optimization method of claim 1, wherein the content stored within the task to be determined comprises a business data object or a business data sub-object.
4. The batch performance optimization method of claim 1, wherein the defining the task of storing the business data object to obtain the task to be determined comprises:
customizing a task;
and encapsulating the task to be judged, wherein the generic object array is used for storing the business data object to obtain the task to be judged.
5. The batch processing performance optimization method according to claim 1, wherein the preset threshold is set by the number of the business data objects and the processing time.
6. A batch process performance optimization apparatus, comprising:
the defining unit is used for defining the task of the business data object to be stored so as to obtain the task to be judged;
the judging unit is used for judging whether the content capacity stored in the task to be judged is smaller than a preset critical value or not;
the segmentation unit is used for segmenting the content stored in the task to be judged into a plurality of service data sub-objects to obtain a target data object if the content capacity stored in the task to be judged is not smaller than a preset critical value;
and the assembling unit is used for assembling the target data object into a plurality of tasks, updating the task to be judged into the task, and executing the judgment of whether the content capacity stored in the task to be judged is smaller than a preset critical value.
7. The batch processing performance optimizing apparatus of claim 6, further comprising:
and the execution unit is used for executing the content stored in the task to be judged if the capacity of the array to be judged is smaller than a preset critical value.
8. The batch processing performance optimizing apparatus according to claim 7, wherein the defining unit includes:
the self-defining subunit is used for self-defining the task;
and the packaging subunit is used for packaging the task and storing the generic object array of the business data object to obtain the task to be judged.
9. A computer arrangement, characterized in that the computer arrangement comprises a memory having stored thereon a computer program and a processor implementing the method according to any of claims 1-5 when executing the computer program.
10. A storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911274496.0A CN111045825A (en) | 2019-12-12 | 2019-12-12 | Batch processing performance optimization method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911274496.0A CN111045825A (en) | 2019-12-12 | 2019-12-12 | Batch processing performance optimization method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111045825A true CN111045825A (en) | 2020-04-21 |
Family
ID=70236408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911274496.0A Pending CN111045825A (en) | 2019-12-12 | 2019-12-12 | Batch processing performance optimization method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111045825A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112256931A (en) * | 2020-10-26 | 2021-01-22 | 北京神州数字科技有限公司 | Data extraction method |
| CN113760520A (en) * | 2020-07-09 | 2021-12-07 | 西安京迅递供应链科技有限公司 | Task processing method and device |
| CN113780903A (en) * | 2021-11-12 | 2021-12-10 | 中国信息通信研究院 | Asset object management method and device, electronic equipment and computer storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013107819A1 (en) * | 2012-01-17 | 2013-07-25 | Thales | Method for optimising the parallel processing of data on a hardware platform |
| US20130297788A1 (en) * | 2011-03-30 | 2013-11-07 | Hitachi, Ltd. | Computer system and data management method |
| CN107689010A (en) * | 2017-07-25 | 2018-02-13 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of batch processing declaration form task |
| US20190036846A1 (en) * | 2017-07-31 | 2019-01-31 | Siemens Aktiengesellschaft | Method and system for uploading data to cloud platform, gateway, and machine-readable medium |
-
2019
- 2019-12-12 CN CN201911274496.0A patent/CN111045825A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130297788A1 (en) * | 2011-03-30 | 2013-11-07 | Hitachi, Ltd. | Computer system and data management method |
| WO2013107819A1 (en) * | 2012-01-17 | 2013-07-25 | Thales | Method for optimising the parallel processing of data on a hardware platform |
| CN107689010A (en) * | 2017-07-25 | 2018-02-13 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of batch processing declaration form task |
| US20190036846A1 (en) * | 2017-07-31 | 2019-01-31 | Siemens Aktiengesellschaft | Method and system for uploading data to cloud platform, gateway, and machine-readable medium |
Non-Patent Citations (4)
| Title |
|---|
| 网页发布者: "ForkJoin之ForkJoinTask框架学习笔记", pages 1 - 10, Retrieved from the Internet <URL:https://www.cnblogs.com/coloz/p/11167883.html> * |
| 网页发布者: "ForkJoin框架（二）：ForkJoinTask", pages 1 - 2, Retrieved from the Internet <URL:https://blog.csdn.net/luanmousheng/article/details/78173342> * |
| 网页发布者: "JAVA并行计算之ForkJoinTask使用样例", pages 1 - 2, Retrieved from the Internet <URL:https://blog.csdn.net/puhaiyang/article/details/77581296> * |
| 青岛东合信息技术有限公司编著: "《Java SE程序设计》", vol. 1, 中国商务出版社, pages: 140 - 142 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760520A (en) * | 2020-07-09 | 2021-12-07 | 西安京迅递供应链科技有限公司 | Task processing method and device |
| CN112256931A (en) * | 2020-10-26 | 2021-01-22 | 北京神州数字科技有限公司 | Data extraction method |
| CN113780903A (en) * | 2021-11-12 | 2021-12-10 | 中国信息通信研究院 | Asset object management method and device, electronic equipment and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111045825A (en) | Batch processing performance optimization method and device, computer equipment and storage medium | |
| CN108829521B (en) | Task processing method and device, computer equipment and storage medium | |
| US10963285B2 (en) | Resource management for virtual machines in cloud computing systems | |
| US7302686B2 (en) | Task management system | |
| US8635405B2 (en) | Computational resource assignment device, computational resource assignment method and computational resource assignment program | |
| US20090037512A1 (en) | Multi-nodal compression techniques for an in-memory database | |
| CN114490566B (en) | Cluster data migration method and device, computer equipment and storage medium | |
| CN112181522A (en) | Data processing method and device and electronic equipment | |
| CN111443959A (en) | Task execution method and device and server | |
| CN109669767B (en) | Task packaging and scheduling method and system for multi-type context dependence | |
| CN111338787A (en) | Data processing method and device, storage medium and electronic device | |
| CN108446300B (en) | Data information scanning method and device | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN113778518B (en) | Data processing method, device, computer equipment and storage medium | |
| CN110851166A (en) | User-unaware application program updating method and device and computer equipment | |
| CN116048756A (en) | Queue scheduling method and device and related equipment | |
| CN112612604B (en) | Task scheduling method and device based on Actor model | |
| CN113377360B (en) | Task execution method, device, electronic equipment, storage medium and program product | |
| JP7141939B2 (en) | industrial controller | |
| CN113886490A (en) | Method and system for realizing high availability of stateless computing instances in distributed database | |
| KR20180071056A (en) | Apparatus and method for dynamic frequency scaling, and recording medium | |
| CN107729772B (en) | Processor | |
| CN113032118A (en) | Asynchronous operation processing method for computer application program and corresponding system | |
| CN112612804A (en) | Service management parameter updating method and device | |
| WO2018230444A1 (en) | Accelerator control device, accelerator control method, and recording medium with accelerator control program stored therein |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |