CN112799813A - Distributed job service scheduling method, distributed server and service terminal - Google Patents
Distributed job service scheduling method, distributed server and service terminal Download PDFInfo
- Publication number
- CN112799813A CN112799813A CN202110117209.6A CN202110117209A CN112799813A CN 112799813 A CN112799813 A CN 112799813A CN 202110117209 A CN202110117209 A CN 202110117209A CN 112799813 A CN112799813 A CN 112799813A
- Authority
- CN
- China
- Prior art keywords
- service
- scheduling
- service scheduling
- message pipeline
- scheduling information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012545 processing Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 12
- 238000004140 cleaning Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 abstract description 56
- 230000008859 change Effects 0.000 abstract description 20
- 238000012986 modification Methods 0.000 abstract description 18
- 230000004048 modification Effects 0.000 abstract description 18
- 230000001360 synchronised effect Effects 0.000 abstract description 12
- 230000008878 coupling Effects 0.000 abstract description 10
- 238000010168 coupling process Methods 0.000 abstract description 10
- 238000005859 coupling reaction Methods 0.000 abstract description 10
- 230000010354 integration Effects 0.000 abstract description 10
- 238000010200 validation analysis Methods 0.000 abstract description 9
- 239000000969 carrier Substances 0.000 abstract description 8
- 238000004891 communication Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 12
- 238000012544 monitoring process Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 5
- 239000000872 buffer Substances 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 210000003205 muscle Anatomy 0.000 description 2
- 239000000725 suspension Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
Abstract
The distributed job service scheduling method, the distributed server and the service terminals can be used in the technical field of big data, firstly, the service scheduling information of each service terminal is obtained, and then the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
Description
Technical Field
The invention relates to the technical field of distributed processing, in particular to a distributed job service scheduling method, a distributed server and a service terminal.
Background
In conventional projects, a job scheduling framework is often coupled with a business system, collectively packaging release and deployment. Synchronous data acquisition among systems requires developers to coordinate service interfaces, changes of interface addresses and protocols cause injuries to bones and muscles, and data consistency cannot be guaranteed. Service functions are not modularized, and the dependence between services is highly coupled. Developers and operation and maintenance personnel do not have visual perception on the operation of the background of the system, the online modification, suspension and starting of operation scheduling cannot be realized, and the execution condition of batch tasks of the background cannot be controlled globally. There is no effective control means for risky background batch tasks.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide an open platform distributed job scheduling and monitoring system, which solves the problems. The invention aims to decouple a job scheduling framework and a business system and provide job scheduling service in a service mode. Flexible service requirements such as on-line modification and start-stop of operation tasks and modification and expansion of service functions are realized in a configuration mode. The continuous availability of the job scheduling service is ensured through a high-availability design, the access end can conveniently schedule the operation job, and a real-time operation result, a log and an analysis report are obtained.
The core concept of the invention is to provide a distributed job service scheduling method, a distributed server and service terminals, firstly, obtaining service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and dependency relations among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
In order to solve the technical problems, the invention provides the following technical scheme:
the embodiment of the invention provides a distributed job service scheduling method, which is executed by a distributed server and comprises the following steps:
acquiring service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and dependency relations among the tasks;
arranging the service scheduling information to a message pipeline;
and performing service scheduling on each service terminal according to the message pipeline.
In a preferred embodiment, further comprising:
and establishing the message pipeline.
In a preferred embodiment, the performing service scheduling on each service terminal according to the message pipeline includes:
scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
acquiring task execution condition information of each service terminal;
and if the task corresponding to the currently scheduled service is executed, cleaning the corresponding service scheduling information in the message pipeline.
The invention further provides a distributed job service scheduling method, which is executed by a service terminal and comprises the following steps:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
The present invention further provides a distributed server comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring service scheduling information of each service terminal, and the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks;
the arrangement module is used for arranging the service scheduling information to a message pipeline;
and the service scheduling module is used for scheduling the service of each service terminal according to the message pipeline.
In a preferred embodiment, further comprising:
and the establishing module is used for establishing the message pipeline.
In a preferred embodiment, the service scheduling module includes:
the service scheduling unit is used for scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
the task execution condition information acquisition unit is used for acquiring task execution condition information of each service terminal;
and the cleaning unit is used for cleaning the corresponding service scheduling information in the message pipeline if the task corresponding to the currently scheduled service is executed.
The present invention further provides a service terminal, specifically configured to:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
The invention further provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the distributed job service scheduling method when executing the program.
The invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the distributed job service scheduling method.
According to the technical scheme, the distributed job service scheduling method, the distributed server and the service terminals, provided by the invention, firstly obtain the service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a diagram of an embodiment of the present invention.
FIG. 2 is a high availability job service plan of the present invention.
FIG. 3 is a diagram of job scheduling operations of the present invention.
FIG. 4 is a flowchart of a distributed job service scheduling method according to the present invention.
Fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the prior art, in a traditional project, a job scheduling framework is often coupled with a business system to jointly package release and deployment. Synchronous data acquisition among systems requires developers to coordinate service interfaces, changes of interface addresses and protocols cause injuries to bones and muscles, and data consistency cannot be guaranteed. Service functions are not modularized, and the dependence between services is highly coupled. Developers and operation and maintenance personnel do not have visual perception on the operation of the background of the system, the online modification, suspension and starting of operation scheduling cannot be realized, and the execution condition of batch tasks of the background cannot be controlled globally. There is no effective control means for risky background batch tasks.
In order to solve the problems, the core concept of the invention is to provide a distributed job service scheduling method, a distributed server and service terminals, firstly, acquiring service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and dependency relations among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
From the foregoing, it can be seen that the present invention is directed to decoupling a job scheduling framework from a business system and providing job scheduling services in a servitization manner. Flexible service requirements such as on-line modification and start-stop of operation tasks and modification and expansion of service functions are realized in a configuration mode. The continuous availability of the job scheduling service is ensured through a high-availability design, the access end can conveniently schedule the operation job, and a real-time operation result, a log and an analysis report are obtained.
The above-described concept of the present invention will be described in detail with reference to the accompanying drawings.
In one or more embodiments of the invention, a distributed job service scheduling system includes a service terminal and a distributed server.
The distributed server is a distributed device and specifically comprises a plurality of distributed servers, wherein application programs are stored in the distributed servers, when the application programs are specifically executed, the service scheduling information of each service terminal is firstly acquired, and the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks; then, the service scheduling information is arranged to a message pipeline; and finally, performing service scheduling on each service terminal according to the message pipeline.
It is to be understood that the present invention is applicable to the field of big data technology, and of course, the present invention is not limited thereto.
The service terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), an in-vehicle device, an intelligent wearable device, and the like.
In an actual application situation, the intelligent wearable system may perform part of the distributed job service scheduling method in the distributed job service scheduling system, which is executed by the distributed server described in the above, or all operations may be completed in the service terminal. Specifically, the selection may be performed according to the processing capability of the service terminal, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. If all the operations are completed in the service terminal, the service terminal may further include a processor for performing specific processing of distributed job service scheduling.
The service terminal may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. For example, the communication unit may transmit traffic service scheduling information for the traffic to the server. The communication unit may also receive service scheduling information returned by the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
The server and the service terminal may communicate using any suitable network protocol, including a network protocol that has not been developed at the filing date of this application. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.
In an embodiment of the present invention, in the distributed job service scheduling method provided by the present invention, the distributed server specifically executes the following steps, as shown in fig. 4:
s100: and acquiring the service scheduling information of each service terminal.
Specifically, the service scheduling information includes a task corresponding to each service terminal and a dependency relationship between the tasks.
S200: arranging the service scheduling information to a message pipeline;
s300: and performing service scheduling on each service terminal according to the message pipeline.
According to the technical scheme, the distributed job service scheduling method provided by the invention comprises the steps of firstly obtaining the service scheduling information of each service terminal, wherein the service scheduling information comprises the task corresponding to each service terminal and the dependency relationship among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
In some embodiments, further comprising:
and establishing the message pipeline.
In some embodiments, the performing service scheduling on each service terminal according to the message pipeline includes:
scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
acquiring task execution condition information of each service terminal;
and if the task corresponding to the currently scheduled service is executed, cleaning the corresponding service scheduling information in the message pipeline.
The invention also provides a distributed job service scheduling method, which is executed by a service terminal and comprises the following steps:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
According to the technical scheme, the distributed job service scheduling method provided by the invention comprises the steps of firstly obtaining the service scheduling information of each service terminal, wherein the service scheduling information comprises the task corresponding to each service terminal and the dependency relationship among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
Specific examples are given below, and as shown in fig. 1 to fig. 3, the service system is divided into the following modules:
and the S100 message notification module provides functions of message notification, load balancing, route forwarding and the like for the service system.
S101, a front-end interface module displays health monitoring information, operation execution information and visual operation configuration of the service system, a task arrangement rule interface and a production line configuration interface.
And S102, the operation service module stores the service configuration information through the configuration information of the front-end service party, generates and manages a task arrangement rule, and monitors the smooth operation of a task pipeline.
S103 task pipeline module, through message driving mode, each business access side monitors corresponding event, actively obtains message, and issues own message, to ensure orderly execution of business function. The service function integration of the cross-system and the decoupling of data synchronization and data acquisition between the systems are realized.
And S104 transaction consistency control module for controlling data consistency of each service system access party and ensuring data consistency across systems.
S105, monitoring and collecting the operation executing information and the system health information for the front-end module to display.
1. Firstly, a service access party configures related information of an access party application and configuration information of timing operation through an S101 front-end interface, and draws a multitask coordination dependency relationship among operations in a flow chart mode. According to the configuration information, the arrangement rule automatically generates a flow chart, meanwhile, the flow chart can be modified or drawn to generate the configuration information and the arrangement rule, and the configuration information is fed back to the operation module for persistence. The addition and deletion of the service scheduling can be adjusted in a page visualization mode, and the operation service module reads the modification of the configuration information and can replan the scheduling rule to achieve the effect of real-time online validation. The method abandons the situation that the scheduling information in the traditional project is packaged and deployed together with the application in the form of a configuration file, and the application needs to be redeployed when the scheduling information is adjusted.
2. The operation service center is used as a core module of the scheduling system, manages and executes scheduling operation, and provides the following functions:
(1) the operation service module functions to manage the configuration information of the access party, arrange the rules, generate the message control series and generate and monitor the task pipeline. Through the xml data format, the front-back dependency relationship and the execution condition of each task on the management task pipeline are configured and persisted, the task execution process is controlled and monitored, and one task service manages and monitors one task pipeline.
(2) The task pipeline module function cooperates with the execution of each task through a message notification mechanism (asynchronization) to complete the pipeline function, as shown in fig. 3. The task is that the service system is used as a carrier for functional modularization of the service access party, and it can be understood that one task is a service functional module which is a minimum granularity unit. All configuration information of the service function on the system is contained, such as: data structure, format and type of access control, timing and scheduling time and mode, etc. One task flow line is an ordered combination of a plurality of system business functions which are cooperatively completed together. The task is assembled and disassembled in a pipeline mode through componentization to reduce the dependence of business functions and the coupling of all business parties, and the expansion and the change of business rules are adapted. In the traditional project, all business parties are coupled with third-party application in an interface calling mode, and the modification of the related interface needs synchronous shutdown deployment. As shown in fig. 3, a certain function X of the service party X needs to be completed by the cooperation of the functions of other service parties, that is, the functions of the service party A, B, C can be combined in a job scheduling manner without being separately developed, so that the function extension can be achieved. If the x function service rule changes on a certain day, the service function b is not needed, and the service function D is needed to be added, the function b can be removed through page pipeline configuration, the function D is added, namely, the function x can be expanded and modified through a pipeline which is effective again, and the application X, A, B, C does not need to be stopped and deployed, and only the configuration of the application D needs to be added. The flexibility and the expandability of the service system are improved.
(3) And the transaction control module controls and manages the full life cycle of the pipeline transaction depending on the monitoring information of the operation service module, and the transaction submission is divided into two stages. The method comprises the following steps:
acquiring a global transaction lock of a scheduling job;
the current business side submits local affairs and records rollback logs according to the execution condition of the current business side;
and acquiring an execution result of the whole task of the pipeline. If the log is successful, each service party of the assembly line is informed to clear the rollback log by itself; if the data is failed, all the service parties of the assembly line are informed to carry out compensation operation by themselves through the rollback log, and data rollback is completed. Thus, the final result consistency of each task data on the pipeline is achieved;
the transaction is completed and the global lock is released.
(4) Each business access side communicates with the operation service module through the message notification module, the communication form adopts RESTful interface style, and the data exchange format is JSON, as shown in figure 1.
And 3, the monitoring module generates a report for acquiring the health information and the operation execution information of the system in real time, and feeds the information back to the front-end module to form a visual interface for a user to check and analyze. The behavior of the monitoring module does not depend on the interaction process of the service party and the service party, and the monitoring module is independently operated as a background process.
4. In the structural design of the service center, the load balancing module, a certain operation service block and the log monitoring module are jointly registered in the registration center, so that service discovery and management are realized. The high availability of services is achieved by the imperceptibility of the application server. In data storage, high availability of data service is realized in a read-write separation mode through two main hot standby modes and one main slave mode and three slave modes.
It can be seen that the invention overcomes the inflexibility and unmonitorable process of the traditional project operation configuration. The method can quickly respond to the change of the service requirement and intuitively monitor the operation condition. Its advantages are as follows:
1. the new modification of the job scheduling is added, and the on-line effect can be achieved without modifying the application redeployment of the service party.
2. The expansion and change of the business rules of the business side only need to modify the assembly line, and the business side application does not need to be redeployed.
3. The task is used as a carrier to modularize the service function, the dependency relationship is completed in a pipeline mode, and the problem of operation execution with a front dependency relationship and a rear dependency relationship in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. And the expandability is provided for the business rule change and the business architecture upgrading of the business system.
4. The service party and the service party are decoupled, so that the service party can pay more attention to the realization of the service function.
5. The operation execution monitoring report and the index graph help the business side to more comprehensively expand, the execution condition of the operation is timed, and the optimization of the business function is promoted.
The present invention further provides a distributed server comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring service scheduling information of each service terminal, and the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks;
the arrangement module is used for arranging the service scheduling information to a message pipeline;
and the service scheduling module is used for scheduling the service of each service terminal according to the message pipeline.
In a preferred embodiment, further comprising:
and the establishing module is used for establishing the message pipeline.
In a preferred embodiment, the service scheduling module includes:
the service scheduling unit is used for scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
the task execution condition information acquisition unit is used for acquiring task execution condition information of each service terminal;
and the cleaning unit is used for cleaning the corresponding service scheduling information in the message pipeline if the task corresponding to the currently scheduled service is executed.
The present invention further provides a service terminal, specifically configured to:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
The distributed server and the service terminal provided by the invention firstly obtain the service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
In terms of hardware, the present invention provides an embodiment of an electronic device for implementing all or part of the contents in the distributed job service scheduling method, where the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission among related devices such as a distributed server, a distributed message middleware device, various databases, a user terminal and the like; the electronic device may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the electronic device may refer to the embodiment of the distributed job service scheduling method in the embodiment, and the contents thereof are incorporated herein, and repeated descriptions are omitted.
Fig. 5 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present invention. As shown in fig. 5, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this FIG. 5 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the distributed job service scheduling functionality may be integrated into the central processor 9100. For example, the central processor 9100 may be configured to control as follows:
s101: and acquiring the service scheduling information of each service terminal.
Specifically, the service scheduling information includes a task corresponding to each service terminal and a dependency relationship between the tasks.
S102: arranging the service scheduling information to a message pipeline;
s103: and performing service scheduling on each service terminal according to the message pipeline.
As can be seen from the above description, in the electronic device provided in the embodiment of the present invention, first, service scheduling information of each service terminal is obtained, where the service scheduling information includes a task corresponding to each service terminal and a dependency relationship between the tasks; then, the service scheduling information is arranged to a message pipeline; finally, service scheduling is carried out on each service terminal according to the message pipeline, and the method and the system can be seen in that the dependency relationship is completed in a pipeline mode by setting the message pipeline and using tasks as carriers to modularize service functions, so that the problem of job execution with front and back dependency relationships in the traditional project is solved. The coupling of synchronous data acquisition and function integration and disassembly among systems is reduced, and the data consistency is ensured. The method provides expandability for the service rule change and the service architecture upgrade of the service system, the modification of the job scheduling is newly increased, the online validation can be realized without modifying the application redeployment of the service party, and the expansion and the change of the service rule of the service party only need to modify the assembly line and do not need the application redeployment of the service party.
In another embodiment, the distributed job service scheduling apparatus may be configured separately from the central processor 9100, for example, the distributed job service scheduling apparatus may be configured as a chip connected to the central processor 9100, and the distributed job service scheduling function may be implemented by the control of the central processor.
As shown in fig. 5, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 5; further, the electronic device 9600 may further include components not shown in fig. 5, which may be referred to in the art.
As shown in fig. 5, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 may also include a distributed job service broker 9143, the distributed job service broker 9143 being configured to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the distributed job service scheduling method in which the execution subject in the above embodiment may be a distributed server, where the computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the computer program implements all the steps in the distributed job service scheduling method in which the execution subject in the above embodiment is a distributed server or a service terminal.
As can be seen from the above description, in the computer-readable storage medium provided in the embodiment of the present invention, whether the current SDK version is the latest version is determined according to the SDK version number, when the SDK is updated, only the incremental upgrade package is needed, and then the version is updated based on the bytecode modification in a manner of being loaded by a class loader, so that the heavy obstacle of pushing the service terminal to cooperate with the upgrade SDK can be alleviated to a great extent by performing the hot upgrade on the SDK, and the pain point that the SDK is difficult to update can be solved by performing the hot update on the quick iteration version without sensing the application.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A distributed job service scheduling method, performed by a distributed server, comprising:
acquiring service scheduling information of each service terminal, wherein the service scheduling information comprises tasks corresponding to each service terminal and dependency relations among the tasks;
arranging the service scheduling information to a message pipeline;
and performing service scheduling on each service terminal according to the message pipeline.
2. The distributed job service scheduling method according to claim 1, further comprising:
and establishing the message pipeline.
3. The distributed job service scheduling method according to claim 1, wherein the performing service scheduling on each service terminal according to the message pipeline includes:
scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
acquiring task execution condition information of each service terminal;
and if the task corresponding to the currently scheduled service is executed, cleaning the corresponding service scheduling information in the message pipeline.
4. A distributed job service scheduling method, characterized in that the method is executed by a service terminal, and comprises:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
5. A distributed server, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring service scheduling information of each service terminal, and the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks;
the arrangement module is used for arranging the service scheduling information to a message pipeline;
and the service scheduling module is used for scheduling the service of each service terminal according to the message pipeline.
6. The distributed server of claim 5, further comprising:
and the establishing module is used for establishing the message pipeline.
7. The distributed server of claim 5, wherein the service scheduling module comprises:
the service scheduling unit is used for scheduling corresponding services in the system one by one according to the sequence of each task in the message pipeline;
the task execution condition information acquisition unit is used for acquiring task execution condition information of each service terminal;
and the cleaning unit is used for cleaning the corresponding service scheduling information in the message pipeline if the task corresponding to the currently scheduled service is executed.
8. A service terminal, characterized in that it is specifically configured to:
and sending service scheduling information to a distributed server together with other service terminals so that the distributed server arranges the service scheduling information to a message pipeline and performs service scheduling on each service terminal according to the message pipeline, wherein the service scheduling information comprises tasks corresponding to each service terminal and the dependency relationship among the tasks.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the distributed job service scheduling method of any one of claims 1 to 4 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the distributed job service scheduling method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110117209.6A CN112799813A (en) | 2021-01-28 | 2021-01-28 | Distributed job service scheduling method, distributed server and service terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110117209.6A CN112799813A (en) | 2021-01-28 | 2021-01-28 | Distributed job service scheduling method, distributed server and service terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112799813A true CN112799813A (en) | 2021-05-14 |
Family
ID=75812461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110117209.6A Pending CN112799813A (en) | 2021-01-28 | 2021-01-28 | Distributed job service scheduling method, distributed server and service terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112799813A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240369A (en) * | 2021-12-17 | 2022-03-25 | 中国工商银行股份有限公司 | Pipeline deployment method and device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339222A (en) * | 2016-08-23 | 2017-01-18 | 联动优势电子商务有限公司 | Service implementing method and device |
| CN107493199A (en) * | 2017-08-31 | 2017-12-19 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies management method and system |
| CN111736969A (en) * | 2020-06-16 | 2020-10-02 | 中国银行股份有限公司 | Distributed job scheduling method and device |
-
2021
- 2021-01-28 CN CN202110117209.6A patent/CN112799813A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339222A (en) * | 2016-08-23 | 2017-01-18 | 联动优势电子商务有限公司 | Service implementing method and device |
| CN107493199A (en) * | 2017-08-31 | 2017-12-19 | 郑州云海信息技术有限公司 | A kind of distributed type assemblies management method and system |
| CN111736969A (en) * | 2020-06-16 | 2020-10-02 | 中国银行股份有限公司 | Distributed job scheduling method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240369A (en) * | 2021-12-17 | 2022-03-25 | 中国工商银行股份有限公司 | Pipeline deployment method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069265B (en) | Synchronization method of configuration data, business data system, computer system and medium | |
| CN111352653B (en) | System development method based on PaaS cloud platform server and server | |
| CN111813601B (en) | Micro-service rollback method and device for stateful distributed cluster | |
| CN111031058A (en) | Websocket-based distributed server cluster interaction method and device | |
| CN111459539B (en) | Continuous integration pipeline operation method and device based on mirror layering | |
| CN105721547B (en) | The method and apparatus for cooperateing with processing business | |
| CN112463535B (en) | Multi-cluster exception handling method and device | |
| CN101917394B (en) | Middleware system for sharing data in mobile phone equipment and working method | |
| CN110764881A (en) | Distributed system background retry method and device | |
| CN113505520A (en) | Method, device and system for supporting heterogeneous federated learning | |
| CN112965814B (en) | Multi-task scheduling method and system | |
| CN113626002A (en) | Service execution method and device | |
| CN110427260B (en) | Host job scheduling method, device and system | |
| CN112799813A (en) | Distributed job service scheduling method, distributed server and service terminal | |
| CN102566485B (en) | Batch loading method and system for engineering data of programmable logic controllers (PLCs) and corresponding PLCs | |
| CN201854301U (en) | Middleware system for sharing data on mobile phones | |
| CN111367561A (en) | Software program remote development method and device | |
| CN101414260B (en) | System and method for creating multi-mode applications | |
| CN114115821A (en) | Application development method and platform, application deployment method and node, system and equipment | |
| CN113138844A (en) | Task issuing method, management component and working component of virtual machine cluster | |
| CN113918436A (en) | Log processing method and device | |
| CN112463212B (en) | Environment building method and device supporting cross-system platform | |
| CN112565173B (en) | Method and apparatus for message processing | |
| CN115580622A (en) | Workflow execution method based on edge synchronization service, electronic device and medium | |
| CN101772003A (en) | Terminal unit management device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |