CN109656782A - Visual scheduling monitoring method, device and server - Google Patents
Visual scheduling monitoring method, device and server Download PDFInfo
- Publication number
- CN109656782A CN109656782A CN201811584022.1A CN201811584022A CN109656782A CN 109656782 A CN109656782 A CN 109656782A CN 201811584022 A CN201811584022 A CN 201811584022A CN 109656782 A CN109656782 A CN 109656782A
- Authority
- CN
- China
- Prior art keywords
- task
- streaming computing
- server cluster
- information
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the present application provides a kind of visual scheduling monitoring method, device and server, one visualization interface is provided, by responding the selected streaming computing task of visualization interface in management server, and obtain the server cluster information for executing the streaming computing task.According to operation resource information and corresponding mission bit stream of streaming computing task needed for streaming computing task configuration.The streaming computing task for carrying operation resource information and mission bit stream is distributed to corresponding server cluster according to the server cluster information for executing the streaming computing task.And the starting monitoring process after the publication successful information for receiving server cluster feedback, it is monitored with the operation information of convection type calculating task.Visual scheduling monitoring scheme provided by the present application is scheduled and monitors to multiple server clusters using management server is unified, and shows operation information by visualization interface, realizes the unified management of the information between server cluster.
Description
Technical field
The present invention relates to streaming computing technical field, in particular to a kind of visual scheduling monitoring method, device and
Server.
Background technique
With computer technology fast development and the sustainable growth of mobile device and internet of things equipment, based on offline batch
The data processing platform (DPP) of calculating can no longer meet the real-time processing requirement of mass data.Since data service was worth with the time
Loss and reduce rapidly, therefore data generation after must it be calculated and be handled as early as possible.In this context, various
Real-time stream process engine comes into being, such as Spark, Storm, Flink.Stream Processing all uses distributed structure/architecture at present, place
Reason ability can be extended with the growth of interstitial content, have good retractility.Stream Processing framework by calculating logic and
Task schedule is separated with monitoring.Specific calculating logic and method need user voluntarily to complete processing logic and calculating in a compiled manner
The definition of unit.The logical code write is submitted in stream process cluster manually, cluster complete task schedule, resource allocation with
And Mission Monitor.The monitor supervision platform that cluster offer can be logged under this mode checks task run situation, can be in monitor supervision platform
Terminate some running task.But under this mode, some cluster is uniformly given in the scheduling and monitoring of streaming computing task
It manages, between multiple clusters can not communicate and interact.Monitoring and dispatch service are relatively independent, can not accomplish unitized monitoring and
Management.
Summary of the invention
In view of this, the purpose of the application is, a kind of visual scheduling monitoring method, device and server are provided to change
The kind above problem.
The embodiment of the present application provides a kind of visual scheduling monitoring method, is applied to management server, the management service
Device and multiple server clusters communicate to connect, which comprises
The selected streaming computing task of visual page in the management server is responded, and is obtained for executing this
The server cluster information of streaming computing task；
According to operation resource information and corresponding of the streaming computing task needed for streaming computing task configuration
Business information；
The streaming computing for carrying the operation resource information and mission bit stream is appointed according to the server cluster information
Business is distributed to corresponding server cluster；
The starting monitoring process after the publication successful information for receiving server cluster feedback, to appoint to the streaming computing
The operation information of business is monitored.
Optionally, the mission bit stream includes the corresponding task ID of streaming computing task, the method also includes:
Calculating task list is created, the corresponding task ID of the streaming computing task for being successfully distributed to server cluster is added
To in the calculating task list；
The step of operation information to the streaming computing task is monitored, comprising:
The task ID for needing the streaming computing task being monitored is obtained from the calculating task list；
The api interface for calling the server cluster obtains streaming corresponding with the task ID by the api interface
The operation information of calculating task.
Optionally, the method also includes:
Receive the implementing result that the server cluster is returned after having executed the streaming computing task；
The implementing result received is stored into database.
Optionally, the method also includes:
At interval of preset duration to each server set pocket transmission heartbeat message；
The heartbeat return information for detecting whether to receive corresponding server cluster feedback, if receiving, it is determined that with it is corresponding
Server cluster normal communication, if not receiving, it is determined that corresponding server cluster present communications are abnormal；
The information of the server cluster of determining communication abnormality is shown by the visual page.
Optionally, the operation resource information includes the memory source and CPU for executing the streaming computing required by task
Resource.
Another embodiment of the application also provides a kind of visual scheduling monitoring device, is applied to management server, the pipe
It manages server and multiple server clusters communicates to connect, described device includes:
Respond module, for responding the selected streaming computing task of visual page in the management server, and
Obtain the server cluster information for executing the streaming computing task；
Configuration module, for according to operation resource information and the streaming computing needed for streaming computing task configuration
The corresponding mission bit stream of task；
Release module, for the operation resource information and mission bit stream will to be carried according to the server cluster information
Streaming computing task be distributed to corresponding server cluster；
Monitoring module, for the starting monitoring process after the publication successful information for receiving server cluster feedback, with right
The operation information of the streaming computing task is monitored.
Optionally, the mission bit stream includes the corresponding task ID of streaming computing task, described device further include:
Creation module will successfully be distributed to the streaming computing task pair of server cluster for creating calculating task list
The task ID answered is added in the calculating task list；
The monitoring module, is specifically used for:
The task ID for needing the streaming computing task being monitored is obtained from the calculating task list；
The api interface for calling the server cluster obtains streaming corresponding with the task ID by the api interface
The operation information of calculating task.
Optionally, described device further include:
Receiving module, the execution returned after having executed the streaming computing task for receiving the server cluster
As a result；
Memory module, for storing the implementing result received into database.
The embodiment of the present application also provides a kind of server, including memory, processor and is stored on the memory simultaneously
The computer program that can be run on the processor, the processor realize above-mentioned method and step when executing described program.
The embodiment of the present application also provides a kind of readable storage medium storing program for executing, is stored with computer program on the readable storage medium storing program for executing,
The computer program executes above-mentioned method and step when being run by processor.
Visual scheduling monitoring method, device and server provided by the embodiments of the present application, provide a visualization interface, lead to
Response is crossed in the selected streaming computing task of visualization interface of management server, and obtains and appoints for executing the streaming computing
The server cluster information of business.According to operation resource information and the streaming computing task needed for streaming computing task configuration
Corresponding mission bit stream.According to the server cluster information for executing the streaming computing task will carry operation resource information and
The streaming computing task of mission bit stream is distributed to corresponding server cluster.And receive server cluster feedback publication at
Starting monitoring process, is monitored after function information with the operation information of convection type calculating task.Visualization provided by the present application is adjusted
Monitoring scheme is spent, multiple server clusters are scheduled and are monitored using management server is unified, and pass through visualization interface
It shows operation information, realizes the unified management of the information between server cluster.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 is the application scenarios schematic diagram of visual scheduling monitoring method provided by the embodiments of the present application.
Fig. 2 is the structural block diagram of server provided by the embodiments of the present application.
Fig. 3 is one of the flow chart of visual scheduling monitoring method provided by the embodiments of the present application.
Fig. 4 is the two of the flow chart of visual scheduling monitoring method provided by the embodiments of the present application.
Fig. 5 is one of the functional block diagram of visual scheduling monitoring device provided by the embodiments of the present application.
Fig. 6 is the two of the functional block diagram of visual scheduling monitoring device provided by the embodiments of the present application.
Icon: 110- processor；120- memory；130- communication interface；140- bus；500- visual scheduling monitoring dress
It sets；501- respond module；502- configuration module；503- release module；504- monitoring module；505- creation module；506- is received
Module；507- memory module.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.
Therefore, the detailed description of the embodiment of the present invention provided in the accompanying drawings is not intended to limit below claimed
The scope of the present invention, but be merely representative of selected embodiment of the invention.Based on the embodiment of the present invention, those skilled in the art
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.
Referring to Fig. 1, being the application scenarios schematic diagram of visual scheduling monitoring method provided by the embodiments of the present application.This
It include management server and multiple server clusters in scape.The management server respectively with each server cluster communication link
It connects.Wherein, one or more server centereds are namely carried out same service by server cluster together.Each server set
Group is provided out unified API (Application Programming Interface, application programming interface) interface,
Interaction for data, information between management server.The management server includes a visualization interface, can by this
It can be realized human-computer interaction depending on changing interface.
Referring to Fig. 2, a kind of structural schematic diagram of the server provided for the disclosure.The server includes memory 120
With processor 110.Wherein, memory 120 is for storing one or more computer instruction, one or more computer instruction quilt
Processor 110 executes, to realize visual scheduling monitoring method provided by the present application.
Server shown in Fig. 2 further includes bus 140 and communication interface 130, processor 110, communication interface 130 and storage
Device 120 is connected by bus 140.
Wherein, memory 120 may include high-speed random access memory (RAM, Random Access Memory),
It may further include non-labile memory (non-volatile memory), for example, at least a magnetic disk storage.By extremely
A few communication interface 130 (can be wired or wireless) is realized logical between the system network element and at least one other network element
Letter connection, can be used internet, wide area network, local network, Metropolitan Area Network (MAN) etc..
Bus 140 can be isa bus, pci bus or eisa bus etc..The bus can be divided into address bus, number
According to bus, control bus etc..Only to be indicated with a four-headed arrow in Fig. 2, it is not intended that an only bus convenient for indicating
Or a type of bus.
Wherein, memory 120 is for storing program, and the processor 110 executes the journey after receiving and executing instruction
Sequence, method provided by the embodiment of the present application can be applied in processor 110, or be realized by processor 110.
Processor 110 may be a kind of IC chip, the processing capacity with signal.During realization, this Shen
Please each step of method provided by embodiment can pass through the integrated logic circuit or software shape of the hardware in processor 110
The instruction of formula is completed.Above-mentioned processor 110 can be general processor, including central processing unit (Central
Processing Unit, abbreviation CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be number
Signal processor (Digital Signal Processing, abbreviation DSP), specific integrated circuit (Application
Specific Integrated Circuit, abbreviation ASIC), ready-made programmable gate array (Field-Programmable
Gate Array, abbreviation FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware
Component.It may be implemented or execute disclosed each method, step and the logic diagram in disclosure embodiment.General processor
It can be microprocessor or the processor be also possible to any conventional processor etc..In conjunction with disclosed in disclosure embodiment
Method the step of can be embodied directly in hardware decoding processor and execute completion, or with hardware in decoding processor and soft
Part block combiner executes completion.Software module can be located at random access memory, and flash memory, read-only memory may be programmed read-only storage
In the storage medium of this fields such as device or electrically erasable programmable memory, register maturation.The storage medium is located at storage
Device 120, processor 110 read the information in memory 120, complete method implementation provided by the present application in conjunction with its hardware
Step.
In conjunction with Fig. 3, the embodiment of the present invention also provides a kind of visual scheduling method for building up that can be applied to management server,
The management server can be above-mentioned server.Wherein, method and step defined in the related process of the method can be by
The processor 110 is realized.Detailed process shown in Fig. 3 will be described in detail below.
Step S110 responds the selected streaming computing task of visual page in the management server, and obtains
For executing the server cluster information of the streaming computing task.
In the present embodiment, the management server may include visualization interface, which can show for example logical
Believe the information of the server cluster of connection, the configuration in accessible type of service, server cluster including server cluster
Information etc..
It by editor's streaming computing shell script and is deployed in management server, each streaming is shown by visualization interface
Calculating task.Administrator can be by selecting the streaming computing task that need to be executed on the visualization interface of management server.Management
Server is in response to the selected streaming computing task.Also, administrator can also select for executing the streaming computing task
Server cluster, management server can get the information of selected server cluster.
In addition, in the present embodiment, management server also needs whether each server cluster of real-time detection communicates normally.It is optional
Ground, referring to Fig. 4, visual scheduling monitoring method provided by the present application is further comprising the steps of:
Step S210, at interval of preset duration to each server set pocket transmission heartbeat message.
Step S220 detects whether the heartbeat return information for receiving corresponding server cluster feedback, if receiving, into
Enter step S230, if not receiving, enters step S240.
Step S230, determining and corresponding server cluster normal communication.
Step S240 determines that corresponding server cluster present communications are abnormal.
Step S250 shows the information of the server cluster of determining communication abnormality by the visual page
Show.
In the present embodiment, heartbeat feedback scheme is established between management server and each server cluster with each for detecting
Whether the communication between server cluster and management server is normal.Management server can at interval of preset duration, such as 1 second or
2 seconds etc. unlimited, to each server set pocket transmission heartbeat message.If server cluster can with management server normal communication,
It can receive the heartbeat message, and return to heartbeat return information to management server.Management server is receiving heartbeat reply
After information, it may be determined that the server cluster present communications for returning to the heartbeat return information are abnormal.Conversely, if management server to certain
A server set pocket transmission heartbeat message, but do not receive the heartbeat return information of server cluster return, then it can determine this
Server cluster present communications are abnormal.
Management server can be shown the information of server cluster that determine communication abnormality by visualization interface, with
This prompt administrator carries out abnormal investigation.
Step S120, according to operation resource information and the streaming computing task needed for streaming computing task configuration
Corresponding mission bit stream.
Step S130 will carry the stream of the operation resource information and mission bit stream according to the server cluster information
Formula calculating task is distributed to corresponding server cluster.
Step S140, the starting monitoring process after the publication successful information for receiving server cluster feedback, to described
The operation information of streaming computing task is monitored.
It can be seen from the above, management server can the streaming computing task selected of response management person and for executing the streaming
The server cluster information of calculating task.Management server can according to the streaming computing task configure needed for operation resource and
The corresponding mission bit stream of streaming computing task.
Wherein, the operation resource of configuration may include the memory source and CPU money for executing streaming computing required by task
The hardware asset informations such as source, the corresponding mission bit stream of the streaming computing task may include task ID.In addition, it may further comprise streaming
Other essential informations of calculating task, such as scripting language, the foot of stream engine type, the corresponding shell script of streaming computing task
This storage location etc..
After operation resource information and mission bit stream are completed in configuration, then it will carry that operation resource is new and mission bit stream
Streaming computing task is distributed to server cluster corresponding with the server cluster information selected.It optionally, can in the present embodiment
Streaming computing task is distributed in yarn server cluster.
Corresponding server cluster after receiving streaming computing task, by according to required operation resource information by streaming
Distribution of computation tasks gives actual motion node, which is that can be used for executing the corresponding module of streaming computing task
Queue, such as when streaming computing task is data statistics calculating task, then can distribute to corresponding data statistics and calculate queue mould
Block.As streaming computing task be information management task when, then can distribute to corresponding information management Queue module.In addition, service
Device cluster by using the task ID in the mission bit stream of the streaming computing task as the streaming computing task in the cluster appoint
The unique identification of business.
In the present embodiment, after server cluster receives streaming computing task, it will feed back to management server and successfully connect
The information of receipts, starting is monitored process after the publication successful information for receiving server cluster feedback by management server, with right
The operation information of streaming computing task is monitored.
Optionally, management server can create a calculating task list, and the streaming that will successfully be distributed to server cluster
The corresponding task ID of calculating task is added in the calculating task list.It is subsequent to the streaming computing for being distributed to server cluster
When the operation information of task is monitored, the streaming computing for needing to be monitored can be obtained from calculating task list according to demand
The task ID of task.And the api interface of corresponding server cluster is called, it is obtained and the task ID pair by the api interface
The operation information for the streaming computing task answered.
Wherein, management server can convection type calculating task implementation procedure and implementing result be monitored, for example including
The operating status of streaming computing task, such as be currently running, end of run, do not bring into operation.And implementing result includes streaming meter
Whether it is executed normally finishes for calculation task, if occurring abnormal, shows abnormal reason, e.g. backstage manager close task,
Or system reason leads to mission failure etc..Optionally, management server can timing acquiring streaming computing task operation information,
To be monitored.
Implementing result is sent to management server after the corresponding streaming computing task that is finished by server cluster.
Management server stores the implementing result received to database after the implementing result for receiving server set pocket transmission
In.Management server is after some streaming computing task execution terminates, by the monitoring of the streaming computing task of release creation
Thread.
Referring to Fig. 5, the embodiment of the present application also provides a kind of visual scheduling monitoring applied to above-mentioned management server
Device 500, the visual scheduling monitoring device 500 include respond module 501, configuration module 502, release module 503 and
Monitoring module 504.
The respond module 501, for responding the selected streaming computing of visual page in the management server
Task, and obtain the server cluster information for executing the streaming computing task.It is appreciated that the respond module 501 can be with
For executing above-mentioned steps S110, the detailed implementation about the respond module 501 is referred to above-mentioned have step S110
The content of pass.
The configuration module 502 for the operation resource information needed for being configured according to the streaming computing task and is somebody's turn to do
The corresponding mission bit stream of streaming computing task.It is appreciated that the configuration module 502 can be used for executing above-mentioned steps S120, close
It is referred in the detailed implementation of the configuration module 502 above-mentioned to the related content of step S120.
The release module 503, for according to the server cluster information will carry the operation resource information and
The streaming computing task of mission bit stream is distributed to corresponding server cluster.It is appreciated that the release module 503 can be used for holding
Row above-mentioned steps S130, the detailed implementation about the release module 503 are referred to above-mentioned related to step S130 interior
Hold.
The monitoring module 504, for the starting monitoring stream after the publication successful information for receiving server cluster feedback
Journey is monitored with the operation information to the streaming computing task.It is appreciated that the monitoring module 504 can be used for executing
Above-mentioned steps S140, the detailed implementation about the monitoring module 504 are referred to above-mentioned to the related content of step S140.
Referring to Fig. 6, in the present embodiment, the mission bit stream includes the corresponding task ID of streaming computing task, described
Visual scheduling monitoring device 500 further includes creation module 505.
The creation module 505 will successfully be distributed to the streaming computing of server cluster for creating calculating task list
The corresponding task ID of task is added in the calculating task list.
The monitoring module 504, is specifically used for:
The task ID for needing the streaming computing task being monitored is obtained from the calculating task list；
The api interface for calling the server cluster obtains streaming corresponding with the task ID by the api interface
The operation information of calculating task.
In addition, in the present embodiment, the visual scheduling monitoring device 500 further includes receiving module 506 and storage mould
Block 507.
The receiving module 506 is returned after having executed the streaming computing task for receiving the server cluster
The implementing result returned.
The memory module 507, for storing the implementing result received into database.
The embodiment of the present application also provides a kind of readable storage medium storing program for executing, the readable storage medium storing program for executing is stored with computer and can hold
The visual scheduling monitoring method in above-mentioned any means embodiment can be performed in row instruction, the computer executable instructions.
It is apparent to those skilled in the art that for convenience and simplicity of description, the device of foregoing description
Specific work process, no longer can excessively be repeated herein with reference to the corresponding process in preceding method.
In conclusion visual scheduling monitoring method, device and server provided by the embodiments of the present application, it is visual to provide one
Change interface, by responding the selected streaming computing task of visualization interface in management server, and obtains for executing this
The server cluster information of streaming computing task.According to operation resource information and stream needed for streaming computing task configuration
The corresponding mission bit stream of formula calculating task.It will be carried and run according to the server cluster information for executing the streaming computing task
Resource information and the streaming computing task of mission bit stream are distributed to corresponding server cluster.And it is anti-receiving server cluster
Starting monitoring process, is monitored after the publication successful information of feedback with the operation information of convection type calculating task.The application provides
Visual scheduling monitoring scheme, multiple server clusters are scheduled and are monitored using management server is unified, and are passed through
Visualization interface shows operation information, realizes the unified management of the information between server cluster.
In embodiment provided herein, it should be understood that disclosed device and method, it can also be by other
Mode realize.The apparatus embodiments described above are merely exemplary, for example, the flow chart and block diagram in attached drawing are shown
Architectural framework in the cards, function and the behaviour of devices in accordance with embodiments of the present invention, method and computer program product
Make.In this regard, each box in flowchart or block diagram can represent a part of a module, section or code, institute
The a part for stating module, section or code includes one or more executable instructions for implementing the specified logical function.
It should also be noted that function marked in the box can also be to be different from attached drawing in some implementations as replacement
The sequence marked occurs.For example, two continuous boxes can actually be basically executed in parallel, they sometimes can also be by
Opposite sequence executes, and this depends on the function involved.It is also noted that each box in block diagram and or flow chart,
And the combination of the box in block diagram and or flow chart, hardware can be based on the defined function of execution or the dedicated of movement
System realize, or can realize using a combination of dedicated hardware and computer instructions.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or equipment for including a series of elements not only includes those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or equipment institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including institute
State in the process, method, article or equipment of element that there is also other identical elements.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.It should also be noted that similar label and letter exist
Similar terms are indicated in following attached drawing, therefore, once being defined in a certain Xiang Yi attached drawing, are then not required in subsequent attached drawing
It is further defined and explained.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.
Claims (10)
1. a kind of visual scheduling monitoring method, which is characterized in that be applied to management server, the management server with it is multiple
Server cluster communication connection, which comprises
The selected streaming computing task of visual page in the management server is responded, and is obtained for executing the streaming
The server cluster information of calculating task；
According to operation resource information needed for streaming computing task configuration and the corresponding task letter of the streaming computing task
Breath；
The streaming computing task for carrying the operation resource information and mission bit stream is sent out according to the server cluster information
Cloth is to corresponding server cluster；
The starting monitoring process after the publication successful information for receiving server cluster feedback, to the streaming computing task
Operation information is monitored.
2. visual scheduling monitoring method according to claim 1, which is characterized in that the mission bit stream includes streaming meter
The corresponding task ID of calculation task, the method also includes:
Calculating task list is created, the corresponding task ID of streaming computing task for being successfully distributed to server cluster is added to institute
It states in calculating task list；
The step of operation information to the streaming computing task is monitored, comprising:
The task ID for needing the streaming computing task being monitored is obtained from the calculating task list；
The api interface for calling the server cluster obtains streaming computing corresponding with the task ID by the api interface
The operation information of task.
3. visual scheduling monitoring method according to claim 1, which is characterized in that the method also includes:
Receive the implementing result that the server cluster is returned after having executed the streaming computing task；
The implementing result received is stored into database.
4. visual scheduling monitoring method according to claim 1, which is characterized in that the method also includes:
At interval of preset duration to each server set pocket transmission heartbeat message；
It detects whether to receive the heartbeat return information that corresponding server cluster is fed back, if receiving, it is determined that with corresponding with service
Device cluster normal communication, if not receiving, it is determined that corresponding server cluster present communications are abnormal；
The information of the server cluster of determining communication abnormality is shown by the visual page.
5. visual scheduling monitoring method according to claim 1, which is characterized in that the operation resource information includes holding
The memory source and cpu resource of the row streaming computing required by task.
6. a kind of visual scheduling monitoring device, which is characterized in that be applied to management server, the management server with it is multiple
Server cluster communication connection, described device include:
Respond module for responding the selected streaming computing task of visual page in the management server, and obtains
For executing the server cluster information of the streaming computing task；
Configuration module, for according to operation resource information and the streaming computing task needed for streaming computing task configuration
Corresponding mission bit stream；
Release module, for the stream of the operation resource information and mission bit stream will to be carried according to the server cluster information
Formula calculating task is distributed to corresponding server cluster；
Monitoring module, for the starting monitoring process after the publication successful information for receiving server cluster feedback, to described
The operation information of streaming computing task is monitored.
7. visual scheduling monitoring device according to claim 6, which is characterized in that the mission bit stream includes streaming meter
The corresponding task ID of calculation task, described device further include:
Creation module, it is for creating calculating task list, the streaming computing task for being successfully distributed to server cluster is corresponding
Task ID is added in the calculating task list；
The monitoring module, is specifically used for:
The task ID for needing the streaming computing task being monitored is obtained from the calculating task list；
The api interface for calling the server cluster obtains streaming computing corresponding with the task ID by the api interface
The operation information of task.
8. visual scheduling monitoring device according to claim 6, which is characterized in that described device further include:
Receiving module, the execution knot returned after having executed the streaming computing task for receiving the server cluster
Fruit；
Memory module, for storing the implementing result received into database.
9. a kind of server, including memory, processor and it is stored on the memory and can runs on the processor
Computer program, which is characterized in that the processor is realized described in claim 1-5 any one when executing described program
Method and step.
10. a kind of readable storage medium storing program for executing, which is characterized in that be stored with computer program on the readable storage medium storing program for executing, the computer
The method and step as described in any one of claim 1-5 is executed when program is run by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811584022.1A CN109656782A (en) | 2018-12-24 | 2018-12-24 | Visual scheduling monitoring method, device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811584022.1A CN109656782A (en) | 2018-12-24 | 2018-12-24 | Visual scheduling monitoring method, device and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109656782A true CN109656782A (en) | 2019-04-19 |
Family
ID=66116608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811584022.1A Pending CN109656782A (en) | 2018-12-24 | 2018-12-24 | Visual scheduling monitoring method, device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109656782A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110324188A (en) * | 2019-07-05 | 2019-10-11 | 上海中通吉网络技术有限公司 | Monitoring warning system and method |
| CN110795223A (en) * | 2019-10-29 | 2020-02-14 | 浪潮云信息技术有限公司 | Cluster scheduling system and method for unified resource management |
| CN110837399A (en) * | 2019-10-31 | 2020-02-25 | 锐捷网络股份有限公司 | Method and device for managing streaming computing application program and computing equipment |
| CN111182061A (en) * | 2019-12-30 | 2020-05-19 | 苏宁云计算有限公司 | Task distribution processing method, system, computer device and storage medium |
| CN111338888A (en) * | 2020-02-13 | 2020-06-26 | 北京奇艺世纪科技有限公司 | Data statistical method and device, electronic equipment and storage medium |
| CN111367680A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Job task allocation method, device, server, system and storage medium |
| CN111597024A (en) * | 2020-05-14 | 2020-08-28 | 科东(广州)软件科技有限公司 | Cross-domain cluster processing method and device, electronic equipment and storage medium |
| CN111767122A (en) * | 2019-05-17 | 2020-10-13 | 北京京东尚科信息技术有限公司 | Distributed task scheduling management method and device |
| CN111858260A (en) * | 2020-07-31 | 2020-10-30 | 中国工商银行股份有限公司 | Information display method, device, equipment and medium |
| CN111930589A (en) * | 2020-07-02 | 2020-11-13 | 上海微亿智造科技有限公司 | Streaming task automatic monitoring system and method |
| CN112019362A (en) * | 2019-05-30 | 2020-12-01 | 中移(苏州)软件技术有限公司 | Data transmission method, device, server, terminal, system and storage medium |
| CN113806177A (en) * | 2021-09-22 | 2021-12-17 | 网易（杭州）网络有限公司 | Cluster monitoring method and device, electronic equipment and storage medium |
| CN114070889A (en) * | 2021-11-10 | 2022-02-18 | 北京百度网讯科技有限公司 | Configuration method, traffic forwarding method, device, storage medium, and program product |
| CN117193990A (en) * | 2023-11-08 | 2023-12-08 | 建信金融科技有限责任公司 | Scheduling management method, device, equipment and storage medium of http interface |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9946577B1 (en) * | 2017-08-14 | 2018-04-17 | 10X Genomics, Inc. | Systems and methods for distributed resource management |
| CN107959705A (en) * | 2016-10-18 | 2018-04-24 | 阿里巴巴集团控股有限公司 | The distribution method and control server of streaming computing task |
| CN108845878A (en) * | 2018-05-08 | 2018-11-20 | 南京理工大学 | The big data processing method and processing device calculated based on serverless backup |
-
2018
- 2018-12-24 CN CN201811584022.1A patent/CN109656782A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107959705A (en) * | 2016-10-18 | 2018-04-24 | 阿里巴巴集团控股有限公司 | The distribution method and control server of streaming computing task |
| US9946577B1 (en) * | 2017-08-14 | 2018-04-17 | 10X Genomics, Inc. | Systems and methods for distributed resource management |
| CN108845878A (en) * | 2018-05-08 | 2018-11-20 | 南京理工大学 | The big data processing method and processing device calculated based on serverless backup |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767122A (en) * | 2019-05-17 | 2020-10-13 | 北京京东尚科信息技术有限公司 | Distributed task scheduling management method and device |
| CN112019362B (en) * | 2019-05-30 | 2022-06-10 | 中移(苏州)软件技术有限公司 | Data transmission method, device, server, terminal, system and storage medium |
| CN112019362A (en) * | 2019-05-30 | 2020-12-01 | 中移(苏州)软件技术有限公司 | Data transmission method, device, server, terminal, system and storage medium |
| CN110324188A (en) * | 2019-07-05 | 2019-10-11 | 上海中通吉网络技术有限公司 | Monitoring warning system and method |
| CN110795223A (en) * | 2019-10-29 | 2020-02-14 | 浪潮云信息技术有限公司 | Cluster scheduling system and method for unified resource management |
| CN110837399A (en) * | 2019-10-31 | 2020-02-25 | 锐捷网络股份有限公司 | Method and device for managing streaming computing application program and computing equipment |
| CN111182061B (en) * | 2019-12-30 | 2023-04-14 | 苏宁云计算有限公司 | Task distribution processing method, system, computer device and storage medium |
| CN111182061A (en) * | 2019-12-30 | 2020-05-19 | 苏宁云计算有限公司 | Task distribution processing method, system, computer device and storage medium |
| CN111338888B (en) * | 2020-02-13 | 2023-12-15 | 北京奇艺世纪科技有限公司 | Data statistics method and device, electronic equipment and storage medium |
| CN111338888A (en) * | 2020-02-13 | 2020-06-26 | 北京奇艺世纪科技有限公司 | Data statistical method and device, electronic equipment and storage medium |
| CN111367680A (en) * | 2020-03-31 | 2020-07-03 | 中国建设银行股份有限公司 | Job task allocation method, device, server, system and storage medium |
| CN111597024A (en) * | 2020-05-14 | 2020-08-28 | 科东(广州)软件科技有限公司 | Cross-domain cluster processing method and device, electronic equipment and storage medium |
| CN111930589A (en) * | 2020-07-02 | 2020-11-13 | 上海微亿智造科技有限公司 | Streaming task automatic monitoring system and method |
| CN111930589B (en) * | 2020-07-02 | 2021-07-27 | 上海微亿智造科技有限公司 | Streaming task automatic monitoring system and method |
| CN111858260A (en) * | 2020-07-31 | 2020-10-30 | 中国工商银行股份有限公司 | Information display method, device, equipment and medium |
| CN113806177A (en) * | 2021-09-22 | 2021-12-17 | 网易（杭州）网络有限公司 | Cluster monitoring method and device, electronic equipment and storage medium |
| CN114070889A (en) * | 2021-11-10 | 2022-02-18 | 北京百度网讯科技有限公司 | Configuration method, traffic forwarding method, device, storage medium, and program product |
| CN114070889B (en) * | 2021-11-10 | 2023-11-14 | 北京百度网讯科技有限公司 | Configuration method, traffic forwarding device, storage medium, and program product |
| CN117193990A (en) * | 2023-11-08 | 2023-12-08 | 建信金融科技有限责任公司 | Scheduling management method, device, equipment and storage medium of http interface |
| CN117193990B (en) * | 2023-11-08 | 2024-02-23 | 建信金融科技有限责任公司 | Scheduling management method, device, equipment and storage medium of http interface |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109656782A (en) | Visual scheduling monitoring method, device and server | |
| CN111625331B (en) | Task scheduling method, device, platform, server and storage medium | |
| CN110611707B (en) | Task scheduling method and device | |
| CN104991821B (en) | method and device for processing monitoring tasks in batches | |
| CN107622060B (en) | Order tracking analysis method and device | |
| CN112416581B (en) | Distributed calling system for timed tasks | |
| CN111400011B (en) | Real-time task scheduling method, system, equipment and readable storage medium | |
| CN110928655A (en) | Task processing method and device | |
| CN106557369A (en) | A kind of management method and system of multithreading | |
| CN111026602A (en) | Health inspection scheduling management method and device of cloud platform and electronic equipment | |
| CN113051019A (en) | Flow task execution control method, device and equipment | |
| CN111104210A (en) | Task processing method and device and computer system | |
| US11782753B2 (en) | Node-local-unscheduler for scheduling remediation | |
| CN113065779B (en) | Data processing method and device and electronic equipment | |
| CN114327894A (en) | Resource allocation method, device, electronic equipment and storage medium | |
| CN113658351A (en) | Product production method and device, electronic equipment and storage medium | |
| CN113190427A (en) | Caton monitoring method and device, electronic equipment and storage medium | |
| CN113377535A (en) | Distributed timing task allocation method, device, equipment and readable storage medium | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN109670932B (en) | Credit data accounting method, apparatus, system and computer storage medium | |
| WO2020001427A1 (en) | Analysis task execution method, apparatus and system, and electronic device | |
| CN111913784A (en) | Task scheduling method and device, network element and storage medium | |
| CN115187097A (en) | Task scheduling method and device, electronic equipment and computer storage medium | |
| CN115344370A (en) | Task scheduling method, device, equipment and storage medium | |
| CN114675954A (en) | Task scheduling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190419 |
|
| RJ01 | Rejection of invention patent application after publication |