CN111190798A - Service data monitoring and warning device and method - Google Patents
Service data monitoring and warning device and method Download PDFInfo
- Publication number
- CN111190798A CN111190798A CN202010006684.1A CN202010006684A CN111190798A CN 111190798 A CN111190798 A CN 111190798A CN 202010006684 A CN202010006684 A CN 202010006684A CN 111190798 A CN111190798 A CN 111190798A
- Authority
- CN
- China
- Prior art keywords
- alarm
- task
- execution
- submodule
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 27
- 230000001960 triggered effect Effects 0.000 claims abstract description 4
- 230000007246 mechanism Effects 0.000 claims description 28
- 235000010627 Phaseolus vulgaris Nutrition 0.000 claims description 14
- 244000046052 Phaseolus vulgaris Species 0.000 claims description 14
- 230000002159 abnormal effect Effects 0.000 claims description 11
- 230000002776 aggregation Effects 0.000 claims description 10
- 238000004220 aggregation Methods 0.000 claims description 10
- 238000009826 distribution Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 28
- 230000008569 process Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000011161 development Methods 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000008571 general function Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3017—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is implementing multitasking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a service data monitoring and warning device and a method, and belongs to the technical field of big data processing. The device comprises: the report task scheduling module is used for scheduling the tasks and monitoring the running condition of the tasks to generate a task queue; the queue management module is used for loading and concurrency control of the task queue; the execution management module is used for returning an alarm task and alarm information according to a preset alarm rule, and comprises an execution determining submodule and an execution submodule, wherein the execution determining submodule is used for the benname to obtain a Spring container, and determining a proxy object from the Spring container to obtain a corresponding execution submodule of the proxy object; the execution submodule is used for acquiring an alarm task and information required by alarm according to the task execution condition and the alarm rule; the alarm rule management module is used for configuring and managing the alarm rule triggered by the alarm task; and the message management module is used for receiving the alarm task and the alarm information, processing the data of the alarm information and then issuing the processed data.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a service data monitoring and warning device and a service data monitoring and warning method.
Background
With the continuous enrichment of services, in order to improve the performance of the system, a large number of asynchronous processing forms are adopted, so that more data to be processed are generated, and the accurate and timely monitoring of the service data generated by the large and complex service scenes becomes an important task.
For each service scene or asynchronous to-be-processed table, JOB is required to be used for monitoring at regular time and giving an alarm to related personnel for processing in time, the task needs development workload, only a small amount of difference exists between different tables, and repeated and complicated monitoring processes result in low monitoring alarm efficiency, the system code amount and the code repetition degree are increased, and a large number of JOBs increase the maintenance cost of scheduling platform tasks.
Disclosure of Invention
In order to solve the problems in the prior art, embodiments of the present invention provide a service data monitoring and warning apparatus and method, which monitor and centrally schedule tasks in different service scenarios, and flexibly adjust configuration parameters of warning rules according to different service monitoring requirements by using a configurable warning manner, thereby not only ensuring efficient monitoring and warning requirements in different service scenarios, but also reducing code development and post-maintenance cost. The technical scheme is as follows:
in one aspect, a traffic data monitoring and warning device is provided, the device includes:
the task scheduling module is used for scheduling tasks and monitoring the task running condition to generate a task queue; the queue management module is used for loading and concurrency control of the task queue; the execution management module is used for returning an alarm task and alarm information according to a preset alarm rule, and comprises an execution determining submodule and an execution submodule, wherein the execution determining submodule is used for acquiring a Spring container through a bean method, determining a proxy object from the Spring container and acquiring a corresponding execution submodule of the proxy object; the execution submodule is used for acquiring an alarm task and information required by alarm according to the task execution condition and the alarm rule; the alarm rule management module is used for configuring and managing the alarm rule triggered by the alarm task; and the message management module is used for receiving the alarm task and the alarm information, processing the data of the alarm information and then issuing the processed data.
Further, the alarm rule at least comprises at least one or more of the group consisting of an alarm execution frequency policy, an alarm threshold policy, an alarm frequency policy, a template dynamic configuration policy, and a personnel configuration policy.
Further, the alarm rule is configured and managed through a bean method; and/or the alarm execution frequency strategy adopts a cron expression; and/or the alarm threshold strategy comprises an abnormal alarm threshold and a backlog alarm threshold.
Furthermore, the task scheduling module comprises a scheduling control submodule, a task monitoring submodule, an abnormal mechanism configuration submodule and an abnormal mechanism execution submodule, wherein the scheduling control submodule is used for scheduling and controlling the concurrent execution of tasks in a cluster environment; the task monitoring submodule is used for monitoring task scheduling conditions and task running conditions, grouping the tasks and generating a task queue; the exception mechanism configuration submodule is used for configuring exception mechanisms including an error retry mechanism and an exception reminding mechanism, and the exception mechanism execution submodule is used for executing corresponding exception mechanisms according to task scheduling execution conditions.
Further, the scheduling control sub-module is also used for subsequent task scheduling and control across business systems.
Further, the queue management module comprises a task data query submodule, a task grouping rule management submodule and a task state management submodule, wherein the task data query submodule is used for querying task data from a database for storing business data; the task grouping rule management submodule is used for configuring and managing a task grouping rule; and the task state management submodule is used for recording and managing the task state.
Further, the database includes a Redis database and/or a MySQL database.
Further, the message management module is configured to perform data processing including information content splicing and alarm event aggregation according to the alarm task and information required for alarm thereof, and manage an alarm information allocation rule including an alarm information issuing mode.
Further, the execution sub-module is used for acquiring the alarm task and the information required by the alarm according to the task execution condition and the alarm rule, and the message management module is used for performing data processing including information content splicing and alarm event aggregation according to the alarm task and the information required by the alarm, and managing the alarm information allocation rule including an alarm information issuing mode, and comprises:
the execution submodule acquires the historical execution time and the alarm time of the alarm task according to the task execution condition and the alarm rule, judges whether the alarm needs to be executed or not according to the execution frequency and the alarm frequency, and triggers a corresponding alarm instruction if the data quantity which is in accordance with the configuration table and the filtering condition statistics of the alarm task is greater than an alarm threshold value; and the message management module splices the information content of the alarm information according to the alarm instruction and sends the information content down through a short message and/or a mailbox.
On the other hand, a service data warning method of the service data monitoring warning device in the above scheme is provided, and the method includes:
the task scheduling module generates a task queue for scheduling tasks;
the queue management module loads the whole amount of a single scheduling task in the same time to a Redis queue, and acquires an alarm task group from the Redis queue according to the alarm rule management module;
the execution management module acquires a Spring container according to the benName, determines a proxy object from the Spring container, acquires an actuator of the proxy object, substitutes execution parameters and the like for execution, judges whether an alarm task reaches an alarm threshold according to the alarm rule, and returns the alarm task and the alarm message meeting the alarm threshold to the message management module;
the message management module carries out alarm information content splicing and alarm event aggregation and issues the alarm information through the alarm information distribution rule of mails, short messages or WeChat.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
1. firstly, task scheduling and monitoring of task running conditions are carried out through a task scheduling module, and a task queue is generated; the loading and concurrency control of the task queue are carried out through a queue management module; the alarm task and the alarm information are returned through the execution management module according to the preset alarm rule; the method comprises the steps that an alarm task and alarm information are received through a message management module, the alarm information is issued after being subjected to data processing, the tasks of different service scenes are monitored and centrally scheduled, and the configurable alarm mode is adopted to flexibly adjust the configuration parameters of alarm rules according to different service monitoring requirements, so that the high-efficiency monitoring alarm requirements of different service scenes are guaranteed, the code development amount and the later maintenance cost are reduced, the general function codes are configurable, the monitoring related workload caused by each new service can be reduced, and the rapid alarm configuration is realized;
2. secondly, as the alarm adopts a configurable mode, and the alarm configuration supports flexible alarm strategy configurations such as alarm execution frequency (cron expression), alarm threshold (abnormal, backlog), alarm frequency (frequent alarm prevention), template dynamic configuration, personnel configuration and the like, a user can flexibly adjust execution parameters, alarm threshold, execution frequency, alarm frequency, template, alarm mode and the like according to the service monitoring requirement, and can also customize alarm implementation, thereby meeting different service monitoring requirements;
3. moreover, various alarm modes and ways are supported, the alarm configuration supports instant communication tools such as mobile phones, mails, bean sprouts and WeChat, the alarm modes are flexibly selected, different alarm modes are used according to the importance of the business, and related personnel can be guaranteed to receive alarm notifications in time;
4. moreover, due to the adoption of flexible configuration of various alarm ways and frequencies, on one hand, the alarm can be configured with different execution beans (the code development is simple), various service alarms are supported, and meanwhile, the bean method can be customized, so that the alarm flexibility is obviously increased; on the other hand, the real-time performance of the alarm can be improved, alarm personnel can process the alarm problem in time, the service function is guaranteed, and the cost of short messages and the like is reduced by flexibly adjusting the alarm threshold value, overstocking the alarm personnel and the like.
5. In addition, the configuration is simple, the related technologies such as big data and the like are not needed, the code reusability aiming at the specific type of service is strong, and the method is suitable for different service systems.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a service data monitoring and warning apparatus according to an embodiment of the present invention;
fig. 2 is a flowchart of a service data monitoring and warning method of a service data monitoring and warning device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the operation of the apparatus components;
FIG. 4 is a business architecture diagram;
FIG. 5 is a system base workflow diagram;
FIG. 6 is a schematic diagram of a Redis queue;
FIG. 7 is a schematic diagram of an alarm rule configuration parameter page;
FIG. 8 is a flowchart of the operation of a particular business data monitoring alert;
FIG. 9 is a diagram of an application scenario example-alarm rules configuration parameter pages;
FIG. 10 is a sample traffic template;
FIG. 11 is a business process flow diagram;
FIG. 12 is a diagram of an application scenario example two alarm rules items configuration parameters page;
FIG. 13 is a sample service template;
FIG. 14 is a business process flow diagram.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
The business data monitoring and warning device and the method provided by the embodiment of the invention carry out task scheduling and monitoring on the task running condition through the task scheduling module to generate a task queue; the loading and concurrency control of the task queue are carried out through a queue management module; the alarm task and the alarm information are returned through the execution management module according to the preset alarm rule; the warning task and the warning information are received through the message management module, the warning information is issued after data processing is carried out on the warning information, the tasks of different service scenes are monitored and centrally scheduled, and the configurable warning mode is adopted to flexibly adjust the configuration parameters of the warning rules according to different service monitoring requirements, so that the rapid monitoring warning requirements of different service scenes are guaranteed, and the code development amount and the later maintenance cost are reduced.
The following describes in detail a service data monitoring and warning apparatus and method provided by the embodiments of the present invention with reference to specific implementations and accompanying drawings.
Fig. 1 is a schematic structural diagram of a service data monitoring and warning device according to an embodiment of the present invention, and as shown in fig. 1, the service data monitoring and warning device 1 according to the embodiment of the present invention includes a task scheduling module 11, a queue management module 12, an execution management module 13, an alarm rule management module 14, and a message management module 15.
The task scheduling module 11 is configured to schedule a task and monitor a task running condition to generate a task queue. Specifically, the task scheduling module 11 includes a scheduling control sub-module 111, a task monitoring sub-module 112, an abnormal mechanism configuration sub-module 113, and an abnormal mechanism execution sub-module 114, where the scheduling control sub-module 111 is configured to schedule and control concurrent execution of tasks in a cluster environment, and preferably, is also configured to schedule and control subsequent tasks of a cross-service system; the task monitoring submodule 112 is configured to monitor a task scheduling condition and a task running condition, group tasks, and generate a task queue; the exception mechanism configuration sub-module 113 is used for configuring an exception mechanism including an error retry mechanism and an exception reminding mechanism; the exception mechanism execution sub-module 114 is used for performing corresponding exception mechanism execution according to the task scheduling execution condition.
And the queue management module 12 is used for loading and concurrency control of the task queue. Specifically, the queue management module 12 includes a task data query submodule 121, a task grouping rule management submodule 122, and a task state management submodule 123, where the task data query submodule 121 is configured to query task data from a database storing business data; the task grouping rule management submodule 122 is configured to configure and manage the task grouping rules; the task status management submodule 123 is configured to record and manage a task status. Preferably, the database comprises a Redis database and/or a MySQL database.
And the execution management module 13 is used for returning an alarm task and alarm information according to a preset alarm rule. Specifically, the execution management module 13 includes an execution determination submodule 131 and an execution submodule 132, where the execution determination submodule 131 is configured to obtain a Spring container by a bean method (i.e., by a bean), determine a proxy object from the Spring container, and obtain a corresponding execution submodule 132 of the proxy object; the execution sub-module 132 is used for acquiring the alarm task and the information required by the alarm thereof according to the task execution condition and the alarm rule. Preferably, the execution sub-module 132 obtains the historical execution time and the alarm time of the alarm task according to the task execution condition and the alarm rule, determines whether the alarm needs to be executed according to the execution frequency and the alarm frequency, and triggers a corresponding alarm instruction if the amount of data according to the configuration table and the filtering condition statistics of the alarm task is greater than the alarm threshold. Here, the alert task is a task that is determined by monitoring to be subsequently required to be alerted.
The alarm rule management module 14 is configured to configure and manage alarm rules triggered by the alarm task, where the alarm rules at least include one or more of a group consisting of an alarm execution frequency policy, an alarm threshold policy, an alarm frequency policy, a template dynamic configuration policy, and a personnel configuration policy. Further preferably, alarm rule configuration and management are carried out through a bean method; and/or, the alarm execution frequency strategy adopts a cron expression; and/or the alarm threshold strategy comprises an abnormal alarm threshold and a backlog alarm threshold.
And the message management module 15 is configured to receive the alarm task and the alarm information, perform data processing on the alarm information, and then issue the processed alarm information. Specifically, the message management module 15 is configured to perform data processing including information content splicing and alarm event aggregation according to the alarm task and the information required for alarm thereof, and manage an alarm information allocation rule including an alarm information issuing mode. Preferably, the message management module 15 splices the information content of the alarm information according to the alarm instruction, and sends the information content to the alarm information through a short message and/or a mailbox.
Fig. 2 is a flowchart of a service data monitoring and warning method of a service data monitoring and warning device according to an embodiment of the present invention, and illustrates a preferred implementation of the service data monitoring and warning device to perform a service data monitoring and warning operation. As shown in fig. 2, the service data monitoring and warning method includes the following steps:
201. the task scheduling module generates a task queue for scheduling tasks;
202. the queue management module loads the whole amount of a single scheduling task in the same time to a Redis queue, and acquires an alarm task group from the Redis queue according to the alarm rule management module;
203. the execution management module acquires a Spring container according to the benName, determines a proxy object from the Spring container, acquires an actuator of the proxy object, substitutes execution parameters and the like for execution, judges whether an alarm task reaches an alarm threshold according to an alarm rule, and returns the alarm task meeting the alarm threshold and information required by the alarm to the message management module;
204. the message management module carries out alarm information content splicing and alarm event aggregation and issues the alarm information through the alarm information distribution rule of mails, short messages or WeChat.
It should be noted that: the service data monitoring and warning device provided in the above embodiment is only illustrated by the division of the above functional modules when triggering the service data monitoring and warning service, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the service data monitoring and warning method provided by the above embodiment and the service data monitoring and warning device embodiment belong to the same concept, and the specific implementation process thereof is described in detail in the device embodiment and is not described herein again.
The following further describes the service data monitoring and warning apparatus and method provided in the embodiment of the present invention with reference to an application example. It should be noted that the application example only exemplarily shows one implementation manner, and the implementation of each functional module and the method flow in the service data monitoring and warning device and method provided by the embodiment of the present invention may also adopt any possible implementation manner in the prior art without departing from the inventive concept of the present invention.
Fig. 3 is a schematic diagram of the operation of the device assembly. FIG. 4 is a business architecture diagram. Fig. 5 is a schematic diagram of the system basic workflow. FIG. 6 is a Redis queue diagram. FIG. 7 is a diagram of an alarm rule configuration parameter page. Fig. 8 is a flowchart of the operation of a specific traffic data monitoring alert. Service risk alarm, data abnormity, backlog alarm and the like in the system need unified menu configuration management, similar monitoring functions can support new service scene alarm by direct configuration, maintenance cost is reduced, and alarm information can be supported to be notified to common communication tools such as mobile phones, mails, WeChat, work groups and the like.
As shown in fig. 3 to fig. 8, the service data monitoring and warning scheme provided in the embodiment of the present invention can be implemented by the following hardware:
a dispatching platform: the centralized management and scheduling of the timed tasks are undertaken, the control of the concurrent execution of the tasks in the cluster environment is intensively solved, the task scheduling execution condition is monitored, an abnormal short message warning and retry mechanism is realized, and meanwhile, the subsequent task scheduling of the cross-service system is also provided;
a queue manager: providing the loading and concurrent control of an alarm task queue, wherein the queue loading comprises the management of task grouping rules and task states;
a scheduling rule manager: the rule management of the alarm task triggering comprises the execution frequency, the alarm frequency and other rules;
an execution manager: acquiring a Spring container according to the benname, determining a proxy object from the Spring container, and acquiring an actuator of the proxy object;
an actuator: according to the threshold value, the business rule and the like configured by the alarm task, the tasks needing to be alarmed and the information needed by the assembly alarm are returned;
a message manager: content splicing of alarm information, alarm event aggregation, and assignment rule management (such as individual, group, public number, etc.).
The following introduces a specific workflow of service data monitoring warning, and distinguishes between first loading and non-first loading, the process is as follows:
1. and (4) starting scheduling, acquiring a task group to be executed from the task queue, and if the task queue is empty (the queue length is equal to 0), representing that the task is a first full load task or a new round of scheduling tasks.
Loading a scene for the first time:
2. and adding a shared lock (setnx) first to ensure that only one scheduling task is loaded into the Redis queue in full at the same time, so as to prevent repeated data in the Redis queue caused by concurrency.
3. And if the locking fails, representing that other schedules are loading (concurrent), ending the process and waiting for the next scheduling.
4. And (3) successfully locking, loading the full amount of tasks into a Redis queue, wherein the full amount of tasks can be stored in the ehcache (the full amount of task data of the ehcache can be loaded from the DB, and meanwhile, the failure time is configured), so that the DB pressure is prevented from being increased through the database.
5. Rule for full load: grouping according to task group number, packing task List, push to Redis queue, and doing so is to avoid repeatedly querying DB. For example, one kind of service data may have various thresholds, alarm types, execution frequencies, etc. according to the monitoring dimension, but all are data to be queried, and this scenario avoids multiple database-penetrating queries, and may be grouped and queried only once.
6. After loading is successful, the shared lock needs to be released, and processing continues from a group of alarm tasks in the Redis queue POP.
Non-first load scenario:
7. and acquiring an alarm task group (List) from the Redis queue, and adding a shared lock (setnx) according to the packet number to prevent concurrency.
8. And acquiring a Spring container according to the benName, determining a proxy object from the Spring container, acquiring an actuator of the proxy object, and substituting the execution parameters and the like into the actuator for execution.
9. The executor judges whether the alarm threshold is reached according to the rules of the backlog threshold, the execution frequency, the alarm frequency and the like, and returns the alarm tasks meeting the alarm threshold to the message manager.
10. The message manager sends messages such as mails, short messages and WeChat by combining task priority, alarm event aggregation, dispatching rules and the like.
Two specific application scenario examples are described below to further explain the service data monitoring and warning apparatus and method provided by the embodiment of the present invention.
Application scenario example one: and the asynchronous scheme backlogs the to-be-processed table and gives an abnormal alarm.
FIG. 9 is a diagram of an application scenario example-alarm rules configuration parameters page. Fig. 10 is a sample traffic template. FIG. 11 is a business process flow diagram.
This application scenario will be described in detail with reference to fig. 9 to 11.
1. For some scenes in the system, an asynchronous processing scheme is designed, for example: complex processing logic of a real-time link, short message sending, data pushing of a peripheral system and the like are not affected. The asynchronous scheme is designed to be decoupled from a real-time link, so that the time consumption of the link is reduced, and the user experience is improved; and using the table to be processed, adding error retry and alarm mechanisms, reducing the dependence on the periphery, pausing the job when a peripheral system is down, and executing the job again after the periphery is recovered.
2. The asynchronous to-be-processed table generally comprises some common fields (processing state, processing times, failure reason and the like), the processing state and the processing times of each to-be-processed record need to be monitored, abnormality can be monitored timely, personnel processing can be allocated, and backlog alarm, failure alarm and the like of the to-be-processed data are similar.
Application scenario example 2: periodic equity actual issuing difference alarm
FIG. 12 is a page diagram of various configuration parameters of an example two alarm rule application scenario. Fig. 13 is a sample traffic template. FIG. 14 is a business process flow diagram.
This application scenario will be described in detail with reference to fig. 12 to 14.
1. The alarm may also be directed to statistics of data or comparison of business differences, for example, a system is a system for executing and issuing rights and interests, there is a business for issuing shipping coupons periodically, shipping coupons are issued regularly every morning, after the issuance is completed, the issuance result is checked, and for the periodic issuance, a multi-dimensional monitoring mechanism is designed, wherein one monitoring mechanism is to compare whether the total amount to be issued corresponds to the actual total amount of details to be issued. The general alarm design scheme can be applied to the scene, the newly added bean is realized, the various rights and interests in the system are distributed in a mode similar to the distribution mode of the freight ticket, and the rapid support can be realized by newly adding configuration on the background.
All the above-mentioned optional technical solutions can be combined arbitrarily to form the optional embodiments of the present invention, and are not described herein again.
In summary, the service data monitoring and warning device and method provided by the embodiments of the present invention have the following beneficial effects, compared with the prior art:
1. firstly, task scheduling and monitoring of task running conditions are carried out through a task scheduling module, and a task queue is generated; the loading and concurrency control of the task queue are carried out through a queue management module; the alarm task and the alarm information are returned through the execution management module according to the preset alarm rule; the method comprises the steps that an alarm task and alarm information are received through a message management module, the alarm information is issued after being subjected to data processing, the tasks of different service scenes are monitored and centrally scheduled, and the configurable alarm mode is adopted to flexibly adjust the configuration parameters of alarm rules according to different service monitoring requirements, so that the high-efficiency monitoring alarm requirements of different service scenes are guaranteed, the code development amount and the later maintenance cost are reduced, the general function codes are configurable, the monitoring related workload caused by each new service can be reduced, and the rapid alarm configuration is realized;
2. secondly, as the alarm adopts a configurable mode, and the alarm configuration supports flexible alarm strategy configurations such as alarm execution frequency (cron expression), alarm threshold (abnormal, backlog), alarm frequency (frequent alarm prevention), template dynamic configuration, personnel configuration and the like, a user can flexibly adjust execution parameters, alarm threshold, execution frequency, alarm frequency, template, alarm mode and the like according to the service monitoring requirement, and can also customize alarm implementation, thereby meeting different service monitoring requirements;
3. moreover, various alarm modes and ways are supported, the alarm configuration supports instant communication tools such as mobile phones, mails, bean sprouts and WeChat, the alarm modes are flexibly selected, different alarm modes are used according to the importance of the business, and related personnel can be guaranteed to receive alarm notifications in time;
4. moreover, due to the adoption of flexible configuration of various alarm ways and frequencies, on one hand, the alarm can be configured with different execution beans (the code development is simple), various service alarms are supported, and meanwhile, the bean method can be customized, so that the alarm flexibility is obviously increased; on the other hand, the real-time performance of the alarm can be improved, alarm personnel can process the alarm problem in time, the service function is guaranteed, and the cost of short messages and the like is reduced by flexibly adjusting the alarm threshold value, overstocking the alarm personnel and the like.
5. In addition, the configuration is simple, the related technologies such as big data and the like are not needed, the code reusability aiming at the specific type of service is strong, and the method is suitable for different service systems.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the true scope of the embodiments of the present application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A traffic data monitoring warning device, said device comprising:
the task scheduling module is used for scheduling tasks and monitoring the task running condition to generate a task queue; the queue management module is used for loading and concurrency control of the task queue; the execution management module is used for returning an alarm task and alarm information according to a preset alarm rule, and comprises an execution determining submodule and an execution submodule, wherein the execution determining submodule is used for acquiring a Spring container through a bean method, determining a proxy object from the Spring container and acquiring a corresponding execution submodule of the proxy object; the execution submodule is used for acquiring an alarm task and information required by alarm according to a task execution condition and the alarm rule; the alarm rule management module is used for configuring and managing the alarm rule triggered by the alarm task; and the message management module is used for receiving the alarm task and the alarm information, processing the data of the alarm information and then issuing the processed data.
2. The apparatus of claim 1, wherein the alarm rules comprise at least one or more of the group consisting of an alarm execution frequency policy, an alarm threshold policy, an alarm frequency policy, a template dynamic configuration policy, and a personnel configuration policy.
3. The apparatus of claim 2, wherein the alarm rule configuration and management is performed by a bean method; and/or the alarm execution frequency strategy adopts a cron expression; and/or the alarm threshold strategy comprises an abnormal alarm threshold and a backlog alarm threshold.
4. The device according to claim 1, wherein the task scheduling module includes a scheduling control sub-module, a task monitoring sub-module, an exception mechanism configuration sub-module, and an exception mechanism execution sub-module, and the scheduling control sub-module is configured to schedule and control concurrent execution of tasks in a cluster environment; the task monitoring submodule is used for monitoring task scheduling conditions and task running conditions, grouping the tasks and generating a task queue; the exception mechanism configuration submodule is used for configuring exception mechanisms including an error retry mechanism and an exception reminding mechanism, and the exception mechanism execution submodule is used for executing corresponding exception mechanisms according to task scheduling execution conditions.
5. The apparatus of claim 4, wherein the scheduling control sub-module is further configured for subsequent task scheduling and control across business systems.
6. The apparatus according to claim 1, wherein the queue management module comprises a task data query submodule, a task grouping rule management submodule and a task state management submodule, and the task data query submodule is used for querying task data from a database storing business data; the task grouping rule management submodule is used for configuring and managing a task grouping rule; and the task state management submodule is used for recording and managing the task state.
7. The apparatus of claim 6, wherein the database comprises a Redis database and/or a MySQL database.
8. The apparatus of claim 1, wherein the message management module is configured to perform data processing including information content splicing and alarm event aggregation according to the alarm task and information required for alarm thereof, and perform alarm information allocation rule management including an alarm information issuing manner.
9. The apparatus of claim 8, wherein the execution sub-module is configured to obtain an alarm task and information required for alarm thereof according to a task execution condition and an alarm rule, and the message management module is configured to perform data processing including information content splicing and alarm event aggregation according to the alarm task and the information required for alarm thereof, and manage an alarm information assignment rule including an alarm information issuing manner, including:
the execution submodule acquires the historical execution time and the alarm time of the alarm task according to the task execution condition and the alarm rule, judges whether the alarm needs to be executed or not according to the execution frequency and the alarm frequency, and triggers a corresponding alarm instruction if the data quantity which is in accordance with the configuration table and the filtering condition statistics of the alarm task is greater than an alarm threshold value; and the message management module splices the information content of the alarm information according to the alarm instruction and sends the information content down through a short message and/or a mailbox.
10. The traffic data warning method of the traffic data monitoring warning apparatus according to claim 1, wherein the method comprises:
the task scheduling module generates a task queue for scheduling tasks;
the queue management module loads the whole amount of a single scheduling task in the same time to a Redis queue, and acquires an alarm task group from the Redis queue according to the alarm rule management module;
the execution management module acquires a Spring container according to the benName, determines a proxy object from the Spring container, acquires an actuator of the proxy object, substitutes execution parameters and the like for execution, judges whether an alarm task reaches an alarm threshold according to the alarm rule, and returns the alarm task and the alarm message meeting the alarm threshold to the message management module;
the message management module carries out alarm information content splicing and alarm event aggregation and issues the alarm information through the alarm information distribution rule of mails, short messages or WeChat.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010006684.1A CN111190798A (en) | 2020-01-03 | 2020-01-03 | Service data monitoring and warning device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010006684.1A CN111190798A (en) | 2020-01-03 | 2020-01-03 | Service data monitoring and warning device and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111190798A true CN111190798A (en) | 2020-05-22 |
Family
ID=70708733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010006684.1A Pending CN111190798A (en) | 2020-01-03 | 2020-01-03 | Service data monitoring and warning device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111190798A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732763A (en) * | 2021-01-20 | 2021-04-30 | 北京千方科技股份有限公司 | Data aggregation method and device, electronic equipment and medium |
| CN113034056A (en) * | 2021-05-06 | 2021-06-25 | 广东鹰视能效科技有限公司 | Early warning identification method and system |
| CN113205666A (en) * | 2021-05-06 | 2021-08-03 | 广东鹰视能效科技有限公司 | Early warning method |
| CN113341739A (en) * | 2021-05-26 | 2021-09-03 | 城家酒店管理有限公司 | Intelligent device management method, system and application device |
| CN113377608A (en) * | 2021-05-19 | 2021-09-10 | 广州有信科技有限公司 | Method and device for alarming task abnormity, terminal equipment and storage medium |
| CN113518006A (en) * | 2021-06-30 | 2021-10-19 | 广东省安心加科技有限公司 | Method and system for processing alarm message of Internet of things equipment |
| CN113657863A (en) * | 2021-08-23 | 2021-11-16 | 交通银行股份有限公司 | Centralized operation processing system and method thereof |
| CN113765685A (en) * | 2020-06-05 | 2021-12-07 | 腾讯科技（深圳）有限公司 | Abnormity management method and device |
| CN114422339A (en) * | 2022-03-29 | 2022-04-29 | 西安塔力科技有限公司 | Automatic scheduling distributed data monitoring system and method |
| CN115599519A (en) * | 2022-10-17 | 2023-01-13 | 安芯网盾（北京）科技有限公司（Cn） | Job scheduling method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104156265A (en) * | 2014-08-08 | 2014-11-19 | 乐得科技有限公司 | Timed task processing method and processing device |
| WO2016177156A1 (en) * | 2015-07-16 | 2016-11-10 | 中兴通讯股份有限公司 | Traffic processing method, device and system |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
-
2020
- 2020-01-03 CN CN202010006684.1A patent/CN111190798A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104156265A (en) * | 2014-08-08 | 2014-11-19 | 乐得科技有限公司 | Timed task processing method and processing device |
| WO2016177156A1 (en) * | 2015-07-16 | 2016-11-10 | 中兴通讯股份有限公司 | Traffic processing method, device and system |
| CN106681882A (en) * | 2015-11-06 | 2017-05-17 | 上海瑞致软件有限公司 | IT-service concentrated monitoring and managing system based on Apriori algorithm |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113765685A (en) * | 2020-06-05 | 2021-12-07 | 腾讯科技（深圳）有限公司 | Abnormity management method and device |
| CN112732763A (en) * | 2021-01-20 | 2021-04-30 | 北京千方科技股份有限公司 | Data aggregation method and device, electronic equipment and medium |
| CN113034056A (en) * | 2021-05-06 | 2021-06-25 | 广东鹰视能效科技有限公司 | Early warning identification method and system |
| CN113205666A (en) * | 2021-05-06 | 2021-08-03 | 广东鹰视能效科技有限公司 | Early warning method |
| CN113377608A (en) * | 2021-05-19 | 2021-09-10 | 广州有信科技有限公司 | Method and device for alarming task abnormity, terminal equipment and storage medium |
| CN113341739A (en) * | 2021-05-26 | 2021-09-03 | 城家酒店管理有限公司 | Intelligent device management method, system and application device |
| CN113518006A (en) * | 2021-06-30 | 2021-10-19 | 广东省安心加科技有限公司 | Method and system for processing alarm message of Internet of things equipment |
| CN113657863A (en) * | 2021-08-23 | 2021-11-16 | 交通银行股份有限公司 | Centralized operation processing system and method thereof |
| CN114422339A (en) * | 2022-03-29 | 2022-04-29 | 西安塔力科技有限公司 | Automatic scheduling distributed data monitoring system and method |
| CN115599519A (en) * | 2022-10-17 | 2023-01-13 | 安芯网盾（北京）科技有限公司（Cn） | Job scheduling method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111190798A (en) | Service data monitoring and warning device and method | |
| CN110377413B (en) | Distributed task asynchronous scheduling and monitoring system based on BPMN standard | |
| CN106406993A (en) | Timed task management method and system | |
| CN106528853B (en) | Data interaction managing device, inter-library data interaction processing unit and method | |
| CN110611707B (en) | Task scheduling method and device | |
| CN107168789B (en) | Multitask serial scheduling method and device | |
| CN107066339A (en) | Distributed job manager and distributed job management method | |
| CN110955506A (en) | Distributed job scheduling processing method | |
| CN105653362A (en) | Method and equipment for managing timed tasks | |
| CN111258741B (en) | Warehouse task execution method, distributed server cluster and computer equipment | |
| CN115525406A (en) | Distributed asynchronous task scheduling system | |
| CN113119127A (en) | Task scheduling method and system for RPA robot | |
| CN110113257B (en) | Unified data access gateway based on big data and implementation method | |
| CN112748990A (en) | Quartz-based data quality task execution method and device and computer equipment | |
| CN110780869A (en) | Distributed batch scheduling | |
| CN115712572A (en) | Task testing method and device, storage medium and electronic device | |
| CN109286610A (en) | Service blocking apparatus, method, equipment and storage medium | |
| CN115617480A (en) | Task scheduling method, device and system and storage medium | |
| CN114697192A (en) | Skywalking-based application performance index alarm system | |
| CN110011832B (en) | Configuration issuing method and device for planned tasks | |
| CN113419835A (en) | Job scheduling method, device, equipment and medium | |
| CN114661432A (en) | Task scheduling method, device, equipment and storage medium | |
| CN113641509A (en) | Internet of things data processing method and device | |
| CN117707794B (en) | Heterogeneous federation-oriented multi-class job distribution management method and system | |
| CN109919749A (en) | A kind of account checking method, system, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200522 |