CN112256533A - Data monitoring alarm processing method, device, medium and electronic equipment - Google Patents

Data monitoring alarm processing method, device, medium and electronic equipment Download PDF

Info

Publication number
CN112256533A
CN112256533A CN201911066387.XA CN201911066387A CN112256533A CN 112256533 A CN112256533 A CN 112256533A CN 201911066387 A CN201911066387 A CN 201911066387A CN 112256533 A CN112256533 A CN 112256533A
Authority
CN
China
Prior art keywords
alarm
data
rule
hit
hit rule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911066387.XA
Other languages
Chinese (zh)
Inventor
辛州
刘敬斌
江龙飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Zhenshi Information Technology Co Ltd
Original Assignee
Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Zhenshi Information Technology Co Ltd filed Critical Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority to CN201911066387.XA priority Critical patent/CN112256533A/en
Publication of CN112256533A publication Critical patent/CN112256533A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/32Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324Display of status information
    • G06F11/327Alarm or error message display

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Alarm Systems (AREA)

Abstract

The invention provides a processing method, a device, a medium and electronic equipment for data monitoring alarm, wherein the method comprises the following steps: and scanning the collected service data according to at least one hit rule registered by the user, generating alarm information if the first hit rule is scanned and sending an alarm notification to at least one user subscribed with the first hit rule according to the alarm information. Through the mode, data are collected, scanning and alarm notification are carried out according to the hit rules of user registration which are not used, the existing resources are integrated, the specific monitoring details can be customized, different scanning requirements can be customized through simple statements, the implementation is simple, and the method can be flexibly applied to different scanning requirements.

Description

Data monitoring alarm processing method, device, medium and electronic equipment
Technical Field
The invention relates to the technical field of data analysis, in particular to a data monitoring alarm processing method, a data monitoring alarm processing device, a data monitoring alarm processing medium and electronic equipment.
Background
In the development process of software, developers need to monitor data in different types according to different requirements, and obtain prompts when monitoring conditions are met, namely, alarms need to be given.
Currently, there is a heavy weight solution in the industry by adding a fixed format execution log to the logic of application execution. Then, big data analysis is carried out by open source software such as kafka, hadoop, and ElasticSearch. And alarming after the retrieval hits the established execution log. In addition, a lighter monitoring alarm solution is provided, namely a plug-in (such as a Sigar plug-in) is implanted into a target machine, then the plug-in scans runtime state parameters such as a memory and a CPU of an application process at regular time, and sends an alarm if sensitive data is found.
However, the first scheme is based on cooperative work of numerous heavyweight open source software, and involves real-time data streaming calculation, the architecture is generally difficult to avoid, bulky and heavy, and is not easy to implement, the second scheme focuses on processing stable or unchangeable index monitoring, and if the index to be monitored is biased to an application layer, and the access party is different, the scene is different, and the corresponding index may change, frequent plug-in modification is required. In summary, the two schemes are complex to implement, cannot be processed universally, and are poor in flexibility.
Disclosure of Invention
The embodiment of the invention provides a processing method, a processing device, a processing medium and electronic equipment for data monitoring alarm, and aims to solve the problems that in the prior art, the implementation is complex, the universal processing cannot be realized, and the flexibility is poor.
In a first aspect, an embodiment of the present invention provides a method for processing a data monitoring alarm, where the method includes:
scanning the collected business data according to at least one hit rule; the at least one hit rule is a received rule of user registration;
if the first hit rule is scanned and hit, alarm information is generated;
and sending an alarm notification to at least one user subscribed to the first hit rule according to the alarm information.
In a specific implementation, the method further includes:
acquiring and acquiring the service data from a service party according to a pre-configured data dimension;
alternatively, the first and second electrodes may be,
and acquiring the service data from a service party according to the pre-configured data dimension and the acquisition frequency.
In a specific implementation, the method further includes:
receiving an alarm rule configured by a service party on service data, wherein the alarm rule comprises: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
In a specific implementation manner, if the alarm rule includes a scanning frequency, the scanning the collected traffic data according to at least one hit rule includes:
and scanning the collected business data according to the scanning frequency according to the at least one hit rule.
In a specific implementation, the method further includes:
and receiving a subscription message sent by at least one terminal device, wherein the subscription message comprises a user identifier, a subscription hit rule and a notification mode.
In a specific implementation manner, the sending, according to the alarm information, an alarm notification to at least one user subscribed to the first hit rule includes:
sending an alarm notification to each user subscribing to the first hit rule according to the alarm information and the user identifier subscribing to the first hit rule and a corresponding notification mode;
alternatively, the first and second electrodes may be,
and sending an alarm notification to each user subscribed with the first hit rule according to the alarm information and the user identifier subscribed with the first hit rule, and according to a corresponding notification mode and a preset notification period.
In a second aspect, an embodiment of the present invention provides a data monitoring alarm processing apparatus, where the method includes:
the processing module is used for scanning the collected business data according to at least one hit rule; the at least one hit rule is a received rule of user registration;
the processing module is further used for generating alarm information if the first hit rule is scanned;
and the notification module is used for sending an alarm notification to at least one user subscribing the first hit rule according to the alarm information.
Optionally, the apparatus further comprises: a data acquisition module to:
acquiring and acquiring the service data from a service party according to a pre-configured data dimension;
alternatively, the first and second electrodes may be,
and acquiring the service data from a service party according to the pre-configured data dimension and the acquisition frequency.
Optionally, the apparatus further comprises:
a first receiving module, configured to receive an alarm rule configured by a service party for service data, where the alarm rule includes: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
Optionally, if the alarm rule includes a scanning frequency, the processing module is specifically configured to:
and scanning the collected business data according to the scanning frequency according to the at least one hit rule.
Optionally, the apparatus further comprises:
and the second receiving module is used for receiving a subscription message sent by at least one terminal device, wherein the subscription message comprises a user identifier, a subscription hit rule and a notification mode.
Optionally, the notification module is specifically configured to:
sending an alarm notification to each user subscribing to the first hit rule according to the alarm information and the user identifier subscribing to the first hit rule and a corresponding notification mode;
alternatively, the first and second electrodes may be,
and sending an alarm notification to each user subscribed with the first hit rule according to the alarm information and the user identifier subscribed with the first hit rule, and according to a corresponding notification mode and a preset notification period.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
a receiver, a processor, a transmitter; and the number of the first and second groups,
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the method of processing a data monitoring alert of any of the first aspect via execution of the executable instructions.
In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the processing method for data monitoring alarm according to any one of the first aspect.
According to the data monitoring alarm processing method, the data monitoring alarm processing device, the data monitoring alarm processing medium and the electronic device, the electronic device scans collected business data according to at least one hit rule registered by a user, generates alarm information if the first hit rule is scanned, and finally sends an alarm notification to at least one user subscribing the first hit rule according to the alarm information. Through the mode, data are collected, scanning and alarm notification are carried out according to the hit rules of user registration which are not used, the existing resources are integrated, the specific monitoring details can be customized, different scanning requirements can be customized through simple statements, the implementation is simple, and the method can be flexibly applied to different scanning requirements.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a first embodiment of a processing method for data monitoring alarm provided by the present invention;
FIG. 2 is a schematic diagram of an application framework of a processing method for data monitoring alarm provided by the present invention;
FIG. 3 is a schematic flow chart of a second embodiment of a data monitoring alarm processing method according to the present invention;
FIG. 4 is a schematic structural diagram of a first embodiment of a data monitoring alarm processing apparatus according to the present invention;
FIG. 5 is a schematic structural diagram of a second embodiment of a data monitoring alarm processing apparatus according to the present invention;
fig. 6 is a schematic structural diagram of a third embodiment of a data monitoring alarm processing apparatus provided in the present invention;
fig. 7 is a schematic structural diagram of a fourth embodiment of a data monitoring alarm processing apparatus provided in the present invention;
fig. 8 is a schematic structural diagram of an electronic device shown in accordance with an example embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments based on the embodiments in the present invention, which can be made by those skilled in the art in light of the present disclosure, are within the scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the two existing solutions provided at present, the first one is, although the function is complete, established on the basis of cooperative work of numerous heavyweight open source software, and involves real-time data streaming calculation, the architecture is generally difficult to avoid being numerous and heavy, and the dependence on server resources, the quality of the whole developer and the investment of operation and maintenance personnel are very large. The second, plug-in approach, generally focuses on handling stable, or constant, index monitoring, such as: and monitoring the state of the application in the runtime. If the indexes needing to be monitored are biased to an application layer, and corresponding indexes may change due to different access parties and different scenes, the scheme may involve frequent plug-in modification. General processing cannot be achieved, and flexibility is lacked.
In order to solve the above problems, the present invention provides a method for processing data monitoring alarm, and mainly provides a solution for a system, and in the specific implementation of the solution, a distributed framework is required, for example, the solution can be implemented on a kyoto jiff platform, and can also be implemented based on Dubbo and spring cloud. The existing resources are integrated with minimum investment without depending on other additional components, and the problem of resource and personnel re-investment brought by the traditional log scanning and big data analysis scheme is eliminated. In addition, in the time limit of the scheme, the monitoring details are customized, all access parties respectively configure concerned dimensions according to needs without mutual interference, and alarm monitoring services from a system operation time layer to a service application layer are perfectly fused.
The data monitoring and alarming scheme provided by the invention is explained in detail by several specific embodiments.
FIG. 1 is a schematic flow chart of a first embodiment of a processing method for data monitoring alarm provided by the present invention; as shown in fig. 1, the scheme can be applied to a system side, that is, a system for providing services to each user and a service party, an execution main body may be an electronic device such as a computer, a server, a cloud server, and the like, and the processing method of the data monitoring alarm specifically includes the following steps:
s101: scanning the collected business data according to at least one hit rule; at least one of the hit rules is a received user registration rule.
In this step, for the executing entity of the scheme, in order to provide the business party and the user who needs to perform data analysis alarm, only the requirement needs to be provided, that is, the data is provided to the system of the scheme, the user needs to provide the analysis alarm of which hit rules need to be performed on which data to the system of the scheme, and the system scans the data according to different hit rules.
In the specific implementation of the scheme, the service data needs to be collected first, the data collection is specifically performed according to what dimension, a user or a service party can perform declaration and authorization from a system interface provided by the scheme in advance, and in the system operation process, relevant service data is acquired from the service party according to the declared data dimension. The hit rules to be scanned may be different for different users, and the number of hit rules to be scanned is also different, so for a specific scan rule, according to various different requirements, a user may design in advance to make a selective subscription in the system, and select to subscribe to one or more hit rules. The user may also input one or more hit rules for subscription according to a subscription interface provided by the system, which is not limited to this scheme.
After the business data are obtained, the system scans according to the hit rule corresponding to the business data. The same service data may correspond to one or more hit rules, the service data may be scanned by the hit rules one by one in the scheme, or the service data may be scanned according to all the hit rules at the same time, which is not limited in this scheme.
S102: and if the first hit rule is scanned and hit, generating alarm information.
In this scheme, the system may not hit all rules during the process of scanning the service data, and therefore, during the scanning process, if there is a first hit rule hit, alarm information is generated, where the alarm information mainly includes data information that satisfies the first hit rule in the service data, including but not limited to: dimensions of data, etc.
S103: and sending an alarm notification to at least one user subscribed to the first hit rule according to the alarm information.
In this step, for the system, after the service data is scanned to satisfy the first hit rule, the corresponding alarm information is generated, and then an alarm notification needs to be performed to at least one user who subscribes to the first hit rule. The specific way of performing the alarm notification may be to send a notification message to the client through the user identifier, or to perform the notification according to the information of the user, such as calling, sending mail, short message, and wechat. This solution is not limited.
Optionally, in a specific implementation of the scheme, after the alarm information is generated, the system may perform an alarm notification to each user subscribed to the first hit rule in real time, or perform an alarm notification to each user subscribed to the first hit rule at a certain time interval through a preset frequency or period.
In a specific implementation, the processing method for data monitoring alarm provided by this embodiment integrates the existing resources by collecting the service data and performing scanning and alarm notification according to the hit rules of different user registrations, customizes specific monitoring details, can simply declare and customize different scanning requirements, is simple to implement, and can be flexibly applied to different scanning requirements.
FIG. 2 is a schematic diagram of an application framework of a processing method for data monitoring alarm provided by the present invention; as shown in fig. 2, based on the above embodiment, it can be seen that the processing method for data monitoring and alarm provided by the present invention mainly relates to two interaction bodies, namely, a user side and a system service side, and the service side of the system mainly implements the following functional modules:
data acquisition, data scanning, alarm rules, alarm information and alarm handling.
In the time limit of the modules, data acquisition is mainly used for acquiring data from a service party, different collectors can be set for different service data, namely the collectors are registered in advance, and data dimensions of the data to be acquired are declared, so that the data acquired by the collectors can correspond to the corresponding data dimensions, for example, Hash data can be formed for storage. In addition, the work of the collector can be performed in real time or at a certain frequency, and if data collection needs to be performed at a certain frequency, the collection frequency (also referred to as collection frequency) needs to be declared in advance to determine how often the system initiates a collection request or how often a service party reports service data.
The alarm rule is mainly used for registering the rule, and for the same service data, at least one hit rule needs to be declared, namely after the service data is collected, the alarm is carried out when the service data meets a certain relation, for example, a mathematical relation between a data dimension and a dimension value. The system may follow such a rule to perform a data scan to determine whether an alarm is hit. Each user in the system can customize one or more hit rules, each business data can be scanned according to the one or more hit rules, and the business data and the hit rules and the relation between the user and the hit rules need to be stored.
Optionally, while declaring the hit rule, the frequency of alarm or the frequency of data scanning may also be declared, that is, the frequency of scanning the service data table in the current hit rule cascade.
And the module is mainly used for scanning the data according to some hit rules corresponding to each service data to obtain corresponding alarm information. After the functions of the above two modules are completed, the data scanning module can also generate corresponding scanning tasks, or timing the scanning tasks.
The module mainly generates and stores the result of data scanning, generates alarm information after the scanning is hit, also called alarm source data, and the function of the module can be realized independently or in an alarm disposal or data scanning module.
The alarm processing module is mainly used for realizing the function of alarming the generated alarm information, and the specific alarm notification mode, the alarm notification time and the like can be realized in the module.
The user side can provide an interface or directly provide a client side for operation, and mainly realizes the following functional modules: rule subscriptions, alert services, and user information.
The user information is mainly used for realizing the function of acquiring the user information, and the user needs to use the service system and needs to register, namely at least needs to acquire the identifier of the user so that a subsequent system can know to which user to send an alarm notification.
The rule subscription mainly provides a function that a user can select and subscribe different rules according to requirements, wherein the user can directly access a plurality of rules provided in the system to select and subscribe, and the user can declare a hit rule needing scanning through a system interface without limitation.
The alarm service is mainly used for realizing the function of alarm notification, and various different templates can be directly provided in the system in the alarm service, and the user selects the template to determine which mode to alarm according to, or the user directly declares a specific alarm notification mode through a system interface, so that the method is not limited.
On the basis of any of the foregoing solutions, fig. 3 is a schematic flow diagram of a second embodiment of a processing method for data monitoring alarm provided by the present invention, and as shown in fig. 3, in a specific implementation manner of the solution, the processing method for data monitoring alarm specifically includes the following steps:
s201: data is collected from the business party.
In this step, the system may have different collectors for different services, and when the registration of data collection frequency, data dimension, and the like is completed, the system may collect data from the service side, and the specific data collection manner at least includes the following:
firstly, acquiring and acquiring the service data from a service party according to a preconfigured data dimension;
if the acquisition frequency is not configured, the data acquisition is carried out from the service side according to the preset data dimensions.
And secondly, acquiring the service data from a service party according to a pre-configured data dimension and acquisition frequency.
If the data dimension and the collection frequency are configured at the same time, the data corresponding to the data dimension can be periodically acquired from the service party according to the collection frequency.
S202: and receiving the alarm rule (namely the hit rule) configured by the service party on the service data.
In this step, the alarm rules (i.e., the hit rules) in the system may be configured by the service party, and the service party may send different hit rules to the service system according to the operation rules of the data and different detection requirements, and the service system stores the rules in the system and provides an interface for the user to select.
In the implementation of the scheme, the alarm rule comprises: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
Optionally, the hit rules may also be declared by the user, and the scheme may also be implemented to receive at least one alarm rule sent by the terminal device of the user, where the alarm rule includes one or more hit rules and may also include a scanning frequency.
S203: and receiving a subscription message sent by at least one terminal device.
In this step, in order to obtain different alarms according to the needs of the user, the user needs to subscribe to a corresponding hit rule, and in a specific implementation, the user may subscribe to the system by sending a subscription message to the system through the terminal device, where the subscription message includes at least one of a user identifier, a hit rule of the subscription, a notification manner, and the like.
After the system receives the subscription message of each user, the information in the subscription message is respectively stored so as to inform the user in a user subscription mode after subsequent scanning.
The implementation sequence of the above steps S201, S202, and S203 is not limited, and may be adjusted according to specific implementations and applications.
S204: the collected traffic data is scanned according to at least one hit rule.
In this step, after the system completes the above steps, the system scans the collected service data according to the hit rules, and detects whether the data satisfies at least one hit rule.
In this process, if the scanning frequency is set, the collected traffic data may be scanned according to the scanning frequency to determine whether the at least one hit rule is met.
S205: and sending an alarm notification to at least one user subscribed to the first hit rule according to the generated alarm information.
In the specific implementation of this step, the implementation can be performed in the following two ways:
in a first implementation manner, according to the alarm information and the user identifier subscribed to the first hit rule, an alarm notification is sent to each user subscribed to the first hit rule according to a corresponding notification manner. The meaning of the scheme is that after a first hit rule subscribed by a user is hit, an alarm notification is sent to the user according to a notification mode selected or input in the subscription process of the user.
In a second implementation manner, according to the alarm information and the user identifier subscribed to the first hit rule, an alarm notification is sent to each user subscribed to the first hit rule according to a corresponding notification manner and a preset notification period.
Different from the above, in this scheme, a notification period, also referred to as a notification frequency, is further provided, and after the scan finds that the first hit rule subscribed by the user is hit, the notification may be performed immediately, or after the notification time expires, according to a certain period, or according to a certain notification frequency.
The data monitoring alarm processing method provided by any embodiment of the above embodiments supports flexible customization of business party and alarm monitoring business, places the type to be monitored below the user, and the user configures the type on the system platform according to the needs of the user, thereby realizing monitoring of business data without additional software resources and reducing cost.
On the basis of any of the above embodiments, the following takes the implementation of the system based on the kyoto jiff distributed environment using JAVA language as an example, and the scheme is specifically described.
In the design of the scheme, a distributed framework (for example, a kyoto jifu platform is taken as an example, and the scheme can be realized based on Dubbo and spring cloud) is required to be relied on in software resources. No other additional components need to be relied upon. Aims to integrate the existing resources with minimum investment and abandon the problem of resource and personnel re-investment brought by the traditional scheme of log scanning and big data analysis. The monitoring details are customized, and all access parties respectively configure concerned dimensions according to needs without mutual interference. And the alarm monitoring service from the system operation time layer to the service application layer is perfectly fused. In one implementation, the following aspects are included.
First, the system architecture of the scheme
As shown in fig. 2, in the design of the present solution, on the service side of the system, the following functional modules are mainly implemented: data acquisition, data scanning, alarm rules, alarm information and alarm handling. The user side can provide an interface or directly provide a client side for operation, and mainly realizes the following functional modules: rule subscriptions, alert services, and user information.
Wherein, the system side includes: a system operation layer: the index indicates whether the application program normally runs or not when the application program runs. Emphasis is placed on describing basic information such as health status indicators of application processes. And (3) a service application layer: which refers to a service index generated by an application when executing a specific service. Conceptually belonging to the upper layer of the system operation time layer.
In the implementation of the scheme, it should be understood that the data dimension refers to an index item concerned by the alarm monitoring system. The index item is customized by the user of the access party. Acquisition frequency (also referred to as acquisition frequency): and quantifying the initiating frequency of the acquisition task. Specifically, cron expression, such as: once per minute, or for some period of time per day. The scan frequency (also referred to as the scan frequency) is the frequency of initiation of the quantitative scan task. The hit rule is used to indicate that the data dimension and the corresponding value are considered as alarm data when they meet a certain mathematical relationship. A data source: i.e. the source of the monitoring alarm data, currently supporting HTTP, INTERFACE types, i.e. the business side.
Second, system time sequence
The main interaction time sequence of the data monitoring system is briefly described as follows:
<1>, register the alarm collector. Firstly, the name of the data dimension to be collected in the future needs to be declared so as to carry out the pair numbering of the collected data and the target dimension to form the Hash data. Second, the acquisition frequency (cron expression) needs to be declared, i.e., tells the system how often to initiate an acquisition request.
<2>, register alarm rules. Firstly, a hit rule needs to be declared, namely, a mathematical relationship between a dimension name and a dimension value is acquired after alarm data is acquired. This is used to determine whether an alarm is hit. Second, the alarm scan frequency (cron expression), i.e., the frequency of scanning alarm data tables cascaded with the current alarm rule, needs to be declared.
And 3, after the first two steps are finished, the system respectively generates an acquisition timing task and an alarm scanning timing task. The former is used for regularly pulling service data and generating alarm source data. The latter is used for scanning alarm source data regularly, and after the scanning, alarm notification and data archiving are initiated.
In the implementation of the above scheme, the design of the system at least needs to include several functions of a database, a data acquisition module, an alarm rule module, and an alarm handling module.
(1) The database is used for storing collected data, user information, different customization rules, user subscription information and the like, can be stored in different storage areas, can be stored respectively through different identifications, can also be stored through a database table collector, and is not limited in the scheme.
(2) Data acquisition module
The data acquisition module is combined with a database table collector and explained as follows:
id: a unique identifier;
collector _ name: the name of the collector;
ds _ type: data source type (only INTERFACE type is presented below).
The time limit of the system may also support the following types:
the HTTP type. That is, the access party issues http service, and then the alarm system makes a call to acquire the data issued by the access party. This type is suitable for any application access capable of issuing http service, such as Python, nodjs-based applications, SpringCloud applications, and ElasticSearch-related applications.
The INTERFACE type, which may also be understood as RPC type, is an INTERFACE declared by the alarm system (refer to the following code fragment), which is then implemented by the access side and issued by the access side. And the alarm platform is called by a timing task. This type is suitable for all applications within the Dubbo ecology, as well as the Kjeff frame ecology of Kyoto.
The following may be defined in the following manner, taking the kyoto jeff framework as an example:
ds _ detail: data source detail. This field is associated with ds _ type.
When ds _ type is the HTTP type, this field is the request address of HTTP.
When ds _ type is an INTERFACE type, this field is a service alias issued by a service party after implementing an AlarmCollector INTERFACE according to its own service.
cron: the alarm platform creates a timing task based on this field to initiate data collection at regular time.
metric _ 1: the data dimension name, that is, the access party accessing the alarm system, needs to inform which indexes are monitored in the target scene. matrix _2, matrix _3 and matrix _4 are the same. After the data are collected by the collection timing task, the data are matched with the dimensionality, meanwhile, expressions of the expression fields of all alarm rules hung under the collector are operated, and if the expressions are hit, the expressions are put in a warehouse (an alarm _ data table).
(3) Alarm rule module
The alarm module is combined with a database table alarm and explained as follows:
id: unique identification
collector _ id: the rule is externally connected with a collector.
Call _ name: rule name
cron: the frequency. The alarm platform will create a timed task based on this field to periodically initiate an alarm data scan. When the data is scanned (alias _ data), the user is notified and the data is archived (archiving to the alias _ history data table).
expression: and (4) regular expressions. The Juel expression can be used to identify a system of dimensions and values as a hit algorithm for an alarm rule. The expression is called at the time of data collection. For example: in a certain scene, the data dimensions are respectively set as follows:
metric_1="app_name",
metric_2="level",
metric_3="count",
metric_4="event_name"。
then the following expression forms a rule,
${app_name=="ql-e2e-worker"&&level=="3"&&count>5&&event_name=='C2CNodeProcessorProxy.executeBusiness'}
that is, in the collected data, if app _ name ═ ql-e2e-worker, "level ═ 3," count >5, and event _ name ═ c2cnodeprocessorproxy.
(4) Alarm processing module
The main table of the alarm handling module is the advicer data table, which is a service-oriented way of identifying a notification, including the notification content (mehtod table) and notifying those persons (subscribers). The method is realized by two additional sub-tables, namely, subsciber and method. External relations, see 3.2.2 database design.
A first sub-table: each piece of data in the list is associated with one entity user, the advicer list and the subscber list are in a one-to-many relationship, namely when a notification operation is triggered, a plurality of users can receive an alarm notification.
Second, sub-table: a method notification mode table stating the notification mode and the corresponding content received by the user when the notification operation is triggered. The advicer table and the method table are in a one-to-many relationship. The method table is associated with another table template, which may be associated with a message template. Different message templates can be created in the template table according to the sending modes (short messages and mails). The template supports the Velocity syntax.
In the system realized by the scheme, the flexible customization of the access service party and the alarm monitoring service is supported. And the specific content to be monitored is released to the access user side. That is, if the user needs to monitor the system runtime state (the alarm platform encapsulates the unified system runtime plug-in), the system runtime parameters can be configured to the alarm system. And if the user needs to monitor the application service data, similarly, the user can go to the system platform configuration. The scheme is based on the ecology of Remote Procedure Call (RPC) of a main stream, and the alarm monitoring function is completed through common RPC interaction. The method is perfectly suitable for Dubbo application and Jeff application. No additional software resources are required. Greatly reducing the cost.
FIG. 4 is a schematic structural diagram of a first embodiment of a data monitoring alarm processing apparatus according to the present invention; as shown in fig. 4, the data monitoring alarm processing device 10 includes:
the processing module 11 is configured to scan the collected service data according to at least one hit rule; the at least one hit rule is a received rule of user registration;
the processing module 11 is further configured to generate alarm information if the first hit rule is scanned;
and a notification module 12, configured to send an alarm notification to at least one user subscribed to the first hit rule according to the alarm information.
The data monitoring alarm processing device provided by the embodiment is used for realizing the technical scheme of any one of the method embodiments, the realization principle and the technical effect are similar, and through the method, data are collected and are scanned and alarmed according to the hit rule of the user registration which is not used, the existing resources are integrated, the specific monitoring details are customized, different scanning requirements can be simply stated to be customized, the implementation is simple, and the data monitoring alarm processing device can be flexibly applied to different scanning requirements.
FIG. 5 is a schematic structural diagram of a second embodiment of a data monitoring alarm processing apparatus according to the present invention; on the basis of the above embodiment, as shown in fig. 5, the data monitoring alarm processing device 10 further includes: a data acquisition module 13 configured to:
acquiring and acquiring the service data from a service party according to a pre-configured data dimension;
alternatively, the first and second electrodes may be,
and acquiring the service data from a service party according to the pre-configured data dimension and the acquisition frequency.
Fig. 6 is a schematic structural diagram of a third embodiment of a data monitoring alarm processing apparatus provided in the present invention; on the basis of the above embodiment, as shown in fig. 6, the data monitoring alarm processing device 10 further includes:
a first receiving module 14, configured to receive an alarm rule configured by a service party for service data, where the alarm rule includes: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
Optionally, if the alarm rule includes a scanning frequency, the processing module 11 is specifically configured to:
and scanning the collected business data according to the scanning frequency according to the at least one hit rule.
Fig. 7 is a schematic structural diagram of a fourth embodiment of a data monitoring alarm processing apparatus provided in the present invention; on the basis of the above embodiment, as shown in fig. 7, the data monitoring alarm processing device 10 further includes:
a second receiving module 15, configured to receive a subscription message sent by at least one terminal device, where the subscription message includes a user identifier, a hit rule of subscription, and a notification manner.
Optionally, the notification module 12 is specifically configured to:
sending an alarm notification to each user subscribing to the first hit rule according to the alarm information and the user identifier subscribing to the first hit rule and a corresponding notification mode;
alternatively, the first and second electrodes may be,
and sending an alarm notification to each user subscribed with the first hit rule according to the alarm information and the user identifier subscribed with the first hit rule, and according to a corresponding notification mode and a preset notification period.
The implementation principle and the technical effect of the processing device for data monitoring and alarming provided by any of the embodiments are similar, and are not described herein again.
Fig. 8 is a schematic structural diagram of an electronic device shown in accordance with an example embodiment of the present invention. As shown in fig. 8, the present embodiment provides an electronic device 20 including:
a receiver 21, a processor 22, a transmitter 23; and the number of the first and second groups,
a memory 24 for storing executable instructions of the processor;
wherein the processor 22 is configured to execute the processing method of data monitoring alarm provided by any of the foregoing method embodiments via executing the executable instructions.
The memory 24 may also be a flash (flash memory);
wherein the processor 22 is configured to perform the steps of the above-described method via execution of the executable instructions. Reference may be made in particular to the description relating to the preceding method embodiment.
Alternatively, the memory 24 may be separate or integrated with the processor 22.
When the memory 24 is a device separate from the processor 22, the electronic device may further include:
a bus for connecting the processor 22 and the memory 24.
The embodiment also provides a readable storage medium, in which a computer program is stored, and when at least one processor of the electronic device executes the computer program, the electronic device executes the processing method for data monitoring and alarming provided by the above various embodiments.
The present embodiment also provides a program product comprising a computer program stored in a readable storage medium. The computer program can be read from a readable storage medium by at least one processor of the electronic device, and the computer program can be executed by the at least one processor to enable the electronic device to implement the processing method for data monitoring alarm provided by the various embodiments described above.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (14)

1. A processing method for data monitoring alarm is characterized by comprising the following steps:
scanning the collected business data according to at least one hit rule; the at least one hit rule is a received rule of user registration;
if the first hit rule is scanned and hit, alarm information is generated;
and sending an alarm notification to at least one user subscribed to the first hit rule according to the alarm information.
2. The method of claim 1, further comprising:
acquiring and acquiring the service data from a service party according to a pre-configured data dimension;
alternatively, the first and second electrodes may be,
and acquiring the service data from a service party according to the pre-configured data dimension and the acquisition frequency.
3. The method of claim 1, further comprising:
receiving an alarm rule configured by a service party on service data, wherein the alarm rule comprises: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
4. The method of claim 3, wherein if the alarm rule comprises a scan frequency, the scanning the collected traffic data according to the at least one hit rule comprises:
and scanning the collected business data according to the scanning frequency according to the at least one hit rule.
5. The method according to any one of claims 1 to 4, further comprising:
and receiving a subscription message sent by at least one terminal device, wherein the subscription message comprises a user identifier, a subscription hit rule and a notification mode.
6. The method of claim 5, wherein sending an alert notification to at least one user subscribed to the first hit rule based on the alert information comprises:
sending an alarm notification to each user subscribing to the first hit rule according to the alarm information and the user identifier subscribing to the first hit rule and a corresponding notification mode;
alternatively, the first and second electrodes may be,
and sending an alarm notification to each user subscribed with the first hit rule according to the alarm information and the user identifier subscribed with the first hit rule, and according to a corresponding notification mode and a preset notification period.
7. A data monitoring alarm processing device, the method comprising:
the processing module is used for scanning the collected business data according to at least one hit rule; the at least one hit rule is a received rule of user registration;
the processing module is further used for generating alarm information if the first hit rule is scanned;
and the notification module is used for sending an alarm notification to at least one user subscribing the first hit rule according to the alarm information.
8. The apparatus of claim 7, further comprising: a data acquisition module to:
acquiring and acquiring the service data from a service party according to a pre-configured data dimension;
alternatively, the first and second electrodes may be,
and acquiring the service data from a service party according to the pre-configured data dimension and the acquisition frequency.
9. The apparatus of claim 7, further comprising:
a first receiving module, configured to receive an alarm rule configured by a service party for service data, where the alarm rule includes: at least one hit rule, or, at least one hit rule and a scan frequency; each hit rule comprises a data dimension needing alarming and a dimension value corresponding to the data dimension.
10. The apparatus of claim 9, wherein if the alarm rule comprises a scan frequency, the processing module is specifically configured to:
and scanning the collected business data according to the scanning frequency according to the at least one hit rule.
11. The apparatus of any one of claims 7 to 9, further comprising:
and the second receiving module is used for receiving a subscription message sent by at least one terminal device, wherein the subscription message comprises a user identifier, a subscription hit rule and a notification mode.
12. The apparatus of claim 11, wherein the notification module is specifically configured to:
sending an alarm notification to each user subscribing to the first hit rule according to the alarm information and the user identifier subscribing to the first hit rule and a corresponding notification mode;
alternatively, the first and second electrodes may be,
and sending an alarm notification to each user subscribed with the first hit rule according to the alarm information and the user identifier subscribed with the first hit rule, and according to a corresponding notification mode and a preset notification period.
13. An electronic device, comprising:
a receiver, a processor, a transmitter; and the number of the first and second groups,
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the method of processing a data monitoring alert of any of claims 1 to 6 via execution of the executable instructions.
14. A storage medium on which a computer program is stored, which program, when being executed by a processor, carries out the method of processing a data monitoring alarm according to any one of claims 1 to 6.
CN201911066387.XA 2019-11-04 2019-11-04 Data monitoring alarm processing method, device, medium and electronic equipment Pending CN112256533A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911066387.XA CN112256533A (en) 2019-11-04 2019-11-04 Data monitoring alarm processing method, device, medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911066387.XA CN112256533A (en) 2019-11-04 2019-11-04 Data monitoring alarm processing method, device, medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN112256533A true CN112256533A (en) 2021-01-22

Family

ID=74223794

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911066387.XA Pending CN112256533A (en) 2019-11-04 2019-11-04 Data monitoring alarm processing method, device, medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN112256533A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103617705A (en) * 2013-12-10 2014-03-05 北京邮电大学 Rule-based method and system for alarming of Internet of things
CN103794033A (en) * 2014-02-27 2014-05-14 广州杰赛科技股份有限公司 Monitoring alarm method and device
CN105049270A (en) * 2015-08-31 2015-11-11 北京奇艺世纪科技有限公司 Information processing method, device and system
CN106100902A (en) * 2016-08-04 2016-11-09 腾讯科技(深圳)有限公司 High in the clouds index monitoring method and apparatus
CN108170580A (en) * 2017-11-22 2018-06-15 链家网(北京)科技有限公司 A kind of rule-based log alarming method, apparatus and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103617705A (en) * 2013-12-10 2014-03-05 北京邮电大学 Rule-based method and system for alarming of Internet of things
CN103794033A (en) * 2014-02-27 2014-05-14 广州杰赛科技股份有限公司 Monitoring alarm method and device
CN105049270A (en) * 2015-08-31 2015-11-11 北京奇艺世纪科技有限公司 Information processing method, device and system
CN106100902A (en) * 2016-08-04 2016-11-09 腾讯科技(深圳)有限公司 High in the clouds index monitoring method and apparatus
CN108170580A (en) * 2017-11-22 2018-06-15 链家网(北京)科技有限公司 A kind of rule-based log alarming method, apparatus and system

Similar Documents

Publication Publication Date Title
JP5677455B2 (en) Cloud computing monitoring and management system
KR20220021438A (en) Management method of machine learning model for network data analytics function device
EP3864880B1 (en) Devices and methods for discovering collectable data and analytics data in a network
CN111966762B (en) Index collection method and device
CN110262851B (en) Interface configuration method and device
CN110928934A (en) Data processing method and device for business analysis
CN111443969A (en) Method and device for recording webpage
CN113495820B (en) Anomaly information collecting and processing method and device and anomaly monitoring system
CN112269902A (en) Data acquisition method for big data
CN110737655B (en) Method and device for reporting data
CN115309767A (en) Data fusion method and device
CN113364670B (en) Data pushing method and device and electronic equipment
CN112256533A (en) Data monitoring alarm processing method, device, medium and electronic equipment
US8510346B2 (en) Efficiently handling information on changes to a UDDI registry including those resulting in virtual deletes
CN111124858A (en) Embedded point-free plug-in data processing method and system, terminal device and storage medium
CN113312242B (en) Interface information management method, device, equipment and storage medium
CN111813765B (en) Method, device, electronic equipment and computer readable medium for processing abnormal data
CN114723397A (en) Flow execution method and device
CN112579325A (en) Business object processing method and device, electronic equipment and storage medium
WO2013049912A1 (en) Notification system
US20240193012A1 (en) Correlation and policy engine system and method of operation
CN113114612B (en) Determination method and device for distributed system call chain
EP4084372A1 (en) Method and apparatus for communication systems comprising management data analytic functions
US9338243B2 (en) Tracking contacts across multiple communications services
CN115174543B (en) Version update release time determining method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination