WO2017107853A1 - Data monitoring management method, and data monitoring method and system - Google Patents

Data monitoring management method, and data monitoring method and system Download PDF

Info

Publication number
WO2017107853A1
WO2017107853A1 PCT/CN2016/110180 CN2016110180W WO2017107853A1 WO 2017107853 A1 WO2017107853 A1 WO 2017107853A1 CN 2016110180 W CN2016110180 W CN 2016110180W WO 2017107853 A1 WO2017107853 A1 WO 2017107853A1
Authority
WO
WIPO (PCT)
Prior art keywords
monitoring
field
rule
data
indicator
Prior art date
Application number
PCT/CN2016/110180
Other languages
French (fr)
Chinese (zh)
Inventor
徐良骥
Original Assignee
阿里巴巴集团控股有限公司
徐良骥
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司, 徐良骥 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2017107853A1 publication Critical patent/WO2017107853A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Definitions

  • the present application relates to data processing technologies in the field of communications, and in particular, to a data monitoring management method, a data monitoring method, and a system.
  • the Data Warehouse (Data Warehouse, DW or DWH for short) provides current and historical data for users to use for decision support.
  • Data warehousing technology is a general term for various technologies and modules that effectively integrate operational data into a unified environment to provide decision-making data access. Based on the data warehouse, users can query the required information more quickly and conveniently, and provide decision support.
  • the data in the data warehouse is processed, summarized and organized on the basis of extracting and cleaning the original distributed database data.
  • it is particularly important to check the data in the data warehouse based on given metrics. Based on the given indicators, the data in the data warehouse can be checked for legality, correctness, and effectiveness.
  • monitoring rules can be configured for critical data tables. After the data of the data table is output, the correctness and validity of the data can be checked according to the configured monitoring rules.
  • the value range of valid data can be used as a monitoring indicator to configure a monitoring rule for a field. If the monitoring indicator of the field is changed (for example, the value range of the valid value changes), or a monitoring indicator needs to be added (for example) The data format of the field needs to be monitored. It is necessary to reconfigure the monitoring rules for all data tables involved in the use of these monitoring indicators, which is cumbersome and easy to miss.
  • the embodiment of the present application provides a data monitoring management method and system.
  • selecting a corresponding monitoring rule for the field from the general rule base including:
  • All or part of the monitoring rules corresponding to the monitoring indicator are selected from the general rule base for the field according to the monitoring indicator used by the field.
  • associating the selected monitoring rule with the field comprises associating the field with index information of the selected monitoring rule in the general rule base.
  • the method further includes: configuring, according to the monitoring indicator used by the field, the field-specific monitoring rule for the field.
  • the method further includes: updating a monitoring rule in the general rule base.
  • the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • An obtaining module configured to acquire a field of a monitoring rule to be configured in the database
  • a selection module configured to select, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from a general rule base; wherein a monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirement of the monitoring indicator ;
  • a configuration module for associating selected monitoring rules with the fields.
  • the selecting module is specifically configured to: according to the monitoring indicator used by the field, All or part of the monitoring rules corresponding to the monitoring indicator are selected in the general rule base for the field.
  • the configuration module is specifically configured to: associate the field with index information of the selected monitoring rule in the general rule base.
  • the configuration module is further configured to configure the field-specific monitoring rule for the field according to the monitoring indicator used by the field.
  • the method further includes: an update module, configured to update the monitoring rule in the general rule base.
  • the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • one or more corresponding monitoring rules are selected from the general rule base for the field, and the selected monitoring rule is selected.
  • This field is associated, which implements the process of configuring the monitoring rules for data. Since the general rule base contains pre-set monitoring rules, when configuring a monitoring rule for a field, you only need to select the appropriate monitoring rule for the field and associate the monitoring rule with the field. When the monitoring rule needs to be adjusted, Simply updating the general rule base without having to re-configure the data table or field to monitor the rules, which simplifies the complexity of monitoring rule configuration and increases system flexibility compared to the prior art.
  • the embodiment of the present application also provides a data monitoring method and system.
  • the data of the field is checked according to the obtained monitoring rule.
  • the monitoring rule associated with the field is obtained from a general rule base, including: And obtaining, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
  • the method further includes: acquiring the field-specific monitoring rule configured for the field, and checking data of the field according to the field-specific monitoring rule.
  • the method further includes: updating a monitoring rule in the general rule base.
  • the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • a first obtaining module configured to acquire a field associated with a monitoring rule in the database
  • a second obtaining module configured to acquire, from a general rule base, a monitoring rule associated with the field
  • the monitoring execution module is configured to check data of the field according to the obtained monitoring rule.
  • the second obtaining module is configured to: obtain, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
  • the first obtaining module is further configured to: obtain the field-specific monitoring rule configured for the field; the monitoring execution module is further configured to: use the field-specific monitoring rule to the field The data is checked.
  • the method further includes: an update module, configured to update the monitoring rule in the general rule base.
  • the monitoring rule associated with the field is all or part of a monitoring rule corresponding to the monitoring indicator used by the field; wherein a monitoring rule corresponding to a monitoring indicator is used to detect Whether the data meets the requirements of this monitoring indicator.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • a field associated with the monitoring rule in the database is obtained, and the monitoring rule associated with the field is obtained from the general rule base, and the field is associated with the general rule base.
  • the monitoring rule is configured for the field by associating the field with the monitoring rule in the general rule base.
  • the general rule base contains preset monitoring rules. When configuring the monitoring rule for a field, just select the appropriate monitoring for the field. Rules, and the monitoring rules are associated with the field. When the monitoring rules need to be adjusted, only the general rule base needs to be updated, and the monitoring rules of the data table or the field need not be reconfigured, compared with the prior art. It simplifies the complexity of monitoring rule configuration and increases system flexibility.
  • FIG. 1 is a schematic diagram of a data monitoring management process provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a data monitoring process provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a data monitoring management system according to an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a data monitoring system according to an embodiment of the present application.
  • a general rule base is preset, and the general rule base includes a preset monitoring rule.
  • a monitoring rule is configured for a field, only a suitable monitoring rule is selected for the field, and the monitoring rule is The fields can be associated.
  • the monitoring rules in the general rule base need to be updated without reconfiguring the monitoring rules of the data tables or fields, which simplifies the monitoring rule configuration compared with the prior art. Complexity, and Increased system flexibility.
  • the embodiments of the present application are applicable to a database system or a data warehouse system.
  • a field or data item In database or data warehousing technology, the nomenclature used to mark the attributes of a data entity (that is, a data record) is called a field or data item. It is the smallest unit of information that can be named, also known as a data element or elementary item.
  • a column in a data table is called a "field”.
  • "contact name” and “contact phone” are attributes common to all the rows in the table, so These columns are referred to as the Contact Name field and the Contact Phone field.
  • Field attributes can include one or more attributes such as data type, length, whether it is allowed to be empty, and so on. Depending on the database system, there may be differences in the types of field attributes. Without loss of generality, the data types may include: numeric types, character types, and other types, such as enumeration type, date type, etc., wherein the numerical types may be further divided into integer types, real numbers, and the like.
  • the monitoring indicators are used to specify the requirements that the data needs to meet in each dimension.
  • the data monitoring system can detect the legality, correctness and validity of the data according to the monitoring indicators.
  • a monitoring metric contains monitoring dimensions that include one or more of the following dimensions: data type, range of values, data format, and whether it is allowed to be empty.
  • the “date” monitoring indicator can be defined from the data type and the data format dimension, and the data type should be “date type” and the data format is “YY-MM-DD”; for example, “ID card number”
  • Monitoring metrics can be defined from the data type, length, and dimension of the range of values.
  • a monitoring indicator can correspond to one or more monitoring rules, and a monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
  • Monitoring rules in the common rule base can be set according to one or more of the following dimensions: data type, range of values, data format, and whether it is allowed to be empty.
  • the control index has at least two corresponding monitoring rules, which are monitoring rules of data types and monitoring rules of data formats.
  • a general rule base may be pre-configured. Monitoring rules are included in the general rule base. Monitoring rules can be set based on monitoring metrics.
  • the monitoring indicator may be configured in advance according to the nature of the data in the database (usually referred to as the business database) or the content of the data table. For example, if the database includes the “ID card number” field, the above “ID card number” needs to be set.
  • the monitoring indicator ; then, according to the monitoring index, the corresponding monitoring rule is configured in the general rule base, and the monitoring index of the above “ID card number” is taken as an example, and the data type, length and value range can be separately selected in the general rule base.
  • the dimension is used to configure the monitoring rule to check the value of the "ID card number” field from the data type, the length, and the dimension of the value range to determine whether it meets the requirements of the "ID card number” monitoring indicator.
  • the monitoring index of the "passport number” needs to be set; then, the corresponding monitoring rule is configured in the general rule base according to the monitoring index, for example, in the general rule base.
  • the monitoring rule is configured from the data type, the length, and the dimension of the value range to check the value of the "passport number” field from the data type, the length, and the dimension of the value range to determine whether the "passport number" monitoring is met. The requirements of the indicator.
  • the data length monitoring rule and the value range monitoring rule in the monitoring rule of the "passport number” are different from the data length monitoring rule and the value range monitoring rule in the monitoring rule of the above "identity number” (because of two kinds of documents) The number of digits and the range of values are different).
  • each monitoring rule is differentiated or identified based on its index information in the general rule base.
  • the index information of a monitoring rule in the general rule base may be a unique identifier of the monitoring rule in the general rule base, or may be formed by the identifier of the monitoring indicator corresponding to the monitoring rule and the identifier of the monitoring rule. For the latter, the identifiers of different monitoring rules can be reused. As long as all the monitoring rules corresponding to one monitoring indicator, the identification of each monitoring rule is unique.
  • Table 1 exemplarily shows an example of a general rule base.
  • the “Monitoring Rule Content” item can fill in the field attribute information that needs to be detected, and can also be used to implement the program code for monitoring the field attribute.
  • the "data type monitoring rule" in the monitoring indicator of the ID card number may specifically be "Int”, indicating whether it is necessary to detect whether the data type is an integer type or a piece of program code, which can be realized by executing the program code: judging data Whether the type is an integer type. If it is an integer type, the detection is passed based on the monitoring rule. Otherwise, the detection fails based on the monitoring rule.
  • a monitoring rule can be indexed by using the ID of the monitoring rule and the monitoring indicator ID corresponding to the monitoring rule.
  • Table 1 can also be split into two tables, one of which is used to store the correspondence between the "monitoring indicator ID" and the “monitoring rule ID”, and the other table is used to store the content of each monitoring rule (can be monitored by the rules) Unique identifier as the primary key).
  • the data monitoring management process provided by the embodiment of the present application can be as shown in FIG. 1 .
  • FIG. 1 is a schematic diagram of a data monitoring management process provided by an embodiment of the present application. This process can be implemented by a data monitoring management system. As shown, the process can include:
  • Step 101 Obtain a field of a monitoring rule to be configured in the database.
  • Step 102 Select a corresponding monitoring rule for the field from the general rule base according to the monitoring indicator used by the field.
  • the monitoring rule corresponding to the monitoring indicator may be selected from the general rule base for the field according to the monitoring indicator used by the field of the monitoring rule to be configured.
  • the current monitoring parameter to be configured is the ID card number.
  • the monitoring indicator used in this field is the monitoring indicator of the ID card number.
  • a monitoring rule when a monitoring rule is selected for a field, all the monitoring rules corresponding to the monitoring indicator used in the field may be selected as the monitoring rule of the field, and some of the monitoring rules may be selected as the monitoring rule of the field as needed.
  • Step 103 Associate the selected monitoring rule with the field of the to-be-configured monitoring rule.
  • the field may be associated with the index information of the monitoring rule selected in step 102 in the general rule base.
  • the identifier of the monitoring indicator may be associated with the field as the index information of the selected monitoring rule; If the part of the monitoring rule corresponding to the monitoring indicator is selected for the field, in step 103, the identifier of the monitoring indicator plus the identifier of the selected monitoring rule may be associated with the field as index information.
  • a mapping table can be created to store the association between the field and the index information of the monitoring rule.
  • a pointer can be created for the fields of the data table, pointing to the monitoring rules selected for the field in the general rule base; as another example, the data monitoring method can be used to select the monitoring rules for a field. This field is associated.
  • the general rule base contains pre-set monitoring rules
  • the monitoring rule needs to be adjusted, Simply updating the general rule base without having to re-configure the data table or field to monitor the rules, which simplifies the complexity of monitoring rule configuration and increases system flexibility compared to the prior art.
  • the field-specific monitoring rule may be configured for the field.
  • the monitoring rules configured in the general rule base cannot satisfy the monitoring requirements of certain aspects of the field.
  • the general monitoring rule is to detect whether the value of the transaction amount is an integer greater than 0, and the unit is a minute.
  • a proprietary monitoring rule can be added to the "transaction amount" field in the fund business to detect Whether the value of this field is greater than 10000 (in minutes) and is an integer multiple of 10000.
  • the configuration of the monitoring rule may be performed according to the foregoing method for each field to be configured by the monitoring rule.
  • FIG. 2 is a schematic diagram of a data monitoring process according to an embodiment of the present invention, which may be implemented by a data monitoring system. As shown, the process can include the following steps:
  • Step 201 Obtain a field associated with a monitoring rule in the database.
  • the method for associating the field with the monitoring rule can be as shown in FIG. 1 and will not be repeated here.
  • the embodiment of the present application does not exclude the use of other methods to implement the association between the monitoring rule and the field.
  • the association between the field and the monitoring rule is taken as an example in the manner of the foregoing embodiment, and the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field;
  • the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
  • the general rule base may include: an identifier of the monitoring indicator and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • the monitoring dimension included in one monitoring indicator may include one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty. Specifically, it can be as described in the foregoing embodiment, and is not repeated here.
  • Step 202 Obtain a monitoring rule associated with the field from a general rule base.
  • the data monitoring system may obtain the index from the general rule base according to the index information of the monitoring rule associated with the field obtained in step 201.
  • the monitoring rule corresponding to the index information.
  • Step 203 Check the data of the field according to the obtained monitoring rule.
  • the field is checked for each monitoring rule in step 203.
  • the obtained monitoring rules associated with the field include: a data type monitoring rule (for detecting whether the data type is an integer type), a length monitoring rule (a bit for detecting data) Whether the number is 18 or not) and the value range monitoring rule (used to detect whether the value range is within the valid value range of the ID card number), whether the value of the "ID card number” field is detected according to the data type monitoring rule It is integer data, whether it is 18-bit data, and whether the value range is within the range of the valid ID number.
  • the data of the “ID card number” field of all (or part of) rows in one or more data tables may be checked according to the monitoring rule associated with the field, and the check result is output.
  • step 201 if the field is further configured with a monitoring rule specific to the field, the specific monitoring rule configured for the field is obtained, and the data of the field is performed according to the proprietary monitoring rule. Check to check whether the data in this field meets the corresponding requirements or regulations, such as whether it is valid or not.
  • the field associated with the monitoring rule in the database is obtained, the monitoring rule associated with the field is obtained from the general rule base, and the field associated with the field is obtained from the general rule base.
  • Monitoring rules Due to the use of fields and general rule base
  • the monitoring rule is associated with the field to configure the monitoring rule.
  • the general rule base contains the preset monitoring rule. When configuring the monitoring rule for a field, just select the appropriate monitoring rule for the field, and the monitoring rule and the The fields can be associated. When the monitoring rules need to be adjusted, only the general rule base needs to be updated, and the monitoring rules of the data tables or fields need not be reconfigured. Compared with the prior art, the complexity of the monitoring rule configuration is simplified. And increased system flexibility.
  • the embodiment of the present application further provides a data monitoring management system, which can implement the process shown in FIG. 1 above.
  • FIG. 3 is a schematic structural diagram of a data monitoring management system according to an embodiment of the present application.
  • the system can include: an obtaining module 301, a selecting module 302, and a configuration module 303, wherein:
  • the obtaining module 301 is configured to acquire a field of a monitoring rule to be configured in the database.
  • the selecting module 302 is configured to select, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from the general rule base; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the monitoring index. Claim;
  • the configuration module 303 is configured to associate the selected monitoring rule with the field.
  • the selecting module 302 is specifically configured to: select all or part of the monitoring rules corresponding to the monitoring indicator for the field from the general rule base according to the monitoring indicator used by the field.
  • the configuration module 303 is specifically configured to: associate the field with the index information of the selected monitoring rule in the general rule base.
  • the configuration module 303 is further configured to: configure the field-specific monitoring rule for the field according to the monitoring indicator used by the field.
  • system may further include an update module 304 (shown in the dashed box in the figure) for updating the monitoring rules in the general rule base.
  • update module 304 shown in the dashed box in the figure for updating the monitoring rules in the general rule base.
  • the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • the embodiment of the present application further provides a data monitoring system, which can implement the foregoing process shown in FIG. 2 .
  • FIG. 4 is a schematic structural diagram of a data monitoring system according to an embodiment of the present disclosure. As shown in the figure, the system may include: a first obtaining module 401, a second obtaining module 402, and a monitoring executing module 403, where:
  • the first obtaining module 401 is configured to acquire a field associated with the monitoring rule in the database
  • a second obtaining module 402 configured to acquire, from a general rule base, a monitoring rule associated with the field
  • the monitoring execution module 403 is configured to check data of the field according to the acquired monitoring rule.
  • the second obtaining module 402 is specifically configured to: obtain, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
  • the first obtaining module 401 is further configured to acquire the field-specific monitoring rule configured for the field.
  • the monitoring execution module 403 is further configured to: check data of the field according to the field-specific monitoring rule.
  • system may further include: an update module 404 (shown by a dashed box in the figure) for updating the monitoring rules in the general rule base.
  • update module 404 shown by a dashed box in the figure for updating the monitoring rules in the general rule base.
  • the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
  • the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  • the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: Data type, range of values, data format, and whether it is allowed to be empty.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Testing And Monitoring For Control Systems (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

A data monitoring management method, and a data monitoring method and system. When a monitoring rule is to be configured for a field, one or more corresponding monitoring rules are selected for the field from a universal rule library according to a monitoring metric used by the field (102), and then the selected monitoring rules are associated with the field (103), thereby realizing a process of configuring a monitoring rule for data. Since the universal rule library contains preset monitoring rules, configuration of a monitoring rule for the field only requires a selection of an appropriate monitoring rule for the field and association of the same with the field. When the monitoring rule needs to be adjusted, it only requires updating the universal rule library, and a monitoring rule reconfiguration does not need to be performed for a data table or the field. Therefore, compared with the prior art, the invention reduces complexity in configuring a monitoring rule, and increases system flexibility.

Description

一种数据监控管理方法、数据监控方法及***Data monitoring management method, data monitoring method and system 技术领域Technical field
本申请涉及通信领域的数据处理技术，尤其涉及一种数据监控管理方法、数据监控方法及***。The present application relates to data processing technologies in the field of communications, and in particular, to a data monitoring management method, a data monitoring method, and a system.
背景技术Background technique
数据仓库(英文为Data Warehouse，简称DW或DWH)提供用户用于决策支持的当前和历史数据。数据仓库技术是为了有效地将操作型数据集成到统一的环境中以提供决策型数据访问的各种技术和模块的总称。基于数据仓库，用户可更快更方便地查询所需要的信息，提供决策支持。The Data Warehouse (Data Warehouse, DW or DWH for short) provides current and historical data for users to use for decision support. Data warehousing technology is a general term for various technologies and modules that effectively integrate operational data into a unified environment to provide decision-making data access. Based on the data warehouse, users can query the required information more quickly and conveniently, and provide decision support.
数据仓库中的数据是在对原有分散的数据库数据进行抽取、清理的基础上经过***加工、汇总和整理得到的。为了更好地提供决策支持，基于给定的指标，对数据仓库中的数据进行检查显得尤为重要。基于给定的指标可对数据仓库中的数据进行合法性、正确性、有效性等各方面的检查。The data in the data warehouse is processed, summarized and organized on the basis of extracting and cleaning the original distributed database data. In order to better provide decision support, it is particularly important to check the data in the data warehouse based on given metrics. Based on the given indicators, the data in the data warehouse can be checked for legality, correctness, and effectiveness.
目前，为了监控数据产出的正确性和有效性，可针对关键性的数据表配置监控规则。在数据表的数据产出后，可根据配置的监控规则检查数据的正确性和有效性。Currently, in order to monitor the correctness and effectiveness of data output, monitoring rules can be configured for critical data tables. After the data of the data table is output, the correctness and validity of the data can be checked according to the configured monitoring rules.
比如，可将有效数据的取值范围作为一个监控指标，为一个字段配置监控规则，如果该字段的监控指标有变更(比如有效数值的取值范围有变化)，或者需要增加一个监控指标(比如需要对该字段的数据格式进行监控)，则需要对涉及使用这些监控指标的所有数据表重新配置监控规则，操作繁琐，并且容易遗漏。For example, the value range of valid data can be used as a monitoring indicator to configure a monitoring rule for a field. If the monitoring indicator of the field is changed (for example, the value range of the valid value changes), or a monitoring indicator needs to be added (for example) The data format of the field needs to be monitored. It is necessary to reconfigure the monitoring rules for all data tables involved in the use of these monitoring indicators, which is cumbersome and easy to miss.
申请内容Application content
本申请实施例提供了一种数据监控管理方法及***。The embodiment of the present application provides a data monitoring management method and system.
本申请实施例提供的数据监控管理方法，包括：The data monitoring management method provided by the embodiment of the present application includes:
获取数据库中待配置监控规则的字段； Obtain the fields of the monitoring rule to be configured in the database;
根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求；Selecting, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from the general rule base; wherein the monitoring rule corresponding to the monitoring indicator is used to detect whether the data meets the requirement of the monitoring indicator;
将选择的监控规则与所述字段进行关联。Associate the selected monitoring rule with the field.
优选地，根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则，包括：Preferably, according to the monitoring indicator used by the field, selecting a corresponding monitoring rule for the field from the general rule base, including:
根据所述字段使用的监控指标，从通用规则库中为所述字段选择所述监控指标对应的全部或部分监控规则。All or part of the monitoring rules corresponding to the monitoring indicator are selected from the general rule base for the field according to the monitoring indicator used by the field.
优选地，将选择的监控规则与所述字段进行关联，包括：将所述字段与所选择的监控规则在所述通用规则库中的索引信息进行关联。Preferably, associating the selected monitoring rule with the field comprises associating the field with index information of the selected monitoring rule in the general rule base.
可选地，还包括：根据所述字段使用的监控指标，为所述字段配置所述字段专用的监控规则。Optionally, the method further includes: configuring, according to the monitoring indicator used by the field, the field-specific monitoring rule for the field.
可选地，还包括：更新所述通用规则库中的监控规则。Optionally, the method further includes: updating a monitoring rule in the general rule base.
优选地，所述通用规则库中的监控规则根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。Preferably, the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
本申请实施例提供的数据监控管理***，包括：The data monitoring management system provided by the embodiment of the present application includes:
获取模块，用于获取数据库中待配置监控规则的字段；An obtaining module, configured to acquire a field of a monitoring rule to be configured in the database;
选择模块，用于根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求；a selection module, configured to select, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from a general rule base; wherein a monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirement of the monitoring indicator ;
配置模块，用于将选择的监控规则与所述字段进行关联。A configuration module for associating selected monitoring rules with the fields.
优选地，所述选择模块具体用于：根据所述字段使用的监控指标，从 通用规则库中为所述字段选择所述监控指标对应的全部或部分监控规则。Preferably, the selecting module is specifically configured to: according to the monitoring indicator used by the field, All or part of the monitoring rules corresponding to the monitoring indicator are selected in the general rule base for the field.
优选地，所述配置模块具体用于：将所述字段与所选择的监控规则在所述通用规则库中的索引信息进行关联。Preferably, the configuration module is specifically configured to: associate the field with index information of the selected monitoring rule in the general rule base.
优选地，所述配置模块还用于：根据所述字段使用的监控指标，为所述字段配置所述字段专用的监控规则。Preferably, the configuration module is further configured to configure the field-specific monitoring rule for the field according to the monitoring indicator used by the field.
可选地，还包括：更新模块，用于更新所述通用规则库中的监控规则。Optionally, the method further includes: an update module, configured to update the monitoring rule in the general rule base.
优选地，所述通用规则库中的监控规则根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。Preferably, the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
本申请的上述实施例中，在为一个字段配置监控规则时，根据该字段使用的监控指标，从通用规则库中为该字段选择一个或多个对应的监控规则，并将选择的监控规则与该字段进行关联，从而实现了对数据进行监控规则的配置过程。由于通用规则库中包含预先设置的监控规则，在为一个字段配置监控规则时，只要为该字段选择合适的监控规则，并将监控规则与该字段进行关联即可，当需要调整监控规则时，只需对通用规则库进行更新，而无需重新对数据表或字段进行监控规则配置，与现有技术相比，简化了监控规则配置的复杂度，并提高了***灵活性。In the foregoing embodiment of the present application, when configuring a monitoring rule for a field, according to the monitoring indicator used by the field, one or more corresponding monitoring rules are selected from the general rule base for the field, and the selected monitoring rule is selected. This field is associated, which implements the process of configuring the monitoring rules for data. Since the general rule base contains pre-set monitoring rules, when configuring a monitoring rule for a field, you only need to select the appropriate monitoring rule for the field and associate the monitoring rule with the field. When the monitoring rule needs to be adjusted, Simply updating the general rule base without having to re-configure the data table or field to monitor the rules, which simplifies the complexity of monitoring rule configuration and increases system flexibility compared to the prior art.
本申请实施例还提供了一种数据监控方法和***。The embodiment of the present application also provides a data monitoring method and system.
本申请实施例提供的数据监控方法，包括：The data monitoring method provided by the embodiment of the present application includes:
获取数据库中关联有监控规则的字段；Obtain a field associated with a monitoring rule in the database;
从通用规则库中获取所述字段所关联的监控规则；Obtaining the monitoring rules associated with the field from the general rule base;
根据获取到的监控规则对所述字段的数据进行检查。The data of the field is checked according to the obtained monitoring rule.
优选地，从通用规则库中获取所述字段所关联的监控规则，包括：根 据所述字段所关联的监控规则的索引信息，从所述通用规则库中获取与所述索引信息对应的监控规则。Preferably, the monitoring rule associated with the field is obtained from a general rule base, including: And obtaining, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
可选地，还包括：获取为所述字段配置的所述字段专用的监控规则，根据所述字段专用的监控规则对所述字段的数据进行检查。Optionally, the method further includes: acquiring the field-specific monitoring rule configured for the field, and checking data of the field according to the field-specific monitoring rule.
可选地，还包括：更新所述通用规则库中的监控规则。Optionally, the method further includes: updating a monitoring rule in the general rule base.
优选地，与所述字段关联的监控规则为所述字段使用的监控指标对应的全部或部分监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。Preferably, the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
本申请实施例提供的数据监控***，包括：The data monitoring system provided by the embodiment of the present application includes:
第一获取模块，用于获取数据库中关联有监控规则的字段；a first obtaining module, configured to acquire a field associated with a monitoring rule in the database;
第二获取模块，用于从通用规则库中获取所述字段所关联的监控规则；a second obtaining module, configured to acquire, from a general rule base, a monitoring rule associated with the field;
监控执行模块，用于根据获取到的监控规则对所述字段的数据进行检查。The monitoring execution module is configured to check data of the field according to the obtained monitoring rule.
优选地，所述第二获取模块具体用于：根据所述字段所关联的监控规则的索引信息，从所述通用规则库中获取与所述索引信息对应的监控规则。Preferably, the second obtaining module is configured to: obtain, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
可选地，所述第一获取模块还用于：获取为所述字段配置的所述字段专用的监控规则；所述监控执行模块还用于：根据所述字段专用的监控规则对所述字段的数据进行检查。Optionally, the first obtaining module is further configured to: obtain the field-specific monitoring rule configured for the field; the monitoring execution module is further configured to: use the field-specific monitoring rule to the field The data is checked.
可选地，还包括：更新模块，用于更新所述通用规则库中的监控规则。Optionally, the method further includes: an update module, configured to update the monitoring rule in the general rule base.
优选地，与所述字段关联的监控规则为所述字段使用的监控指标对应的全部或部分监控规则；其中，一个监控指标所对应的监控规则用来检测 数据是否符合该监控指标的要求。Preferably, the monitoring rule associated with the field is all or part of a monitoring rule corresponding to the monitoring indicator used by the field; wherein a monitoring rule corresponding to a monitoring indicator is used to detect Whether the data meets the requirements of this monitoring indicator.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
本申请的上述实施例中，在进行数据监控时，获取数据库中关联有监控规则的字段，从通用规则库中获取所述字段所关联的监控规则，从通用规则库中获取所述字段所关联的监控规则。由于采用将字段与通用规则库中的监控规则进行关联的方式为字段配置监控规则，通用规则库中包含预先设置的监控规则，在为一个字段配置监控规则时，只要为该字段选择合适的监控规则，并将监控规则与该字段进行关联即可，当需要调整监控规则时，只需对通用规则库进行更新，而无需重新对数据表或字段进行监控规则配置，与现有技术相比，简化了监控规则配置的复杂度，并提高了***灵活性。In the foregoing embodiment of the present application, when data monitoring is performed, a field associated with the monitoring rule in the database is obtained, and the monitoring rule associated with the field is obtained from the general rule base, and the field is associated with the general rule base. Monitoring rules. The monitoring rule is configured for the field by associating the field with the monitoring rule in the general rule base. The general rule base contains preset monitoring rules. When configuring the monitoring rule for a field, just select the appropriate monitoring for the field. Rules, and the monitoring rules are associated with the field. When the monitoring rules need to be adjusted, only the general rule base needs to be updated, and the monitoring rules of the data table or the field need not be reconfigured, compared with the prior art. It simplifies the complexity of monitoring rule configuration and increases system flexibility.
附图说明DRAWINGS
图1为本申请实施例提供的数据监控管理流程示意图；1 is a schematic diagram of a data monitoring management process provided by an embodiment of the present application;
图2为本申请实施例提供的数据监控流程示意图；2 is a schematic diagram of a data monitoring process provided by an embodiment of the present application;
图3为本申请实施例提供的数据监控管理***结构示意图；3 is a schematic structural diagram of a data monitoring management system according to an embodiment of the present application;
图4为本申请实施例提供的数据监控***结构示意图。FIG. 4 is a schematic structural diagram of a data monitoring system according to an embodiment of the present application.
具体实施方式detailed description
本申请的上述实施例中，预先设置通用规则库，通用规则库中包含预先设置的监控规则，在为一个字段配置监控规则时，只要为该字段选择合适的监控规则，并将监控规则与该字段进行关联即可，当需要调整监控规则时，只需对通用规则库中的监控规则进行更新，而无需重新配置数据表或字段的监控规则，与现有技术相比，简化了监控规则配置的复杂度，并 提高了***灵活性。In the foregoing embodiment of the present application, a general rule base is preset, and the general rule base includes a preset monitoring rule. When a monitoring rule is configured for a field, only a suitable monitoring rule is selected for the field, and the monitoring rule is The fields can be associated. When the monitoring rules need to be adjusted, the monitoring rules in the general rule base need to be updated without reconfiguring the monitoring rules of the data tables or fields, which simplifies the monitoring rule configuration compared with the prior art. Complexity, and Increased system flexibility.
本申请实施例可适用于数据库***或者数据仓库***。The embodiments of the present application are applicable to a database system or a data warehouse system.
在描述本申请实施例之前，首先对本申请实施例中涉及到的一些术语进行说明。Before describing the embodiments of the present application, some terms involved in the embodiments of the present application are first described.
(1)字段以及字段属性(1) fields and field attributes
在数据库或数据仓库技术中，用于标记数据实体(即一条数据记录)属性的命名单位称为字段或数据项。它是可以命名的最小信息单位，又称为数据元素或初等项。对于关系型数据库，一个数据表中的一列称为一个“字段”，比如，在“通讯录”数据库中，“联系人姓名”、“联系电话”这些都是表中所有行共有的属性，所以将这些列称为“联系人姓名”字段和“联系电话”字段。In database or data warehousing technology, the nomenclature used to mark the attributes of a data entity (that is, a data record) is called a field or data item. It is the smallest unit of information that can be named, also known as a data element or elementary item. For a relational database, a column in a data table is called a "field". For example, in the "Contacts" database, "contact name" and "contact phone" are attributes common to all the rows in the table, so These columns are referred to as the Contact Name field and the Contact Phone field.
字段属性可包括数据类型、长度、是否允许为空等一种或多种属性。根据数据库***的不同，字段属性的种类也可能存在差别。不失一般性地，数据类型可包括：数值型、字符型以及其他类型，比如枚举型、日期型等，其中，数值型可进一步分为整数型、实数型等。Field attributes can include one or more attributes such as data type, length, whether it is allowed to be empty, and so on. Depending on the database system, there may be differences in the types of field attributes. Without loss of generality, the data types may include: numeric types, character types, and other types, such as enumeration type, date type, etc., wherein the numerical types may be further divided into integer types, real numbers, and the like.
(2)监控指标(2) Monitoring indicators
监控指标用来规定数据在各维度上需要满足的要求，数据监控***可根据监控指标检测数据的合法性、正确性、有效性等。一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。比如，“日期”的监控指标可以从数据类型、数据格式的维度进行定义，规定数据类型应为“日期型”、数据格式为“YY-MM-DD”；再例如，“身份证号码”的监控指标可从数据类型、长度以及取值范围的维度进行定义。The monitoring indicators are used to specify the requirements that the data needs to meet in each dimension. The data monitoring system can detect the legality, correctness and validity of the data according to the monitoring indicators. A monitoring metric contains monitoring dimensions that include one or more of the following dimensions: data type, range of values, data format, and whether it is allowed to be empty. For example, the “date” monitoring indicator can be defined from the data type and the data format dimension, and the data type should be “date type” and the data format is “YY-MM-DD”; for example, “ID card number” Monitoring metrics can be defined from the data type, length, and dimension of the range of values.
(3)监控规则(3) Monitoring rules
一个监控指标可对应一个或多个监控规则，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。A monitoring indicator can correspond to one or more monitoring rules, and a monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
通用规则库中的监控规则可根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。以上述“日期”的监 控指标为例，对应的监控规则至少有两个，分别是数据类型的监控规则和数据格式的监控规则。Monitoring rules in the common rule base can be set according to one or more of the following dimensions: data type, range of values, data format, and whether it is allowed to be empty. Supervised by the above "date" For example, the control index has at least two corresponding monitoring rules, which are monitoring rules of data types and monitoring rules of data formats.
本申请实施例中，可预先配置通用规则库。通用规则库中包含监控规则。监控规则可根据监控指标来设置。In the embodiment of the present application, a general rule base may be pre-configured. Monitoring rules are included in the general rule base. Monitoring rules can be set based on monitoring metrics.
在具体实施中，可预先根据数据库(通常指业务数据库)中数据的性质或者是数据表的内容，配置监控指标，比如，数据库中包含“身份证号码”字段，则需要设置上述“身份证号码”的监控指标；然后，根据监控指标在通用规则库中配置对应的监控规则，以上述“身份证号码”的监控指标为例，可在通用规则库中分别从数据类型、长度以及取值范围的维度来配置监控规则，以用来对“身份证号码”字段的取值从数据类型、长度以及取值范围的维度进行检查，判断是否符合“身份证号码”监控指标的要求。In a specific implementation, the monitoring indicator may be configured in advance according to the nature of the data in the database (usually referred to as the business database) or the content of the data table. For example, if the database includes the “ID card number” field, the above “ID card number” needs to be set. The monitoring indicator; then, according to the monitoring index, the corresponding monitoring rule is configured in the general rule base, and the monitoring index of the above “ID card number” is taken as an example, and the data type, length and value range can be separately selected in the general rule base. The dimension is used to configure the monitoring rule to check the value of the "ID card number" field from the data type, the length, and the dimension of the value range to determine whether it meets the requirements of the "ID card number" monitoring indicator.
再例如，如果数据库中还包含“护照号码”字段，则需要设置“护照号码”的监控指标；然后，根据监控指标在通用规则库中配置对应的监控规则，比如，可在通用规则库中分别从数据类型、长度以及取值范围的维度来配置监控规则，以用来对“护照号码”字段的取值从数据类型、长度以及取值范围的维度进行检查，判断是否符合“护照号码”监控指标的要求。其中，“护照号码”的监控规则中的数据长度监控规则和取值范围监控规则，与上述“身份证号码”的监控规则中的数据长度监控规则和取值范围监控规则不同(因为两种证件的号码位数以及取值范围不同)。For example, if the "passport number" field is also included in the database, the monitoring index of the "passport number" needs to be set; then, the corresponding monitoring rule is configured in the general rule base according to the monitoring index, for example, in the general rule base. The monitoring rule is configured from the data type, the length, and the dimension of the value range to check the value of the "passport number" field from the data type, the length, and the dimension of the value range to determine whether the "passport number" monitoring is met. The requirements of the indicator. The data length monitoring rule and the value range monitoring rule in the monitoring rule of the "passport number" are different from the data length monitoring rule and the value range monitoring rule in the monitoring rule of the above "identity number" (because of two kinds of documents) The number of digits and the range of values are different).
在通用规则库中，每个监控规则基于其在该通用规则库中的索引信息进行区分或标识。一个监控规则在通用规则库中的索引信息可以是该监控规则在该通用规则库中的唯一标识，也可以由该监控规则对应的监控指标的标识以及该监控规则的标识构成。对于后者，不同监控规则的标识可以重用，只要一个监控指标对应的所有监控规则中，每个监控规则的标识唯一即可。In the general rule base, each monitoring rule is differentiated or identified based on its index information in the general rule base. The index information of a monitoring rule in the general rule base may be a unique identifier of the monitoring rule in the general rule base, or may be formed by the identifier of the monitoring indicator corresponding to the monitoring rule and the identifier of the monitoring rule. For the latter, the identifiers of different monitoring rules can be reused. As long as all the monitoring rules corresponding to one monitoring indicator, the identification of each monitoring rule is unique.
表1示例性地示出了一种通用规则库的示例。 Table 1 exemplarily shows an example of a general rule base.
表1Table 1
Figure PCTCN2016110180-appb-000001
Figure PCTCN2016110180-appb-000001
表1中，“监控规则内容”一项可填写需要检测的字段属性信息，也可以用于实现对字段属性进行监控的程序代码。比如，身份证号码的监控指标中的“数据类型监控规则”具体可以是“Int”，表示需要检测数据类型是否为整数型；也可以是一段程序代码，通过执行该程序代码可实现：判断数据类型是否为整数型，如果是整数型，则基于该监控规则检测通过，否则基于该监控规则检测失败。In Table 1, the “Monitoring Rule Content” item can fill in the field attribute information that needs to be detected, and can also be used to implement the program code for monitoring the field attribute. For example, the "data type monitoring rule" in the monitoring indicator of the ID card number may specifically be "Int", indicating whether it is necessary to detect whether the data type is an integer type or a piece of program code, which can be realized by executing the program code: judging data Whether the type is an integer type. If it is an integer type, the detection is passed based on the monitoring rule. Otherwise, the detection fails based on the monitoring rule.
表1中，一个监控规则可用该监控规则的ID与该监控规则对应的监控指标ID进行索引。In Table 1, a monitoring rule can be indexed by using the ID of the monitoring rule and the monitoring indicator ID corresponding to the monitoring rule.
表1也可拆分为两个表，其中一个表用于存储“监控指标ID”与“监控规则ID”的对应关系，另一个表用于存储每个监控规则的内容(可以以监控规则的唯一标识作为主键)。Table 1 can also be split into two tables, one of which is used to store the correspondence between the "monitoring indicator ID" and the "monitoring rule ID", and the other table is used to store the content of each monitoring rule (can be monitored by the rules) Unique identifier as the primary key).
上述通用规则库可根据需要进行更新。以上述表1为例，比如，当需要增加一个新的监控指标时(监控指标ID＝300)，则在上述通用规则库中增加该监控指标对应的一个或多个监控规则；当需要对原有的某个监控规则(监控指标ID＝100，监控规则ID＝3)进行调整时(比如调整取值范围)，则只要将监控规则的内容进行调整即可；如果需要针对某个监控指标(监 控指标ID＝300)，增加一个监控规则时，只要在表1中ID＝300的监控指标中增加相应的监控规则即可。The above general rule base can be updated as needed. Take the above Table 1 as an example. For example, when a new monitoring indicator needs to be added (monitoring indicator ID=300), one or more monitoring rules corresponding to the monitoring indicator are added to the general rule base; When some monitoring rules (monitoring indicator ID=100, monitoring rule ID=3) are adjusted (such as adjusting the value range), you only need to adjust the content of the monitoring rules; if you need to target a certain monitoring indicator ( Supervisor Control indicator ID=300), when adding a monitoring rule, just add the corresponding monitoring rule in the monitoring indicator of ID=300 in Table 1.
基于上述通用规则库，本申请实施例提供的数据监控管理流程可如图1所示。Based on the above-mentioned general rule base, the data monitoring management process provided by the embodiment of the present application can be as shown in FIG. 1 .
参见图1，为本申请实施例提供的数据监控管理流程示意图。该流程可由数据监控管理***实现。如图所示，该流程可包括：FIG. 1 is a schematic diagram of a data monitoring management process provided by an embodiment of the present application. This process can be implemented by a data monitoring management system. As shown, the process can include:
步骤101：获取数据库中待配置监控规则的字段。Step 101: Obtain a field of a monitoring rule to be configured in the database.
步骤102：根据该字段使用的监控指标，从通用规则库中为该字段选择对应的监控规则。Step 102: Select a corresponding monitoring rule for the field from the general rule base according to the monitoring indicator used by the field.
以前述通用规则库的数据结构为例，可根据待配置监控规则的字段使用的监控指标，从通用规则库中为该字段选择该监控指标对应的监控规则。比如，当前待配置监控规则的字段为“身份证号码”，该字段使用的监控指标为“身份证号码”的监控指标，其对应的监控指标ID＝100，则可根据监控指标ID＝100在通用规则库中查询到对应有3个监控规则，则将这3个监控规则选择为该字段的监控规则。Taking the data structure of the foregoing general rule base as an example, the monitoring rule corresponding to the monitoring indicator may be selected from the general rule base for the field according to the monitoring indicator used by the field of the monitoring rule to be configured. For example, the current monitoring parameter to be configured is the ID card number. The monitoring indicator used in this field is the monitoring indicator of the ID card number. The corresponding monitoring indicator ID=100 can be based on the monitoring indicator ID=100. If there are 3 monitoring rules in the general rule base, the 3 monitoring rules are selected as the monitoring rules of the field.
进一步地，在为一个字段选择监控规则时，可将该字段使用的监控指标对应的全部监控规则选择为该字段的监控规则，也可以根据需要将其中部分监控规则选择为该字段的监控规则。Further, when a monitoring rule is selected for a field, all the monitoring rules corresponding to the monitoring indicator used in the field may be selected as the monitoring rule of the field, and some of the monitoring rules may be selected as the monitoring rule of the field as needed.
步骤103：将选择的监控规则与该待配置监控规则的字段进行关联。Step 103: Associate the selected monitoring rule with the field of the to-be-configured monitoring rule.
具体地，该步骤中，可将该字段与步骤102中所选择的监控规则在通用规则库中的索引信息进行关联。Specifically, in this step, the field may be associated with the index information of the monitoring rule selected in step 102 in the general rule base.
进一步地，如果步骤102中为该字段选择监控指标对应的全部监控规则，则在步骤103中，可将该监控指标的标识作为所选择的监控规则的索引信息与该字段进行关联；如果步骤102中为该字段选择监控指标对应的部分监控规则，则在步骤103中，可将该监控指标的标识加上所选择的监控规则的标识一起作为索引信息与该字段进行关联。Further, if all the monitoring rules corresponding to the monitoring indicator are selected for the field in step 102, in step 103, the identifier of the monitoring indicator may be associated with the field as the index information of the selected monitoring rule; If the part of the monitoring rule corresponding to the monitoring indicator is selected for the field, in step 103, the identifier of the monitoring indicator plus the identifier of the selected monitoring rule may be associated with the field as index information.
监控规则的索引信息与字段的关联方式可以有多种。作为一个例子，可创建一个映射表，用于存储字段与监控规则的索引信息之间的关联关 系；作为另一个例子，可以针对数据表的字段创建指针，指向通用规则库中为该字段选择的监控规则；作为另一个例子，可采用数据引用方式，将为某个字段选择的监控规则与该字段进行关联。There are many ways to associate index information of a monitoring rule with a field. As an example, a mapping table can be created to store the association between the field and the index information of the monitoring rule. As another example, a pointer can be created for the fields of the data table, pointing to the monitoring rules selected for the field in the general rule base; as another example, the data monitoring method can be used to select the monitoring rules for a field. This field is associated.
通过以上描述可以看出，在为一个字段配置监控规则时，根据该字段使用的监控指标，从通用规则库中为该字段选择一个或多个对应的监控规则，并将选择的监控规则与该字段进行关联，从而实现了对数据进行监控规则的配置过程。由于通用规则库中包含预先设置的监控规则，在为一个字段配置监控规则时，只要为该字段选择合适的监控规则，并将监控规则与该字段进行关联即可，当需要调整监控规则时，只需对通用规则库进行更新，而无需重新对数据表或字段进行监控规则配置，与现有技术相比，简化了监控规则配置的复杂度，并提高了***灵活性。As can be seen from the above description, when configuring a monitoring rule for a field, according to the monitoring indicator used by the field, one or more corresponding monitoring rules are selected from the general rule base for the field, and the selected monitoring rule is The fields are associated, thus implementing the process of configuring the monitoring rules for the data. Since the general rule base contains pre-set monitoring rules, when configuring a monitoring rule for a field, you only need to select the appropriate monitoring rule for the field and associate the monitoring rule with the field. When the monitoring rule needs to be adjusted, Simply updating the general rule base without having to re-configure the data table or field to monitor the rules, which simplifies the complexity of monitoring rule configuration and increases system flexibility compared to the prior art.
可选地，在上述实施例的基础上，在另外一些实施例中，可为字段配置该字段专用的监控规则。考虑到对于某些字段，由于其特殊性，通用规则库中配置的监控规则无法满足对该字段的某些方面的监控需求，这种情况下，通过为该字段配置该字段专用的监控规则，可以较全面地对该字段进行监控。举例来说，比如关于交易金额字段，通用的监控规则是检测交易金额的取值是否是一个大于0的整数，单位为分。但是在具体业务中，比如基金业务，有些基金要求是100元起申购，并且要求是100的整数倍，因此可针对基金业务中的“交易金额”字段增加一个专有的监控规则，用来检测该字段的取值是否大于10000(单位为分)，并且是10000的整数倍。Optionally, based on the foregoing embodiment, in other embodiments, the field-specific monitoring rule may be configured for the field. Considering that for some fields, due to their particularity, the monitoring rules configured in the general rule base cannot satisfy the monitoring requirements of certain aspects of the field. In this case, by configuring the field-specific monitoring rules for the field, This field can be monitored more comprehensively. For example, for example, regarding the transaction amount field, the general monitoring rule is to detect whether the value of the transaction amount is an integer greater than 0, and the unit is a minute. However, in specific businesses, such as fund business, some funds require a subscription of 100 yuan, and the requirement is an integer multiple of 100. Therefore, a proprietary monitoring rule can be added to the "transaction amount" field in the fund business to detect Whether the value of this field is greater than 10000 (in minutes) and is an integer multiple of 10000.
在具体实施时，可针对每个待进行监控规则配置的字段，按照上述方法进行监控规则的配置。In a specific implementation, the configuration of the monitoring rule may be performed according to the foregoing method for each field to be configured by the monitoring rule.
参见图2，为本发明实施例提供的数据监控流程示意图，该流程可由数据监控***实现。如图所示，该流程可包括如下步骤：FIG. 2 is a schematic diagram of a data monitoring process according to an embodiment of the present invention, which may be implemented by a data monitoring system. As shown, the process can include the following steps:
步骤201：获取数据库中关联有监控规则的字段。Step 201: Obtain a field associated with a monitoring rule in the database.
其中，字段与监控规则的关联方法可如图1所示，在此不再重复。当然，本申请实施例也不排除采用其他方式实现监控规则与字段的关联。The method for associating the field with the monitoring rule can be as shown in FIG. 1 and will not be repeated here. Of course, the embodiment of the present application does not exclude the use of other methods to implement the association between the monitoring rule and the field.
以采用前述实施例的方式对字段和监控规则进行关联为例，与该字段关联的监控规则为该字段使用的监控指标对应的全部或部分监控规则；其 中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。The association between the field and the monitoring rule is taken as an example in the manner of the foregoing embodiment, and the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field; The monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
所述通用规则库中可包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。其中，一个监控指标包含的监控维度可包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。具体可如前述实施例所述，在此不再重复。The general rule base may include: an identifier of the monitoring indicator and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule. The monitoring dimension included in one monitoring indicator may include one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty. Specifically, it can be as described in the foregoing embodiment, and is not repeated here.
步骤202：从通用规则库中获取该字段所关联的监控规则。Step 202: Obtain a monitoring rule associated with the field from a general rule base.
以采用前述实施例的方法对字段和监控规则进行关联为例，本步骤中，数据监控***可根据步骤201中获取到的字段所关联的监控规则的索引信息，从通用规则库中获取与该索引信息对应的监控规则。For example, the data monitoring system may obtain the index from the general rule base according to the index information of the monitoring rule associated with the field obtained in step 201. The monitoring rule corresponding to the index information.
步骤203：根据获取到的监控规则对该字段的数据进行检查。Step 203: Check the data of the field according to the obtained monitoring rule.
其中，如果步骤202中获取到多个监控规则，则步骤203中针对每个监控规则对该字段进行检查。例如，以“身份证号码”字段为例，获取到的与该字段关联的监控规则包括：数据类型监控规则(用于检测数据类型是否是整数型)、长度监控规则(用于检测数据的位数是否是18位)以及取值范围监控规则(用于检测取值范围是否在身份证号码有效取值范围之内)，则根据数据类型监控规则检测该“身份证号码”字段的取值是否是整数型数据、是否是18位数据、取值范围是否在有效身份证号码取值范围之内。具体实施时，可针对一个或多个数据表中的所有(或部分)行的“身份证号码”字段的数据，根据该字段关联的监控规则进行检查，并输出检查结果。If the multiple monitoring rules are obtained in step 202, the field is checked for each monitoring rule in step 203. For example, taking the “ID card number” field as an example, the obtained monitoring rules associated with the field include: a data type monitoring rule (for detecting whether the data type is an integer type), a length monitoring rule (a bit for detecting data) Whether the number is 18 or not) and the value range monitoring rule (used to detect whether the value range is within the valid value range of the ID card number), whether the value of the "ID card number" field is detected according to the data type monitoring rule It is integer data, whether it is 18-bit data, and whether the value range is within the range of the valid ID number. In a specific implementation, the data of the “ID card number” field of all (or part of) rows in one or more data tables may be checked according to the monitoring rule associated with the field, and the check result is output.
进一步地，在步骤201中，如果该字段还配置有该字段所专有的监控规则，则获取为该字段配置的专有的监控规则，并根据该专有的监控规则对该字段的数据进行检查，以检查该字段的数据是否符合相应的要求或规定，比如是否有效、是否合法等。Further, in step 201, if the field is further configured with a monitoring rule specific to the field, the specific monitoring rule configured for the field is obtained, and the data of the field is performed according to the proprietary monitoring rule. Check to check whether the data in this field meets the corresponding requirements or regulations, such as whether it is valid or not.
通过以上描述可以看出，在进行数据监控时，获取数据库中关联有监控规则的字段，从通用规则库中获取所述字段所关联的监控规则，从通用规则库中获取所述字段所关联的监控规则。由于采用将字段与通用规则库 中的监控规则进行关联的方式为字段配置监控规则，通用规则库中包含预先设置的监控规则，在为一个字段配置监控规则时，只要为该字段选择合适的监控规则，并将监控规则与该字段进行关联即可，当需要调整监控规则时，只需对通用规则库进行更新，而无需重新对数据表或字段进行监控规则配置，与现有技术相比，简化了监控规则配置的复杂度，并提高了***灵活性。As can be seen from the above description, in the data monitoring, the field associated with the monitoring rule in the database is obtained, the monitoring rule associated with the field is obtained from the general rule base, and the field associated with the field is obtained from the general rule base. Monitoring rules. Due to the use of fields and general rule base The monitoring rule is associated with the field to configure the monitoring rule. The general rule base contains the preset monitoring rule. When configuring the monitoring rule for a field, just select the appropriate monitoring rule for the field, and the monitoring rule and the The fields can be associated. When the monitoring rules need to be adjusted, only the general rule base needs to be updated, and the monitoring rules of the data tables or fields need not be reconfigured. Compared with the prior art, the complexity of the monitoring rule configuration is simplified. And increased system flexibility.
基于相同的技术构思，本申请实施例还提供了一种数据监控管理***，可实现如前述图1所示的流程。Based on the same technical concept, the embodiment of the present application further provides a data monitoring management system, which can implement the process shown in FIG. 1 above.
参见图3，为本申请实施例提供的数据监控管理***的结构示意图。该***可包括：获取模块301、选择模块302、配置模块303，其中：FIG. 3 is a schematic structural diagram of a data monitoring management system according to an embodiment of the present application. The system can include: an obtaining module 301, a selecting module 302, and a configuration module 303, wherein:
获取模块301，用于获取数据库中待配置监控规则的字段；The obtaining module 301 is configured to acquire a field of a monitoring rule to be configured in the database.
选择模块302，用于根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求；The selecting module 302 is configured to select, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from the general rule base; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the monitoring index. Claim;
配置模块303，用于将选择的监控规则与所述字段进行关联。The configuration module 303 is configured to associate the selected monitoring rule with the field.
优选地，选择模块302可具体用于：根据所述字段使用的监控指标，从通用规则库中为所述字段选择所述监控指标对应的全部或部分监控规则。Preferably, the selecting module 302 is specifically configured to: select all or part of the monitoring rules corresponding to the monitoring indicator for the field from the general rule base according to the monitoring indicator used by the field.
优选地，配置模块303可具体用于：将所述字段与所选择的监控规则在所述通用规则库中的索引信息进行关联。Preferably, the configuration module 303 is specifically configured to: associate the field with the index information of the selected monitoring rule in the general rule base.
优选地，配置模块303还可用于：根据所述字段使用的监控指标，为所述字段配置所述字段专用的监控规则。Preferably, the configuration module 303 is further configured to: configure the field-specific monitoring rule for the field according to the monitoring indicator used by the field.
可选地，该***还可包括更新模块304(如图中虚线框所示)，用于更新所述通用规则库中的监控规则。Optionally, the system may further include an update module 304 (shown in the dashed box in the figure) for updating the monitoring rules in the general rule base.
优选地，所述通用规则库中的监控规则根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。Preferably, the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。 Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
基于相同的技术构思，本申请实施例还提供了一种数据监控***，该***可实现前述图2所示的流程。Based on the same technical concept, the embodiment of the present application further provides a data monitoring system, which can implement the foregoing process shown in FIG. 2 .
参见图4，为本申请实施例提供的数据监控***的结构示意图，如图所示，该***可包括：第一获取模块401、第二获取模块402、监控执行模块403，其中：FIG. 4 is a schematic structural diagram of a data monitoring system according to an embodiment of the present disclosure. As shown in the figure, the system may include: a first obtaining module 401, a second obtaining module 402, and a monitoring executing module 403, where:
第一获取模块401，用于获取数据库中关联有监控规则的字段；The first obtaining module 401 is configured to acquire a field associated with the monitoring rule in the database;
第二获取模块402，用于从通用规则库中获取所述字段所关联的监控规则；a second obtaining module 402, configured to acquire, from a general rule base, a monitoring rule associated with the field;
监控执行模块403，用于根据获取到的监控规则对所述字段的数据进行检查。The monitoring execution module 403 is configured to check data of the field according to the acquired monitoring rule.
优选地，第二获取模块402可具体用于：根据所述字段所关联的监控规则的索引信息，从所述通用规则库中获取与所述索引信息对应的监控规则。Preferably, the second obtaining module 402 is specifically configured to: obtain, according to the index information of the monitoring rule associated with the field, a monitoring rule corresponding to the index information from the general rule base.
优选地，第一获取模块401还可用于获取为所述字段配置的所述字段专用的监控规则。相应地，监控执行模块403还可用于：根据所述字段专用的监控规则对所述字段的数据进行检查。Preferably, the first obtaining module 401 is further configured to acquire the field-specific monitoring rule configured for the field. Correspondingly, the monitoring execution module 403 is further configured to: check data of the field according to the field-specific monitoring rule.
可选地，该***还可包括：更新模块404(如图中的虚线框所示)，用于更新所述通用规则库中的监控规则。Optionally, the system may further include: an update module 404 (shown by a dashed box in the figure) for updating the monitoring rules in the general rule base.
优选地，与所述字段关联的监控规则为所述字段使用的监控指标对应的全部或部分监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。Preferably, the monitoring rule associated with the field is all or part of the monitoring rule corresponding to the monitoring indicator used by the field; wherein the monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator.
优选地，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Preferably, the general rule base includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
优选地，一个监控指标包含的监控维度包括以下维度中的一种或多种： 数据类型、取值范围、数据格式，以及是否允许为空。Preferably, the monitoring dimension included in one monitoring indicator includes one or more of the following dimensions: Data type, range of values, data format, and whether it is allowed to be empty.
本申请是参照根据本申请实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。While the preferred embodiment of the present application has been described, it will be apparent that those skilled in the art can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。 It will be apparent to those skilled in the art that various modifications and changes can be made in the present application without departing from the spirit and scope of the application. Thus, it is intended that the present invention cover the modifications and variations of the present invention.

Claims (30)

  1. 一种数据监控管理方法，其特征在于，包括：A data monitoring management method, comprising:
    获取数据库中待配置监控规则的字段；Obtain the fields of the monitoring rule to be configured in the database;
    根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求；Selecting, according to the monitoring indicator used by the field, a corresponding monitoring rule for the field from the general rule base; wherein the monitoring rule corresponding to the monitoring indicator is used to detect whether the data meets the requirement of the monitoring indicator;
    将选择的监控规则与所述字段进行关联。Associate the selected monitoring rule with the field.
  2. 如权利要求1所述的方法，其特征在于，根据所述字段使用的监控指标，从通用规则库中为所述字段选择对应的监控规则，包括：The method according to claim 1, wherein the corresponding monitoring rule is selected for the field from the general rule base according to the monitoring indicator used by the field, including:
    根据所述字段使用的监控指标，从通用规则库中为所述字段选择所述监控指标对应的全部或部分监控规则。All or part of the monitoring rules corresponding to the monitoring indicator are selected from the general rule base for the field according to the monitoring indicator used by the field.
  3. 如权利要求1所述的方法，其特征在于，将选择的监控规则与所述字段进行关联，包括：The method of claim 1 wherein associating the selected monitoring rule with the field comprises:
    将所述字段与所选择的监控规则在所述通用规则库中的索引信息进行关联。The field is associated with index information of the selected monitoring rule in the general rule base.
  4. 如权利要求1所述的方法，其特征在于，还包括：The method of claim 1 further comprising:
    根据所述字段使用的监控指标，为所述字段配置所述字段专用的监控规则。The field-specific monitoring rules are configured for the field according to the monitoring indicators used by the field.
  5. 如权利要求1所述的方法，其特征在于，还包括：The method of claim 1 further comprising:
    更新所述通用规则库中的监控规则。Update the monitoring rules in the general rule base.
  6. 如权利要求1至5中任一项所述的方法，其特征在于，所述通用规则库中的监控规则根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。The method according to any one of claims 1 to 5, wherein the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format And whether it is allowed to be empty.
  7. 如权利要求1至5中任一项所述的方法，其特征在于，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。The method according to any one of claims 1 to 5, wherein the monitoring dimension included in one monitoring indicator comprises one or more of the following dimensions: data type, value range, data format, and whether to allow Is empty.
  8. 如权利要求1至5中任一项所述的方法，其特征在于，所述通用规 则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。Method according to any one of claims 1 to 5, wherein said general gauge The library includes: an identifier of the monitoring indicator, and an identifier of the monitoring rule; or the general rule base includes: an identifier of the monitoring indicator, an identifier of the monitoring rule, and a content of the monitoring rule.
  9. 一种数据监控方法，其特征在于，包括：A data monitoring method, comprising:
    获取数据库中关联有监控规则的字段；Obtain a field associated with a monitoring rule in the database;
    从通用规则库中获取所述字段所关联的监控规则；Obtaining the monitoring rules associated with the field from the general rule base;
    根据获取到的监控规则对所述字段的数据进行检查。The data of the field is checked according to the obtained monitoring rule.
  10. 如权利要求9所述的方法，其特征在于，从通用规则库中获取所述字段所关联的监控规则，包括：The method according to claim 9, wherein the obtaining the monitoring rule associated with the field from the general rule base comprises:
    根据所述字段所关联的监控规则的索引信息，从所述通用规则库中获取与所述索引信息对应的监控规则。Obtaining a monitoring rule corresponding to the index information from the general rule base according to the index information of the monitoring rule associated with the field.
  11. 如权利要求9所述的方法，其特征在于，还包括：The method of claim 9 further comprising:
    获取为所述字段配置的所述字段专用的监控规则；Obtaining the field-specific monitoring rules configured for the field;
    根据所述字段专用的监控规则对所述字段的数据进行检查。The data of the field is checked according to the monitoring rules specific to the field.
  12. 如权利要求9所述的方法，其特征在于，还包括：The method of claim 9 further comprising:
    更新所述通用规则库中的监控规则。Update the monitoring rules in the general rule base.
  13. 如权利要求9所述的方法，其特征在于，与所述字段关联的监控规则为所述字段使用的监控指标对应的全部或部分监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。The method according to claim 9, wherein the monitoring rule associated with the field is all or part of a monitoring rule corresponding to the monitoring indicator used by the field; wherein a monitoring rule corresponding to a monitoring indicator is used to detect Whether the data meets the requirements of this monitoring indicator.
  14. 如权利要求9至13中任一项所述的方法，其特征在于，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。The method according to any one of claims 9 to 13, wherein the general rule base includes: an identifier of a monitoring indicator, and an identifier of a monitoring rule; or the general rule base includes: a monitoring indicator The identity, the identity of the monitoring rule, and the content of the monitoring rule.
  15. 如权利要求13或14所述的方法，其特征在于，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。The method according to claim 13 or 14, wherein the monitoring indicator comprises a monitoring dimension comprising one or more of the following dimensions: a data type, a range of values, a data format, and whether it is allowed to be empty.
  16. 一种数据监控管理***，其特征在于，包括：A data monitoring management system, comprising:
    获取模块，用于获取数据库中待配置监控规则的字段；An obtaining module, configured to acquire a field of a monitoring rule to be configured in the database;
    选择模块，用于根据所述字段使用的监控指标，从通用规则库中为所 述字段选择对应的监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求；a selection module for using the monitoring indicators used by the fields from the general rule base The field selects a corresponding monitoring rule; wherein a monitoring rule corresponding to a monitoring indicator is used to detect whether the data meets the requirements of the monitoring indicator;
    配置模块，用于将选择的监控规则与所述字段进行关联。A configuration module for associating selected monitoring rules with the fields.
  17. 如权利要求16所述的***，其特征在于，所述选择模块具体用于：The system of claim 16 wherein said selection module is specifically for:
    根据所述字段使用的监控指标，从通用规则库中为所述字段选择所述监控指标对应的全部或部分监控规则。All or part of the monitoring rules corresponding to the monitoring indicator are selected from the general rule base for the field according to the monitoring indicator used by the field.
  18. 如权利要求16所述的***，其特征在于，所述配置模块具体用于：The system of claim 16 wherein said configuration module is specifically configured to:
    将所述字段与所选择的监控规则在所述通用规则库中的索引信息进行关联。The field is associated with index information of the selected monitoring rule in the general rule base.
  19. 如权利要求16所述的***，其特征在于，所述配置模块还用于：The system of claim 16 wherein said configuration module is further configured to:
    根据所述字段使用的监控指标，为所述字段配置所述字段专用的监控规则。The field-specific monitoring rules are configured for the field according to the monitoring indicators used by the field.
  20. 如权利要求16所述的***，其特征在于，还包括：The system of claim 16 further comprising:
    更新模块，用于更新所述通用规则库中的监控规则。An update module for updating the monitoring rules in the general rule base.
  21. 如权利要求16至20中任一项所述的***，其特征在于，所述通用规则库中的监控规则根据以下维度中的一种或多种进行设置：数据类型、取值范围、数据格式以及是否允许为空。The system according to any one of claims 16 to 20, wherein the monitoring rule in the general rule base is set according to one or more of the following dimensions: data type, value range, data format And whether it is allowed to be empty.
  22. 如权利要求16至20中任一项所述的***，其特征在于，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。The system according to any one of claims 16 to 20, wherein a monitoring indicator comprises a monitoring dimension comprising one or more of the following dimensions: data type, value range, data format, and whether or not to allow Is empty.
  23. 如权利要求16至20中任一项所述的***，其特征在于，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。The system according to any one of claims 16 to 20, wherein the general rule base includes: an identifier of a monitoring indicator, and an identifier of a monitoring rule; or the general rule base includes: a monitoring indicator The identity, the identity of the monitoring rule, and the content of the monitoring rule.
  24. 一种数据监控***，其特征在于，包括：A data monitoring system, comprising:
    第一获取模块，用于获取数据库中关联有监控规则的字段；a first obtaining module, configured to acquire a field associated with a monitoring rule in the database;
    第二获取模块，用于从通用规则库中获取所述字段所关联的监控规则；a second obtaining module, configured to acquire, from a general rule base, a monitoring rule associated with the field;
    监控执行模块，用于根据获取到的监控规则对所述字段的数据进行检 查。a monitoring execution module, configured to check data of the field according to the obtained monitoring rule check.
  25. 如权利要求24所述的***，其特征在于，所述第二获取模块具体用于：The system of claim 24, wherein the second acquisition module is specifically configured to:
    根据所述字段所关联的监控规则的索引信息，从所述通用规则库中获取与所述索引信息对应的监控规则。Obtaining a monitoring rule corresponding to the index information from the general rule base according to the index information of the monitoring rule associated with the field.
  26. 如权利要求24所述的***，其特征在于，所述第一获取模块还用于：获取为所述字段配置的所述字段专用的监控规则；The system according to claim 24, wherein the first obtaining module is further configured to: acquire the field-specific monitoring rule configured for the field;
    所述监控执行模块还用于：根据所述字段专用的监控规则对所述字段的数据进行检查。The monitoring execution module is further configured to: check data of the field according to the monitoring rule specific to the field.
  27. 如权利要求24所述的***，其特征在于，还包括：The system of claim 24, further comprising:
    更新模块，用于更新所述通用规则库中的监控规则。An update module for updating the monitoring rules in the general rule base.
  28. 如权利要求24所述的***，其特征在于，与所述字段关联的监控规则为所述字段使用的监控指标对应的全部或部分监控规则；其中，一个监控指标所对应的监控规则用来检测数据是否符合该监控指标的要求。The system according to claim 24, wherein the monitoring rule associated with the field is all or part of a monitoring rule corresponding to the monitoring indicator used by the field; wherein a monitoring rule corresponding to a monitoring indicator is used to detect Whether the data meets the requirements of this monitoring indicator.
  29. 如权利要求24至28中任一项所述的***，其特征在于，所述通用规则库中包括：监控指标的标识、监控规则的标识；或者，所述通用规则库中包括：监控指标的标识、监控规则的标识，以及监控规则的内容。The system according to any one of claims 24 to 28, wherein the general rule base includes: an identifier of a monitoring indicator, and an identifier of a monitoring rule; or the general rule base includes: a monitoring indicator The identity, the identity of the monitoring rule, and the content of the monitoring rule.
  30. 如权利要求28或29所述的***，其特征在于，一个监控指标包含的监控维度包括以下维度中的一种或多种：数据类型、取值范围、数据格式，以及是否允许为空。 The system according to claim 28 or 29, wherein the monitoring dimension included in one monitoring indicator comprises one or more of the following dimensions: data type, value range, data format, and whether it is allowed to be empty.
PCT/CN2016/110180 2015-12-25 2016-12-15 Data monitoring management method, and data monitoring method and system WO2017107853A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510997452.6A CN106919602A (en) 2015-12-25 2015-12-25 A kind of data monitoring management method, data monitoring method and system
CN201510997452.6 2015-12-25

Publications (1)

Publication Number Publication Date
WO2017107853A1 true WO2017107853A1 (en) 2017-06-29

Family

ID=59089037

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/110180 WO2017107853A1 (en) 2015-12-25 2016-12-15 Data monitoring management method, and data monitoring method and system

Country Status (2)

Country Link
CN (1) CN106919602A (en)
WO (1) WO2017107853A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110580248A (en) * 2019-09-04 2019-12-17 北京明略软件***有限公司 Data management method and device based on rule base expansion
CN110704100A (en) * 2018-06-25 2020-01-17 沃尔缇夫能源***公司 Method and device for configuring functional module software
CN111177176A (en) * 2019-11-18 2020-05-19 腾讯科技（深圳）有限公司 Data detection method, device and storage medium
CN111221956A (en) * 2019-12-26 2020-06-02 国网宁夏电力有限公司中卫供电公司 PMS distribution network equipment data quality checking method for power management system
CN111324511A (en) * 2020-02-24 2020-06-23 北京达佳互联信息技术有限公司 Alarm rule generation method and device, electronic equipment and storage medium
CN111338817A (en) * 2020-02-21 2020-06-26 中国农业银行股份有限公司 Interface preprocessing method and device
CN112579571A (en) * 2020-12-17 2021-03-30 招商局金融科技有限公司 Monitoring data configuration method, data monitoring method, device, equipment and storage medium

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109358943B (en) * 2018-11-09 2022-03-22 金色熊猫有限公司 Page configuration method and device, electronic equipment and storage medium
CN109510824B (en) * 2018-11-12 2021-06-04 中国银行股份有限公司 Interface message checking method and device
CN111381768A (en) * 2018-12-29 2020-07-07 北京亿阳信通科技有限公司 Data monitoring method and device
CN110297739A (en) * 2019-06-04 2019-10-01 平安科技（深圳）有限公司 The data monitoring method and relevant device of platform are administered based on data
CN110400046A (en) * 2019-06-18 2019-11-01 平安科技（深圳）有限公司 Monitor control index method of adjustment, device, equipment and computer readable storage medium
CN111080834A (en) * 2019-10-31 2020-04-28 北京航天自动控制研究所 Method for configuring criteria for polling data of test and launch control software by using index
CN111045874B (en) * 2019-10-31 2023-05-12 北京航天自动控制研究所 Universal method for modeling solving flow step data interpretation by test initiation control software

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101625686A (en) * 2008-07-08 2010-01-13 阿里巴巴集团控股有限公司 Method and system for monitoring data consistency between plurality of databases
US20130014106A1 (en) * 2011-07-05 2013-01-10 Fujitsu Limited Information processing apparatus, computer-readable medium storing information processing program, and management method
CN104636483A (en) * 2015-02-16 2015-05-20 广东省公安厅 Data monitoring method
CN104636484A (en) * 2015-02-16 2015-05-20 广东省公安厅 Monitoring task generating method and device based on data monitoring
CN105095056A (en) * 2015-08-14 2015-11-25 焦点科技股份有限公司 Method for monitoring data in data warehouse

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101625686A (en) * 2008-07-08 2010-01-13 阿里巴巴集团控股有限公司 Method and system for monitoring data consistency between plurality of databases
US20130014106A1 (en) * 2011-07-05 2013-01-10 Fujitsu Limited Information processing apparatus, computer-readable medium storing information processing program, and management method
CN104636483A (en) * 2015-02-16 2015-05-20 广东省公安厅 Data monitoring method
CN104636484A (en) * 2015-02-16 2015-05-20 广东省公安厅 Monitoring task generating method and device based on data monitoring
CN105095056A (en) * 2015-08-14 2015-11-25 焦点科技股份有限公司 Method for monitoring data in data warehouse

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704100A (en) * 2018-06-25 2020-01-17 沃尔缇夫能源***公司 Method and device for configuring functional module software
CN110580248A (en) * 2019-09-04 2019-12-17 北京明略软件***有限公司 Data management method and device based on rule base expansion
CN111177176A (en) * 2019-11-18 2020-05-19 腾讯科技（深圳）有限公司 Data detection method, device and storage medium
CN111177176B (en) * 2019-11-18 2023-05-16 腾讯科技（深圳）有限公司 Data detection method, device and storage medium
CN111221956A (en) * 2019-12-26 2020-06-02 国网宁夏电力有限公司中卫供电公司 PMS distribution network equipment data quality checking method for power management system
CN111338817A (en) * 2020-02-21 2020-06-26 中国农业银行股份有限公司 Interface preprocessing method and device
CN111338817B (en) * 2020-02-21 2023-11-03 中国农业银行股份有限公司 Interface preprocessing method and device
CN111324511A (en) * 2020-02-24 2020-06-23 北京达佳互联信息技术有限公司 Alarm rule generation method and device, electronic equipment and storage medium
CN111324511B (en) * 2020-02-24 2023-10-31 北京达佳互联信息技术有限公司 Alarm rule generation method and device, electronic equipment and storage medium
CN112579571A (en) * 2020-12-17 2021-03-30 招商局金融科技有限公司 Monitoring data configuration method, data monitoring method, device, equipment and storage medium
CN112579571B (en) * 2020-12-17 2024-03-19 招商局金融科技有限公司 Monitoring data configuration, data monitoring method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN106919602A (en) 2017-07-04

Similar Documents

Publication Publication Date Title
WO2017107853A1 (en) Data monitoring management method, and data monitoring method and system
AU2015315203B2 (en) Conditional validation rules
CN110309125B (en) Data verification method, electronic device and storage medium
WO2020024356A1 (en) Warehouse asset inventory method and apparatus, computer device and storage medium
CN110795455A (en) Dependency relationship analysis method, electronic device, computer device and readable storage medium
CN108958959B (en) Method and device for detecting hive data table
CN111061833A (en) Data processing method and device, electronic equipment and computer readable storage medium
EP2862101B1 (en) Method and a consistency checker for finding data inconsistencies in a data repository
CN107291745B (en) Data index management method and device
WO2019055553A1 (en) Dynamic representation of exploration and/or production entity relationships
CN108073707B (en) Financial business data updating method and device and computer readable storage medium
WO2023134134A1 (en) Method and apparatus for generating association viewing model, and computer device and storage medium
CN111061733B (en) Data processing method, device, electronic equipment and computer readable storage medium
CN112817810A (en) Equipment data monitoring method and device, terminal equipment and storage medium
CN108427637B (en) Test case recommendation method, electronic device and readable storage medium
CN107330031B (en) Data storage method and device and electronic equipment
CN112988780A (en) Data checking method and device, storage medium and electronic equipment
CN114741392A (en) Data query method and device, electronic equipment and storage medium
CN114357032A (en) Data quality monitoring method and device, electronic equipment and storage medium
CN113934729A (en) Data management method based on knowledge graph, related equipment and medium
CN111143356B (en) Report retrieval method and device
CN113778996A (en) Large data stream data processing method and device, electronic equipment and storage medium
CN109299613B (en) Database partition authority setting method and terminal equipment
CN110618991A (en) Method, system, equipment and storage medium for automatically inputting parameter values
CN111078949A (en) Product knowledge storage method and device, computer equipment and readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16877664

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16877664

Country of ref document: EP

Kind code of ref document: A1