CN112835991A - System, method, device and storage medium for monitoring data - Google Patents
System, method, device and storage medium for monitoring data Download PDFInfo
- Publication number
- CN112835991A CN112835991A CN201911165756.0A CN201911165756A CN112835991A CN 112835991 A CN112835991 A CN 112835991A CN 201911165756 A CN201911165756 A CN 201911165756A CN 112835991 A CN112835991 A CN 112835991A
- Authority
- CN
- China
- Prior art keywords
- data
- storage unit
- dimension storage
- user
- strategy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 98
- 238000013500 data storage Methods 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 23
- 230000006399 behavior Effects 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000003672 processing method Methods 0.000 description 4
- 238000013480 data collection Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
- G06F16/287—Visualization; Browsing
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a system, method, device and storage medium for monitoring data, the system comprising: the data storage module comprises a data dimension storage unit, a strategy dimension storage unit and a processing dimension storage unit: the data dimension storage unit is used for storing the user-related data in a classified manner based on data dimensions; the strategy dimension storage unit is used for storing strategy data; the processing dimension storage unit is used for storing processing mode data corresponding to the strategy data; the data query module is used for acquiring related data from the data dimension storage unit, the strategy dimension storage unit and the processing dimension storage unit; and the data summarization module is used for generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data. The data filtering method and device can support a multi-dimensional data filtering scheme and can improve efficiency and accuracy of data acquisition and use.
Description
Technical Field
The present disclosure relates to the field of information processing technologies, and in particular, to a system, a method, a device, and a storage medium for monitoring data.
Background
In policy making, personnel such as products and operators often need to obtain output data meeting expectations by means of data meeting required characteristics due to various factors. In order to read and use such data, searching from various processes and databases is usually required, and problems of great difficulty and low integrity of data collection exist.
Disclosure of Invention
The present disclosure provides a system, a method, a device and a storage medium for monitoring data, so as to at least solve the technical problems that in the related art, searching from each process and database is usually required, the difficulty of data collection is large, and the integrity of data collection is low. The technical scheme of the disclosure is as follows:
according to a first aspect of embodiments of the present disclosure, there is provided a system for monitoring data, including:
the data storage module comprises a data dimension storage unit, a strategy dimension storage unit and a processing dimension storage unit, wherein:
the data dimension storage unit is used for storing the user-related data in a classified manner based on data dimensions;
the policy dimension storage unit is used for storing policy data, and the policy data comprises the user related data stored based on the policy dimension;
the processing dimension storage unit is used for storing processing mode data corresponding to the strategy data;
the data query module is used for acquiring user related data which accord with set rules from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit and acquiring corresponding processing mode data from the processing dimension storage unit;
and the data summarization module is used for generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data.
In one embodiment, the system for monitoring data further includes:
the data alarm module is used for:
setting an index threshold value for an index to be monitored, and determining an index value of the index to be monitored according to data in the data table;
and generating alarm information in response to the index value exceeding the threshold value.
In an embodiment, the system for monitoring data further includes:
and the data display module is used for visually displaying the data table.
In one embodiment, the data presentation module is further configured to visually present data in the data table in at least one of a table, a chart, and a line graph.
In an embodiment, the data dimension storage unit is a relational database, the policy dimension storage unit is a non-relational database, and the processing dimension storage unit is a local cache;
the data query module comprises a data query unit;
the data query unit is used for tracking the user behavior log which accords with the set rule in the relational database, synchronizing the user behavior log to the non-relational database, acquiring associated policy data from the non-relational database, and acquiring associated processing mode data in the local cache according to the policy data.
In one embodiment, the data summarization module further comprises:
and the data warehouse is used for storing the user-related data which accords with the setting rule and one or more items of the associated strategy data and the associated processing mode data in the data warehouse and providing a setting function based on the synchronous data, wherein the setting function comprises at least one of an offline calculation function and a data reporting function.
In an embodiment, the setting rule includes a negative-direction pressing rule, the user-related data includes at least one of user data, video data, and live data, and the user data, the video data, and the live data are classified and stored according to policy dimensions.
In one embodiment, the data query module further comprises:
and the marking unit is used for marking the data acquired from the data dimension storage unit, the strategy dimension storage unit or the processing dimension storage unit.
According to a first aspect of embodiments of the present disclosure, there is provided a method of monitoring data, including:
classifying and storing user-related data in a data dimension storage unit based on data dimensions, storing strategy data in a strategy dimension storage unit, wherein the strategy data comprises the user-related data stored based on strategy dimensions, and storing processing mode data corresponding to the strategy data in a processing dimension storage unit;
acquiring user-related data which accords with a set rule from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit, and acquiring corresponding processing mode data from the processing dimension storage unit;
and generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data.
In an embodiment, the method for monitoring data further includes:
setting an index threshold value for an index to be monitored, and determining an index value of the index to be monitored according to data in the data table;
and generating alarm information in response to the index value exceeding the threshold value.
In an embodiment, the method for monitoring data further includes:
and visually displaying the data table.
In an embodiment, the step of visually displaying the data table further includes:
the data in the data sheet is visually presented in at least one of a table, a chart, and a line graph.
In an embodiment, the data dimension storage unit is a relational database, the policy dimension storage unit is a non-relational database, and the processing dimension storage unit is a local cache;
the step of obtaining the user-related data meeting the set rule from the data dimension storage unit, obtaining the corresponding policy data from the policy dimension storage unit, and obtaining the corresponding processing mode data from the processing dimension storage unit further includes:
tracking the user behavior log which accords with the set rule in the relational database, synchronizing the user behavior log to the non-relational database, acquiring associated strategy data from the non-relational database, and acquiring associated processing mode data in the local cache according to the strategy data.
In an embodiment, the step of generating a data table based on the user-related data meeting the setting rule and one or more of the associated policy data and processing manner data further includes:
storing the user-related data meeting the setting rules and one or more of the associated policy data and processing mode data in a data repository, and providing setting functions based on the synchronized data, the setting functions including at least one of an offline calculation function and a data reporting function.
In an embodiment, the setting rule includes a negative-direction pressing rule, the user-related data includes at least one of user data, video data, and live data, and the user data, the video data, and the live data are classified and stored according to policy dimensions.
In an embodiment, the method for monitoring data further includes:
and marking the data acquired from the data dimension storage unit, the strategy dimension storage unit or the processing dimension storage unit.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device for monitoring data, including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement a method of monitoring data as described in any of the above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a storage medium having instructions that, when executed by a processor of an electronic device for monitoring data, enable the electronic device for monitoring data to perform the method for monitoring data as described in any one of the above.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product, which, when executed by a processor of an electronic device for monitoring data, enables the electronic device for monitoring data to perform the method of monitoring data as defined in any one of the above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the system for monitoring data of the present disclosure includes: the data storage module comprises a data dimension storage unit, a strategy dimension storage unit and a processing dimension storage unit, wherein: the data dimension storage unit is used for storing the user-related data in a classified manner based on data dimensions; the policy dimension storage unit is used for storing policy data, and the policy data comprises the user related data stored based on the policy dimension; the processing dimension storage unit is used for storing processing mode data corresponding to the strategy data; the data query module is used for acquiring user related data which accord with set rules from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit and acquiring corresponding processing mode data from the processing dimension storage unit; and the data summarization module is used for generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data. The data filtering method and device can support a multi-dimensional data filtering scheme and can improve efficiency and accuracy of data acquisition and use.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a block diagram illustrating an apparatus for monitoring data according to an exemplary embodiment.
Fig. 2 is a block diagram illustrating an apparatus for monitoring data according to yet another exemplary embodiment.
FIG. 3 is a flow chart illustrating a method of monitoring data in accordance with an exemplary embodiment.
FIG. 4 is a flow chart illustrating a method of monitoring data in accordance with yet another exemplary embodiment.
FIG. 5 is a flow chart illustrating a method of monitoring data in accordance with another exemplary embodiment.
FIG. 6 is a block diagram illustrating an electronic device monitoring data in accordance with an exemplary embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
FIG. 1 is a block diagram illustrating an apparatus for monitoring data according to an exemplary embodiment. As shown in fig. 1, the system includes: data storage module 110, data query module 120, and data summarization module 130, wherein:
the data storage module 110 further includes a data dimension storage unit 111, a policy dimension storage unit 112, and a processing dimension storage unit 113, wherein:
the data dimension storage unit 111 is used for storing the user-related data in a classified manner based on the data dimension.
In this embodiment, the user-related data may include one or more of user data, live data, and video data, which is not limited in this embodiment.
The policy dimension storage unit 112 is configured to store policy data, which includes the user-related data stored based on the policy dimension.
In this embodiment, the policy dimension storage unit may store the user-related data based on the policy dimension, where the user-related data may include data stored in the data dimension storage unit in the data dimension.
The processing dimension storage unit 113 is configured to store processing manner data corresponding to the policy data.
Further, the data query module 120 is configured to obtain the user-related data meeting the set rule from the data dimension storage unit, obtain the corresponding policy data from the policy dimension storage unit, and obtain the corresponding processing manner data from the processing dimension storage unit.
In this embodiment, the setting rule may be freely set by a developer according to an actual application scenario and a service requirement, for example, the setting rule is a negative-direction pressing rule, which is not limited in this embodiment.
It should be noted that the user data, the video data, and the live data may be classified and stored according to different policies.
The data summarization module 130 is configured to generate a data table based on the user-related data that meets the set rule, and one or more of the associated policy data and the processing manner data.
For example, after the user sets the rule for acquiring the data based on the user's own requirement, the data summarization module may generate the data table based on the user-related data that meets the set rule and one or more of the policy data and the processing manner data associated with the user-related data.
As can be seen from the above description, the system for monitoring data of the present embodiment includes: the data storage module comprises a data dimension storage unit, a strategy dimension storage unit and a processing dimension storage unit, wherein: the data dimension storage unit is used for storing the user-related data in a classified manner based on data dimensions; the policy dimension storage unit is used for storing policy data, and the policy data comprises the user related data stored based on the policy dimension; the processing dimension storage unit is used for storing processing mode data corresponding to the strategy data; the data query module is used for acquiring user related data which accord with set rules from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit and acquiring corresponding processing mode data from the processing dimension storage unit; and the data summarization module is used for generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data. The system of the embodiment stores the user-related data by adopting the data dimension, the strategy dimension and the processing dimension, supports to acquire the user-related data which accords with the set rule from the data dimension storage unit based on the data query module, acquires the corresponding strategy data from the strategy dimension storage unit, and acquires the corresponding processing mode data from the processing dimension storage unit, so that the data summarization module supports to generate the data table based on the user-related data which accords with the set rule and one or more of the associated strategy data and processing mode data, a data filtering scheme which supports multiple dimensions can be realized, and the efficiency and the accuracy of acquiring and using the data can be improved.
Fig. 2 is a block diagram illustrating an apparatus for monitoring data according to yet another exemplary embodiment. As shown in fig. 2, the data storage module 210, the data query module 220, and the data summarization module 230 of the present embodiment have the same functions as the data storage module 110, the data query module 120, and the data summarization module 130 in the embodiment shown in fig. 1, and are not described herein again. As shown in fig. 2, the system further includes:
a data alert module 240 to:
setting an index threshold value for an index to be monitored, and determining an index value of the index to be monitored according to data in the data table;
and generating alarm information in response to the index value exceeding the threshold value.
For example, the data alarm module 240 may set an index threshold for the index to be monitored, and determine the index value of the index to be monitored according to the data in the data table, so as to generate alarm information when it is monitored that the index value exceeds the threshold.
It will be understood that, in practice, the relevant threshold values, such as the same ratio, the ring ratio, the value or the standard deviation, etc., may be set for some important indexes of the data in the data table based on the needs of the user. Further, when the index value of the related data exceeds the threshold value, alarm information may be generated.
In an embodiment, the content and the form of the alarm information may be set by a developer according to actual business needs, for example, the alarm information may be sent to a relevant user through an email, so as to achieve the purpose of reminding the user of finding a problem in time, which is not limited in this embodiment.
In another embodiment, as shown in fig. 2, the system for monitoring data may further include:
and a data display module 250, configured to perform visual display on the data table.
In this embodiment, the data display module 250 may be further configured to visually display data in the data table by using at least one of a table, a chart, and a line graph.
For example, the data presentation module 250 may perform visual presentation based on the acquired data, such as flexible data analysis and data mining from multiple dimensions of time, criminal name, work type, presentation page, pressing manner, and AB test. Optionally, the data table may adopt a form of a repeated table and chart, the important indexes may be displayed in a line chart manner, and a plurality of indexes are displayed in detail by using the table, which may help a user to realize a quick overview and an accurate analysis of data.
In the above embodiment, the data dimension storage unit 211 may also be a relational database, the policy dimension storage unit 212 may be a non-relational database, and the processing dimension storage unit 213 may be a local cache;
in this embodiment, the data query module 220 may further include a data query unit 221, configured to track, in the relational database, a user behavior log that meets the setting rule, synchronize the user behavior log with the non-relational database, obtain associated policy data from the non-relational database, and obtain associated processing mode data in the local cache according to the policy data.
For example, each table in the relational database may be used to store data tagged with the same policy tag information, respectively, and the non-relational database may be used to store the target data with the unique identifier of the target data as a data handle and the policy tag information of the target data as a data value. In an embodiment, the local cache may be configured to store the data marked with the processing flag information, so that the accuracy of subsequent data searching and acquiring efficiency may be improved, and the user requirement may be met.
In an optional embodiment, the data summarization module 230 may further include a data repository 231; the data warehouse 231 may be configured to store the user-related data meeting the setting rule and one or more of the associated policy data and processing manner data in the data warehouse, and provide a setting function based on the synchronized data, wherein the setting function includes at least one of an offline calculation function and a data reporting function.
For example, the data warehouse may be used for data synchronization based on logs of a relational database, and may provide an offline calculation function, a data reporting function, and the like based on synchronized data.
In another alternative embodiment, the data query module 220 may further include a marking unit 222. The marking unit 222 may be configured to mark data acquired from the data dimension storage unit 211, the policy dimension storage unit 212, or the processing dimension storage unit 213 in the above implementation.
It should be noted that, the specific marking manner of the marking unit for the data acquired from each storage unit may be set by a developer according to actual business needs, and this embodiment does not limit this.
In the foregoing embodiments, the setting rule may include a negative suppressing rule, and the user-related data includes at least one of user data, video data, and live data. That is to say, the system of this embodiment may finally implement generating at least one of negative-direction push general data, negative-direction user general data, negative-direction live broadcast general data, and negative-direction video general data according to the obtained service data.
As can be seen from the above description, in this embodiment, since the data alarm module sets an index threshold for the to-be-monitored index, determines the index value of the to-be-monitored index according to the data in the data table, and generates the alarm information in response to the index value exceeding the threshold, the purpose of reminding the user of finding a problem in time can be achieved, and the data in the data table is visually displayed by the data display module in at least one of a table, a chart and a line graph, the visualization degree of the data can be improved on the basis of realizing a data filtering scheme supporting multiple dimensions, and the efficiency and accuracy of acquiring and using the data are further improved.
FIG. 3 is a flow chart illustrating a method of monitoring data in accordance with an exemplary embodiment. As shown in fig. 3, the method comprises the following steps S101-S103:
in step S101, the user-related data is classified and stored based on the data dimension in the data dimension storage unit, the policy data is stored in the policy dimension storage unit, and the processing manner data corresponding to the policy data is stored in the processing dimension storage unit.
Wherein the policy data comprises the user-related data stored based on a policy dimension.
In step S102, user-related data meeting the set rule is obtained from the data dimension storage unit, corresponding policy data is obtained from the policy dimension storage unit, and corresponding processing manner data is obtained from the processing dimension storage unit.
In step S103, a data table is generated based on the user-related data that meets the setting rule and one or more of the associated policy data and processing method data.
As can be seen from the above description, in the present embodiment, by classifying and storing user-related data based on data dimensions in a data dimension storage unit, and storing policy data in a policy dimension storage unit, and acquiring user-related data that meets a set rule from the data dimension storage unit, and acquiring corresponding policy data from the policy dimension storage unit, and acquiring corresponding processing manner data from the processing dimension storage unit, and generating a data table based on one or more of the user-related data that meets the set rule, and associated policy data and processing manner data, since the data dimensions, policy dimensions, and processing dimensions are used to store the user-related data, and the acquisition of the user-related data that meets the set rule from the data dimension storage unit is supported, and the corresponding policy data is acquired from the policy dimension storage unit, and acquiring corresponding processing mode data from the processing dimension storage unit, so that a data table can be generated based on the user related data which accords with the set rule and one or more items of associated strategy data and processing mode data, a multi-dimensional data filtering scheme can be supported, and the efficiency and the accuracy of acquiring and using data can be improved.
FIG. 4 is a flow chart illustrating a method of monitoring data in accordance with yet another exemplary embodiment. As shown in fig. 4, the method comprises the following steps S201-S206:
in step S201, the user-related data is classified and stored based on the data dimension in the data dimension storage unit, the policy data is stored in the policy dimension storage unit, and the processing manner data corresponding to the policy data is stored in the processing dimension storage unit.
Wherein the policy data comprises the user-related data stored based on a policy dimension.
In step S202, user-related data meeting the set rule is obtained from the data dimension storage unit, corresponding policy data is obtained from the policy dimension storage unit, and corresponding processing manner data is obtained from the processing dimension storage unit.
In step S203, a data table is generated based on the user-related data that meets the setting rule and one or more of the associated policy data and processing method data.
For the explanation and description of steps S201 to S203, reference may be made to the above embodiments, which are not described herein again.
In step S204, an index threshold is set for the to-be-monitored index, and the index value of the to-be-monitored index is determined according to the data in the data table.
In step S205, in response to the index value exceeding the threshold value, alarm information is generated.
In step S206, the data table is visually displayed.
For example, the step of visually displaying the data table in step S206 may further include:
the data in the data sheet is visually presented in at least one of a table, a chart, and a line graph.
In an alternative embodiment, the data dimension storage unit may be a relational database, the policy dimension storage unit may be a non-relational database, and the processing dimension storage unit may be a local cache.
On this basis, the acquiring, in the step S202, the user-related data meeting the setting rule from the data dimension storage unit, acquiring the corresponding policy data from the policy dimension storage unit, and acquiring the corresponding processing manner data from the processing dimension storage unit may further include:
tracking the user behavior log which accords with the set rule in the relational database, synchronizing the user behavior log to the non-relational database, acquiring associated strategy data from the non-relational database, and acquiring associated processing mode data in the local cache according to the strategy data.
In an embodiment, the generating a data table based on the user-related data meeting the setting rule and one or more of the associated policy data and the processing method data in step S203 may further include:
storing the user-related data meeting the setting rules and one or more of the associated policy data and processing mode data in a data repository, and providing setting functions based on the synchronized data, the setting functions including at least one of an offline calculation function and a data reporting function.
In an actual application scenario, the setting rule may include a negative-direction suppressing rule, the user-related data includes at least one of user data, video data, and live data, and the user data, the video data, and the live data are classified and stored according to policy dimensions.
As can be seen from the above description, in this embodiment, an index threshold is set for an index to be monitored, an index value of the index to be monitored is determined according to data in the data table, and alarm information is generated in response to the index value exceeding the threshold, so that a user can be reminded of finding a problem in time, and by visually displaying data in the data table by using at least one of a table, a chart and a line graph, the visualization degree of the data can be improved on the basis of realizing a data filtering scheme supporting multiple dimensions, and the efficiency and accuracy of acquiring and using the data are further improved.
FIG. 5 is a flow chart illustrating a method of monitoring data in accordance with another exemplary embodiment. As shown in FIG. 5, the method includes the following steps S301-S306
In step S301, the user-related data is classified and stored in the data dimension storage unit based on the data dimension, the policy data is stored in the policy dimension storage unit, and the processing manner data corresponding to the policy data is stored in the processing dimension storage unit.
Wherein the policy data comprises the user-related data stored based on a policy dimension.
In step S302, user-related data meeting a set rule is obtained from the data dimension storage unit, corresponding policy data is obtained from the policy dimension storage unit, corresponding processing manner data is obtained from the processing dimension storage unit, and then data obtained from the data dimension storage unit, the policy dimension storage unit, or the processing dimension storage unit is marked.
For example, after the user-related data meeting the set rule is acquired from the data dimension storage unit, the corresponding policy data is acquired from the policy dimension storage unit, and the corresponding processing manner data is acquired from the processing dimension storage unit, the acquired data may be marked based on a preset marking manner.
It should be noted that, the specific marking manner of the data obtained from each storage unit may be set by a developer according to actual business needs, and this embodiment does not limit this.
In step S303, a data table is generated based on the user-related data that meets the setting rule and one or more of the associated policy data and processing method data.
In step S304, an index threshold is set for the index to be monitored, and the index value of the index to be monitored is determined according to the data in the data table.
In step S305, in response to the index value exceeding the threshold value, alarm information is generated.
In step S306, the data table is visually displayed.
For the explanation and description of steps S301 and S303-S306, reference may be made to the foregoing embodiments, which are not repeated herein.
With regard to the method in the above-described embodiment, the specific manner in which the respective steps are performed has been described in detail in the embodiment related to the system, and will not be described in detail herein.
It should be noted that, all the above-mentioned optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described in detail herein.
Embodiments of the system for monitoring data of the present invention may be applied to network devices. The system embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. Taking a software implementation as an example, as a system in a logical sense, the system is formed by reading, by a processor of a device in which the system is located, a corresponding computer program instruction in a nonvolatile memory into an internal memory for running, where the computer program is used to execute the method for monitoring data provided by the embodiment shown in fig. 3 to 5. From a hardware level, as shown in fig. 6, which is a hardware structure diagram of the device for monitoring data according to the present invention, besides the processor, the network interface, the memory and the nonvolatile memory shown in fig. 6, the device may also include other hardware, such as a forwarding chip responsible for processing packets, etc.; the device may also be a distributed device in terms of hardware structure, and may include multiple interface cards to facilitate expansion of message processing at the hardware level.
On the other hand, the present application also provides a computer-readable storage medium, which, when a computer program stored in the storage medium is executed by a processor of an electronic device for monitoring data, enables the electronic device for monitoring data to execute the method for monitoring data provided in the embodiments shown in fig. 3 to 5.
On the other hand, the present application also provides a computer program product, which, when executed by a processor of an electronic device for monitoring data, enables the electronic device for monitoring data to execute the method for monitoring data provided by the embodiments shown in fig. 3 to 5.
For the system embodiment, since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment for relevant points. The above-described system embodiments are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A system for monitoring data, comprising:
the data storage module comprises a data dimension storage unit, a strategy dimension storage unit and a processing dimension storage unit, wherein:
the data dimension storage unit is used for storing the user-related data in a classified manner based on data dimensions;
the policy dimension storage unit is used for storing policy data, and the policy data comprises the user related data stored based on the policy dimension;
the processing dimension storage unit is used for storing processing mode data corresponding to the strategy data;
the data query module is used for acquiring user related data which accord with set rules from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit and acquiring corresponding processing mode data from the processing dimension storage unit;
and the data summarization module is used for generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data.
2. The system for monitoring data according to claim 1, further comprising:
the data alarm module is used for:
setting an index threshold value for an index to be monitored, and determining an index value of the index to be monitored according to data in the data table;
and generating alarm information in response to the index value exceeding the threshold value.
3. The system for monitoring data according to claim 1, further comprising:
and the data display module is used for visually displaying the data table.
4. The system for monitoring data of claim 3, wherein the data presentation module is further configured to visually present data in the spreadsheet in at least one of a table, a chart, and a line graph.
5. The system for monitoring data according to claim 1, wherein the data dimension storage unit is a relational database, the policy dimension storage unit is a non-relational database, and the processing dimension storage unit is a local cache;
the data query module comprises a data query unit;
the data query unit is used for tracking the user behavior log which accords with the set rule in the relational database, synchronizing the user behavior log to the non-relational database, acquiring associated policy data from the non-relational database, and acquiring associated processing mode data in the local cache according to the policy data.
6. The system for monitoring data of claim 5, wherein the data summarization module further comprises:
and the data warehouse is used for storing the user-related data which accords with the setting rule and one or more items of the associated strategy data and the associated processing mode data in the data warehouse and providing a setting function based on the synchronous data, wherein the setting function comprises at least one of an offline calculation function and a data reporting function.
7. The system for monitoring data according to claim 1, wherein the set rule includes a negative-direction pressing rule, the user-related data includes at least one of user data, video data and live data, and the user data, the video data and the live data are respectively sorted and stored according to policy dimensions.
8. A method of monitoring data, comprising:
classifying and storing user-related data in a data dimension storage unit based on data dimensions, storing strategy data in a strategy dimension storage unit, wherein the strategy data comprises the user-related data stored based on strategy dimensions, and storing processing mode data corresponding to the strategy data in a processing dimension storage unit;
acquiring user-related data which accords with a set rule from the data dimension storage unit, acquiring corresponding strategy data from the strategy dimension storage unit, and acquiring corresponding processing mode data from the processing dimension storage unit;
and generating a data table based on the user related data which accords with the set rule and one or more items of the associated strategy data and the processing mode data.
9. An electronic device for monitoring data, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of monitoring data of claim 8.
10. A storage medium having instructions that, when executed by a processor of an electronic device that monitors data, enable the electronic device that monitors data to perform the method of monitoring data of claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911165756.0A CN112835991B (en) | 2019-11-25 | 2019-11-25 | System, method, device and storage medium for monitoring data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911165756.0A CN112835991B (en) | 2019-11-25 | 2019-11-25 | System, method, device and storage medium for monitoring data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112835991A true CN112835991A (en) | 2021-05-25 |
| CN112835991B CN112835991B (en) | 2023-11-21 |
Family
ID=75922239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911165756.0A Active CN112835991B (en) | 2019-11-25 | 2019-11-25 | System, method, device and storage medium for monitoring data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112835991B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090064037A1 (en) * | 2007-08-28 | 2009-03-05 | International Business Machines Corporation | Apparatus, system, and method for xml based disconnected data access for multivalued/hierarchical databases |
| US20130006966A1 (en) * | 2011-06-09 | 2013-01-03 | International Business Machines Corporation | Relational Query Planning for Non-Relational Data Sources |
| CN104572401A (en) * | 2015-02-09 | 2015-04-29 | 浪潮软件股份有限公司 | Alarming method and alarming system |
| CN105208098A (en) * | 2015-08-24 | 2015-12-30 | 用友网络科技股份有限公司 | Cloud monitoring system realization device and method |
| US20160019249A1 (en) * | 2014-07-18 | 2016-01-21 | Wipro Limited | System and method for optimizing storage of multi-dimensional data in data storage |
| CN107491549A (en) * | 2017-08-29 | 2017-12-19 | 云熠信息科技有限公司 | A kind of data processing method and system |
| CN107741955A (en) * | 2017-09-15 | 2018-02-27 | 平安科技（深圳）有限公司 | Business datum monitoring method, device, terminal device and storage medium |
| CN108021809A (en) * | 2017-12-19 | 2018-05-11 | 北京明朝万达科技股份有限公司 | A kind of data processing method and system |
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN109165905A (en) * | 2018-06-26 | 2019-01-08 | 北京炎黄盈动科技发展有限责任公司 | Processing method, device, equipment and the readable storage medium storing program for executing of operation flow data |
| CN109189790A (en) * | 2018-08-23 | 2019-01-11 | 平安科技（深圳）有限公司 | Data managing method, device, computer equipment and storage medium |
| CN109376176A (en) * | 2018-08-22 | 2019-02-22 | 阿里巴巴集团控股有限公司 | A kind of data configuration method and device, a kind of calculating equipment and storage medium |
| CN109446253A (en) * | 2018-09-25 | 2019-03-08 | 平安科技（深圳）有限公司 | Data query control method, device, computer equipment and storage medium |
| CN109783512A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
| CN110032560A (en) * | 2018-11-06 | 2019-07-19 | 阿里巴巴集团控股有限公司 | A kind of generation method and device monitoring chart |
-
2019
- 2019-11-25 CN CN201911165756.0A patent/CN112835991B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090064037A1 (en) * | 2007-08-28 | 2009-03-05 | International Business Machines Corporation | Apparatus, system, and method for xml based disconnected data access for multivalued/hierarchical databases |
| US20130006966A1 (en) * | 2011-06-09 | 2013-01-03 | International Business Machines Corporation | Relational Query Planning for Non-Relational Data Sources |
| US20160019249A1 (en) * | 2014-07-18 | 2016-01-21 | Wipro Limited | System and method for optimizing storage of multi-dimensional data in data storage |
| CN104572401A (en) * | 2015-02-09 | 2015-04-29 | 浪潮软件股份有限公司 | Alarming method and alarming system |
| CN105208098A (en) * | 2015-08-24 | 2015-12-30 | 用友网络科技股份有限公司 | Cloud monitoring system realization device and method |
| CN108132958A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | A kind of multi-level buffer data storage, inquiry, scheduling and processing method and processing device |
| CN107491549A (en) * | 2017-08-29 | 2017-12-19 | 云熠信息科技有限公司 | A kind of data processing method and system |
| CN107741955A (en) * | 2017-09-15 | 2018-02-27 | 平安科技（深圳）有限公司 | Business datum monitoring method, device, terminal device and storage medium |
| CN108021809A (en) * | 2017-12-19 | 2018-05-11 | 北京明朝万达科技股份有限公司 | A kind of data processing method and system |
| CN109165905A (en) * | 2018-06-26 | 2019-01-08 | 北京炎黄盈动科技发展有限责任公司 | Processing method, device, equipment and the readable storage medium storing program for executing of operation flow data |
| CN109376176A (en) * | 2018-08-22 | 2019-02-22 | 阿里巴巴集团控股有限公司 | A kind of data configuration method and device, a kind of calculating equipment and storage medium |
| CN109189790A (en) * | 2018-08-23 | 2019-01-11 | 平安科技（深圳）有限公司 | Data managing method, device, computer equipment and storage medium |
| CN109446253A (en) * | 2018-09-25 | 2019-03-08 | 平安科技（深圳）有限公司 | Data query control method, device, computer equipment and storage medium |
| CN110032560A (en) * | 2018-11-06 | 2019-07-19 | 阿里巴巴集团控股有限公司 | A kind of generation method and device monitoring chart |
| CN109783512A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112835991B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105447184B (en) | Information extraction method and device | |
| CN110413634B (en) | Data query method, system, device and computer readable storage medium | |
| WO2016107155A1 (en) | Asset management method and system | |
| CN107977396B (en) | Method and device for updating data table of KeyValue database | |
| CN111339171B (en) | Data query method, device and equipment | |
| CN103702053A (en) | Video storage and search method and system as well as monitoring system | |
| CN109472017B (en) | Method and device for obtaining relevant information of text court deeds of referee to be generated | |
| CN111737443B (en) | Answer text processing method and device and key text determining method | |
| CN104504028A (en) | Index value calculation method, device and system | |
| JP2023069915A (en) | Trade mark image monitoring device, trade mark image monitoring method, and trade mark image monitoring program | |
| CN110442439B (en) | Task process processing method and device and computer equipment | |
| US20210064660A1 (en) | Graph search using index vertices | |
| CN112835991B (en) | System, method, device and storage medium for monitoring data | |
| CN111209284B (en) | Meta-data-based sub-table method and device | |
| CN111143329B (en) | Data processing method and device | |
| CN114238823A (en) | Method and device for accessing website, computer equipment and storage medium | |
| US20180189699A1 (en) | A method and system for locating regulatory information | |
| CN113934729A (en) | Data management method based on knowledge graph, related equipment and medium | |
| CN111611451A (en) | Trademark infringement early warning method and system | |
| CN111597235A (en) | Data processing method and device and electronic equipment | |
| CN111078949A (en) | Product knowledge storage method and device, computer equipment and readable storage medium | |
| CN111324792A (en) | Big data platform | |
| CN112181995B (en) | Data processing method, device, equipment and storage medium of data table | |
| CN117312283A (en) | Database and table data verification method and device, computer equipment and storage medium | |
| CN114817237A (en) | Data collection method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |