CN110008291B - Data early warning method and device, storage medium and electronic equipment - Google Patents
Data early warning method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN110008291B CN110008291B CN201910285517.2A CN201910285517A CN110008291B CN 110008291 B CN110008291 B CN 110008291B CN 201910285517 A CN201910285517 A CN 201910285517A CN 110008291 B CN110008291 B CN 110008291B
- Authority
- CN
- China
- Prior art keywords
- data
- production
- early warning
- upstream
- data warehouse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- General Factory Administration (AREA)
Abstract
The embodiment of the disclosure discloses a data early warning method, a data early warning device, a storage medium and electronic equipment, wherein the method comprises the following steps: determining the production relation between an upstream data warehouse table and a corresponding downstream data warehouse table in the task in a breadth-first traversal mode; acquiring current production data in an upstream database table; and when the current production data does not meet the preset production data requirement, sending early warning information to a downstream data warehouse corresponding to the upstream data warehouse according to the production relation. This disclosed embodiment when downstream data warehouse table has not produced data output yet, just can accomplish the early warning in advance, when the production data of downstream data warehouse table output really has the problem, can be quick know which upstream data warehouse table leads to, the business side can know the problem place in time, timely analysis problem reason, user experience is better.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a data early warning method, an apparatus, a storage medium, and an electronic device.
Background
For the existing production operation data, such as daily activity, playing times and the like, a business side knows which data have problems after seeing the statistical report.
However, the scheduling time of the statistics report is delayed relative to the task execution completion time, and the business side knows that the problem is late, so that the time for analyzing the cause of the problem is later.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a data early warning method, an apparatus, a storage medium, and an electronic device, so as to solve the following problems in the prior art: the scheduling time of the statistical report is delayed relative to the task execution completion time, and the business side knows that the problem is late, so that the time for analyzing the cause of the problem is later.
On one hand, the embodiment of the disclosure provides a data early warning method, which includes: determining the production relation between an upstream data warehouse table and a corresponding downstream data warehouse table in the task in a breadth-first traversal mode; acquiring current production data in the upstream database table; and when the current production data does not meet the preset production data requirement, sending out early warning information to a downstream database table corresponding to the upstream database according to the production relation.
In some embodiments, in a case that the production data is a data volume, the sending an early warning message to the downstream database table corresponding to the upstream database according to the production relationship when the current production data does not meet a predetermined production data requirement includes: judging whether the data volume is within a preset data volume range or not; and when the data volume is not in the preset data volume range, sending an early warning message that the data volume is abnormal to a downstream database table corresponding to the upstream database according to the production relation.
In some embodiments, in a case that the production data is data output time, when the current production data does not meet a predetermined production data requirement, sending an early warning message to a downstream database table corresponding to the upstream database according to the production relationship includes: judging whether the data output time exceeds a preset output time or not; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
In some embodiments, after the determining, in a breadth-first traversal manner, a production relationship between an upstream data warehouse table and a corresponding downstream data warehouse table in a task, the method further includes: establishing marks respectively corresponding to the data warehouse tables; and constructing a production relation graph according to the production relation between the database tables and the marks.
On the other hand, the embodiment of the present disclosure provides a data early warning device, including: the determining module is configured to determine the production relation between an upstream data warehouse table and a corresponding downstream data warehouse table in the task in a breadth-first traversal mode; the acquisition module is configured to acquire current production data in the upstream database table; and the early warning module is configured to send out early warning information to the downstream database table corresponding to the upstream database according to the production relation when the current production data does not accord with the preset production data requirement.
In some embodiments, the early warning module is specifically configured to: under the condition that the production data is the data volume, judging whether the data volume is in a preset data volume range; and when the data volume is not in the preset data volume range, sending an early warning message that the data volume is abnormal to a downstream database table corresponding to the upstream database according to the production relation.
In some embodiments, the early warning module is specifically configured to: under the condition that the production data is data output time, judging whether the data output time exceeds preset output time; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
In some embodiments, further comprising: the building module is configured to build marks corresponding to the data warehouse tables respectively; and constructing a production relation graph according to the production relation between the database tables and the marks.
In another aspect, an embodiment of the present disclosure provides a storage medium storing a computer program, where the computer program is executed by a processor to implement the method provided in any embodiment of the present disclosure.
In another aspect, an embodiment of the present disclosure provides an electronic device, which at least includes a memory and a processor, where the memory stores a computer program thereon, and the processor implements the method provided in any embodiment of the present disclosure when executing the computer program on the memory.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
Drawings
Fig. 1 is a flowchart of a data early warning method according to a first embodiment of the disclosure;
fig. 2 is a flowchart of a data early warning method according to a second embodiment of the disclosure;
FIG. 3 is a production relationship diagram provided by a second embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a data early warning apparatus according to a third embodiment of the disclosure;
FIG. 5 is a production relationship diagram provided by a third embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described below clearly and completely with reference to the accompanying drawings of the embodiments of the present disclosure. It is to be understood that the described embodiments are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the disclosure without any inventive step, are within the scope of protection of the disclosure.
Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The use of "first," "second," and similar terms in this disclosure is not intended to indicate any order, quantity, or importance, but rather is used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
To maintain the following description of the embodiments of the present disclosure clear and concise, a detailed description of known functions and known components have been omitted from the present disclosure.
A first embodiment of the present disclosure provides a data early warning method, where a flow of the method is shown in fig. 1, and the method includes steps S101 to S103:
and S101, determining the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal mode.
The breadth-first traversal method is to search all nodes on a next level of a certain level after searching all nodes on the certain level.
Each task is composed of a plurality of database tables, that is, data corresponding to the database table at the bottom end of one task is finally obtained through the plurality of database tables, for example, the database table 11 is an upstream database table of the database table 21, the database table 21 and the database table 31 are upstream database tables of the database table 41, and the database table 41 is a database table at the bottom end of the task.
The method and the device for acquiring the production relation in the task acquire the production relation in the task in a breadth-first traversal mode, and during specific implementation, the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of executing the task, namely the codes of the task are acquired, and the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of analyzing the codes through a preset analysis program.
And S102, acquiring current production data in the upstream database table.
The production data may be the obtained data amount or the obtained data output time.
And S103, when the current production data does not meet the preset production data requirement, sending an early warning message to a downstream database table corresponding to the upstream database according to the production relation.
Because the production relation of the upstream data warehouse table and the corresponding downstream data warehouse table in the task is known, the downstream data warehouse table corresponding to the upstream data warehouse table can be determined according to the production relation, and then early warning is given to the produced data which is not produced by the downstream data warehouse table so as to remind a user that the downstream data warehouse table may have problems.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
A second embodiment of the present disclosure provides a data early warning method, where a flow of the method is shown in fig. 2, and the method includes steps S201 to S205:
s201, determining the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal mode.
Each task is composed of a plurality of database tables, that is, data corresponding to the database table at the bottom end of one task is finally obtained through the plurality of database tables, for example, the database table 11 is an upstream database table of the database table 21, the database table 21 and the database table 31 are upstream database tables of the database table 41, and the database table 41 is a database table at the bottom end of the task.
The method and the device for acquiring the production relation in the task acquire the production relation in the task in a breadth-first traversal mode, and during specific implementation, the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of executing the task, namely the codes of the task are acquired, and the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of analyzing the codes through a preset analysis program.
S202, establishing marks corresponding to the database tables respectively.
For example, the corresponding label of the data warehouse table is a circular pattern, and the blank in the middle of the circular pattern can fill out the table name of the data warehouse table.
And S203, constructing a production relation graph according to the production relation and the marks among the database tables.
For example, if the database table 11 is an upstream database table of the database table 21, and the database table 21, the database table 31, and the database table 41 are upstream database tables of the database table 51, the production relationship graph thereof may be as shown in fig. 3. Through the production relational graph, the relation among the database tables can be seen clearly more intuitively, and even if the task content is complex, the relation among the database tables can be known clearly. The production relation graph can be directly displayed to a user in a visual mode.
And S204, acquiring the current production data in the upstream database table.
The production data may be the obtained data amount or the obtained data output time.
And S205, when the current production data does not meet the preset production data requirement, sending out an early warning message to a downstream database table corresponding to the upstream database according to the production relation diagram.
Because the production relation of the upstream data warehouse table and the corresponding downstream data warehouse table in the task is known, the downstream data warehouse table corresponding to the upstream data warehouse table can be determined according to the production relation, and then early warning is given to the produced data which is not produced by the downstream data warehouse table so as to remind a user that the downstream data warehouse table may have problems.
In specific implementation, under the condition that the production data is the data volume, whether the data volume is in a preset data volume range can be judged; and when the data volume is not in the preset data volume range, sending out an early warning message that the data volume is abnormal to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation. Judging whether the data output time exceeds the preset output time or not under the condition that the production data is the data output time; and when the data output time exceeds the preset output time, sending an early warning message with delay in the data output time of a downstream database table corresponding to the upstream database according to the production relation. Through this process, no matter data volume or output time goes wrong, early warning all can be fine.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
A third embodiment of the present disclosure provides a data early warning device, a structural schematic of the device is shown in fig. 4, and the device includes:
the determining module 10 is configured to determine a production relationship between an upstream data warehouse table and a corresponding downstream data warehouse table in a task in a breadth-first traversal manner; the acquisition module 20 is coupled with the determination module 10 and is used for acquiring current production data in an upstream database table; and the early warning module 30 is coupled to the obtaining module 20 and configured to send an early warning message to a downstream database table corresponding to the upstream database according to the production relationship when the current production data does not meet the predetermined production data requirement.
Each task is composed of a plurality of database tables, that is, data corresponding to the database table at the bottom end of one task is finally obtained through the plurality of database tables, for example, the database table 11 is an upstream database table of the database table 21, the database table 21 and the database table 31 are upstream database tables of the database table 41, and the database table 41 is a database table at the bottom end of the task.
The method and the device for acquiring the production relation in the task acquire the production relation in the task in a breadth-first traversal mode, and during specific implementation, the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of executing the task, namely the codes of the task are acquired, and the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of analyzing the codes through a preset analysis program.
The production data may be the obtained data amount or the obtained data output time.
Correspondingly, the early warning module may be specifically configured to: judging whether the data volume exceeds a preset data volume range or not under the condition that the production data is the data volume; and when the data volume is not in the preset data volume range, sending out an early warning message that the data volume is abnormal to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation. The early warning module may be further configured to: judging whether the data output time exceeds the preset output time or not under the condition that the production data is the data output time; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream database table corresponding to the upstream database according to the production relation.
Because the production relation of the upstream data warehouse table and the corresponding downstream data warehouse table in the task is known, the downstream data warehouse table corresponding to the upstream data warehouse table can be determined according to the production relation, and then early warning is given to the produced data which is not produced by the downstream data warehouse table so as to remind a user that the downstream data warehouse table may have problems.
The above apparatus may further include: the construction module is coupled with the determination module and the acquisition module and is used for establishing marks respectively corresponding to the data warehouse tables; and constructing a production relation graph according to the production relation and the marks.
For example, the corresponding label of the data warehouse table is a square pattern, and the blank in the middle of the square pattern can fill out the table name of the data warehouse table. The database table 11 and the database table 12 are upstream database tables of the database table 21, and the database table 21, the database table 31 and the database table 41 are upstream database tables of the database table 51, so that the production relationship diagram thereof can be as shown in fig. 5, in the production relationship diagram, no arrow indicates the direction of the production relationship, but an up-down relationship is defaulted, and the square length of the downstream database table is at least equal to the lengths of all the upstream database tables. Through the production relational graph, the relation among the database tables can be seen clearly more intuitively, and even if the task content is complex, the relation among the database tables can be known clearly. The production relation graph can be directly displayed to a user in a visual mode.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
A fourth embodiment of the present disclosure provides a storage medium storing a computer program, which when executed by a processor implements the method provided in any of the embodiments of the present disclosure, including the following steps S1 to S3:
s1, determining the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal mode;
s2, acquiring current production data in an upstream database table;
and S3, when the current production data do not meet the preset production data requirement, sending out an early warning message to a downstream database table corresponding to the upstream database according to the production relation.
When the computer program is executed by the processor and the current production data does not meet the preset production data requirement, and the step of sending the early warning message to the downstream data warehouse table corresponding to the upstream data warehouse according to the production relation is executed by the processor, the following steps are specifically executed by the processor: judging whether the data volume is in a preset data volume range or not under the condition that the production data is the data volume; and when the data volume is not in the preset data volume range, sending out an early warning message that the data volume is abnormal to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
When the computer program is executed by the processor and the current production data does not meet the preset production data requirement, and the step of sending the early warning message to the downstream data warehouse table corresponding to the upstream data warehouse according to the production relation is executed by the processor, the following steps are specifically executed by the processor: judging whether the data output time exceeds the preset output time or not under the condition that the production data is the data output time; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream database table corresponding to the upstream database according to the production relation.
After the computer program is executed by the processor to determine the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal mode, the computer program can also be executed by the processor to execute the following steps: establishing marks respectively corresponding to the data warehouse tables; and constructing a production relation graph according to the production relation and the marks among the database tables.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-only memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes. Optionally, in this embodiment, the processor executes the method steps described in the above embodiments according to the program code stored in the storage medium. Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again. It will be apparent to those skilled in the art that the modules or steps of the present disclosure described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. As such, the present disclosure is not limited to any specific combination of hardware and software.
A fifth embodiment of the present disclosure provides an electronic device, as shown in fig. 6, the electronic device at least includes a memory 901 and a processor 902, the memory 901 stores a computer program thereon, the processor 902 implements the method provided in any embodiment of the present disclosure when executing the computer program on the memory 901, for example, the computer program has the following steps S11 to S13:
s11, determining the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal mode;
s12, acquiring current production data in an upstream database table;
and S13, when the current production data do not meet the preset production data requirement, sending out an early warning message to a downstream database table corresponding to the upstream database according to the production relation.
When the processor 902 executes the computer program, which is stored in the memory 901 and used for sending an early warning message to the downstream database table corresponding to the upstream database according to the production relationship when the current production data does not meet the predetermined production data requirement, the following computer program may be specifically executed: judging whether the data volume is in a preset data volume range or not under the condition that the production data is the data volume; and when the data volume is not in the preset data volume range, sending out an early warning message that the data volume is abnormal to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
When the processor 902 executes the computer program, which is stored in the memory 901 and used for sending an early warning message to the downstream database table corresponding to the upstream database according to the production relationship when the current production data does not meet the predetermined production data requirement, the following computer program may be specifically executed: judging whether the data output time exceeds the preset output time or not under the condition that the production data is the data output time; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
The processor 902, after executing the computer program stored on the memory 901 for determining the production relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in the task in a breadth-first traversal manner, may further execute the following computer program: establishing marks respectively corresponding to the data warehouse tables; and constructing a production relation graph according to the production relation and the marks among the database tables.
The embodiment of the disclosure determines the relationship between the upstream data warehouse table and the corresponding downstream data warehouse table in a task, when the current production data in the upstream data warehouse table is not in accordance with expectations, the determined production relationship is used for sending out an early warning to the downstream data warehouse table corresponding to the upstream data warehouse table, and then when the downstream data warehouse table does not produce production data, the early warning can be achieved in advance.
Moreover, although exemplary embodiments have been described herein, the scope thereof includes any and all embodiments based on the disclosure with equivalent elements, modifications, omissions, combinations (e.g., of various embodiments across), adaptations or alterations. The elements of the claims are to be interpreted broadly based on the language employed in the claims and not limited to examples described in the present specification or during the prosecution of the application, which examples are to be construed as non-exclusive. It is intended, therefore, that the specification and examples be considered as exemplary only, with a true scope and spirit being indicated by the following claims and their full scope of equivalents.
The above description is intended to be illustrative and not restrictive. For example, the above-described examples (or one or more versions thereof) may be used in combination with each other. For example, other embodiments may be used by those of ordinary skill in the art upon reading the above description. In addition, in the foregoing detailed description, various features may be grouped together to streamline the disclosure. This should not be interpreted as an intention that a disclosed feature not claimed is essential to any claim. Rather, the subject matter of the present disclosure may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description as examples or embodiments, with each claim standing on its own as a separate embodiment, and it is contemplated that these embodiments may be combined with each other in various combinations or permutations. The scope of the disclosure should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
While the present disclosure has been described in detail with reference to the embodiments, the present disclosure is not limited to the specific embodiments, and those skilled in the art can make various modifications and alterations based on the concept of the present disclosure, and the modifications and alterations should fall within the scope of the present disclosure as claimed.
Claims (10)
1. A data early warning method is characterized by comprising the following steps:
determining a production relation between an upstream data warehouse table and a corresponding downstream data warehouse table in a task in a breadth-first traversal mode, wherein the production relation comprises acquiring a code of the task, determining the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table in the process of analyzing the code through a preset analysis program, and the task is composed of a plurality of data warehouse tables;
acquiring current production data in the upstream database table;
when the current production data do not meet the preset production data requirement, sending an early warning message to the downstream database table corresponding to the upstream database according to the production relation, wherein the early warning message comprises judging whether the data volume is in a preset data volume range or judging whether the data output time exceeds preset output time under the condition that the production data are data volume.
2. The method of claim 1, wherein, in the case that the production data is a data volume, the issuing an early warning message to the downstream database table corresponding to the upstream data warehouse according to the production relationship when the current production data does not meet a predetermined production data requirement comprises:
and when the data volume is not in the preset data volume range, sending an early warning message that the data volume is abnormal to a downstream database table corresponding to the upstream database according to the production relation.
3. The method of claim 1, wherein, in the case that the production data is data output time, the issuing an early warning message to a downstream database table corresponding to the upstream database according to the production relationship when the current production data does not meet a predetermined production data requirement comprises:
and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
4. The method of any one of claims 1 to 3, wherein after said determining a production relationship between an upstream data warehouse table and a corresponding downstream data warehouse table in a task by way of breadth-first traversal, further comprising:
establishing marks respectively corresponding to the data warehouse tables;
and constructing a production relation graph according to the production relation between the database tables and the marks.
5. A data early warning device, characterized by comprising:
the system comprises a determining module, a processing module and a processing module, wherein the determining module is configured to determine a production relation between an upstream data warehouse table and a corresponding downstream data warehouse table in a task in a breadth-first traversal mode, the production relation comprises a code of the task, the production relation between the upstream data warehouse table and the corresponding downstream data warehouse table is determined in the process of analyzing the code through a preset analysis program, and the task is composed of a plurality of data warehouse tables;
the acquisition module is configured to acquire current production data in the upstream database table;
the early warning module is configured to send an early warning message to the downstream database table corresponding to the upstream database according to the production relation when the current production data does not meet a preset production data requirement, and the early warning message includes judging whether the data volume is in a preset data volume range or whether the data output time exceeds preset output time when the production data is data output time under the condition that the production data is data volume.
6. The apparatus of claim 5,
the early warning module is specifically configured to: under the condition that the production data is the data volume, judging whether the data volume is in a preset data volume range; and when the data volume is not in the preset data volume range, sending an early warning message that the data volume is abnormal to a downstream database table corresponding to the upstream database according to the production relation.
7. The apparatus of claim 5,
the early warning module is specifically configured to: under the condition that the production data is data output time, judging whether the data output time exceeds preset output time; and when the data output time exceeds the preset output time, sending an early warning message that the data output time is delayed to a downstream data warehouse table corresponding to the upstream data warehouse according to the production relation.
8. The apparatus of any of claims 5 to 7, further comprising:
the building module is configured to build marks corresponding to the data warehouse tables respectively; and constructing a production relation graph according to the production relation between the database tables and the marks.
9. A storage medium storing a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 4 when executed by a processor.
10. An electronic device comprising at least a memory, a processor, the memory having a computer program stored thereon, wherein the processor, when executing the computer program on the memory, is adapted to carry out the steps of the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285517.2A CN110008291B (en) | 2019-04-10 | 2019-04-10 | Data early warning method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285517.2A CN110008291B (en) | 2019-04-10 | 2019-04-10 | Data early warning method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008291A CN110008291A (en) | 2019-07-12 |
| CN110008291B true CN110008291B (en) | 2022-03-11 |
Family
ID=67170832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910285517.2A Active CN110008291B (en) | 2019-04-10 | 2019-04-10 | Data early warning method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008291B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489261A (en) * | 2019-07-31 | 2019-11-22 | 上海艾融软件股份有限公司 | Task handles alarm method, device and electronic equipment, storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034554A (en) * | 2012-12-30 | 2013-04-10 | 焦点科技股份有限公司 | ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting |
| CN103902653A (en) * | 2014-02-28 | 2014-07-02 | 珠海多玩信息技术有限公司 | Method and device for creating data warehouse table blood relationship graph |
| CN103927378B (en) * | 2014-04-29 | 2017-04-12 | 智业软件股份有限公司 | Automatic positioning and early-warning method of problem data of report output results |
| CN106933857A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | The dispatching method of task, device in a kind of data warehouse |
| CN107025224A (en) * | 2016-01-29 | 2017-08-08 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of monitor task operation |
| CN107665215A (en) * | 2016-07-29 | 2018-02-06 | 银联数据服务有限公司 | A kind of bank return generation method and device |
| CN109254986A (en) * | 2018-08-31 | 2019-01-22 | 阿里巴巴集团控股有限公司 | A kind of determination method and device of abnormal data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150378574A1 (en) * | 2014-06-25 | 2015-12-31 | Allied Telesis Holdings Kabushiki Kaisha | Graphical user interface of a sensor based detection system |
| CN109582660B (en) * | 2018-12-06 | 2021-08-10 | 深圳前海微众银行股份有限公司 | Data blood margin analysis method, device, equipment, system and readable storage medium |
-
2019
- 2019-04-10 CN CN201910285517.2A patent/CN110008291B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034554A (en) * | 2012-12-30 | 2013-04-10 | 焦点科技股份有限公司 | ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting |
| CN103902653A (en) * | 2014-02-28 | 2014-07-02 | 珠海多玩信息技术有限公司 | Method and device for creating data warehouse table blood relationship graph |
| CN103927378B (en) * | 2014-04-29 | 2017-04-12 | 智业软件股份有限公司 | Automatic positioning and early-warning method of problem data of report output results |
| CN106933857A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | The dispatching method of task, device in a kind of data warehouse |
| CN107025224A (en) * | 2016-01-29 | 2017-08-08 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of monitor task operation |
| CN107665215A (en) * | 2016-07-29 | 2018-02-06 | 银联数据服务有限公司 | A kind of bank return generation method and device |
| CN109254986A (en) * | 2018-08-31 | 2019-01-22 | 阿里巴巴集团控股有限公司 | A kind of determination method and device of abnormal data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008291A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108536601B (en) | Evaluation method, device, server and storage medium | |
| CN112311617A (en) | Configured data monitoring and alarming method and system | |
| CN108833584B (en) | Message pushing method, terminal, server and computer storage medium | |
| CN104869048A (en) | Grouping processing method, device and system of MicroBlog data | |
| CN107784068A (en) | Acquisition methods, device, storage medium, processor and the service end of data variation | |
| CN110362563A (en) | The processing method and processing device of tables of data, storage medium, electronic device | |
| CN114757639A (en) | Data processing method, device, equipment and storage medium | |
| CN110008291B (en) | Data early warning method and device, storage medium and electronic equipment | |
| US20180293089A1 (en) | Method and device for adding indicative icon in interactive application | |
| CN109885467B (en) | Data fluctuation alarming method and device, storage medium and electronic equipment | |
| CN113722141B (en) | Method and device for determining delay reason of data task, electronic equipment and medium | |
| CN109857818B (en) | Method and device for determining production relation, storage medium and electronic equipment | |
| CN109299765B (en) | Anti-collision method and device for radio frequency identification equipment of multiple internet of things and storage medium | |
| CN112965992B (en) | Multi-parameter constraint data retrieval man-machine interaction method and device | |
| CN109683994B (en) | Method and device for determining view construction time, storage medium and electronic equipment | |
| CN110309211B (en) | Method for positioning ETL process problem and related equipment | |
| CN111178044B (en) | Method and device for determining target text elements | |
| CN107704533B (en) | User terminal online quantity statistical method and system | |
| CN110532758A (en) | A kind of Risk Identification Method and device for group | |
| CN110866037A (en) | Message filtering method and device | |
| CN111127065A (en) | Method and device for acquiring user occupation place | |
| CN116433197B (en) | Information reporting method, device, reporting end and storage medium | |
| CN112819358B (en) | Data processing method, device and equipment | |
| CN109213848B (en) | Method and device for determining related search terms, storage medium and electronic equipment | |
| CN110909289B (en) | Data processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |