CN102279886A - Method and equipment for processing metadata - Google Patents
Method and equipment for processing metadata Download PDFInfo
- Publication number
- CN102279886A CN102279886A CN2011102351367A CN201110235136A CN102279886A CN 102279886 A CN102279886 A CN 102279886A CN 2011102351367 A CN2011102351367 A CN 2011102351367A CN 201110235136 A CN201110235136 A CN 201110235136A CN 102279886 A CN102279886 A CN 102279886A
- Authority
- CN
- China
- Prior art keywords
- metadata
- data
- order
- ddl
- treatment facility
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and equipment for processing metadata. The method comprises the following steps that: the equipment for processing the metadata acquires an extraction-transformation-loading (ETL) execution log and acquires a first data definition language (DDL) command, wherein the first DDL command is used for performing node definition on a data object included by the ETL execution log; the equipment for processing the metadata analyzes the acquired ETL execution log to acquire a second DDL command and a data manipulation language (DML) command, wherein the second DDL command is used for performing node definition on a temporary object included by the ETL execution log, and the DML command is used for recording the conversion operation of the data object and/or the temporary object in an ETL execution process; and the equipment for processing the metadata acquires the metadata which is used for representing the data flow direction of the data object and/or the temporary object by analyzing the first DDL command, the second DDL command and the DML command. By the method and the equipment for processing the metadata, the metadata which reflects the real data flow direction can be acquired.
Description
Technical field
The present invention relates to the metadata treatment technology, relate in particular to a kind of metadata disposal route and equipment.
Background technology
Along with the fast development of office automation, information has become the valuable source of modern enterprise, is the basis of enterprise's utilization scientific management, decision analysis, is used to realize that the data warehouse of independent data management has also obtained using very widely.
Data pick-up, conversion and loading (Extraction-Transformation-Loading, ETL) be the important step that makes up data warehouse, it is responsible for data that distribute, in the heterogeneous data source, as relation data and flat data file etc., clean, change and load after being drawn into interim middle layer, and be incorporated in the data warehouse, become the basis of on-line analytical processing and data mining.The ETL process adopts metadata approach to manage concentratedly.Wherein, metadata is the data (data about data) of data of description, and it concentrates on for ETL's: the position in definition of data source and the attribute of data source, the definite rule of correspondence, definite relevant service logic and other necessary preliminary works before the data actual loaded etc. from the source data to the target data.
In the prior art, when realizing data management based on data warehouse, the data flow in the time of need knowing the structure data warehouse by the metadata that before carrying out ETL, sets, thus set up data association, avoid producing data silo or information blind area.But owing in the ETL of reality process, may there are differences owing to some reasons make real data flow to the preset data flow direction.At this moment, if still carry out data management, just can't accurately know the flow direction relation between the data, thereby can't carry out reliably the data warehouse, effectively manage based on predefined metadata.
Summary of the invention
At above-mentioned defective, the invention provides a kind of metadata disposal route and equipment, in order to realize obtaining the metadata that can reflect that True Data flows to.
According to an aspect of the present invention, provide a kind of metadata disposal route, comprising:
The metadata treatment facility receives the metadata processing instruction that the user sends, respond described metadata processing instruction and obtain data pick-up, conversion and loading ETL execution journal, and obtain the first database definition language DDL from data warehouse and order, wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition;
Described metadata treatment facility is resolved the ETL execution journal of being obtained, to obtain the 2nd DDL order and data manipulation language (DML) DML order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
Described metadata treatment facility obtains the metadata of the data flow that is used to represent described data object and described temporary object by resolving described DDL order, described the 2nd DDL order and described DML order, and to the described metadata of described user feedback.
According to a further aspect in the invention, also provide a kind of metadata treatment facility, it is characterized in that, comprising:
Acquisition module is used to obtain the ETL execution journal, and obtains DDL order from data warehouse, and wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition;
Log analyzing module, be used for the ETL execution journal of being obtained is resolved, to obtain the 2nd DDL order and data manipulation language (DML) DML order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
The metadata acquisition module is used for obtaining the metadata of the data flow that is used to represent described data object and described temporary object by resolving described DDL order, described the 2nd DDL order and described DML order.
According to metadata disposal route of the present invention and equipment, owing to be that DDL resolves and DML resolves the flow direction relation of obtaining between the object by the ETL execution journal of true operation generation is carried out, so the metadata of the relation of the flow direction between the sign object that is obtained is consistent with true implementation status, can reflect efficiently and accurately that the True Data in the ETL implementation flows to, thereby can carry out reliable data dependence analysis and system function optimization etc. based on this metadata.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of metadata disposal route of the present invention.
Fig. 2 is for using the data warehouse Organization Chart of metadata disposal route of the present invention.
Fig. 3 is the structural representation of metadata treatment facility of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that is obtained under the creative work prerequisite.
Fig. 1 is the schematic flow sheet of metadata disposal route of the present invention.As shown in Figure 1, this metadata disposal route comprises:
Step S101, the metadata treatment facility obtains the ETL execution journal, and (wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition for Data Definition Language, DDL) order to obtain first database definition language from data warehouse;
Step S102, described metadata treatment facility is resolved the ETL execution journal of being obtained, to obtain the 2nd DDL order and data manipulation language (DML) (Data Manipulation Language, DML) order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
Step S103, described metadata treatment facility obtains the metadata of the data flow that is used to represent described data object and described temporary object by resolving described DDL order, described the 2nd DDL order and described DML order.
Fig. 2 is for using the data warehouse Organization Chart of metadata disposal route of the present invention.As shown in Figure 2, in data warehouse, comprise a plurality of data sources, a plurality of ETL equipment (being that example describes only among Fig. 2) and data warehouse with 2, wherein ETL equipment is connected between data source and the data warehouse, is used for the data of data source are extracted, change and be loaded into data warehouse.
Wherein, data warehouse is a kind of data organization mode of subject-oriented, it can the data to analytic target provide complete, consistent description on higher level, every data of the enterprise that each analytic target of portrayal that can be complete, unified is related and the contact between the data, thereby adapt to the business activity characteristics of each department of enterprise and the behavioral characteristics of business data, fundamentally realize separating of data and application.The extraction process of ETL equipment is meant data and promptly reads out in the data source from various original operation systems; Transfer process is meant according to pre-designed rule the data that extract is changed, cleaned, and the data of handling some redundancies, ambiguity, the data layout of original isomery can be united, and transformation rule and function for example comprise the symbol of selecting specific field, translation coding, symbol, calculated value and the merging source etc. that coding is new; Loading procedure is meant that the data importing that will convert is in data warehouse.
In data warehouse shown in Figure 2, also comprise the metadata treatment facility that is connected with each ETL equipment respectively.Above-mentioned steps S101～S103 is carried out by this metadata treatment facility.
Particularly, the metadata treatment facility for example receives the metadata processing instruction of user's input by self-contained human-computer interaction interface, or receives the metadata processing instruction that the user sends by the user terminal (not shown) that is arranged in the data warehouse, communicate with this metadata treatment facility.The metadata treatment facility responds this metadata processing instruction, obtains the ETL daily record from 1ETL equipment and 2ETL equipment respectively.Wherein, the ETL daily record is divided three classes: the first kind is an execution journal, is the record in one step of every execution in the ETL implementation, with the zero-time of day-to-day account form record each run, each step, influenced how many line data and carried out content; Second class is an error log, needs the write error daily record when certain module is made mistakes, the information that is used to write down the time of at every turn makeing mistakes, the module of makeing mistakes and makes mistakes; Whether successful the 3rd class is overall daily record, only write down ETL start time, concluding time information.
The metadata treatment facility is ordered by DDL whole objects related in the ETL execution journal and is carried out node definition.Wherein, whole objects related in the ETL execution journal comprise data object and temporary object, wherein data object is the data object that defines in the data warehouse, comprise former data object and target data object, temporary object is at the intermediate virtual object of being set up to the transfer process of target data object by former data object, for example variable temporary table for performance optimization.Particularly, when related whole objects carry out the DDL node definition in to the ETL execution journal, need to obtain the node definition of data object from data warehouse, and the DDL that parses from the log file of ETL execution journal in the ETL implementation orders, the i.e. node definition that temporary object carried out to being created, for example defined node structure etc.To order from the DDL that data warehouse obtains and resolve the 2nd DDL order merging that obtains and be stored as a ddl file by log.For example comprise the DDL order of from data warehouse, reading " create table T1 (F1x32; F2x10) " in this ddl file and from the log file, parse DDL order " create table t2 (f1x32; f2x10) ", wherein, definition of data Object table T1 comprises data object F1 and F2, and definition temporary object table t2 comprises temporary object f1 and f2.
In addition, also need in the process that the log file of ETL execution journal is resolved, to extract the DML order that is comprised.Wherein, the DML order is the process that forms data stream.More specifically, for example extract the DML order in the following ways from the log file: the metadata treatment facility word for word reads the log file, after having read a complete statement, comprise the key word of DML order (for example being " INSERT " or " UPDATE ") and meet the statement condition for validity if know this statement, know that then this statement is that DML orders and it is stored to separately in the DML file.Wherein, owing to also comprise non-exectorial multiple interfere information in the ETL execution journal, the notes content that for example starts and end up etc. with " */" with "/* ", so whether need to be used to distinguish this statement to the input of metadata treatment facility in advance is the condition for validity of interfere information, for example this statement is not to start with "/* ".In this way, for example find DML order " INSERT INTO T1 SELETE*from t2 ".
After obtaining ddl file and DML file, DML resolves and DLL resolves the data flow of obtaining data object and temporary object related in the ETL execution journal by carrying out.Particularly, the metadata treatment facility is at first handled each the bar DML order in the DML file, for example according to the operations such as data computation in the semanteme table removal DML order of presetting, thereby obtains object and relation between objects.Wherein, store a large amount of key words in the semantic table, for example comprise key word MAX, then when in the DML order, reading key word MAX, it is ignored, do not consider.Afterwards,, for example be table according to related object in the DML order, and the definition of from ddl file, searching this table, obtain the included concrete object of table, thereby set up the data flow of data object and temporary object.For example at DML order " INSERT INTO T1S ELETE*from t2 " (promptly the object from t2 is inserted among the T1), the definition of the definition of look-up table T1 and table t2 from ddl file, know that data object T1 comprises concrete object F1 and F2, temporary object t2 comprises concrete object f1 and f2, and mate according to the order that concrete object defines in data object T1 and temporary object t2, then obtain data flow metadata " T1.F1＜-t2.f1 " and " T1.F2＜-t2.f2 ".The metadata treatment facility feeds back to the user terminal of transmission metadata processing instruction with the metadata that is obtained or shows to the user by human-computer interaction interface.
Metadata disposal route according to the foregoing description, owing to be that DDL resolves and DML resolves the flow direction relation of obtaining between the object by the ETL execution journal of true operation generation is carried out, so the metadata of the relation of the flow direction between the sign object that is obtained is consistent with true implementation status, can reflect efficiently and accurately that the True Data in the ETL implementation flows to, thereby can carry out reliable data dependence analysis and system function optimization etc. based on this metadata.
Further, in the metadata disposal route of the foregoing description, after the step of the metadata of obtaining the data flow that is used to represent described data object and described temporary object, also comprise:
Temporary object in the metadata of the data flow of described metadata treatment facility shielding described data object of expression and described temporary object is to obtain the metadata of the data flow that is used to represent described data object.
Particularly because in the ETL implementation, the naming rule of the temporary object of being created for " the script name. table. temporary object ", for example be " pl_name.t2.f1 " that wherein pl_name is the script name; And in data warehouse, the naming rule of defined data object be " library name. table/view. data object ", for example be " Db_name.T1.F1 " that wherein Db_name is a library name.Therefore, the metadata treatment facility can be distinguished data object and temporary object by the title of identifying object.
Next, the metadata of object relationship that is used to characterize in twos that the metadata disposal route by the foregoing description is obtained is put in order.Particularly, the metadata treatment facility detect each in twos the destination object in the relation whether be temporary object, for example object A is former object in relational expression " object A → object B " first-class, object B is a destination object, this moment, destination object was a temporary object; Then continuing to search with this temporary object is the relation in twos of former object, flows to relational expression " object B → object C " thereby find second; Detect whether second destination object that flows in the relational expression is temporary object this moment once more, carries out that this to search until the final objective object be not temporary object.For example C is a data object, then this moment the first-class temporary object B that flows in the relational expression to relational expression and second is shielded, thereby the data flow that obtains " object A → object C " concerns.
More specifically, for example, by utilizing the metadata disposal route of the foregoing description, data flow " T1.F1＜-t2.f1 ", " T1.F2＜-t2.f2 ", " t2.f1＜-V1.f1 " and " t2.f2＜-V1.f2 " have been obtained, wherein t2 is a temporary object, T1 and V1 are data object, then by shielding temporary object t2, can directly obtain flow direction relation " T1.F1＜-V1.f1 " and " T1.F2＜-V1.f2 " between the data object.
Metadata disposal route according to the foregoing description, because temporary object only is at the intermediate virtual object of being set up to the transfer process of target data object by former data object for performance optimization, do not possess essential meaning for the data management in the data warehouse in later stage, so the flow direction relation of directly setting up between the data object helps carrying out the data warehouse management that the later use metadata is carried out.
Further, in the metadata disposal route of the foregoing description, described data object comprises table and view; Correspondingly, after the step of the metadata of obtaining the data flow that is used to represent described data object, also comprise:
View in the metadata of the data flow of the described data object of expression that the shielding of described metadata treatment facility is generated is used for the metadata of the data flow of expression table with generation.
Wherein, the query-defined Virtual table that view is made up of the SELECT statement, it is made up of the data in one or more table, and it is the same with real table, comprises a series of row and line data that have title.But view does not exist with the data set form of storage in database.The table that number of lines and columns are quoted according to the inquiry that freely defines view, and when quoting view, dynamically generate.
For example, flow direction relation between the data object " T1.F1＜-V1.f1 ", " T1.F2＜-V1.f2 ", " V1.f1＜-T3.f1 " and " V1.f2＜-T3.f2 " have been obtained, wherein T1 and T3 are table, V1 is a view, then by shielding view V1, get access to flow direction relation between table T1 and the table T3 " T1.F1＜-T3.f1 " and " T1.F2＜-T3.f2 ".
According to the metadata disposal route of the foregoing description, can directly obtain to be stored in table in the data warehouse and the flow direction relation between the table.
Further, in the metadata disposal route of the foregoing description, described metadata treatment facility comprises service end and browser end; Correspondingly, described metadata disposal route also comprises: the service end of described metadata treatment facility represents that with being used to of being generated the metadata of data flow is sent to described browser end and shows.
Fig. 3 is the structural representation of metadata treatment facility of the present invention.As shown in Figure 3, this metadata treatment facility comprises:
Acquisition module 11, be used to receive the metadata processing instruction that the user sends, respond described metadata processing instruction and obtain the ETL execution journal, and obtain DDL order from data warehouse, wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition;
Log analyzing module 12, be used for the ETL execution journal of being obtained is resolved, to obtain the 2nd DDL order and data manipulation language (DML) DML order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
The metadata treatment facility of the foregoing description for example adopts the J2EE platform technology to realize, and the running environment of dependence Java Sdk more than 1.6, its flow process of obtaining the metadata that is used to represent data flow is identical with the metadata disposal route of the foregoing description, so locate to repeat no more.
Metadata treatment facility according to the foregoing description, owing to be that DDL resolves and DML resolves the flow direction relation of obtaining between the object by the ETL execution journal of true operation generation is carried out, so the metadata of the relation of the flow direction between the sign object that is obtained is consistent with true implementation status, can reflect efficiently and accurately that the True Data in the ETL implementation flows to, thereby can carry out reliable data dependence analysis and system function optimization etc. based on this metadata.
Further, in the metadata treatment facility of the foregoing description, described metadata acquisition module also is used for shielding the temporary object of described data flow, to obtain the metadata of the data flow that is used to represent described data object.
Metadata treatment facility according to the foregoing description, because temporary object only is at the intermediate virtual object of being set up to the transfer process of target data object by former data object for performance optimization, do not possess essential meaning for the data management in the data warehouse in later stage, so the flow direction relation of directly setting up between the data object helps carrying out the data warehouse management that the later use metadata is carried out.
Further, in the metadata treatment facility of the foregoing description, described metadata acquisition module also is used for shielding the view of metadata of the data flow of the described data object of expression that is generated, and is used for the metadata of the data flow of expression table with generation.
According to the metadata treatment facility of the foregoing description, can directly obtain to be stored in table in the data warehouse and the flow direction relation between the table.
Further, in the metadata treatment facility of the foregoing description, comprise service end and browser end; Wherein, described acquisition module, described log analyzing module and described metadata acquisition module are arranged on described service end, and described metadata acquisition module also is used for being used to of being generated represented that the metadata of data flow is sent to described browser end and shows.
According to the metadata treatment facility of the foregoing description, can be simultaneously at the provide convenience friendly interface of the B/S pattern used of domestic consumer or business personnel.In addition, this metadata treatment facility also can adopt the current design of the relational database support of standard to store the metadata of being obtained, and with support several data storehouse, and can realize carrying out metadata query easily by the technician.
It should be noted that at last: above embodiment only in order to technical scheme of the present invention to be described, is not intended to limit; Although with reference to previous embodiment the present invention is had been described in detail, those of ordinary skill in the art is to be understood that: it still can be made amendment to the technical scheme that aforementioned each embodiment put down in writing, and perhaps part technical characterictic wherein is equal to replacement; And these modifications or replacement do not make the essence of appropriate technical solution break away from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (8)
1. a metadata disposal route is characterized in that, comprising:
The metadata treatment facility receives the metadata processing instruction that the user sends, respond described metadata processing instruction and obtain data pick-up, conversion and loading ETL execution journal, and obtain the first database definition language DDL from data warehouse and order, wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition;
Described metadata treatment facility is resolved the ETL execution journal of being obtained, to obtain the 2nd DDL order and data manipulation language (DML) DML order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
Described metadata treatment facility obtains the metadata of the data flow that is used to represent described data object and described temporary object by resolving described DDL order, described the 2nd DDL order and described DML order, and to the described metadata of described user feedback.
2. metadata disposal route according to claim 1 is characterized in that, also comprises after the step of the metadata of obtaining the data flow that is used to represent described data object and described temporary object:
Temporary object in the metadata of the data flow of described metadata treatment facility shielding described data object of expression and described temporary object is to obtain the metadata of the data flow that is used to represent described data object.
3. metadata disposal route according to claim 2 is characterized in that, described data object comprises table and view; Correspondingly, after the step of the metadata of obtaining the data flow that is used to represent described data object, also comprise:
View in the metadata of the data flow of the described data object of expression that the shielding of described metadata treatment facility is generated is used for the metadata of the data flow of expression table with generation.
4. according to the arbitrary described metadata disposal route of claim 1 to 3, it is characterized in that described metadata treatment facility comprises service end and browser end; Correspondingly, described metadata disposal route also comprises: the service end of described metadata treatment facility represents that with being used to of being generated the metadata of data flow is sent to described browser end and shows.
5. a metadata treatment facility is characterized in that, comprising:
Acquisition module is used to obtain the ETL execution journal, and obtains DDL order from data warehouse, and wherein said DDL order is used for the data object that described ETL execution journal comprises is carried out node definition;
Log analyzing module, be used for the ETL execution journal of being obtained is resolved, to obtain the 2nd DDL order and data manipulation language (DML) DML order, wherein said the 2nd DDL order is used for the temporary object that described ETL execution journal comprises is carried out node definition, and described DML order is used to be recorded in the conversion operations of data object described in the ETL implementation and/or described temporary object;
The metadata acquisition module is used for obtaining the metadata of the data flow that is used to represent described data object and described temporary object by resolving described DDL order, described the 2nd DDL order and described DML order.
6. metadata treatment facility according to claim 5 is characterized in that described metadata acquisition module also is used for shielding the temporary object of described data flow, to obtain the metadata of the data flow that is used to represent described data object.
7. metadata treatment facility according to claim 6, it is characterized in that, described metadata acquisition module also is used for shielding the view of metadata of the data flow of the described data object of expression that is generated, and is used for the metadata of the data flow of expression table with generation.
8. according to the arbitrary described metadata treatment facility of claim 5 to 7, it is characterized in that, comprise service end and browser end; Wherein, described acquisition module, described log analyzing module and described metadata acquisition module are arranged on described service end, and described metadata acquisition module also is used for being used to of being generated represented that the metadata of data flow is sent to described browser end and shows.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110235136A CN102279886B (en) | 2011-08-16 | 2011-08-16 | Method and equipment for processing metadata |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110235136A CN102279886B (en) | 2011-08-16 | 2011-08-16 | Method and equipment for processing metadata |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102279886A true CN102279886A (en) | 2011-12-14 |
| CN102279886B CN102279886B (en) | 2012-10-17 |
Family
ID=45105328
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110235136A Active CN102279886B (en) | 2011-08-16 | 2011-08-16 | Method and equipment for processing metadata |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102279886B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577586A (en) * | 2013-11-08 | 2014-02-12 | 北京国双科技有限公司 | Method and device for processing log records |
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN105447014A (en) * | 2014-08-15 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Metadata management method based on binglog, and method and device used for providing metadata |
| CN108427699A (en) * | 2017-09-22 | 2018-08-21 | 平安科技（深圳）有限公司 | The method, apparatus and storage medium of fast initialization system database |
| CN112567347A (en) * | 2018-09-06 | 2021-03-26 | 欧姆龙株式会社 | Data processing device, data processing method, and data processing program |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101069200A (en) * | 2004-12-16 | 2007-11-07 | 国际商业机器公司 | Journaling to capture workflow and convert to workflow markup language |
| CN101105793A (en) * | 2006-07-11 | 2008-01-16 | 阿里巴巴公司 | Data processing method and system of data library |
| CN101305365A (en) * | 2004-12-17 | 2008-11-12 | 亚马逊科技公司 | Apparatus and method for data warehousing |
| US7512614B2 (en) * | 2003-06-12 | 2009-03-31 | International Business Machines Corporation | System and method for data ETL in a data warehouse environment |
| CN101477543A (en) * | 2008-01-03 | 2009-07-08 | 埃森哲环球服务有限公司 | System and method for automating ETL application |
| US20100280990A1 (en) * | 2009-04-30 | 2010-11-04 | Castellanos Maria G | Etl for process data warehouse |
-
2011
- 2011-08-16 CN CN201110235136A patent/CN102279886B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7512614B2 (en) * | 2003-06-12 | 2009-03-31 | International Business Machines Corporation | System and method for data ETL in a data warehouse environment |
| CN101069200A (en) * | 2004-12-16 | 2007-11-07 | 国际商业机器公司 | Journaling to capture workflow and convert to workflow markup language |
| CN101305365A (en) * | 2004-12-17 | 2008-11-12 | 亚马逊科技公司 | Apparatus and method for data warehousing |
| CN101105793A (en) * | 2006-07-11 | 2008-01-16 | 阿里巴巴公司 | Data processing method and system of data library |
| CN101477543A (en) * | 2008-01-03 | 2009-07-08 | 埃森哲环球服务有限公司 | System and method for automating ETL application |
| US20100280990A1 (en) * | 2009-04-30 | 2010-11-04 | Castellanos Maria G | Etl for process data warehouse |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577586A (en) * | 2013-11-08 | 2014-02-12 | 北京国双科技有限公司 | Method and device for processing log records |
| CN103577586B (en) * | 2013-11-08 | 2017-03-15 | 北京国双科技有限公司 | The processing method and processing device of log recording |
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN105447014A (en) * | 2014-08-15 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Metadata management method based on binglog, and method and device used for providing metadata |
| CN105447014B (en) * | 2014-08-15 | 2019-03-15 | 阿里巴巴集团控股有限公司 | Metadata management method based on binlog and for providing the method and device of metadata |
| CN108427699A (en) * | 2017-09-22 | 2018-08-21 | 平安科技（深圳）有限公司 | The method, apparatus and storage medium of fast initialization system database |
| WO2019056707A1 (en) * | 2017-09-22 | 2019-03-28 | 平安科技（深圳）有限公司 | Method and device for quickly initializing system database and storage medium |
| CN112567347A (en) * | 2018-09-06 | 2021-03-26 | 欧姆龙株式会社 | Data processing device, data processing method, and data processing program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102279886B (en) | 2012-10-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11068439B2 (en) | Unsupervised method for enriching RDF data sources from denormalized data | |
| CN101504672B (en) | Method and system for dynamically configuring physical data table | |
| US8527451B2 (en) | Business semantic network build | |
| US10698916B1 (en) | Data preparation context navigation | |
| US20140297592A1 (en) | Computer-readable medium storing program and version control method | |
| CN103890709A (en) | Cache based key-value store mapping and replication | |
| CN104598376A (en) | Data driving layered automation test system and method | |
| CN102279886B (en) | Method and equipment for processing metadata | |
| CN111324610A (en) | Data synchronization method and device | |
| CN101452487B (en) | Data loading method and system, and data loading unit | |
| US8880463B2 (en) | Standardized framework for reporting archived legacy system data | |
| US9406018B2 (en) | Systems and methods for semantic data integration | |
| CN101681377A (en) | User-defined relevance ranking for search | |
| CN105760520A (en) | Data control platform and architecture | |
| CN103309904A (en) | Method and device for generating data warehouse ETL (Extraction, Transformation and Loading) codes | |
| CN101675415B (en) | Program pattern analyzer, pattern appearance status information production method, pattern information generating device, and program | |
| US20150066977A1 (en) | Method and system for managing digital resources | |
| CN112579625A (en) | Multi-source heterogeneous data treatment method and device | |
| CN104715032A (en) | Mapping system and method of Chinese and English table name and field name of report system | |
| CN102024039A (en) | Form generating method based on electric operation maintenance system (EOMS) | |
| CN104216961A (en) | Method and device for data processing | |
| US11537785B1 (en) | Spreadsheet flat data extractor | |
| CN105426481B (en) | Handle the method and device of data | |
| CN114416868A (en) | Data synchronization method, device, equipment and storage medium | |
| CN104573053B (en) | A kind of configuration item template dynamic customization method based on XML |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |