CN109242445A - System, the method and device of development management are carried out to data product lifecycle - Google Patents
System, the method and device of development management are carried out to data product lifecycle Download PDFInfo
- Publication number
- CN109242445A CN109242445A CN201811147270.XA CN201811147270A CN109242445A CN 109242445 A CN109242445 A CN 109242445A CN 201811147270 A CN201811147270 A CN 201811147270A CN 109242445 A CN109242445 A CN 109242445A
- Authority
- CN
- China
- Prior art keywords
- data
- design
- model specification
- demand
- management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Economics (AREA)
- Marketing (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- General Factory Administration (AREA)
Abstract
The invention discloses system, method and devices that a kind of pair of data product lifecycle carries out development management, and system includes: design module, for realizing the data-linkage between sector of demand and engineering department；Execution module, for execute data processing operation stream, and carrying out data production and data quality control according to the data model Specification Design and Business Stream G- Design after confirmation；Management module, for optimizing analysis management to design module and execution module, the optimization analysis management includes: to carry out monitoring of tracing to the source to the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control process.The workflow of traditional demand analysis, Demand Design and design confirmation is switched to realize on line under line by the present invention, it can be improved and cooperate with efficiency between sector of demand and engineering department, reduce time cost, moreover it is possible to carry out monitoring of tracing to the source in real time, can be widely applied to technical field of data processing.
Description
Technical field
The present invention relates to technical field of data processing, especially a kind of pair of data product lifecycles to carry out development management
System, method and device.
Background technique
Traditional data product processing and production process generally after by business department's proposition demand, be sent to engineering department into
Row is realized, is during which needed by a series of activities processes such as business demand analysis, demand exploitation, result confirmations, this section interminable
Workflow brings a large amount of time cost, and the demand of Chang Yin business department changes or both sides link up and exist partially
Difference, and lead to the inefficiency that cooperates between business department and engineering department, even result in final delivery result and demand not
It is corresponding.In addition, over time, many demands can not also be traced because of " technology black box " problem.
Summary of the invention
In order to solve the above technical problems, it is an object of the invention to: provide it is a kind of can trace to the source to the full life of data product
Period carries out system, the method and device of development management, cooperates with efficiency between sector of demand and engineering department to improve.
One aspect of the present invention is adopted the technical scheme that:
The system that development management is carried out to data product lifecycle, comprising:
Module is designed, for realizing the data-linkage between sector of demand and engineering department, the data-linkage includes: head
The Demand Design data of sector of demand are first obtained, then design data carries out data model Specification Design and Business Stream according to demand
G- Design finally confirms the design of data model specification and Business Stream G- Design；
Execution module, for according to the data model Specification Design and Business Stream G- Design after confirmation, to execute at data
Job stream is managed, and carries out data production and data quality control；Wherein, the data quality control include: to data model,
Data warehouse, cloud data exchange process, quality of data rule and data dispatch process are controlled；
Management module, for optimizing analysis management, the optimization analysis management packet to design module and execution module
It includes: to the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control process
Carry out monitoring of tracing to the source.
Further, the design module includes:
Model specification definition module carries out additions and deletions for defining data model specification, and to the data model specification of definition
Change and looks into operation；
Model specification correction verification module carries out data model specification for changing operation according to the additions and deletions of data model specification
Verification；
Model specification retrieval module is multiplexed temporal data model specification for design data according to demand.
Further, the model specification correction verification module includes stage unit, and the stage unit is used for according to data model
The check results of specification, the data model specification changed after operation to additions and deletions are classified.
Further, the design module further includes development management module, and the development management module includes:
Predefined unit, for being predefined to development specifications；
Visualization, for providing pattern manipulation interface and filling template for development specifications；
Control unit is managed for the risk data operation to development specifications, and the risk data operation includes pair
The additions and deletions of development specifications change operation.
Further, the management module includes:
Model specification management module divides domain information accuracy, entity name for the model layers to data model specification
Specification degree of conformity, data dictionary information integrity, storage period reasonability and table association reasonability are checked；
Model specification diagnostic module, for the service condition to data model specification, vigor/isolated object, model storage
Occupancy and operation duration carry out analyzing and diagnosing, and are exported analyzing and diagnosing result based on the preset period；
Process auditing module passes through sector of demand's logarithm for carrying out audit confirmation to business flow graph by sector of demand
Audit confirmation is carried out according to quality, and audit confirmation is carried out to source code by engineering department；
Dispatching management module, for scheduling relationship, dispatching cycle, dispatch state and dispatching resource according to business flow graph
It is managed collectively；
Metadata management module is divided for collecting the metadata in design process and development process, and to metadata
Class management；
Data quality management module, it is raw for acquisition, data processing operation process and the data to Demand Design data
Production process carries out whole process monitoring and audit, carries out consistency guarantee to demand and development result.
Another aspect of the present invention is adopted the technical scheme that:
The method that development management is carried out to data product lifecycle, comprising the following steps:
Data between sector of demand and engineering department are connected, the data-linkage includes: to obtain demand first
The Demand Design data of department, then design data carries out data model Specification Design and Business Stream G- Design according to demand, most
The design of data model specification and Business Stream G- Design are confirmed afterwards；
According to the data model Specification Design and Business Stream G- Design after confirmation, to execute data processing operation stream, and
Carry out data production and data quality control；Wherein, the data quality control includes: to data model, data warehouse, cloud number
It is controlled according to interactive process, quality of data rule and data dispatch process；
To the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control
Process carries out monitoring of tracing to the source.
Further, the step for data between sector of demand and engineering department are connected, further include following
Step:
Data model specification is defined, and the progress additions and deletions of the data model specification of definition are changed and look into operation；
Change operation according to the additions and deletions of data model specification, data model specification is verified；
Design data according to demand is multiplexed temporal data model specification.
Further, the step for data between sector of demand and engineering department are connected, further include following
Step:
Development specifications is predefined；
Pattern manipulation interface and filling template are provided for development specifications；
The risk data operation of development specifications is managed, the risk data operation includes the additions and deletions to development specifications
Change operation.
Further, it is described to the design of data model specification, Business Stream G- Design, data processing operation stream, data production and
Data quality control process carries out the step for monitoring of tracing to the source, comprising the following steps:
Divide domain information accuracy, entity Naming conventions degree of conformity, data dictionary letter to the model layers of data model specification
Breath integrality, storage period reasonability and table association reasonability are checked；
The service condition of data model specification, vigor/isolated object, model storage occupancy and operation duration are carried out
Analyzing and diagnosing, and exported analyzing and diagnosing result based on the preset period；
Audit confirmation is carried out to business flow graph by sector of demand, the quality of data audit really by sector of demand
Recognize, and audit confirmation is carried out to source code by engineering department；
According to business flow graph, scheduling relationship, dispatching cycle, dispatch state and scheduling resource are managed collectively；
The metadata in design process and development process is collected, and Classification Management is carried out to metadata；
To the acquisition of Demand Design data, data processing operation process and data production process carry out whole process monitoring and
Audit carries out consistency guarantee to demand and development result.
Another aspect of the present invention is adopted the technical scheme that:
The device of development management is carried out to data product lifecycle, comprising:
Memory, for storing program；
Processor is used for loading procedure, to execute the side for carrying out development management to data product lifecycle
Method.
The beneficial effects of the present invention are: the workflow that the present invention confirms traditional demand analysis, Demand Design and design
Journey switchs to realize on line under line, can be improved and cooperate with efficiency between sector of demand and engineering department, reduce and link up under line
Time cost, moreover it is possible to the data in workflow trace to the source monitoring in real time, practicability height.
Detailed description of the invention
Fig. 1 is the overall step flow chart of the embodiment of the present invention.
Specific embodiment
The present invention is further explained and is illustrated with specific embodiment with reference to the accompanying drawings of the specification.For of the invention real
The step number in example is applied, is arranged only for the purposes of illustrating explanation, any restriction is not done to the sequence between step, is implemented
The execution sequence of each step in example can be adaptively adjusted according to the understanding of those skilled in the art.
The embodiment of the invention provides the system that a kind of pair of data product lifecycle carries out development management, the systems
Include:
Module is designed, for realizing the data-linkage between sector of demand and engineering department, the data-linkage includes: head
The Demand Design data of sector of demand are first obtained, then design data carries out data model Specification Design and Business Stream according to demand
G- Design finally confirms the design of data model specification and Business Stream G- Design；
Wherein, design module is responsible for linking business and technology, is the core hinge of data processing.
Execution module, for according to the data model Specification Design and Business Stream G- Design after confirmation, to execute at data
Job stream is managed, and carries out data production and data quality control；Wherein, the data quality control include: to data model,
Data warehouse, cloud data exchange process, quality of data rule and data dispatch process are controlled；
Wherein, execution module and ETL are decoupled, and have merged data model management, data processing, metadata in data warehouse
The necessary functions such as interactive maintenance, quality of data regulation management and management and running are also responsible for execution, the data of data processing operation stream
Production and the work such as data quality control.
Management module, for optimizing analysis management, the optimization analysis management packet to design module and execution module
It includes: to the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control process
Carry out monitoring of tracing to the source.
Wherein, management module is the module of data processing daily monitoring and integrated management, is responsible for development specifications and data
The control of specification, specifically, the control of the development specifications includes exploitation, test, online process, code specification etc.；It is described
The control of data standard: data definition, quality rule, SQL access right etc.；It is held in addition, management module is also responsible for data processing
The monitoring and optimization analysis management of row process.
With the arriving of big data era, the present invention is able to solve the following problems in big data whole process: this
Invention can clean data before data enter data warehouse, to eliminate the inconsistency in source data, and then guarantee
Information integrity in data warehouse；In addition, the present invention will also carry out various verifications to the data for entering data warehouse, than
Such as:
Data format verification carries out format control, such as the format to date data, number, the amount of money etc., length to data
Degree and data format carry out unified setting；
Logical relation verification, audits the equilibrium relation between data, as declaration form table in, relationship between table, correlation time
Sequencing；
Qualification verification is arranged corresponding qualification permission to the object of data description, the people of different identity is checked
The data of different qualifications；
Timeliness verification, judges the timeliness of data.
For the present invention by design module, execution module and management module, can ensure that the overall process of data processing is transparent can
Control realizes the exploitation of tool and procedure, O&M and management to data, improves the effect of data product exploitation and production
Rate, it can be ensured that the quality of data product.In addition, the present invention realizes by data model abundant and component and passes through dragging
Mode carry out design data, production and management.
It is further used as preferred embodiment, the design module includes:
Model specification definition module carries out additions and deletions for defining data model specification, and to the data model specification of definition
Change and looks into operation；
Model specification correction verification module carries out data model specification for changing operation according to the additions and deletions of data model specification
Verification；
Model specification retrieval module is multiplexed temporal data model specification for design data according to demand.
Specifically, design module of the invention is also with the following functions:
1), support model specification defined function can formulate data model specification, and support to carry out data model specification
Flexible additions and deletions, which change, looks into operation；
2), when developer carries out newly-increased modification or case-deleted models, the present invention will verify data model specification,
It is the corresponding classification of data model norm-setting and rank, to be confirmed whether that developer is allowed to submit and according to check results
Modified data model specification；
3), model index function is provided, for new business demand, to reduce model redundancy and fast implementing demand, this
Invention can be multiplexed existing data model specification.
It is further used as preferred embodiment, the model specification correction verification module includes stage unit, and the classification is single
Member for the check results according to data model specification, classify by the data model specification changed after operation to additions and deletions.
It is further used as preferred embodiment, the design module further includes development management module, the development management
Module includes:
Predefined unit, for being predefined to development specifications；
Visualization, for providing pattern manipulation interface and filling template for development specifications；
Control unit is managed for the risk data operation to development specifications, and the risk data operation includes pair
The additions and deletions of development specifications change operation.
Wherein, design module of the invention also has the function of development management, specific as follows:
Development process and code reasonability are managed, reduce different developers work out application quality difference and
The influence that system may cause.Major function includes:
1) predefined development specifications function is provided, and has scalability, defining a good development specifications will act on
In entire development process；
2) visual pattern manipulation interface is provided, developer can realize flow chart of data processing by filling template
Exploitation, and then reduce written in code amount；
3) risk data operation control function is provided.
It is further used as preferred embodiment, the management module includes:
Model specification management module divides domain information accuracy, entity name for the model layers to data model specification
Specification degree of conformity, data dictionary information integrity, storage period reasonability and table association reasonability are checked；
Model specification diagnostic module, for the service condition to data model specification, vigor/isolated object, model storage
Occupancy and operation duration carry out analyzing and diagnosing, and are exported analyzing and diagnosing result based on the preset period；
Process auditing module passes through sector of demand's logarithm for carrying out audit confirmation to business flow graph by sector of demand
Audit confirmation is carried out according to quality, and audit confirmation is carried out to source code by engineering department.
Dispatching management module, for scheduling relationship, dispatching cycle, dispatch state and dispatching resource according to business flow graph
It is managed collectively；
Metadata management module is divided for collecting the metadata in design process and development process, and to metadata
Class management；
Data quality management module, it is raw for acquisition, data processing operation process and the data to Demand Design data
Production process carries out whole process monitoring and audit, carries out consistency guarantee to demand and development result.
Specifically, management module of the invention is also with the following functions:
1), data model specification is checked, judges that the model layers of data model specification divide domain information accuracy, reality
Body Naming conventions degree of conformity, data dictionary information integrity, storage period reasonability are associated with reasonability etc. with table and are checked；
2), data model specification diagnostic function is provided, and periodically export diagnostic result, wherein diagnosis content is main
It include: service condition (refineing to field), vigor/isolated object analysis, model storage occupancy situation and the operation duration of model
Analysis etc.；
3), process audit examination & approval function is provided, business personnel is allowed to participate in the overall process the working process overall process of data product, it is main
Wanting function includes: the business flow graph audit confirmation of (1) business side；(2) quality of data of business side audits confirmation；(3) technology side
Source code audit confirmation.
4), unified management and running, the task schedule for data Layer and application layer are provided.
Under United Dispatching, each task is data model objects one by one, the context of task in strict accordance with
Job stream (business flow graph automatic conversion) relationship is scheduled, and realizes whole job stream and data of the data from source to target
The consistency of scheduling.Under United Dispatching, relation management, management dispatching cycle, dispatch state management and scheduling can be scheduled
Resource management.
5), metadata management function is provided, obtain and manage all kinds of metadata letter precipitated in design, development process
Breath, and provide and open view, to support administrative staff, business personnel and technical staff rapidly and accurately to understand system.
Wherein, the metadata management function specifically includes the following contents: (1), metadata obtain: provide by technical element number
Classify according to obtaining with data service metadata acquisition, allows developer while completion application is designed, developed, complete the receipts of metadata
Collection；(2) data service metadata and technology metadata, technology metadata and technology realization, job stream metadata consistency guarantee: are provided
With the security function of the consistency such as data dispatch.
6), data quality management function is provided: by being monitored to design data and Quality Process, result data
The quality problems generated with the misfitting of demand, derived data and process of machining/processing.
Wherein, the data quality management function in the present embodiment specifically includes following two aspect:
(1), Quality Design: being monitored Business Stream G- Design, ensures the consistency between design and demand；Data mould
Type Specification Design: the consistency between exploitation and design is ensured；Scheduling Design: Business Stream -> job stream -> scheduling prison is realized
Control ensures the online consistency between exploitation；Business Stream audit, code audit and quality audit are carried out one by one, and then are formed
Closed loop audit, ensures the consistency between exploitation and result.
(2) it process monitoring and audit: provides from the whole process monitoring that result exports to the end of data source, process and examines
Count function, mainly include following functions: quality rule management, monitoring management, alarm management, interrupt alarm and quality are examined
Meter.
Referring to Fig.1, the embodiments of the present invention also provide a kind of pair of data product lifecycles to carry out development management
Method, comprising the following steps:
Data between sector of demand and engineering department are connected, the data-linkage includes: to obtain demand first
The Demand Design data of department, then design data carries out data model Specification Design and Business Stream G- Design according to demand, most
The design of data model specification and Business Stream G- Design are confirmed afterwards；
According to the data model Specification Design and Business Stream G- Design after confirmation, to execute data processing operation stream, and
Carry out data production and data quality control；Wherein, the data quality control includes: to data model, data warehouse, cloud number
It is controlled according to interactive process, quality of data rule and data dispatch process；
To the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control
Process carries out monitoring of tracing to the source.
It is further used as preferred embodiment, the data between sector of demand and engineering department carry out being connected this
One step, further comprising the steps of:
Data model specification is defined, and the progress additions and deletions of the data model specification of definition are changed and look into operation；
Change operation according to the additions and deletions of data model specification, data model specification is verified；
Design data according to demand is multiplexed temporal data model specification.
It is further used as preferred embodiment, the data between sector of demand and engineering department carry out being connected this
One step, further comprising the steps of:
Development specifications is predefined；
Pattern manipulation interface and filling template are provided for development specifications；
The risk data operation of development specifications is managed, the risk data operation includes the additions and deletions to development specifications
Change operation.
It is further used as preferred embodiment, it is described to the design of data model specification, Business Stream G- Design, data processing
Job stream, data production and data quality control process carry out the step for monitoring of tracing to the source, comprising the following steps:
Divide domain information accuracy, entity Naming conventions degree of conformity, data dictionary letter to the model layers of data model specification
Breath integrality, storage period reasonability and table association reasonability are checked；
The service condition of data model specification, vigor/isolated object, model storage occupancy and operation duration are carried out
Analyzing and diagnosing, and exported analyzing and diagnosing result based on the preset period；
Audit confirmation is carried out to business flow graph by sector of demand, the quality of data audit really by sector of demand
Recognize, and audit confirmation is carried out to source code by engineering department；
According to business flow graph, scheduling relationship, dispatching cycle, dispatch state and scheduling resource are managed collectively；
The metadata in design process and development process is collected, and Classification Management is carried out to metadata；
To the acquisition of Demand Design data, data processing operation process and data production process carry out whole process monitoring and
Audit carries out consistency guarantee to demand and development result.
The embodiment of the invention also provides a kind of data processing equipments, comprising:
Memory, for storing program；
Processor is used for loading procedure, to execute the side for carrying out development management to data product lifecycle
Method.
Suitable for present apparatus embodiment, present apparatus embodiment is implemented content in above method embodiment
Function is identical as above method embodiment, and the beneficial effect reached and above method embodiment beneficial effect achieved
It is identical.
In conclusion the present invention with solve to link up between business and technology " pain spot " that cooperates be it is basic, allow business department
The overall process for participating in data product processing is jointly carried on a shoulder pole data product quality with engineering department and is blamed, and realization adds data product
Work process is visual, can manage, controllably, it is ensured that the accuracy of data to business.
Compared to traditional data product development management method, the invention has the following advantages that
1), the present invention is the critical role promoted business to design data compared to the maximum advantage of existing method, is solved
The business and data inconsistence problems being prevalent in data mining.
2), through the invention, the cooperation of business and data will not be bottleneck, and the invention enables the work quickenings of data analysis
Into division of labor elaboration phase, the target optimized to design data quality is realized, the business for greatly improving data processing is satisfied
Degree.
3), the present invention switchs to the workflow of traditional demand analysis, Demand Design and design confirmation on line under line
It realizes, can be improved and cooperate with efficiency between sector of demand and engineering department, reduce the time cost linked up under line, moreover it is possible to right
Data in workflow carry out monitoring of tracing to the source in real time, and practicability is high.
It is to be illustrated to preferable implementation of the invention, but the present invention is not limited to the embodiment above, it is ripe
Various equivalent deformation or replacement can also be made on the premise of without prejudice to spirit of the invention by knowing those skilled in the art, this
Equivalent deformation or replacement are all included in the scope defined by the claims of the present application a bit.
Claims (10)
1. the system that pair data product lifecycle carries out development management, it is characterised in that: include:
Module is designed, for realizing the data-linkage between sector of demand and engineering department, the data-linkage includes: to obtain first
The Demand Design data of sector of demand are taken, then design data carries out data model Specification Design according to demand and business flow graph is set
Meter finally confirms the design of data model specification and Business Stream G- Design；
Execution module, for making to execute data processing according to the data model Specification Design and Business Stream G- Design after confirmation
Industry stream, and carry out data production and data quality control；Wherein, the data quality control includes: to data model, data
Warehouse, cloud data exchange process, quality of data rule and data dispatch process are controlled；
Management module, for optimizing analysis management to design module and execution module, the optimization analysis management includes: pair
Data model Specification Design, Business Stream G- Design, data processing operation stream, data production and data quality control process trace back
Source monitoring.
2. the system according to claim 1 for carrying out development management to data product lifecycle, it is characterised in that: institute
Stating design module includes:
Model specification definition module for defining data model specification, and changes the progress additions and deletions of the data model specification of definition and looks into
Operation；
Model specification correction verification module verifies data model specification for changing operation according to the additions and deletions of data model specification；
Model specification retrieval module is multiplexed temporal data model specification for design data according to demand.
3. the system according to claim 2 for carrying out development management to data product lifecycle, it is characterised in that: institute
Stating model specification correction verification module includes stage unit, and the stage unit is used for the check results according to data model specification, right
The data model specification that additions and deletions change after operation is classified.
4. the system according to claim 2 for carrying out development management to data product lifecycle, it is characterised in that: institute
Stating design module further includes development management module, and the development management module includes:
Predefined unit, for being predefined to development specifications；
Visualization, for providing pattern manipulation interface and filling template for development specifications；
Control unit is managed for the risk data operation to development specifications, and the risk data operation includes to exploitation
The additions and deletions of specification change operation.
5. the system according to claim 1 for carrying out development management to data product lifecycle, it is characterised in that: institute
Stating management module includes:
Model specification management module divides domain information accuracy, entity Naming conventions for the model layers to data model specification
Degree of conformity, data dictionary information integrity, storage period reasonability and table association reasonability are checked；
Model specification diagnostic module is occupied for the service condition to data model specification, vigor/isolated object, model storage
Rate and operation duration carry out analyzing and diagnosing, and are exported analyzing and diagnosing result based on the preset period；
Process auditing module, for carrying out audit confirmation to business flow graph by sector of demand, by sector of demand to data matter
Amount carries out audit confirmation, and carries out audit confirmation to source code by engineering department；
Dispatching management module, for being carried out to scheduling relationship, dispatching cycle, dispatch state and scheduling resource according to business flow graph
Unified management；
Metadata management module carries out classification pipe for collecting the metadata in design process and development process, and to metadata
Reason；
Data quality management module was produced for acquisition, data processing operation process and the data to Demand Design data
The monitoring of Cheng Jinhang whole process and audit carry out consistency guarantee to demand and development result.
6. the method that pair data product lifecycle carries out development management, it is characterised in that: the following steps are included:
Data between sector of demand and engineering department are connected, the data-linkage includes: the sector of demand of acquisition first
Demand Design data, then design data carries out data model Specification Design and Business Stream G- Design according to demand, finally right
Data model Specification Design and Business Stream G- Design are confirmed；
According to the data model Specification Design and Business Stream G- Design after confirmation, to execute data processing operation stream, and carry out
Data production and data quality control；Wherein, the data quality control includes: to hand over data model, data warehouse, cloud data
Mutual process, quality of data rule and data dispatch process are controlled；
To the design of data model specification, the production of Business Stream G- Design, data processing operation stream, data and data quality control process
Carry out monitoring of tracing to the source.
7. the method according to claim 6 for carrying out development management to data product lifecycle, it is characterised in that: institute
The step for being connected to the data between sector of demand and engineering department is stated, further comprising the steps of:
Data model specification is defined, and the progress additions and deletions of the data model specification of definition are changed and look into operation；
Change operation according to the additions and deletions of data model specification, data model specification is verified；
Design data according to demand is multiplexed temporal data model specification.
8. the method according to claim 7 for carrying out development management to data product lifecycle, it is characterised in that: institute
The step for being connected to the data between sector of demand and engineering department is stated, further comprising the steps of:
Development specifications is predefined；
Pattern manipulation interface and filling template are provided for development specifications；
The risk data operation of development specifications is managed, the risk data operation includes changing behaviour to the additions and deletions of development specifications
Make.
9. the method according to claim 6 for carrying out development management to data product lifecycle, it is characterised in that: institute
State to the design of data model specification, Business Stream G- Design, data processing operation stream, data production and data quality control process into
Capable the step for monitoring of tracing to the source, comprising the following steps:
Divide domain information accuracy, entity Naming conventions degree of conformity, data dictionary information complete the model layers of data model specification
Whole property, storage period reasonability and table association reasonability are checked；
The service condition of data model specification, vigor/isolated object, model storage occupancy and operation duration are analyzed
Diagnosis, and exported analyzing and diagnosing result based on the preset period；
Audit confirmation is carried out to business flow graph by sector of demand, audit confirmation is carried out to the quality of data by sector of demand, with
And audit confirmation is carried out to source code by engineering department；
According to business flow graph, scheduling relationship, dispatching cycle, dispatch state and scheduling resource are managed collectively；
The metadata in design process and development process is collected, and Classification Management is carried out to metadata；
Whole process monitoring is carried out to the acquisition of Demand Design data, data processing operation process and data production process and is examined
Meter carries out consistency guarantee to demand and development result.
10. the device that pair data product lifecycle carries out development management, it is characterised in that: include:
Memory, for storing program；
Processor, be used for loading procedure, with execute as claim in any one of claims 6-9 to data product lifecycle into
The method of row development management.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811147270.XA CN109242445A (en) | 2018-09-29 | 2018-09-29 | System, the method and device of development management are carried out to data product lifecycle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811147270.XA CN109242445A (en) | 2018-09-29 | 2018-09-29 | System, the method and device of development management are carried out to data product lifecycle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109242445A true CN109242445A (en) | 2019-01-18 |
Family
ID=65054549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811147270.XA Pending CN109242445A (en) | 2018-09-29 | 2018-09-29 | System, the method and device of development management are carried out to data product lifecycle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109242445A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825725A (en) * | 2019-10-12 | 2020-02-21 | 国网安徽省电力有限公司 | Data quality checking method and system based on double helix management |
| CN110851426A (en) * | 2019-11-19 | 2020-02-28 | 重庆华龙网海数科技有限公司 | Data DNA visualization relation analysis system and method |
| CN112905329A (en) * | 2021-03-24 | 2021-06-04 | 武汉众邦银行股份有限公司 | Full life cycle management and control method for improving standard falling rate of data |
| CN113190805A (en) * | 2021-05-31 | 2021-07-30 | 世纪龙信息网络有限责任公司 | Code asset management system |
| CN115017662A (en) * | 2022-06-27 | 2022-09-06 | 江苏汤谷智能科技有限公司 | Digital circuit design IP management system for full-flow monitoring |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080127040A1 (en) * | 2006-08-31 | 2008-05-29 | Jon Barcellona | Enterprise-Scale Application Development Framework Utilizing Code Generation |
| CN104036365A (en) * | 2014-06-26 | 2014-09-10 | 浪潮软件股份有限公司 | Method for constructing enterprise-level data service platform |
| CN104766151A (en) * | 2014-12-29 | 2015-07-08 | 国家电网公司 | Quality management and control method for electricity transaction data warehouses and management and control system thereof |
| CN105893593A (en) * | 2016-04-18 | 2016-08-24 | 国网山东省电力公司信息通信公司 | Data fusion method |
| CN106777970A (en) * | 2016-12-15 | 2017-05-31 | 北京锐软科技股份有限公司 | The integrated system and method for a kind of medical information system data template |
| CN107463603A (en) * | 2017-06-16 | 2017-12-12 | 中国科学院计算机网络信息中心 | It is a kind of that control method and system are customized based on the scientific research project life cycle data management for quantifying DMP |
-
2018
- 2018-09-29 CN CN201811147270.XA patent/CN109242445A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080127040A1 (en) * | 2006-08-31 | 2008-05-29 | Jon Barcellona | Enterprise-Scale Application Development Framework Utilizing Code Generation |
| CN104036365A (en) * | 2014-06-26 | 2014-09-10 | 浪潮软件股份有限公司 | Method for constructing enterprise-level data service platform |
| CN104766151A (en) * | 2014-12-29 | 2015-07-08 | 国家电网公司 | Quality management and control method for electricity transaction data warehouses and management and control system thereof |
| CN105893593A (en) * | 2016-04-18 | 2016-08-24 | 国网山东省电力公司信息通信公司 | Data fusion method |
| CN106777970A (en) * | 2016-12-15 | 2017-05-31 | 北京锐软科技股份有限公司 | The integrated system and method for a kind of medical information system data template |
| CN107463603A (en) * | 2017-06-16 | 2017-12-12 | 中国科学院计算机网络信息中心 | It is a kind of that control method and system are customized based on the scientific research project life cycle data management for quantifying DMP |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110825725A (en) * | 2019-10-12 | 2020-02-21 | 国网安徽省电力有限公司 | Data quality checking method and system based on double helix management |
| CN110851426A (en) * | 2019-11-19 | 2020-02-28 | 重庆华龙网海数科技有限公司 | Data DNA visualization relation analysis system and method |
| CN112905329A (en) * | 2021-03-24 | 2021-06-04 | 武汉众邦银行股份有限公司 | Full life cycle management and control method for improving standard falling rate of data |
| CN113190805A (en) * | 2021-05-31 | 2021-07-30 | 世纪龙信息网络有限责任公司 | Code asset management system |
| CN113190805B (en) * | 2021-05-31 | 2024-06-25 | 天翼数字生活科技有限公司 | Code asset management system |
| CN115017662A (en) * | 2022-06-27 | 2022-09-06 | 江苏汤谷智能科技有限公司 | Digital circuit design IP management system for full-flow monitoring |
| CN115017662B (en) * | 2022-06-27 | 2023-08-22 | 江苏汤谷智能科技有限公司 | Digital circuit design IP management system for full-flow monitoring |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109242445A (en) | System, the method and device of development management are carried out to data product lifecycle | |
| CN105574593B (en) | Track state static detection and control system and method based on cloud computing and big data | |
| CN104899143B (en) | The software peer review system implementation device of data mining is provided | |
| CN112181955B (en) | Data standard management method for information sharing of heavy haul railway comprehensive big data platform | |
| CN110221975A (en) | Create the method and device of interface use-case automatic test script | |
| CN112527791A (en) | Intelligent urban brain big data system | |
| Lagerström et al. | Visualizing and measuring enterprise architecture: an exploratory biopharma case | |
| CN108985471B (en) | Aircraft management system based on 3D lightweight model | |
| CN113706099A (en) | Data labeling and deep learning model training and service publishing system | |
| US20180012181A1 (en) | Method of collaborative software development | |
| CN107301498A (en) | Generalized information management system for the military service monitoring of high-speed railway wheel rail relation | |
| CN110544035A (en) | internal control detection method, system and computer readable storage medium | |
| CN111079997A (en) | Modeling and collaborative optimization method | |
| Likhitruangsilp et al. | A BIM-enabled system for evaluating impacts of construction change orders | |
| CN105956287A (en) | System and method for integrating product design and environment influence evaluation in CAD system environment | |
| CN111260327A (en) | Project construction cooperation management and control platform based on cloud service | |
| CN110399182A (en) | A kind of CUDA thread placement optimization method | |
| CN109255389A (en) | A kind of equipment evaluation method, device, equipment and readable storage medium storing program for executing | |
| CN105138612A (en) | Method and system for analyzing and locating reasons of data consistency difference | |
| CN117519656A (en) | Software development system based on intelligent manufacturing | |
| CN117575222A (en) | Production management method, system, equipment and storage medium | |
| CN109992251B (en) | Business variable-oriented software development scheme generation method | |
| CN112801768A (en) | Internet + government affair-based natural fund and performance data sharing system | |
| Zhao et al. | Research on the impact of big data technology on management accounting | |
| CN114971547A (en) | Component product level borrowing analysis method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190118 |