CN117076728A - Data management method and system - Google Patents

Abstract

The invention relates to the technical field of data processing, and provides a data management method and a system, which are used for acquiring stock data in at least one data module and screening key data in the stock data; carrying out standardized processing on the key data and determining a data standard; and generating rules based on data standards, checking stock data and/or standard newly-added data in the data module based on the rules, taking the checked stock data and/or the standard newly-added data as standard data, and storing the standard newly-added data into a corresponding data module. Through implementation of the method and application of the system, standards are formulated according to stock data in each data module, rules are generated according to the standards, and finally, the stock data in the data module and newly-added data to be stored in the data module are standardized based on the rules, so that interfaces and data formats of each data module are unified, the difficulty of integrating the data is reduced, the usability of the data is improved, and closed-loop management of the data is realized.

Description

Data management method and system

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a data management method and system.

Background

Currently, digital economics have become an important engine that drives the development of economics of high quality. As a core asset of an enterprise, making full use of data-driven business innovation and development has become a consensus of various industries. In the process of digital transformation of a traditional manufacturing enterprise, data becomes a key production element for promoting the development of enterprise digitization, networking and intellectualization, but the manufacturing enterprise has long flow, the data penetrates through various links of product research and development design, production and manufacturing, process flow, supply and marketing, logistics, after-sales service, enterprise operation management, operation and maintenance service and the like, the data volume of each system is huge and redundant, and how to solve data island, improve the data quality of the enterprise and unify data standards, release the data value, so that the data is better served and the service is enabled is an important challenge facing the digitization of the manufacturing enterprise.

As shown in fig. 1, the structure of a conventional data system in a manufacturing industry is shown, in this system, a plurality of data modules distributed among different subsystems, such as a product data management (Product Data Management, PDM) data module 41, an enterprise resource planning (Enterprise Resource Planning, ERP) data module 42, a production execution system (Manufacturing Execution System, MES) data module 43, a Maintenance (MRO) data module 44, a quality management system (Quality management system, QMS) data module 45, a warehouse management system (Warehouse Management System, WMS) data module 46, etc., are included, and these data modules can store different data, and the different data modules are respectively connected with each other, so that the data formats in the data modules are disordered, the difficulty of integrating the data is high, which makes it difficult to normalize the stock data in each data module, and it is difficult to constrain the newly added data to conform to the standard, so that the usability of the data is poor.

Disclosure of Invention

The invention provides a data management method and a system, which are used for solving the defects that in the prior art, the data format of each data module in a data system is disordered, the data integration difficulty is high, the timeliness and the accuracy of data maintenance are inconsistent, so that the stock data in each data module is difficult to normalize, the newly added data is difficult to restrict to ensure that the newly added data accords with the standard, the stock data in each module is standardized, and the newly added data is restricted to ensure that the newly added data accords with the standard.

The invention provides a data management method, which comprises the following steps:

acquiring stock data in at least one data module;

screening out key data in the stock data;

carrying out standardization processing on the key data to determine a data standard;

generating rules based on the data criteria;

and verifying the stock data and/or the standard newly-added data based on the rule, and taking the verified stock data and/or the standard newly-added data as standard data.

According to the data management method provided by the invention, the key data in the stock data are screened out, and the method comprises the following steps:

calculating the importance degree of each data in the stock data;

ranking the individual data based on the importance;

the preset number of data arranged in front is selected as key data.

According to the data management method provided by the invention, the importance degree of each data in the stock data is calculated, and the method comprises the following steps:

calculating an initial reference degree of each datum in the stock data:

(αCI _n +(1-α)*(DI _n +SA _x ))，

wherein alpha is a damping coefficient, CI _n For the degree of reference of the nth data, DI _n For the integrity of the nth data, SA _x Stability under x cycles, x being the time period;

overlapping the reference degree according to the cited condition of the data:

wherein p is the total number of data; out is provided with _n The number of outgoing chains of the nth data reference node;

calculating importance of data:

wherein TR is importance;

and performing iterative calculation until the importance degree tends to be stable, and obtaining a final importance degree result.

According to the data management method provided by the invention, the calculation of the reference degree comprises the following steps:

CI＝(SQ _n +50*US _n )

wherein SQ _n Number of queries for the nth service; US (US) _n The number of users for downstream use of the nth service;

the calculation of the stability comprises:

calculating a statistical period:

Cycle _h ＝(30*24-En _h )/(30*24)

Cycle _d ＝(30-En _d )/30

Cycle _w ＝(7-En _w )/7

wherein, cycle _h Cycle is a statistical period in hours _d Cycle is a statistical period in days _w En is the statistical period in units of weeks _h In order to give an alarm in time period of unit of hour, en _d For number of alarms not generated on time in time period of days, en _w The number of alarms which are not generated on time in the time period taking the week as a unit;

calculation stability:

SA _x ＝(Cycle _x *100)，

where x is the time period.

According to the data management method provided by the invention, the key data is subjected to standardized processing, and the data standard is determined, which comprises the following steps:

distinguishing the attribute information of the key data, and deleting the attribute information which accords with the preset redundant information condition;

determining characteristic attributes of each attribute information in the key data, and constructing a characteristic attribute set of each attribute information;

and combining the characteristic attribute sets of the attribute information to construct a data standard set.

According to the data governance method provided by the invention, rules are generated based on the data standard, and the data governance method comprises the following steps:

generating rules containing data formats based on the data standard set; the data format includes a definition of data, a data structure, and a list of values.

According to the data management method provided by the invention, the stock data and/or the specification newly-added data are checked based on the rule, and the data management method comprises the following steps:

checking the stock data in the data module one by one according to the rule, screening the stock data with the data format conforming to the rule as standard data, correcting the stock data with the screened data format not conforming to the rule according to the rule, and taking the corrected stock data as standard data;

and/or checking the newly added data according to the rule, and screening the newly added data with the data format conforming to the rule as standard data.

According to the data management method provided by the invention, after the checked stock data and/or the normalized newly-added data are used as standard data, the method further comprises the following steps:

and constructing panoramic data based on the association relation between the standard data and the data corresponding party.

The invention also provides a data management system for executing any one of the data management methods, wherein the data management system comprises a data management unit and at least one data module, and the data module contains stock data; the data management unit is used for acquiring stock data in at least one data module, screening out key data in the stock data, carrying out standardized processing on the key data, determining a data standard, generating a rule based on the data standard, checking the stock data based on the rule, standardizing newly-added data based on the rule, and taking the checked stock data and the standardized newly-added data as standard data;

the data management unit comprises a basic data supporting layer, wherein the basic data supporting layer comprises a standard unit and a rule unit; the standard unit comprises the data standard; the rule unit contains the rule.

According to the data management system provided by the invention, the data management unit further comprises a panoramic data construction layer, wherein the panoramic data construction layer is used for constructing panoramic data based on the association relation between the standard data and the data corresponding party.

According to the data management method and system provided by the invention, stock data in at least one data module is obtained, and key data in the stock data are screened out; carrying out standardized processing on the key data and determining a data standard; and generating rules based on data standards, checking stock data and/or standard newly-added data in a data module based on the rules, and taking the checked stock data and/or the standard newly-added data as standard data. Through implementation of the method and application of the system, standards are formulated according to stock data in each data module, rules are generated according to the standards, and finally, the stock data in the data module and newly-added data to be stored in the data module are standardized based on the rules, so that data formats of the data in each data module are unified, the difficulty of integrating the data is reduced, the usability of the data is improved, and closed-loop management of the data is realized.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a prior art data system;

FIG. 2 is a schematic flow chart of a data governance method provided by the present invention;

FIG. 3 is a flow chart of an alternative data governance method in accordance with an embodiment of the present invention;

FIG. 4 is a schematic diagram of an alternative data management system according to the present invention;

FIG. 5 is a schematic diagram of a data management unit according to an embodiment of the present invention;

FIG. 6 is a second schematic diagram of a data management unit according to an embodiment of the invention.

Reference numerals:

40: a data management unit; 41: a product data management data module; 42: an enterprise resource planning data module; 43: a production execution system data module; 44: operating a maintenance data module; 45: a quality management system data module; 46: a warehouse management system data module; 50: a base data support layer; 51: a standard cell; 52: a rule unit; 60: a panoramic data construction layer; 61: a service carding module; 62: a data identification module; 63: a data classification and grading module; 64: a table structure module; 65: and a data association module.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The data governance method of the present invention is described below in conjunction with fig. 2 and 3.

As shown in fig. 1, the data management method provided by the invention comprises the following steps:

s1, acquiring stock data in at least one data module.

In this step, stock data needs to be acquired from the data module. It should be understood that the present invention is not limited to the specific field, distribution characteristics, number and variety of data modules, and is described below by way of example in terms of a data system for a manufacturing enterprise:

the manufacturing data systems include data modules distributed among different subsystems (e.g., production execution systems, quality management systems, etc.), which may include product data management (Product Data Management, PDM) data modules, enterprise resource planning (Enterprise Resource Planning, ERP) data modules, production execution system (Manufacturing Execution System, MES) data modules, maintenance of operation (MRO) data modules, quality management system (Quality management system, QMS) data modules, warehouse management system (Warehouse Management System, WMS) data modules, etc. It should be noted that the above data module is only an example, and other data modules may be included in the present invention in addition to the above data module, where the data modules may store different data.

The product data management data module is used for managing all data related to the product (including part information, configuration, documents, CAD files, structures, authority information and the like) and all data related to the product (including process definition and management). The enterprise resource planning data module is used for effectively sharing and utilizing enterprise resources, and fully tidying and effectively transmitting information through the information system, so that the resources of the enterprise can be reasonably configured and utilized in various aspects of purchasing, storing, producing, selling, people, finance, things and the like, and the enterprise operation efficiency is improved. The production execution system data module is used for forming a complete production informatization system, meeting the requirements of enterprises in different planning stages in the field of informatization production management, and realizing the stable transition and gradual improvement of the informatization process. The operation maintenance data module is used for ordering consumable parts (stationery, office supplies, repair parts of equipment and machines and the like) from an enterprise by utilizing the Internet, so that the enterprise can acquire and purchase the consumable parts in daily life to realize efficient distribution. The quality management system data module is used for selecting a plurality of system elements to be combined according to the characteristics of enterprises, enhancing the quality management activities of the whole process of design development, production, inspection, sales and use, and making the quality management activities systematic and standardized, and becoming the requirements and the activity programs of the quality work in the enterprises. The warehouse management system data module is used for managing information, resources, behaviors, inventory and distribution operations according to the operational business rules and algorithms, and improving efficiency.

For a manufacturing enterprise, there is typically a large amount of data in the several modules, which are referred to as stock data, which is critical to the enterprise and is the basis for the normal operation of the enterprise. In addition to inventory data, as the production of the enterprise proceeds, new data, such as newly purchased equipment, new suppliers, new standards, etc., may be generated, which may be referred to as new add-on data, which is an integral part of the enterprise's development. In the prior art, the stock data generally has a plurality of different formats, for example, different data modules adopt different data formats, and the data formats can be in the form of numerical values, characters or binary numbers, etc., which causes the problems of difficult acquisition, difficult organization, difficult sharing, difficult application, etc. of the data between the different data modules. The difficulty in acquiring mainly because the same data are simultaneously maintained in different systems, the timeliness and the accuracy of the maintenance are inconsistent, and the accurate and authoritative data sources are difficult to be definitely and accurately acquired; the difficult organization is mainly because the equipment global data are scattered in a plurality of service systems for storage, and the data integration difficulty and the cost are increased; the difficulty in sharing is mainly due to the fact that a large number of interfaces are generated in data sharing among systems, when the same data are maintained in a plurality of systems, the interfaces among the systems are also required to be maintained simultaneously, and rapid sharing is difficult to achieve; the difficult application mainly causes that the data accuracy cannot be ensured due to different data sources between the systems. For stock data, even data in the same data module may exist in multiple different formats, which further exacerbates the difficulty of sharing and application of the data.

S2, screening out key data in the stock data.

In this step, it is necessary to calculate the importance degree of each data in the stock data, rank each data based on the importance degree, and select a preset number of data in the top order as key data. Specifically, the importance degree sorting and screening of the stock data are completed by utilizing the circulation relation among the data, the key data to be standardized is determined, and if one data is applied to a plurality of times, the data is proved to be important, namely the TR value is relatively high; if a data having a high TR value is applied to another data, the TR value of the data to be applied is correspondingly increased.

The algorithm for calculating the importance of the stock data in the stock data comprises three parts of an reference degree (CI), a data integrity Degree (DI) and a stability Score (SA), wherein the specific meaning and the calculation formula are as follows:

(1) Calculation of the degree of reference

The score of the reference degree is divided into two parts, namely the self score and the transmitted numerical value, and the specific calculation formula is as follows:

CI＝(SQ _n +50*US _n )

wherein SQ _n Number of queries for the nth service; US (US) _n The number of users is used downstream of the nth service.

(2) Definition of data integrity

Data integrity represents the degree of data integrity, and in an alternative embodiment of the invention, the total score of data integrity is 100 points, the score composition of which is shown in table 1.

TABLE 1 data integrity score formation table

Description of the invention	Score value
		Data source	25 minutes
Description of data	20 minutes
		Data annotation	25 minutes
Chinese name in table	15 minutes
		Data hierarchy	10 minutes
Business theme	10 minutes

(3) Stability scoring

In an alternative embodiment of the invention, the statistical period is calculated as follows, including hours, days and weeks, scheduled by hours, scheduled by days and scheduled by weeks, respectively:

Cycle _h ＝(30*24-En _h )/(30*24)

Cycle _d ＝(30-En _d )/30

Cycle _w ＝(7-En _w )/7

wherein, cycle _h Cycle is a statistical period in hours _d Cycle is a statistical period in days _w En is the statistical period in units of weeks _h In order to give an alarm in time period of unit of hour, en _d For number of alarms not generated on time in time period of days, en _w The number of alarms that are not generated on time in a time period in weeks.

Calculating data stability:

SA _x ＝(Cycle _x *100)，

where x is a time period including, but not limited to, hours, days or weeks; stability of the data is calculated based on whether the data is produced periodically.

The algorithm of importance degree of each data in the data stock data is as follows:

(1) Calculating the initial self-reference degree of each datum in the stock data:

(αCI _n +(1-α)*(DI _n +SA _x ))，

wherein alpha is a damping coefficient, CI _n For the degree of reference of the nth data, DI _n For the integrity of the nth data, SA _x For stability at x cycles, x is hours (hor), days (day) or weeks (week).

(2) Superimposing the reference degree according to the referenced condition of each data:

wherein p is the total number of data; out is provided with _n The number of outgoing chains of the nth data reference node;

(3) Calculating importance of data:

where TR is importance.

The TR value of the data can be calculated according to the formula, and a final importance result is obtained.

After the importance of each data in the stock data is calculated, ranking each data based on the importance, and selecting the data with the preset number which is ranked at the front as key data. It should be understood that the preset number may be selected according to different data types, and the present invention is not limited thereto, for example, data with the first importance of 50% of the total amount of all the stock data may be selected as the key data.

And S3, carrying out standardization processing on the key data, and determining a data standard.

After screening the key data, carrying out standardized processing on the key data according to the contents of the service subject, the data source, the data description and the like in the data integrity, thereby determining the data standard. The main idea is to normalize data information, represent the data information as a set of all attributes, determine the characteristic attribute of each attribute information (including but not limited to null value, text description, code and other attribute information) in key data, form a characteristic attribute set of each attribute information, and then combine the characteristic attribute set of each attribute information into a standard set, thereby forming a data standard set. Taking the data in the RRP data module as an example, assuming that the characteristic attributes of each attribute in the key data respectively comprise name, age and telephone in the related personnel information in the data, the standard set comprises three attributes of name, age and telephone.

In an alternative embodiment of the invention, for critical data to be normalized, a classification analysis of the data may be performed by a neural network.

In an alternative embodiment of the present invention, the attribute information of the key data may be distinguished, and the attribute information meeting the preset redundant information condition may be deleted, for example, attribute information that is meaningless or has less influence on the work may be deleted, so that the effective expression of the attribute information may be improved. The purpose of the normalization process is to remove the compression redundant information from the original data and to strengthen the useful information.

In an optional embodiment of the present invention, similar clustering may be performed on data with the same data type of the feature attribute according to the data type of the feature attribute in the attribute information of the key data (data to be standardized), so as to generate the corresponding identification system. For example, after feature attribute sets of each attribute information are combined, the obtained data standard set is { device domain-source layer-table 1-temperature-nn°c } and the corresponding code is xxx-yyyy-zzzz-tem-00nn.

S4, generating rules based on the data standard.

Specifically, rules are generated based on the generated criteria, including naming rules, encoding rules, and quality feature auditing rules. The naming rules are used for realizing unification of all naming standards, including but not limited to Chinese naming in a table, english naming, chinese naming in a field and English naming rules in a field; the encoding rules are used in part to constrain the encoding of the value dictionary and in part to verify the encoding of device-related data, such as device encoding; quality characteristic auditing rules are used to verify the integrity, uniqueness, timeliness, validity, accuracy, consistency of data, e.g., to verify the integrity of data without empty constraints. Rules contain definitions for data formats, including definitions of data, data structures, and lists of values. Taking the rule related to the ERP data as an example for illustration, for example, after the data standard is determined, the personnel occupation of 5 items is counted to be larger, and the rule related to the personnel information in the ERP data can be 5 items including name, age, gender, address and telephone, wherein the data formats of the three items of name, gender and address are text formats, and the data formats of the age and the telephone are numerical formats.

S5, checking the stock data and/or the standard newly-added data based on the rule, and taking the checked stock data and/or the standard newly-added data as standard data.

In particular, for massive stock data, it is obviously not practical to check whether the stock data meets the data management requirement manually one by one. Taking personnel information data in the ERP data module as an example, although statistics find that the personnel information contains 5 items with a personnel ratio of 80% and contains 3 or other items with a personnel ratio of only 20%, the picking of 20% of the personnel which do not meet the rule still has huge work, and the personnel information of other 80% of the personnel also has the condition of not meeting the rule, for example, the age in the data may be in a text format rather than a numerical format. In this regard, it is necessary to acquire the stock data in each data module, verify the stock data according to the rule, and convert the stock data into standard data conforming to the standard. For example, personnel information data in the ERP data module is obtained, the personnel information data is checked one by one, the data conforming to the rule is standard data, for example, the data containing 5 items of name, age, sex, address and telephone, wherein the data formats of the three items of name, sex and address are text formats, and the data formats of the age and the telephone are numerical formats, namely the standard data.

And correcting the stock data with the data format which does not accord with the rule according to the rule, including but not limited to manual correction and automatic program correction, and obtaining standard data after correction. For example, for personnel information lacking telephone and address, manual supplementary recording can be performed; data which is not satisfactory in terms of data format may be automatically converted into data format by a computer program, for example, text format data in age is converted into numeric data.

For the newly-added data of different subsystems, the formats of the newly-added data are more disordered because the newly-added data are generally irregularly constrained data, and the newly-added data further cause the confusion of the data after being added into the stock data. The new data may come from different data modules and have different formats, and the non-standard new data may bring disastrous consequences to enterprises, so that the new data needs to be standardized according to rules, so that the new data also meets the standard requirements, i.e. the new data is also converted into standard data.

Taking the ERP data module as an example, when personnel change exists in an enterprise, for example, when new personnel enter, new personnel information is needed to be input through the ERP system at the moment, according to personnel information rules, for example, 5 items including name, age, sex, address and telephone, wherein the data formats of the name, the sex and the address are text formats, the data formats of the age and the telephone are numerical value format data, and when a certain item is absent in personnel information input or the numerical value format does not meet the rule requirement, the ERP system can send out error warning, and the error data cannot be accepted. Only after the new employee data meets the constraints of the rule, the new data can be accepted, thereby obtaining standardized new data.

In an optional embodiment of the present invention, step S5 further includes constructing panoramic data based on the association relationship between the standard data and the data counterpart.

Specifically, the standard data are associated with all relevant data modules to construct a panoramic view. And supporting the accuracy of the associated data through the standard and the rule, and optimizing the standard and the rule through the use condition of the panoramic service data association to form a closed loop. The data counterpart includes, but is not limited to, personnel, equipment, products, etc. to which the standard data corresponds. For example, for a certain device (i.e. a data counterpart), standard data corresponding to the device in each data module can be correlated, that is, panoramic data of the device is constructed, the panoramic data includes data corresponding to the device in each data module, and integration of device data is achieved.

The following specifically describes a data management method provided by the present invention by taking equipment data of a manufacturing industry as an example:

as shown in fig. 3, a method for managing data of manufacturing equipment specifically includes the following steps:

and step 1, combing the equipment related business process, and identifying the equipment related data from top to bottom. The device is a physical entity, taking a practical situation as an example, and for a certain device, the device exists under a certain factory and exists in a certain unit in the factory, and the device comprises a certain device, and the device comprises a certain component, and the component comprises a certain part; the factory area, the machine set, the equipment, the device, the component and the part are the related data of the equipment from top to bottom in the equipment business flow.

And 2, researching a service system, and identifying the equipment related data from bottom to top. Taking a practical case as an example, service system data from an underlying internet of things sensor, to a data storage, to a monitoring system and to a system application, and thus a logic hierarchy division is the equipment related data from bottom to top in the service system.

And step 3, classifying and grading the equipment related data according to the identified related data and combining the safety control requirements.

Step 4, based on the identified device-related data, making standards according to the existing data, wherein the standards comprise, but are not limited to, naming, data structures, value lists and the like; steps 4, 5, 6 may be performed in conjunction with synchronous unfolding.

And 5, formulating rules for generating standards according to the content of the standards, and checking standardized data and simultaneously restricting newly added data in the future, wherein the rules comprise naming rules, coding rules, quality characteristic rules and the like.

And 6, carding the related data table structure of the equipment, wherein the step mainly realizes the condition that the fields of the same field are inconsistent among different tables, for example, the deviceCode and the deviceNo both represent 'equipment codes' but are identified by different fields, and the field unification can be realized through the carding table structure. Steps 4, 5, 6 may be performed in conjunction with synchronous unfolding.

And 7, associating the device data based on the identified and standardized device related data to form a panoramic view of the device.

The above-described exemplary manufacturing facility data governance method is a closed-loop manufacturing facility data governance method employing "set criteria-generate rules-associated data". The management of equipment data is emphasized, and the equipment data can be classified into design, production, operation and operation according to different equipment dimension association data, for example, according to the full life cycle dimension of equipment, and can be classified into static data and dynamic data according to the characteristic dimension of the equipment data. By formulating standards, stock device data is standardized, rules are generated through the standards, future newly-added device data is verified, the standards and the rules can support device data association, and the device association data can optimize the standards and the rules. By the closed-loop data management method, stock data are standardized, newly-added data are restrained, standard data sources are unified, and data quality is improved.

In summary, the invention provides a data management method, which obtains stock data in at least one data module and screens key data in the stock data; carrying out standardized processing on the key data and determining a data standard; and generating rules based on data standards, checking stock data and/or standard newly-added data in the data module based on the rules, taking the checked stock data and/or the standard newly-added data as standard data, and storing the standard newly-added data into a corresponding data module. Through implementation of the method and application of the system, standards are formulated according to stock data in each data module, rules are generated according to the standards, and finally, the stock data in the data module and newly-added data to be stored in the data module are standardized based on the rules, so that closed loop management of the data is realized, interfaces and data formats of each data module are unified, the difficulty of integrating the data is reduced, the usability of the data is improved, and the timeliness and accuracy of data maintenance are guaranteed.

Based on the same inventive concept, the invention also provides a data management system, which is used for executing any one of the data management methods. The data management system provided by the invention is described below, and the data management system described below and the data management method described above can be referred to correspondingly.

As shown in FIG. 4, the invention provides a data governance system comprising a data governance unit 40 and at least one data module, such as a product data management data module 41, an enterprise resource planning data module 42, a production execution system data module 43, an operation maintenance data module 44, a quality management system data module 45, a warehouse management system data module 46, and the like.

The data management unit is used for acquiring the stock data in at least one data module, screening out key data in the stock data, carrying out standardized processing on the key data, determining a data standard, generating rules based on the data standard, checking the stock data based on the rules, and newly-added data based on the rules, wherein the checked stock data and the newly-added data after the rules are used as standard data;

as shown in fig. 5, the data administration unit 40 includes a base data support layer 50 including a standard unit 51 and a regular unit 52; the standard unit 51 includes data standards for summarizing related data in the data modules across the subsystems; the rule unit 52 includes rules for checking the standard, and implementing checking the accuracy of the stock data and normalizing the newly added data.

The standard unit 51 includes generated related standards including, for example, naming standards, data structure standards, value list standards, and the like. The naming standards are used for realizing the induction and summarization of all cross-system standard naming, including but not limited to table Chinese naming, english naming, field Chinese naming dictionary; the data structure standard is used for realizing the standardization of the related data table structure of the equipment, including but not limited to field Chinese name, field English name, data type, data length, value field and filling specification; the value list is used for realizing normalization of the filling of the value range and ensuring that the value range in the data structure is an accurate and up-to-date value.

The rule unit 52 includes a rule generated by the generated standard for normalizing the stock data in each data module and normalizing the newly added data. The rules comprise naming rules, coding rules and quality characteristic auditing rules, and standard formulation comprises definition standards, data structure standards and value list standards. The naming rules are used for realizing unification of all naming standards, including but not limited to Chinese naming in a table, english naming, chinese naming in a field and English naming rules in a field; the encoding rules are used in part to constrain the encoding of the value dictionary and in part to verify the encoding of device-related data, such as device encoding; quality characteristic auditing rules are used to verify the integrity, uniqueness, timeliness, validity, accuracy, consistency of data, e.g., to verify the integrity of data without empty constraints. The data modules of the respective subsystems transmit the stock data to the rule unit 52, and the rule unit 52 checks the stock data according to the rules, and converts the stock data into standard data conforming to the standard. The rule unit 52 is also used to send standard data to the panorama data construction layer 60. For newly added data, rule element 52 constrains and normalizes the newly added data. The newly added data may come from different data modules, have different formats, and the non-canonical newly added data may have disastrous consequences for the enterprise. In this regard, the rule unit 52 provides the data module with the new data rule to normalize the new data so that the new data also meets the standard requirement, i.e. the new data is also converted into standard data.

In an alternative embodiment of the present invention, as shown in fig. 5 and 6, the data governance unit 40 further includes a panorama data construction layer 60, and the panorama data construction layer 60 associates all data modules based on the standard data acquired by the base data support layer 50, forming panorama data, supporting accuracy of the associated data through the standards and rules, and optimizing the standards and rules through the case of panorama data association. The base data support layer 50 is at the bottom layer and the panoramic data construction layer 60 is at the top layer. The underlying base data support layer 50 provides the upper panoramic data construction layer 60 with the treated data, and the panoramic data construction layer 60 constructs panoramic data based on the treated data.

Specifically, the panorama data construction layer 60 includes a service carding module 61, a data recognition module 62, a data classification and grading module 63, a table structure module 64, and a data association module 65, which are sequentially connected.

The service carding module 61 is used for planning service data from top to bottom, firstly carding the primary service flow and refining the secondary and tertiary service flow on the basis. The data identification module 62 is configured to identify equipment related data, including PDM data, ERP data, MES data, MRQ data, QMS data, and WMS data, to ensure that the business entity is not missing. The data classification and classification module 63 is configured to implement rapid management of device data as data assets while satisfying the security requirement classification. The table structure module 64 is used to implement device-dependent data table structure formulation. The data association module 65 is configured to determine an association relationship between the data counterpart and the related data according to the standard data acquired from the basic data support layer, and construct global data of the data counterpart.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of data management comprising:

acquiring stock data in at least one data module;

screening out key data in the stock data;

carrying out standardization processing on the key data to determine a data standard;

generating rules based on the data criteria;

and verifying the stock data and/or the standard newly-added data based on the rule, and taking the verified stock data and/or the standard newly-added data as standard data.

2. The data governance method of claim 1, wherein screening out critical data in the inventory data comprises:

calculating the importance degree of each data in the stock data;

ranking the individual data based on the importance;

the preset number of data arranged in front is selected as key data.

3. The data governance method of claim 2, wherein calculating the importance of each of the inventory data comprises:

calculating the initial reference degree of each datum in the stock data:

(αCI _n +(1-α)*(DI _n +SA _x ))，

wherein alpha is a damping coefficient, CI _n For the degree of reference of the nth data, DI _n For the integrity of the nth data, SA _x Stability under x cycles, x being the time period;

overlapping the reference degree according to the cited condition of the data:

wherein p is the total number of data; out is provided with _n The number of outgoing chains of the nth data reference node;

calculating importance of data:

wherein TR is importance;

and performing iterative calculation until the importance degree tends to be stable, and obtaining a final importance degree result.

4. A data governance method according to claim 3, wherein said calculation of degree of reference comprises:

CI＝(SQ _n +50*US _n )

wherein SQ _n Number of queries for the nth service; US (US) _n The number of users for downstream use of the nth service;

the calculation of the stability comprises:

calculating a statistical period:

Cycle _h ＝(30*24-En _h )/(30*24)

Cycle _d ＝(30-En _d )/30

Cycle _w ＝(7-En _w )/7

wherein, cycle _h Cycle is a statistical period in hours _d Cycle is a statistical period in days _w En is the statistical period in units of weeks _h In order to give an alarm in time period of unit of hour, en _d For number of alarms not generated on time in time period of days, en _w The number of alarms which are not generated on time in the time period taking the week as a unit;

calculation stability:

SA _x ＝(Cycle _x *100)。

5. the data governance method of claim 1, wherein normalizing the critical data to determine a data standard comprises:

distinguishing the attribute information of the key data, and deleting the attribute information which accords with the preset redundant information condition;

determining characteristic attributes of each attribute information in the key data, and constructing a characteristic attribute set of each attribute information;

and combining the characteristic attribute sets of the attribute information to construct a data standard set.

6. The data governance method of claim 5, wherein generating rules based on the data criteria comprises:

generating rules containing data formats based on the data standard set; the data format includes a definition of data, a data structure, and a list of values.

7. The data governance method of claim 6, wherein verifying said inventory data and/or specification new data based on said rules comprises:

checking the stock data in the data module one by one according to the rule, screening the stock data with the data format conforming to the rule as standard data, correcting the stock data with the screened data format not conforming to the rule according to the rule, and taking the corrected stock data as standard data;

and/or checking the newly added data according to the rule, and screening the newly added data with the data format conforming to the rule as standard data.

8. The data governance method according to any one of claims 1 to 7, further comprising, after taking the checked stock data and/or the normalized new data as standard data:

and constructing panoramic data based on the association relation between the standard data and the data corresponding party.

9. A data governance system for performing the data governance method of any of claims 1 to 8, the system comprising a data governance unit and at least one data module containing inventory data; the data management unit is used for acquiring stock data in at least one data module, screening out key data in the stock data, carrying out standardized processing on the key data, determining a data standard, generating a rule based on the data standard, checking the stock data based on the rule, standardizing newly-added data based on the rule, and taking the checked stock data and the standardized newly-added data as standard data;

the data management unit comprises a basic data supporting layer, wherein the basic data supporting layer comprises a standard unit and a rule unit; the standard unit comprises the data standard; the rule unit contains the rule.

10. The data governance system of claim 9, wherein the data governance unit further comprises a panoramic data construction layer for constructing panoramic data based on an association of the standard data with a data counterpart.