CN115936296A - Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake - Google Patents
Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake Download PDFInfo
- Publication number
- CN115936296A CN115936296A CN202211641822.9A CN202211641822A CN115936296A CN 115936296 A CN115936296 A CN 115936296A CN 202211641822 A CN202211641822 A CN 202211641822A CN 115936296 A CN115936296 A CN 115936296A
- Authority
- CN
- China
- Prior art keywords
- data
- submodule
- module
- management
- production
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 168
- 238000007726 management method Methods 0.000 claims abstract description 100
- 238000012544 monitoring process Methods 0.000 claims abstract description 56
- 238000011161 development Methods 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims abstract description 24
- 238000004458 analytical method Methods 0.000 claims abstract description 15
- 238000013500 data storage Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims description 33
- 238000013523 data management Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000002776 aggregation Effects 0.000 claims description 7
- 238000004220 aggregation Methods 0.000 claims description 7
- 230000010354 integration Effects 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 7
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000000737 periodic effect Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 5
- 238000012550 audit Methods 0.000 claims description 3
- 238000013475 authorization Methods 0.000 claims description 3
- 239000008280 blood Substances 0.000 claims description 3
- 210000004369 blood Anatomy 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 238000007418 data mining Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 claims description 3
- 230000036541 health Effects 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 2
- 238000011156 evaluation Methods 0.000 claims description 2
- 238000004886 process control Methods 0.000 claims description 2
- 238000012558 master data management Methods 0.000 claims 1
- 238000012800 visualization Methods 0.000 abstract description 9
- 230000018109 developmental process Effects 0.000 description 30
- 230000014759 maintenance of location Effects 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000013439 planning Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention discloses a production and manufacturing data cockpit system of a discrete manufacturing enterprise based on an industrial internet big data lake, which is based on the upper application of the industrial internet big data lake and helps a manufacturing enterprise user to more quickly and intuitively know the whole theme domain information of the enterprise production and manufacturing plan, progress, quality, process and equipment by comprehensively assisting the visualization of workshop production monitoring, production line production monitoring visualization, and the visualization display of key index data of workshop-level, production line-level and equipment-level production and manufacturing processes of equipment monitoring management from data acquisition, data storage, data governance, data development and data service to data operation monitoring analysis.
Description
Technical Field
The invention belongs to the technical field of industrial internet big data lakes, and particularly relates to a production and manufacturing data cockpit system of a discrete manufacturing enterprise based on the industrial internet big data lake.
Background
A production and manufacturing data cockpit system of a discrete manufacturing enterprise based on an industrial internet big data lake is based on data aggregation and data processing capacity of the industrial internet big data lake, and is based on a big data cluster, a data warehouse multilayer planning scheme based on a subject library, an index system and a dimensional model design is completed, metadata management, main data management, data governance management, data standard management, data asset management and data safety management of various service system data under a production and manufacturing theme in the big data lake are completed in the process, upper layer data development, data operation monitoring and data service of the big data lake are supported to realize production and manufacturing index system development of the manufacturing enterprise, visual display of production and manufacturing data of the manufacturing enterprise, workshop and equipment based on the index system is supported, and a decision basis based on data drive of historical data, real-time data and the like is provided for production and manufacturing management of the enterprise.
The manufacturing data cockpit system needs to display a plurality of service domain data and analysis data of discrete manufacturing enterprise planning, production, quality, storage and the like, so that a large amount of real-time data in the manufacturing process of the enterprise and service data of various existing service systems in the manufacturing process need to be collected.
However, in the face of complex data conditions such as various data sources, complex data structures, dirty data, and unclear data semantics, the problems of low accuracy of data query results, slow data query speed, and the like occur in the production and manufacturing of the data cockpit system, and once the system index system is adjusted, a large amount of development needs to be performed for new indexes, so that the system adaptability is poor.
Disclosure of Invention
The embodiment of the invention provides a production and manufacturing data cockpit system of a discrete manufacturing enterprise based on an industrial internet big data lake, which aims to solve the problems that the production and manufacturing data cockpit system in the prior art is low in data query result accuracy, slow in data query speed and the like.
In order to solve the technical problem, the embodiment of the invention discloses the following technical scheme:
one aspect of the invention provides a manufacturing data cockpit system for a discrete manufacturing enterprise based on an industrial internet big data lake, which comprises an application unit, a data lake unit and a heterogeneous data storage unit, and is connected with an external business system; wherein:
the application unit comprises a workshop production monitoring module, a production line production monitoring module and an equipment health management module;
the data lake unit comprises a data integration management module, a data warehouse module, a data management module, a data security module, a data development module, a data service module and a data operation monitoring module; wherein:
the data integration management module comprises a data aggregation sub-module, a data processing sub-module, a data file/interface sub-module and an online filling sub-module, and is used for transmitting, loading, cleaning, converting and integrating cross-department data, and supporting custom scheduling and graphical monitoring;
the data warehouse module comprises a data pasting source sub-module, a public dimension sub-module, a detail data sub-module, a summarized data sub-module and a data application sub-module;
the data management module is used for processing, formatting and standardizing data;
the data security module is used for carrying out unified security management on data resources stored in different types, shielding details of different storage technologies and establishing a security control mechanism and a security control process of data; the data security module comprises a user authentication submodule, a data authority management submodule, a data security processing submodule, a data service security submodule and a log management submodule;
the data development module is used for constructing model services and applications of various scene data services, big data decision services and operation management analysis services;
the data service module is used for packaging the data asset calculation logic and providing the whole life cycle management of data API service registration, auditing, testing and release; the data service module comprises a service registration and release sub-module, a service gateway sub-module, a service authorization sub-module, a service calling and monitoring sub-module and a service full life cycle management sub-module;
the data operation monitoring module is used for monitoring data asset conditions, data use conditions and data platform operation conditions, constructing a uniform data operation monitoring view and realizing all-round monitoring of a data operation process; the data operation monitoring module comprises a platform monitoring submodule, a data processing process monitoring submodule and a service resource monitoring submodule;
the heterogeneous data storage unit comprises a relational database module, a semi-structured data module, a graph storage module, a distributed file system module, a time sequence database module and a service interface module.
Optionally, the data pasting source sub-module stores full detail data, wherein the full detail data is obtained by performing light cleaning, type conversion and incremental summation processing on production and manufacturing data subjected to data aggregation, third-party data interface and online filling, and comprises full service and log original data;
the detail data submodule is connected with the data pasting source submodule, a detail fact table is created according to the service object, the detail fact table is obtained by associating and combining data from a plurality of external service system sources, complete detailed historical data are stored, the detail fact layer comprises an affair fact table, a periodic snapshot fact table and an accumulated snapshot fact table, the affair fact table is used for describing a service process, tracking a measurement event of a space point or a time point, and storing original data; the periodic snapshot fact table is used for recording facts at regular and predictable time intervals; the accumulated snapshot fact table is used for expressing key step events between the beginning and the end of the process, covers the whole life cycle of the process and is provided with a plurality of date fields for recording key time points;
the public dimension submodule is based on dimension modeling and is used for providing statistical dimension data for data statistics of a data warehouse, and the dimension data comprises a date dimension table, a workshop dimension table, a production line dimension table and an equipment dimension table;
the summarized data submodule takes the analyzed subject object as a modeling drive, and constructs an index fact table based on the index requirement of the upper-layer production manufacturing data cockpit system for quickly responding to the query of a user;
the data application submodule stores statistical index data of the production and manufacturing data cockpit system, an index system of a production and manufacturing process of a discrete manufacturing enterprise is established according to data application requirements of the collected production and manufacturing data cockpit system, data are obtained from the data pasting source submodule, the detail data submodule and the summarized data submodule according to the calculation logic of each index item, corresponding indexes are calculated, and index calculation results are stored in a table of the data application submodule.
Optionally, the data governance module includes a metadata management submodule, a main data management submodule, a data standard management submodule, a data quality management submodule, a data asset management submodule and a data resource portal submodule; wherein:
the metadata management submodule is used for generating a metadata model, establishing a uniform metadata structure and uniformly managing and storing the service metadata, the technical metadata and the management metadata; scanning a database, a file and an interface to obtain metadata information; organizing the collected metadata in a directory mode, and maintaining the metadata; and, managing to audit the metadata quality based on the data standard; constructing a data map by utilizing metadata blood relationship analysis and metadata influence analysis, so that a user can quickly clear data resources;
the main data management submodule is used for providing full life cycle management of main data, and the full life cycle management comprises acquisition, creation, application, change, verification, examination and approval, validation, distribution and invalidation;
the data standard management submodule maintains standard data by using a standard dictionary and finishes the comparison between the data and the standard dictionary; the data comparison proportion is checked through comparison statistics;
the data quality management submodule is used for problem data feedback tracking processing, data quality processing, output quality report, quality metadata standard access and data tracing management;
and the data asset management submodule establishes the process control of the assets through data asset inventory, data asset registration and asset evaluation.
Optionally, the data development module comprises an offline development submodule, a real-time development submodule and an algorithm development submodule; wherein:
the off-line development submodule calls related services of the distributed computing subsystem according to the data of the data pasting source submodule, and a system-data bin, label data and application data are respectively formed after computing processing and conversion;
the real-time development submodule customizes a corresponding streaming data processing task according to real-time acquired data and calculates in real time;
the algorithm development comprises big data AI modeling, a one-stop data mining and machine learning platform is provided, and multi-environment, multi-cluster and multi-form model servitization and algorithm development scenes are supported.
Optionally, the external business system includes an ERP ordering system, an equipment access system, an EAM equipment full life cycle management system, an APS advanced scheduling system, a WMS material warehousing system, an MES production and manufacturing execution system, an MOM manufacturing management system, and a QIS quality information system.
According to the production and manufacturing data cockpit system of the discrete manufacturing enterprise based on the industrial internet big data lake, provided by the embodiment of the invention, the big data lake comprehensively assists the visualization of workshop production monitoring, production line production monitoring and the visualization of workshop level, production line production monitoring and the visualization of key index data of equipment level production and manufacturing processes in workshop level, production line level and equipment level production and manufacturing process from data acquisition, data storage, data management, data development and data service to data operation monitoring and analysis, and helps a manufacturing enterprise user to know the whole subject area information of enterprise production and manufacturing plan, progress, quality, process and equipment more quickly and intuitively.
Drawings
FIG. 1 is a schematic structural diagram of a manufacturing data cockpit system of a discrete manufacturing enterprise based on an industrial Internet big data lake according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data lake unit in fig. 1 according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
The production and manufacturing process of the discrete manufacturing enterprise mainly comprises the steps of receiving a production order, carrying out production scheduling, generating a production plan, finishing material receiving according to a production flow card, starting production processing execution, finishing line side warehouse material preparation according to the production progress, finishing production, checking product quality and finally warehousing products. In the process, a large amount of heterogeneous data is involved, including non-structural data such as real-time data of the internet of things equipment, production and manufacturing business system structured data, production monitoring videos, production daily documents and the like, and data from a third-party service platform and other platforms. The production and manufacturing business data mainly come from business information systems such as an ERP order system, an equipment access system, an EAM equipment full life cycle management system, an APS advanced scheduling system, a WMS material storage system, an MES production and manufacturing execution system, an MOM manufacturing management system, a QIS quality information system and the like. Meanwhile, different data storages are used by each business system, and comprise various heterogeneous data storages such as MySQL/Oracle and other relational databases, mongoDB/CouchDB and other document databases, redis/Memached and other key value storages, HBase/Cassandra wide column storages, neo4j/AllegroGraph graph storages, influxDB time sequence data storages, HDFS distributed file storages and the like. The large-scale heterogeneous production and manufacturing data relates to the businesses of different departments of an enterprise, the reusability of the data among the departments is poor, and the planning and control of the whole layer are lacked, so that the efficiency of data development and data sharing is low.
With the continuous emergence of industrial internet platforms, the number of connected industrial devices is continuously increased, the acquired large-scale heterogeneous big data needs to be stably stored, and meanwhile, on the basis of massive real-time data, big data processing and analysis and calculation in the production and manufacturing process are completed to support data results obtained after application layer use and processing, for example, real-time monitoring and prediction of the devices are completed in the production and manufacturing process. Therefore, the demand on the industrial internet big data lake is newly expanded, and the index demand and the data response of the cross-topic data are expressed in that the support on real-time data is required to be met, and the agile support capability of a system and the demands on the consistency, integrity and accuracy of the data when the business index data changes are also required to be met. In order to adapt to the requirement expansion, the embodiment of the invention provides the following big data lake system architecture, data acquisition is taken as an inlet of a big data lake, data management and data safety are the foundation of the big data lake, data development and data service mode sharing are realized under the support of data operation monitoring based on the design of a 5-layer data warehouse, and the redundancy is reduced, the multiplexing is increased, the user requirement is quickly responded, and the data driving service innovation is realized.
Fig. 1 is a schematic structural diagram of a manufacturing data cockpit system produced by a discrete manufacturing enterprise based on an industrial internet big data lake according to an embodiment of the present invention, as shown in fig. 1, the system includes an application unit 11, a data lake unit 12, and a heterogeneous data storage unit 13, and is connected to an external business system 14. Wherein:
the application unit 11 includes a workshop production monitoring module, a production line production monitoring module and an equipment health management module.
As shown in fig. 2, the data lake unit 12 includes a data integration management module 21, a data warehouse module 22, a data governance module 23, a data security module 24, a data development module 25, a data service module 26 and a data operation monitoring module 27. Wherein:
the data integration management module 21 comprises a data aggregation sub-module, a data processing sub-module, a data file/interface sub-module and an online filling sub-module, and is used for transmitting, loading, cleaning, converting and integrating cross-department data, and supporting custom scheduling and graphical monitoring. Unified scheduling and unified monitoring are realized, the visual requirement of operation and maintenance is met, and the operation and maintenance management work efficiency is improved.
The data warehouse module 22 includes a data posting source sub-module (data posting layer), a common dimension sub-module (common dimension layer), a detailed data sub-module (detailed data layer), a summarized data sub-module (summarized data layer), and a data application sub-module (data application layer).
In one embodiment disclosed by the invention, the data pasting layer is a source data layer, full detailed data obtained by performing incremental integration processing on slightly cleaned and type-converted production and manufacturing data subjected to data aggregation, third-party data interface and online filling are stored, the full detailed data comprises full service and log original data, the data pasting layer is a centralized storage place of the service data, the unstructured data is structurally processed and the same data is integrated, excessive cleaning processing is not performed on the service data, the original state of the data is kept as much as possible, the original data of an enterprise production and manufacturing domain is gathered to a big data lake platform, all service system data can be inquired in a big data lake, and preparation is made for providing data services for a detailed data layer, a summary data layer and a data application layer later.
The detail data layer is used for creating a detail fact table according to the business object on the basis of the data posting layer, and the detail fact table can be associated and combined by data from a plurality of business systems and is used for storing complete detailed historical data. The fine-grained fact layers are generally divided into three categories: the system comprises a transaction fact table, a periodic snapshot fact table and an accumulated snapshot fact table, wherein the transaction fact table is used for describing a business process, tracking a measurement event at a certain point in space or time, and storing most atomic data, also called an atomic fact table. The periodic snapshot fact table records facts at regular, predictable intervals. The cumulative snapshot fact table is used to represent the key step events between the beginning and the end of the process, covering the entire life cycle of the process, and typically has multiple date fields to record the key points in time. Meanwhile, the detail data layer carries out data standardization on data from each service information system such as an ERP ordering system, an equipment access system, an EAM equipment full life cycle management system, an APS advanced scheduling system, a WMS material storage system, an MES production and manufacturing execution system, an MOM manufacturing management system, QIS quality information and the like, unifies data with the same description object but with different values of each service system, such as a gender field, wherein some service systems are represented by 0 and 1 or Man and Women, and are uniformly mapped into standard approved names through data standard management. And meanwhile, the repeated data is deleted, and for the same business object maintained in a plurality of business systems, the retention rule is ensured, and only a unique data item is left.
The public dimension layer provides support for statistical dimension data for data statistics of the data warehouse based on a dimension modeling concept, dimension tables of an enterprise production and manufacturing process comprise a date dimension table, a workshop dimension table (workshop name, workshop ID, product quality and product type), a production line dimension table (production line name, production line ID, product quality and product type), an equipment dimension table (workshop name, workshop ID and equipment state) and the like, main data maintained by the main data management system are converted into a dimension table of a DIM layer, and uniqueness of dimensions in the enterprise data warehouse is guaranteed.
The summarized data layer takes the analyzed subject object as a modeling drive, an index fact table is constructed based on index requirements of an upper-layer production and manufacturing data cockpit system, and usually one table of the summarized data layer at least corresponds to one derived index. By gathering detailed data layer data, the query of a user is responded quickly, and taking an equipment operation table as an example, equipment operation data acquired by equipment at different time intervals every day is recorded in the table, and the data are gathered according to days to obtain the operation state of the equipment every day, so that a large number of repeated calculation processes in statistics of subsequent indexes according to months, seasons and years are reduced, and the calculation and query efficiency of the indexes is improved greatly.
The data application layer stores the individualized statistical index data of the production and manufacturing data cockpit system, and the index system of the production and manufacturing process of the discrete manufacturing enterprise is combed by collecting the data application requirements of the production and manufacturing data cockpit system. And aiming at the calculation logic of each index item, data are taken from the data pasting layer, the detail data layer and the summary data layer, corresponding indexes are calculated, and index calculation results are stored in a table of the data application layer.
And the data governance module 23 is used for processing, formatting and normalizing the data. Data governance is the process of handling, formatting and normalizing data, and is the core management means and management paradigm of data.
In one embodiment of the present disclosure, the data governance module 23 includes a metadata management submodule, a main data management submodule, a data standard management submodule, a data quality management submodule, a data asset management submodule, and a data resource portal submodule. Wherein:
the metadata management submodule is used for generating a metadata model, establishing a uniform metadata structure and uniformly managing and storing the service metadata, the technical metadata and the management metadata; scanning a database, a file and an interface to obtain metadata information; organizing the collected metadata in a directory mode, and maintaining the metadata; and, managing to audit the metadata quality based on the data standard; and constructing a data map by utilizing metadata blood relationship analysis and metadata influence analysis, so that a user can quickly clear data resources and know the data coming and going.
The main data is the most core data needing to be shared most in a plurality of business systems of the enterprise. The main data management submodule provides full life cycle management of main data acquisition, creation, application, change, verification, examination and approval, validation, distribution, invalidation and the like, realizes maintenance, retrieval, distribution and quality management of main data, unifies standard, normative flow, timely distribution and efficient sharing, and provides comprehensive, reliable, timely and accurate main data service for businesses.
Data standards are regulatory constraints that guarantee the consistency and accuracy of internal and external use and exchange of data. Data standard management is a series of activities that specify the formulation and implementation of data standards. The data standard management submodule maintains standard data by using a standard dictionary and finishes the comparison between the data and the standard dictionary; and finishing the check of the data by contrast statistics. The integrity, the effectiveness, the consistency and the normalization of the big data lake data are realized, and a standard basis is provided for data quality inspection and data safety management.
The data quality management is to realize tracking and processing of data asset quality problems, guarantee system data correctness, and support implementation control on key processes and important links influencing the informationized data quality so as to realize continuous improvement of the data quality of the big data lake. The data quality management submodule provides quality rule management, task editing management and task control operation, and functions of problem data feedback tracking processing, data quality processing, output quality report, quality metadata standard access, data tracing management and the like are completed through simple operation means such as wizard, visualization and the like, so that a complete data quality management closed loop is formed.
The data asset management is a platform for providing all-round inventory and management and control capability for organized data assets, the data asset management submodule establishes flow management and control of the assets through means of data asset inventory, data asset registration, asset assessment and the like, visual inventory, standardized management, intelligent analysis, procedural management and control of the assets and full life cycle management of the assets are achieved, a basis for sustainable management and development of the assets is provided, and the value of the data assets is better mined.
The data security module 24 is based on a data asset directory system, and is used for implementing unified security management on data resources stored in different types, shielding details of different storage technologies, and establishing a security management and control mechanism and a security management and control flow of data in combination with actual requirements.
The data security module 24 includes a user authentication sub-module, a data authority management sub-module, a data security processing sub-module, a data service security sub-module, and a log management sub-module.
The data development module 25 is a cloud IDE which is oriented to industrial data application and is constructed quickly, so as to support quick construction of a data service model and data application development, realize model service and application construction of various scene data services, big data decision services and operation management analysis services, improve user development efficiency, reduce development cost and accelerate user knowledge sharing.
In one embodiment of the invention, the data development module comprises an offline development submodule, a real-time development submodule and an algorithm development submodule. Wherein:
and the off-line development submodule calls related services of the distributed computing subsystem according to the data of the data pasting source submodule, and a system-data bin, label data and application data are respectively formed after computing processing and conversion.
And the real-time development submodule customizes a corresponding streaming data processing task according to the real-time acquired data and calculates in real time.
The algorithm development comprises big data AI modeling, a one-stop data mining and machine learning platform is provided, and multi-environment, multi-cluster and multi-form model servitization and algorithm development scenes are supported.
A data services module 26 for computing logical packaging of data assets and providing whole lifecycle management of data API service registration, auditing, testing, publishing. The data service module 26 includes a service registration issuing sub-module, a service gateway sub-module, a service authorization sub-module, a service calling and monitoring sub-module, and a service full life cycle management sub-module.
The data operation monitoring module 27 is used for monitoring data asset conditions, data use conditions and data platform operation conditions, and constructing a unified data operation monitoring view to realize all-around monitoring of a data operation process. The data operation monitoring module 27 includes a platform monitoring submodule, a data processing procedure monitoring submodule, and a service resource monitoring submodule.
The heterogeneous data storage unit 13 includes a relational database module, a semi-structured data module, a graph storage module, a distributed file system module, a time sequence database module, and a service interface module.
In one embodiment of the present disclosure, the external business system 14 includes an ERP ordering system, a device access system, an EAM device full life cycle management system, an APS advanced scheduling system, a WMS material warehousing system, an MES production manufacturing execution system, an MOM manufacturing management system, and a QIS quality information system.
The system provided by the embodiment of the invention is based on the upper-layer application of the industrial internet big data lake, and can help a manufacturing enterprise user to know the whole theme domain information of the enterprise production and manufacturing plan, progress, quality, process and equipment more quickly and intuitively from data acquisition, data storage, data management, data development, data service to data operation monitoring analysis, and the big data lake comprehensively assists the workshop production monitoring visualization, production line production monitoring visualization, and the visual display of the key index data in the workshop level, production line level and equipment level production and manufacturing processes of equipment monitoring management.
It will be understood that the above embodiments are merely exemplary embodiments taken to illustrate the principles of the present invention, which is not limited thereto. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit and substance of the invention, and these modifications and improvements are also considered to be within the scope of the invention.
Claims (5)
1. A manufacturing data cockpit system for a discrete manufacturing enterprise based on an industrial Internet big data lake is characterized by comprising an application unit, a data lake unit and a heterogeneous data storage unit, wherein the application unit, the data lake unit and the heterogeneous data storage unit are connected with an external business system; wherein:
the application unit comprises a workshop production monitoring module, a production line production monitoring module and an equipment health management module;
the data lake unit comprises a data integration management module, a data warehouse module, a data management module, a data security module, a data development module, a data service module and a data operation monitoring module; wherein:
the data integration management module comprises a data aggregation sub-module, a data processing sub-module, a data file/interface sub-module and an online filling sub-module, and is used for transmitting, loading, cleaning, converting and integrating cross-department data, and supporting custom scheduling and graphical monitoring;
the data warehouse module comprises a data paste source submodule, a public dimension submodule, a detail data submodule, a summarized data submodule and a data application submodule;
the data management module is used for processing, formatting and standardizing data;
the data security module is used for carrying out unified security management on data resources stored in different types, shielding details of different storage technologies and establishing a security control mechanism and a security control process of data; the data security module comprises a user authentication submodule, a data authority management submodule, a data security processing submodule, a data service security submodule and a log management submodule;
the data development module is used for constructing model services and applications of various scene data services, big data decision services and operation management analysis services;
the data service module is used for computing and logically packaging data assets and providing whole life cycle management of data API service registration, auditing, testing and release; the data service module comprises a service registration and release submodule, a service gateway submodule, a service authorization submodule, a service calling and monitoring submodule and a service full life cycle management submodule;
the data operation monitoring module is used for monitoring data asset conditions, data use conditions and data platform operation conditions, constructing a uniform data operation monitoring view and realizing all-round monitoring of a data operation process; the data operation monitoring module comprises a platform monitoring submodule, a data processing process monitoring submodule and a service resource monitoring submodule;
the heterogeneous data storage unit comprises a relational database module, a semi-structured data module, a graphic storage module, a distributed file system module, a time sequence database module and a service interface module.
2. The system of claim 1,
the data pasting source submodule stores full detailed data, the full detailed data is obtained after slight cleaning and type conversion are carried out on production and manufacturing data which are gathered, connected with a third-party data interface and filled on line, and then incremental aggregation processing is carried out on the production and manufacturing data, and the full detailed data comprises full business and log original data;
the detail data submodule is connected with the data paste source submodule, a detail fact table is created according to the service object, the detail fact table is obtained by carrying out correlation and combination on data from a plurality of external service systems, complete detailed historical data are stored, and the detail fact layer comprises a transaction fact table, a periodic snapshot fact table and an accumulated snapshot fact table; the transaction fact table is used for describing a business process, tracking a measurement event of a space point or a time point and storing original data; the periodic snapshot fact table is used for recording facts at regular and predictable time intervals; the accumulated snapshot fact table is used for expressing key step events between the beginning and the end of the process, covers the whole life cycle of the process and is provided with a plurality of date fields for recording key time points;
the public dimension submodule is based on dimension modeling and is used for providing statistical dimension data for data statistics of a data warehouse, and the dimension data comprises a date dimension table, a workshop dimension table, a production line dimension table and an equipment dimension table;
the summarized data submodule takes the analyzed subject object as a modeling drive, and constructs an index fact table based on the index requirement of the upper-layer production manufacturing data cockpit system for quickly responding to the query of a user;
the data application submodule stores statistical index data of the production and manufacturing data cockpit system, an index system of a production and manufacturing process of a discrete manufacturing enterprise is established according to data application requirements of the collected production and manufacturing data cockpit system, data are obtained from the data pasting source submodule, the detail data submodule and the summarized data submodule according to the calculation logic of each index item, corresponding indexes are calculated, and index calculation results are stored in a table of the data application submodule.
3. The system of claim 1, wherein the data governance module comprises a metadata management submodule, a master data management submodule, a data standard management submodule, a data quality management submodule, a data asset management submodule, and a data resource portal submodule; wherein:
the metadata management submodule is used for generating a metadata model, establishing a uniform metadata structure and uniformly managing and storing the service metadata, the technical metadata and the management metadata; scanning a database, a file and an interface to obtain metadata information; organizing the collected metadata in a directory mode, and maintaining the metadata; and, managing to audit the metadata quality based on the data standard; constructing a data map by utilizing metadata blood relationship analysis and metadata influence analysis, so that a user can quickly clear data resources;
the main data management submodule is used for providing full life cycle management of main data, and the full life cycle management comprises acquisition, creation, application, change, verification, examination and approval, taking effect, distribution and invalidation;
the data standard management submodule maintains standard data by using a standard dictionary and finishes the comparison between the data and the standard dictionary; the data comparison proportion is checked through comparison statistics;
the data quality management submodule is used for problem data feedback tracking processing, data quality processing, output quality report, quality metadata standard access and data tracing management;
and the data asset management submodule establishes the process control of the assets through data asset inventory, data asset registration and asset evaluation.
4. The system of claim 1, wherein the data development module comprises an offline development sub-module, a real-time development sub-module, and an algorithm development sub-module; wherein:
the off-line development submodule calls related services of the distributed computing subsystem according to the data of the data pasting source submodule, and a system-data bin, label data and application data are respectively formed after computing processing and conversion;
the real-time development submodule customizes a corresponding streaming data processing task according to real-time acquired data and calculates in real time;
the algorithm development comprises big data AI modeling, a one-stop data mining and machine learning platform is provided, and multi-environment, multi-cluster and multi-form model servitization and algorithm development scenes are supported.
5. The system of claim 1, wherein the external business systems include ERP ordering systems, equipment access systems, EAM equipment full lifecycle management systems, APS advanced scheduling systems, WMS material warehousing systems, MES manufacturing execution systems, MOM manufacturing management systems, and QIS quality information systems.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211641822.9A CN115936296A (en) | 2022-12-20 | 2022-12-20 | Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211641822.9A CN115936296A (en) | 2022-12-20 | 2022-12-20 | Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115936296A true CN115936296A (en) | 2023-04-07 |
Family
ID=86652565
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211641822.9A Pending CN115936296A (en) | 2022-12-20 | 2022-12-20 | Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115936296A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501788A (en) * | 2023-06-20 | 2023-07-28 | 东华软件智能科技有限公司 | Storehouse lake integrated data management and control platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460236A (en) * | 2020-04-26 | 2020-07-28 | 天津七一二通信广播股份有限公司 | Big data acquisition administers quick retrieval system based on data lake |
| CN112241402A (en) * | 2020-10-16 | 2021-01-19 | 中国民用航空华东地区空中交通管理局 | Empty pipe data supply chain system and data management method |
| CN113468159A (en) * | 2021-07-19 | 2021-10-01 | 广东电网有限责任公司 | Data application full-link management and control method and system |
| CN114490886A (en) * | 2021-12-29 | 2022-05-13 | 北京航天智造科技发展有限公司 | Industrial operation system data lake construction method based on data warehouse |
| CN114880405A (en) * | 2022-03-31 | 2022-08-09 | 华能信息技术有限公司 | Data lake-based data processing method and system |
-
2022
- 2022-12-20 CN CN202211641822.9A patent/CN115936296A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460236A (en) * | 2020-04-26 | 2020-07-28 | 天津七一二通信广播股份有限公司 | Big data acquisition administers quick retrieval system based on data lake |
| CN112241402A (en) * | 2020-10-16 | 2021-01-19 | 中国民用航空华东地区空中交通管理局 | Empty pipe data supply chain system and data management method |
| CN113468159A (en) * | 2021-07-19 | 2021-10-01 | 广东电网有限责任公司 | Data application full-link management and control method and system |
| CN114490886A (en) * | 2021-12-29 | 2022-05-13 | 北京航天智造科技发展有限公司 | Industrial operation system data lake construction method based on data warehouse |
| CN114880405A (en) * | 2022-03-31 | 2022-08-09 | 华能信息技术有限公司 | Data lake-based data processing method and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501788A (en) * | 2023-06-20 | 2023-07-28 | 东华软件智能科技有限公司 | Storehouse lake integrated data management and control platform |
| CN116501788B (en) * | 2023-06-20 | 2023-10-27 | 东华软件智能科技有限公司 | Storehouse lake integrated data management and control platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114490886A (en) | Industrial operation system data lake construction method based on data warehouse | |
| CN112181960B (en) | Intelligent operation and maintenance framework system based on AIOps | |
| CN115934680B (en) | One-stop big data analysis processing system | |
| CN112527886A (en) | Data warehouse system based on urban brain | |
| CN111538720B (en) | Method and system for cleaning basic data of power industry | |
| CN113723822A (en) | Power supply service data management system | |
| CN115936296A (en) | Production and manufacturing data cockpit system of discrete manufacturing enterprise based on industrial internet big data lake | |
| Kun et al. | Application of big data technology in scientific research data management of military enterprises | |
| CN113688396A (en) | Automobile information safety risk assessment automation system | |
| Hu et al. | E‐maintenance platform design for public infrastructure maintenance based on IFC ontology and Semantic Web services | |
| CN116307570A (en) | Subway vehicle electronic record system based on block chain technology | |
| CN114298550A (en) | Method for treating cement production operation data | |
| US8688499B1 (en) | System and method for generating business process models from mapped time sequenced operational and transaction data | |
| CN116450620B (en) | Database design method and system for multi-source multi-domain space-time reference data | |
| Li et al. | Industrial internet platforms: Applications in BF ironmaking | |
| CN117744914A (en) | BIM-based construction digital management method, platform and computer equipment | |
| CN115016902B (en) | Industrial flow digital management system and method | |
| CN115496337A (en) | Data system for supporting brain of enterprise | |
| Tytenko et al. | Software and information support for business analysis in enterprise management | |
| CN112784129A (en) | Pump station equipment operation and maintenance data supervision platform | |
| CN111435466A (en) | Integrated machine room operation and maintenance management system | |
| Weijie et al. | Research on the big data construction of equipment support | |
| Wu et al. | Application of Real-Time Big Data Technology in Complex Power Engineering Management | |
| CN116975043B (en) | Data real-time transmission construction method based on stream frame | |
| Wu et al. | Research on decision support system of automobile service based on distributed data warehouse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230407 |