CN113138963A - Man-machine interaction system of large industrial data platform for aluminum/copper plate strips - Google Patents
Man-machine interaction system of large industrial data platform for aluminum/copper plate strips Download PDFInfo
- Publication number
- CN113138963A CN113138963A CN202110470285.5A CN202110470285A CN113138963A CN 113138963 A CN113138963 A CN 113138963A CN 202110470285 A CN202110470285 A CN 202110470285A CN 113138963 A CN113138963 A CN 113138963A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- unit
- user
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 title claims abstract description 42
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 title claims abstract description 42
- 229910052782 aluminium Inorganic materials 0.000 title claims abstract description 42
- 229910052802 copper Inorganic materials 0.000 title claims abstract description 42
- 239000010949 copper Substances 0.000 title claims abstract description 42
- 230000003993 interaction Effects 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 claims abstract description 62
- 230000002159 abnormal effect Effects 0.000 claims abstract description 59
- 238000004140 cleaning Methods 0.000 claims abstract description 48
- 230000008569 process Effects 0.000 claims abstract description 33
- 230000008676 import Effects 0.000 claims abstract description 30
- 238000007405 data analysis Methods 0.000 claims abstract description 26
- 238000004519 manufacturing process Methods 0.000 claims abstract description 15
- 238000012544 monitoring process Methods 0.000 claims description 43
- 238000012545 processing Methods 0.000 claims description 38
- 238000004458 analytical method Methods 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 8
- 238000004148 unit process Methods 0.000 claims description 7
- 238000012800 visualization Methods 0.000 claims description 7
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 6
- 238000013079 data visualisation Methods 0.000 claims description 5
- 230000001419 dependent effect Effects 0.000 claims description 5
- 238000013461 design Methods 0.000 claims description 4
- 238000005266 casting Methods 0.000 claims description 3
- 238000005097 cold rolling Methods 0.000 claims description 3
- 238000005098 hot rolling Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 11
- 230000000007 visual effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- WPPDFTBPZNZZRP-UHFFFAOYSA-N aluminum copper Chemical compound [Al].[Cu] WPPDFTBPZNZZRP-UHFFFAOYSA-N 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013450 outlier detection Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/156—Query results presentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
- G06F16/287—Visualization; Browsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/358—Browsing; Visualisation therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Library & Information Science (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to an aluminum/copper plate strip industrial big data platform man-machine interaction system, which comprises a data import module, a data analysis module and a data analysis module, wherein the data import module is used for importing the unstructured data collected on site into a big data platform in a multi-thread high-speed manner; the data query module is used for displaying the multi-source structured data of the whole process of aluminum/copper plate strip production in a time-space layered mode and displaying the unstructured data in a graph mode; the data cleaning module is used for constructing a human-computer interaction interface suitable for the aluminum/copper plate strip data cleaning system, displaying statistical information such as repeated values, abnormal values, single values and missing values of a selected data source and designing cleaning operation logic of a user; and the data analysis module is used for analyzing and displaying data facing the whole aluminum/copper plate strip production process, designing different human-computer interaction systems according to different data analysis methods, and providing different interfaces corresponding to the back-end data analysis function. The invention can effectively solve the problems of scattered production data, poor link hooking performance, low data query speed and the like.
Description
Technical Field
The invention relates to the technical field of big data and data visualization, in particular to a man-machine interaction system of an aluminum/copper plate strip industrial big data platform.
Background
The high-dimensional dynamic multi-space-time scale aluminum/copper strip industrial big data visual analysis man-machine interaction technology is mainly based on a big data platform to provide visual page interaction operation for users, is an important implementation link for CPS-oriented aluminum/copper strip production full-flow big data acquisition and platform construction, is specifically represented as the design and implementation of a big data platform software system, and a popular and general aluminum/copper strip industrial big data processing software system is not found at present. Therefore, the design and realization of the aluminum/copper strip industrial big data processing software system by combining the construction requirements of the big data platform and adopting the visual analysis interaction technology are of profound significance.
With the fusion and breakthrough of new generation big data information technologies such as distributed storage, distributed computation, data mining analysis and the like, the processing of large-batch data becomes simpler and simpler, and data visualization plays an increasingly important role in the whole big data ecology. The reasonable display and man-machine interaction method is particularly important for processing high-dimensional multi-space-time data generated in the production process of the aluminum-copper plate strip. Meanwhile, for high-noise industrial field data, a corresponding visualization technology is also needed for cleaning and analyzing to connect a user and a system platform, so that a reasonable visualization technology is extremely important for the whole large data platform in the aluminum-copper strip industry.
The original data source usually has a large amount of expired data and invalid data, and if the data and the valid data are imported into the database together when the data is imported, time and labor are wasted, and a lot of unnecessary troubles are brought to a user in subsequent data use. Meanwhile, because an integrated platform is not provided, the association between data of different processes and different devices cannot be effectively utilized, and a tool is lacked to provide visual data display for process personnel; and secondly, the historical data contains a large number of missing values, abnormal values, repeated values and the like, so that the data information value is not high, and therefore, a cleaning and analyzing tool which is intuitive and easy to operate needs to be provided for workers, so that the field data can exert higher value.
Disclosure of Invention
Aiming at the problems, the invention aims to provide a man-machine interaction system of an aluminum/copper plate strip industrial large data platform, which can effectively solve the problems of production data dispersion, poor hooking performance, low data query speed and the like.
The technical scheme adopted by the invention is as follows:
the man-machine interaction system of the aluminum/copper plate strip industrial big data platform comprises a data import module, a data query module, a data cleaning module and a data analysis module;
the data import module comprises a file reading unit, a data uploading unit and a display unit and is used for importing the unstructured data collected on site into the big data platform in a multithreading high-speed manner;
the data query module comprises a product number and time retrieval module, an equipment number and time retrieval module and a process path and time retrieval module, and is used for displaying multi-source structured data of the whole process of aluminum/copper plate strip production in a time-space layered mode and displaying non-structured data in a graph mode;
the data cleaning module comprises a data set selection module, an original data state display module and a cleaning operation module, and is used for constructing a human-computer interaction interface suitable for the aluminum/copper plate strip data cleaning system, displaying statistical information such as a repeated value, an abnormal value, a single value and a missing value of a selected data source and designing a user cleaning operation logic;
the data analysis module comprises a classification module, a clustering module and a regression module, is used for analyzing and displaying data of the whole aluminum/copper plate strip production process, designs different human-computer interaction systems according to different data analysis methods, and provides different interfaces corresponding to the back-end data analysis function.
Further, the file reading unit is configured to read a locally stored IBA file and provide the file name for a user to select, and provide the file for the data uploading unit before importing so as to perform an importing operation;
the data uploading unit is used for performing data import from local data to the big data platform and feeding back a main module of a result; the method comprises the steps of obtaining file data of a file reading unit, packaging, compressing and transmitting the file, decompressing and storing the file in an HDFS (Hadoop distributed File System) at a server side, returning a real-time state through Websocket communication and providing the real-time state for a display unit;
the display unit is used for providing selectable files and executable operations for a user; including time selection, file path selection, and import schedule display.
Furthermore, the product number and time retrieval module indexes all the volume number information of the current time period according to the time period and then queries specific volume number data;
the equipment number and time retrieval module is used for classifying and displaying data generated in different time periods on different equipment according to equipment division data;
the 'process path + time' retrieval module is used for displaying products produced in different processes at different times according to processes such as casting, hot rolling and cold rolling as a primary index, and displaying corresponding L2-L3 data and IBA data after a specific product is selected.
Further, the data set selection module is used for providing selectable data for a user, and after the user selection is obtained, providing the selectable data for the single value detection unit, the repeated value detection unit, the abnormal value monitoring unit and the missing value monitoring unit respectively; the user selection comprises a data selection unit and an abnormal value judgment unit;
the original data state display module comprises a single value monitoring unit, a repeated value monitoring unit, a missing value monitoring unit and an abnormal value monitoring unit; the system is used for providing a state display function before cleaning of the original data, visually embodying the basic states of a single value, a repeated value, an abnormal value and a missing value of the original data on a man-machine interaction page, and providing a basis for data cleaning operation; the data selected by the user and provided by the data set selection module is acquired and then processed, and the data is provided for the data cleaning module;
the cleaning operation module comprises a missing value processing unit, an abnormal value processing unit and a default processing unit; the field names processed by the original data state display module are acquired and provided for a user to perform cleaning operation selection, the default processing function and the user-defined file name function for repeated values and single values are provided mainly for missing values and abnormal values, and the user can conveniently search.
Furthermore, the data selection unit is used for selecting a data source, providing a locally downloaded source data file for a user to select, and supporting the user to specify a file path;
the abnormal value judgment unit is used for selecting an abnormal value judgment method, is mainly used for judging an abnormal value during cleaning, and comprises three methods of a box diagram, a Lauda criterion and an absolute median difference.
Further, the single value monitoring unit is used for monitoring and displaying field names with a row of repetition values exceeding 90% in the data;
the repetition value monitoring unit is used for monitoring data with two completely repeated upper and lower lines in the data, and only the repeated line number and the repeated volume number are displayed because the field data generally has more repeated values and large data volume;
the missing value monitoring unit is used for monitoring missing and meaningless data in the data; displaying the number of the missing values, the specific fields where the missing values are located, the missing proportion of the corresponding fields and other statistical information;
the abnormal value monitoring unit is used for judging abnormal values in the monitoring data according to the abnormal values selected by the user, and displaying the field names of the abnormal values and the positions of the corresponding abnormal values;
further, the missing value processing unit processes the missing value detected in the previous module; providing four modes of deletion, filling median, filling mode and filling average for the user to select;
the abnormal value processing unit processes the abnormal value detected in the previous module; five modes of deletion, no processing, filling median, filling mode and filling average are provided for the user to select;
the default processing unit processes the repeated value and the single value, and has little meaning on data analysis and data visualization, so that the data is deleted by default when the back end is cleaned.
Further, the classification module, the clustering module and the regression module respectively correspond to different data characteristics and different process backgrounds; aiming at the characteristic, a man-machine interaction system aiming at different interfaces of different methods is constructed, wherein each module comprises the following specific units:
the data selection unit is used for providing input parameter settings of algorithms such as data source selection, independent variable selection, dependent variable selection, parameter setting and the like for a user at the front end of the classification and regression algorithm and supporting a default transfer mode; providing data set selection, column selection and algorithm parameter setting for the front end of the clustering method;
the data processing unit is used for transmitting the data to the java back end after the data selection unit obtains the user input, the back end transmits the data to the Python program for analysis, and the analysis result is generated and then transmitted to the analysis result visualization unit through the java end;
and the analysis result visualization unit is used for displaying the returned result of the algorithm processing unit to the user in a form and graphic mode so as to visually display the analysis result.
Compared with the prior art, the invention has the following beneficial effects:
1. the data import module accelerates the data import speed by using a compression and decompression mode, and simultaneously ensures that the display module can acquire the data import progress in real time and can capture and report the data import progress to a user in time when the import is abnormally interrupted by using WebSocket full duplex communication, thereby greatly improving the robustness of the data import system;
2. the data query module is designed with three different query modes, so that detailed display of multi-source heterogeneous data of aluminum \ copper plate strips is realized, and a combined query mode of site structured data and unstructured data is provided;
3. the data cleaning module is designed with a man-machine interaction system integrating selection, display and cleaning, so that a user can quickly clean a set of industrial data for analysis under the condition of minimum operation, and visual display and cleaning operation selection can be provided for the user.
Drawings
FIG. 1 is a schematic diagram of the module composition of the man-machine interaction system of the large data platform in the aluminum/copper plate strip industry of the present invention;
FIG. 2 is a schematic diagram of a data cleaning module of the large data platform man-machine interaction system for aluminum/copper plate strip industry according to the present invention;
FIG. 3 is a schematic view of a data cleaning operation flow of the large data platform man-machine interaction system for aluminum/copper plate strip industry according to the present invention;
4-6 are schematic diagrams of the data import module system interface of the large data platform man-machine interaction system of the aluminum/copper plate strip industry of the present invention;
FIGS. 7-8 are schematic diagrams of the interface of the data query module of the coil number index of the large data platform man-machine interactive system for aluminum/copper plate strip industry according to the present invention;
FIG. 9 is a schematic view of an equipment index data query module interface of the large data platform man-machine interactive system for aluminum/copper plate strip industry of the present invention;
FIGS. 10-11 are process index data query module interfaces of the large data platform man-machine interactive system for aluminum/copper plate strip industry of the present invention;
FIG. 12 is a schematic diagram of the interface of the data cleaning system of the large data platform man-machine interactive system for the aluminum/copper plate strip industry of the present invention;
fig. 13 is a schematic interface diagram of a data analysis system of the man-machine interaction system of the large data platform in the aluminum/copper plate strip industry.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
The invention provides a man-machine interaction system of an aluminum/copper plate strip industrial large data platform, which comprises a data import module, a data query module, a data cleaning module and a data analysis module as shown in figure 1.
The data import module comprises a file reading unit, a data uploading unit and a display unit, and is used for importing the unstructured data collected on site into the big data platform in a multi-thread high-speed manner; as shown in the figures 4-6 of the drawings,
the file reading unit is used for reading the locally stored IBA file and providing the file name to the display unit, and simultaneously providing the file to the data transmission unit for the importing operation before the importing operation, so as to provide help for the selection of the file before the importing operation and the reading of the data during the importing operation. The specific functions include: when the user appoints time, the unit returns all file names in the time period according to the time appointed by the user, and when the user selects the file names, the module sequentially provides the files to the data uploading unit according to the file names to prepare for importing. And simultaneously, file path selection and batch selection functions are provided for the user, and simultaneously, the files with more user selection times are ranked in front when being displayed by providing a memory function. The interface is shown in fig. 4, in which a user first designates time and a path for storing a file, transmits the time and the path to a file reading unit through a display unit, reads the file, and returns the read file to the display unit, as shown in fig. 5, to provide options for the user to select the file.
The data uploading unit is used for performing data import of local data to the big data platform and feeding back a main module of a result. The method comprises the steps of obtaining file data of a file reading unit, packing and compressing a plurality of IBA files, then transmitting the IBA files to a server end at a high speed by multithreading, and decompressing and storing the IBA files in an HDFS (Hadoop distributed File System) by using a C # program at the server end. Meanwhile, the java side provides real-time return of the import state with the WebSocket communication of the front end during the whole import process, and timely prompts a user to ensure that the import work can be normally carried out when the import is blocked.
The display unit is a bridge connecting the data uploading unit, the file reading unit and the user and is used for providing selectable files and executable operations for the user. The method comprises the steps of providing a user with time selection and file path selection popup as shown in fig. 6, obtaining an import progress returned by a data transmission unit during import to display a progress bar as shown in fig. 6, and capturing abnormality and prompting the user by the popup when the import is abnormal.
The data query module comprises a product number and time retrieval module, an equipment number and time retrieval module and a process path and time retrieval module, and is used for displaying multi-source structured data of the whole process of aluminum/copper plate strip production in a time-space layered mode and displaying non-structured data in a graph mode;
the product number and time retrieval module firstly indexes all the volume number information in the current time period according to the time period as shown in fig. 7, and the volume number can be selected by double-clicking; after the user specifies the volume number, the user jumps to the data display page shown in fig. 8 to display that the current volume number passes through all the devices, and finally, the user clicks one device to display specific data; the indexing method is mainly used for displaying the data of the volume number and the time period of the equipment, and is used for field workers to index the equipment data according to products, and simultaneously, equipment joint check is supported, and all equipment which passes through the equipment can be quickly searched when the produced products have problems.
The "device number + time" retrieval module, as shown in fig. 9, performs classified display on data generated in different time periods on different devices according to device division data; and requesting data to a background after the field and time are selected after the equipment is specified, displaying the production information of the current field of the current equipment in the time, and simultaneously supporting Chinese and English field switching.
The process path and time retrieval module is used for displaying products produced by different processes at different times according to the processes of casting, hot rolling, cold rolling and the like as a primary index; firstly, as shown in fig. 10, a user selects a process and time to display all product information produced by the current process in the current time period, including a production daily report and a quality inspection report; when a specific product is selected, the corresponding L2-L3 structured data and the IBA unstructured data as shown in FIG. 11 can be displayed, and the export support function can check the data click export button in the table to automatically export the data.
The data cleaning module comprises a data set selection module, an original data state display module and a cleaning operation module, a man-machine interaction interface suitable for the aluminum/copper plate strip data cleaning system is constructed, statistical information such as a repeated value, an abnormal value, a single value and a missing value of a selected data source is displayed, and user cleaning operation is provided; as shown in figure 12 of the drawings,
the data set selection module is mainly used for providing selectable data for a user, acquiring user selection and then respectively providing the data to the single value detection unit, the repeated value detection unit, the abnormal value monitoring unit and the missing value monitoring unit; the device comprises a data selection unit and an abnormal value judgment unit.
The data selection unit, embodied mainly as fig. 12 selecting the folder part, will browse all files ending with the names xls and csv in the user's computer. Namely, the locally downloaded source data file is provided to be selected by the user, and simultaneously, the user is supported to specify the file path.
The abnormal value judging unit provides options for a user to select a specific abnormal value judging method through an interface, and the method comprises three methods, namely a boxed graph, a Lauda criterion and an absolute median difference; respectively corresponding to different processes of the rear-end abnormal value monitoring unit.
The original data state display module comprises a single value monitoring unit, a repeated value monitoring unit, a missing value monitoring unit and an abnormal value monitoring unit; providing a state display function before cleaning of original data; the method comprises the steps of receiving data specified by a user of a data set selection module, and visually representing the basic states of a single value, a repeated value, an abnormal value and a missing value of original data on a man-machine interaction page after processing so as to provide data for a cleaning operation module, wherein the middleware is used for connecting the data set selection module and the cleaning operation module;
the single value monitoring unit is shown in the single value display column of fig. 12, and monitors field names with a row of repetition values exceeding 90% in the data through all data points of each feature of the cycle data, and performs statistical display.
The repeated value monitoring unit is embodied in the repeated value display column of fig. 12, and is mainly used for monitoring completely repeated data in upper and lower rows in the data, and only the repeated row number and the repeated volume number are displayed because the repeated value in the field data is generally more and the data volume is large.
The missing value monitoring unit is embodied in the missing value display column of fig. 12, and mainly monitors missing and meaningless data in the data. The number of the missing values and the specific field where the missing values are located, and the missing proportion of the corresponding field and other statistical information are mainly displayed.
The abnormal value monitoring unit is embodied in the abnormal value display column of fig. 12, and mainly displays the field name of the abnormal value and the position of the corresponding abnormal value according to the outlier detection method monitoring data selected by the user in the abnormal value judgment unit of the data set selection module.
The cleaning operation module is mainly embodied in front and back end programs corresponding to the functional area in fig. 12, and includes a missing value processing unit, an abnormal value processing unit and a default processing unit; the field names processed by the original state display module are mainly acquired and provided for a user to perform cleaning operation selection aiming at missing values and abnormal values, and the rest detected abnormal data are deleted by a default processing unit. Meanwhile, the user-defined file name function is provided, so that the subsequent search of the user is facilitated.
The missing value processing unit is embodied in the missing value processing part shown in fig. 12, and processes the missing value detected by the missing value unit in the original state display module; and acquiring fields generated by a missing value monitoring unit in an original state display module, providing four modes of deletion, median filling, mode filling and average filling for a user to select, and after the fields and corresponding selections are acquired by the user, submitting the fields and corresponding selections to a back end in a Map form in batches for processing.
The abnormal value processing unit is embodied in the abnormal value processing section shown in fig. 12, and processes the abnormal value detected in the previous block. And acquiring fields generated by an abnormal value monitoring unit in the original state display module, and providing five modes of deletion, no processing, median filling, mode filling and average filling for a user to select.
The default processing unit processes the repeated values and the single values, and the meaning of the default processing unit on data analysis and data visualization is not great, so that the default processing unit directly deletes the data by default when the back end is cleaned, wherein the data belongs to noise generated on site or artificial misoperation data.
The data analysis module comprises a classification module, a clustering module and a regression module, and is used for analyzing and displaying data of the whole aluminum/copper plate strip production process and designing different human-computer interaction systems according to different data analysis methods; providing different interfaces corresponding to the backend data analysis function; reasonably displaying the analysis result; where the effect of the classification module is shown in figure 13,
the data analysis of the aluminum/copper plate strip mainly comprises three methods of classification, clustering and regression, which respectively correspond to different data characteristics and different process backgrounds; aiming at the characteristic, a man-machine interaction system aiming at different interfaces of different methods is constructed, and comprises the following specific units:
data cells are selected, including data selection, data proportion, independent variable selection, dependent variable selection, and parameter setting as shown in fig. 13. And the input parameter setting of algorithms such as data source selection, independent variable selection, dependent variable selection, parameter setting and the like is provided for a user for the front end of the classification and regression algorithm, and meanwhile, a default transfer mode is supported. And providing data set selection, column selection and algorithm parameter setting for the clustering method front end.
And the data processing unit comprises three data analysis methods of classification, clustering and regression, when the selected data unit obtains user input, the selected data unit transmits the input to the java back end, the back end transmits the original data information and the user specified method to the Python program together, the data analysis is carried out by using skleran, an analysis result is generated, and then the analysis result is transmitted to the analysis result visual unit through the java back end.
And the analysis result visualization unit is mainly used for displaying the returned result of the algorithm processing unit to the user in a form and graph mode, and visually displaying the analysis result, namely the result formed by random forest classification and a confusion matrix at the bottom of the graph 13.
The use of the method mainly comprises four parts, namely data import, data query, data cleaning and data analysis, wherein a data import stage is firstly as shown in FIG. 4, a time period in which data to be imported is located is selected, after the data is selected, clicking is determined as shown in FIG. 5, all files in the current time period are displayed, after the data to be imported is selected, a start button is clicked as shown in FIG. 6, a file selection button is hidden, and the name and the import progress of the imported file are displayed at the same time;
the data query is divided into a product number + time query, a device number + time query and a process path + time query, wherein the product number + time query is firstly shown in FIG. 7, a volume number selection button is clicked after the time is selected, the volume number at the current time is displayed in a volume number list block, the volume number is placed in a current selection frame by double clicking, an interface shown in FIG. 8 is jumped by clicking a volume number confirmation button after the volume number is selected, a device corresponding to a device flow chart is lightened after the volume number is selected by clicking, and bottom data can be displayed after the device is clicked; the query of "equipment number + time" is shown in fig. 9, and data can be displayed at the bottom by first selecting a machine, selecting information types, then selecting start and end time, and finally clicking the query; the 'process path + time' query mode is as shown in fig. 10, firstly, the process and the starting and ending time are designated, the quality inspection information daily report and the production information are displayed at the bottom after the query is clicked, a certain line is clicked to check a check box, IBA query or process data query L2-L3 or IBA data can be clicked after one volume is selected, and as shown in fig. 11, the IBA query or the process data query L2-L3 or the IBA data can be clicked after a plurality of volumes are selected and exported to the local;
data cleaning is as shown in fig. 12, firstly, clicking browsing to find local data to be cleaned, then clicking an abnormal value judgment method by a drop-down box, displaying a single value, a missing value, an abnormal value and repeated value data after clicking confirmation, finally, dropping a designated field and corresponding operation of the drop-down box in a bottom operation area, clicking a cleaning start button after an input box defines a cleaned file name to complete the operation, and finally popping up to prompt that cleaning operation is completed after cleaning operation is completed;
the data analysis function is as shown in fig. 13, first, a data source is designated in a data selection part drop-down box, data is displayed in a data display box, then training set proportion and dependent variable independent variable setting are sequentially performed, after model parameters are set in a parameter setting part, an analysis button is clicked to perform analysis, and algorithm results and models are displayed in a bottom return box in a graph form.
The above-mentioned embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solution of the present invention by those skilled in the art should fall within the protection scope defined by the claims of the present invention without departing from the spirit of the present invention.
Claims (8)
1. The man-machine interaction system of the industrial big data platform of the aluminum/copper plate strip is characterized in that: the human-computer interaction system comprises a data import module, a data query module, a data cleaning module and a data analysis module;
the data import module comprises a file reading unit, a data uploading unit and a display unit and is used for importing the unstructured data collected on site into the big data platform in a multithreading high-speed manner;
the data query module comprises a product number and time retrieval module, an equipment number and time retrieval module and a process path and time retrieval module, and is used for displaying multi-source structured data of the whole process of aluminum/copper plate strip production in a time-space layered mode and displaying non-structured data in a graph mode;
the data cleaning module comprises a data set selection module, an original data state display module and a cleaning operation module, and is used for constructing a human-computer interaction interface suitable for the aluminum/copper plate strip data cleaning system, displaying statistical information such as a repeated value, an abnormal value, a single value and a missing value of a selected data source and designing a user cleaning operation logic;
the data analysis module comprises a classification module, a clustering module and a regression module, is used for analyzing and displaying data of the whole aluminum/copper plate strip production process, designs different human-computer interaction systems according to different data analysis methods, and provides different interfaces corresponding to the back-end data analysis function.
2. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 1, wherein: the file reading unit is used for reading the locally stored IBA file and providing the file name for a user, and simultaneously providing the file for the data uploading unit before importing so as to carry out importing operation;
the data uploading unit is used for performing data import from local data to the big data platform and feeding back a main module of a result; after file data of the file reading unit is obtained, the file is packed, compressed and transmitted, decompressed and stored at the server side, and returned to a real-time state to be provided to the display unit;
the display unit is used for providing selectable files and executable operations for a user; including time selection, file path selection, and import schedule display.
3. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 1, wherein: the product number and time retrieval module is used for indexing and searching all volume number information of the current time period according to the time period and then inquiring specific volume number data;
the equipment number and time retrieval module is used for classifying and displaying data generated in different time periods on different equipment according to equipment division data;
the 'process path + time' retrieval module is used for displaying products produced in different processes at different times according to processes such as casting, hot rolling and cold rolling as a primary index, and displaying corresponding L2-L3 data and IBA data after a specific product is selected.
4. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 1, wherein: the data set selection module is used for providing selectable data for a user and providing the selectable data for the single value detection unit, the repeated value detection unit, the abnormal value monitoring unit and the missing value monitoring unit after the user selection is obtained; the user selection comprises a data selection unit and an abnormal value judgment unit;
the original data state display module comprises a single value monitoring unit, a repeated value monitoring unit, a missing value monitoring unit and an abnormal value monitoring unit; the system is used for providing a state display function before cleaning of the original data, visually embodying the basic states of a single value, a repeated value, an abnormal value and a missing value of the original data on a man-machine interaction page, and providing a basis for data cleaning operation; the data selected by the user and provided by the data set selection module is acquired and then processed, and the data is provided for the data cleaning module;
the cleaning operation module comprises a missing value processing unit, an abnormal value processing unit and a default processing unit; the field names processed by the original data state display module are acquired and provided for a user to perform cleaning operation selection, the default processing function and the user-defined file name function for repeated values and single values are provided mainly for missing values and abnormal values, and the user can conveniently search.
5. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 4, wherein: the data selection unit is used for selecting a data source, providing a locally downloaded source data file for a user to select, and supporting the user to specify a file path;
the abnormal value judgment unit is used for selecting an abnormal value judgment method, is mainly used for judging an abnormal value during cleaning, and comprises three methods of a box diagram, a Lauda criterion and an absolute median difference.
6. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 4, wherein: the single value monitoring unit is used for monitoring and displaying field names with a row of repeated values exceeding 90% in the data;
the repetition value monitoring unit is used for monitoring data with two completely repeated upper and lower lines in the data, and only the repeated line number and the repeated volume number are displayed because the field data generally has more repeated values and large data volume;
the missing value monitoring unit is used for monitoring missing and meaningless data in the data; displaying the number of the missing values, the specific fields where the missing values are located, the missing proportion of the corresponding fields and other statistical information;
the abnormal value monitoring unit is used for judging abnormal values in the monitoring data according to the abnormal values selected by the user, and displaying the field names of the abnormal values and the positions of the corresponding abnormal values;
7. the aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 4, wherein: the missing value processing unit processes the missing value detected in the previous module; providing four modes of deletion, filling median, filling mode and filling average for the user to select;
the abnormal value processing unit processes the abnormal value detected in the previous module; five modes of deletion, no processing, filling median, filling mode and filling average are provided for the user to select;
the default processing unit processes the repeated value and the single value, and has little meaning on data analysis and data visualization, so that the data is deleted by default when the back end is cleaned.
8. The aluminum/copper plate and strip industrial big data platform man-machine interaction system as claimed in claim 1, wherein: the classification module, the clustering module and the regression module respectively correspond to different data characteristics and different process backgrounds; aiming at the characteristic, a man-machine interaction system aiming at different interfaces of different methods is constructed, wherein each module comprises the following specific units:
the data selection unit is used for providing input parameter settings of algorithms such as data source selection, independent variable selection, dependent variable selection, parameter setting and the like for a user at the front end of the classification and regression algorithm and supporting a default transfer mode; providing data set selection, column selection and algorithm parameter setting for the front end of the clustering method;
the data processing unit is used for transmitting the data to the java back end after the data selection unit obtains the user input, the back end transmits the data to the Python program for analysis, and the analysis result is generated and then transmitted to the analysis result visualization unit through the java end;
and the analysis result visualization unit is used for displaying the returned result of the algorithm processing unit to the user in a form and graphic mode so as to visually display the analysis result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110470285.5A CN113138963A (en) | 2021-04-28 | 2021-04-28 | Man-machine interaction system of large industrial data platform for aluminum/copper plate strips |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110470285.5A CN113138963A (en) | 2021-04-28 | 2021-04-28 | Man-machine interaction system of large industrial data platform for aluminum/copper plate strips |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113138963A true CN113138963A (en) | 2021-07-20 |
Family
ID=76816388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110470285.5A Pending CN113138963A (en) | 2021-04-28 | 2021-04-28 | Man-machine interaction system of large industrial data platform for aluminum/copper plate strips |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113138963A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050120030A1 (en) * | 2003-10-14 | 2005-06-02 | Medicel Oy | Visualization of large information networks |
| CN105956029A (en) * | 2016-04-25 | 2016-09-21 | 吉林大学 | Data access method under hybrid storage architecture |
| CN108229828A (en) * | 2018-01-04 | 2018-06-29 | 上海电气集团股份有限公司 | A kind of analysis system based on industrial data |
| CN109739922A (en) * | 2019-01-10 | 2019-05-10 | 江苏徐工信息技术股份有限公司 | A kind of industrial data intelligent analysis system |
| CN110134674A (en) * | 2019-05-22 | 2019-08-16 | 中国人民银行成都分行 | A kind of money and credit big data monitoring analysis system |
| CN110298614A (en) * | 2019-06-05 | 2019-10-01 | 西安理工大学 | A kind of food packaging shop production line administrative system and its management method |
| CN110705807A (en) * | 2019-10-23 | 2020-01-17 | 哈工大机器人（合肥）国际创新研究院 | Semiconductor production quality prediction method and system based on field data |
| US20200026397A1 (en) * | 2018-07-17 | 2020-01-23 | Methodical Mind, Llc. | Graphical user interface system |
| CN111914037A (en) * | 2020-07-15 | 2020-11-10 | 国网能源研究院有限公司 | Power grid development-oriented multivariate information mining and analyzing method and system |
-
2021
- 2021-04-28 CN CN202110470285.5A patent/CN113138963A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050120030A1 (en) * | 2003-10-14 | 2005-06-02 | Medicel Oy | Visualization of large information networks |
| CN105956029A (en) * | 2016-04-25 | 2016-09-21 | 吉林大学 | Data access method under hybrid storage architecture |
| CN108229828A (en) * | 2018-01-04 | 2018-06-29 | 上海电气集团股份有限公司 | A kind of analysis system based on industrial data |
| US20200026397A1 (en) * | 2018-07-17 | 2020-01-23 | Methodical Mind, Llc. | Graphical user interface system |
| CN109739922A (en) * | 2019-01-10 | 2019-05-10 | 江苏徐工信息技术股份有限公司 | A kind of industrial data intelligent analysis system |
| CN110134674A (en) * | 2019-05-22 | 2019-08-16 | 中国人民银行成都分行 | A kind of money and credit big data monitoring analysis system |
| CN110298614A (en) * | 2019-06-05 | 2019-10-01 | 西安理工大学 | A kind of food packaging shop production line administrative system and its management method |
| CN110705807A (en) * | 2019-10-23 | 2020-01-17 | 哈工大机器人（合肥）国际创新研究院 | Semiconductor production quality prediction method and system based on field data |
| CN111914037A (en) * | 2020-07-15 | 2020-11-10 | 国网能源研究院有限公司 | Power grid development-oriented multivariate information mining and analyzing method and system |
Non-Patent Citations (2)
| Title |
|---|
| SHEN YAN等: "Quality prediction method for aluminum alloy ingot based on XGBoost", 《2020 CHINESE CONTROL AND DECISION CONFERENCE》, pages 1 - 7 * |
| 刘贤康: "基于Hadoop生态圈的工业数据平台设计与研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 138 - 345 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bendix et al. | Parallel sets: visual analysis of categorical data | |
| CN106372194B (en) | Method and system for presenting search results | |
| CN106682213B (en) | Internet of things task customizing method and system based on Hadoop platform | |
| US7890519B2 (en) | Summarizing data removed from a query result set based on a data quality standard | |
| CN104462167B (en) | Data analysis accessory system | |
| WO2014148176A1 (en) | Information-gathering system, information-gathering method, and storage medium | |
| CN105205686A (en) | Method and system for obtaining product price information | |
| CN111474904A (en) | Nuclear power plant control system configuration data visualization platform and method | |
| CN106682218A (en) | Industrial enterprise Web data graphical system and generation method | |
| CN103377240A (en) | Information providing method, processing server and combination server | |
| CN114238388A (en) | Heterogeneous data collection and retrieval system based on multiple protocols | |
| CN111505444B (en) | Comprehensive ranging traveling wave analysis optimization system and method | |
| CN113919813A (en) | Production line dynamic value flow analysis method and system based on production line dynamic value flow graph | |
| CN113138963A (en) | Man-machine interaction system of large industrial data platform for aluminum/copper plate strips | |
| CN112162896A (en) | InfluxDB-based cluster data monitoring method | |
| CN117009610A (en) | K line graph visualization system and method based on power data | |
| CN111104683A (en) | Key information content matching and identifying method based on big data | |
| CN115809226A (en) | Data docking method and system for intelligent management system of coal washery | |
| CN114780625A (en) | Data management and comprehensive analysis system for civil aircraft | |
| CN110032560B (en) | Method and device for generating monitoring chart | |
| CN115062133A (en) | Data query method and device based on data model, computer equipment and medium | |
| CN107203607B (en) | Online cultural and creative interaction platform based on intelligent robot | |
| CN112884349A (en) | Stock technical index display method, device and system | |
| KR20100117837A (en) | System for and method of displaying virtual images of results in search system | |
| CN111429987B (en) | Index file-based searching method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20240322 |