CN116795776A - Engine test data management system - Google Patents
Engine test data management system Download PDFInfo
- Publication number
- CN116795776A CN116795776A CN202310767327.0A CN202310767327A CN116795776A CN 116795776 A CN116795776 A CN 116795776A CN 202310767327 A CN202310767327 A CN 202310767327A CN 116795776 A CN116795776 A CN 116795776A
- Authority
- CN
- China
- Prior art keywords
- data
- test data
- management system
- engine test
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013523 data management Methods 0.000 title claims abstract description 27
- 238000005516 engineering process Methods 0.000 claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 20
- 238000004458 analytical method Methods 0.000 claims abstract description 18
- 238000007726 management method Methods 0.000 claims abstract description 18
- 238000013500 data storage Methods 0.000 claims abstract description 8
- 230000008901 benefit Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 5
- 230000015556 catabolic process Effects 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 238000013499 data model Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012916 structural analysis Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an engine test data management system which comprises a metadata model, file block uploading, file block analysis, big data storage, timing task management and data through mark. The metadata model technique refers to a technique of modeling and managing metadata information of data. Metadata is data describing data, including information such as definition, structure, format, attribute, relationship, etc. of the data, and the metadata model technology models the information and manages and maintains the information through a unified metadata management system. According to the invention, the engine test data management system is established through the data model, file block uploading, file block analysis, big data storage, timing task management and data standard crossing technology, so that the problems that test data are scattered and independent, standardized management is lacking, test data safety cannot be ensured and test data is lacking in effective analysis in the management at present are solved.
Description
Technical Field
The invention relates to the technical field of data management, in particular to an engine test data management system.
Background
At present, a large amount of test data is stored in each computer in a scattered manner in an electronic document mode, a unified storage management mechanism is lacked, and the data of each test is basically in an independent state. Test data is difficult to share among general, test, professional, etc. departments; the collection, storage, collection, transmission and analysis of the data are basically in a punctiform scattered state, so that the test data are difficult to collect and comprehensively analyze; the related data is not correlated, so that the utilization rate of the data is low, and a lot of valuable data is not mined. In addition, the naming modes of the existing test methods, test projects, test parameters and the like are inconsistent in different professional departments, so that data sharing is difficult. The standardization and standardization of the test, the standardization of the data analysis and the standardization of the information such as test terms and symbols are also enhanced by informatization means. Because the acquisition, storage, output and application of test data by each test are basically in point distribution, and various test data are stored in a scattered manner in the hands of each professional designer, the safety of the data is difficult to ensure. In addition, the engine test process has high data sampling rate, long sampling time and more acquisition parameters, so that the engine test data file has large capacity, and the current lack of effective tools for checking, comparing and analyzing the test data has low data utilization rate.
Disclosure of Invention
The invention provides an engine test data management system for solving the defects existing in the prior background technology, and the specific technical scheme is as follows:
an engine test data management system comprises a metadata model, file block uploading, file block analysis, big data storage, timing task management and data through mark.
Further, the metadata model technology refers to a technology for modeling and managing metadata information of data. Metadata is data describing data, including information such as definition, structure, format, attribute, relationship, etc. of the data, and the metadata model technology models the information and manages and maintains the information through a unified metadata management system.
Further, the file block uploading is a technology of dividing a large file into a plurality of smaller blocks for uploading. The main advantage of the technology is that the problems of network interruption, failure in uploading and the like can be avoided when large files are uploaded, so that the reliability and the efficiency of uploading are improved.
Furthermore, the engine test data management system realizes the rapid and stable uploading of massive test data and high-definition test video files through a file partitioning technology, and provides a data base for engine test data management.
Further, the file chunk parsing is a technique of dividing a large file into a plurality of smaller chunks for formatting parsing. The technology has the main advantages that the problems of unstable system or system breakdown and the like caused by overlarge occupied memory of the system due to direct analysis of big data files can be avoided, and therefore the stability of the system and the reliability of file analysis are improved.
Furthermore, the big data storage adopts HBase, which is an extensible distributed NoSQL database and is specially used for storing big data. The method is established on the Hadoop and has the advantages of high reliability, high expandability, high performance and the like. The HBase supports the storage and access of mass data, can process PB-level data, and has rapid reading and writing capability.
By adopting the technical scheme, the method has the following technical effects:
according to the invention, the engine test data management system is established through the data model, file block uploading, file block analysis, big data storage, timing task management and data standard crossing technology, so that the problems that test data are scattered and independent, standardized management is lacking, test data safety cannot be ensured and test data is lacking in effective analysis in the management at present are solved.
Drawings
FIG. 1 is a schematic diagram of a metadata model of the present invention;
FIG. 2 is a schematic diagram of file block upload according to the present invention;
FIG. 3 is a schematic diagram of a big data store of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The engine test data management system in the embodiment mainly comprises a metadata model technology, a file block uploading technology, a file block analysis technology, a big data storage technology, a timing task management technology and a data through mark technology.
As shown in fig. 1, the metadata model technique refers to a technique of modeling and managing metadata information of data. Metadata is data describing data, including information such as definition, structure, format, attribute, relationship, etc. of the data, and the metadata model technology models the information and manages and maintains the information through a unified metadata management system.
As shown in fig. 2, file block uploading is a technique of uploading a large file into a plurality of smaller blocks. The main advantage of the technology is that the problems of network interruption, failure in uploading and the like can be avoided when large files are uploaded, so that the reliability and the efficiency of uploading are improved. And the engine test data management system realizes the rapid and stable uploading of massive test data and high-definition test video files through a file partitioning technology, and provides a data basis for engine test data management.
File chunk parsing is a technique that separates a large file into multiple smaller chunks for formatting parsing. The technology has the main advantages that the problems of unstable system or system breakdown and the like caused by overlarge occupied memory of the system due to direct analysis of big data files can be avoided, and therefore the stability of the system and the reliability of file analysis are improved.
The engine test data management system realizes the structural analysis of the high-capacity slow-variable data file and the quick-change data file by a file block analysis technology, and meets the management requirement of structural storage of the engine test data.
As shown in fig. 3, the big data storage technology, HBase is a scalable distributed NoSQL database, dedicated to storing big data. The method is established on the Hadoop and has the advantages of high reliability, high expandability, high performance and the like. The HBase supports the storage and access of mass data, can process PB-level data, and has rapid reading and writing capability.
The engine test data management system realizes the storage and management of mass data files by combining the structured database mysql with the HBase database, and provides a data basis for engine test data management, comparison analysis and the like.
A timed task refers to a function of automatically executing a certain task at a predetermined point in time or time interval. The engine test data management system executes data formatting analysis and data synchronization operation in idle time of the system through timing tasks, and ensures system access stability and data integrity.
The data cross-marking technology refers to the process of gradually standardizing test data through rule definition of test projects and test parameters. The engine test data management system checks the corresponding standard every time the data is put into storage through a through-mark technology, continuously updates the data standard, and finally realizes standardized management of the data.
While the basic principles and main features of the present invention have been described above, it will be understood by those skilled in the art that the present invention is not limited by the above-described embodiments, which are described in the foregoing description merely illustrate the principles of the present invention, and various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined in the appended claims and their equivalents.
Claims (6)
1. The engine test data management system is characterized by comprising a metadata model, file block uploading, file block analysis, big data storage, timing task management and data through mark.
2. An engine test data management system according to claim 1, wherein the metadata model technique is a technique of modeling and managing metadata information of data. Metadata is data describing data, including information such as definition, structure, format, attribute, relationship, etc. of the data, and the metadata model technology models the information and manages and maintains the information through a unified metadata management system.
3. An engine test data management system as in claim 1 wherein said file block upload is a technique of uploading a large file into a plurality of smaller blocks. The main advantage of the technology is that the problems of network interruption, failure in uploading and the like can be avoided when large files are uploaded, so that the reliability and the efficiency of uploading are improved.
4. The engine test data management system according to claim 1, wherein the engine test data management system realizes fast and stable uploading of massive test data and high-definition test video files through a file partitioning technology, and provides a data basis for engine test data management.
5. An engine test data management system as in claim 1 wherein said file chunk parsing is a technique of formatting a large file into a plurality of smaller chunks. The technology has the main advantages that the problems of unstable system or system breakdown and the like caused by overlarge occupied memory of the system due to direct analysis of big data files can be avoided, and therefore the stability of the system and the reliability of file analysis are improved.
6. The engine test data management system of claim 1, wherein HBase is employed in the big data store, the HBase being a scalable distributed NoSQL database dedicated to storing big data. The method is established on the Hadoop and has the advantages of high reliability, high expandability, high performance and the like. The HBase supports the storage and access of mass data, can process PB-level data, and has rapid reading and writing capability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310767327.0A CN116795776A (en) | 2023-06-27 | 2023-06-27 | Engine test data management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310767327.0A CN116795776A (en) | 2023-06-27 | 2023-06-27 | Engine test data management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116795776A true CN116795776A (en) | 2023-09-22 |
Family
ID=88046554
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310767327.0A Pending CN116795776A (en) | 2023-06-27 | 2023-06-27 | Engine test data management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116795776A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118010361A (en) * | 2024-04-08 | 2024-05-10 | 西安航天动力试验技术研究所 | Engine ground test command decision method, system, equipment and medium |
-
2023
- 2023-06-27 CN CN202310767327.0A patent/CN116795776A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118010361A (en) * | 2024-04-08 | 2024-05-10 | 西安航天动力试验技术研究所 | Engine ground test command decision method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147411B (en) | Data synchronization method, device, computer equipment and storage medium | |
| CN102332029B (en) | Hadoop-based mass classifiable small file association storage method | |
| CN107169083B (en) | Mass vehicle data storage and retrieval method and device for public security card port and electronic equipment | |
| CN101446984B (en) | Method and device for storing files and method and device for deleting files | |
| CN106682077B (en) | Mass time sequence data storage implementation method based on Hadoop technology | |
| CN104317800A (en) | Hybrid storage system and method for mass intelligent power utilization data | |
| CN104281533B (en) | A kind of method and device of data storage | |
| US8578112B2 (en) | Data management system and data management method | |
| CN110727406B (en) | Data storage scheduling method and device | |
| DE202012013469U1 (en) | Data Processing Service | |
| CN104462389A (en) | Method for implementing distributed file systems on basis of hierarchical storage | |
| CN116795776A (en) | Engine test data management system | |
| CN102779138A (en) | Hard disk access method of real time data | |
| US9298733B1 (en) | Storing files in a parallel computing system based on user or application specification | |
| CN103605617A (en) | Method and device for recording changes in stored data | |
| CN113177090A (en) | Data processing method and device | |
| CN106780157B (en) | Ceph-based power grid multi-temporal model storage and management system and method | |
| CN115114370B (en) | Master-slave database synchronization method and device, electronic equipment and storage medium | |
| CN103984507A (en) | Storage configuration and optimizing strategy for bioinformatics high-performance computing platform | |
| CN103942301A (en) | Distributed file system oriented to access and application of multiple data types | |
| CN115391307A (en) | Database optimization method and device, electronic equipment and computer-readable storage medium | |
| CN113553325A (en) | Synchronization method and system for aggregation objects in object storage system | |
| CN102855391A (en) | Enhanced message service (EMS) multi-professional-data holographic storage and panoramic accident inversion method | |
| CN117520270A (en) | Method and device for optimizing, analyzing and processing HDFS small file | |
| CN115470235A (en) | Data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |