CN116501757A - ER diagram-based simulation data construction method and device - Google Patents
ER diagram-based simulation data construction method and device Download PDFInfo
- Publication number
- CN116501757A CN116501757A CN202310733667.1A CN202310733667A CN116501757A CN 116501757 A CN116501757 A CN 116501757A CN 202310733667 A CN202310733667 A CN 202310733667A CN 116501757 A CN116501757 A CN 116501757A
- Authority
- CN
- China
- Prior art keywords
- data
- diagram
- rule
- simulation data
- generation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010586 diagram Methods 0.000 title claims abstract description 149
- 238000004088 simulation Methods 0.000 title claims abstract description 87
- 238000010276 construction Methods 0.000 title claims abstract description 37
- 230000000007 visual effect Effects 0.000 claims abstract description 69
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000009877 rendering Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims description 20
- 230000002159 abnormal effect Effects 0.000 claims description 12
- 230000014509 gene expression Effects 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 10
- 230000009466 transformation Effects 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 9
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000000844 transformation Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000009960 carding Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a simulated data construction method and device based on an ER diagram, wherein the method comprises the following steps: acquiring an ER diagram, analyzing the ER diagram, and generating SQL table information corresponding to the ER diagram; performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram; receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction; and generating simulation data according to the data generation rule. According to the invention, the simulation data is generated by configuring the data generation rule of the entity in the ER diagram on line, so that the visual configuration of the data generation rule on the operation interface is realized, the adjustment is convenient, the simulation data conforming to the corresponding service scene can be generated through simple interface operation, and the efficiency of generating the simulation data is improved.
Description
Technical Field
The invention relates to the field of system testing and related information technology processing, in particular to an ER diagram-based simulation data construction method and device.
Background
Information systems require data support in trials, trial, and exercises, but often do not use real data. Typical reasons are as follows: 1. for security reasons, the use of real data is not allowed. In order to solve the problem, some simple transformation is often adopted to disguise real data in some occasions, but the disguised data can be restored into the real data through inverse transformation, and potential safety hazards still exist. Meanwhile, some camouflage data can reduce the rationality of the data and influence the application effect. 2. It is difficult to use real data for time or cost requirements. The real data is required to be acquired according to a certain flow, corresponding personnel support and equipment guarantee are required, and the minimum time is ensured. In the case of time limitation or investment limitation, it is difficult to acquire real data satisfying the requirements. 3. Different tasks have specific data requirements, and real data cannot meet the application. The information system has definite test, trial and exercise task targets and needs data support which is suitable for the targets. In general, existing real data is not necessarily obtained based on the data requirement of the task, and deviations exist between the existing real data and the actual data requirement in the aspects of integrity, timeliness, availability, correlation, utility and the like. For the above reasons, the need for fast generation of analog data to meet the actual application scenario is urgent for those skilled in the art, and needs to be resolved. A reasonable model and algorithm are adopted, and simulation data are generated rapidly in a computer simulation mode, so that the actual requirements of information system tests, trial and exercise are met.
In the existing technology for quickly generating analog data, it is common practice to configure related business rules according to ER diagrams designed by a user database and directly insert related data into the database. However, in the method, the business rules are manually configured, the configuration process is complicated, quick modification cannot be performed, and the efficiency of generating the analog data is reduced.
Accordingly, the prior art has drawbacks and needs to be improved and developed.
Disclosure of Invention
The invention aims to solve the technical problems that aiming at the defects in the prior art, a test scene data construction method and device based on an ER diagram are provided, and aims to solve the problems that in the prior art, service rules are manually configured according to the ER diagram designed by a user database, the configuration process is complicated, quick modification is impossible, and the efficiency of generating analog data is reduced.
The technical scheme adopted for solving the technical problems is as follows:
an ER graph-based analog data construction method, comprising:
acquiring an ER diagram, analyzing the ER diagram, and generating SQL table information corresponding to the ER diagram;
performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction;
and generating simulation data according to the data generation rule.
In one implementation manner, the obtaining the ER graph, analyzing the ER graph, and generating the SQL table information corresponding to the ER graph includes:
acquiring an ER diagram, analyzing the ER diagram by using a DDL plug-in unit, and generating SQL table information corresponding to the ER diagram;
the SQL table information is original entity information; alternatively, the SQL table information includes: original entity information and an external key of an original entity father entity; alternatively, the SQL table information includes: foreign keys of all entities related to the relationship and attribute information of the relationship.
In one implementation, performing visual rendering on the ER graph according to the SQL table information to obtain a visual interface corresponding to the ER graph, including:
performing visual rendering on the relation between the entity table structure and the entity table in the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
wherein the visual interface comprises: a scene list area, an operation area and a parameter configuration area; the scene list area is used for creating various business scenes; the operation region includes: the device comprises an operation button list, an operation canvas and a running log list, wherein the operation button list is provided with function buttons, the operation canvas is used for configuring and checking data generation rules of the ER diagram, and the log list is used for checking generation logs of simulation data corresponding to the ER diagram; the parameter configuration area is used for configuring entity attribute generation rules and relationship attribute generation rules corresponding to the ER diagram.
In one implementation, the receiving a rule configuration instruction, before configuring data generation rules on the visual interface according to the rule configuration instruction, further includes:
metadata information is stored in advance, wherein the metadata information is obtained according to a data standard corresponding to the current scene service, and the data standard comprises a metadata standard and/or a dictionary table standard.
In one implementation, the receiving a rule configuration instruction, configuring data on the visual interface according to the rule configuration instruction, and generating a rule includes:
receiving a rule configuration instruction, and configuring data on the visual interface according to the metadata information and the rule configuration instruction to generate a rule;
wherein the data generation rule includes: and the entity attribute rule corresponding to the ER diagram, the association degree parameter between the entities and the target data type.
In one implementation, the entity attribute rule includes: simulating data generation rules, normal data settings and abnormal data settings; the simulation data generation rule includes: fixed values, random values, regular expressions, increments, lexicon generation, and custom functions.
Specifically, the user can customize the function on the configuration list page to meet specific rule requirements, so that the application range is wider and the method is more targeted.
In one implementation, the step of configuring the association parameter between the entities includes:
acquiring a database table corresponding to the ER diagram, wherein the database table comprises a relational database SQL table and a non-relational database SQL table;
analyzing the SQL sentences in the database tables, and determining the association relation among the database tables according to the types of the association fields in the SQL sentences, wherein the association relation of the database tables comprises an external key relation, a same-dimensional relation and a main sub relation;
receiving a user setting instruction, and setting the association relation on the visual interface;
and scheduling the generation sequence of the simulation data generated by each database table according to the association relation.
In one implementation, the generating sequence of each database table to generate the simulation data is arranged according to the association relation, including:
taking a database table in the same-dimensional relationship as a same-dimensional table, taking a database table in a main-sub relationship as a main sub table, hiding the sub table in the main sub table by attaching to the main table, and taking an external key table of the sub table as an external key table of the main table;
assigning the same initial label to each database table, and updating the initial label according to the number of all external key tables of the database table to obtain a final label;
the order of the final marks from small to large is used as the generation order of the simulation data generated by each database table.
In one implementation, the configuration of the target data type includes: setting a data generation rate, setting normal data, setting abnormal data, and setting a data format.
In one implementation, generating simulated data according to the data generation rule includes:
generating simulation data for each database table in turn according to the generation order;
and after the simulation data of the main table are generated, randomly acquiring an external key value from the main key of the main table.
The invention also provides an ER diagram-based simulation data construction device, which comprises:
the analysis module is used for acquiring an ER diagram, analyzing the ER diagram and generating SQL table information corresponding to the ER diagram;
the rendering module is used for performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
the configuration module is used for receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction;
and the generation module is used for generating simulation data according to the data generation rule.
The invention also provides a terminal, which comprises: the device comprises a memory, a processor and an ER diagram based simulation data construction program stored on the memory and capable of running on the processor, wherein the ER diagram based simulation data construction program realizes the steps of the ER diagram based simulation data construction method when being executed by the processor.
The present invention also provides a computer-readable storage medium storing a computer program executable for implementing the steps of the ER graph-based simulation data constructing method as described above.
The invention provides a simulated data construction method and device based on an ER diagram, wherein the method comprises the following steps: acquiring an ER diagram, analyzing the ER diagram, and generating SQL table information corresponding to the ER diagram; performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram; receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction; and generating simulation data according to the data generation rule. According to the invention, the simulation data is generated by configuring the data generation rule of the entity in the ER diagram on line, so that the visual configuration of the data generation rule on the operation interface is realized, the adjustment is convenient, the simulation data conforming to the corresponding service scene can be generated through simple interface operation, and the efficiency of generating the simulation data is improved.
Drawings
FIG. 1 is an overall flow chart of the invention for constructing simulation data based on ER diagrams.
FIG. 2 is a flow chart of a preferred embodiment of the ER diagram-based simulated data construction method of the present invention.
FIG. 3 is a layout of a visual operation interface in the present invention.
FIG. 4 is a schematic diagram of the visual operation interface according to the present invention.
FIG. 5 is a functional block diagram of a preferred embodiment of the ER diagram based analog data construction device of the present invention.
Fig. 6 is a functional block diagram of a terminal in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Information systems require data support in trials, trial, and exercises, but often do not use real data. Typical reasons are as follows: 1. for security reasons, the use of real data is not allowed. In order to solve the problem, some simple transformation is often adopted to disguise real data in some occasions, but the disguised data can be restored into the real data through inverse transformation, and potential safety hazards still exist. Meanwhile, some camouflage data can reduce the rationality of the data and influence the application effect. 2. It is difficult to use real data for time or cost requirements. The real data is required to be acquired according to a certain flow, corresponding personnel support and equipment guarantee are required, and the minimum time is ensured. In the case of time limitation or investment limitation, it is difficult to acquire real data satisfying the requirements. 3. Different tasks have specific data requirements, and real data cannot meet the application. The information system has definite test, trial and exercise task targets and needs data support which is suitable for the targets. In general, existing real data is not necessarily obtained based on the data requirement of the task, and deviations exist between the existing real data and the actual data requirement in the aspects of integrity, timeliness, availability, correlation, utility and the like. For the above reasons, the need for fast generation of analog data to meet the actual application scenario is urgent for those skilled in the art, and needs to be resolved. A reasonable model and algorithm are adopted, and simulation data are generated rapidly in a computer simulation mode, so that the actual requirements of information system tests, trial and exercise are met.
In the existing technology for quickly generating a large amount of simulation data, one common method is to simulate a real user scene through an interface written by a developer to generate the data, and the method is suitable in some aspects because the user scene is simulated by writing codes, the service data is complete, and the data cannot be lost in the data transmission process. However, the method has large and complicated workload because of the need of writing codes to process each step of the business flow, interface definition, signature, encryption and the like. In addition, the expansibility is poor, and a set of codes is required to be written in one business scene.
Another common approach is to configure related business rules according to ER diagrams designed in the user database, and directly insert related data into the database. However, in the method, the service rules are manually configured, the configuration process is complicated, the service rules cannot be timely adjusted according to the change of the service rules, and a large amount of test data conforming to the test scene cannot be quickly generated.
The invention provides an ER diagram-based simulation data construction method, which is used for solving the following requirements:
(1) Providing ER diagram upload. And uploading ER diagrams of databases of different types by a user, configuring target data types and generating rules, dynamically adapting the target data source types by a system, and automatically generating business simulation data.
(2) And supporting function plugins. Support a variety of rules for generating analog data, such as fixed values, random values, regular expressions, incremental values; supporting the selection of word stock to generate random values within a specific range; support word stock, table design, field information sharing; support to build dictionary tables directly using existing word stock; support the generation of analog data in multiple formats, such as json, xml, sql, cvs.
Meanwhile, the invention provides a simulated data construction method based on the ER diagram, which supports the following three application scenes:
(1) Testing the scene. In the test process, the function or performance of the data back end needs to be verified, and at the moment, the coupling with the data generating end needs to be reduced.
(2) Continuously integrating the scene. In the continuous integration scenario, one or more modules form a platform, and continuous data is required to enter the continuous integration environment, so as to automatically complete testing and iteration work.
(3) A production scenario. After testing and iteration are completed on one project and released to a production environment, continuous function or availability monitoring is usually required, various normal or abnormal data are required to be continuously and stably produced and sent back to a platform according to certain rules and definitions, and production verification can be performed only through certain rule configuration according to service requirements in a continuous integration scene so as to meet the requirements.
As shown in fig. 1, the process of constructing simulation data based on ER diagram in the present invention mainly includes parsing ER diagram 12, rendering ER diagram 13, carding metadata 14, configuration rules 15, and simulation data 16.
The invention provides a simulated data construction method based on an ER diagram, which is characterized in that SQL is generated by analyzing ER diagram metadata, and the ER diagram is visually rendered by combining G6 and Vue3 frames; the method comprises the steps of configuring data rules of entities and association rules of relationships between the entities in an ER diagram on line, configuring output data source types, constructing business rules of simulation data, and finally dynamically adapting, completing data format conversion and generating the simulation data according to the business rules.
Referring to fig. 2, fig. 2 is a flowchart of an ER diagram-based simulation data construction method according to the present invention. As shown in fig. 2, the ER diagram-based simulation data construction method according to the embodiment of the present invention includes the following steps:
step S100, an ER diagram is obtained, the ER diagram is analyzed, and SQL table information corresponding to the ER diagram is generated.
Specifically, the ER diagram is a database design that reflects relationships between tables with relationships between graphs. And analyzing the ER diagram to generate SQL table information corresponding to the ER diagram, wherein the SQL table information is completed by using a DDL plug-in. The invention utilizes the DDL plug-in to convert the ER diagram into SQL table information, can rapidly complete conversion, supports the ER diagram designed by the client required by analyzing different database types, realizes ER diagram metadata standardization, and has good compatibility.
In one implementation, the step S100 specifically includes:
acquiring an ER diagram, analyzing the ER diagram by using a DDL plug-in unit, and generating SQL table information corresponding to the ER diagram;
the SQL table information is original entity information; alternatively, the SQL table information includes: original entity information and an external key of an original entity father entity; alternatively, the SQL table information includes: foreign keys of all entities related to the relationship and attribute information of the relationship.
Specifically, SQL in a relational database is transformed from ER diagrams and can be generally divided into three categories:
(1) The converted SQL table and the original entity contain the same information content. Such transformations are generally applicable to:
in a binary "many-to-many" relationship, entities at either end;
in the binary one-to-many relationship, the entity at one end of one;
in a binary "one-to-one" relationship, the entity at one end;
in the binary 'many-to-many' regression relationship, entities at any one end (note: both ends of the relationship point to the same entity);
entities at either end in a ternary or n-ary relationship;
in the hierarchical generalization relationship, superclass entities.
(2) The converted SQL table contains the information content of the original entity and the foreign key of the parent entity of the original entity. Such transformations are generally applicable to:
in the binary one-to-many relationship, the entity at one end of the many;
in a binary "one-to-one" relationship, the entity at one end;
the entities at either end of a binary "one-to-one" or "one-to-many" regression relationship.
This transformation is one of the common methods of handling relationships, namely adding foreign key information in the child table that points to the primary key in the parent table.
(3) An SQL table obtained by converting the relation contains foreign keys of all entities related to the relation and attribute information of the relation. Such transformations are generally applicable to:
binary "many-to-many" relationship;
binary "many-to-many" regression relationship;
ternary or n-ary relationships;
this transformation is another commonly used relationship processing method. For a "many-to-many" relationship, it is desirable to define a separate table containing the primary keys of two related entities, which table can also contain attribute information for the relationship.
The invention supports the three types of conversion and generates SQL table information corresponding to the ER diagram. The basic conversion steps from ER diagram to SQL table are as follows:
(1) Each entity is converted into a table containing key and non-key attributes.
In the entity conversion, if the two entities are in a one-to-many relationship, a main key of one end entity is added into a table of the multiple end entities to serve as an external key. If the two entities are in a one-to-one relationship, the main key of one entity at one end is put into the table of the other entity to be used as an external key, and the entity added with the external key can be selected theoretically, but generally follows the following principle: placing keys of parent entities into child entities according to the most natural parent-child relationship among the entities; another strategy is to add foreign keys to a table with fewer rows based on efficiency.
Each entity in the generalization hierarchy is converted into a table. Each table will contain keys for superclass entities. In fact the primary key of the sub-class entity is also the foreign key. The superclass table also contains common non-key attributes of all related entities, and the other tables contain non-key attributes specific to each sub-class entity.
The resulting SQL table may contain constraints such as not null, unique, foreign key, etc. Each table must have a primary key that implies non null and unique constraints.
(2) Each "many-to-many" binary or binary regression relationship is converted into a table containing keys for the entities and attributes for the relationship.
In the "many-to-many" binary relationship conversion, each "many-to-many" binary relationship can be converted into
A table contains attributes of keys and relationships of two entities.
The resulting SQL table of the translation may contain non null constraints. The reason that the unique constraint is not used here is that the primary key of the relationship table is composed of a foreign key composite of entities, and the unique constraint is implicit.
(3) Ternary and higher (n-ary) relationships are converted into a table.
In ternary relationship conversion, each ternary (or n-ary) relationship is converted into a table containing n primary keys of related entities and attributes of the relationship.
The resulting table of transformations must contain the not null constraint. The primary key of the relational table is composed of a composite of foreign keys of the entities. The n-gram has n foreign keys. In addition to the primary key constraint, other candidate keys (candidates) should also be subject to the unique constraint.
Meanwhile, the processing rule for NULL value in the conversion process is as follows:
(1) When the relationship between entities is optional, the foreign key column in the SQL table is allowed to be NULL;
(2) When the relationship between entities is mandatory, the foreign key column in the SQL table is not allowed to be NULL;
(3) SQL tables translated from the "many-to-many" relationship do not allow any foreign key columns to be NULL.
The invention completes the generation of SQL table information corresponding to the ER diagram through the rules and steps, supports various relation processing methods, has wide application scene and can meet the complex service requirement.
As shown in fig. 2, the ER graph-based simulation data construction method further includes the following steps:
and step 200, performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram.
Specifically, the invention performs ER diagram visual rendering through the combined use of G6, X6 and Vuedraggble, jsplumb, supports the uploading of ER diagrams of databases of different types, and has high compatibility.
In one embodiment, the step S200 specifically includes:
performing visual rendering on the relation between the entity table structure and the entity table in the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
the visual interface comprises: a scene list area, an operation area and a parameter configuration area; the scene list area is used for creating various business scenes; the operation region includes: the device comprises an operation button list, an operation canvas and a running log list, wherein the operation button list is provided with function buttons, the operation canvas is used for configuring and checking data generation rules of the ER diagram, and the log list is used for checking generation logs of simulation data corresponding to the ER diagram; the parameter configuration area is used for configuring entity attribute generation rules and relationship attribute generation rules corresponding to the ER diagram.
Specifically, as shown in fig. 3, the ER diagram visualization IDE is composed of three parts, namely a scene list area on the left, an operation area in the middle, and a parameter configuration area on the right.
As shown in fig. 4, the specific functions of the respective parts are as follows:
(1) Scene area. The scene area contains a scene list whose function is to maintain different scene functions, such as creating traffic scenes, medical insurance scenes.
(2) An operating region. The operation area is composed of an operation button list, an operation canvas and a running log list. The operation button list mainly comprises function buttons such as zoom-in, zoom-out, operation, selection and the like and is used for controlling operation; the operation canvas is used for configuring and viewing the entity/relationship attribute generation rule in the ER diagram; the log list is used to view a log of ER diagram simulation data.
(3) And a parameter configuration area. The method mainly realizes rule configuration of each entity and different relation attributes of the ER diagram, and the parameter configuration is that the back end of the database is configured and rendered, and a user only needs to fill in related parameter values.
According to the method, the visual interface corresponding to the ER diagram is obtained through visual rendering of the ER diagram. The invention supports the rendering of the entity table structure, can display the table structure, the field type, the primary key, the default value, the index, the remark information and the like according to the types of different databases, and simultaneously supports the rendering of the entity table front relation, and has simple operation.
The receiving rule configuration instruction, before configuring data generation rules on the visual interface according to the rule configuration instruction, further comprises:
metadata information is stored in advance, wherein the metadata information is obtained according to a data standard corresponding to the current scene service, and the data standard comprises a metadata standard and/or a dictionary table standard.
Specifically, metadata is the concept of information and the connection between information, to simulate the data close to the service, and according to various standards of the service data in the actual scene of the user, such as metadata standards, dictionary table standards and the like, metadata combing is required for the entity and the relation between the entities in the ER diagram.
Such as entity student tables in ER diagrams, attributes contain student_id, six, student_name, city, etc. This information is metadata describing a student table. According to metadata standards of users, real business meanings represented by various attributes, such as city representing city fields, value ranges in actual business, business rules and the like, are combed.
By combing the metadata, the requirements for understanding business and data are met, the upstream-downstream relationship and the meaning of the data can be quickly known, the time cost of data research is reduced, and the efficiency is improved; the method can clearly understand the incoming pulse, the service processing rule, the conversion condition and the like of the data flow in the ER diagram.
The invention stores the metadata information in advance, the metadata is combined and carded by manpower and codes, and the metadata information is accurate and reliable and is convenient for later use.
As shown in fig. 1, the ER graph-based simulation data construction method further includes the following steps:
and step S300, receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction.
Specifically, the rule configuration instructions are generated by user operations in a configuration zone of the visual interface.
In one implementation, the step S300 specifically includes:
receiving a rule configuration instruction, and configuring data on the visual interface according to the metadata information and the rule configuration instruction to generate a rule;
wherein the data generation rule includes: and the entity attribute rule corresponding to the ER diagram, the association degree parameter between the entities and the target data type.
Specifically, according to the carded metadata information and the rule configuration instruction, the data generation rule of each entity/relation is configured in the configuration area of the visual interface. According to the method and the device, the rule is generated by configuring the data in the visual interface configuration area, manual rule configuration is not needed, the configuration process is concise and convenient, and the rule can be timely adjusted according to the change of the business rule.
In one implementation, the entity attribute rule includes: simulating data generation rules, normal data settings and abnormal data settings; the simulation data generation rule includes: fixed values, random values, regular expressions, increments, lexicon generation, and custom functions.
Specifically, simulation data generation rules, such as identification card number generation rules of a designated region, users of designated gender and the like, are set on a configuration list page of the visual interface. The operation is visual, the flow is simple and convenient, and the service test of different scenes can be satisfied. The simulation data generation rule comprises a fixed value, a random value, a regular expression, increment, word stock generation and a custom function. The support function plug-in specifically includes: supporting the selection of word stock to generate random values within a specific range; support word stock, table design, field information sharing; support to build dictionary tables directly using existing word stock; support generation of analog data in multiple formats, such as json, xml, sql, cvs; and supporting the user-defined function by supporting the user-defined function uploading by selecting the supporting field dictionary. By configuring the business rules and the support function plug-ins on the interface, the user can conveniently select and use, the expansion is convenient, and the applicability of the data is improved.
The normal data and the abnormal data are set, the normal data meet the normal use scene of the business process, and the abnormal data meet the abnormal use scene of the business process. By setting and generating the normal data and the abnormal data respectively, the test scene can be comprehensively covered.
In one implementation, the step of configuring the association parameter between the entities includes:
a1, acquiring a database table corresponding to the ER diagram, wherein the database table comprises a relational database SQL table and a non-relational database SQL table;
a2, analyzing SQL sentences in the database tables, and determining association relations among the database tables according to the types of association fields in the SQL sentences, wherein the association relations of the database tables comprise foreign key relations, same-dimensional relations and main sub-relations;
a3, receiving a user setting instruction, and setting the association relation on the visual interface;
and A4, scheduling the generation sequence of the simulation data generated by each database table according to the association relation.
The association relation of the database table can be divided into three situations of external key relation, same-dimensional relation and main sub relation. The association relationship may be distinguished according to whether the association field is a primary key (or a part of the primary key) of the table. Wherein the association field is parsed from the entity-to-entity relationship on the ER diagram.
Specifically, according to the SQL statement, it is analyzed from the JOIN clause therein, and it is determined which association is based on whether the JOIN field is the primary key or part of the primary key of the table. If the JOIN field is the primary key of the table, then the JOIN is full, and common data is obtained; if the JOIN field is not the primary key of the table, then it is either a left JOIN or a right JOIN. When the same table is a sub-table of two tables at the same time, the two main tables are regarded as external key tables of the sub-tables, and are not treated as main sub-tables, so that the situation that no sub-table has a plurality of main tables can be ensured.
It should be noted that some external keys are implicit relations, for example, date and area codes exist in the identification card number, if the operation has the action of providing the part of information from the identification card number, then the operation needs to be manually operated on a visual interface, so that the implicit relations are found out, which is equivalent to splitting the field of the identification number into a plurality of sections.
The invention carries out configuration of association degree parameters between entities at the visual interface, so that each database table automatically generates the generation sequence of the simulation data, and can respond to configuration modification in time. According to the invention, the simulation data is carried out according to the actual service ER relationship diagram, the service simulation data is generated, the association of entity attributes and the association between entities can be better embodied, and the service data has strong association.
In one implementation, the generating sequence of each database table to generate the simulation data is arranged according to the association relation, including:
b1, taking a database table in the same-dimensional relationship as a same-dimensional table, taking a database table in a main-sub relationship as a main sub table, attaching the sub table in the main sub table to the main table for hiding, and taking an external key table of the sub table as an external key table of the main table;
b2, giving the same initial label to each database table, and updating the initial label according to the number of all external key tables of the database table to obtain a final label;
b3, taking the order of the final marks from small to large as the generation order of the simulation data generated by each database table.
Specifically, after the association relationship is set, the order in which the data is generated may be ordered. The same-dimension table is treated as a logic table, the sub-table in the main sub-table is firstly hidden by being attached to the main table, other external key tables of the sub-table are also treated as external key tables of the main table, and the main table is treated and the sub-table is treated again. Each table is marked with the number 1. For each table, if the maximum value of the labels of all foreign key tables it has is n, the label of itself is changed to n+1, and this action is repeatedly performed until all labels are no longer changed. The invention automatically schedules the order of generating the data after setting the association relation, so that the data can be quickly generated after setting the rule and can be changed in time according to the change of the configuration rule.
In one implementation, the configuration of the target data type includes: setting a data generation rate, setting normal data, setting abnormal data, and setting a data format.
Specifically, the data generation rate is set in the configuration list of the visual interface, so that the data can be generated according to the actual demand, and the influence on the performance of an operating system due to the data generation is avoided. Setting normal data and abnormal data to more comprehensively meet the test scene. The data format is set in the configuration list of the visual interface, which can support json, xml, sql, cvs and the like, thereby facilitating the subsequent butt joint of test data and other test systems and test environments.
As shown in fig. 1, the ER graph-based simulation data construction method further includes the following steps:
step 400, generating simulation data according to the data generation rule.
Specifically, after the configuration list page of the visual interface completes rule configuration, the simulation data can be generated immediately in response. When the configuration rule is modified, the modified simulation data can be generated in time. After the user uploads ER diagrams of databases of different types, the user can dynamically adapt to the type of the target data source to automatically generate business simulation data after configuring the generation rule. For example, the ER diagram of Mysql is provided, and the ER diagram can be dynamically converted into data of an Opengauss data source, so that various application scenes are satisfied. Meanwhile, the invention provides the online ER diagram visualization, and the data generation state, such as the generation flow of a table, error information and the like, can be observed in real time on a component operation log list interface of a visualization interface. The state of the generated data can be timely judged through real-time observation of the data generation state, and the next operation is convenient. If the generation speed is lower or higher, the generation speed can be adjusted in time according to the actual situation. If the generated data has error reporting information, the error cause can be stopped and inquired in time. The situation that the resource waste or the data is not expected due to the fact that the data generation state cannot be observed in real time is avoided, and the use of a user is facilitated.
In one implementation, the generating the simulation data according to the data generation rule includes:
c1, generating simulation data for each database table in turn according to the generation sequence;
and C2, after the simulation data of the main table are generated, randomly acquiring an external key value from the main key of the main table.
Specifically, the data is generated in the order of the numbers of the tables from small to large. When data is generated for a table numbered n+1, none of the foreign key tables it references exceed n. If it has been generated, its foreign key value is taken randomly from the primary keys of these generated primary tables; the same dimension table can be generated together, and the sub-table waits for the main table to be regenerated after finishing. The invention carries out the simulation data according to the practical service ER relation diagram to generate the service simulation data, and has strong service data relevance and simple operation.
Further, as shown in fig. 5, based on the above-mentioned simulated data construction method based on the ER diagram, the present invention further correspondingly provides a simulated data construction device based on the ER diagram, which includes:
the analysis module 100 is configured to obtain an ER diagram, analyze the ER diagram, and generate SQL table information corresponding to the ER diagram;
the rendering module 200 is configured to perform visual rendering on the ER graph according to the SQL table information, so as to obtain a visual interface corresponding to the ER graph;
the configuration module 300 is configured to receive a rule configuration instruction, and configure data on the visual interface according to the rule configuration instruction to generate a rule;
the generating module 400 is configured to generate simulation data according to the data generating rule.
As shown in fig. 6, the present invention further provides a terminal, including: the device comprises a memory 20, a processor 10 and an ER diagram based simulation data construction program 30 stored on the memory 20 and capable of running on the processor 10, wherein the ER diagram based simulation data construction program 30 realizes the steps of the ER diagram based simulation data construction method as described above when being executed by the processor 10.
The present invention also provides a computer-readable storage medium storing a computer program executable for implementing the steps of the ER graph-based simulation data constructing method as described above.
In summary, the method and apparatus for constructing analog data based on ER diagram provided by the present invention, the method includes: acquiring an ER diagram, analyzing the ER diagram, and generating SQL table information corresponding to the ER diagram; performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram; receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction; and generating simulation data according to the data generation rule. According to the invention, the data rule of the entity in the ER diagram, the association rule of the relation between the entity and the entity, the type of the output data source and the business rule of the simulation data are configured on line, and finally the simulation data are generated according to the business rule, so that the visual configuration of the business rule on an operation interface is realized, the adjustment is convenient, a large amount of simulation data conforming to the business scene can be generated through simple interface operation, and the simulation data can be generated rapidly. The invention has strong service data relevance, supports ER diagram analysis of different database types, dynamically adapts to different data sources, supports function plugin, has observable data, is convenient and intuitive to use, has strong compatibility and wide application range.
It is to be understood that the invention is not limited in its application to the examples described above, but is capable of modification and variation in light of the above teachings by those skilled in the art, and that all such modifications and variations are intended to be included within the scope of the appended claims.
Claims (13)
1. A method of modeling data construction based on an ER diagram, comprising:
acquiring an ER diagram, analyzing the ER diagram, and generating SQL table information corresponding to the ER diagram;
performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction;
and generating simulation data according to the data generation rule.
2. The ER graph-based simulated data construction method of claim 1, wherein the obtaining the ER graph, parsing the ER graph, and generating SQL table information corresponding to the ER graph, comprises:
acquiring an ER diagram, analyzing the ER diagram by using a DDL plug-in unit, and generating SQL table information corresponding to the ER diagram;
the SQL table information is original entity information; alternatively, the SQL table information includes: original entity information and an external key of an original entity father entity; alternatively, the SQL table information includes: foreign keys of all entities related to the relationship and attribute information of the relationship.
3. The ER graph-based simulated data construction method of claim 1, wherein performing visual rendering on the ER graph according to the SQL table information to obtain a visual interface corresponding to the ER graph, comprises:
performing visual rendering on the relation between the entity table structure and the entity table in the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
wherein the visual interface comprises: a scene list area, an operation area and a parameter configuration area; the scene list area is used for creating various business scenes; the operation region includes: the device comprises an operation button list, an operation canvas and a running log list, wherein the operation button list is provided with function buttons, the operation canvas is used for configuring and checking data generation rules of the ER diagram, and the log list is used for checking generation logs of simulation data corresponding to the ER diagram; the parameter configuration area is used for configuring entity attribute generation rules and relationship attribute generation rules corresponding to the ER diagram.
4. The ER graph-based simulated data construction method of claim 1, wherein said receiving a rule configuration instruction, prior to configuring a data generation rule on said visual interface in accordance with said rule configuration instruction, further comprises:
metadata information is stored in advance, wherein the metadata information is obtained according to a data standard corresponding to the current scene service, and the data standard comprises a metadata standard and/or a dictionary table standard.
5. The ER graph-based simulated data construction method of claim 4, wherein said receiving a rule configuration instruction, configuring data generation rules on said visual interface in accordance with said rule configuration instruction, comprises:
receiving a rule configuration instruction, and configuring data on the visual interface according to the metadata information and the rule configuration instruction to generate a rule;
wherein the data generation rule includes: and the entity attribute rule corresponding to the ER diagram, the association degree parameter between the entities and the target data type.
6. The ER graph-based simulated data construction method of claim 5, wherein said entity attribute rules comprise: simulating data generation rules, normal data settings and abnormal data settings; the simulation data generation rule includes: fixed values, random values, regular expressions, increments, lexicon generation, and custom functions.
7. The ER diagram-based simulation data construction method of claim 5, wherein the configuring of the association parameter between the entities comprises:
acquiring a database table corresponding to the ER diagram, wherein the database table comprises a relational database SQL table and a non-relational database SQL table;
analyzing the SQL sentences in the database tables, and determining the association relation among the database tables according to the types of the association fields in the SQL sentences, wherein the association relation of the database tables comprises an external key relation, a same-dimensional relation and a main sub relation;
receiving a user setting instruction, and setting the association relation on the visual interface;
and scheduling the generation sequence of the simulation data generated by each database table according to the association relation.
8. The ER graph-based simulation data construction method of claim 7, wherein the scheduling of the generation order of the respective database tables to generate the simulation data according to the association relation comprises:
taking a database table in a same-dimensional relationship as a same-dimensional table, taking a database table in a main-sub relationship as a main sub table, hiding the sub table in the main sub table by attaching to the main table, and taking an external key table of the sub table as an external key table of the main table;
assigning the same initial label to each database table, and updating the initial label according to the number of all external key tables of the database table to obtain a final label;
the order of the final marks from small to large is used as the generation order of the simulation data generated by each database table.
9. The ER graph-based simulated data construction method of claim 5, wherein the configuration of the target data type comprises: setting a data generation rate, setting normal data, setting abnormal data, and setting a data format.
10. The ER graph-based simulation data construction method of claim 8, wherein generating simulation data according to the data generation rule comprises:
generating simulation data for each database table in turn according to the generation order;
and after the simulation data of the main table are generated, randomly acquiring an external key value from the main key of the main table.
11. An ER diagram-based analog data construction apparatus, comprising:
the analysis module is used for acquiring an ER diagram, analyzing the ER diagram and generating SQL table information corresponding to the ER diagram;
the rendering module is used for performing visual rendering on the ER diagram according to the SQL table information to obtain a visual interface corresponding to the ER diagram;
the configuration module is used for receiving a rule configuration instruction, and configuring data on the visual interface to generate a rule according to the rule configuration instruction;
and the generation module is used for generating simulation data according to the data generation rule.
12. A terminal, comprising: memory, processor and store on the memory and can be run on the processor based on ER diagram simulation data construction procedure, the simulation data construction procedure based on ER diagram is carried out by the processor and realizes the steps of the simulation data construction method based on ER diagram according to any one of claims 1-10.
13. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program executable for implementing the steps of the ER graph-based simulation data construction method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310733667.1A CN116501757B (en) | 2023-06-20 | 2023-06-20 | ER diagram-based simulation data construction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310733667.1A CN116501757B (en) | 2023-06-20 | 2023-06-20 | ER diagram-based simulation data construction method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116501757A true CN116501757A (en) | 2023-07-28 |
| CN116501757B CN116501757B (en) | 2023-10-03 |
Family
ID=87316806
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310733667.1A Active CN116501757B (en) | 2023-06-20 | 2023-06-20 | ER diagram-based simulation data construction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116501757B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130187926A1 (en) * | 2011-07-08 | 2013-07-25 | Steamfunk Labs, Inc. | Automated presentation of information using infographics |
| CN111736813A (en) * | 2020-06-24 | 2020-10-02 | 深圳前海微众银行股份有限公司 | JPA code generation method and device, terminal equipment and storage medium |
| CN112597171A (en) * | 2020-12-31 | 2021-04-02 | 平安银行股份有限公司 | Table relation visualization method and device, electronic equipment and storage medium |
| CN114637811A (en) * | 2022-03-15 | 2022-06-17 | 平安国际智慧城市科技股份有限公司 | Data table entity relation graph generation method, device, equipment and storage medium |
| CN115543428A (en) * | 2022-10-31 | 2022-12-30 | 中国人民解放军31007部队 | Simulated data generation method and device based on strategy template |
| CN115617776A (en) * | 2022-09-30 | 2023-01-17 | 国家石油天然气管网集团有限公司 | Data management system and method |
| CN115858513A (en) * | 2022-12-07 | 2023-03-28 | 腾讯医疗健康(深圳)有限公司 | Data governance method, data governance device, computer equipment and storage medium |
| CN116028653A (en) * | 2023-03-29 | 2023-04-28 | 鹏城实验室 | Method and system for constructing map by visually configuring multi-source heterogeneous data |
-
2023
- 2023-06-20 CN CN202310733667.1A patent/CN116501757B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130187926A1 (en) * | 2011-07-08 | 2013-07-25 | Steamfunk Labs, Inc. | Automated presentation of information using infographics |
| CN111736813A (en) * | 2020-06-24 | 2020-10-02 | 深圳前海微众银行股份有限公司 | JPA code generation method and device, terminal equipment and storage medium |
| CN112597171A (en) * | 2020-12-31 | 2021-04-02 | 平安银行股份有限公司 | Table relation visualization method and device, electronic equipment and storage medium |
| CN114637811A (en) * | 2022-03-15 | 2022-06-17 | 平安国际智慧城市科技股份有限公司 | Data table entity relation graph generation method, device, equipment and storage medium |
| CN115617776A (en) * | 2022-09-30 | 2023-01-17 | 国家石油天然气管网集团有限公司 | Data management system and method |
| CN115543428A (en) * | 2022-10-31 | 2022-12-30 | 中国人民解放军31007部队 | Simulated data generation method and device based on strategy template |
| CN115858513A (en) * | 2022-12-07 | 2023-03-28 | 腾讯医疗健康(深圳)有限公司 | Data governance method, data governance device, computer equipment and storage medium |
| CN116028653A (en) * | 2023-03-29 | 2023-04-28 | 鹏城实验室 | Method and system for constructing map by visually configuring multi-source heterogeneous data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116501757B (en) | 2023-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210174006A1 (en) | System and method for facilitating complex document drafting and management | |
| US8825695B2 (en) | Mapping dataset elements | |
| Moreau et al. | A templating system to generate provenance | |
| US20050066263A1 (en) | System and method for generating data validation rules | |
| Danaparamita et al. | QueryViz: helping users understand SQL queries and their patterns | |
| CN106445536B (en) | Automatic business design management system | |
| KR101917038B1 (en) | Device of reference document integration management and procedure display for making nuclear power plant commissioning procedures | |
| Wang et al. | A full life cycle nuclear knowledge management framework based on digital system | |
| Kiviluoma et al. | Spine Toolbox: A flexible open-source workflow management system with scenario and data management | |
| CN111176613A (en) | Collaborative task automatic decomposition system based on architecture model | |
| Muslim et al. | A modular and extensible framework for open learning analytics | |
| CN115687649A (en) | Automatic image examination system based on BIM and knowledge graph | |
| CN112783475B (en) | Embedded software demand analysis method | |
| Sauerbier | UMDBS-a new tool for dynamic microsimulation | |
| CN112817964B (en) | Index billboard design and development system based on process robot | |
| CN116501757B (en) | ER diagram-based simulation data construction method and device | |
| CN116842076A (en) | Data analysis method, device, analysis equipment and readable storage medium | |
| CN115827885A (en) | Operation and maintenance knowledge graph construction method and device and electronic equipment | |
| Barnes | NASA's Advanced Multimission Operations System: a case study in software architecture evolution | |
| CN112286902A (en) | Intelligent application development system based on cloud computing and big data | |
| Lee et al. | GEA: A Goal-Driven Approach toDiscovering Early Aspects | |
| Liu | Integrating process mining with discrete-event simulation modeling | |
| Voinov et al. | Community-based software tools to support participatory modelling: a vision | |
| Meinecke et al. | Visualizing RCE Workflow Executions via W3C Provenance | |
| Skotnik et al. | An Approach towards Reusability in Hybrid Avionic Software Development by Using a Unified Graph Representation of the Software System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |