CN110245270A - Data genetic connection storage method, system, medium and equipment based on graph model - Google Patents
Data genetic connection storage method, system, medium and equipment based on graph model Download PDFInfo
- Publication number
- CN110245270A CN110245270A CN201910385135.7A CN201910385135A CN110245270A CN 110245270 A CN110245270 A CN 110245270A CN 201910385135 A CN201910385135 A CN 201910385135A CN 110245270 A CN110245270 A CN 110245270A
- Authority
- CN
- China
- Prior art keywords
- graph model
- data
- name
- initial graph
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2291—User-Defined Types; Storage management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16B—BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G16B50/00—ICT programming tools or database systems specially adapted for bioinformatics
- G16B50/30—Data warehousing; Computing architectures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioethics (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biotechnology (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data genetic connection storage method based on graph model, comprising: the SQL statement in parsing data mart modeling script；Create initial graph model；By the parsing result and initial graph model interaction；It repeats above operation, traverses the SQL statement in all data mart modeling scripts, generate a genetic connection graph model.The above scheme of the embodiment of the present invention, directly using data as the node of figure, relationship, attribute storage into chart database, without being pre-designed complicated relational data table structure, significantly reduces the design difficulty and complexity of such scene using graph model；Second, have benefited from the memory computer system of chart database Neo4j and the data structure of optimization, under mass data, data blood relationship upstream and downstream level quantity, the statistics of dependence table quantity can be rapidly completed in several milliseconds, and the retrieval of data field and tables of data dependence is rapidly completed.
Description
Technical field
The present invention relates to software technology fields, store in particular to a kind of data genetic connection based on graph model
Method, system, medium and electronic equipment.
Background technique
In the prior art, data warehouse works as investigation to support different business that can generate a large amount of tables of data and data
When data quality problem, cleaning redundant data and data flow to link, it is difficult to quickly clear between mass data
Blood relationship dependence.It generallys use manual record or is stored based on the form of the relevant databases such as mysql based on graph model
Data genetic connection, however, this mode complexity is higher, is easy error, can not support complicated data consanguinity analysis, difficulty
To cope with the performance requirement under large-scale data.
Therefore, in long-term research and development, inventor has carried out greatly the storage of the data genetic connection based on graph model
One of quantifier elimination proposes a kind of data genetic connection storage method based on graph model, to solve the above technical problems.
Summary of the invention
The data genetic connection storage method that the purpose of the present invention is to provide a kind of based on graph model, system, medium and
Electronic equipment is able to solve at least one technical problem mentioned above.Concrete scheme is as follows:
Specific embodiment according to the present invention, in a first aspect, the present invention provides a kind of data blood relationship based on graph model
Relationship storage method characterized by comprising
Parse the SQL statement in data mart modeling script；
Create initial graph model；
By the parsing result and initial graph model interaction；
It repeats above operation, traverses the SQL statement in all data mart modeling scripts, generate a genetic connection graph model.
Wherein, after the SQL statement in the parsing data mart modeling script, comprising:
Obtain Data source table title and field name, datum target table name and field name, Data source table and data mesh
Mark the relationship between literary name section.
Wherein, the initial graph model of creation specifically includes:
Initial graph model is created in chart database Neo4j.
Wherein, described to include: by the parsing result and initial graph model interaction
Using the Data source table field name and the datum target literary name name section as the initial graph model
Node, the chart database Neo4j is written.
It is wherein, described by the parsing result and initial graph model interaction further include:
Using the Data source table title and the datum target table name as the category of the initial graph model node
Property, the chart database Neo4j is written.
Wherein, described to include: by the parsing result and initial graph model interaction
Using the Data source table field name and the relationship of the datum target literary name name section as the initial artwork
The chart database Neo4j is written in the side of type.
Wherein, including the genetic connection graph model is visualized.
Wherein, the genetic connection graph model be a network diagramming, wherein the network diagramming centered on a node,
Its node is associated with central node, while carrying out color differentiation to different nodes according to the depth of genetic connection.
Wherein, the nodal information in the genetic connection graph model includes: table name, the upstream number of plies, the downstream number of plies, upstream
Table quantity, downstream table quantity, immediately upstream table quantity, direct downstream table quantity and direct downstream literary name section list.
Wherein, the information of each list includes Data source table field name and data in the direct downstream literary name section list
Relationship between object table field name.
Specific embodiment according to the present invention, second aspect, the present invention provide a kind of data blood relationship based on graph model
Relationship storage system characterized by comprising
Parsing module, for parsing the SQL statement in data mart modeling script；
Creation module, for creating an initial graph model in a chart database Neo4j；
Node writing module, for making the Data source table field name and the datum target literary name name section respectively
For the node of the initial graph model, the chart database Neo4j is written；
Attribute writing module, for using the Data source table title and the datum target table name as the initial graph
The chart database Neo4j is written in the attribute of model node；
Relationship writing module, for by the relationship of the Data source table field name and the datum target literary name name section
As the side of the initial graph model, the chart database Neo4j is written；
Spider module generates a genetic connection artwork for traversing the SQL statement parsed in all data mart modeling scripts
Type.
Wherein, the parsing module is also used to obtain Data source table title and field name, datum target table name and word
Relationship between name section, Data source table and datum target literary name section.
Wherein, the nodal information in the genetic connection graph model includes: table name, the upstream number of plies, the downstream number of plies, upstream
Table quantity, downstream table quantity, immediately upstream table quantity, direct downstream table quantity and direct downstream literary name section list.
Specific embodiment according to the present invention, the third aspect, the present invention provide a kind of computer readable storage medium,
On be stored with computer program, when described program is executed by processor realize as above described in any item data based on graph model
Genetic connection storage method.
Specific embodiment according to the present invention, fourth aspect, the present invention provide a kind of electronic equipment, comprising: one or
Multiple processors；Storage device, for storing one or more programs, when one or more of programs are by one or more
When a processor executes, so that one or more of processors realize as above described in any item data blood based on graph model
Edge relationship storage method.
The above scheme of the embodiment of the present invention is directly stored data as the node of figure, relationship, attribute using graph model
Into chart database, without being pre-designed complicated relational data table structure, the design difficulty of such scene is significantly reduced
And complexity；Second, have benefited from the memory computer system of chart database Neo4j and the data structure of optimization, under mass data,
Data blood relationship upstream and downstream level quantity, the statistics of dependence table quantity can be rapidly completed in several milliseconds, and number is rapidly completed
According to the retrieval of field and tables of data dependence.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and be used to explain the principle of the present invention together with specification.It should be evident that the accompanying drawings in the following description is only the present invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.In the accompanying drawings:
Fig. 1 shows a kind of data genetic connection storage method based on graph model of a certain embodiment according to the present invention
Flow chart；
Fig. 2 shows a kind of data genetic connection storage methods based on graph model according to another embodiment of the present invention
Flow chart；
Fig. 3 shows a kind of structure of data genetic connection storage system based on graph model according to an embodiment of the present invention
Schematic diagram；
Fig. 4 shows the structural schematic diagram of the electronic equipment of embodiment according to the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention make into
It is described in detail to one step, it is clear that described embodiments are only a part of the embodiments of the present invention, rather than whole implementation
Example.Based on the embodiments of the present invention, obtained by those of ordinary skill in the art without making creative efforts
All other embodiment, shall fall within the protection scope of the present invention.
The term used in embodiments of the present invention is only to be not intended to be limiting merely for for the purpose of describing particular embodiments
The present invention.In the embodiment of the present invention and the "an" of singular used in the attached claims, " described " and "the"
It is also intended to including most forms, unless the context clearly indicates other meaning, " a variety of " generally comprise at least two.
It should be appreciated that term "and/or" used herein is only a kind of incidence relation for describing affiliated partner, indicate
There may be three kinds of relationships, for example, A and/or B, can indicate: individualism A, exist simultaneously A and B, individualism B these three
Situation.In addition, character "/" herein, typicallys represent the relationship that forward-backward correlation object is a kind of "or".
It will be appreciated that though may be described in embodiments of the present invention using term first, second, third, etc..,
But these ... it should not necessarily be limited by these terms.These terms be only used to by ... distinguish.For example, not departing from implementation of the present invention
In the case where example range, first ... can also be referred to as second ..., and similarly, second ... can also be referred to as the
One ....
Depending on context, word as used in this " if ", " if " can be construed to " ... when " or
" when ... " or " in response to determination " or " in response to detection ".Similarly, context is depended on, phrase " if it is determined that " or " such as
Fruit detection (condition or event of statement) " can be construed to " when determining " or " in response to determination " or " when detection (statement
Condition or event) when " or " in response to detection (condition or event of statement) ".
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
Include, so that commodity or device including a series of elements not only include those elements, but also including not clear
The other element listed, or further include for this commodity or the intrinsic element of device.In the feelings not limited more
Under condition, the element that is limited by sentence "including a ...", it is not excluded that in the commodity or device for including the element also
There are other identical elements.
Big data era, data contain unlimited value.Mobile Internet flourishes, and makes each Internet company long-pending
The user data and business datum of PB rank are tired out.Under powerful requirement drive, the also mature development steadily of big data technology is led to
Cross HDFS, HBase, MongoDB, the storage assemblies such as Kafka, the data having recorded magnanimity and having continued to increase.
Generation, processing fusion, the circulation circulation of data, wither away to final, will form a kind of relationship naturally between data.It borrows
A kind of similar relationship expresses this relationship between data, the referred to as genetic connection of data in mirror human society.
Embodiment 1
Referring to Fig. 1, the embodiment of the present invention provides a kind of data genetic connection storage method based on graph model, this method
Include the following steps:
Parse the SQL statement in data mart modeling script；
Create initial graph model；
By the parsing result and initial graph model interaction；
It repeats above operation, traverses the SQL statement in all data mart modeling scripts, generate a genetic connection graph model.
Wherein, after the SQL statement in the parsing data mart modeling script, comprising:
Obtain Data source table title and field name, datum target table name and field name, Data source table and data mesh
Mark the relationship between literary name section.
Wherein, the initial graph model of creation specifically includes:
Initial graph model is created in chart database Neo4j.
Wherein, described to include: by the parsing result and initial graph model interaction
Using the Data source table field name and the datum target literary name name section as the initial graph model
Node, the chart database Neo4j is written.
It is wherein, described by the parsing result and initial graph model interaction further include:
Using the Data source table title and the datum target table name as the category of the initial graph model node
Property, the chart database Neo4j is written.
Wherein, described to include: by the parsing result and initial graph model interaction
Using the Data source table field name and the relationship of the datum target literary name name section as the initial artwork
The chart database Neo4j is written in the side of type.
Wherein, including the genetic connection graph model is visualized.
Wherein, the genetic connection graph model be a network diagramming, wherein the network diagramming centered on a node,
Its node is associated with central node, while carrying out color differentiation to different nodes according to the depth of genetic connection.
Wherein, the nodal information in the genetic connection graph model includes: table name, the upstream number of plies, the downstream number of plies, upstream
Table quantity, downstream table quantity, immediately upstream table quantity, direct downstream table quantity and direct downstream literary name section list.
Wherein, the information of each list includes Data source table field name and data in the direct downstream literary name section list
Relationship between object table field name.
Embodiment 2
Referring to Fig. 2, the embodiment of the present invention provides a kind of data genetic connection storage method based on graph model, this method
Include the following steps:
S1 parses the SQL statement in data mart modeling script, obtains Data source table title and field name, datum target table
Relationship between title and field name, Data source table and datum target literary name section.
Specifically, using certain way parse data mart modeling script, obtain data warehouse in tables of data, data field it
Between genetic connection, as building the data genetic connection graph model based on graph model data basis.Since the present invention is implemented
The storage for focusing on genetic connection of example, is not illustrated script resolving herein.In the present embodiment, by parsing journey
Sequence parses the SQL statement in data mart modeling script, obtains Data source table title (S_TABLE), Data source table field name (S_
COLUMN), target table name (T_TABLE), object table field name (T_COLUMN), Data source table field and datum target table
Relationship between field.
S2 creates an initial graph model in a chart database Neo4j.
Specifically, the data model according to Neo4j chart database creates after the genetic connection being resolved between data
Initial graph model, it is subsequent to be stored in data in the initial graph model.
S3, using the Data source table field name and the datum target literary name name section as the initial artwork
The chart database Neo4j is written in the node of type.
Specifically, setting the Data source table field name as the node Node_A of the initial graph model, and figure is written
In database Neo4j；The node Node_B of the entitled initial graph model of the datum target literary name section is concurrently set, and is write
Enter in chart database Neo4j.
S4, using the Data source table title and the datum target table name as the category of the initial graph model node
Property, the chart database Neo4j is written.
Specifically, setting the attribute of the entitled initial graph model node Node_A of the Data source table, and figure is written
In database Neo4j；The attribute that the datum target table name is known as the initial graph model node Node_B is concurrently set, and is write
Enter in chart database Neo4j.
S5, using the Data source table field name and the relationship of the datum target literary name name section as the initial graph
The chart database Neo4j is written in the side of model.
Specifically, the relationship between the node Node_A and Node_B of the setting initial graph model, and diagram data is written
In the Neo4j of library.In the present embodiment, using application programming interface, specify chart database object (Graph) link address and
Account name establishes connection with chart database Neo4j；Then the field name of Data source table and object table is designed as described initial
The vertex of graph model creates node object using create method, saves using Data source table and target table name as corresponding
The name attribute of point object；Then using the relationship object of create method creation chart database Neo4j, the ginseng of the relationship object
Number specified first is data source field, second parameter is appointed as the description in ' to ' representation relation direction, third parameter refers to
It is set to datum target field.
S6 repeats the above steps, and traversal parses the SQL statement in all data mart modeling scripts, generates a genetic connection figure
Model.
Specifically, the S1-S5 that repeats the above steps, traversal parses the SQL statement in all data mart modeling scripts, is owned
Data source table and data object table field, table name, and using it as the node (Nodes) of the initial graph model, pass
It is (Relationships), attribute (Properties), and is written in chart database Neo4j, forms one big complete
Figure, i.e. genetic connection graph model.
The genetic connection graph model is a network diagramming, wherein the network diagramming is centered on a node, other nodes
It is associated with central node, while color differentiation is carried out to different nodes according to the depth of genetic connection.The genetic connection
Nodal information in graph model include: table name, the upstream number of plies, the downstream number of plies, upstream table quantity, downstream table quantity, directly on
Swim table quantity, direct downstream table quantity and direct downstream literary name section list.Each list in the direct downstream literary name section list
Information include relationship between Data source table field name and datum target literary name name section.
Further, the data genetic connection storage method based on graph model includes visualizing the genetic connection
Graph model.Specifically, the Cypher query grammar based on Neo4j obtains data, completed in combination with front end frame vue.js.
Data genetic connection storage method provided in an embodiment of the present invention based on graph model is direct using initial graph model
Using data as the node of figure, relationship, attribute storage into chart database, without being pre-designed complicated relational data table knot
Structure significantly reduces the design difficulty and complexity of such scene；Memory computer system and optimization based on chart database Neo4j
Data structure data blood relationship upstream and downstream level quantity, dependence table number can be rapidly completed in several milliseconds under mass data
The statistics of amount, and the retrieval of data field and tables of data dependence is rapidly completed；In combination with good visualization interface
Functional Design, user are clicked by mouse without writing code and quickly can check and search for tables of data, intuitively see data
Between blood relationship flow to relationship, blood relationship relies on the key messages such as level.
Embodiment 3
Referring to Fig. 3, the embodiment of the present invention provides a kind of data genetic connection storage system 200 based on graph model, it should
System 200 includes: parsing module 210, creation module 220, node writing module 230, attribute writing module 240, relationship write-in
Module 250 and spider module 260.
The parsing module 210 obtains Data source table title and word for parsing the SQL statement in data mart modeling script
Relationship between name section, datum target table name and field name, Data source table and datum target literary name section.Specifically, institute
It states parsing module 210 and data mart modeling script is parsed using certain way, between the tables of data, data field in acquisition data warehouse
Genetic connection, as building the data genetic connection graph model based on graph model data basis.Due to the embodiment of the present invention
The storage for focusing on genetic connection, herein not to script parsing illustrate.In the present embodiment, the parsing module 210 is logical
The SQL statement in analysis program parsing data mart modeling script is crossed, Data source table title (S_TABLE), Data source table field are obtained
Title (S_COLUMN), target table name (T_TABLE), object table field name (T_COLUMN), Data source table field and number
According to the relationship between target literary name section.
The creation module 220, for creating an initial graph model in a chart database Neo4j.Specifically, when parsing
To after the genetic connection between data, the creation module 220 creates initial graph according to the data model of Neo4j chart database
Model, it is subsequent to be stored in data in the initial graph model.
The node writing module 230 is used for the Data source table field name and the datum target literary name section name
Claim the node respectively as the initial graph model, the chart database Neo4j is written.Specifically, the node writing module
230 set the Data source table field name as the node Node_A of the initial graph model, and chart database Neo4j is written
In；The node Node_B of the entitled initial graph model of the datum target literary name section is concurrently set, and chart database is written
In Neo4j.
The attribute writing module 240, for using the Data source table title and the datum target table name as institute
The chart database Neo4j is written in the attribute for stating initial graph model node.Specifically, the attribute writing module 240 sets institute
The attribute of the entitled initial graph model node Node_A of Data source table is stated, and is written in chart database Neo4j；It concurrently sets
The datum target table name is known as the attribute of the initial graph model node Node_B, and is written in chart database Neo4j.
The relationship writing module 250 is used for the Data source table field name and the datum target literary name section name
Side of the relationship of title as the initial graph model, is written the chart database Neo4j.Specifically, the relationship writing module
Relationship between the node Node_A and Node_B of the 250 setting initial graph models, and be written in chart database Neo4j.This
In inventive embodiments, using application programming interface, the link address and account name of chart database object (Graph) are specified,
Connection is established with chart database Neo4j；Then the field name of Data source table and object table is designed as the initial graph model
Vertex, node object is created using create method, using Data source table and target table name as corresponding node object
Name attribute；Then using the relationship object of create method creation chart database Neo4j, the parameter of the relationship object is specified
First is data source field, second parameter is appointed as the description in ' to ' representation relation direction, third parameter is appointed as counting
According to aiming field.
The spider module 260 generates a blood relationship and closes for traversing the SQL statement parsed in all data mart modeling scripts
It is graph model.Specifically, the traversal of spider module 260 parses the SQL statement in all data mart modeling scripts, all numbers are obtained
According to source table and data object table field, table name, and using it as the node (Nodes) of the initial graph model, relationship
(Relationships), attribute (Properties), and be written in chart database Neo4j, a big complete figure is formed,
That is genetic connection graph model.
The genetic connection graph model is a network diagramming, wherein the network diagramming is centered on a node, other nodes
It is associated with central node, while color differentiation is carried out to different nodes according to the depth of genetic connection.The genetic connection
Nodal information in graph model include: table name, the upstream number of plies, the downstream number of plies, upstream table quantity, downstream table quantity, directly on
Swim table quantity, direct downstream table quantity and direct downstream literary name section list.Each list in the direct downstream literary name section list
Information include relationship between Data source table field name and datum target literary name name section.
Further, the data genetic connection storage system 200 based on graph model includes a visual presentation module
270, for visualizing the genetic connection graph model.Specifically, the visual presentation module 270 is based on Neo4j's
Cypher query grammar obtains data, completes to visualize in combination with front end frame vue.js.In the present embodiment, based on figure
After the data genetic connection building of model, by designing a kind of matched visualization interface, to be based on described in inquiring and analyzing
The data genetic connection of graph model.
The genetic connection graph model that the visualization interface is shown defaults the section centered on some tables of data node
Point uses red display；Show all tables of data nodes being associated in the form of network diagramming, is connected by grey with the arrow
Wiring connects each tables of data node, and according to the depth of genetic connection, the color of nodes at different levels shoals step by step.Wherein, the section
Point is labeled with the data table name of this node on behalf with the displaying of round icon, icon；Tables of data includes Data source table and data mesh
Mark table.The node can pass through click after play frame show relevant information, comprising: table name, the upstream number of plies, the downstream number of plies, on
Swim table quantity, downstream table quantity, immediately upstream table quantity, direct downstream table quantity and direct downstream literary name section list.It is described straight
It connects in the literary name section list of downstream, clicks some table, the relationship between expansion display data field, comprising: Data source table field name
Title, datum target literary name name section and the connecting line with direction arrow.
The visualization interface further comprises the search box that can input text, clicks search after input table name and presses
Button, genetic connection figure will repaint, and directly show the relevant genetic connection figure of specific tables of data of search.It is described visual
Data genetic connection figure based on graph model such as can be dragged, be amplified, being reduced at the operation, to make user that can intuitively check some
Tables of data flows to situation entire data link.
Data genetic connection storage system 200 provided in an embodiment of the present invention based on graph model is straight using initial graph model
It connects using data as the node of figure, relationship, attribute storage into chart database, without being pre-designed complicated relational data table
Structure significantly reduces the design difficulty and complexity of such scene；Memory computer system based on chart database Neo4j and excellent
Data blood relationship upstream and downstream level quantity, dependence table can be rapidly completed under mass data in the data structure of change in several milliseconds
The statistics of quantity, and the retrieval of data field and tables of data dependence is rapidly completed；In combination with good visualization circle
Face Functional Design, user are clicked by mouse without writing code and quickly can check and search for tables of data, intuitively see number
Blood relationship between flows to relationship, blood relationship relies on the key messages such as level.
Embodiment 4
As shown in figure 4, the present embodiment provides a kind of electronic equipment, comprising: at least one processor；And with it is described extremely
The memory of few processor communication connection；Wherein,
The memory is stored with the instruction that can be executed by one processor, and described instruction is by described at least one
Manage device execute so that at least one described processor can:
The SQL statement in data mart modeling script is parsed, Data source table title and field name, datum target table name are obtained
And the relationship between field name, Data source table and datum target literary name section；
An initial graph model is created in a chart database Neo4j；
Using the Data source table field name and the datum target literary name name section as the initial graph model
Node, the chart database Neo4j is written；
Using the Data source table title and the datum target table name as the category of the initial graph model node
Property, the chart database Neo4j is written；
Using the Data source table field name and the relationship of the datum target literary name name section as the initial artwork
The chart database Neo4j is written in the side of type；
It repeats the above steps, traversal parses the SQL statement in all data mart modeling scripts, generates a genetic connection artwork
Type.
Embodiment 4
The embodiment of the present disclosure provides a kind of nonvolatile computer storage media, and the computer storage medium is stored with
Computer executable instructions, the loophole component version which can be performed in above-mentioned any means embodiment are looked into
Look for method.
It should be noted that the above-mentioned computer-readable medium of the disclosure can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In open, computer-readable signal media may include in a base band or as the data-signal that carrier wave a part is propagated,
In carry computer-readable program code.The data-signal of this propagation can take various forms, including but not limited to
Electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable and deposit
Any computer-readable medium other than storage media, the computer-readable signal media can send, propagate or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF (radio frequency) etc. are above-mentioned
Any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof
Machine program code, above procedure design language include object oriented program language-such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present disclosure can be realized by way of software, can also be by hard
The mode of part is realized.Wherein, the title of unit does not constitute the restriction to the unit itself under certain conditions, for example, the
One acquiring unit is also described as " obtaining the unit of at least two internet protocol addresses ".
Claims (10)
1. a kind of data genetic connection storage method based on graph model characterized by comprising
Parse the SQL statement in data mart modeling script；
Create initial graph model；
By the parsing result and initial graph model interaction；
It repeats above operation, traverses the SQL statement in all data mart modeling scripts, generate a genetic connection graph model.
2. the method according to claim 1, wherein wherein it is described parsing data mart modeling script in SQL statement
Later, comprising:
Obtain Data source table title and field name, datum target table name and field name, Data source table and datum target table
Relationship between field.
3. according to the method described in claim 2, it is characterized in that, wherein the initial graph model of creation specifically includes:
Initial graph model is created in chart database Neo4j.
4. according to the method described in claim 3, it is characterized in that, wherein described close the parsing result and initial graph model
Connection includes:
Using the Data source table field name and the datum target literary name name section as the section of the initial graph model
The chart database Neo4j is written in point.
5. according to the method described in claim 3, it is characterized in that, wherein described close the parsing result and initial graph model
Connection further include:
Using the Data source table title and the datum target table name as the attribute of the initial graph model node, write
Enter the chart database Neo4j.
6. according to the method described in claim 3, it is characterized in that, wherein described close the parsing result and initial graph model
Connection includes:
Using the Data source table field name and the relationship of the datum target literary name name section as the initial graph model
The chart database Neo4j is written in side.
7. a kind of data genetic connection storage system based on graph model characterized by comprising
Parsing module, for parsing the SQL statement in data mart modeling script；
Creation module, for creating an initial graph model in a chart database Neo4j；
Node writing module, for using the Data source table field name and the datum target literary name name section as institute
The chart database Neo4j is written in the node for stating initial graph model；
Attribute writing module, for using the Data source table title and the datum target table name as the initial graph model
The chart database Neo4j is written in the attribute of node；
Relationship writing module, for using the Data source table field name and the relationship of the datum target literary name name section as
The chart database Neo4j is written in the side of the initial graph model；
Spider module generates a genetic connection graph model for traversing the SQL statement parsed in all data mart modeling scripts.
8. system according to claim 7, which is characterized in that the parsing module be also used to obtain Data source table title and
Relationship between field name, datum target table name and field name, Data source table and datum target literary name section.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that described program is by processor
Such as method described in any one of claims 1 to 6 is realized when execution.
10. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs, when one or more of programs are by one or more of processing
When device executes, so that one or more of processors realize such as method described in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910385135.7A CN110245270A (en) | 2019-05-09 | 2019-05-09 | Data genetic connection storage method, system, medium and equipment based on graph model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910385135.7A CN110245270A (en) | 2019-05-09 | 2019-05-09 | Data genetic connection storage method, system, medium and equipment based on graph model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110245270A true CN110245270A (en) | 2019-09-17 |
Family
ID=67883891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910385135.7A Pending CN110245270A (en) | 2019-05-09 | 2019-05-09 | Data genetic connection storage method, system, medium and equipment based on graph model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245270A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781520A (en) * | 2019-10-30 | 2020-02-11 | 上海观安信息技术股份有限公司 | Sensitive table group discovery method and system |
| CN111427997A (en) * | 2020-03-09 | 2020-07-17 | 北京明略软件***有限公司 | Method and device for displaying blood relationship, computer storage medium and terminal |
| CN111475682A (en) * | 2020-04-06 | 2020-07-31 | 武汉智领云科技有限公司 | Intelligent operation and maintenance platform based on super-large-scale data system |
| CN111538743A (en) * | 2020-04-22 | 2020-08-14 | 电子科技大学 | SQL-based data blood relationship analysis method and system |
| CN111723253A (en) * | 2020-05-25 | 2020-09-29 | 贵州华泰智远大数据服务有限公司 | Data blood relationship query method and query system based on graph database |
| CN111782641A (en) * | 2020-06-28 | 2020-10-16 | 中国工商银行股份有限公司 | Data error repairing method and system |
| CN112749186A (en) * | 2021-01-22 | 2021-05-04 | 广州虎牙科技有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN112749229A (en) * | 2019-10-31 | 2021-05-04 | 北京国双科技有限公司 | Data conversion method, device, storage medium and electronic equipment |
| CN112783887A (en) * | 2019-11-07 | 2021-05-11 | 北京沃东天骏信息技术有限公司 | Data processing method and device based on data warehouse |
| CN113094776A (en) * | 2021-04-19 | 2021-07-09 | 城云科技（中国）有限公司 | Method and system for constructing visual component model data and electronic equipment |
| CN113127442A (en) * | 2020-01-10 | 2021-07-16 | 马上消费金融股份有限公司 | Visualization method and device of data model and storage medium |
| CN113486108A (en) * | 2021-07-06 | 2021-10-08 | 建信金融科技有限责任公司 | Data processing method and device, electronic equipment and computer readable medium |
| CN113672674A (en) * | 2021-07-15 | 2021-11-19 | 浙江大华技术股份有限公司 | Method, electronic device and storage medium for automatically arranging service flow |
| CN113672774A (en) * | 2021-07-29 | 2021-11-19 | 国电南瑞科技股份有限公司 | Distribution network equipment topology coloring method and device based on distribution cloud master station and graph database |
| CN116010444A (en) * | 2023-03-27 | 2023-04-25 | 中国人民解放军国防科技大学 | Low-code interactive graph query statement construction method |
| CN117688217A (en) * | 2024-02-02 | 2024-03-12 | 北方健康医疗大数据科技有限公司 | System, method and medium for realizing data blood relationship structure based on directed graph |
| CN117786023A (en) * | 2024-02-28 | 2024-03-29 | 北方健康医疗大数据科技有限公司 | Medical data blood-edge analysis method, system, terminal and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3327991A1 (en) * | 2016-11-29 | 2018-05-30 | Alcatel Lucent | Storage of coverage-related information of a telecommunication network |
| CN108170847A (en) * | 2018-01-18 | 2018-06-15 | 国网福建省电力有限公司 | A kind of big data storage method based on Neo4j chart databases |
| CN108804701A (en) * | 2018-06-19 | 2018-11-13 | 苏州大学 | Personage's portrait model building method based on social networks big data |
| CN109446279A (en) * | 2018-10-15 | 2019-03-08 | 顺丰科技有限公司 | Based on neo4j big data genetic connection management method, system, equipment and storage medium |
-
2019
- 2019-05-09 CN CN201910385135.7A patent/CN110245270A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3327991A1 (en) * | 2016-11-29 | 2018-05-30 | Alcatel Lucent | Storage of coverage-related information of a telecommunication network |
| CN108170847A (en) * | 2018-01-18 | 2018-06-15 | 国网福建省电力有限公司 | A kind of big data storage method based on Neo4j chart databases |
| CN108804701A (en) * | 2018-06-19 | 2018-11-13 | 苏州大学 | Personage's portrait model building method based on social networks big data |
| CN109446279A (en) * | 2018-10-15 | 2019-03-08 | 顺丰科技有限公司 | Based on neo4j big data genetic connection management method, system, equipment and storage medium |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781520A (en) * | 2019-10-30 | 2020-02-11 | 上海观安信息技术股份有限公司 | Sensitive table group discovery method and system |
| CN112749229A (en) * | 2019-10-31 | 2021-05-04 | 北京国双科技有限公司 | Data conversion method, device, storage medium and electronic equipment |
| CN112783887A (en) * | 2019-11-07 | 2021-05-11 | 北京沃东天骏信息技术有限公司 | Data processing method and device based on data warehouse |
| CN113127442A (en) * | 2020-01-10 | 2021-07-16 | 马上消费金融股份有限公司 | Visualization method and device of data model and storage medium |
| CN113127442B (en) * | 2020-01-10 | 2023-12-22 | 马上消费金融股份有限公司 | Method, device and storage medium for visualizing data model |
| CN111427997A (en) * | 2020-03-09 | 2020-07-17 | 北京明略软件***有限公司 | Method and device for displaying blood relationship, computer storage medium and terminal |
| CN111475682A (en) * | 2020-04-06 | 2020-07-31 | 武汉智领云科技有限公司 | Intelligent operation and maintenance platform based on super-large-scale data system |
| CN111538743A (en) * | 2020-04-22 | 2020-08-14 | 电子科技大学 | SQL-based data blood relationship analysis method and system |
| CN111538743B (en) * | 2020-04-22 | 2023-08-18 | 电子科技大学 | SQL-based data blood relationship analysis method and system |
| CN111723253A (en) * | 2020-05-25 | 2020-09-29 | 贵州华泰智远大数据服务有限公司 | Data blood relationship query method and query system based on graph database |
| CN111782641A (en) * | 2020-06-28 | 2020-10-16 | 中国工商银行股份有限公司 | Data error repairing method and system |
| CN111782641B (en) * | 2020-06-28 | 2023-07-28 | 中国工商银行股份有限公司 | Data error repairing method and system |
| CN112749186A (en) * | 2021-01-22 | 2021-05-04 | 广州虎牙科技有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN112749186B (en) * | 2021-01-22 | 2024-02-09 | 广州虎牙科技有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN113094776A (en) * | 2021-04-19 | 2021-07-09 | 城云科技（中国）有限公司 | Method and system for constructing visual component model data and electronic equipment |
| CN113486108A (en) * | 2021-07-06 | 2021-10-08 | 建信金融科技有限责任公司 | Data processing method and device, electronic equipment and computer readable medium |
| CN113672674A (en) * | 2021-07-15 | 2021-11-19 | 浙江大华技术股份有限公司 | Method, electronic device and storage medium for automatically arranging service flow |
| CN113672774A (en) * | 2021-07-29 | 2021-11-19 | 国电南瑞科技股份有限公司 | Distribution network equipment topology coloring method and device based on distribution cloud master station and graph database |
| CN113672774B (en) * | 2021-07-29 | 2023-09-29 | 国电南瑞科技股份有限公司 | Distribution network equipment topology coloring method and device based on distribution cloud master station and graph database |
| CN116010444A (en) * | 2023-03-27 | 2023-04-25 | 中国人民解放军国防科技大学 | Low-code interactive graph query statement construction method |
| CN117688217A (en) * | 2024-02-02 | 2024-03-12 | 北方健康医疗大数据科技有限公司 | System, method and medium for realizing data blood relationship structure based on directed graph |
| CN117786023A (en) * | 2024-02-28 | 2024-03-29 | 北方健康医疗大数据科技有限公司 | Medical data blood-edge analysis method, system, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110245270A (en) | Data genetic connection storage method, system, medium and equipment based on graph model | |
| US10824632B2 (en) | System and method for generating event visualizations | |
| US11580680B2 (en) | Systems and interactive user interfaces for dynamic retrieval, analysis, and triage of data items | |
| US11681694B2 (en) | Systems and methods for grouping and enriching data items accessed from one or more databases for presentation in a user interface | |
| US10599313B2 (en) | System for high volume data analytic integration and channel-independent advertisement generation | |
| US10311050B2 (en) | Crowdsourced discovery of paths in a knowledge graph | |
| Dourish | No SQL: The shifting materialities of database technology | |
| US7734619B2 (en) | Method of presenting lineage diagrams representing query plans | |
| US9466041B2 (en) | User selected flow graph modification | |
| US9223549B1 (en) | User interface generation using a model layer | |
| US20150007084A1 (en) | Chaining applications | |
| US11727325B2 (en) | User interface to analyze and navigate through decision logic | |
| US10552423B2 (en) | Semantic tagging of nodes | |
| Lavbič et al. | Facilitating information system development with Panoramic view on data | |
| US10585926B2 (en) | Managing structuring of large sets of unstructured data | |
| US11275485B2 (en) | Data processing pipeline engine | |
| US20190384615A1 (en) | Containerized runtime environments | |
| EP3786810A1 (en) | Automatic generation of computing artifacts for data analysis | |
| US20190384577A1 (en) | Customizing operator nodes for graphical representations of data processing pipelines | |
| US20170270697A1 (en) | Hierarchical information visualizer and editor | |
| JP2010128870A (en) | Data processor | |
| Öztürk et al. | Onyx: A new Canvas‐based tool for visualizing biomedical and health ontologies | |
| Canbaz | Visuall: a quickly customizable library for jumpstarting visual graph analysis components | |
| Tailhardat et al. | NORIA UI: Efficient Incident Management on Large-Scale ICT Systems Represented as Knowledge Graphs | |
| da Costa Capa | Data Center Visualization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190917 |
|
| RJ01 | Rejection of invention patent application after publication |