CN112528037A - Edge relation prediction method, device, equipment and storage medium based on knowledge graph - Google Patents
Edge relation prediction method, device, equipment and storage medium based on knowledge graph Download PDFInfo
- Publication number
- CN112528037A CN112528037A CN202011406864.5A CN202011406864A CN112528037A CN 112528037 A CN112528037 A CN 112528037A CN 202011406864 A CN202011406864 A CN 202011406864A CN 112528037 A CN112528037 A CN 112528037A
- Authority
- CN
- China
- Prior art keywords
- knowledge
- graph
- nodes
- node
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 239000013598 vector Substances 0.000 claims abstract description 88
- 238000012545 processing Methods 0.000 claims abstract description 27
- 230000002776 aggregation Effects 0.000 claims abstract description 18
- 238000004220 aggregation Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 14
- 230000015654 memory Effects 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000000873 masking effect Effects 0.000 claims description 2
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000010276 construction Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 208000024891 symptom Diseases 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 206010028748 Nasal obstruction Diseases 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010033425 Pain in extremity Diseases 0.000 description 1
- 206010057190 Respiratory tract infections Diseases 0.000 description 1
- 206010046306 Upper respiratory tract infection Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 208000020029 respiratory tract infectious disease Diseases 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a knowledge graph-based edge relation prediction method, a knowledge graph-based edge relation prediction device, knowledge graph-based edge relation prediction equipment and a storage medium, and relates to the technical field of deep learning and the technical field of knowledge graphs. The method comprises the following steps: acquiring a first knowledge graph, wherein the first knowledge graph is constructed based on structured data and unstructured data of a target knowledge domain; acquiring a first knowledge representation model, wherein the first knowledge representation model is generated by training a first BERT model through a corpus data set of the target knowledge field; and inputting the marked word vectors and the position word vectors of the plurality of nodes of the first knowledge graph into a second BERT model for information aggregation processing to obtain the edge relation information between the plurality of nodes of the first knowledge graph and respective neighbor nodes. By using the method and the device, the side relation information among the nodes in the knowledge graph can be obtained.
Description
Technical Field
The application relates to the technical field of deep learning and the technical field of knowledge maps, in particular to a knowledge map-based edge relationship prediction method, device, equipment and storage medium.
Background
Knowledge Graph (Knowledge Graph) generally refers to a semantic network capable of revealing relationships between entities, and based on means such as data mining, information processing, Graph drawing and the like, a visual Graph is used for vividly displaying a complex Knowledge field, so that the development rule of the Knowledge field can be reflected to a certain extent. Knowledge representation (knowledge representation) is a description of knowledge that includes a data structure that is computer-acceptable for describing knowledge. Knowledge Inference (Knowledge Inference) generally refers to a process of simulating an intelligent human Inference mode in a computer or an intelligent system, and using formalized Knowledge to think about and solve a problem by a machine according to an Inference control strategy. In the knowledge inference calculation, inference prediction needs to be carried out by relying on a complete knowledge base, taking the medical knowledge field as an example, diagnosis, medical advice, medication and the like in the medical field all depend on artificial knowledge storage and experience, corresponding conclusions are obtained through the inference calculation, and the inference prediction can be realized by using knowledge inference modeling, wherein the currently mainly adopted schemes comprise 1) knowledge inference based on rules, 2) knowledge inference based on knowledge graph spectrogram walking and 3) knowledge inference based on distributed representation, and the schemes need to rely on complete knowledge graph information to realize inference prediction and have higher requirements on the consistency and completeness of the knowledge graph. However, the construction cost of the knowledge graph in the current professional field is high, and completeness is difficult to guarantee, so that the accuracy of knowledge inference calculation is low.
Disclosure of Invention
The application provides a method, a device, equipment and a storage medium for predicting an edge relationship based on a knowledge graph, which are used for solving at least one technical problem.
According to a first aspect of the present application, there is provided a method for knowledge-graph-based edge relation prediction, comprising:
acquiring a first knowledge graph, wherein the first knowledge graph is constructed based on structured data and unstructured data of a target knowledge domain;
acquiring a first knowledge representation model, wherein the first knowledge representation model is generated by training a first BERT model through a corpus data set of a target knowledge field;
inputting the first knowledge graph into a first knowledge representation model to obtain the output labeled word vectors of a plurality of nodes in the first knowledge graph;
respectively determining position word vectors of all nodes according to the distances between the nodes in the first knowledge graph and respective neighbor nodes;
and inputting the marked word vectors and the position word vectors of the plurality of nodes of the first knowledge graph into the second BERT model for information aggregation processing to obtain the edge relation information between the plurality of nodes of the first knowledge graph and respective neighbor nodes.
According to a second aspect of the present application, there is provided a knowledge-graph-based edge relation prediction apparatus comprising:
the system comprises a map acquisition module, a map generation module and a map generation module, wherein the map acquisition module is used for acquiring a first knowledge map, and the first knowledge map is constructed based on structured data and unstructured data of a target knowledge field;
the model acquisition module is used for acquiring a first knowledge representation model, and the first knowledge representation model is generated by training a first BERT model through a corpus data set of a target knowledge field;
the tagged word vector acquisition module is used for inputting the first knowledge graph into the first knowledge representation model to obtain tagged word vectors of a plurality of nodes in the output first knowledge graph;
the position word vector determining module is used for respectively determining the position word vectors of all the nodes according to the distances between the plurality of nodes and respective neighbor nodes in the first knowledge graph;
and the side relation information acquisition module is used for inputting the marked word vectors and the position word vectors of the nodes of the first knowledge graph into the second BERT model to carry out information aggregation processing so as to obtain side relation information between the nodes of the first knowledge graph and respective neighbor nodes.
According to a third aspect of the present application, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to a fourth aspect of the present application, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described above.
According to a fifth aspect of the present application, there is provided a computer program product comprising computer instructions which, when executed by a processor, implement the method as described above.
According to the embodiment of the application, the unstructured data are added when the knowledge graph is constructed, graph information can be comprehensive as much as possible, the completeness of the constructed knowledge graph is improved, the BERT model is used for conducting aggregation processing on word vectors of nodes to obtain side relation information among the nodes in the graph, the side relation information in knowledge inference is supplemented, and the accuracy of side relation prediction is improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a block flow diagram of a method for knowledge-graph based edge relationship prediction in an embodiment of the present application;
FIG. 2 is a schematic illustration of a medical knowledge-map of an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating processing logic for obtaining inter-node relationship information according to an embodiment of the present application;
FIG. 4 is a block diagram of an apparatus for knowledge-graph based edge relationship prediction in accordance with an embodiment of the present application;
FIG. 5 is a block diagram of an electronic device implementing a method for knowledge-graph based edge relationship prediction in accordance with an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 shows a flow chart of a method for predicting an edge relationship based on a knowledge graph according to an embodiment of the present application, where the method includes:
s101, acquiring a first knowledge graph, wherein the first knowledge graph is constructed based on structured data and unstructured data of a target knowledge field;
s102, acquiring a first knowledge representation model, wherein the first knowledge representation model is generated by training a first BERT model through a corpus data set of a target knowledge field;
s103, inputting the first knowledge graph into the first knowledge representation model to obtain the output labeled word vectors of a plurality of nodes in the first knowledge graph;
s104, respectively determining position word vectors of each node according to the distance between each node and each neighbor node in the first knowledge graph;
and S105, inputting the marked word vectors and the position word vectors of the nodes of the first knowledge graph into the second BERT model for information aggregation processing to obtain the edge relation information between the nodes of the first knowledge graph and respective neighbor nodes.
According to the embodiment of the application, the unstructured data are added during the construction of the knowledge graph, the information loss during the construction of the knowledge graph can be reduced, the information in the graph is comprehensive as much as possible, the completeness of the constructed knowledge graph is improved, the word vectors of the nodes are subjected to aggregation processing by using the BERT model, the side relation information among the nodes in the graph can be obtained, the side relation information in knowledge inference is supplemented, the accuracy of side relation prediction can be improved, and the accuracy of knowledge inference can be improved.
The BERT model, which is an unsupervised Natural Language Processing (NLP) training model, is referred to collectively as "Bidirectional Encoder replication from converters," which is a bi-directional Encoder of a converter "Transformer," and is structurally a coding portion of a Transformer tool. The training of the BERT model framework can comprise two stages of model pre-training and model fine-tuning (fine-tune) in specific tasks, in the model pre-training stage, a large amount of data can be used for training the model to conduct unsupervised training, and in the model fine-tuning stage, the model is fine-tuned according to the specific tasks so as to achieve good effects.
In some embodiments of the present application, optionally, after the first knowledge graph is input into the first knowledge representation model, for a first node in the first knowledge graph, the first knowledge representation model calculates a word vector (embedding) of the first node and word vectors of one or more neighbor nodes of the first node, and concatenates the calculated word vectors to obtain a token word vector (token embedding) of the first node. By processing each node in the knowledge graph in the above manner, a labeled word vector of each node can be obtained for calculation in the subsequent steps.
In some embodiments of the present application, optionally, a distance between the first node and the neighboring node is a number of edges between the first node and one or more neighboring nodes of the first node, a position word vector (position embedding) of the first node is the same as a dimension of a tag word vector of the first node, and each dimension corresponds to the number of edges. By processing each node in the knowledge graph in the above manner, the position word vector of each node can be obtained for calculation in the subsequent steps.
In some embodiments of the present application, optionally, the information aggregation process includes learning semantic information and graph structure relationships of a plurality of nodes of the first knowledge-graph through an Attention Mask (Attention Mask) tool of the second BERT model, obtaining edge relationship information between the nodes in the first knowledge-graph. Information aggregation can be realized by using the BERT model as an aggregation function, and the information of the edge relation among the nodes can be obtained.
In some embodiments of the present application, optionally, after obtaining the edge relation information between the plurality of nodes of the first knowledge-graph and the respective neighboring nodes, the method further includes: and inputting the edge relation information into the activation function to calculate the confidence of each edge relation, and reserving the edge relation with the confidence greater than or equal to the threshold value. The edge relation with too low confidence can not be used, the quality of the edge relation information can be improved, and the accuracy of knowledge reasoning can be improved.
In some embodiments of the present application, optionally, the first knowledge-map comprises a medical knowledge-map; the structured data comprises different types of entities in the medical field and attribute data related to each entity; the unstructured data comprises long text description data of the medical field; the corpus data set of the target knowledge domain includes chinese medical corpus long text. According to the embodiment of the application, the existing corpus data in the medical and medical field is utilized, the required medical field knowledge can be acquired through the structured data and non-structured data mining technology, and the data acquisition cost is reduced.
In some embodiments of the present application, optionally, the first BERT model is a BERT model to be trained, and the second BERT model is a pre-trained BERT model. The first BERT model and the second BERT model are not the same model, and can respectively complete the processing of the corresponding steps.
By utilizing at least one embodiment of the application, the graph neural network can be constructed based on the entity semantics and the incomplete knowledge base structure information, the side relation information of the knowledge base is supplemented, and the accuracy of side relation prediction is improved.
While various implementations of the embodiments of the present application are described above, at least one of the embodiments described above may be applied to various knowledge fields, such as news fields, construction fields, medical fields, military fields, and so on. The processing procedure of the embodiment of the present application is described below by taking "medical field" as an example and by a specific example. The data of the corresponding domain can be adopted by the application of other knowledge domains, and the specific processing mode and the process are similar.
Fig. 2 schematically shows a schematic diagram of a medical knowledge-graph in the medical field, and in order to implement the medical knowledge-graph-based edge relation prediction method of the embodiment of the present application, the following operation steps can be performed.
Construction of medical knowledge map (or called medical knowledge map)
For the application of knowledge reasoning, a domain knowledge graph needs to be constructed. The present embodiment builds a required medical domain knowledge graph based on structured data and unstructured data.
Acquiring structured data and unstructured data.
The structured data can be captured from authoritative medical verticals, and merged into a map after necessary data governance and processing.
The unstructured data can be obtained from medical authority books, and medical knowledge in the unstructured data can be mined by using machine learning technologies such as entity recognition, edge relation mining and the like.
And secondly, constructing a medical knowledge graph based on the acquired structured data and the acquired unstructured data.
Referring to fig. 2, a constructed knowledge graph may include different types of entities and associated attributes of each entity, the entities may include entities such as diseases, symptoms, examinations, surgeries, signs, etc., wherein some of the entity attributes may contain structured descriptions and unstructured descriptions, e.g., the primary "symptom" of a "disease" contains both structured symptom entities and unstructured long text descriptions, which may be stored with two attribute fields, respectively.
(II) generating a medical knowledge representation model
In this embodiment, the BERT model may be trained using a chinese medical corpus data set, a chinese medical corpus long text may be input to the BERT model frame to be trained, and unsupervised training learning may be performed, and the generated model may learn semantic and entity related information about the medical field, which may be referred to as a medical knowledge representation model, and the output of the model is a word vector of the medical text.
For example, a plurality of medical terms with similar or frequently-occurring semantics are input into a trained medical knowledge representation model, the similarity of the obtained word vectors is high, while the similarity of the obtained word vectors is low when the terms with non-similar semantics are input, such as: the word vectors corresponding to "cold" and "cold" (or "upper respiratory infection") are relatively close, while the word vectors corresponding to "cold" and "leg pain" are less similar.
(III) obtaining the edge relation information between the nodes
Fig. 3 schematically shows a logic diagram of a processing procedure for acquiring inter-node edge relationship information according to an embodiment of the present application. Firstly, word vectors are calculated for the information of the nodes and the neighbor nodes in the medical knowledge graph by using the medical knowledge representation model, and the word vectors are spliced.
For example, "cold" is a disease node, and its attribute nodes may include a nose node of a disease occurrence location and a nasal obstruction node, and word vectors may be calculated for the above 3 node information, and then spliced together, for example, the word vectors of the 3 nodes are recorded as A, B, C, and the spliced word vectors are obtained as "a (separator) B (separator) C (separator)", and used as a marker word vector of "cold".
Wherein, the cold node is a central node (or called core node), and the nose node and the nasal obstruction node are neighbor nodes (or called adjacent nodes) of the cold. The number of the nodes adjacent to the splicing is determined according to the number of first-order nodes around the central node, wherein the first-order nodes are nodes with edges directly connected, and the first-order nodes do not include indirectly connected nodes. It should be noted that each node in the graph may be regarded as a central node, and its neighboring nodes are neighboring nodes corresponding to the central node.
Then, a position word vector representation is obtained according to the distance between the node and the adjacent node, wherein the distance between the nodes can be represented by the number of spacing edges between the nodes.
Specifically, the dimension of the position word vector is consistent with that of the aforementioned label word vector, and the number of the interval edges is recorded at positions with different dimensions, for example, the aforementioned "ABC", where a is a core node, B, C is an adjacent node, B is 1 edge apart from a, C is 2 edges apart from a, and the dimension of the label word vector is 3, then the following can be obtained:
the position word vector for A is [0,0,0],
the position word vector for B is [0,1,0],
the position word vector for C is [0,0,2 ].
Then, the tagged word vectors and the position word vectors of the nodes are transmitted into a pre-trained BERT model for information aggregation (or called information aggregation), so that the interrelations between the central node and the neighboring nodes, namely the edge relations between the nodes, can be captured, for example, whether node a and node B have synonym relations, whether node B is a symptom of node a, and the like.
The information aggregation processing is to learn semantic information and graph structure relations of a plurality of nodes of the graph through an attention covering tool of the BERT model, obtain edge relation information among the nodes in the graph, and realize edge relation modeling among the nodes of the graph.
And finally, transmitting the output result into an activation function, such as a sigmoid function, to calculate the confidence level of the edge relationship among the nodes, wherein the edge relationship is considered to exist when the confidence level is greater than or equal to a threshold value.
For example, assuming that a central node is A, a node of an edge relation to be predicted is X, and a neighbor node of A is B, C, introducing A, B, C, X nodes into a knowledge representation model to calculate a tagged word vector and a position word vector, splicing the tagged word vector and the position word vector, introducing the spliced tagged word vector and the position word vector into a BERT model to perform information aggregation calculation, introducing an output result into a sigmoid function to calculate the edge relation confidence degrees of the A node and the X node, and considering that the A node and the X node have the edge relation if the confidence degree is greater than or equal to a threshold value.
The method and the device have the advantages that the unstructured long text described by the attribute knowledge is added when the knowledge graph is constructed, information loss caused when the graph is constructed can be reduced, semantic features of the unstructured long text attribute information can be fully utilized in a modeling stage, modeling is performed by combining graph structure information of the knowledge graph, semantic and structural relations of nodes of the nodes and neighbor nodes are modeled by taking BERT as an aggregation function, supplement of side relation information can be achieved, accuracy of side relation prediction is improved, and accuracy of knowledge reasoning can be improved.
The specific arrangement and implementation of the embodiments of the present application are described above from different perspectives by way of a plurality of embodiments. In correspondence with the processing method of at least one of the above embodiments, the present application further provides a knowledge-graph-based edge relation prediction apparatus 100, referring to fig. 4, which includes:
the map acquisition module 110 is configured to acquire a first knowledge map, where the first knowledge map is constructed based on structured data and unstructured data of a target knowledge domain;
a model obtaining module 120, configured to obtain a first knowledge representation model, where the first knowledge representation model is generated by training a first BERT model through a corpus data set in a target knowledge domain;
a tagged word vector obtaining processing module 130, configured to input the first knowledge graph into the first knowledge representation model, so as to obtain tagged word vectors of multiple nodes in the output first knowledge graph;
a position word vector determining module 140, configured to determine a position word vector of each node according to distances between the plurality of nodes in the first knowledge graph and respective neighboring nodes;
and the edge relation information acquisition module 150 is configured to input the tagged word vectors and the position word vectors of the plurality of nodes of the first knowledge graph into the second BERT model to perform information aggregation processing, so as to obtain edge relation information between the plurality of nodes of the first knowledge graph and respective neighboring nodes.
Optionally, for a first node in the first knowledge graph, the first knowledge representation model is configured to calculate a word vector of the first node and word vectors of one or more neighbor nodes of the first node, and splice the calculated word vectors to obtain a tagged word vector of the first node.
Optionally, the distance between the first node and the neighboring node is the number of edges between the first node and one or more neighboring nodes of the first node, the position word vector of the first node has the same dimension as the tagged word vector of the first node, and each dimension corresponds to the number of edges.
Optionally, the second BERT model is configured to learn semantic information and graph structure relationships of a plurality of nodes of the first knowledge-graph through an attention masking tool, and obtain edge relationship information between the nodes in the first knowledge-graph.
Optionally, the edge relation prediction apparatus 100 further includes: and the confidence processing module is used for inputting the edge relation information into the activation function to calculate the confidence of each edge relation and reserving the edge relation with the confidence greater than or equal to the threshold.
The functions of each module in each apparatus in the embodiment of the present application may refer to the processing correspondingly described in the foregoing method embodiment, and are not described herein again.
There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product. Fig. 5 is a block diagram of an electronic device for a knowledge-graph-based edge relation prediction method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 5, the electronic apparatus includes: one or more processors 1001, memory 1002, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display Graphical information for a Graphical User Interface (GUI) on an external input/output device, such as a display device coupled to the Interface. In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 5 illustrates an example of a processor 1001.
The memory 1002 is a non-transitory computer readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for knowledge-graph based edge relationship prediction provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method for knowledge-graph based edge-relationship prediction provided herein. The computer program product of the present application includes computer instructions that, when executed by a processor, provide a method of knowledge-graph based edge relationship prediction as provided herein.
The memory 1002, as a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the method for knowledge-graph-based edge-relationship prediction in the embodiments of the present application (e.g., the graph obtaining module 110, the model obtaining module 120, the tagged word vector obtaining module 130, the position word vector determining module 140, and the edge-relationship information obtaining module 150 shown in fig. 4). The processor 1001 executes various functional applications of the server and data processing, i.e., implements the knowledge-graph based edge relation prediction method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 1002.
The memory 1002 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created from analysis of the search result processing use of the electronic device, and the like. Further, the memory 1002 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 1002 may optionally include memory located remotely from the processor 1001, which may be connected to the analysis processing electronics of the search results over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device corresponding to the knowledge graph-based edge relation prediction method in the embodiment of the application may further include: an input device 1003 and an output device 1004. The processor 1001, the memory 1002, the input device 1003 and the output device 1004 may be connected by a bus or other means, and the embodiment of fig. 5 in the present application is exemplified by the bus connection.
The input device 1003 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device for analysis processing of search results, such as an input device like a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer, one or more mouse buttons, a track ball, a joystick, etc. The output devices 1004 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The Display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) Display, and a plasma Display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, Integrated circuitry, Application Specific Integrated Circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (Cathode Ray Tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present invention is not limited herein.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (17)
1. A knowledge graph-based edge relation prediction method comprises the following steps:
acquiring a first knowledge graph, wherein the first knowledge graph is constructed based on structured data and unstructured data of a target knowledge domain;
acquiring a first knowledge representation model, wherein the first knowledge representation model is generated by training a first BERT model through a corpus data set of the target knowledge field;
inputting the first knowledge graph into the first knowledge representation model to obtain output tagged word vectors of a plurality of nodes in the first knowledge graph;
respectively determining position word vectors of all nodes according to the distances between the plurality of nodes in the first knowledge graph and respective neighbor nodes;
and inputting the marked word vectors and the position word vectors of the plurality of nodes of the first knowledge graph into a second BERT model for information aggregation processing to obtain the edge relation information between the plurality of nodes of the first knowledge graph and respective neighbor nodes.
2. The method of claim 1, wherein after the first knowledge-graph is input into the first knowledge-graph representation model, for a first node in the first knowledge-graph, the first knowledge-representation model calculates a word vector of the first node and word vectors of one or more neighboring nodes of the first node, and concatenates the calculated word vectors to obtain a tagged word vector of the first node.
3. The method of claim 2, wherein the distance between the first node and a neighboring node is the number of edges between the first node and one or more neighboring nodes of the first node, the position word vector of the first node has the same dimensions as the tag word vector of the first node, and each dimension corresponds to the number of edges.
4. The method of claim 1, wherein the information aggregation process comprises: and learning semantic information and graph structure relations of a plurality of nodes of the first knowledge graph through an attention covering tool of the second BERT model, and acquiring edge relation information among the nodes in the first knowledge graph.
5. The method of claim 1, after obtaining edge relationship information between a plurality of nodes of the first knowledge-graph and respective neighboring nodes, the method further comprising:
and inputting the edge relation information into an activation function to calculate the confidence coefficient of each edge relation, and reserving the edge relation with the confidence coefficient larger than or equal to a threshold value.
6. The method of any one of claims 1-5,
the first knowledge-graph comprises a medical knowledge-graph;
the structured data comprises different types of entities of the medical field and attribute data related to each entity;
the unstructured data comprises long text description data of a medical field;
the corpus data set of the target knowledge domain includes chinese medical corpus long text.
7. The method of any one of claims 1-5,
the first BERT model is a BERT model to be trained, and the second BERT model is a pretrained BERT model.
8. A knowledge-graph-based edge relation prediction apparatus, comprising:
the system comprises a map acquisition module, a map generation module and a map generation module, wherein the map acquisition module is used for acquiring a first knowledge map, and the first knowledge map is constructed based on structured data and unstructured data of a target knowledge field;
a model obtaining module, configured to obtain a first knowledge representation model, where the first knowledge representation model is generated by training a first BERT model through a corpus data set of the target knowledge domain;
a tagged word vector acquisition module, configured to input the first knowledge graph into the first knowledge representation model, to obtain tagged word vectors of multiple nodes in the output first knowledge graph;
the position word vector determining module is used for respectively determining the position word vector of each node according to the distance between the plurality of nodes in the first knowledge graph and respective neighbor nodes;
and the side relation information acquisition module is used for inputting the marked word vectors and the position word vectors of the nodes of the first knowledge graph into a second BERT model to carry out information aggregation processing so as to obtain side relation information between the nodes of the first knowledge graph and respective neighbor nodes.
9. The apparatus of claim 8, wherein for a first node in the first knowledge-graph, the first knowledge representation model is configured to compute a word vector for the first node and word vectors for one or more neighboring nodes of the first node, and concatenate the computed word vectors to obtain a tagged word vector for the first node.
10. The apparatus of claim 9, wherein the distance between the first node and a neighboring node is a number of edges between the first node and one or more neighboring nodes of the first node, the position word vector of the first node has the same dimensions as the tag word vector of the first node, and each dimension corresponds to the number of edges.
11. The apparatus of claim 8, wherein the second BERT model is configured to learn semantic information and graph structure relationships of a plurality of nodes of the first knowledgegraph through an attention masking tool to obtain edge relationship information between nodes in the first knowledgegraph.
12. The apparatus of claim 8, further comprising:
and the confidence processing module is used for inputting the edge relation information into an activation function to calculate the confidence of each edge relation and reserving the edge relation with the confidence greater than or equal to the threshold.
13. The apparatus of any one of claims 8-12,
the first knowledge-graph comprises a medical knowledge-graph;
the structured data comprises different types of entities of the medical field and attribute data related to each entity;
the unstructured data comprises long text description data of a medical field;
the corpus data set of the target knowledge domain includes chinese medical corpus long text.
14. The apparatus of any one of claims 8-12,
the first BERT model is a BERT model to be trained, and the second BERT model is a pretrained BERT model.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-9.
17. A computer program product comprising computer instructions which, when executed by a processor, implement the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406864.5A CN112528037B (en) | 2020-12-04 | 2020-12-04 | Side relation prediction method, device, equipment and storage medium based on knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011406864.5A CN112528037B (en) | 2020-12-04 | 2020-12-04 | Side relation prediction method, device, equipment and storage medium based on knowledge graph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112528037A true CN112528037A (en) | 2021-03-19 |
| CN112528037B CN112528037B (en) | 2024-04-09 |
Family
ID=74998486
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011406864.5A Active CN112528037B (en) | 2020-12-04 | 2020-12-04 | Side relation prediction method, device, equipment and storage medium based on knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528037B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139483A (en) * | 2021-04-28 | 2021-07-20 | 北京百度网讯科技有限公司 | Human behavior recognition method, apparatus, device, storage medium, and program product |
| CN113204961A (en) * | 2021-05-31 | 2021-08-03 | 平安科技（深圳）有限公司 | Language model construction method, device, equipment and medium for NLP task |
| CN113628726A (en) * | 2021-08-10 | 2021-11-09 | 海南榕树家信息科技有限公司 | Traditional Chinese medicine dialectical treatment recommendation system and method based on graph neural network and electronic equipment |
| CN114168799A (en) * | 2021-11-26 | 2022-03-11 | 四川云从天府人工智能科技有限公司 | Method, device and medium for acquiring characteristics of node adjacency relation in graph data structure |
| CN114490884A (en) * | 2021-12-21 | 2022-05-13 | 北京三快在线科技有限公司 | Method and device for determining entity association relationship, electronic equipment and storage medium |
| CN114880551A (en) * | 2022-04-12 | 2022-08-09 | 北京三快在线科技有限公司 | Method and device for acquiring upper-lower relation, electronic equipment and storage medium |
| CN116150392A (en) * | 2022-12-12 | 2023-05-23 | 首都师范大学 | Threat information knowledge graph processing method, threat information knowledge graph processing device, threat information knowledge graph processing equipment and storage medium |
| CN116523039A (en) * | 2023-04-26 | 2023-08-01 | 华院计算技术(上海)股份有限公司 | Continuous casting knowledge graph generation method and device, storage medium and terminal |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017185887A1 (en) * | 2016-04-29 | 2017-11-02 | Boe Technology Group Co., Ltd. | Apparatus and method for analyzing natural language medical text and generating medical knowledge graph representing natural language medical text |
| CN110390023A (en) * | 2019-07-02 | 2019-10-29 | 安徽继远软件有限公司 | A kind of knowledge mapping construction method based on improvement BERT model |
| DE202020102105U1 (en) * | 2020-04-16 | 2020-04-29 | Robert Bosch Gmbh | Device for the automated generation of a knowledge graph |
| CN111475658A (en) * | 2020-06-12 | 2020-07-31 | 北京百度网讯科技有限公司 | Knowledge representation learning method, device, equipment and storage medium |
-
2020
- 2020-12-04 CN CN202011406864.5A patent/CN112528037B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017185887A1 (en) * | 2016-04-29 | 2017-11-02 | Boe Technology Group Co., Ltd. | Apparatus and method for analyzing natural language medical text and generating medical knowledge graph representing natural language medical text |
| CN110390023A (en) * | 2019-07-02 | 2019-10-29 | 安徽继远软件有限公司 | A kind of knowledge mapping construction method based on improvement BERT model |
| DE202020102105U1 (en) * | 2020-04-16 | 2020-04-29 | Robert Bosch Gmbh | Device for the automated generation of a knowledge graph |
| CN111475658A (en) * | 2020-06-12 | 2020-07-31 | 北京百度网讯科技有限公司 | Knowledge representation learning method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 张小亮;***;王永庆;郭建军;刘云;: "基于自然语言处理的临床合理用药知识图谱构建", 中华医学图书情报杂志, no. 09 * |
| 马满福;刘元?;李勇;王霞;贾海;史彦斌;张小康;: "基于LCN的医疗知识问答模型", 西南大学学报(自然科学版), no. 10 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139483B (en) * | 2021-04-28 | 2023-09-29 | 北京百度网讯科技有限公司 | Human behavior recognition method, device, apparatus, storage medium, and program product |
| CN113139483A (en) * | 2021-04-28 | 2021-07-20 | 北京百度网讯科技有限公司 | Human behavior recognition method, apparatus, device, storage medium, and program product |
| CN113204961A (en) * | 2021-05-31 | 2021-08-03 | 平安科技（深圳）有限公司 | Language model construction method, device, equipment and medium for NLP task |
| CN113204961B (en) * | 2021-05-31 | 2023-12-19 | 平安科技（深圳）有限公司 | Language model construction method, device, equipment and medium for NLP task |
| CN113628726A (en) * | 2021-08-10 | 2021-11-09 | 海南榕树家信息科技有限公司 | Traditional Chinese medicine dialectical treatment recommendation system and method based on graph neural network and electronic equipment |
| CN113628726B (en) * | 2021-08-10 | 2023-12-26 | 海南榕树家信息科技有限公司 | Traditional Chinese medicine diagnosis and treatment recommendation system and method based on graph neural network and electronic equipment |
| CN114168799A (en) * | 2021-11-26 | 2022-03-11 | 四川云从天府人工智能科技有限公司 | Method, device and medium for acquiring characteristics of node adjacency relation in graph data structure |
| CN114168799B (en) * | 2021-11-26 | 2024-06-11 | 四川云从天府人工智能科技有限公司 | Method, device and medium for acquiring characteristics of node adjacency in graph data structure |
| CN114490884A (en) * | 2021-12-21 | 2022-05-13 | 北京三快在线科技有限公司 | Method and device for determining entity association relationship, electronic equipment and storage medium |
| CN114880551B (en) * | 2022-04-12 | 2023-05-02 | 北京三快在线科技有限公司 | Method and device for acquiring upper and lower relationship, electronic equipment and storage medium |
| CN114880551A (en) * | 2022-04-12 | 2022-08-09 | 北京三快在线科技有限公司 | Method and device for acquiring upper-lower relation, electronic equipment and storage medium |
| CN116150392A (en) * | 2022-12-12 | 2023-05-23 | 首都师范大学 | Threat information knowledge graph processing method, threat information knowledge graph processing device, threat information knowledge graph processing equipment and storage medium |
| CN116523039A (en) * | 2023-04-26 | 2023-08-01 | 华院计算技术(上海)股份有限公司 | Continuous casting knowledge graph generation method and device, storage medium and terminal |
| CN116523039B (en) * | 2023-04-26 | 2024-02-09 | 华院计算技术(上海)股份有限公司 | Continuous casting knowledge graph generation method and device, storage medium and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112528037B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112528037B (en) | Side relation prediction method, device, equipment and storage medium based on knowledge graph | |
| CN112668671B (en) | Method and device for acquiring pre-training model | |
| CN112560496B (en) | Training method and device of semantic analysis model, electronic equipment and storage medium | |
| CN112507040B (en) | Training method and device for multivariate relation generation model, electronic equipment and medium | |
| JP7228662B2 (en) | Event extraction method, device, electronic device and storage medium | |
| CN112560479B (en) | Abstract extraction model training method, abstract extraction device and electronic equipment | |
| CN111079442B (en) | Vectorization representation method and device of document and computer equipment | |
| CN111753543A (en) | Medicine recommendation method and device, electronic equipment and storage medium | |
| KR20210040851A (en) | Text recognition method, electronic device, and storage medium | |
| CN111710412B (en) | Diagnostic result verification method and device and electronic equipment | |
| CN111967256B (en) | Event relation generation method and device, electronic equipment and storage medium | |
| CN112347769B (en) | Entity recognition model generation method and device, electronic equipment and storage medium | |
| CN111144507B (en) | Emotion analysis model pre-training method and device and electronic equipment | |
| KR20220011082A (en) | Natural language processing model training method, device, electric equipment and storage medium | |
| CN111241838B (en) | Semantic relation processing method, device and equipment for text entity | |
| CN111507111B (en) | Pre-training method and device of semantic representation model, electronic equipment and storage medium | |
| CN111950291A (en) | Semantic representation model generation method and device, electronic equipment and storage medium | |
| CN112560505A (en) | Recognition method and device of conversation intention, electronic equipment and storage medium | |
| CN111326251A (en) | Method and device for outputting inquiry questions and electronic equipment | |
| CN111966782B (en) | Multi-round dialogue retrieval method and device, storage medium and electronic equipment | |
| CN113868519A (en) | Information searching method and device, electronic equipment and storage medium | |
| CN113553411B (en) | Query statement generation method and device, electronic equipment and storage medium | |
| CN112507705B (en) | Position code generation method and device and electronic equipment | |
| JP7186255B2 (en) | Medical collision detection method, device, electronic device and storage medium | |
| CN113902005A (en) | Method, device, equipment and storage medium for pre-training language model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |