CN115858821B - Knowledge graph processing method and device and training method of knowledge graph processing model - Google Patents
Knowledge graph processing method and device and training method of knowledge graph processing model Download PDFInfo
- Publication number
- CN115858821B CN115858821B CN202310132855.9A CN202310132855A CN115858821B CN 115858821 B CN115858821 B CN 115858821B CN 202310132855 A CN202310132855 A CN 202310132855A CN 115858821 B CN115858821 B CN 115858821B
- Authority
- CN
- China
- Prior art keywords
- relation
- entity
- connection
- relationship
- connection relation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 106
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012549 training Methods 0.000 title claims abstract description 26
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 239000011159 matrix material Substances 0.000 claims description 112
- 239000013598 vector Substances 0.000 claims description 97
- 230000006870 function Effects 0.000 claims description 38
- 230000002776 aggregation Effects 0.000 claims description 21
- 238000004220 aggregation Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 13
- 230000003213 activating effect Effects 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 11
- 230000000873 masking effect Effects 0.000 claims description 9
- 230000004931 aggregating effect Effects 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims 2
- 238000010801 machine learning Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 6
- 235000013305 food Nutrition 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000015654 memory Effects 0.000 description 5
- 241000238421 Arthropoda Species 0.000 description 3
- 241000607479 Yersinia pestis Species 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 235000005911 diet Nutrition 0.000 description 3
- 230000037213 diet Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a knowledge graph processing method and device and a training method of a knowledge graph processing model, which can be applied to the technical field of computers and the technical field of machine learning. The method comprises the following steps: acquiring a first entity relation pair and a second entity relation pair of an initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity; determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair; and processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph.
Description
Technical Field
The present invention relates to the field of computer technology and the field of machine learning technology, and in particular, to a knowledge graph processing method, a device, an electronic apparatus, and a training method for a knowledge graph processing model.
Background
The Knowledge Graph (KG) can display complex Knowledge in the fields of medicine, finance, electronic commerce and the like through data mining, information processing, knowledge metering and Graph drawing, so as to reveal the dynamic development rule of the Knowledge field and provide practical and valuable references for discipline research, such as medicine discovery, user modeling, dialogue systems and the like. However, the existing knowledge graph has serious information deletion problem, and compared with the high cost of manual labeling, the knowledge graph completion (Knowledge Graph Completion, KGC) can automatically predict the missing entity relation pair based on the incomplete knowledge graph.
In the process of implementing the inventive concept, the inventor finds that at least the following problems exist in the related art: the knowledge graph completion method in the related art cannot fully utilize the connection relation in the knowledge graph, so that the knowledge graph is not fully completed.
Disclosure of Invention
In view of the above problems, the invention provides a knowledge graph processing method, a knowledge graph processing device, an electronic device and a training method of a knowledge graph processing model.
According to a first aspect of the present invention, there is provided a knowledge-graph processing method, comprising:
Acquiring a first entity relation pair and a second entity relation pair of an initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity;
determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity;
and processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph.
According to an embodiment of the invention, wherein determining the third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair comprises:
determining a first attention score of the first entity-relationship pair and a second attention score of the second entity-relationship pair using an attention mechanism;
Determining a third connection relationship satisfying a preset condition based on the first attention score and the second attention score;
a third entity-relationship pair is determined based on the third connection relationship.
According to an embodiment of the present invention, the first connection relationship, the second connection relationship, and the third connection relationship each include I dimensions, and processing the first connection relationship, the second connection relationship, and the third connection relationship to obtain a target knowledge graph includes:
the method comprises the steps of carrying out aggregation updating on a first connection relation of an ith dimension, a second connection relation of the ith dimension and a third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively, wherein I is more than or equal to 1 and less than or equal to I, and I and I are integers;
and processing the first updated connection relation, the second updated connection relation and the third updated connection relation to obtain a target knowledge graph.
According to an embodiment of the present invention, aggregating and updating a first connection relationship in an ith dimension, a second connection relationship in the ith dimension, and a third connection relationship in the ith dimension to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively, includes:
Generating a first initial relation matrix based on the first connection relation, the second connection relation and the third connection relation, wherein a column vector in the first initial relation matrix represents a dimension, and a row vector represents the connection relation;
determining a target connection relationship according to the first connection relationship, the second connection relationship and the third connection relationship;
performing a relationship mask on the target connection relationship in the first initial relationship matrix to obtain a first intermediate relationship matrix;
determining an ith column vector from the first intermediate relation matrix based on the first connection relation of the ith dimension, the second connection relation of the ith dimension and the third connection relation of the ith dimension;
and performing aggregation updating on the ith column vector to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively.
According to an embodiment of the present invention, processing a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship to obtain a target knowledge graph includes:
generating a second initial relation matrix based on the first updated connection relation, the second updated connection relation and the third updated connection relation, wherein column vectors in the second initial relation matrix represent dimensions, and row vectors represent connection relations;
Performing dimension masking on an nth column vector in a second initial relation matrix based on an nth dimension to obtain a second intermediate relation matrix, wherein n is a positive integer;
and processing each row vector in the second intermediate relation matrix to obtain a target knowledge graph.
According to an embodiment of the present invention, performing aggregation update on an ith column vector to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively, including:
activating the ith column vector by using a first activating function to obtain an intermediate column vector;
and carrying out aggregation updating on the intermediate column vectors according to the first weight matrix to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation.
According to an embodiment of the present invention, processing each row vector in the second intermediate relation matrix to obtain a target knowledge graph includes:
activating the row vector by using a second activation function to obtain an intermediate row vector;
and aggregating the intermediate row vectors according to the second weight matrix to obtain a target knowledge graph.
The second aspect of the invention provides a training method of a knowledge graph processing model, which comprises the following steps:
obtaining an initial knowledge graph sample, wherein the initial knowledge graph sample comprises a first entity relation pair sample and a second entity relation pair sample, the first entity relation pair sample is an unprocessed entity relation pair in the initial knowledge graph, and the second entity relation pair sample is a processed entity relation pair in the initial knowledge graph;
and training the knowledge graph processing model by taking the second entity relation pair sample pair as a label and taking the first entity relation pair sample pair as an input to obtain a trained knowledge graph processing model.
A third aspect of the present invention provides a knowledge-graph processing apparatus, including:
the first acquisition module is used for acquiring a first entity relation pair and a second entity relation pair of the initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity;
A determining module configured to determine a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity;
the first obtaining module is used for processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph.
A fourth aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above.
According to the knowledge graph processing method, the knowledge graph processing device, the electronic equipment and the training method of the knowledge graph processing model, the first entity relation pair and the second entity relation pair of the initial knowledge graph are obtained, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity; determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity; and processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph. The third entity relation pair is determined through the first attention score and the second attention score, more entity relation pairs are obtained, the target knowledge graph is obtained based on the first connection relation, the second connection relation and the third connection relation, and the utilization of the connection relation in the initial knowledge graph is enhanced, so that the technical problem that the knowledge graph is not fully complemented due to the fact that the connection relation in the knowledge graph cannot be fully utilized by the knowledge graph complement method in the related technology is at least partially solved.
Drawings
The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:
fig. 1 shows an application scenario diagram of a knowledge-graph processing method according to an embodiment of the present invention;
FIG. 2 shows a flow chart of a knowledge-graph processing method, according to an embodiment of the invention;
FIG. 3 shows a schematic process diagram of a first initial relationship matrix and a second initial relationship matrix according to an embodiment of the invention;
FIG. 4 shows a flowchart of a training method of a knowledge-graph processing model, in accordance with an embodiment of the invention;
FIG. 5 shows a training schematic of a knowledge-graph processing model, in accordance with an embodiment of the invention;
FIG. 6 is a schematic diagram showing an example of knowledge-graph processing model processing according to an embodiment of the present invention;
fig. 7 shows a block diagram of a knowledge-graph processing apparatus, according to an embodiment of the present invention;
FIG. 8 shows a block diagram of a training apparatus of a knowledge-graph processing model, in accordance with an embodiment of the invention;
fig. 9 shows a block diagram of an electronic device adapted to implement the knowledge-graph processing method, according to an embodiment of the invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the technical scheme of the invention, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing, applying and the like of the personal information of the user all accord with the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.
In the technical scheme of the invention, the processes of data acquisition, collection, storage, use, processing, transmission, provision, disclosure, application and the like all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.
In the related art, KGC aims to use existing information in KG to complete inference of unknown entities, and can be broadly classified into an embedding-based method and a multi-hop inference-based method. The basic idea of the embedding-based approach is to learn representations of entities and connection relationships to model semantic associations between pairs of entity relationships. They typically evaluate the rationality of unknown entities by well-designed scoring functions. These methods can be categorized into translation-based, semantic matching-based, and neural network-based, according to different design criteria for the scoring function. These methods train in an end-to-end fashion, which balances efficiency and performance well. In contrast, multi-hop reasoning-based approaches sacrifice some performance to improve interpretability, the basic idea being to discover possible paths between head and tail entities to infer missing entities. These efforts typically include reinforcement learning-based and neural symbol rule-based models. The former mainly uses reinforcement learning to advance entities and/or relationships as states or actions along an existing graph structure, while the latter helps to build inference paths by mining logical rules.
To improve the embedded KGC-based approach, a graph convolution network (Graph Convolutional Network, GCN) is introduced to model the graph structure of the knowledge graph, usually using GCN as encoder to complete the representation learning of entities and connection relations, and then using the embedded scoring-based function as decoder to evaluate the rationality of the facts. The strategy further improves the performance of the embedding-based approach, since the optimized representation contains rich structural information. However, most of these approaches focus on updating the connection relationships between entities, ignoring updates to the intrinsic connections between the connection relationships.
In view of this, an embodiment of the present invention provides a knowledge-graph processing method, including: acquiring a first entity relation pair and a second entity relation pair of an initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity; determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity; and processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph. The third entity relation pair is determined through the first attention score and the second attention score, more entity relation pairs are obtained, the target knowledge graph is obtained based on the first connection relation, the second connection relation and the third connection relation, and the utilization of the connection relation in the initial knowledge graph is enhanced, so that the technical problem that the knowledge graph is not fully complemented due to the fact that the connection relation in the knowledge graph cannot be fully utilized by the knowledge graph complement method in the related technology is at least partially solved.
Fig. 1 shows an application scenario diagram of a knowledge-graph processing method according to an embodiment of the present invention.
As shown in fig. 1, an application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium used to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the first terminal device 101, the second terminal device 102, the third terminal device 103, to receive or send messages etc. Various communication client applications, such as a shopping class application, a web browser application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only) may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.
The first terminal device 101, the second terminal device 102, the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that, the knowledge graph processing method provided in the embodiment of the present invention may be generally executed by the server 105. Accordingly, the knowledge graph processing apparatus provided in the embodiment of the present invention may be generally disposed in the server 105. The knowledge graph processing method provided by the embodiment of the present invention may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105. Accordingly, the knowledge graph processing apparatus provided by the embodiment of the present invention may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and/or the server 105.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 shows a flowchart of a knowledge-graph processing method according to an embodiment of the invention.
As shown in FIG. 2, the knowledge graph processing method of the embodiment includes operations S210-S230.
In operation S210, a first entity-relationship pair and a second entity-relationship pair of the initial knowledge-graph are obtained, wherein the first entity-relationship pair includes a first header entity, a first tail entity, and a first connection relationship between the first header entity and the first tail entity, and the second entity-relationship pair includes a second header entity, a second tail entity, and a second connection relationship between the second header entity and the second tail entity.
According to an embodiment of the present invention, the initial knowledge-graph may be a sparse knowledge-graph of entity relationship pairs.
According to an embodiment of the present invention, the first entity-relationship pair may represent a triplet in the initial knowledge-graph, the first entity-relationship pair including a first head entity, a first tail entity and a first connection relationship, e.g., the first entity-relationship pair may be (a, colleague, B) where a is the first head entity, B is the first tail entity, and "colleague" is the first connection relationship.
According to the embodiment of the invention, the second entity relation pair can represent a triplet different from the first entity relation pair in the initial knowledge graph, specifically, the first connection relation and the second connection relation are different, for example, the first entity relation is (A, colleague, B), the second entity relation pair is (A, friend, B), wherein A is a first head entity and a second head entity, and B is a first tail entity and a second tail entity; the first entity-relationship pair and the second entity-relationship pair may also be physically distinct, e.g., the first entity-relationship pair is (a, colleague, B), the second entity-relationship pair is (B, friend, C), B is both the first tail entity and the second head entity, and the second entity-relationship pair may also be (C, colleague, B).
According to an embodiment of the present invention, there may be a plurality of pairs of first entity relationships, and there may be a plurality of second entity relationships.
In operation S220, a third entity-relationship pair is determined based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes the first head entity, the second tail entity, and a third connection relationship between the first head entity and the second tail entity.
According to an embodiment of the invention, the first attention score may characterize a degree of tightness between the first head entity and the first tail entity, and the second attention score may characterize a degree of tightness between the second head entity and the second tail entity.
According to the embodiment of the invention, the second tail entity can be used as the tail entity in the third entity relation pair to be complemented, and the relation path between the first head entity and the second tail entity is determined by the first attention score and the second attention score, so that the third connection relation is determined, and the third entity relation pair is obtained.
In operation S230, the first connection relationship, the second connection relationship, and the third connection relationship are processed to obtain a target knowledge graph.
According to the embodiment of the invention, the first connection relationship, the second connection relationship and the third connection relationship can be used for connecting the entities in the initial knowledge graph to obtain the target knowledge graph.
According to the embodiment of the invention, a first entity relation pair and a second entity relation pair of an initial knowledge graph are obtained, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity; determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity; and processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph. The third entity relation pair is determined through the first attention score and the second attention score, more entity relation pairs are obtained, the target knowledge graph is obtained based on the first connection relation, the second connection relation and the third connection relation, and the utilization of the connection relation in the initial knowledge graph is enhanced, so that the technical problem that the knowledge graph is not fully complemented due to the fact that the connection relation in the knowledge graph cannot be fully utilized by the knowledge graph complement method in the related technology is at least partially solved.
According to an embodiment of the invention, wherein determining the third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair comprises:
determining a first attention score of the first entity-relationship pair and a second attention score of the second entity-relationship pair using an attention mechanism;
determining a third connection relationship satisfying a preset condition based on the first attention score and the second attention score;
a third entity-relationship pair is determined based on the third connection relationship.
According to an embodiment of the present invention, the attention mechanism may be established based on a relationship-aware, weighting-free GCN, and in particular, the third entity-relationship pair may be described by an aggregation function.
wherein ,
representing a third tail entity in a third entity-relationship pair,/->
Representing a first entity relation pair and a second entity relation pair in the initial knowledge-graph, ++>
Representing a first head entity in a first entity relationship pair and a first connection relationship or a second head entity in a second entity relationship pair and a second connection relationship, respectively,/->
Respectively represent the passes throughlA first head entity after a plurality of iterations, a first connection relationship and a first tail entity, or a first connection relationship after a plurality of iterations lA second head entity, a second connection and a second tail entity after the second iteration,/->
The representation represents an update function.
According to an embodiment of the present invention, a first head entity and a first tail entity in a first entity-relationship pair and a second head entity and a second tail entity in a second entity-relationship pair may be mapped to a relationship space, in particular, a sparse matrix may be used as a projection matrix by using a diagonalization constraint:
wherein ,
representing a first head entity and a first connection relationship or a firstProjection matrix of two head entities and second connection relation, < >>
Projection matrix representing a first tail entity and a first connection relation or a second tail entity and a second connection relation,/a first tail entity and a second tail entity>
A diagonal matrix representing the first connection or the second connection.
According to an embodiment of the present invention, the third entity relationship pair may be updated by the projection matrix described above:
wherein ,d s representing the number of first header entities or second header entities,d t representing the number of first tail entities or second tail entities,
representing either the first attention score or the second attention score.
In accordance with an embodiment of the present invention,
calculated by the following formula (5):
Wherein, in the formula (5), use is made of
(hyperbolic tangent function) rather than conventional
(normalization function), mainly because hyperbolic tangent function can better aggregate information from the first entity relationship pair and the second entity relationship pair.
According to an embodiment of the invention, the third entity-relationship pair may be determined by a first attention score of the plurality of first entity-relationship pairs and a second attention score of the plurality of second entity-relationship pairs, e.g. assuming that there is a relationship set of third connection relationships between the first head entity and the second tail entity
The relation set consists of at least one first connection and at least one second connection, i.e.>
（nIs the number of first connections and second connections, < >>
Representing the first connection relationship or the second connection relationship), the following can be obtained:
wherein ,
representing the second tail entity in the third entity-relationship pair,/->
Are respectively->
Is the corresponding first attention score or second attention score. In the event that the attention scores on one path both exceed a threshold, then a third entity-relationship pair may be determined.
According to an embodiment of the present invention, the first connection relationship, the second connection relationship, and the third connection relationship each include I dimensions, and processing the first connection relationship, the second connection relationship, and the third connection relationship to obtain a target knowledge graph includes:
The method comprises the steps of carrying out aggregation updating on a first connection relation of an ith dimension, a second connection relation of the ith dimension and a third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively, wherein I is more than or equal to 1 and less than or equal to I, and I and I are integers;
and processing the first updated connection relation, the second updated connection relation and the third updated connection relation to obtain a target knowledge graph.
According to the embodiment of the invention, the first connection relation, the second connection relation and the third connection relation comprise I dimensions, the first connection relation, the second connection relation and the third connection relation in the same dimension can be aggregated to obtain the correlation of the first connection relation, the second connection relation and the third connection relation in the same dimension, and the first updated connection relation, the second updated connection relation and the third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation are obtained after aggregation and updating.
According to an embodiment of the present invention, the first updated connection relation, the second updated connection relation, and the third updated connection relation are obtained as follows:
wherein ,
a first updated connection relation, a second updated connection relation or a third updated connection relation representing an ith dimension,/->
A first relational inference function is represented,Rrepresenting a set of relationships comprising a first connection relationship, a second connection relationship and a third connection relationship,krepresenting any connection relationship in the set of relationships, +.>
Representing the connection relation obtained in the upper network, the first in the relation setThe first connection relation of the i dimension, the second connection relation of the i dimension and the third connection relation of the i dimension.
According to the embodiment of the invention, the first entity relation pair may be (food, including fruit and apple), the tightness degree between the food and the apple can be reflected by the distance between the food and the apple in the initial knowledge graph, that is, the first connection relation includes the attention score of the fruit, for example, the attention score of the first connection relation is 0.1, the attention score of the first connection relation obtained after aggregation update may be 0.2, the distance between the food and the apple is reduced, so that the connection between the food and the apple is enhanced, the discovery of a new entity relation pair is more facilitated, and the second update connection relation and the third update connection relation are similar to the first update connection relation and are not repeated herein.
According to an embodiment of the present invention, the processing of the first connection relationship, the second connection relationship, and the third connection relationship may be updating of I dimensions of the connection relationship itself, as follows:
wherein ,
representing the connection relation in the target knowledge graph, +.>
Representing a second relationship-reasoning function,
any one of the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship is represented.
According to the embodiment of the invention, the first connection relation, the second connection relation and the third connection relation are aggregated and processed, so that the internal relation between the relations can be deduced from the connection relations, and the initial knowledge graph can be better complemented, and the target knowledge graph can be obtained.
According to an embodiment of the present invention, aggregating and updating a first connection relationship in an ith dimension, a second connection relationship in the ith dimension, and a third connection relationship in the ith dimension to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively, includes:
generating a first initial relation matrix based on the first connection relation, the second connection relation and the third connection relation, wherein a column vector in the first initial relation matrix represents a dimension, and a row vector represents the connection relation;
Determining a target connection relationship according to the first connection relationship, the second connection relationship and the third connection relationship;
performing a relationship mask on the target connection relationship in the first initial relationship matrix to obtain a first intermediate relationship matrix;
determining an ith column vector from the first intermediate relation matrix based on the first connection relation of the ith dimension, the second connection relation of the ith dimension and the third connection relation of the ith dimension;
and performing aggregation updating on the ith column vector to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively.
According to the embodiment of the invention, in order to facilitate the update processing of the initial map by the computer, a first initial relation matrix can be generated based on the first connection relation, the second connection relation and the third connection relation.
According to the embodiment of the invention, in order to enable the first initial relation matrix to better learn the internal correlation between the connection relations, a target connection relation can be randomly determined from the first connection relation, the second connection relation and the third connection relation, and a column vector corresponding to the target connection relation is subjected to relation masking in the first initial relation matrix to obtain a first intermediate relation matrix, and specifically, elements corresponding to the target connection relation in the first initial relation matrix can be set to zero.
According to the embodiment of the invention, the number of column vectors in the first intermediate relation matrix corresponds to the dimensions of the first connection relation, the second connection relation and the third connection relation, i.e. the first intermediate relation matrix has I column vectors, and the ith column vector corresponding to the ith dimension can be determined from the first intermediate relation matrix based on the ith dimension.
According to the embodiment of the invention, the i column vector is aggregated and updated, which can be to analyze semantic information of the connection relation of the i dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation.
According to the embodiment of the invention, the internal relation between the connection relations can be further learned by processing the connection relations under the same dimension.
According to an embodiment of the present invention, processing a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship to obtain a target knowledge graph includes:
generating a second initial relation matrix based on the first updated connection relation, the second updated connection relation and the third updated connection relation, wherein column vectors in the second initial relation matrix represent dimensions, and row vectors represent connection relations;
Performing dimension masking on an nth column vector in a second initial relation matrix based on an nth dimension to obtain a second intermediate relation matrix, wherein n is a positive integer;
and processing each row vector in the second intermediate relation matrix to obtain a target knowledge graph.
According to the embodiment of the invention, the second initial relation matrix can be generated by the first updated connection relation, the second updated connection relation and the third updated connection relation, so that the initial knowledge graph can be updated and complemented better.
According to the embodiment of the invention, the column vector of the second initial relation matrix corresponds to n dimensions of the connection relation, the n-th dimension can be determined randomly, the corresponding relation between the dimension and the column vector can be determined according to the n-th column vector of the second initial relation matrix determined by the n-th dimension, so that the second intermediate relation matrix is obtained by carrying out dimension masking on the n-th column vector, and specifically, elements corresponding to the n-th column vector in the second initial relation matrix can be set to zero.
According to the embodiment of the invention, the I dimensions of the entity relation pair corresponding to the row vector can be subjected to semantic analysis to obtain the target knowledge graph.
According to an embodiment of the present invention, performing aggregation update on an ith column vector to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively, including:
Activating the ith column vector by using a first activating function to obtain an intermediate column vector;
and carrying out aggregation updating on the intermediate column vectors according to the first weight matrix to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation.
According to an embodiment of the present invention, processing each row vector in the second intermediate relation matrix to obtain a target knowledge graph includes:
activating the row vector by using a second activation function to obtain an intermediate row vector;
and aggregating the intermediate row vectors according to the second weight matrix to obtain a target knowledge graph.
According to an embodiment of the present invention, the first weight matrix may include a first sub-weight matrix, the second sub-weight matrix, and the second weight matrix may include a third sub-weight matrix and a fourth sub-weight matrix, and the above process may be represented by the following formula:
wherein ,
a first result matrix representing the output after aggregation of the first connection relationship of the ith dimension, the second connection relationship of the ith dimension and the third connection relationship of the ith dimension, and->
Represent the first lA second result matrix of layer outputs +.>
Representing a target result matrix corresponding to the target knowledge graph, < ->
Representing the first adjustment parameter,/->
Representing a relation mask->
Representing a second adjustment parameter,/->
Representing an activation function->
Representing dimension mask->
A first matrix of sub-weights is represented,
representing a second matrix of sub-weights->
Representing a third sub-weight matrix,>
representing a fourth sub-weight matrix. />
FIG. 3 shows a schematic process diagram of a first initial relationship matrix and a second initial relationship matrix according to an embodiment of the invention.
As shown in fig. 3, a first intermediate relationship matrix is obtained by performing random relationship masking on a first initial relationship matrix, a first result relationship matrix is obtained by performing aggregation updating on column vectors in the first intermediate relationship matrix, wherein the first result relationship matrix comprises a first updated connection relationship, a second updated connection relationship and a third updated connection relationship which respectively correspond to the first connection relationship, the second connection relationship and the third connection relationship, a second intermediate relationship matrix is obtained by performing random dimension masking on a second initial relationship matrix, a target relationship matrix is obtained by processing row vectors in the second intermediate relationship matrix through a multi-layer perception mechanism, and a target knowledge graph is obtained based on the connection relationship in the target relationship matrix.
Fig. 4 shows a flowchart of a training method of the knowledge-graph processing model, according to an embodiment of the invention.
As shown in FIG. 4, the method includes operations S410-S420.
An operation S410, obtaining an initial knowledge-graph sample, where the initial knowledge-graph sample includes a first entity-relationship pair sample and a second entity-relationship pair sample, the first entity-relationship pair sample is an unprocessed entity-relationship pair in the initial knowledge-graph, and the second entity-relationship pair sample is a processed entity-relationship pair in the initial knowledge-graph;
in operation S420, the second entity relationship pair sample pair is used as a label, and the first entity relationship pair sample pair is used as an input to train the knowledge-graph processing model, so as to obtain a trained knowledge-graph processing model.
According to an embodiment of the present invention, the initial knowledge-graph sample may be obtained from a knowledge-graph database.
According to the embodiment of the invention, the first entity relation pair sample can be a sparse entity relation pair set in the initial knowledge graph, and the second entity relation pair sample can be a richer entity relation pair obtained after the initial knowledge graph is complemented.
According to the embodiment of the invention, the completion on the sparse KG can be completed through multiple iterations, and the optimized input scoring function is input 
To complete entity relation pair->
By selecting TransE (based on translation), distMult (based on semantic matching) and ConvE (based on neural network) as representative scoring functions
And determining a loss value of the knowledge-graph processing model according to the cross entropy function, wherein the loss value is represented by the following formula (11):
wherein ,
representing the loss value of the knowledge-graph processing model obtained from the cross entropy function, < >>
Is an activation (logistic signature) function,/->
Sample set representing first entity relation pair samples, < ->
Representing the first entity relation pair in the sample of entity relation pairs,>
representing the number of head and tail entities in the sample set of the first entity relationship pair samples,/>
the scoring function is represented by a number of points,ois the tail entity in the initial knowledge-graph sample,Vrepresenting a set of entities->
Representing the network layer number of the knowledge graph processing model, +.>
Representing the pair of entity relations inferred by the knowledge-graph processing model, < ->
Representing an indicator if and only if +.>
In case the second entity relation pair exists in the sample,/I>
1.
In order to avoid knowledge-graph processing models producing too close results during training, mutual information is used to normalize their representations, according to embodiments of the present invention. Specifically, using InfoNCE (self-supervised contrast learning) loss as a constraint of the connection relationship, the following formula (12):
wherein ,
representing the connection relation versus the loss value, < >>
Representing similarity between two pairs of entity relationships, which are set as cosine similarity functions; />
Representing a first entity offIs a set of connection relations to the sample, +.>
Representing the relation in the set of connection relations of the first entity relation to the sample,/for>
Sample set representing first entity relation pair samples different from +.>
Other relations of->
Representing a first hyper-parameter.
According to the embodiment of the invention, the connection relation is used as an auxiliary part to further improve the learning ability of the knowledge graph processing model. Finally, the following loss functions are used for training of the knowledge-graph processing model:
wherein ,
representing the loss value obtained by training the knowledge graph processing model by taking the connection relation comparison loss value as an auxiliary part, and ++>
Representing a second superparameter,/->
Representing the third hyper-parameter, ">
Model parameters representing a knowledge-graph processing model.
FIG. 5 shows a training schematic of a knowledge-graph processing model, in accordance with an embodiment of the invention.
As shown in fig. 5, the knowledge graph processing model may include multiple network layers, and connection relations in the initial knowledge graph may be fully mined through multiple iterations, so that for convenience in presentation, only two network layers are shown in the figure. And inputting the initial knowledge graph into the knowledge graph processing model, wherein in each layer of updating, the attention score of the entity relation pair can be obtained, the entity relation pair is updated according to the attention score, the relation updating is carried out according to the connection relation, and finally, the training of the knowledge graph processing model is completed through a scoring function, so that the trained knowledge graph model is obtained.
In order to intuitively understand the rationality and effectiveness of knowledge-graph processing model design, a NELL23K dataset is used as an example in accordance with an embodiment of the present invention.
Fig. 6 shows a schematic diagram of an example of knowledge-graph processing model processing according to an embodiment of the invention.
As shown in fig. 6, (pest, invertebrate diet, leaf) is one pair of entity relationships to be judged from the NELL23K test set. From the attention score, we can find a valuable path between the head entity "pest" and the tail entity "leaf": both links between "pests" and "leaves" linked via "food comprising arthropods" and "invertebrates" have a high attention score. Obviously, these paths reflect the following rules: given (x, including arthropods, y) and (y, invertebrate diet, z), we may have (x, invertebrate diet, z). This not only indicates that the knowledge-graph processing model can provide interpretability for the predicted result, but also that the knowledge-graph processing model can effectively learn the endophytic relationship (invertebrates contain arthropods) between the connection relationships in the knowledge graph.
Based on the knowledge graph processing method, the invention further provides a knowledge graph processing device. The device will be described in detail below in connection with fig. 7.
Fig. 7 shows a block diagram of a knowledge-graph processing apparatus, according to an embodiment of the present invention.
As shown in fig. 7, the knowledge-graph processing apparatus 700 of this embodiment includes a first acquisition module 710, a determination module 720, and a first obtaining module 730.
A first obtaining module 710, configured to obtain a first entity-relationship pair and a second entity-relationship pair of the initial knowledge-graph, where the first entity-relationship pair includes a first header entity, a first tail entity, and a first connection relationship between the first header entity and the first tail entity, and the second entity-relationship pair includes a second header entity, a second tail entity, and a second connection relationship between the second header entity and the second tail entity; in an embodiment, the first obtaining module 710 may be configured to perform the operation S210 described above, which is not described herein.
A determining module 720, configured to determine a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, where the third entity-relationship pair includes a first head entity, a second tail entity, and a third connection relationship between the first head entity and the second tail entity; in an embodiment, the determining module 720 may be configured to perform the operation S220 described above, which is not described herein.
The first obtaining module 730 is configured to process the first connection relationship, the second connection relationship, and the third connection relationship to obtain a target knowledge graph. In an embodiment, the first obtaining module 730 may be used to perform the operation S230 described above, which is not described herein.
According to an embodiment of the present invention, wherein the determining module 720 for determining the third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair comprises:
a first determination sub-module for determining a first attention score of a first entity-relationship pair and a second attention score of a second entity-relationship pair using an attention mechanism;
a second determining sub-module for determining a third connection relationship satisfying a preset condition based on the first attention score and the second attention score;
and the third determining submodule is used for determining a third entity relation pair based on the third connection relation.
According to an embodiment of the present invention, the first obtaining module 760 for obtaining the target knowledge graph, where the first connection relationship, the second connection relationship, and the third connection relationship each include I dimensions, processes the first connection relationship, the second connection relationship, and the third connection relationship, and includes:
The first obtaining submodule is used for carrying out aggregation update on a first connection relation of an ith dimension, a second connection relation of the ith dimension and a third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively, I is more than or equal to 1 and less than or equal to I, and I and I are integers;
and the second obtaining submodule is used for processing the first updated connection relation, the second updated connection relation and the third updated connection relation to obtain a target knowledge graph.
According to an embodiment of the present invention, a first obtaining submodule configured to aggregate and update a first connection relation of an ith dimension, a second connection relation of the ith dimension, and a third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation, and a third updated connection relation respectively corresponding to the first connection relation, the second connection relation, and the third connection relation includes:
the first obtaining unit is used for generating a first initial relation matrix based on the first connection relation, the second connection relation and the third connection relation, wherein a column vector in the first initial relation matrix represents a dimension, and a row vector represents the connection relation;
The second obtaining unit is used for determining a target connection relation according to the first connection relation, the second connection relation and the third connection relation;
a third obtaining unit, configured to perform a relationship mask on the target connection relationship in the first initial relationship matrix, to obtain a first intermediate relationship matrix;
a fourth obtaining unit, configured to determine an ith column vector from the first intermediate relation matrix based on the first connection relation of the ith dimension, the second connection relation of the ith dimension, and the third connection relation of the ith dimension;
and a fifth obtaining unit, configured to aggregate and update the ith column vector to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship that respectively correspond to the first connection relationship, the second connection relationship, and the third connection relationship.
According to an embodiment of the present invention, the second obtaining submodule for obtaining the target knowledge-graph includes:
a sixth obtaining unit, configured to generate a second initial relationship matrix based on the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship, where a column vector in the second initial relationship matrix represents a dimension, and a row vector represents the connection relationship;
A seventh obtaining unit, configured to perform dimension masking on an nth column vector in the second initial relationship matrix based on an nth dimension, to obtain a second intermediate relationship matrix, where n is a positive integer;
and an eighth obtaining unit, configured to process each row vector in the second intermediate relation matrix to obtain a target knowledge graph.
According to an embodiment of the present invention, a fifth obtaining unit configured to aggregate and update an ith column vector to obtain a first updated connection relationship, a second updated connection relationship, and a third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively, includes:
the first obtaining subunit is used for performing activation operation on the ith column vector by using a first activation function to obtain an intermediate column vector;
and the second obtaining subunit is used for carrying out aggregation updating on the intermediate column vectors according to the first weight matrix to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation.
According to an embodiment of the present invention, the eighth obtaining unit for obtaining the target knowledge-graph by processing each row vector in the second intermediate relation matrix includes:
A third obtaining subunit, configured to perform activation processing on the row vector by using a second activation function to obtain an intermediate row vector;
and fourthly, obtaining a subunit, configured to aggregate the intermediate row vectors according to the second weight matrix, so as to obtain a target knowledge graph.
Based on the training method of the knowledge graph processing model, the invention also provides a training device of the knowledge graph processing model. The device will be described in detail below in connection with fig. 8.
Fig. 8 shows a block diagram of a training apparatus of a knowledge-graph processing model, according to an embodiment of the invention.
As shown in fig. 8, the training device 800 for the knowledge-graph processing model of this embodiment includes a second acquisition module 810 and a second obtaining module 820.
A second obtaining module 810, configured to obtain an initial knowledge-graph sample, where the initial knowledge-graph sample includes a first entity-relationship pair sample and a second entity-relationship pair sample, the first entity-relationship pair sample is an unprocessed entity-relationship pair in the initial knowledge-graph, and the second entity-relationship pair sample is a processed entity-relationship pair in the initial knowledge-graph; in an embodiment, the second obtaining module 810 may be configured to perform the operation S410 described above, which is not described herein.
The second obtaining module 820 is configured to train the knowledge-graph processing model with the second entity-relationship pair sample pair as a label and the first entity-relationship pair sample pair as an input, to obtain a trained knowledge-graph processing model. In an embodiment, the second obtaining module 820 may be used to perform the operation S420 described above, which is not described herein.
Any of the first acquisition module 710, the determination module 720, and the first obtaining module 730 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules, according to an embodiment of the present invention. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first acquisition module 710, the determination module 720, and the first obtaining module 730 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuits, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the first acquisition module 710, the determination module 720 and the first obtaining module 730 may be at least partially implemented as a computer program module, which when executed may perform the respective functions.
Fig. 9 shows a block diagram of an electronic device adapted to implement the knowledge-graph processing method, according to an embodiment of the invention.
As shown in fig. 9, an electronic device 900 according to an embodiment of the present invention includes a processor 901 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. The processor 901 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 901 may also include on-board memory for caching purposes. Processor 901 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the invention.
In the RAM 903, various programs and data necessary for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. The processor 901 performs various operations of the method flow according to an embodiment of the present invention by executing programs in the ROM 902 and/or the RAM 903. Note that the program may be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to embodiments of the present invention by executing programs stored in the one or more memories.
According to an embodiment of the invention, the electronic device 900 may also include an input/output (I/O) interface 905, the input/output (I/O) interface 905 also being connected to the bus 904. The electronic device 900 may also include one or more of the following components connected to the I/O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output portion 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 908 including a hard disk or the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 910 so that a computer program read out therefrom is installed into the storage section 908 as needed.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the invention and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the invention. In particular, the features recited in the various embodiments of the invention and/or in the claims can be combined in various combinations and/or combinations without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.
The embodiments of the present invention are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the invention, and such alternatives and modifications are intended to fall within the scope of the invention.
Claims (5)
1. A knowledge graph processing method, comprising:
acquiring a first entity relation pair and a second entity relation pair of an initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity;
Determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes the first head entity, the second tail entity, and a third connection relationship between the first head entity and the second tail entity;
processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph;
the first connection relationship, the second connection relationship and the third connection relationship all comprise I dimensions, and the processing of the first connection relationship, the second connection relationship and the third connection relationship to obtain a target knowledge graph comprises the following steps:
performing aggregation updating on the first connection relation of the ith dimension, the second connection relation of the ith dimension and the third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively, wherein I is more than or equal to 1 and less than or equal to I, and I and I are integers;
Processing the first updated connection relation, the second updated connection relation and the third updated connection relation to obtain the target knowledge graph;
the aggregating and updating the first connection relation of the ith dimension, the second connection relation of the ith dimension and the third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which respectively correspond to the first connection relation, the second connection relation and the third connection relation, and the aggregating and updating comprises the following steps:
generating a first initial relation matrix based on the first connection relation, the second connection relation and the third connection relation, wherein a column vector in the first initial relation matrix represents a dimension, and a row vector represents a connection relation;
determining a target connection relationship according to the first connection relationship, the second connection relationship and the third connection relationship;
performing a relation mask on the target connection relation in the first initial relation matrix to obtain a first intermediate relation matrix;
determining an ith column vector from the first intermediate relation matrix based on the first connection relation of the ith dimension, the second connection relation of the ith dimension and the third connection relation of the ith dimension;
Performing aggregation update on the ith column vector to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively;
the processing the first updated connection relationship, the second updated connection relationship and the third updated connection relationship to obtain a target knowledge graph includes:
generating a second initial relationship matrix based on the first updated connection relationship, the second updated connection relationship and the third updated connection relationship, wherein column vectors in the second initial relationship matrix represent dimensions and row vectors represent connection relationships;
performing dimension masking on an nth column vector in the second initial relation matrix based on an nth dimension to obtain a second intermediate relation matrix, wherein n is a positive integer;
processing each row vector in the second intermediate relation matrix to obtain the target knowledge graph;
wherein, the aggregation updating of the ith column vector is to analyze semantic information of the connection relation of the ith dimension;
the aggregating and updating the ith column vector to obtain the first updated connection relationship, the second updated connection relationship and the third updated connection relationship corresponding to the first connection relationship, the second connection relationship and the third connection relationship respectively, including:
Activating the ith column vector by using a first activating function to obtain an intermediate column vector;
performing aggregation updating on the intermediate column vector according to a first weight matrix to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which correspond to the first connection relation, the second connection relation and the third connection relation respectively;
wherein the processing of each row vector in the second intermediate relation matrix is processing of performing semantic analysis on I dimensions of an entity relation pair corresponding to each row vector;
wherein the processing each row vector in the second intermediate relation matrix to obtain the target knowledge graph includes:
activating the row vector by using a second activating function to obtain an intermediate row vector;
and aggregating the intermediate row vectors according to a second weight matrix to obtain the target knowledge graph.
2. The method of claim 1, wherein the determining a third entity-relationship pair based on the first attention score of the first entity-relationship pair and the second attention score of the second entity-relationship pair comprises:
Determining a first attention score for the first entity-relationship pair and a second attention score for the second entity-relationship pair using an attention mechanism;
determining the third connection relation meeting a preset condition based on the first attention score and the second attention score;
and determining a third entity relation pair based on the third connection relation.
3. A training method of a knowledge graph processing model comprises the following steps:
acquiring an initial knowledge graph sample, wherein the initial knowledge graph sample comprises a first entity relation pair sample and a second entity relation pair sample, the first entity relation pair sample is an unprocessed entity relation pair in the initial knowledge graph, and the second entity relation pair sample is a processed entity relation pair in the initial knowledge graph;
training the knowledge graph processing model by taking the second entity relation pair sample pair as a label and taking the first entity relation pair sample pair as an input to obtain a trained knowledge graph processing model;
the completion of the initial knowledge graph is completed through multiple iterations, the rationality assessment of the first entity relation on the entity relation pairs in the sample is completed through a scoring function, the loss value of the knowledge graph processing model is obtained according to a cross entropy function by selecting functions based on translation TransE, on semantic matching DistMult and on a neural network ConvE as the scoring function, and the cross entropy function obtains the calculation of the loss value of the knowledge graph processing model as shown in the following formula (1):
wherein ,
representing a loss value of the knowledge-graph processing model obtained from the cross entropy function, < >>
Representing an activation function->
Sample set representing said first entity relation pair samples,/for a sample>
Representing the entity relationship pairs in the first entity relationship pair sample, +.>
Representing the number of head and tail entities in the sample set of said first entity relation pair samples,/->
Representing a scoring function->
Representing tail entities in said initial knowledge-graph sample,/->
Representing a set of entities->
Representing the network layer number of the knowledge graph processing model, < >>
Representing the pair of entity relations inferred by the knowledge graph processing model,>
representing an indicator if and only if +.>
In case said second entity-relationship pair is present in the sample,/o->
1 is shown in the specification;
taking the comparison loss of the connection relation of the knowledge graph processing model as the constraint of the connection relation, and calculating the comparison loss value of the connection relation of the knowledge graph processing model as the following formula (2):
wherein ,
representing the connection relation versus the loss value, < >>
Representing similarity between two entity-relationship pairs, +.>
Representing a set of connection relations of the first entity relation to the sample,/ >
Representing the first entity relationshipRelationships in the set of connection relationships to the sample, +.>
A sample set representing the first entity relation pair sample different from +.>
Other relations of->
Representing a first hyper-parameter;
training the knowledge graph processing model by taking the connection relation comparison loss value as an auxiliary part, and training the knowledge graph processing model by taking the connection relation comparison loss value as an auxiliary part to obtain a calculation formula (3) of the loss value:
wherein ,
representing the loss value obtained by training the knowledge graph processing model by taking the connection relation comparison loss value as an auxiliary part, and performing +_f>
Representing a second superparameter,/->
Representing the third hyper-parameter, ">
Model parameters representing a knowledge-graph processing model.
4. A knowledge graph processing apparatus comprising:
the first acquisition module is used for acquiring a first entity relation pair and a second entity relation pair of the initial knowledge graph, wherein the first entity relation pair comprises a first head entity, a first tail entity and a first connection relation between the first head entity and the first tail entity, and the second entity relation pair comprises a second head entity, a second tail entity and a second connection relation between the second head entity and the second tail entity;
A determining module configured to determine a third entity-relationship pair based on a first attention score of the first entity-relationship pair and a second attention score of the second entity-relationship pair, wherein the third entity-relationship pair includes the first head entity, the second tail entity, and a third connection relationship between the first head entity and the second tail entity;
the first obtaining module is used for processing the first connection relation, the second connection relation and the third connection relation to obtain a target knowledge graph;
the first connection relationship, the second connection relationship and the third connection relationship all comprise I dimensions, and the first obtaining module comprises:
the first obtaining submodule is used for carrying out aggregation update on a first connection relation of an ith dimension, a second connection relation of the ith dimension and a third connection relation of the ith dimension to obtain a first updated connection relation, a second updated connection relation and a third updated connection relation which are respectively corresponding to the first connection relation, the second connection relation and the third connection relation, I is more than or equal to 1 and less than or equal to I, and I and I are integers;
a second obtaining sub-module, configured to process the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship, to obtain the target knowledge graph;
Wherein the first obtaining submodule includes:
the first obtaining unit is used for generating a first initial relation matrix based on the first connection relation, the second connection relation and the third connection relation, wherein a column vector in the first initial relation matrix represents a dimension, and a row vector represents a connection relation;
a second obtaining unit, configured to determine a target connection relationship according to the first connection relationship, the second connection relationship, and the third connection relationship;
a third obtaining unit, configured to perform a relationship mask on the target connection relationship in the first initial relationship matrix, to obtain a first intermediate relationship matrix;
a fourth obtaining unit, configured to determine an ith column vector from the first intermediate relation matrix based on the first connection relation of the ith dimension, the second connection relation of the ith dimension, and the third connection relation of the ith dimension;
a fifth obtaining unit, configured to aggregate and update the ith column vector to obtain the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship corresponding to the first connection relationship, the second connection relationship, and the third connection relationship, respectively;
Wherein the second obtaining submodule includes:
a sixth obtaining unit, configured to generate a second initial relationship matrix based on the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship, where a column vector in the second initial relationship matrix represents a dimension, and a row vector represents a connection relationship;
a seventh obtaining unit, configured to perform dimension masking on an nth column vector in the second initial relationship matrix based on an nth dimension, to obtain a second intermediate relationship matrix, where n is a positive integer;
an eighth obtaining unit, configured to process each row vector in the second intermediate relationship matrix to obtain the target knowledge graph;
wherein, the aggregation updating of the ith column vector is to analyze semantic information of the connection relation of the ith dimension;
wherein the fifth obtaining unit includes:
a first obtaining subunit, configured to perform an activating operation on the ith column vector by using a first activating function to obtain an intermediate column vector;
a second obtaining subunit, configured to aggregate and update the intermediate column vector according to a first weight matrix, to obtain the first updated connection relationship, the second updated connection relationship, and the third updated connection relationship that respectively correspond to the first connection relationship, the second connection relationship, and the third connection relationship;
Wherein the processing of each row vector in the second intermediate relation matrix is processing of performing semantic analysis on I dimensions of an entity relation pair corresponding to each row vector;
wherein the eighth obtaining unit includes:
a third obtaining subunit, configured to perform activation processing on the row vector by using a second activation function to obtain an intermediate row vector;
and fourth, obtaining a subunit, configured to aggregate the intermediate row vectors according to a second weight matrix, so as to obtain the target knowledge graph.
5. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310132855.9A CN115858821B (en) | 2023-02-20 | 2023-02-20 | Knowledge graph processing method and device and training method of knowledge graph processing model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310132855.9A CN115858821B (en) | 2023-02-20 | 2023-02-20 | Knowledge graph processing method and device and training method of knowledge graph processing model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115858821A CN115858821A (en) | 2023-03-28 |
| CN115858821B true CN115858821B (en) | 2023-06-16 |
Family
ID=85658392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310132855.9A Active CN115858821B (en) | 2023-02-20 | 2023-02-20 | Knowledge graph processing method and device and training method of knowledge graph processing model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115858821B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110064833A (en) * | 2009-12-09 | 2011-06-15 | 한국전자통신연구원 | Apparatus and method for knowledge graph stabilization |
| CN111260064A (en) * | 2020-04-15 | 2020-06-09 | 中国人民解放军国防科技大学 | Knowledge inference method, system and medium based on knowledge graph of meta knowledge |
| CN111949764A (en) * | 2020-08-18 | 2020-11-17 | 桂林电子科技大学 | Knowledge graph completion method based on bidirectional attention mechanism |
| CN112084344A (en) * | 2020-09-11 | 2020-12-15 | 清华大学 | Knowledge graph reasoning method, device and storage medium |
| KR20210098820A (en) * | 2020-02-03 | 2021-08-11 | 삼성전자주식회사 | Electronic device, method for controlling the electronic device and readable recording medium |
| CN113535984A (en) * | 2021-08-11 | 2021-10-22 | 华侨大学 | Attention mechanism-based knowledge graph relation prediction method and device |
-
2023
- 2023-02-20 CN CN202310132855.9A patent/CN115858821B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110064833A (en) * | 2009-12-09 | 2011-06-15 | 한국전자통신연구원 | Apparatus and method for knowledge graph stabilization |
| KR20210098820A (en) * | 2020-02-03 | 2021-08-11 | 삼성전자주식회사 | Electronic device, method for controlling the electronic device and readable recording medium |
| CN111260064A (en) * | 2020-04-15 | 2020-06-09 | 中国人民解放军国防科技大学 | Knowledge inference method, system and medium based on knowledge graph of meta knowledge |
| CN111949764A (en) * | 2020-08-18 | 2020-11-17 | 桂林电子科技大学 | Knowledge graph completion method based on bidirectional attention mechanism |
| CN112084344A (en) * | 2020-09-11 | 2020-12-15 | 清华大学 | Knowledge graph reasoning method, device and storage medium |
| CN113535984A (en) * | 2021-08-11 | 2021-10-22 | 华侨大学 | Attention mechanism-based knowledge graph relation prediction method and device |
Non-Patent Citations (3)
| Title |
|---|
| 一种改进的基于TransE知识图谱表示方法;陈文杰;文奕;张鑫;杨宁;赵爽;计算机工程(第05期);第58-64页 * |
| 基于Att＿GCN模型的知识图谱推理算法;王红;林海舟;卢林燕;计算机工程与应用(第09期);第69-75页 * |
| 知识图谱补全算法综述;丁建辉;贾维嘉;信息通信技术(第01期);第189-195页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115858821A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12008459B2 (en) | Multi-task machine learning architectures and training procedures | |
| WO2020048445A1 (en) | End-to-end structure-aware convolutional networks for knowledge base completion | |
| WO2022063151A1 (en) | Method and system for relation learning by multi-hop attention graph neural network | |
| WO2023097929A1 (en) | Knowledge graph recommendation method and system based on improved kgat model | |
| CN113535984B (en) | Knowledge graph relation prediction method and device based on attention mechanism | |
| US7194466B2 (en) | Object clustering using inter-layer links | |
| US20190164084A1 (en) | Method of and system for generating prediction quality parameter for a prediction model executed in a machine learning algorithm | |
| US20230229677A9 (en) | Artificial intelligence-based property data linking system | |
| US20190303535A1 (en) | Interpretable bio-medical link prediction using deep neural representation | |
| WO2021143267A1 (en) | Image detection-based fine-grained classification model processing method, and related devices | |
| US11360928B2 (en) | Systems and methods for improved anomaly detection in attributed networks | |
| WO2023065859A1 (en) | Item recommendation method and apparatus, and storage medium | |
| WO2017181866A1 (en) | Making graph pattern queries bounded in big graphs | |
| US20230195809A1 (en) | Joint personalized search and recommendation with hypergraph convolutional networks | |
| US10956825B1 (en) | Distributable event prediction and machine learning recognition system | |
| US11874798B2 (en) | Smart dataset collection system | |
| EP4120137A1 (en) | System and method for molecular property prediction using edge conditioned identity mapping convolution neural network | |
| US20230049817A1 (en) | Performance-adaptive sampling strategy towards fast and accurate graph neural networks | |
| CN114065048A (en) | Article recommendation method based on multi-different-pattern neural network | |
| WO2024041483A1 (en) | Recommendation method and related device | |
| CN114358111A (en) | Object clustering model obtaining method, object clustering method and device | |
| US20230041339A1 (en) | Method, device, and computer program product for user behavior prediction | |
| Abbas et al. | Predicting the future popularity of academic publications using deep learning by considering it as temporal citation networks | |
| CN111198905A (en) | Visual analytics framework for understanding missing links in bipartite networks | |
| US20230245210A1 (en) | Knowledge graph-based information recommendation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |