CN111209408A - Time-carrying knowledge graph embedding method based on hybrid translation model - Google Patents

Time-carrying knowledge graph embedding method based on hybrid translation model Download PDF

Info

Publication number
CN111209408A
CN111209408A CN201911335182.7A CN201911335182A CN111209408A CN 111209408 A CN111209408 A CN 111209408A CN 201911335182 A CN201911335182 A CN 201911335182A CN 111209408 A CN111209408 A CN 111209408A
Authority
CN
China
Prior art keywords
entity
translation model
knowledge graph
time
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911335182.7A
Other languages
Chinese (zh)
Inventor
王治豪
李鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
East China Normal University
Original Assignee
East China Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by East China Normal University filed Critical East China Normal University
Priority to CN201911335182.7A priority Critical patent/CN111209408A/en
Publication of CN111209408A publication Critical patent/CN111209408A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)

Abstract

The invention discloses a time-based knowledge graph embedding method based on a hybrid translation model, which comprises the following steps of: 1) inputting a data set related to the knowledge graph, and performing initialization setting according to the condition of the data set; 2) updating the embedded representation of the knowledge graph by using a hybrid translation model to obtain an embedded representation result; 3) and performing completion operation on the related knowledge graph data set according to the embedded representation result. The method can better solve the problem that the prior algorithm can not process the relation between time and multiple elements simultaneously, and can improve the accuracy of judgment.

Description

Time-carrying knowledge graph embedding method based on hybrid translation model
Technical Field
The invention relates to the field of knowledge graph embedding, in particular to a time-carrying knowledge graph embedding method based on a hybrid translation model.
Background
The knowledge graph is a directed graph, and takes different entities in the real world as nodes and different relations as edges on the directed graph. A set of facts existing in the real world is usually represented in the form of triples (h, r, t), where h represents the head entity, t represents the tail entity, and r represents the relationship between two entities. Although the constructed knowledge-graph contains a large number of facts, it is often the case that the knowledge-graph needs to be complemented. The task of knowledge-graph completion is to predict the most likely missing entities and relationships, the head entity prediction is the most likely head entity under this situation (. Since the knowledge-graph constructed from the real world is large and structurally complex, the symbolization-based approach and the traditional logic-based approach are neither scalable nor suitable for the task of current knowledge-graph completion.
Knowledge graph embedding has become one of the important methods for knowledge graph completion. In knowledge graph representation learning, a knowledge graph embedding method based on a translation model is simple and effective, and has good performance on related prediction tasks. It attempts to learn the low-dimensional embedding of each entity and the relationships in the knowledge-graph in a continuous vector space and evaluate the authenticity of triples using scoring functions defined on the entities and relationship embedding. In the representation learning method, the embedding based on translation considers the simplicity of the model and the accuracy of prediction. The translation-based model is derived from TransE, and when (h, r, t) is a valid triplet, it is expected that h + r ≈ t remains unchanged in the embedding space. TransE is suitable for modeling of 1-1 relationships, but has some drawbacks when dealing with reflexive types and multivariate relationships. To address this issue, related researchers have proposed more translation-based models, including TransH, TransR/CTRANsR, TransD, etc., to further address the problems faced by TransE in order to efficiently model diverse types of multivariate relational knowledge-graphs.
Previous translation-based knowledge-graph embedding models (including TransH, TransR/CTRANsR and TransD) focused on modeling static knowledge-graphs. The triples in the static knowledge-graph should be universally correct. In fact, there are many time-dependent facts in KGS, such as triplets (Einstein, diedlin, Princeton)
A method of networking that models temporal relationships as events that vary over time. It uses RNNs as event coders to model the case of temporal and multivariate relational interactions between entities and uses a so-called neighborhood aggregator to model concurrent interactions at the same time.
Disclosure of Invention
The invention aims to provide a time-carrying knowledge graph embedding method based on a mixed translation model, which achieves the purpose of simultaneously processing time and multivariate relation in a time sequence knowledge graph by mixing two models of TransD and TransH, improves the completion accuracy of the knowledge graph and learns better knowledge graph embedding expression.
The specific technical scheme for realizing the purpose of the invention is as follows:
a time-based knowledge graph embedding method based on a hybrid translation model comprises the following steps:
step 1: inputting a time sequence knowledge graph data set needing to be completed, and carrying out initialization setting according to the size of the data set; wherein the initialization setting specifically is:
step A1: randomly initializing entities and relations in the data set, and displaying the entities and relations in a vector form to obtain initial relation vectors and entity vectors;
step A2: for each relation vector, adding the relation vector into a relation set R, and for each entity vector, adding the relation vector into an entity set E;
step 2: updating the embedded representation of the knowledge graph by using a hybrid translation model to obtain an embedded representation result; wherein, the using of the hybrid translation model to update the embedded representation of the knowledge graph specifically comprises:
step B1: sampling a batch of data with a fixed size of batch from a training set of a data set;
step B2: constructing a negative sample data set, and randomly replacing h, r or t for the triple in the batch to generate an error triple to be added into the batch of data to form training data used by the current batch;
step B3: the correct triplet (h, r, t) is compared with the incorrect triplet [ (h, r ', t), (h ', r, t) or (h, r, t ')]Mapping from the entity space to the relationship space, resulting in the correct triplet (h) in the relationship space⊥,r,t⊥) And error triplet [ (h)⊥,r`,t⊥)、(h⊥`,r,t⊥) Or (h ⊥, r, t)⊥`)]；
Step B4: projecting all triples in the relation space onto a hyperplane constructed by the corresponding time tau;
step B5: calculating a loss function, and updating the embedded expression of the entity and the relation by adopting a gradient descent algorithm;
step B6: repeating the steps B1-B5 until the obtained result is stable;
in step B2, the method for constructing the negative sample data set is as follows:
Figure BDA0002330756930000021
Figure BDA0002330756930000022
Figure BDA0002330756930000023
h. r, t, h ', r', t 'represent embedded representations of entities and relationships, where h' represents a randomly substituted head entity, r 'represents a randomly substituted relationship, t' represents a randomly substituted tail entity, D+Representing a set of positive samples, Dx,τ-representing a negative sample data set;
in step B3, the mapping from the entity space to the relationship space is as follows:
Figure BDA0002330756930000031
Figure BDA0002330756930000032
wherein M isrh、MrtTo map the matrix, rpA projection vector representing the relationship is calculated,
Figure BDA0002330756930000033
an identity matrix of size m × n, hp TProjection transpose vector, h, representing head entity⊥Representing an embedded representation of the head entity in the relational space after mapping, tp TProjection transpose vector, t, representing the tail entity⊥Representing an embedded representation of the tail entity in the relationship space after mapping;
in step B4, the projection of all triples in the relationship space onto the hyperplane constructed at the corresponding time τ is as follows:
Figure BDA0002330756930000034
Figure BDA0002330756930000035
Figure BDA0002330756930000036
wherein, ω isτA hyperplane representing the constructed corresponding time τ;
in step B5, the loss function is:
Figure BDA0002330756930000037
wherein f isτ(x)、fτ(y) a scoring function, x, y representing positive and negative examples, f | | | hτ+rτ-tτ||L1/L2L1, L2 are regularization terms, γ is the minimum interval between positive and negative samples, T represents the set of all times;
and step 3: performing completion operation on the used knowledge graph data set according to the embedded representation result; the method specifically comprises the following steps:
step C1: for a triple (h, R, t), firstly checking whether h and t belong to E or not, whether R belongs to R or not, if not, then h, t or R is not used as a head-tail entity or relation, the triple does not hold, if so, then executing the next step;
step C2: calculating the scoring function f | | | h according to the established triplesτ+rτ-tτ||L1/L2And sequencing all established triples, wherein the result in the front sequencing is the optimal result and is used for completing.
The invention has the following advantages:
1) the invention can use the traditional translation algorithm in the field to simultaneously process the time and multivariate relation in the knowledge graph.
2) The invention provides a novel negative sample construction method.
3) The invention has better performance on tasks such as link prediction, relation prediction and the like.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided in order to better understand the present invention for those skilled in the art with reference to the accompanying drawings.
The effectiveness of the method is verified by comparison experiments on YAGO11K and Wikidata 12K. All of these data sets consist of a training set, a validation set, and a test stage. Table 1 lists the data statistics for the data set. Each training entry is a triplet (h, r, t) indicating that h and t have a relationship r.
TABLE 1 data set partitioning for Wikidata12K and YAGO11K (k denotes 1000)
Figure BDA0002330756930000041
The following describes the learning knowledge graph embedding representation of the present invention by taking YAGO11K as an example, and uses the obtained entity vector and relationship vector for link prediction and relationship prediction. As shown in fig. 1, a knowledge graph embedding method based on a hybrid translation model includes the following specific steps:
s1-1 randomly initializing entities and relations in the data set, displaying in a vector form to obtain initial relation vectors and entity vectors, namely randomly generating a representation vector for 10 relations and 10623 entities in the data set;
s1-2, adding each appeared relation vector into a relation set R, and adding each appeared entity vector into an entity set E;
s2.1, sampling a batch of data with fixed size of batch from 16400 training sets in YAGO 11K;
s2.2, constructing a negative sample data set, randomly replacing h, r or t with the triples in the batch to generate an error triplet, and constructing training data used by the current batch, wherein the method for constructing the negative sample is as follows:
Figure BDA0002330756930000042
Figure BDA0002330756930000043
Figure BDA0002330756930000044
h. r, t, h ', r', t 'represent embedded representations of entities and relationships, where h' represents a randomly substituted head entity, r 'represents a randomly substituted relationship, t' represents a randomly substituted tail entity, D+Representing a set of positive samples, Dx,τ-representing a set of negative examples. By randomly replacing the head entity, the relation and the tail entity, a new triplet is formed, which does not belong to the positive sample set, and the erroneous triplets are put into the training data.
S2.3, mapping the triplet (h, r, t) from the entity space to the relation space to obtain the triplet (h) of the relation space⊥,r,t⊥) The way of mapping from the entity space to the relationship space is:
Figure BDA0002330756930000051
Figure BDA0002330756930000052
wherein M isrh、MrtTo map the matrix, rpA projection vector representing the relationship is calculated,
Figure BDA0002330756930000053
an identity matrix of size m × n, hp TProjection transpose vector, h, representing head entity⊥Representing an embedded representation of the head entity in the relational space after mapping, tp TProjection transpose vector, t, representing the tail entity⊥An embedded representation of the tail entity in the relationship space after the representation mapping.
S2.4 grouping triples (h) in relationship space⊥,r,t⊥) Projecting the data to a hyperplane constructed by corresponding time to obtain a triplet (h)τ,r,tτ) The way to project the triplets onto the hyperplane constructed in response time is:
Figure BDA0002330756930000054
Figure BDA0002330756930000055
Figure BDA0002330756930000056
wherein, ω isτRepresenting the constructed hyperplane corresponding to time τ.
S2.5, calculating a loss function, and updating the embedded representation of the entity and the relation by adopting a gradient descent algorithm, wherein the loss function is
Figure BDA0002330756930000057
Wherein f isτ(x)、fτ(y) a scoring function, wherein x and y represent positive and negative samplesThis, f | | | hτ+rτ-tτ||L1/L2L1, L2 are regularization terms, and γ is the minimum spacing between positive and negative samples.
And (5) repeating the steps (2-1) to (2-5) until the result obtained by the whole algorithm is stable.
S3-1, for a triple (h, R, t), firstly checking whether h and t belong to E, whether R belongs to R, if not, then h, t or R is not suitable as a head-tail entity or relationship, the triple is not established, if yes, then executing the next step;
s3-2 calculates the result of the scoring function according to the triple of the candidate result, wherein the scoring function is shown in the form of- (f | | | h)τ+rτ-tτ||L1/L2And sorting all the candidate results, wherein the result in the top sorting is the optimal result and is used for completing.
The experiment is compared with the current optimal model, HyTE and RE-NET, on a test set. The two indexes of average ranking (Mean Rank) and top 10 recall (Hits @10) are mainly compared. For each triplet (h, r, t) in the test set, when it is determined that the head entity h and the relationship r are to predict the tail entity t, scores are calculated for all possible tail entities in the data set by using an evaluation function, and then the scores are sorted according to the scores, so that if the score of the tail entity t to be predicted is low, the result before comparison can successfully predict that the triplet (h, r, t) is established. Therefore, the ranking of the tail entity t and whether the tail entity t ranks in the top ten can be used as indexes for measuring the algorithm. Similarly, the relationship r and the tail entity t can be determined to predict the head entity h, which can result in the ranking of the head entities and whether the head entities are ranked in the top ten. From tables 2, 3 and 4, it can be seen that the present invention has, overall, better performance than the same type of process.
TABLE 2 comparison with the results of the translation-based model experiments on the link prediction task (for hits @10, the larger the result is the better; for mean rank, the smaller the result is the better, the black bold is the inventive result)
Figure BDA0002330756930000061
TABLE 3 comparison with the results of the translation-based model experiments on the task of relationship prediction (for hits @1, the larger the result is the better, for mean rank, the smaller the result is the better, the black bold is the inventive result)
Figure BDA0002330756930000062
Figure BDA0002330756930000071
TABLE 4 comparison with the experimental results of the neural network model on the link prediction task (for hits @1, hits @3, hits @10, the larger the result the better, for MRR, the larger the result the better, the black bold is the inventive result)
Figure BDA0002330756930000072

Claims (8)

1. A time-based knowledge graph embedding method based on a hybrid translation model is characterized by comprising the following steps:
step 1: inputting a time sequence knowledge graph data set needing to be completed, and carrying out initialization setting according to the size of the data set;
step 2: updating the embedded representation of the knowledge graph by using a hybrid translation model to obtain an embedded representation result;
and step 3: and performing completion operation on the used knowledge graph data set according to the embedded representation result.
2. The method for embedding the knowledge graph with time based on the hybrid translation model according to claim 1, wherein in step 1, the initialization setting comprises the following specific steps:
step A1: randomly initializing entities and relations in the data set, and displaying the entities and relations in a vector form to obtain initial relation vectors and entity vectors;
step A2: for each relationship vector, add to the relationship set R, and for each entity vector, add to the entity set E.
3. The hybrid translation model-based time-of-knowledge-graph immersion method according to claim 1, wherein in step 2, the embedded representation of the knowledge graph is updated using the hybrid translation model by the specific steps of:
step B1: sampling a batch of data with a fixed size of batch from a training set of a data set;
step B2: constructing a negative sample data set, and randomly replacing h, r or t for the triple in the batch to generate an error triple to be added into the batch of data to form training data used by the current batch;
step B3: the correct triplet (h, r, t) is compared with the incorrect triplet [ (h, r ', t), (h ', r, t) or (h, r, t ')]Mapping from the entity space to the relationship space, resulting in the correct triplet (h) in the relationship space⊥,r,t⊥) And error triplet [ (h)⊥,r`,t⊥)、(h⊥`,r,t⊥) Or (h)⊥,r,t⊥`)]；
Step B4: projecting all triples in the relation space onto a hyperplane constructed by the corresponding time tau;
step B5: calculating a loss function, and updating the embedded expression of the entity and the relation by adopting a gradient descent algorithm;
step B6: and repeating the steps B1-B5 until the obtained result is stable.
4. The hybrid translation model-based time-of-knowledge map embedding method according to claim 3, wherein in step B2, the manner of constructing the negative sample data set is:
Figure FDA0002330756920000011
h. r, t, h ', r', t 'represent embedded representations of entities and relationships, where h' represents a randomly substituted head entity, r 'represents a randomly substituted relationship, t' represents a randomly substituted tail entity, D+Representing a set of positive samples, Dx,τ -Representing a negative sample data set.
5. The hybrid translation model-based temporal knowledge graph embedding method according to claim 3, wherein in step B3, the entity space is mapped to the relationship space by:
Figure FDA0002330756920000021
Figure FDA0002330756920000022
wherein M isrh、MrtTo map the matrix, rpProjection vector representing a relation, Im×nAn identity matrix of size m × n, hp TProjection transpose vector, h, representing head entity⊥Representing an embedded representation of the head entity in the relational space after mapping, tp TProjection transpose vector, t, representing the tail entity⊥An embedded representation of the tail entity in the relationship space after the representation mapping.
6. The hybrid translation model-based time-of-knowledge map embedding method according to claim 3, wherein in step B4, all the triples in the relationship space are projected onto the hyperplane constructed by the corresponding time τ in a manner that:
Figure FDA0002330756920000023
Figure FDA0002330756920000024
Figure FDA0002330756920000025
wherein, ω isτRepresenting the constructed hyperplane corresponding to time τ.
7. The hybrid translation model-based time-of-knowledge map embedding method according to claim 3, wherein in step B5, the loss function is:
Figure FDA0002330756920000026
wherein f isτ(x)、fτ(y) a scoring function, x, y representing positive and negative examples, f | | | hτ+rτ-tτ||L1/L2L1, L2 are regularization terms, γ is the minimum interval between positive and negative samples, and T represents the set of all times.
8. The hybrid translation model-based time-of-knowledge map embedding method according to claim 1, wherein the step 3 is specifically:
step C1: for a triple (h, R, t), firstly checking whether h and t belong to E or not, whether R belongs to R or not, if not, then h, t or R is not used as a head-tail entity or relation, the triple does not hold, if so, then executing the next step;
step C2: calculating the scoring function f | | | h according to the established triplesτ+rτ-tτ||L1/L2And sequencing all established triples, wherein the result in the front sequencing is the optimal result and is used for completing.
CN201911335182.7A 2019-12-23 2019-12-23 Time-carrying knowledge graph embedding method based on hybrid translation model Pending CN111209408A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911335182.7A CN111209408A (en) 2019-12-23 2019-12-23 Time-carrying knowledge graph embedding method based on hybrid translation model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911335182.7A CN111209408A (en) 2019-12-23 2019-12-23 Time-carrying knowledge graph embedding method based on hybrid translation model

Publications (1)

Publication Number Publication Date
CN111209408A true CN111209408A (en) 2020-05-29

Family

ID=70786374

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911335182.7A Pending CN111209408A (en) 2019-12-23 2019-12-23 Time-carrying knowledge graph embedding method based on hybrid translation model

Country Status (1)

Country Link
CN (1) CN111209408A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112364108A (en) * 2020-11-13 2021-02-12 四川省人工智能研究院(宜宾) Time sequence knowledge graph completion method based on space-time architecture
CN112380864A (en) * 2020-11-03 2021-02-19 广西大学 Text triple labeling sample enhancement method based on translation
CN112380355A (en) * 2020-11-20 2021-02-19 华南理工大学 Method for representing and storing time slot heterogeneous knowledge graph
CN112395423A (en) * 2020-09-09 2021-02-23 北京邮电大学 Recursive time-series knowledge graph completion method and device
CN113033914A (en) * 2021-04-16 2021-06-25 哈尔滨工业大学 Entity and relation prediction method for machining process knowledge graph
CN113312492A (en) * 2021-05-24 2021-08-27 哈尔滨理工大学 Knowledge graph representation learning method based on dynamic translation
CN114357191A (en) * 2021-12-31 2022-04-15 科大讯飞(苏州)科技有限公司 Knowledge extraction, question answering and recommendation method, related device, equipment and medium

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112395423A (en) * 2020-09-09 2021-02-23 北京邮电大学 Recursive time-series knowledge graph completion method and device
CN112395423B (en) * 2020-09-09 2022-08-26 北京邮电大学 Recursive time sequence knowledge graph completion method and device
CN112380864A (en) * 2020-11-03 2021-02-19 广西大学 Text triple labeling sample enhancement method based on translation
CN112364108A (en) * 2020-11-13 2021-02-12 四川省人工智能研究院(宜宾) Time sequence knowledge graph completion method based on space-time architecture
CN112364108B (en) * 2020-11-13 2021-06-15 四川省人工智能研究院(宜宾) Time sequence knowledge graph completion method based on space-time architecture
CN112380355A (en) * 2020-11-20 2021-02-19 华南理工大学 Method for representing and storing time slot heterogeneous knowledge graph
CN113033914A (en) * 2021-04-16 2021-06-25 哈尔滨工业大学 Entity and relation prediction method for machining process knowledge graph
CN113033914B (en) * 2021-04-16 2022-03-25 哈尔滨工业大学 Entity and relation prediction method for machining process knowledge graph
CN113312492A (en) * 2021-05-24 2021-08-27 哈尔滨理工大学 Knowledge graph representation learning method based on dynamic translation
CN114357191A (en) * 2021-12-31 2022-04-15 科大讯飞(苏州)科技有限公司 Knowledge extraction, question answering and recommendation method, related device, equipment and medium

Similar Documents

Publication Publication Date Title
CN111209408A (en) Time-carrying knowledge graph embedding method based on hybrid translation model
US10719301B1 (en) Development environment for machine learning media models
US20230195845A1 (en) Fast annotation of samples for machine learning model development
US11537506B1 (en) System for visually diagnosing machine learning models
CN111753101B (en) Knowledge graph representation learning method integrating entity description and type
WO2020232874A1 (en) Modeling method and apparatus based on transfer learning, and computer device and storage medium
CN109840595B (en) Knowledge tracking method based on group learning behavior characteristics
JP6962123B2 (en) Label estimation device and label estimation program
CN113377964B (en) Knowledge graph link prediction method, device, equipment and storage medium
Pan et al. Identifying protein complexes from protein-protein interaction networks based on fuzzy clustering and GO semantic information
CN114707641A (en) Training method, device, equipment and medium for neural network model of double-view diagram
CN117828377B (en) Educational perception clustering method and system based on fairness weighting factors
CN106156857A (en) The method and apparatus selected for mixed model
CN110491443B (en) lncRNA protein correlation prediction method based on projection neighborhood non-negative matrix decomposition
CN115269861A (en) Reinforced learning knowledge graph reasoning method based on generative confrontation and imitation learning
Fang et al. Flexible variable selection for recovering sparsity in nonadditive nonparametric models
Manolopoulou et al. BPEC: An R package for Bayesian phylogeographic and ecological clustering
Lonij et al. Open-world visual recognition using knowledge graphs
Li et al. Robust multi-label semi-supervised classification
Song et al. Toward robustness in multi-label classification: A data augmentation strategy against imbalance and noise
CN116128013B (en) Temporary collaboration method and device based on diversity population training and computer equipment
KR20180087069A (en) A method for predicting drug-target interactions via self-training
CN115129029B (en) Industrial system fault diagnosis method and system based on sub-field adaptive dictionary learning
Mosin et al. Comparing input prioritization techniques for testing deep learning algorithms
CN116108189A (en) Quaternion-based knowledge graph embedding method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200529