WO2020232943A1 - 用于事件预测的知识图构建方法与事件预测方法 - Google Patents

用于事件预测的知识图构建方法与事件预测方法 Download PDF

Info

Publication number
WO2020232943A1
WO2020232943A1 PCT/CN2019/108129 CN2019108129W WO2020232943A1 WO 2020232943 A1 WO2020232943 A1 WO 2020232943A1 CN 2019108129 W CN2019108129 W CN 2019108129W WO 2020232943 A1 WO2020232943 A1 WO 2020232943A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
events
relationship
knowledge graph
candidate
Prior art date
Application number
PCT/CN2019/108129
Other languages
English (en)
French (fr)
Inventor
张洪铭
刘昕
潘浩杰
宋阳秋
Original Assignee
广州市香港科大***研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州市香港科大***研究院 filed Critical 广州市香港科大***研究院
Priority to US17/613,940 priority Critical patent/US20220309357A1/en
Publication of WO2020232943A1 publication Critical patent/WO2020232943A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/169—Annotation, e.g. comment data or footnotes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]

Definitions

  • the present invention relates to the technical field of natural language processing, in particular to a knowledge graph construction method for event prediction and an event prediction method.
  • Natural language processing is an important direction in the field of computer science and artificial intelligence.
  • natural language processing involves the area of human-computer interaction.
  • Many of the challenges involve natural language understanding, that is, the meaning of computers derived from human or natural language input, and others involve natural language generation.
  • Understanding human language requires complex knowledge of the world.
  • the current large-scale knowledge graphs only focus on entity relationships.
  • knowledge graphs (KGs) 3 formalize words and enumerate their categories and relationships.
  • Typical KGs include WordNet for words, FrameNet for events, and CYc for common sense knowledge. Since the existing knowledge graphs only focus on entity relationships and are limited in size, the application of KGs knowledge graphs in practical applications is limited.
  • the present invention provides a knowledge graph construction method and an event prediction method for event prediction, which can effectively mine activities, states, events and the relationship between them, and can improve the quality and effectiveness of the knowledge graph .
  • embodiments of the present invention provide a knowledge graph construction method for event prediction, including:
  • a knowledge graph of the event is generated.
  • the extraction of multiple events from the candidate sentences according to a preset dependency relationship, so that each event retains the complete semantic information of the corresponding candidate sentence specifically includes:
  • the preset dependency relationship is used to match the event pattern corresponding to the candidate sentence where the verb is located;
  • an event centered on the verb is extracted from the candidate sentence.
  • the preset dependency relationship includes multiple event patterns, and the event pattern includes a connection relationship between one or more words among nouns, prepositions, and adjectives, verbs, and marginal terms.
  • the preprocessing of the pre-collected corpus and extracting multiple candidate sentences from the corpus specifically includes:
  • Natural language processing is performed on the corpus to extract multiple candidate sentences.
  • the use of the preset dependency relationship to match the event pattern corresponding to the candidate sentence where the verb is located specifically includes:
  • syntactic analysis is performed on the candidate sentence where the verb is located, and the event mode corresponding to the candidate sentence where the verb is located is obtained.
  • the extracting the seed relationship between the events from the corpus specifically includes:
  • annotated connectives and the event global statistics are performed on the annotated corpus, and the seed relationship between the events is extracted.
  • the possibility relationship of the event is extracted through the pre-built relationship self-recommendation network model to obtain the candidate event relationship between the events.
  • the embodiments of the present invention have the following beneficial effects: use text mining to extract common grammatical patterns based on dependencies to extract events from the corpus.
  • the event extraction is simpler and has low complexity.
  • the grammatical patterns are based on sentences. With the verb as the center, it can effectively dig out the relationship between activities, states, events and them, and construct a high-quality, effective accidental/possible event knowledge graph.
  • an event prediction method including:
  • event reasoning is performed through the knowledge graph to obtain an accidental event of any one of the events.
  • performing event reasoning on any one of the events through the knowledge graph to obtain an accidental event of any one of the events specifically includes:
  • an event search is performed on any one of the events, and the event corresponding to the maximum event probability is obtained as the accidental event.
  • performing event reasoning on any one of the events through the knowledge graph to obtain an accidental event of any one of the events specifically includes:
  • a relationship search is performed on any one of the events, and an event whose event probability is greater than a preset probability threshold is obtained as the accidental event.
  • the embodiments of the present invention have the following beneficial effects: using text mining to extract common grammatical patterns from dependencies to extract events from the corpus, the event extraction is simpler, and the complexity is low.
  • the verb of the sentence is the center, which can effectively dig out about activities, states, events and the relationship between them, and construct a high-quality and effective accidental/possible event knowledge map.
  • the application of this knowledge map can accurately predict accidental events. It can generate better dialogue responses and has a wide range of application scenarios in the field of human-computer interaction dialogues such as question answering and dialogue systems.
  • Fig. 1 is a flowchart of a method for constructing a knowledge graph for event prediction according to a first embodiment of the present invention
  • Figure 2 is a schematic diagram of an event mode provided by an embodiment of the present invention.
  • Fig. 3 is a schematic diagram of an event extraction algorithm provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a seed mode provided by an embodiment of the present invention.
  • FIG. 5 is a framework diagram of ASER knowledge extraction provided by an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of event relationship types provided by an embodiment of the present invention.
  • Fig. 7 is a flowchart of an event prediction method provided by the second embodiment of the present invention.
  • State is usually described by static verbs and cannot be described as action. For example, "I am knowing” or “I am loving” means action, not status. A typical status expression is "The coffee machine is ready for brewing coffee”.
  • Activities are also called processes. Activities and events are described by event (action) verbs. For example, "The coffee machine is brewing coffee” is an activity.
  • Event The distinguishing feature of an event is that it defines an event as an event that is essentially a countable noun (see Alexander P.D. Mourelatos. 1978. Events, Processes, and States). For the same activity that uses coffee as an example, there is the event "The coffee machine has brewed coffee twice half hour ago", which recognizes the basic adverbial.
  • the first embodiment of the present invention provides a knowledge graph construction method for event prediction, which is executed by a knowledge graph construction device for event prediction, and the knowledge graph construction device for event prediction It can be a computing device such as a computer, a mobile phone, a tablet, a notebook computer, or a server.
  • the method for constructing a knowledge graph for event prediction can be integrated with the device for constructing a knowledge graph for event prediction as one of the functional modules.
  • the knowledge graph construction device for event prediction is executed.
  • the method specifically includes the following steps:
  • S11 Preprocess the pre-collected corpus, and extract multiple candidate sentences from the corpus
  • relevant comments, news articles, etc. can be crawled from the Internet platform, or the corpus can be directly downloaded from a specific corpus.
  • the corpus includes e-books, movie subtitles, news articles, comments, etc. Specifically, you can crawl several comments from the Yelp social media platform, crawl several post records from the Reddit forum, and crawl from the New York Times Take several news articles, crawl several pieces of text data from Wikipedia, obtain movie subtitles from the Opensubtitles2016 corpus, and so on.
  • S15 Generate a knowledge graph of the event according to the event and the candidate event relationship between the events.
  • Forming events based on dependencies can effectively dig out the relationship between activities, states, events and them, and construct a high-quality and effective knowledge graph (ASER KG).
  • the knowledge graph is a mixed graph of events, and each event is a hyper-edge connected to a set of vertices.
  • Each vertex is a word in the vocabulary.
  • set words Represents the set of vertices; and E ⁇ , ⁇ represents the set of hyper-edges, that is, the set of events.
  • (V) ⁇ 0 ⁇ is a subset of the vertex set V power set.
  • Knowledge graph H is a hybrid graph combining hypergraph ⁇ V, ⁇ and traditional graph ⁇ , R ⁇ , where the hyper edges of hypergraph ⁇ V, ⁇ are constructed between vertices, graph ⁇ , R ⁇ The edge is built between events.
  • words conforming to a specific grammatical pattern are used to express contingency, so as to avoid sparse accidents extracted.
  • the grammatical pattern of English is fixed; (2) The semantics of the event is determined by the words inside the event; then the definition of the event can be obtained as follows: a kind of accidental event E i is a plurality of word based on ⁇ w i, 1, ..., w i, Ni ⁇ super edge, where N i is the number of words to be displayed in the event E i, w i, 1, ..., w i, Ni ⁇ V, V represents a vocabulary; E i a pair of words (w i, j, w i , k) follows the syntactic relations e i, (i.e., event pattern given in FIG.
  • j k. w i, j represents a different word, v i represents the only word in the vocabulary. It extracts events from a large-scale unlabeled corpus by analyzing the dependence between words. For example, for accidents (dog, bark), a relationship nsubj is adopted between these two words to indicate that there is a subject-verb relationship between the two words.
  • a fixed event pattern (n 1 -nsubj-v 1 ) is used to extract simple and semantically complete verb phrases to form an event. Since the event pattern is highly accurate, the accuracy of event extraction can be improved.
  • S11 preprocessing the pre-collected corpus, and extracting multiple candidate sentences from the corpus, specifically includes:
  • Natural language processing is performed on the corpus to extract multiple candidate sentences.
  • the natural language processing process mainly includes word segmentation, data cleaning, annotation processing, feature extraction, and modeling based on classification algorithms, similarity algorithms, and the like. It should be noted that the corpus can be English text or Chinese text. When the corpus is English text, the corpus is also required for spell checking, stemming and morphological restoration.
  • S12 said extracting multiple events from the candidate sentences according to the preset dependency relationship, so that each of the events retains the complete semantic information of the corresponding candidate sentence, specifically include:
  • each candidate sentence may contain multiple events, and the verb is the center of each event, in this embodiment of the present invention, the Stanford Dependency Parser8 parser is used to parse each candidate sentence and extract each candidate sentence All verbs in.
  • the preset dependency relationship includes multiple event patterns, and the event pattern includes a connection relationship between one or more words among nouns, prepositions, and adjectives, verbs, and marginal terms.
  • the use of the preset dependency relationship to match the event pattern corresponding to the candidate sentence in which the verb is located specifically includes:
  • syntactic analysis is performed on the candidate sentence where the verb is located, and the event mode corresponding to the candidate sentence where the verb is located is obtained.
  • the'v' in the event pattern pattern listed in Figure 2 represents the verbs in the sentence other than'be','be' represents the'be' verb in the sentence,'n' represents the noun, and'a' represents the adjective ,'P' stands for preposition.
  • Code represents the unique code of the event mode.
  • nsubj nominal subject, noun subject
  • xcomp open clausal complement
  • iobj indirect object, indirect object, that is, all indirect object
  • dobj direct object direct object
  • cop copula, co-verb (such as be ,seem,appear, etc.), (the connection between the proposition subject and the predicate)
  • case, nmod, nsubjpass passive nominal subject, passive noun subject
  • the additional elements of the event are extracted from the candidate sentences to characterize the dependency of the syntax.
  • the code can be loaded into a syntactic analysis tool, such as a Stanford syntactic analysis tool, to perform part-of-speech tagging, syntactic analysis, and entity recognition on the candidate sentence to obtain the event pattern corresponding to the candidate sentence where the verb is located.
  • a syntactic analysis tool such as a Stanford syntactic analysis tool
  • the Stanford Syntactic Analysis Tool integrates three algorithms: Probabilistic Context-Free Grammar (PCFG), Neural Network-based Dependency Syntax Analysis and Conversion-based Dependency Syntax Analysis (ShiftReduce).
  • the embodiment of the present invention defines optional dependencies for each event mode, including but not limited to: advmod (adverbial modifier), amod (adjectival modifier), aux (auxiliary, non-primary verbs and auxiliary words, such as BE, HAVE) SHOULD/C legally wait) and neg (negation modifier), etc.
  • advmod adverbial modifier
  • amod adjectival modifier
  • aux auxiliary, non-primary verbs and auxiliary words, such as BE, HAVE
  • neg neg
  • S123 Extract an event centered on the verb from the candidate sentence according to the event pattern corresponding to the candidate sentence where the verb is located.
  • adding a negative margin term neg to each event mode further ensures that all the extracted events have complete semantics. For example: match the candidate sentence with all event patterns in the dependency relationship to obtain a dependency relationship graph; when a negative dependency edge item neg is found in the dependency relationship graph, the result extracted from the corresponding event pattern is judged as unqualified. Therefore, when the candidate sentence has no object/object connection, the first event mode is used for event extraction; otherwise, the next event mode is used for event extraction in turn.
  • the time complexity of possible event extraction is O(
  • the complexity of event extraction is low.
  • S13: extracting the seed relationship between the events from the corpus specifically includes:
  • annotated connectives and the event global statistics are performed on the annotated corpus, and the seed relationship between the events is extracted.
  • S14 According to the event and the seed relationship between the events, extract the possibility relationship of the event through a pre-built relationship self-recommendation network model to obtain the candidate event relationship between the events , Specifically including:
  • the second is to use a self-recommendation strategy to incrementally annotate more possible relationships to increase the coverage of relationship search.
  • the bootstrapping strategy is a kind of information extraction technology, for example, the Eugene Agichtein and Luis Gravano.2000 tool can be used for bootstrapping strategy.
  • a neural network-based machine learning algorithm is used to perform the bootstrapping of event relationships. For details, refer to the knowledge extraction framework diagram of ASER shown in FIG. 5.
  • the candidate sentence S and the two events E1 and E2 extracted in step 12 are used.
  • E1 and E2 use the GloVe algorithm to map its corresponding word vector to a semantic vector space; among them, one layer of two-way LSTM network is used to encode the word sequence of possible events, and the other layer is two-way
  • the LSTM network is used to encode word sequences.
  • the sequence information is encoded in the final hidden states h E1 , h E2 and h s .
  • the candidate event relationship T includes: temporal relationship (Temporal), contingency relationship (Contingency), comparison relationship (Comparison), development relationship (Expansion), and co-occurrence relationship (Co-Occurrence) .
  • temporal relationship includes the relationship of precedence, succession, and synchronization;
  • the contingency relationship includes the relationship of Reason, Result and Condition;
  • comparison relationship includes contrast (Contrast) and concession (Concession) relationships;
  • development relationship includes connection (Conjunction), instantiation (Instantiation), restatement (Restatement), optional (Alternative), alternative (Chosen Alternative) Relationship with Exception; Co-Occurrence. Please refer to Figure 6 for specific event relationship types.
  • the embodiment of the present invention adopts a pure data-driven text mining method. Since the state is described by static verbs, and the activity event is described by (action) verbs, the embodiment of the present invention takes the verb of the sentence as the center, and digs out information about activities, states, The relationship between events and them constructs a high-quality, effective accidental/possible event knowledge graph.
  • the two-step method of combining PDTB and neural network classifiers is used to extract the possibility relationship between events.
  • the overall complexity can be reduced, and on the other hand, the relationship between more events can be filled incrementally and self-recommended. Improve the coverage and accuracy of relationship search.
  • the second embodiment of the present invention provides an event prediction method, which is executed by an event prediction device, and the event prediction device may be a computing device such as a computer, a mobile phone, a tablet, a laptop, or a server.
  • the event prediction method can be integrated with the event prediction device as one of the functional modules and executed by the event prediction device.
  • the method specifically includes the following steps:
  • S21 Pre-process the pre-collected corpus, and extract multiple candidate sentences from the corpus;
  • the embodiment of the present invention applies the knowledge graph constructed in the first embodiment, adopts the preset accidental event matching mode and the knowledge graph, and can accurately find the matched accidental event through probability statistical reasoning. For example, given a sentence “The dog is chasing the cat, suddenly it barks.” It is necessary to clarify what "it” refers to. Two events “dog is chasing cat” and “it barks” are extracted through step S21-22.
  • performing event reasoning on any one of the events through the knowledge graph to obtain an accidental event of any one of the events specifically includes:
  • an event search is performed on any one of the events, and the event corresponding to the maximum event probability is obtained as the accidental event.
  • Event retrieval includes single-hop reasoning and multi-hop reasoning.
  • single-hop reasoning and two-hop reasoning are used to illustrate the process of event retrieval.
  • f(E h , R 1 , E t ) represents edge strength. If there is no event related to E h through the edge R1, then P(E t
  • R 1 ,E h ) 0, then for any accidental event E′ ⁇ . Among them, ⁇ is the set of accidental events E'. Therefore, by sorting the probabilities, the relevant accident Et corresponding to the maximum probability can be easily retrieved.
  • S represents the number of sentences
  • t represents the set of relations.
  • ⁇ m is the set of intermediate event E m such that (E h, R 1, E m) and (E m, R 2, E t) ⁇ ASER.
  • performing event reasoning on any one of the events through the knowledge graph to obtain an accidental event of any one of the events specifically includes:
  • a relationship search is performed on any one of the events, and an event whose event probability is greater than a preset probability threshold is obtained as the accidental event.
  • Relation retrieval also includes single-hop reasoning and multi-hop reasoning.
  • single-hop reasoning and two-hop reasoning are used to illustrate the event retrieval process.
  • T is the type of relation R, It is the relation collection of relation type T. Where T ⁇ T. Then you can get the most likely relationship:
  • P represents the likelihood scoring function in the above formula (3)
  • R represents the relationship set.
  • E h ) represents the probability of the relationship R based on the event E h , and the specific formula is as follows:
  • the embodiments of the present invention provide many conditional probabilities to display different semantics, to test language understanding problems, and event prediction is more accurate.
  • the knowledge graph construction device used for event prediction includes: at least one processor, such as a CPU, at least one network interface or other user interface, memory, and at least one communication bus.
  • the communication bus is used to implement connection and communication between these components.
  • the user interface may optionally include a USB interface, other standard interfaces, and wired interfaces.
  • the network interface may optionally include a Wi-Fi interface and other wireless interfaces.
  • the memory may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the memory may optionally include at least one storage device located far away from the foregoing processor.
  • the memory stores the following elements, executable modules or data structures, or their subsets, or their extended sets:
  • the processor is used to call a program stored in the memory to execute the method for constructing a knowledge graph for event prediction described in the foregoing embodiment, for example, step S11 shown in FIG. 1. Or, when the processor executes the computer program, the function of each module/unit in the foregoing device embodiments is realized.
  • the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory and executed by the processor to complete the present invention.
  • the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the knowledge graph construction device for event prediction.
  • the knowledge graph construction equipment for event prediction may be computing equipment such as desktop computers, notebooks, palmtop computers, and cloud servers.
  • the knowledge graph construction device for event prediction may include, but is not limited to, a processor and a memory.
  • Those skilled in the art can understand that the schematic diagram is only an example of the knowledge graph construction device for event prediction, and does not constitute a limitation on the knowledge graph construction device for event prediction, and may include more or less components than shown. Or combine some parts, or different parts.
  • the so-called processor can be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), ready-made Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc.
  • the processor is the control center of the knowledge graph construction equipment for event prediction, and connects the entire network with various interfaces and lines.
  • the knowledge graph of event prediction constructs various parts of the equipment.
  • the memory may be used to store the computer program and/or module, and the processor executes the computer program and/or module stored in the memory and calls the data stored in the memory to implement the The knowledge graph of event prediction constructs various functions of the equipment.
  • the memory may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.); the storage data area may store Data (such as audio data, phone book, etc.) created based on the use of mobile phones.
  • the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card , Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • non-volatile memory such as hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card , Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the module/unit integrated in the knowledge graph construction device for event prediction is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the present invention implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electrical carrier signal telecommunications signal
  • software distribution media etc.
  • the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction.
  • the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Machine Translation (AREA)

Abstract

本发明公开了一种用于事件预测的知识图构建方法与事件预测方法，该知识图构建方法通过对预先采集的语料进行预处理，从语料中抽取出多个候选句子；根据预设的依赖关系，从候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；从语料中抽取所述事件之间的种子关系；根据事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对事件进行可能性关系提取，获得事件之间的候选事件关系；根据事件及事件之间的候选事件关系，生成事件的知识图，依据依赖关系中提取常见的语法模式，以从语料中抽取出语义完整的事件，能有效挖掘出关于活动、状态、事件和他们之间的关系，构建出高质量、有效性的知识图。

Description

用于事件预测的知识图构建方法与事件预测方法 技术领域
本发明涉及自然语言处理技术领域，尤其涉及一种用于事件预测的知识图构建方法与事件预测方法。
背景技术
自然语言处理(NLP)计算机科学领域与人工智能领域中的一个重要方向。在自然语言处理面临很多挑战，包括自然语言理解，因此，自然语言处理涉及人机交互的面积。在诸多挑战涉及自然语言理解，即计算机源于人为或自然语言输入的意思，和其他涉及到自然语言生成。理解人类语言需要复杂的世界知识。然而，目前的大规模知识图只关注实体关系，例如知识图(KGs)3通过形式化单词，并枚举单词的类别和关系，典型的KGs包括用于单词的WordNet、用于事件的FrameNet和用于常识知识的CYc。由于现有的知识图只关注实体关系且大小受限，限制了KGs知识图在实际应用中的应用。
发明内容
基于此，本发明提供了一种用于事件预测的知识图构建方法与事件预测方法，其能有效挖掘出关于活动、状态、事件和他们之间的关系，能够提高知识图的质量、有效性。
第一方面，本发明实施例提供了一种用于事件预测的知识图构建方法，包括：
对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
从所述语料中抽取所述事件之间的种子关系；
根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
根据所述事件及事件之间的候选事件关系，生成事件的知识图。
在其中一种实施例中，所述根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息，具体包括：
提取所述候选句子中的动词；
对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式；
根据所述动词所在的候选句子对应的事件模式，从所述候选句子中抽取出以所述动词为中心的事件。
在其中一种实施例中，所述预设的依赖关系包括多种事件模式，所述事件模式包括名词、介词、形容词中一种或多种词语与动词、边缘项之间的连接关系。
在其中一种实施例中，所述对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子，具体包括：
对所述语料进行自然语言处理，抽取出多个候选句子。
在其中一种实施例中，所述对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式，具体包括：
对所述预设的依赖关系中每种事件模式构建一一对应的代码；
根据所述代码，对所述动词所在的候选句子进行句法分析，获得所述动词所在的候选句子对应的事件模式。
在其中一种实施例中，所述从所述语料中抽取所述事件之间的种子关系，具体包括：
利用PDTB中定义的关系，对所述语料中的连接词进行注释；
根据注释后的连接词以及所述事件，对注释后的语料进行全局统计，抽取出所述事件之间的种子关系。
在其中一种实施例中，所述根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系，具体包括：
将种子关系N及其对应的两个事件初始化为一个实例X；
利用所述实例X训练预先构建的神经网络分类器，获得自动标记关系的关系自荐网络模型以及两个事件的可能性关系；
对所述可能性关系进行全局统计，并将置信度大于预设阈值的可能性关系添加到所述实例X中，重新输入到所述关系自荐网络模型进行训练，获得两个事件之间的候选事件关系。
相对于现有技术，本发明实施例具有如下有益效果：使用文本挖掘依据依赖关系中提取常见的语法模式，以从语料中抽取出事件，事件的提取更加简单，复杂度低，语法模式以句子的动词为中心，能有效挖掘出关于活动、状态、事件和他们之间的关系，构建出高质量、有效性的偶然/可能性事件知识图。
第二方面，本发明实施例提供了一种事件预测方法，包括：
对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
从所述语料中抽取所述事件之间的种子关系；
根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
根据所述事件及事件之间的候选事件关系，生成事件的知识图；
对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件。
在其中一种实施例中，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
根据所述知识图，对任意一个所述事件进行事件检索，获取最大事件概率对应的事件，作为所述偶然事件。
在其中一种实施例中，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
根据所述知识图，对任意一个所述事件进行关系检索，获取事件概率大于预设概率阈值的事件，作为所述偶然事件。
相对于现有技术，本发明实施例具有如下有益效果：使用文本挖掘依据依赖关系中提取常见的语法模式从，以从语料中抽取以事件，事件的提取更加简单，复杂度低，语法模式以句子的动词为中心，能有效挖掘出关于活动、状态、事件和他们之间的关系，构建出高质量、有效性的偶然/可能性事件知识图，应用该知识图能够准确预测出偶然事件，能够生成更好的对话响应，在问题解答、对话***等人机交互对话领域上有广泛的应用场景。
附图说明
为了更清楚地说明本发明的技术方案，下面将对实施方式中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明第一实施例提供的用于事件预测的知识图构建方法的流程图；
图2是本发明实施例提供的事件模式的示意图；
图3是本发明实施例提供的事件提取算法的示意图；
图4是本发明实施例提供的种子模式的示意图；
图5是本发明实施例提供的ASER的知识提取框架图；
图6是本发明实施例提供的事件关系类型的示意图；
图7是本发明第二实施例提供的一种事件预测方法的流程图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
在描述本法明实施例前，先对常用的术语进行说明：
状态：状态通常用静态动词来描述，不能被描述为行动。例如“I am knowing”或“I am loving”表示行动，不能表示状态。一个典型的状态表达是“The coffe machine is ready for brewing coffe”。
活动：活动也称为过程。活动和事件都是由事件(动作)动词来描述。例如，“The coffee machine is brewing coffee”是一个活动。
事件：事件的显著特征是其定义了一个事件作为一个本质上是可数名词的事件(参见Alexander P.D.Mourelatos.1978.Events,Processes,and States)。同样使用咖啡作为示例的活动，则有事件“The coffee machine has brewed coffee twice half hour ago”，该事件承认基本状语。
关系：采用了Penn DiscourseTree Bank(PDTB)中定义的关系，例如COMPARISON(比较关系)、CONTINGENCY.(因果关系)。
请参照图1，本发明第一实施例提供了一种用于事件预测的知识图构建方法，该方法由用于事件预测的知识图构建设备执行，所述用于事件预测的知识图构建设备可为电脑、手机、平板电脑、笔记本电脑或者服务器等计算设备，所述用于事件预测的知识图构建方法可作为其中一个功能模块集成与所述用于事件预测的知识图构建设备上，由所述用于事件预测的知识图构建设备来执行。
该方法具体包括以下步骤：
S11：对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
需要说明的是，本发明实施例对语料的采集方式不做具体的限定，例如可以从互联网平台中爬取相关评论、新闻文章等，或者从特定的语料库中直接下载语料集。所述语料包括如电子书、电影字幕、新闻文章、评论等，具体地，可以通过从Yelp社交媒体平台中爬取若干条评论、从Reddit论坛中爬取若干条post记录、从***中爬取若干片新闻文章、从***爬取若干条文本数据、从Opensubtitles2016语料库中获取电影字幕等等。
S12：根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
S13：从所述语料中抽取所述事件之间的种子关系；
S14：根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
S15：根据所述事件及事件之间的候选事件关系，生成事件的知识图。
基于依赖关系形成事件，能有效挖掘出关于活动、状态、事件和他们之间的关系，构建出高质量、有效性的知识图(ASER KG)。该知识图是一个关于事件的混合图，每个事件都是一个连接到一组顶点的超边缘。每个顶点都是词汇表中的一个单词。例如设定单词
Figure PCTCN2019108129-appb-000001
表示顶点集合；以及E∈ε，ε表示超边缘集合，即事件集合。
Figure PCTCN2019108129-appb-000002
(V)\{0}是顶点集合V幂集的子集。同时定义一个事件E i和E j的关系R i,j∈R，R表示关系集合；一种关系类型T∈T，T表示关系类型集合，则知识图H＝{V，ε，R，T}。知识图H是一个结合超图{V，ε}和传统的图{ε，R}的混合图，其中，超图{V，ε}的超边构建在顶点之间，图{ε，R}的边缘建立在事件之间。例如，各包含3个单词的两个偶然事件：E 1＝(i，be,hungry)和E 2＝(i，eat，anything)，这两个偶然事件间有一个关系R 1,2＝Result，Result表示一个关系类型；则可以构建一个基于超图{V，ε}的二部图，该二部图的边建立在单词和事件之间。
本发明实施例采用符合特定语法模式的单词来表示偶然性，避免提取的偶然事件过于稀疏。对于事件，假设均符合以下两个条件：(1)英语的语法模式固定；(2)事件的语义是由事件内部的词语决定；则可以得到事件的定义如下：一种可偶然事件E i是一个基于多个单词{w i,1，…，w i,Ni}的超边缘，其中N i是在事件E i中显示的单词数量，w i,1，…，w i,Ni∈V，V表示词汇表；E i中的一对词(w i,j,w i,k)遵循句法关系e i,j,k(即图2中给出的事件模式)。w i,j表示不同的单词,而v i表示词汇表中唯一的单词。通过对单词间的依赖分析从无标签的大规模语料库中提取事件。例如，偶然事件(dog,bark)，对这两个词之间采用了一个关系nsubj来表示这两个词之间有一个主语-动词关系。使用固定的事件模式(n 1-nsubj-v 1)来提取简单且语义完整的动词短语以形成事件，由于事件模式是高精度，可以提高事件提取的准确性。
在一种可选的实施例中，S11：所述对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子，具体包括：
对所述语料进行自然语言处理，抽取出多个候选句子。
所述自然语言处理的过程主要包括分词、数据清洗、标注化处理、特征提取以及基于分类算法、相似度算法等的建模。需要说明的是所述语料可以是英文文本或中文文本。当语料为英文文本时，还需要语料进行拼写检查处理、词干提取和词形还原处理。
在一种可选的实施例中，S12：所述根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息，具体包括：
S121：提取所述候选句子中的动词；
需要说明的是，由于每个候选句子可能包含了多个事件，而动词是每个事件的中心，在本发明实施例中，采用Stanford Dependency Parser8解析器解析每个候选句子，提取每个候选句子中的所有动词。
S122：对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式；
进一步地，所述预设的依赖关系包括多种事件模式，所述事件模式包括名词、介词、形容词中一种或多种词语与动词、边缘项之间的连接关系。
在一种可选的实施例中，所述对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式，具体包括：
对所述预设的依赖关系中每种事件模式构建一一对应的代码；
根据所述代码，对所述动词所在的候选句子进行句法分析，获得所述动词所在的候选句子对应的事件模式。
本发明实施例采用的事件模式请参见图2。其中，图2中列出的事件模式pattern中的‘v’表示句子中除‘be’外的动词，‘be’表示句子中的‘be’动词，‘n’表示名词，‘a’表示形容词，‘p’表示介词。Code表示事件模式的唯一代码。nsubj(nominal subject，名词主语)、xcomp(open clausal complement)、iobj(indirect object，非直接宾语，也就是所以的间接宾语)、dobj(direct object直接宾语)、cop(copula，系动词(如be,seem,appear等)，(命题主词与谓词间的)连系)、case、nmod、nsubjpass(passive nominal subject，被动的名词主语)分别为连接不同词性词语之间的边缘项，该边缘项为从候选句子中提取事件的附加元素，表征句法的依存关系。
具体地，可以将所述代码加载到句法分析工具，例如斯坦福句法分析工具，对所述候选句子进行词性标注、句法分析和实体识别，获得所述动词所在的候选句子对应的事件模式。斯坦福句法分析工具集成了三种算法：概率上下文无关文法(PCFG)、基于神经网络的依存句法分析和基于转换的依存句法分析(ShiftReduce)。本发明实施例对每个事件模式定义了可选的依存关系，包括但不限于：advmod(adverbial modifier状语)、amod(adjectival modifier形容词)、aux(auxiliary，非主要动词和助词，如BE,HAVE SHOULD/COULD等到)和neg(negation modifier否定词)等，具体可参考斯坦福依存关系。
S123：根据所述动词所在的候选句子对应的事件模式，从所述候选句子中抽取出以所述动词为中心的事件。
进一步地，对每种事件模式添加否定的边缘项neg,进一步确保了抽取出的所有事件具有完整语义。例如：将所述候选句子匹配依赖关系中的所有事件模式，得到依赖关系图；当在依赖关系图中发现否定的依赖边缘项neg，则以对应的事件模式提取出的结果判定为不合格。因此，当所述候选句子没有对象/客体连接时，采用第一种事件模式进行事件提取；否则，依次采用下一事件模式进行事件提取。例如：把句子“I have a book”作为一个例子,通过事件抽取得到<“I”“have”“book”>,而不是<“I”“have”>或<“have”“book”>,作为一个有效的可能性事件,因为<“I”“have”>或<“have”“book”>语义不是完整。
对于语料中候选句子的每一个可能发生的事件模式Pi和动词v,检查所有积极的边缘项(即上表中给出的边缘项)是否被发现与动词v关联。然后将所有匹配边缘项添加到提取的可能性的事件E中；同时将包括所有匹配的潜在的边缘项添加到事件E中，得到该语料的依赖关系图。如果在依赖关系图找到任何在否定的边缘项,则取消提取的事件并返回Null。依据句法分析工具，采用某一事件模式Pi提取可能性的事件的具体的提取算法如图3所示。可能性的事件提取的时间复杂度是O(|S|·|D|·|V|)，|S|是句子的数量,|D|是在依赖解析树中的平均边缘数量,|V|是动词在一个句子的平均数量。事件提取的复杂度低。
在一种可选的实施例中，S13：从所述语料中抽取所述事件之间的种子关系，具体包括：
利用PDTB中定义的关系，对所述语料中的连接词进行注释；
根据注释后的连接词以及所述事件，对注释后的语料进行全局统计，抽取出所述事件之间的种子关系。
在一种可选的实施例中，S14：根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系，具体包括：
将种子关系N及其对应的两个事件初始化为一个实例X；
利用所述实例X训练预先构建的神经网络分类器，获得自动标记关系的关系自荐网络模型以及两个事件的可能性关系；
对所述可能性关系进行全局统计，并将置信度大于预设阈值的可能性关系添加到所述实例X中，重新输入到所述关系自荐网络模型进行训练，获得两个事件之间的候选事件关系。
在本发明实施例中，在从语料中提取事件后，采用两步法提取事件之间的关系：
一是：采用PDTB中定义的显性连接词，采用预设的种子模式挖掘语料的种子关系；所述预设的种子模式如图4所示。由于PDTB中的部分连接词比其他连接词更加含糊不清，例如，在PDTB注释中，连接词while被注释为连接词39次，对比词111次，期望词79次，让步词85次，等等；当识别该连接词时，由于不能确定与之相关的两个事件之间的关系。有些连接词是确定的，例如，连接词so that，它被注释了31次，只与结果关联。在本发明实施 例中，采用特定的连接词，其中，每一个超过90％的注释表示为相同关系，作为提取种子关系的种子模式。
假设一个连接词及其对应关系为c和R，设定一个实例<E 1,c,E 2>表示一个候选句子S；其中，根据依赖解析，两个事件E 1和E2采用连接词c连接。将这个实例作为关系R的一个例子，通过PDTB注释，当被注释为模糊不清的关系越来越少后，为确保提取的种子关系的例子，对每个种子关系R进行全局统计，以查找事件的关系，并将查找到的事件的关系作为种子关系。
二是：采用自荐策略增量地注释更多的可能性关系，以提高关系查找的覆盖率。自举策略是一种信息提取技术，例如可以通过Eugene Agichtein and Luis Gravano.2000工具进行自举策略。本发明实施例中采用基于神经网络的机器学习算法进行事件关系的自举,具体可参见图5所示的ASER的知识提取框架图。
例如：构建基于神经网络的分类器。对于每个提取出的实例X，使用候选句子S和步骤12提取的两个事件E1和E2。对于在S，E1和E2中的每个词采用GloVe算法将其对应的词向量映射到一个语义向量空间；其中，一层双向LSTM网络用于对可能性事件词序列进行编码，另一层双向LSTM网络用于对词序列进行编码。序列信息编码在最后的隐藏状态h E1,h E2和h s中。我们将hE1,hE2,hE1hE2,hE1hE2,hE1hE2和hs串联起来，然后将串联的结果通过ReLU激活函数和损失函数，送入一个两层前馈网络。Softmax函数用于生成此实例的概率分布。我们把交叉熵损失加到每个关系的训练例子上。神经网络分类器的输出预测一对事件被分类到每一个关系的概率。假设对于Ti类型的关系R＝Ti。对于实例X＝<S,E1,E2>，输出P(Ti|X)。在自荐过程中,如果P(Ti|X)>τ，τ为预设的阈值,标签实例作为关系类型Ti。这样，在使用神经网络分类器处理整个语料库的每一步之后，就可以增量地、自动地为神经网络分类器标注更多的训练示例。进一步地，采用Adam optimizer作为分类器，所以复杂度与LSTM的单元L中的参数数量、迭代中自动标注的实例Nt的平均数量、关系类型|T|的数量以及自荐迭代Iter max的数量是线性的，复杂度为O(L·Nt·|T|·Iter max)，总体复杂度更低。
在一种可选的实施例中，所述候选事件关系T包括：时间关系(Temporal)、偶然性关系(Contingency)、比较关系(Comparison)、发展关系(Expansion)、共现关系(Co-Occurrence)。
具体地，时间关系(Temporal)包括优先级(Precedence)、继承(Succession)和同步(Synchronous)关系；偶然性关系(Contingency)包括原因(Reason)、结果(Result)和条件(Condition)关系；比较关系(Comparison)包括对比(Contrast)和让步(Concession)关系；发展关系(Expansion)包括连接(Conjunction)、实例化(Instantiation)、重述(Restatement)、 可选(Alternative)、备选(Chosen Alternative)和异常(Exception)关系；共现关系(Co-Occurrence)。具体的事件关系类型请参看图6。
相对于现有技术，本发明实施例的有益效果在于：
1、本发明实施例采用基于纯数据驱动的文本挖掘方法，由于状态以静态动词描述，活动事件基于(动作)动词描述，本发明实施例以句子的动词为中心，挖掘出关于活动、状态、事件和他们之间的关系，构建出高质量、有效性的偶然/可能性事件知识图。
2、采用PDTB和神经网络分类器结合的两步法提取事件之间的可能性关系，一方面可以降低总体复杂度，另一方方面可以增量、自荐地填充到更多事件之间的关系，提高关系查找的覆盖率和准确性。
3、使用文本挖掘从依赖关系图中提取常见的语法模式以形成事件，事件的提取更加简单，复杂度低。
请参见图7，本发明第二实施例提供了一种事件预测方法，该方法由事件预测设备执行，所述事件预测设备可为电脑、手机、平板电脑、笔记本电脑或者服务器等计算设备，所述事件预测方法可作为其中一个功能模块集成与所述事件预测设备上，由所述事件预测设备来执行。
该方法具体包括以下步骤：
S21：对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
S22：根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
S23：从所述语料中抽取所述事件之间的种子关系；
S24：根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
S25：根据所述事件及事件之间的候选事件关系，生成事件的知识图；
S26：对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件。
本发明实施例应用第一实施例构建的知识图，采用预设的偶然事件匹配模式和知识图，通过概率统计推理能够准确查找匹配的偶然事件。例如给出一个句子“The dog is chasing the cat,suddenly it barks.”这里需要理清“it”具体指代什么内容。通过步骤S21-22抽取两个事件“dog is chasing cat”和“it barks”。由于代词“it”在例子中是无用信息，将“it”替换成“dog”和“cat”生成两个伪事件，并将这四个事件“dog is chasing cat”、“it barks”、“dog barks”和“cat barks”作为知识图的输入，得到“dog barks”出现65次，“cat barks”出现1次，从得到偶然事件为“dog  barks”，偶然事件预测更加准确。具体的三种不同级别的偶然事件匹配模式(单词、框架单词、动词)请参见图7。
在一种可选的实施例中，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
根据所述知识图，对任意一个所述事件进行事件检索，获取最大事件概率对应的事件，作为所述偶然事件。
事件检索包括单跳推理和多跳推理，在本发明实施例中，以单跳推理和两跳推理对事件检索的过程进行说明。事件检索的定义为：设定一个事件E h和一个关系表L＝(R 1，R 2…R k)，找到相关事件E t，则可以找到一条路径，其包含知识图ASER中从E h到E t的所有关系L。
单跳推理：对于单跳推断，由于两个事件之间只有一条边缘，假设该边缘为关系R 1。则任何可能的事件E t的概率如下：
Figure PCTCN2019108129-appb-000003
其中，f(E h,R 1,E t)表示边缘强度。如果通过边缘R1不存在与E h相关的事件，则P(E t|R 1,E h)＝0，那么对于任意偶然事件E′∈ε。其中，ε为偶然事件E′的集合。因此，可以通过对概率进行排序，轻松地检索出最大概率对应的相关的偶然事件Et。S表示句子数量，t表示关系集合。
两跳推理：假设两个事件之间的两个关系依次为R 1和R 2，在公式1的基础上，定义两跳设置下偶然事件E t的概率如下：
Figure PCTCN2019108129-appb-000004
其中,ε m是中间事件E m的集合,使得(E h，R 1，E m)和(E m，R 2，E t)∈ASER。
下面举例对事件检索进行说明：
给定一个事件“I go to the restaurant”,在从知识图ASER检索到相关的偶然事件之后，得到原因关系下的事件为“I am hungry”，继承关系下的事件为“I order food”，即事件“I go to the restaurant”主要因为“I am hungry”，并发生在“I order food”之前。通过知识图ASER了解这些关系后，可以推理出这样的问题“Why do you go to the restaurant？”、“What will you do next？”，而不需要更多的上下文，复杂度低，推理效率更快。
在一种可选的实施例中，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
根据所述知识图，对任意一个所述事件进行关系检索，获取事件概率大于预设概率阈值的事件，作为所述偶然事件。
关系检索也包括单跳推理和多跳推理，在本发明实施例中，以单跳推理和两跳推理对事件检索的过程进行说明。
单跳推理：设定任意两个事件E h和E t，则从E h到E t存在一种关系R的概率为：
Figure PCTCN2019108129-appb-000005
其中，T为关系R的类型，
Figure PCTCN2019108129-appb-000006
为关系类型T的关系集合。其中T∈T。则可以得到最可能的关系为：
Figure PCTCN2019108129-appb-000007
其中，P表示上述公式(3)中似然性评分函数，R表示关系集合。当P(R max|E h,E t)大于0.5时，知识图将返回R max；否则将返回“NULL”。
两跳推理：同样设定任意两个事件E h和E t，则从E h到E t存在一个两跳连接(R 1,R 2)的概率为：
Figure PCTCN2019108129-appb-000008
其中,P(R|E h)表示基于事件E h的关系R的概率，具体公式如下：
Figure PCTCN2019108129-appb-000009
则可以得到对可能的一对关系为：
Figure PCTCN2019108129-appb-000010
与单跳推理类似，P(E h,R 1,max,R 2,max,E t)大于0.5时，知识图将返回R 1,max,R 2,max；否则将返回“NULL”。
相对于现有技术，本发明实施例的有益效果在于：
1、基于上述构建的高质量、有效性的知识图，能够准确预测出偶然事件，能够生成更好的对话响应，在问题解答、对话***等人机交互对话领域上有广泛的应用场景。
2、本发明实施例提供许多条件概率来显示不同的语义，以测试语言理解问题，事件预测更加准确。
该用于事件预测的知识图构建设备包括：至少一个处理器，例如CPU，至少一个网络接口或者其他用户接口，存储器，至少一个通信总线，通信总线用于实现这些组件之间的连接通信。其中，用户接口可选的可以包括USB接口以及其他标准接口、有线接口。网络接口可选的可以包括Wi-Fi接口以及其他无线接口。存储器可能包含高速RAM存储器，也可能还包括非不稳定的存储器(non-volatilememory)，例如至少一个磁盘存储器。存储器可选的可以包含至少一个位于远离前述处理器的存储装置。
在一些实施方式中，存储器存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集:
操作***，包含各种***程序，用于实现各种基础业务以及处理基于硬件的任务；
程序。
具体地，处理器用于调用存储器中存储的程序，执行上述实施例所述的于事件预测的知识图构建方法，例如图1所示的步骤S11。或者，所述处理器执行所述计算机程序时实现上述各装置实施例中各模块/单元的功能。
示例性的，所述计算机程序可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器中，并由所述处理器执行，以完成本发明。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序在所述于事件预测的知识图构建设备中的执行过程。
所述于事件预测的知识图构建设备可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。所述于事件预测的知识图构建设备可包括，但不仅限于，处理器、存储器。本领域技术人员可以理解，所述示意图仅仅是于事件预测的知识图构建设备的示例，并不构成对于事件预测的知识图构建设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件。
所称处理器可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，所述处理器是所述于事件预测的知识图构建设备的控制中心，利用各种接口和线路连接整个于事件预测的知识图构建设备的各个部分。
所述存储器可用于存储所述计算机程序和/或模块，所述处理器通过运行或执行存储在所述存储器内的计算机程序和/或模块，以及调用存储在存储器内的数据，实现所述于事件预测的知识图构建设备的各种功能。所述存储器可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、 插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
其中，所述于事件预测的知识图构建设备集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
以上所述是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也视为本发明的保护范围。

Claims (10)

  1. 一种用于事件预测的知识图构建方法，其特征在于，包括：
    对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
    根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
    从所述语料中抽取所述事件之间的种子关系；
    根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
    根据所述事件及事件之间的候选事件关系，生成事件的知识图。
  2. 如权利要求1所述的用于事件预测的知识图构建方法，其特征在于，所述根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息，具体包括：
    提取所述候选句子中的动词；
    对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式；
    根据所述动词所在的候选句子对应的事件模式，从所述候选句子中抽取出以所述动词为中心的事件。
  3. 如权利要求2所述的用于事件预测的知识图构建方法，其特征在于，所述预设的依赖关系包括多种事件模式，所述事件模式包括名词、介词、形容词中一种或多种词语与动词、边缘项之间的连接关系。
  4. 如权利要求1所述的用于事件预测的知识图构建方法，其特征在于，所述对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子，具体包括：
    对所述语料进行自然语言处理，抽取出多个候选句子。
  5. 如权利要求3所述的用于事件预测的知识图构建方法，其特征在于，所述对每个所述动词，采用所述预设的依赖关系来匹配所述动词所在的候选句子对应的事件模式，具体包括：
    对所述预设的依赖关系中每种事件模式构建一一对应的代码；
    根据所述代码，对所述动词所在的候选句子进行句法分析，获得所述动词所在的候选句子对应的事件模式。
  6. 如权利要求1所述的用于事件预测的知识图构建方法，其特征在于，所述从所述语料中抽取所述事件之间的种子关系，具体包括：
    利用PDTB中定义的关系，对所述语料中的连接词进行注释；
    根据注释后的连接词以及所述事件，对注释后的语料进行全局统计，抽取出所述事件之间的种子关系。
  7. 如权利要求1所述的用于事件预测的知识图构建方法，其特征在于，所述根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系，具体包括：
    将种子关系N及其对应的两个事件初始化为一个实例X；
    利用所述实例X训练预先构建的神经网络分类器，获得自动标记关系的关系自荐网络模型以及两个事件的可能性关系；
    对所述可能性关系进行全局统计，并将置信度大于预设阈值的可能性关系添加到所述实例X中，重新输入到所述关系自荐网络模型进行训练，获得两个事件之间的候选事件关系。
  8. 一种事件预测方法，其特征在于，包括：
    对预先采集的语料进行预处理，从所述语料中抽取出多个候选句子；
    根据预设的依赖关系，从所述候选句子中提取出多个事件，以使得每个所述事件保留对应候选句子的完整语义信息；
    从所述语料中抽取所述事件之间的种子关系；
    根据所述事件及事件之间的种子关系，通过预先构建的关系自荐网络模型对所述事件进行可能性关系提取，获得事件之间的候选事件关系；
    根据所述事件及事件之间的候选事件关系，生成事件的知识图；
    对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件。
  9. 如权利要求8所述的事件预测方法，其特征在于，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
    根据所述知识图，对任意一个所述事件进行事件检索，获取最大事件概率对应的事件，作为所述偶然事件。
  10. 如权利要求8所述的事件预测方法，其特征在于，所述对任意一个所述事件，通过所述知识图进行事件推理，获得任意一个所述事件的偶然事件，具体包括：
    根据所述知识图，对任意一个所述事件进行关系检索，获取事件概率大于预设概率阈值的事件，作为所述偶然事件。
PCT/CN2019/108129 2019-05-23 2019-09-26 用于事件预测的知识图构建方法与事件预测方法 WO2020232943A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/613,940 US20220309357A1 (en) 2019-05-23 2019-09-26 Knowledge graph (kg) construction method for eventuality prediction and eventuality prediction method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910434546.0A CN110263177B (zh) 2019-05-23 2019-05-23 用于事件预测的知识图构建方法与事件预测方法
CN201910434546.0 2019-05-23

Publications (1)

Publication Number Publication Date
WO2020232943A1 true WO2020232943A1 (zh) 2020-11-26

Family

ID=67915181

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/108129 WO2020232943A1 (zh) 2019-05-23 2019-09-26 用于事件预测的知识图构建方法与事件预测方法

Country Status (3)

Country Link
US (1) US20220309357A1 (zh)
CN (1) CN110263177B (zh)
WO (1) WO2020232943A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112633483A (zh) * 2021-01-08 2021-04-09 中国科学院自动化研究所 四元组门图神经网络事件预测方法、装置、设备及介质
CN116108204A (zh) * 2023-02-23 2023-05-12 广州世纪华轲科技有限公司 基于知识图谱融合多维嵌套泛化模式的作文评语生成方法
CN118228079A (zh) * 2024-05-23 2024-06-21 湘江实验室 模糊超图生成方法、装置、计算机设备及存储介质

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263177B (zh) * 2019-05-23 2021-09-07 广州市香港科大***研究院 用于事件预测的知识图构建方法与事件预测方法
CN112417104B (zh) * 2020-12-04 2022-11-11 山西大学 一种句法关系增强的机器阅读理解多跳推理模型及方法
CN112463970B (zh) * 2020-12-16 2022-11-22 吉林大学 一种基于时间关系对文本包含的因果关系进行抽取的方法
CN113569572B (zh) * 2021-02-09 2024-05-24 腾讯科技（深圳）有限公司 文本实体生成方法、模型训练方法及装置
US11954436B2 (en) * 2021-07-26 2024-04-09 Freshworks Inc. Automatic extraction of situations
CN114357197B (zh) * 2022-03-08 2022-07-26 支付宝(杭州)信息技术有限公司 事件推理方法和装置
US20230359825A1 (en) * 2022-05-06 2023-11-09 Sap Se Knowledge graph entities from text
CN115826627A (zh) * 2023-02-21 2023-03-21 白杨时代(北京)科技有限公司 一种编队指令的确定方法、***、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103999081A (zh) * 2011-12-12 2014-08-20 国际商业机器公司 生成用于信息领域的自然语言处理模型
US20150127323A1 (en) * 2013-11-04 2015-05-07 Xerox Corporation Refining inference rules with temporal event clustering
CN107358315A (zh) * 2017-06-26 2017-11-17 深圳市金立通信设备有限公司 一种信息预测方法及终端
CN107656921A (zh) * 2017-10-10 2018-02-02 上海数眼科技发展有限公司 一种基于深度学习的短文本依存分析方法
CN109446341A (zh) * 2018-10-23 2019-03-08 国家电网公司 知识图谱的构建方法及装置
CN110263177A (zh) * 2019-05-23 2019-09-20 广州市香港科大***研究院 用于事件预测的知识图构建方法与事件预测方法

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7505989B2 (en) * 2004-09-03 2009-03-17 Biowisdom Limited System and method for creating customized ontologies
JP5594225B2 (ja) * 2011-05-17 2014-09-24 富士通株式会社 知識獲得装置、知識取得方法、及びプログラム
CN103699689B (zh) * 2014-01-09 2017-02-15 百度在线网络技术（北京）有限公司 事件知识库的构建方法及装置
US10102291B1 (en) * 2015-07-06 2018-10-16 Google Llc Computerized systems and methods for building knowledge bases using context clouds
CN107038263B (zh) * 2017-06-23 2019-09-24 海南大学 一种基于数据图谱、信息图谱和知识图谱的搜索优化方法
CN107480137A (zh) * 2017-08-10 2017-12-15 北京亚鸿世纪科技发展有限公司 用语义迭代提取网络突发事件并识别外延事件关系的方法
CN107908671B (zh) * 2017-10-25 2022-02-01 南京擎盾信息科技有限公司 基于法律数据的知识图谱构建方法及***
CN109657074B (zh) * 2018-09-28 2023-11-10 北京信息科技大学 基于地址树的新闻知识图谱构建方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103999081A (zh) * 2011-12-12 2014-08-20 国际商业机器公司 生成用于信息领域的自然语言处理模型
US20150127323A1 (en) * 2013-11-04 2015-05-07 Xerox Corporation Refining inference rules with temporal event clustering
CN107358315A (zh) * 2017-06-26 2017-11-17 深圳市金立通信设备有限公司 一种信息预测方法及终端
CN107656921A (zh) * 2017-10-10 2018-02-02 上海数眼科技发展有限公司 一种基于深度学习的短文本依存分析方法
CN109446341A (zh) * 2018-10-23 2019-03-08 国家电网公司 知识图谱的构建方法及装置
CN110263177A (zh) * 2019-05-23 2019-09-20 广州市香港科大***研究院 用于事件预测的知识图构建方法与事件预测方法

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112633483A (zh) * 2021-01-08 2021-04-09 中国科学院自动化研究所 四元组门图神经网络事件预测方法、装置、设备及介质
CN112633483B (zh) * 2021-01-08 2023-05-30 中国科学院自动化研究所 四元组门图神经网络事件预测方法、装置、设备及介质
CN116108204A (zh) * 2023-02-23 2023-05-12 广州世纪华轲科技有限公司 基于知识图谱融合多维嵌套泛化模式的作文评语生成方法
CN116108204B (zh) * 2023-02-23 2023-08-29 广州世纪华轲科技有限公司 基于知识图谱融合多维嵌套泛化模式的作文评语生成方法
CN118228079A (zh) * 2024-05-23 2024-06-21 湘江实验室 模糊超图生成方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN110263177A (zh) 2019-09-20
US20220309357A1 (en) 2022-09-29
CN110263177B (zh) 2021-09-07

Similar Documents

Publication Publication Date Title
WO2020232943A1 (zh) 用于事件预测的知识图构建方法与事件预测方法
US20210232613A1 (en) Automatically generating natural language responses to users&#39; questions
Qi et al. Openhownet: An open sememe-based lexical knowledge base
US20220318505A1 (en) Inducing rich interaction structures between words for document-level event argument extraction
US11720756B2 (en) Deriving multiple meaning representations for an utterance in a natural language understanding (NLU) framework
CN111143576A (zh) 一种面向事件的动态知识图谱构建方法和装置
Ma et al. Easy-to-deploy API extraction by multi-level feature embedding and transfer learning
WO2013088287A1 (en) Generation of natural language processing model for information domain
US11397859B2 (en) Progressive collocation for real-time discourse
Li et al. A relation extraction method of Chinese named entities based on location and semantic features
US20220245353A1 (en) System and method for entity labeling in a natural language understanding (nlu) framework
Bahcevan et al. Deep neural network architecture for part-of-speech tagging for turkish language
US20220245361A1 (en) System and method for managing and optimizing lookup source templates in a natural language understanding (nlu) framework
Ferrario et al. The art of natural language processing: classical, modern and contemporary approaches to text document classification
US20220238103A1 (en) Domain-aware vector encoding (dave) system for a natural language understanding (nlu) framework
US20220229994A1 (en) Operational modeling and optimization system for a natural language understanding (nlu) framework
US20220237383A1 (en) Concept system for a natural language understanding (nlu) framework
US11954436B2 (en) Automatic extraction of situations
Gao et al. Chinese causal event extraction using causality‐associated graph neural network
Shams et al. Intent Detection in Urdu Queries Using Fine-Tuned BERT Models
US20230229936A1 (en) Extraction of tasks from documents using weakly supervision
Nasim et al. Modeling POS tagging for the Urdu language
US20220229990A1 (en) System and method for lookup source segmentation scoring in a natural language understanding (nlu) framework
US20220229986A1 (en) System and method for compiling and using taxonomy lookup sources in a natural language understanding (nlu) framework
US20220229987A1 (en) System and method for repository-aware natural language understanding (nlu) using a lookup source framework

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19929359

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19929359

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19929359

Country of ref document: EP

Kind code of ref document: A1