CN113688233A - Text understanding method for semantic search of knowledge graph - Google Patents
Text understanding method for semantic search of knowledge graph Download PDFInfo
- Publication number
- CN113688233A CN113688233A CN202110870572.5A CN202110870572A CN113688233A CN 113688233 A CN113688233 A CN 113688233A CN 202110870572 A CN202110870572 A CN 202110870572A CN 113688233 A CN113688233 A CN 113688233A
- Authority
- CN
- China
- Prior art keywords
- entity
- text
- semantic
- classification
- semantic vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Animal Behavior & Ethology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a text understanding method for knowledge graph semantic search, which aims at the input text to be understood and comprises the following steps: obtaining semantic information of each word element in the text through a large-scale pre-training model, and generating a semantic vector; identifying an entity type and a relationship type through a convolutional neural network, a first softmax classifier for entity classification and a second softmax classifier for relationship classification based on the semantic vector; based on the semantic vector, performing sequence annotation through a CRF (conditional random access memory), and extracting an entity; classifying the text through a Bi-LSTM model and a third softmax classifier for classifying question sentences based on the semantic vector; and retrieving knowledge graph acquisition information as feedback based on the recognized entity type and relationship type, the extracted entity and the classification result of the text. The invention uses a uniform method to complete four tasks simultaneously, so that the system is more concise.
Description
Technical Field
The invention belongs to the field of natural language processing, and particularly relates to a text understanding method for semantic search of a knowledge graph.
Background
Semantic search refers to a search engine that, instead of being limited to the literal of a requested sentence input by a user, accurately captures the true intention behind the sentence input by the user by looking at the essence through phenomena, and searches the sentence so as to more accurately return a search result that best meets the needs of the user.
In the knowledge graph, semantic search can be carried out through anthropomorphic question asking, under the scene, the question to be retrieved can be clearly described like a person, the text of the question description is analyzed and understood, the result is searched and matched from the knowledge graph, and the result required by the user is returned.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a text understanding method for semantic search of a knowledge graph.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method of text understanding for a knowledge-graph semantic search, the method comprising, for an input text to be understood, the steps of:
obtaining semantic information of each word element in the text through a large-scale pre-training model, and generating a semantic vector;
identifying an entity type and a relationship type through a convolutional neural network, a first softmax classifier for entity classification and a second softmax classifier for relationship classification based on the semantic vector;
based on the semantic vector, performing sequence annotation through a CRF (conditional random access memory), and extracting an entity;
classifying the text through a Bi-LSTM model and a third softmax classifier for classifying question sentences based on the semantic vector;
and retrieving knowledge graph acquisition information as feedback based on the recognized entity type and relationship type, the extracted entity and the classification result of the text.
Preferably, the large scale pre-training model uses one of BERT, RoBERTa, ERINE, Albert, GPT.
Preferably, the classification result of the text includes a fact class, a statistic class, a non-class and a relation class.
A computer storage medium having a computer program stored therein which, when executed, implements the method.
A system for text understanding for knowledge-graph semantic search, the system comprising:
the semantic acquisition module is used for acquiring semantic information of each word element in the text through a large-scale pre-training model and generating a semantic vector;
the entity classification module is used for identifying entity types through a convolutional neural network and a first softmax classifier for entity classification based on the semantic vector;
the relation classification module is used for identifying the relation type through a convolutional neural network and a second softmax classifier for relation classification based on the semantic vector;
the entity extraction module is used for carrying out sequence marking through CRF based on the semantic vector to extract an entity;
the question classification module is used for classifying the texts through a Bi-LSTM model and a third softmax classifier for question classification based on the semantic vector;
and the retrieval feedback module is used for retrieving the knowledge graph acquisition information as feedback based on the recognized entity type and relationship type, the extracted entity and the classification result of the text.
Compared with the prior art, the invention has the beneficial effects that:
1. the intention recognition is realized through question classification, and answers which a user wants are understood, specifically, the problem is divided into four types, namely an actual type, a statistic type, a non-type, a relation type and the like through the model, but the model is not limited to the four types, and the problem can be subdivided or adjusted according to specific conditions, for example, in the professional field of a power grid, the problem can be divided into a maintenance type, an operation and maintenance type, a financial type, a party building type and the like;
2. the information needed for better retrieval in semantic search is guided by identifying the input text segments as different entity types or relationship types;
3. extracting the entity of the input text by an entity extraction method so as to directly match the entity from the knowledge graph;
4. the invention uses a uniform method to complete four tasks simultaneously, so that the system is more concise;
5. since these four tasks are closely related in a single text entry, the effect is better than four separate tasks.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of 4 downstream tasks performed simultaneously according to an embodiment of the present invention.
Fig. 2 is a system architecture diagram of an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
In the description of the present invention, it is to be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.
As shown in fig. 1-2, semantic information of each token (tokens) in the input text is obtained through the same large-scale pre-training model. The large-scale pre-training model may use a general pre-training model such as BERT, RoBERTA, ERINE, Albert, GPT, etc., and may be a special pre-training model similar to a network structure, such as Fin-BERT trained with financial corpora, Ele-BERT trained with power grid domain corpora, Ship-BERT trained with Ship domain, etc.
After the semantic vectors are obtained, the downstream task uses four different models to achieve the four tasks of text understanding.
The entity type identification and the relation type identification share a Convolutional Neural Network (CNN), and the entity type classification and the relation type classification are realized by using respective softmax classifiers, so that the corresponding entity type and the corresponding relation type are understood, and the corresponding key text segment is identified. For example, in the architecture diagram, "who" characterizes "people" in this entity category, and two lemmas, "look" and "play", characterize the relationship of "< people, rehearsal, people >".
And the entity identification task directly uses CRF to label the sequence after the semantic vector, so that entity extraction is realized, and the entities in the input text and the corresponding entity types are extracted. Note that the entity extraction and the entity type identification are different, the entity extraction is to extract which text segments in the text are an entity, and the corresponding entity type, for example, "sunny and Wen" in the architecture diagram is an entity of "person". The entity type identification identifies that the entity is not an entity, but words which may indicate the entity type, such as "who" in the architecture diagram indicates "person" in the entity category.
In question classification, a Bi-LSTM model is used for further understanding from semantic vectors of the lemmas to obtain global semantic information, and the question is classified into different categories, such as four categories in the architecture diagram, including fact category, statistic category, non-category and relationship category. However, the present invention is not limited to a specific category, and may be adjusted to suit the category in different application scenarios. The above figure returns "fact class", and as a result, the entity of a person and various attribute information corresponding to the entity should be returned. For the statistical class, various aggregation algorithms are called to perform statistical calculation, and a statistical result is returned. For example, "do you have several prefectures in Fujian province? "then need to calculate the number of grade city of Fujian province, return the calculated result. For "nongeneric" the logic judgment result is returned, for example, "how is the city of building belong to the province of Fujian? "return result" yes ". For the "relationship class", the relationship between the entities needs to be obtained from the graph. For example, "what is a relationship between the city of the good fortune and the province of the good fortune? "return" < location, province, location > ".
Although the present invention has been described in detail with respect to the above embodiments, it will be understood by those skilled in the art that modifications or improvements based on the disclosure of the present invention may be made without departing from the spirit and scope of the invention, and these modifications and improvements are within the spirit and scope of the invention.
Claims (5)
1. A method of text understanding for knowledge-graph semantic search, characterized in that, for an input text to be understood, the method comprises the steps of:
obtaining semantic information of each word element in the text through a large-scale pre-training model, and generating a semantic vector; identifying an entity type and a relationship type through a convolutional neural network, a first softmax classifier for entity classification and a second softmax classifier for relationship classification based on the semantic vector; based on the semantic vector, performing sequence annotation through a CRF (conditional random access memory), and extracting an entity;
classifying the text through a Bi-LSTM model and a third softmax classifier for classifying question sentences based on the semantic vector;
and retrieving knowledge graph acquisition information as feedback based on the recognized entity type and relationship type, the extracted entity and the classification result of the text.
2. The method for textual understanding of knowledge-graph semantic search of claim 1, wherein the large scale pre-training model uses one of BERT, RoBERTa, ERINE, Albert, GPT.
3. The method for textual understanding of knowledge-graph semantic search of claim 1, wherein the classification results of text include fact, statistical, non-category, and relationship categories.
4. A computer storage medium, characterized in that the storage medium has stored therein a computer program which, when executed, implements the method of any one of claims 1-3.
5. A system for text understanding for knowledge-graph semantic search, the system comprising:
the semantic acquisition module is used for acquiring semantic information of each word element in the text through a large-scale pre-training model and generating a semantic vector;
the entity classification module is used for identifying entity types through a convolutional neural network and a first softmax classifier for entity classification based on the semantic vector;
the relation classification module is used for identifying the relation type through a convolutional neural network and a second softmax classifier for relation classification based on the semantic vector;
the entity extraction module is used for carrying out sequence marking through CRF based on the semantic vector to extract an entity;
the question classification module is used for classifying the texts through a Bi-LSTM model and a third softmax classifier for question classification based on the semantic vector;
and the retrieval feedback module is used for retrieving the knowledge graph acquisition information as feedback based on the recognized entity type and relationship type, the extracted entity and the classification result of the text.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110870572.5A CN113688233A (en) | 2021-07-30 | 2021-07-30 | Text understanding method for semantic search of knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110870572.5A CN113688233A (en) | 2021-07-30 | 2021-07-30 | Text understanding method for semantic search of knowledge graph |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688233A true CN113688233A (en) | 2021-11-23 |
Family
ID=78578347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110870572.5A Pending CN113688233A (en) | 2021-07-30 | 2021-07-30 | Text understanding method for semantic search of knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688233A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114462398A (en) * | 2022-02-15 | 2022-05-10 | 平安科技（深圳）有限公司 | Entity searching method, system, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200103157A (en) * | 2019-02-12 | 2020-09-02 | 주식회사 자이냅스 | Recording midium |
| CN112035635A (en) * | 2020-08-28 | 2020-12-04 | 康键信息技术（深圳）有限公司 | Medical field intention recognition method, device, equipment and storage medium |
| CN112084790A (en) * | 2020-09-24 | 2020-12-15 | 中国民航大学 | Relation extraction method and system based on pre-training convolutional neural network |
| CN112131883A (en) * | 2020-09-30 | 2020-12-25 | 腾讯科技（深圳）有限公司 | Language model training method and device, computer equipment and storage medium |
-
2021
- 2021-07-30 CN CN202110870572.5A patent/CN113688233A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200103157A (en) * | 2019-02-12 | 2020-09-02 | 주식회사 자이냅스 | Recording midium |
| CN112035635A (en) * | 2020-08-28 | 2020-12-04 | 康键信息技术（深圳）有限公司 | Medical field intention recognition method, device, equipment and storage medium |
| CN112084790A (en) * | 2020-09-24 | 2020-12-15 | 中国民航大学 | Relation extraction method and system based on pre-training convolutional neural network |
| CN112131883A (en) * | 2020-09-30 | 2020-12-25 | 腾讯科技（深圳）有限公司 | Language model training method and device, computer equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114462398A (en) * | 2022-02-15 | 2022-05-10 | 平安科技（深圳）有限公司 | Entity searching method, system, equipment and medium |
| CN114462398B (en) * | 2022-02-15 | 2023-05-09 | 平安科技（深圳）有限公司 | Entity searching method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113283551B (en) | Training method and training device of multi-mode pre-training model and electronic equipment | |
| Demir et al. | Improving named entity recognition for morphologically rich languages using word embeddings | |
| CN111325029B (en) | Text similarity calculation method based on deep learning integrated model | |
| CN108595708A (en) | A kind of exception information file classification method of knowledge based collection of illustrative plates | |
| CN111177591B (en) | Knowledge graph-based Web data optimization method for visual requirements | |
| CN107590133A (en) | The method and system that position vacant based on semanteme matches with job seeker resume | |
| CN108763510A (en) | Intension recognizing method, device, equipment and storage medium | |
| CN106951438A (en) | A kind of event extraction system and method towards open field | |
| CN110674252A (en) | High-precision semantic search system for judicial domain | |
| CN106997341B (en) | A kind of innovation scheme matching process, device, server and system | |
| CN113505586A (en) | Seat-assisted question-answering method and system integrating semantic classification and knowledge graph | |
| CN109002473A (en) | A kind of sentiment analysis method based on term vector and part of speech | |
| CN108509521B (en) | Image retrieval method for automatically generating text index | |
| CN111191051B (en) | Method and system for constructing emergency knowledge map based on Chinese word segmentation technology | |
| CN112069312B (en) | Text classification method based on entity recognition and electronic device | |
| CN113360582B (en) | Relation classification method and system based on BERT model fusion multi-entity information | |
| Celikyilmaz et al. | A graph-based semi-supervised learning for question-answering | |
| CN114881043A (en) | Deep learning model-based legal document semantic similarity evaluation method and system | |
| CN111178080A (en) | Named entity identification method and system based on structured information | |
| CN110347812A (en) | A kind of search ordering method and system towards judicial style | |
| CN112380346B (en) | Financial news emotion analysis method and device, computer equipment and storage medium | |
| CN113901224A (en) | Knowledge distillation-based secret-related text recognition model training method, system and device | |
| CN113688233A (en) | Text understanding method for semantic search of knowledge graph | |
| CN117216617A (en) | Text classification model training method, device, computer equipment and storage medium | |
| Ronghui et al. | Application of Improved Convolutional Neural Network in Text Classification. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |