CN114818737A - Method, system and storage medium for extracting semantic features of scientific and technological paper data text - Google Patents
Method, system and storage medium for extracting semantic features of scientific and technological paper data text Download PDFInfo
- Publication number
- CN114818737A CN114818737A CN202210745539.4A CN202210745539A CN114818737A CN 114818737 A CN114818737 A CN 114818737A CN 202210745539 A CN202210745539 A CN 202210745539A CN 114818737 A CN114818737 A CN 114818737A
- Authority
- CN
- China
- Prior art keywords
- scientific
- paper
- matrix
- semantic
- semantic features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The invention provides a method, a system and a storage medium for extracting semantic features of scientific and technical paper data texts, wherein the method comprises the following steps: acquiring text information of a scientific and technological paper, and constructing an entity relationship graph based on the acquired text information of the scientific and technological paper, wherein the text information comprises a paper title and a keyword, a node in the entity relationship graph is the paper title or the keyword, and an edge in the entity relationship graph is an association relationship between the nodes; extracting semantic features based on the acquired text information of the scientific and technical paper to obtain a semantic feature matrix; determining an original adjacency matrix based on an entity relationship graph, and inputting a semantic feature matrix and the original adjacency matrix into a graph network model to obtain a spatial feature matrix; and performing feature fusion on the semantic feature matrix and the spatial feature matrix to obtain the final semantic features of the scientific and technological paper. The feature extraction method can well extract the semantic features of the scientific and technical paper by utilizing the spatial correlation of the knowledge map on the basis of extracting the semantic features of the linguistic data of the scientific and technical paper.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a system for extracting semantic features of scientific and technological paper data texts and a storage medium.
Background
Scientific and technological papers are taken as important research result showing and information obtaining sources, a large number of scientific and technological papers are published almost every day, the academic results comprise a plurality of latest professional field information, and the technology and technological papers are effectively and quickly obtained and semantic feature representation and learning are particularly important. However, scientific and technical paper data often contains a large number of complex attributes, such as abstracts, keywords, cited documents, and the like of the papers, the association between the papers is tighter, and in addition, a large amount of professional knowledge in the papers covers a wide range of disciplines, so that a large amount of professional knowledge is required for feature extraction of the scientific and technical papers; the feature of effectively extracting the thesis data can provide support for processing the scientific and technical thesis data.
TF-IDF (term frequency-inverse document frequency) is a traditional text feature extraction mode. It uses word frequency and inverse document frequency to represent documents as a multidimensional vector representation of keyword weights, which is a typical vector space model. Mikolov et al introduced a Word vector representation model Word2Vec based on the continuous bag of words model (CBOW) and Skip-Gram model, and the whole NLP field quickly entered the embedded world. The traditional coding mode mainly takes onehot coding as a main mode, vectors obtained by the coding mode are often sparse, Word vectors trained by Word2Vec are low-dimensional and dense, context information of words is effectively utilized, and semantic information of the vectors is richer. Li et al uses Word2vec algorithm to process semantic gap and implement Word frequency-document reciprocal frequency (TF-IDF) weighted mapping of HTTP traffic to construct low-dimensional paragraph vector representation to reduce complexity; after Word2vec is retrained, the semantic vector of each Word is not changed, and different vectors cannot be obtained by combining context semantics. Peters et al propose an ELMo model in order to solve the problem of lack of context adaptation in Word2 vec; different from the characteristic that semantic expression vectors acquired by a static word embedding expression model are not changed, ELMo needs to be pre-trained on a large-scale corpus at first, and after the pre-training is finished, fine adjustment is performed according to a specific application field, so that the purpose of field adaptation is achieved, and a word can acquire special vectors according to the current context. The GPT also adopts a corpus to obtain a pre-training model, and then fine tuning is carried out through a small-scale corpus, compared with ELMo, the GPT and the ELMo have the main difference that the network structures adopted for feature extraction are different, the GPT adopts a Transformer, and the ELMo adopts an LSTM. The Transformer is an end-to-end sequence model proposed by ***, and on the basis of the model, a plurality of improved methods are widely applied to the fields of natural language processing, even images and the like; compared with the traditional sequence model, the Transformer completely adopts an attention mechanism to form a network, and the whole network is formed by the structures of an encoder and a decoder; based on the method, a BERT model is further provided, the BERT model shields partial words in the corpus to perform a prediction task through a mask mechanism, so that pre-training of the model is performed, and context semantics of the text are effectively extracted by adopting a bidirectional coding mode.
The traditional pre-training model mainly adopts a one-way network or simply splices a two-way network, the context relation of a text cannot be effectively mined by the network structure, BERT emphasizes that context information is obtained in an omnidirectional manner, and the whole model is constructed by a deep two-way Transformer unit, so that a deep two-way language representation capable of fusing the context information is finally generated. In the pre-training process, the BERT respectively adopts a mask Language Model and a Next sequence Prediction task to perform pre-training, wherein the mask Language Model randomly replaces partial words in the corpus and enables the Model to predict the replaced words so as to realize pre-training, so that the Model can effectively learn the characteristics of a single Sentence, however, for common questions, answers and other sentences, the context relationship of the Sentence needs to be captured, and the mask Language Model task tends to extract the word level representation and cannot directly acquire the Sentence level characteristics. In order to enable the model to understand the relationship between sentences, BERT uses a Next sequence Prediction task for pre-training, namely, a training task is carried out by judging whether two sentences are context-related; the BERT carries out the two pre-training tasks through a deep network and large-scale corpora to obtain rich text semantic representation, and on the basis, the best performance at the time is obtained in a plurality of downstream tasks of the NLP.
In the existing method for extracting features by using a pre-training corpus-based model, semantic features are extracted aiming at text contexts, however, for scientific and technical papers, a large number of associations exist among various attributes of the papers, particularly keywords and titles, and the papers can also be associated through the co-occurrence relationship of the keywords, the attributes cover main semantic information of the papers, and the existing feature extraction method ignores the association among the attributes of the papers and cannot extract features from the contexts and the association relationship of the papers at the same time, so that the existing feature extraction method cannot well extract semantic features of the scientific and technical papers. Therefore, how to better extract semantic features of the scientific and technical papers is a technical problem to be solved urgently.
Disclosure of Invention
In view of the above, the present invention provides a method, a system, and a storage medium for extracting semantic features of a scientific and technological paper data text, so as to solve one or more problems in the prior art.
According to one aspect of the invention, the invention discloses a method for extracting semantic features of a scientific and technical paper data text, which comprises the following steps:
acquiring text information of a scientific and technological thesis, and constructing an entity relationship graph based on the acquired text information of the scientific and technological thesis, wherein the text information comprises a thesis title and a keyword, nodes in the entity relationship graph are the thesis title or the keyword, and edges in the entity relationship graph are incidence relations among the nodes;
extracting semantic features based on the acquired text information of the scientific and technical paper to obtain a semantic feature matrix;
determining an original adjacency matrix based on the entity relationship graph, and inputting the semantic feature matrix and the original adjacency matrix into a graph network model to obtain a spatial feature matrix;
and performing feature fusion on the semantic feature matrix and the spatial feature matrix to obtain the final semantic features of the scientific and technological thesis.
In some embodiments of the present invention, extracting semantic features based on the obtained text information of the scientific and technological paper to obtain a semantic feature matrix, where the semantic feature matrix includes:
and inputting the acquired text information of the scientific and technological paper to a BERT model to obtain a semantic feature matrix.
In some embodiments of the present invention, constructing an entity relationship graph based on the obtained text information of the scientific and technical paper includes:
and calculating the correlation between any two nodes through a point-to-point mutual information algorithm based on the acquired text information of the scientific and technological paper, and constructing an entity relation graph based on the calculated correlation.
In some embodiments of the present invention, the calculation formula of the point-by-point mutual information algorithm is:
wherein the content of the first and second substances,W i representing nodesi,W j Representing nodesj,,,,MShows the text abstract length of scientific paper, CW i, W j ) Representing nodesiAnd nodejNumber of co-occurrences in the text abstract of the same scientific paper, C: (W i ) Representing nodesiTotal number of occurrences in the summary, C: (W j ) Representing nodesjTotal number of occurrences in the summary.
In some embodiments of the invention, the graphical network model comprises a plurality of convolutional layers, the output of each convolutional layer being:
wherein, the first and the second end of the pipe are connected with each other,L (i) is as followsiThe output of the layer(s) is,L (i-1) is as followsi-1The output of the layer(s) is,ρin order to activate the function(s),W (i) in order to be the parameters of the model,is the laplace transform of the adjacency matrix,and D is a degree matrix for normalization.
In some embodiments of the invention, the method further comprises:
calculating the cosine similarity of any two nodes in the entity relationship graph based on the obtained final semantic features;
obtaining a reconstructed adjacency matrix based on the cosine similarity;
and calculating loss values of the original adjacency matrix and the reconstructed adjacency matrix, and optimizing parameters of the graph network model based on the loss values.
In some embodiments of the invention, the loss function of the graph network model is:
wherein;Sis a matrix of the cosine similarity, and,Ain the form of an original adjacency matrix,Fis composed ofFAnd (4) norm.
In some embodiments of the invention, the data is represented by a formulaZ=λZ GCN +(1-λ) Z BERT Performing feature fusion on the semantic feature matrix and the spatial feature matrix; wherein the content of the first and second substances,Z GCN in the form of a matrix of spatial features,Z BERT in the form of a matrix of semantic features,λis hyper-parametric, andλ∈(0,1)。
according to another aspect of the present invention, a scientific paper data text semantic feature extraction system is also disclosed, which comprises a processor and a memory, wherein the memory stores computer instructions, the processor is used for executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system realizes the steps of the method according to any one of the above embodiments.
According to yet another aspect of the present invention, a computer-readable storage medium is also disclosed, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the steps of the method according to any of the embodiments above.
The method and the system for extracting the text semantic features of the scientific and technological paper data disclosed by the invention are characterized in that a paper semantic feature matrix is obtained firstly, a spatial feature matrix is determined through a graph network model, and finally the semantic feature matrix and the spatial feature matrix are subjected to feature fusion to obtain the final semantic features of the scientific and technological paper; the method and the system enrich semantic representation of the thesis by using spatial correlation of the knowledge map on the basis of extracting semantic features of the linguistic data of the scientific and technical thesis, thereby being capable of well extracting the semantic features of the scientific and technical thesis.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to what has been particularly described hereinabove, and that the above and other objects that can be achieved with the present invention will be more clearly understood from the following detailed description.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. For purposes of illustrating and describing some portions of the present invention, corresponding parts of the drawings may be exaggerated, i.e., may be larger, relative to other components in an exemplary apparatus actually manufactured according to the present invention. In the drawings:
fig. 1 is a schematic flow chart of a method for extracting semantic features of a scientific and technical paper data text according to an embodiment of the present invention.
Fig. 2 is a schematic flow chart of a method for extracting semantic features of a scientific paper data text according to another embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a system for extracting semantic features of a scientific paper data text according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not closely related to the present invention are omitted.
It should be emphasized that the term "comprises/comprising/comprises/having" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same or similar parts, or the same or similar steps.
Fig. 1 is a flowchart illustrating a method for extracting semantic features of a scientific paper data text according to an embodiment of the present invention, and as shown in fig. 1, the method for extracting semantic features of a scientific paper data text at least includes steps S10 to S40.
Step S10: the method comprises the steps of obtaining text information of a scientific and technical paper, and constructing an entity relation graph based on the obtained text information of the scientific and technical paper, wherein the text information comprises a paper title and a keyword, nodes in the entity relation graph are the paper title or the keyword, and edges in the entity relation graph are incidence relations among the nodes.
In this step, the text data of the scientific paper may be preprocessed first, and then the keywords of the scientific paper may be extracted to obtain the text information including the title content and the keywords content. Furthermore, based on the structural characteristics of the scientific paper, the association between the title of the scientific paper entity and the keyword is established. Specifically, an entity relationship graph G is constructed based on the acquired text information of the scientific and technological paper, the G is an undirected graph, nodes in the undirected graph are titles or keywords, and edges between the nodes represent the association relationship between two corresponding nodes. In constructing a relationship graphGAnd then, establishing edges of title nodes and keyword nodes of the scientific paper entity by using the association relationship between the title and the keyword or between the keyword and the keyword of the scientific paper.
The constructing an entity relationship graph based on the acquired text information of the scientific and technical paper may include: and calculating the correlation between any two nodes through a point-to-point mutual information algorithm based on the acquired text information of the scientific and technological paper, and constructing an entity relation graph based on the calculated correlation. It should be understood that the construction of the entity relationship graph based on the correlation between nodes listed herein is merely an example, and in addition, the entity relationship graph can be constructed according to other methods; besides the point-by-point mutual information algorithm, other algorithms can also be adopted when calculating the correlation between the two nodes.
Further, the calculation formula of the point-by-point mutual information algorithm is as follows:
wherein the content of the first and second substances,W i representing nodesi,W j Representing nodesjExemplary, a nodeiAs title node, nodejAs the nodes of the key words,PMI(W i ,W j ) Representing nodesiAnd nodejThe score of the correlation between the two or more,,,,Mrepresenting the number of headings in the entity relationship graph, which can also be considered as the number of documents,C(W i, W j )representing nodesiAnd nodejThe number of co-occurrences in the text abstract of the same scientific paper,C(W i )representing nodesiThe total number of times that it occurs in the summary,C(W j )representing nodesjTotal number of occurrences in the summary. In the embodiment, association between the paper and the keywords adopts an index based on word frequency-inverse document frequency (TF-IDF) to define the association degree of the words and the documents, namely two nodesi And j the weight of the edge in between is defined as: for the association between keywords, the point-by-Point Mutual Information (PMI) is used to calculate the score, two nodes with scores larger than a certain value can be considered to have the association relationship, and at the moment, the two nodes corresponding to the nodes have the association relationshipGenerating edges between nodes of the relationship; similarly, two nodes with scores smaller than a certain value can be considered to have no association relationship, and then no edge is generated between the two corresponding nodes without association relationship.
Step S20: and extracting semantic features based on the acquired text information of the scientific and technical paper to obtain a semantic feature matrix.
In this step, the semantic features of the scientific paper may be extracted based on the pre-trained language model, that is, the text information of the scientific paper acquired in step S10 is input into the pre-trained language model to obtain the semantic feature matrix of the scientific paper.
The pre-training language model may be a BERT model, a Word2Vec model, an ELMo model, etc. When the pre-training language model is a BERT model, extracting semantic features based on the acquired text information of the scientific and technical paper to obtain a semantic feature matrix, which specifically comprises the following steps: and inputting the acquired text information of the scientific and technological thesis into a BERT model to obtain a semantic feature matrix. The network structure of the BERT model is mainly formed by a Transformer and is trained in an unsupervised mode. When the model is trained, a data sample set is determined, and an initial network model is trained on the basis of the data samples to optimize network parameters, so that a trained BERT model is obtained.
Illustratively, the output of the BERT model may be expressed asX,XA semantic feature matrix representing a scientific paper;document node embedding code composed ofIs shown in whichdIs the dimension of the embedding, and,X doc a semantic feature matrix representing correspondence of a title in the text information, andX word representing a semantic feature matrix corresponding to a keyword in the text information,n doc indicates the number of titles in the text information, andn word indicating the number of keywords in the text information,whilen doc +n word But can also be viewed as the total number of nodes in the entity relationship graph.
Step S30: and determining an original adjacent matrix based on the entity relationship graph, and inputting the semantic feature matrix and the original adjacent matrix into a graph network model to obtain a spatial feature matrix.
The original adjacency matrix may be written asA,AIs composed ofN×NThe matrix is a matrix of a plurality of matrices,Nis the number of nodes in the entity relationship graph, anA∈R N×N In a contiguous matrixAIn, if the nodeiAnd nodejConnected and then correspondingA ij =1; if nodeiAnd nodejIf not connected, it is correspondingA ij = 0; from this, the matrixAThe element in (1) is 0 or 1. It should be understood that setting the elements of the adjacency matrix to 0 or 1 respectively is only a preferred example, and in other application scenarios, the adjustment may be performed according to the actual application.
In this step, the original adjacency matrix is divided intoAAnd inputting the semantic feature matrix acquired in the step S20 into the trained graph network model for learning, thereby obtaining a spatial feature matrix of the scientific and technological paper. Wherein the embedded features are obtained based on the BERT model and are taken as a relational graphGCharacteristic representation of middle nodes, i.e.XAnd inputting the initial characteristic matrix serving as the node into a trained graph network model for learning. When the graph network model is trained, similar to the BERT model, a training sample set is firstly constructed, the training sample set comprises a plurality of sample data, the initial graph network model is trained based on the sample data to update the network parameters of the network model, and therefore the trained graph network model is obtained.
Illustratively, the graphical network model comprises a plurality of convolutional layers, i.e. an initial feature matrix as input of the graphical network model and an original adjacency matrix, in particular as input of the first convolutional layer of the graphical network model. In the graph network modeliThe output of a layer convolution layer can be expressed as:(ii) a Wherein the content of the first and second substances,L (i) is as followsiThe output of the layer(s) is,L (i-1) is as followsi-1The output of the layer(s) is,ρin order to activate the function(s),W (i) in order to be the parameters of the model,is the laplace transform of the adjacency matrix,and D is a degree matrix. And the final output of the graphical network model can be represented as:Z GCN =g(X,A);Ain the form of an original adjacency matrix,Xin the form of a matrix of semantic features,Z GCN is a spatial feature matrix.
Step S40: and performing feature fusion on the semantic feature matrix and the spatial feature matrix to obtain the final semantic features of the scientific and technological paper.
In this step, the semantic feature vector obtained in step S20 and the spatial feature vector obtained in step S30 are subjected to vector fusion, so as to obtain the final semantic features of the scientific paper. In the step, by combining the advantages of the BERT module and the GCN module, large-scale pre-training of a large amount of original scientific and technical paper data is utilized, and on the basis of extracting semantic features of the scientific and technical paper corpus, the semantic representation of the scientific and technical paper is enriched by utilizing the spatial correlation of the knowledge map.
In fact, directly utilizeWhen the downstream classification task is carried out, the convergence speed and the final classification result are not as good as the characteristic vector of the original BERT, so the invention constructs a weighted output by utilizing a residual error network to synthesize the characteristic representation and the semantic characteristic representation of the GCN; i.e. by the formulaZ=λZ GCN +(1-λ) Z BERT Performing feature fusion on the semantic feature matrix and the spatial feature matrix; it is composed ofIn (1),Z GCN in the form of a matrix of spatial features,Z BERT in the form of a matrix of semantic features,λis hyper-parametric, andλ∈(0,1)。λa weight between the feature representation used to control the GCN output and the original semantic features;λ=1 means that representation learning is performed on scientific papers using GCN model only, andmeaning that only the BERT module is adopted to carry out representation learning on scientific and technical papers; thus in the present inventionWhen is coming into contact withIn time, the predictions from the two models can be balanced, so that the BERT-GCN model can better optimize the output result; operating BERT directly on GCNλValue of, ensure GCNλThe values are adjusted and optimized towards the target; this helps the multilayer GCN model to overcome inherent defects such as gradient disappearance or excessive smoothness, and thus better performance is obtained. According to the embodiment, the method for extracting the text semantic features of the scientific and technological paper data can be found out that the original semantic features acquired by the BERT and the features output by the graph convolution network are weighted by using the residual error network, and the semantic information and the associated information in the paper text data are fully mined.
In an embodiment, the method for extracting semantic features of a scientific and technological paper data text further includes: calculating the cosine similarity of any two nodes in the entity relationship graph based on the obtained final semantic features; obtaining a reconstructed adjacency matrix based on the cosine similarity; and calculating loss values of the original adjacency matrix and the reconstructed adjacency matrix, and optimizing parameters of the graph network model based on the loss values.
Namely, in the pre-training process, link prediction is adopted as a pre-training task, so that the final output is realizedThe structural features of the graph can be learned. For nodei、jIn other words, the coded characteristics areZi,ZjThen calculating the cosine similarity of the two pointsS ij Finally, the reconstructed adjacency matrix is obtainedS. And calculating a loss value between the original adjacent matrix A and the reconstruction matrix S through a cross entropy loss function to serve as an optimization target. Illustratively, the loss function of the graph network model is:(ii) a Wherein; s is a cosine similarity matrix, A is an adjacency matrix, and F isFAnd (4) norm.
Fig. 2 is a schematic flow chart of a method for extracting semantic features of a scientific and technological paper data text according to another embodiment of the present invention, as shown in fig. 2, in the method for extracting semantic features of a scientific and technological paper data text, first, original scientific and technological paper data is obtained; further extracting keywords from the scientific and technical papers; constructing a keyword relation based on the relation among the keywords; further extracting original semantic features of the scientific and technical paper to obtain a semantic feature matrix; performing spatial correlation feature extraction on the scientific paper based on the adjacent matrix corresponding to the entity relational graph and the obtained semantic feature matrix to obtain a spatial feature matrix; and finally, carrying out vector fusion on the spatial feature matrix and the semantic feature matrix to obtain the final semantic representation of the scientific and technological paper. The method enriches the semantic representation of the scientific paper by using the spatial correlation of the knowledge map on the basis of extracting the semantic features of the linguistic data of the scientific paper, thereby better extracting the semantic features of the scientific paper.
Correspondingly, the invention also provides a system for extracting semantic features of a scientific paper data text, which comprises a processor and a memory, wherein the memory stores computer instructions, the processor is used for executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system realizes the steps of the method according to any one of the above embodiments. Fig. 3 is a schematic diagram of an architecture of a text semantic feature extraction system of scientific and technological paper data according to another embodiment of the present invention, and as shown in fig. 3, the system first extracts semantic features of the scientific and technological paper data based on a BERT model, further extracts spatial features of the scientific and technological paper based on a GCN network model, and further performs feature fusion on the semantic features and the spatial features to finally obtain final semantic features of the scientific and technological paper.
Through the embodiment, when the feature extraction is carried out on the scientific and technical paper data, the association between the scientific and technical papers is combined, the co-occurrence relation of the keywords is utilized to construct the heterogeneous network of the paper documents and the keywords, and the vector representation of the paper is obtained from the triple of the heterogeneous network of the paper and the keywords by adopting the Graph Convolution Network (GCN) on the basis of the original semantic features extracted by the BERT. That is, in the BERT-GCN model proposed by the present invention, the BERT model is used to initialize the representations of the document nodes in the text graph, which are used as inputs to the GCN, and then the document representations are iteratively updated based on the graph structure using the GCN, and the outputs thereof are regarded as final representations of the document nodes, so that they can utilize the advantages of the pre-trained model and the graph model.
In addition, the knowledge graph is constructed by linguistic data with extremely high requirements on professional knowledge in the fields of scientific and technological papers to train, so that the feature representation capability of the model in the professional fields is improved. Firstly, constructing a document and keyword heteromorphic graph for a corpus, wherein nodes are words or documents, node vectors are initialized by using a pre-trained BERT model, and features are aggregated by using a Graph Convolution Network (GCN). By jointly training the BERT module and the GCN module, the method can utilize the advantages of the BERT module and the GCN module, utilize large-scale pre-training of a large amount of original scientific and technological thesis data, and enrich semantic representation of the thesis by utilizing spatial correlation of a knowledge map on the basis of extracting semantic features of the scientific and technological thesis corpus.
In addition, the invention also discloses a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the method according to any of the above embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein may be implemented as hardware, software, or combinations of both. Whether this is done in hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this patent describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments in the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for extracting semantic features of a scientific paper data text is characterized by comprising the following steps:
acquiring text information of a scientific and technological paper, and constructing an entity relationship graph based on the acquired text information of the scientific and technological paper, wherein the text information comprises a paper title and a keyword, a node in the entity relationship graph is the paper title or the keyword, and an edge in the entity relationship graph is an incidence relation between the nodes;
extracting semantic features based on the acquired text information of the scientific and technical paper to obtain a semantic feature matrix;
determining an original adjacency matrix based on the entity relationship graph, and inputting the semantic feature matrix and the original adjacency matrix into a graph network model to obtain a spatial feature matrix;
and performing feature fusion on the semantic feature matrix and the spatial feature matrix to obtain the final semantic features of the scientific and technological paper.
2. The method as claimed in claim 1, wherein extracting semantic features based on the obtained text information of the scientific paper to obtain a semantic feature matrix comprises:
and inputting the acquired text information of the scientific and technological paper to a BERT model to obtain a semantic feature matrix.
3. The method for extracting semantic features of the text of the scientific paper data according to claim 1, wherein constructing an entity relationship diagram based on the acquired text information of the scientific paper comprises:
and calculating the correlation between any two nodes through a point-to-point mutual information algorithm based on the acquired text information of the scientific and technological paper, and constructing an entity relation graph based on the calculated correlation.
4. The method of claim 3, wherein the computation formula of the point-by-point mutual information algorithm is as follows:
wherein the content of the first and second substances,W i representing nodesi,W j Representing nodesj,,,,MThe length of the text abstract of the scientific paper is shown,C(W i, W j )representing nodesiAnd nodejNumber of co-occurrences in the text abstract of the same scientific paper, C: (W i ) Representing nodesiNumber of occurrences in the summary, C: (W j ) Representing nodesjNumber of occurrences in the summary.
5. The method of claim 1, wherein the graph network model comprises a plurality of convolutional layers, and an output of each convolutional layer is:
wherein the content of the first and second substances,L (i) is as followsiThe output of the layer(s) is,L (i-1) is as followsi-1The output of the layer(s) is,ρin order to activate the function(s),W (i) in order to be the parameters of the model,is the laplace transform of the adjacency matrix,,Dis a degree matrix for normalization.
6. The method of claim 1, wherein the method further comprises:
calculating the cosine similarity of any two nodes in the entity relationship graph based on the obtained final semantic features;
obtaining a reconstructed adjacency matrix based on the cosine similarity;
and calculating loss values of the original adjacency matrix and the reconstructed adjacency matrix, and optimizing parameters of the graph network model based on the loss values.
8. The method for extracting semantic features of scientific paper data texts according to any one of claims 1 to 7, characterized in that the semantic features are extracted by formulasZ=λZ GCN +(1-λ) Z BERT Performing feature fusion on the semantic feature matrix and the spatial feature matrix; wherein the content of the first and second substances,Z GCN in the form of a matrix of spatial features,Z BERT in the form of a matrix of semantic features,λis hyper-parametric, andλ∈(0,1)。
9. a scientific paper data text semantic feature extraction system comprising a processor and a memory, wherein the memory stores computer instructions, the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system realizes the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210745539.4A CN114818737B (en) | 2022-06-29 | 2022-06-29 | Method, system and storage medium for extracting semantic features of scientific and technological paper data text |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210745539.4A CN114818737B (en) | 2022-06-29 | 2022-06-29 | Method, system and storage medium for extracting semantic features of scientific and technological paper data text |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114818737A true CN114818737A (en) | 2022-07-29 |
CN114818737B CN114818737B (en) | 2022-11-18 |
Family
ID=82523321
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210745539.4A Active CN114818737B (en) | 2022-06-29 | 2022-06-29 | Method, system and storage medium for extracting semantic features of scientific and technological paper data text |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114818737B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115858725A (en) * | 2022-11-22 | 2023-03-28 | 广西壮族自治区通信产业服务有限公司技术服务分公司 | Method and system for screening text noise based on unsupervised graph neural network |
CN117831698A (en) * | 2024-03-04 | 2024-04-05 | 吉林大学 | Intelligent quality control system and method for nursing medical records |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110688474A (en) * | 2019-09-03 | 2020-01-14 | 西北工业大学 | Embedded representation obtaining and citation recommending method based on deep learning and link prediction |
CN110705260A (en) * | 2019-09-24 | 2020-01-17 | 北京工商大学 | Text vector generation method based on unsupervised graph neural network structure |
CN113254616A (en) * | 2021-06-07 | 2021-08-13 | 佰聆数据股份有限公司 | Intelligent question-answering system-oriented sentence vector generation method and system |
US20210271822A1 (en) * | 2020-02-28 | 2021-09-02 | Vingroup Joint Stock Company | Encoder, system and method for metaphor detection in natural language processing |
CN113378547A (en) * | 2021-06-16 | 2021-09-10 | 武汉大学 | GCN-based Chinese compound sentence implicit relation analysis method and device |
CN113704415A (en) * | 2021-09-09 | 2021-11-26 | 北京邮电大学 | Vector representation generation method and device for medical text |
CN114048350A (en) * | 2021-11-08 | 2022-02-15 | 湖南大学 | Text-video retrieval method based on fine-grained cross-modal alignment model |
US20220058464A1 (en) * | 2020-08-20 | 2022-02-24 | Fujifilm Business Innovation Corp. | Information processing apparatus and non-transitory computer readable medium |
-
2022
- 2022-06-29 CN CN202210745539.4A patent/CN114818737B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110688474A (en) * | 2019-09-03 | 2020-01-14 | 西北工业大学 | Embedded representation obtaining and citation recommending method based on deep learning and link prediction |
CN110705260A (en) * | 2019-09-24 | 2020-01-17 | 北京工商大学 | Text vector generation method based on unsupervised graph neural network structure |
US20210271822A1 (en) * | 2020-02-28 | 2021-09-02 | Vingroup Joint Stock Company | Encoder, system and method for metaphor detection in natural language processing |
US20220058464A1 (en) * | 2020-08-20 | 2022-02-24 | Fujifilm Business Innovation Corp. | Information processing apparatus and non-transitory computer readable medium |
CN113254616A (en) * | 2021-06-07 | 2021-08-13 | 佰聆数据股份有限公司 | Intelligent question-answering system-oriented sentence vector generation method and system |
CN113378547A (en) * | 2021-06-16 | 2021-09-10 | 武汉大学 | GCN-based Chinese compound sentence implicit relation analysis method and device |
CN113704415A (en) * | 2021-09-09 | 2021-11-26 | 北京邮电大学 | Vector representation generation method and device for medical text |
CN114048350A (en) * | 2021-11-08 | 2022-02-15 | 湖南大学 | Text-video retrieval method based on fine-grained cross-modal alignment model |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115858725A (en) * | 2022-11-22 | 2023-03-28 | 广西壮族自治区通信产业服务有限公司技术服务分公司 | Method and system for screening text noise based on unsupervised graph neural network |
CN117831698A (en) * | 2024-03-04 | 2024-04-05 | 吉林大学 | Intelligent quality control system and method for nursing medical records |
CN117831698B (en) * | 2024-03-04 | 2024-05-24 | 吉林大学 | Intelligent quality control system and method for nursing medical records |
Also Published As
Publication number | Publication date |
---|---|
CN114818737B (en) | 2022-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109840287B (en) | Cross-modal information retrieval method and device based on neural network | |
CN108733792B (en) | Entity relation extraction method | |
CN110717017B (en) | Method for processing corpus | |
WO2021179640A1 (en) | Graph model-based short video recommendation method, intelligent terminal and storage medium | |
CN112380435B (en) | Document recommendation method and system based on heterogeneous graph neural network | |
CN106484674B (en) | Chinese electronic medical record concept extraction method based on deep learning | |
CN114818737B (en) | Method, system and storage medium for extracting semantic features of scientific and technological paper data text | |
JP5128629B2 (en) | Part-of-speech tagging system, part-of-speech tagging model training apparatus and method | |
CN111190997B (en) | Question-answering system implementation method using neural network and machine learning ordering algorithm | |
WO2023039901A1 (en) | Text recommendation method and apparatus, model training method and apparatus, and readable storage medium | |
CN111985520A (en) | Multi-mode classification method based on graph convolution neural network | |
CN112597296A (en) | Abstract generation method based on plan mechanism and knowledge graph guidance | |
US20230281400A1 (en) | Systems and Methods for Pretraining Image Processing Models | |
CN116628171B (en) | Medical record retrieval method and system based on pre-training language model | |
Mittal et al. | Deep graph-long short-term memory: a deep learning based approach for text classification | |
CN114896377A (en) | Knowledge graph-based answer acquisition method | |
CN111914061B (en) | Radius-based uncertainty sampling method and system for text classification active learning | |
Han et al. | Augmented sentiment representation by learning context information | |
CN114841140A (en) | Dependency analysis model and Chinese combined event extraction method based on dependency analysis | |
CN114281982B (en) | Book propaganda abstract generation method and system adopting multi-mode fusion technology | |
CN111930981A (en) | Data processing method for sketch retrieval | |
Ma et al. | E2timt: Efficient and effective modal adapter for text image machine translation | |
CN116414988A (en) | Graph convolution aspect emotion classification method and system based on dependency relation enhancement | |
CN115630223A (en) | Service recommendation method and system based on multi-model fusion | |
CN115631504A (en) | Emotion identification method based on bimodal graph network information bottleneck |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |