CN112052318A

CN112052318A - Semantic recognition method and device, computer equipment and storage medium

Info

Publication number: CN112052318A
Application number: CN202010840252.0A
Authority: CN
Inventors: 施晓明; 陈曦; 张子恒; 郑冶枫; 车万翔; 刘挺
Original assignee: Harbin Institute of Technology; Tencent Technology Shenzhen Co Ltd
Current assignee: Harbin Institute of Technology; Tencent Technology Shenzhen Co Ltd
Priority date: 2020-08-18
Filing date: 2020-08-18
Publication date: 2020-12-08

Abstract

The application discloses a semantic recognition method, a semantic recognition device, computer equipment and a storage medium, which can realize semantic recognition based on an artificial intelligence technology, extract text characteristic information of a first user input information sample through a semantic recognition model so as to recognize predicted semantic information of the sample, and adjust parameters of the semantic recognition model based on a label and the predicted semantic information of the sample; extracting text characteristic information of a second user input information sample through the adjusted semantic identification model so as to identify the predicted semantic information of the sample; adjusting parameters of a semantic recognition model based on the label of the sample and the predicted semantic information to obtain a trained semantic recognition model; the semantic information is obtained by performing semantic recognition on the target user input information based on the semantic recognition model, wherein the first user input information sample does not need manual labeling, dependence on manual labeling can be reduced, and the second user input information sample is adopted to train the model, so that the semantic recognition accuracy can be improved.

Description

Semantic recognition method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of natural language processing technologies, and in particular, to a semantic recognition method, apparatus, and storage medium.

Background

Generally, in order to achieve a good recognition effect, a model used by a dialog system needs a large amount of fine labeling data, for different application scenes, the fine labeling data may need different professionals to label, and a large amount of time is consumed.

Disclosure of Invention

The embodiment of the invention provides a semantic recognition method, a semantic recognition device and a storage medium, which can reduce the dependence of a semantic recognition model on manual labeling to a certain extent.

The embodiment of the invention provides a semantic recognition method, which comprises the following steps:

extracting text characteristic information from a first user input information sample through a semantic identification model, and identifying predicted semantic information of the first user input information sample based on the text characteristic information of the first user input information sample, wherein a label of the first user input information sample comprises semantic information obtained by performing semantic identification on the first user input information sample and/or information obtained from reply information of the first user input information sample;

performing parameter adjustment on the semantic recognition model based on the labels and predicted semantic information of the first user input information sample;

extracting text characteristic information from a second user input information sample through the adjusted semantic identification model, and identifying predicted semantic information of the second user input information sample based on the text characteristic information of the second user input information sample, wherein a label of the second user input information sample comprises the semantic information of the second user input information sample;

performing parameter adjustment on the semantic recognition model based on the label of the second user input information sample and the predicted semantic information to obtain a trained semantic recognition model;

and performing semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user.

The embodiment of the invention also provides a semantic recognition device, which comprises:

the first semantic prediction unit is used for extracting text characteristic information from a first user input information sample through a semantic recognition model, and recognizing the predicted semantic information of the first user input information sample based on the text characteristic information of the first user input information sample, wherein the label of the first user input information sample comprises semantic information obtained by performing semantic recognition on the first user input information sample and/or information obtained from reply information of the first user input information sample;

a first parameter adjusting unit, configured to perform parameter adjustment on the semantic recognition model based on the label and predicted semantic information of the first user input information sample;

the second semantic prediction unit is used for extracting text characteristic information from a second user input information sample through the adjusted semantic identification model, and identifying the predicted semantic information of the second user input information sample based on the text characteristic information of the second user input information sample, wherein the label of the second user input information sample comprises the semantic information of the second user input information sample;

the second parameter adjusting unit is used for adjusting parameters of the semantic recognition model based on the labels and the predicted semantic information of the second user input information samples to obtain a trained semantic recognition model;

and the semantic recognition unit is used for carrying out semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user.

In an alternative example, the semantic recognition unit is configured to:

performing text feature extraction on target user input information through a trained semantic recognition model to obtain text feature information of the target user input information;

identifying semantic information of the target user input information based on text characteristic information of the target user input information through the semantic identification model

In an alternative example, the semantic recognition unit is configured to:

mapping target user input information into a feature space through a trained semantic recognition model to obtain text feature information of the target user input information in the feature space;

comparing the text characteristic information of the target user input information with the characteristic distance between the preset text characteristic information of at least one semantic information in the characteristic space through the semantic recognition model;

and identifying semantic information of the target user input information based on the characteristic distance.

In an optional example, when the tag of the first user input information sample includes semantic information obtained by performing semantic recognition on the first user input information sample, the apparatus further includes: a sample acquisition unit for:

before text feature extraction is carried out on a first user input information sample through a semantic recognition model to obtain text feature information of the first user input information sample,

obtaining a first user input information sample without labels;

performing semantic recognition on the first user input information sample to obtain first semantic information of the first user input information sample;

generating a pseudo label for the first sample of user input information based on the first semantic information.

In an optional example, when the tag of the first user input information sample includes information obtained from reply information of the first user input information sample, the sample obtaining unit is further configured to:

before text feature extraction is carried out on the first user input information sample through a semantic recognition model to obtain text feature information of the first user input information sample, acquiring reply information of the first user input information sample in dialog information where the first user input information sample is located;

acquiring second semantic information of the first user input information sample from the reply information;

generating a weak label for the first user input information sample based on the second semantic information.

In an optional example, the sample acquiring unit is configured to:

training the semantic recognition model through a labeled second user input information sample;

performing semantic recognition on the first user input information sample by using the trained semantic recognition model to obtain predicted semantic information of the first user input information sample;

and using the obtained predicted semantic information as first semantic information of the first user input information sample.

In an optional example, the sample acquiring unit is further configured to:

before text feature extraction is carried out on the first user input information sample through the semantic recognition model to obtain text feature information of the first user input information sample, acquiring reply information of the first user input information sample in dialog information where the first user input information sample is located;

generating a pseudo label for the first user input information sample based on the second semantic information;

and merging the pseudo label and the weak label of the first user input information sample to obtain a merged label of the second semantic information.

In an optional example, the sample acquiring unit is configured to:

matching the reply information with a preset semantic information set;

and determining second semantic information of the first user input information sample based on the matched semantic information in the semantic information set.

In an optional example, the first parameter adjusting unit is configured to:

calculating the loss of the semantic recognition model based on the predicted semantic information of the first user input information sample and the semantic information in the merged tag of the first user input information sample;

adjusting parameters of the semantic recognition model based on the loss.

In an optional example, the semantic recognition apparatus of this embodiment further includes a replying unit, configured to:

after semantic recognition is carried out on input information of a target user based on a trained semantic recognition model to obtain semantic information of the input information of the target user, intelligent reply information of the input information of the target user is generated based on the semantic information and a current conversation scene;

and outputting the intelligent reply information.

In some embodiments of the present invention, there may also be provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method as described above when executing the computer program.

In some embodiments of the invention, there may also be provided a storage medium having stored thereon a computer program which, when run on a computer, causes the computer to perform the steps of the method as described above.

By adopting the embodiment of the application, the text characteristic information of the first user input information sample can be extracted through the semantic recognition model, the predicted semantic information of the first user input information sample is recognized based on the text characteristic information, and the semantic recognition model is subjected to parameter adjustment based on the label and the predicted semantic information of the first user input information sample; extracting text characteristic information of a second user input information sample through the adjusted semantic identification model, and identifying predicted semantic information of the second user input information sample based on the text characteristic information; performing parameter adjustment on the semantic recognition model based on the label of the second user input information sample and the predicted semantic information to obtain a trained semantic recognition model; the method comprises the steps of carrying out semantic recognition on target user input information based on a trained semantic recognition model to obtain semantic information of the target user input information, wherein labels of a first user input information sample of the embodiment comprise the semantic information obtained through the semantic recognition and/or information obtained from reply information of the first user input information sample, so that the labels of the first user input information sample do not need manual labeling, dependence of training of the semantic recognition model on the manual labeling can be reduced, time required by model training is shortened, after the semantic recognition model is trained based on the first user input information sample, the model is further finely tuned by adopting a finely labeled second user input information sample, and the semantic recognition accuracy of the model can be effectively improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1a is a schematic structural diagram of a semantic recognition system provided by an embodiment of the present invention;

FIG. 1b is a flow chart of a semantic recognition method provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a two-step training method for providing a semantic recognition model according to an embodiment of the present invention;

FIG. 3 is a flowchart of a semantic recognition method in a medical scenario according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a semantic recognition apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides a semantic recognition method, a semantic recognition device, computer equipment and a storage medium.

The embodiment of the invention provides a semantic recognition system which comprises a semantic recognition device suitable for computer equipment. The computer device may be a terminal or a server.

The terminal can be a mobile phone, a tablet computer, a notebook computer and other terminal equipment, and also can be wearable equipment, an intelligent television or other intelligent terminals with display modules.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, but is not limited thereto.

The semantic recognition device of this embodiment may be integrated in a terminal or a server, and optionally, may be integrated in the terminal or the server in the form of an application program or the like.

Referring to fig. 1a, the semantic recognition system provided in the present embodiment includes a terminal 10, a server 20, and the like.

The terminal 10 may be configured to obtain input information of a target user and transmit the input information of the target user to the server 20.

The server 20 may be configured to extract text feature information from a first user input information sample through a semantic recognition model, and recognize predicted semantic information of the first user input information sample based on the text feature information of the first user input information sample, where a tag of the first user input information sample includes semantic information obtained by performing semantic recognition on the first user input information sample and/or information obtained from reply information of the first user input information sample; performing parameter adjustment on the semantic recognition model based on the labels and predicted semantic information of the first user input information sample; extracting text characteristic information from a second user input information sample through the adjusted semantic identification model, and identifying predicted semantic information of the second user input information sample based on the text characteristic information of the second user input information sample, wherein a label of the second user input information sample comprises the semantic information of the second user input information sample; performing parameter adjustment on the semantic recognition model based on the label of the second user input information sample and the predicted semantic information to obtain a trained semantic recognition model; and performing semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user.

The following are detailed below. It should be noted that the following description of the embodiments is not intended to limit the preferred order of the embodiments.

The embodiments of the present invention will be described from the perspective of a semantic recognition device, which may be specifically integrated in a terminal and/or a server, for example, may be integrated in the terminal or the server in the form of an application program.

The semantic recognition method provided by the embodiment of the invention can be executed by a processor of a terminal or a server, the semantic recognition is realized based on a semantic recognition model in the embodiment, the semantic recognition method is an application based on a Natural Language Processing (NLP) technology, and the NLP technology is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.

As shown in fig. 1b, the flow of the semantic recognition method may be as follows:

101. extracting text characteristic information from the first user input information sample through a semantic identification model, and identifying predicted semantic information of the first user input information sample based on the text characteristic information of the first user input information sample, wherein a label of the first user input information sample comprises semantic information obtained by performing semantic identification on the first user input information sample and/or information obtained from reply information of the first user input information sample;

in the Semantic recognition of the embodiment, the Semantic recognition model uses Semantic understanding (Semantic interpretation) technology, especially Semantic analysis (Semantic analysis) technology therein, and robot question-answering (similar to siri/microsoft ice class) technology, especially question understanding query interpretation technology therein. Training of the semantic recognition model is realized based on an AI (Artificial intelligence) technology, particularly based on a Machine Learning (ML) technology in the Artificial intelligence technology, and more particularly, may be realized by Deep Learning (Deep Learning) in the Machine Learning.

The semantic recognition model in this embodiment may be constructed based on any structure of an artificial neural network (artificial neural network) that can be used for semantic recognition, for example, the semantic recognition model may be implemented based on BERT (Bidirectional Encoder from converters), which is a pre-training language model and can help a downstream natural language processing task to improve performance by embedding words or sentences. The semantic recognition model may be implemented based on a Recurrent Neural Network (RNN), which is a Recurrent Neural Network in which sequence data is used as an input, recursion is performed in the evolution direction of the sequence, and all nodes (cyclic units) are connected in a chain manner to form a closed loop. The semantic recognition model can be implemented based on CNN (Convolutional Neural Networks), which is a type of feed-forward Neural network that contains convolution or correlation calculations and has a deep structure. The semantic recognition model may be implemented based on RCNN (Rich hierarchical Neural Network), and the like, which is not limited by the embodiment.

In this embodiment, the field of semantic recognition model recognition is not limited, and may be any divided field, for example, the semantic recognition model in this embodiment may be a model of a medical dialogue recognition field, or the semantic recognition model in this embodiment may be a model of a music field, or may also be a model of a chemical field, a model of a physical field, a model of a game field, a model of an intelligent robot field, and so on.

In this embodiment, the first user input information sample and the second user input information sample include user input information, and both samples are labeled, where the labeled information is semantic information of the user input information in the samples.

The user input information in the first and second samples of user input information in this embodiment may be user input information from dialog information. The label of the second user input information sample is finely labeled manually, and in the manual labeling, the semantic information of the user input information is determined manually, the semantic information can be understood as the conversation purpose of the user, the intention which the user wants to express, and the like, for example, the medical field is taken as an example, in the field, the conversation information is medical conversation information, the user input information is generally inquiry information input by a patient, the inquiry information can be illness state description information to a great extent, the description information generally has the characteristic of spoken language, for example, the user input information is 'doctor, my belly is very painful', and in the manual labeling, the label of the user input information is 'belly' according to the intention which the user input information wants to express. However, it is conceivable that, if all samples of the model need to be labeled manually, the data labeling difficulty is high, and time and labor are wasted.

In view of reducing the cost of annotation, the data source of the sample is improved in this embodiment, so that the annotation of the first user input information sample is not manually completed, but is obtained based on semantic recognition and from the reply information of the user input information.

The semantic recognition model in this embodiment is essentially a classification model, and the semantic recognition model may include a feature extraction layer and a classification layer, where n semantic classifications are defined in the classification layer, and a semantic recognition result of the semantic recognition model is a classification probability of user input information in each semantic classification.

102. Performing parameter adjustment on the semantic recognition model based on the label of the first user input information sample and the predicted semantic information;

103. extracting text characteristic information from the second user input information sample through the adjusted semantic identification model, and identifying predicted semantic information of the second user input information sample based on the text characteristic information of the second user input information sample, wherein the label of the second user input information sample comprises the semantic information of the second user input information sample;

104. performing parameter adjustment on the semantic recognition model based on the label of the second user input information sample and the predicted semantic information to obtain a trained semantic recognition model;

in this embodiment, the steps 101-102 can be regarded as a pre-training process for the semantic recognition model, in which the parameters of the model are coarsely adjusted, and the steps 103-104 can be regarded as a fine-tuning process for the semantic recognition model, in which the parameters of the semantic recognition model are finely tuned.

Optionally, this embodiment further describes, with reference to fig. 2, a detailed example of a training process of the semantic recognition model, where the two steps of pre-training and model fine-tuning of the semantic recognition model in this embodiment correspond to step 201 and step 202 in fig. 2, respectively.

In the step of "pre-training" of the model, that is, in step 101, a specific process of obtaining the predicted semantic information of the first user input information sample through the semantic recognition model may include: mapping the first user input information sample to a feature space through (a feature extraction layer of) a semantic recognition model to obtain text feature information of the first user input information sample in the feature space; comparing the text characteristic information of the first user input information sample with the characteristic distance between the preset text characteristic information of at least one semantic information in the characteristic space through (the classification layer of) the semantic recognition model; based on the feature distance, predictive semantic information for the first user input information sample is identified.

In this embodiment, the output of the model includes, in addition to the predicted semantic information, the prediction probability of each predicted semantic information, wherein before the model outputs the predicted semantic information, the model may determine the prediction probability of each preset semantic information set by the classification layer of the model based on the characteristic distance, discard the semantic information with the prediction probability lower than the lowest prediction probability threshold, and output only the classification result of the preset semantic information with the prediction probability higher than the lowest prediction probability threshold.

Wherein the model may determine a prediction probability of each preset semantic information based on the feature distance.

In the step 103 of "fine tuning" of the model, similarly, the specific process of obtaining the predicted semantic information of the second user input information sample through the semantic recognition model may include: mapping the second user input information sample to a feature space through (a feature extraction layer of) a semantic recognition model to obtain text feature information of the second user input information sample in the feature space; comparing the text characteristic information of the second user input information sample with the characteristic distance between the preset text characteristic information of at least one semantic information in the characteristic space through (the classification layer of) the semantic recognition model; based on the feature distance, predictive semantic information for the second user input information sample is identified.

It can be understood that, in this embodiment, the detailed steps of obtaining the semantic recognition result of the user input information are different if the semantic recognition models have different structures, and are not limited to the above-mentioned refinement steps.

Optionally, in this embodiment, the tag information of the first user input information sample includes three cases:

the first label information of the first user input information sample comprises: and performing semantic recognition on the first user input information sample to obtain first semantic information.

Second, the label information of the first user input information sample includes: and second semantic information obtained from reply information of the first user input information sample.

The third type of label information of the first user input information sample comprises: the method comprises the steps of carrying out semantic recognition on a first user input information sample to obtain first semantic information and obtaining second semantic information from reply information of the first user input information sample.

For the first case, the labels, which are obtained based on semantic recognition, may be considered pseudo-labels of the first user input information sample. Before extracting text feature information from the first user input information sample through the semantic recognition model, the obtaining of the pseudo tag may include: obtaining a first user input information sample without labels; performing semantic recognition on the first user input information sample to obtain first semantic information of the first user input information sample; pseudo-labels for the first sample of user input information are generated based on the first semantic information.

In the case that the label of the first user input information sample only includes a pseudo label, the step "performing parameter adjustment on the semantic recognition model based on the label of the first user input information sample and the predicted semantic information" may include:

based on the predicted semantic information of the first user input information sample and the semantic information in the pseudo-label of the first user input information sample, a loss (of the pre-training phase) of the semantic recognition model is calculated, and parameters of the semantic recognition model are adjusted based on the loss.

The loss can be calculated using any available classification loss function, which is not limited in this embodiment.

For the tags in the second case, which are obtained from the reply information of the first user input information sample, and may be considered as weak tags of the first user input information sample, before extracting the text feature information of the first user input information sample through the semantic recognition model, the specific obtaining process of the weak tags may include: acquiring reply information of the first user input information sample in the dialog information of the first user input information sample; acquiring second semantic information of the first user input information sample from the reply information; weak labels of the first user input information samples are generated based on the second semantic information.

In the case that the labels of the first user-input information sample only include weak labels, the step of "performing parameter adjustment on the semantic recognition model based on the labels of the first user-input information sample and the predicted semantic information" may include:

based on the predicted semantic information of the first user input information sample and the semantic information in the weak label of the first user input information sample, a loss (of the pre-training phase) of the semantic recognition model is calculated, and parameters of the semantic recognition model are adjusted based on the loss. The loss may also be calculated by using any available classification loss function, which is not limited in this embodiment.

For the tag in the third case, it may be understood as being composed of a pseudo tag and a weak tag, and the acquisition process includes an acquisition process of the pseudo tag and the weak tag, where the acquisition order of the pseudo tag and the weak tag is not limited.

It will be appreciated that in the third case, the loss of the semantic recognition model needs to be calculated based on weak and pseudo-labels.

Referring to fig. 2, in this embodiment, in a model training phase, a weak supervision enforcement module is provided for a model, and the module may be connected to a semantic recognition model or may be provided relatively independently of the semantic recognition model, and specifically functions to provide the first user input information sample for the semantic recognition model, and in an actual use phase of the semantic recognition model, the module stops working.

Referring to fig. 2, a weak supervision strengthening module provides a teacher model, which has a semantic recognition function, and can recognize the user's conversation intention from the user input information, so as to obtain semantic information. The classification categories set in the teacher model comprise all classification categories set in the semantic recognition model.

As can be seen from the above description, in an example, the first user input information sample may have only a weak label, which may play a weak supervision role with respect to the semantic recognition model, so the first user input information sample having the weak label (refer to icon 5 in fig. 2) is a weak supervision data (refer to fig. 2), which may be represented by D, and the second user input information sample labeled with a high precision may be represented by D.

Intuitively, the semantic recognition model trained on the fine labeled data (icon 2 in FIG. 2), i.e., the second user input information sample, has a higher F1 score than the semantic recognition model trained on the weakly supervised data. Therefore, the fine labeled data can provide more accurate label information for weakly supervised learning. From the above analysis, the parameters learned from the fine labeled data can be considered a priori estimates of the parameters of the weakly supervised data. The scheme for this a priori estimation is formalized as follows:

wherein w represents a parameter of the semantic recognition model, D represents weakly supervised data, D represents fine labeled data, w represents an optimal parameter from the pre-training stage, P (D | w) represents a semantic recognition accuracy of the weakly supervised data D when the parameter of the semantic recognition model is w, and P (D | w) represents a semantic recognition accuracy of the fine labeled data D when the parameter of the semantic recognition model is w.

However, the amount of the weakly supervised data D is much larger than the amount of the well annotated fine labeled data D. Therefore, D will overrule the entire training process of the semantic recognition model, resulting in that the model can learn only a small amount of knowledge from D. Furthermore, D may differ from D in label distribution and expression, which may also be detrimental to the training of the model.

In order to alleviate the above problem, the present embodiment proposes a scheme for adding a pseudo tag to make tag information of weak supervised data more accurate (i.e. a scheme in which a source of the tag information includes a weak tag and a pseudo tag), in order to solve a problem that accuracy of a tag in the weak supervised data is not sufficient.

In this scheme, the pseudo tag may be added before the weak tag, or may be added after the weak tag, which is not limited in this embodiment.

Taking the example that the pseudo tag is added after the weak tag is added, after the step "generating the weak tag of the first user input information sample based on the second semantic information", the method may further include:

generating a pseudo label of the first user input information sample based on the first semantic information;

The method includes merging pseudo labels and weak labels of a first user input information sample, specifically merging first semantic information and second semantic information, and performing deduplication processing on merged semantic information if the same semantic information exists in the two semantic information during merging, so that no repeated semantic information exists in the merged labels.

In this embodiment, the pseudo label of the first user input information sample may be obtained by performing semantic recognition on the first user input information sample by a teacher model in the weak supervision enforcement module.

Optionally, in an example, the teacher model may be obtained by training based on the fine-label data, that is, by training based on the second user input information sample of this embodiment, in an example, the step "performing semantic recognition on the first user input information sample to obtain the first semantic information of the first user input information sample" may include:

performing semantic recognition training on the teacher model (labeled 3 in the figure 2) through the labeled second user input information sample (labeled 2 in the figure 2);

performing semantic recognition on the first user input information sample by adopting the trained teacher model to obtain predicted semantic information of the first user input information sample;

and taking the obtained predicted semantic information as first semantic information of a first user input information sample.

Referring to fig. 2, the learned first semantic information may be used to generate a pseudo label (labeled as 4 in fig. 2) of a first user input information sample, and in an example, the teacher model may be a model completely identical to the semantic recognition model in this embodiment, or the semantic recognition model itself in this embodiment, that is, in this embodiment, the semantic recognition model is trained by using the fine labeling data, and then the trained semantic recognition model is used to perform semantic recognition on the first user input information sample (i.e., weak supervision data) with a weak label, so as to obtain first semantic information, and generate the pseudo label based on the first semantic information.

In the example that the pseudo tag is added before the weak tag, the generation mode of the pseudo tag is similar, and after the pseudo tag is generated by obtaining the first semantic information, the reply information of the first user input information sample in the dialog information where the first user input information sample is located can be obtained; acquiring second semantic information of the first user input information sample from the reply information; generating a pseudo label of the first user input information sample based on the second semantic information; and merging the pseudo label and the weak label of the first user input information sample to obtain a merged label of the second semantic information.

In this embodiment, in order to extract the more accurate second semantic information from the reply information, a corresponding semantic information set may be set for the field of the model application, where the semantic information set may include semantic information that is more commonly used in the field, and for example, in the medical field, a semantic information set including medical professional vocabularies in the medical field may be set, and the set may include medical professional vocabularies such as abdominal pain, abdominal distension, diarrhea, and the like.

Optionally, the step of obtaining the second semantic information of the first user input information sample from the reply information may include: matching the reply information with a preset semantic information set; and determining second semantic information of the first user input information sample based on the matched semantic information in the semantic information set.

In an example, a preset semantic information set may be set according to a category set by a classification layer of a semantic recognition model, and assuming that the classification layer of the semantic recognition model is provided with n categories k1-kn, semantic association expansion may be performed on the names of the categories ki to obtain semantic associated words corresponding to the categories, and each category name and the corresponding semantic associated words (without limitation in number) are used as semantic information and are correspondingly stored in the semantic information set.

For example, abdominal pain, the semantic association "abdominal pain" may be set, and so on.

When the reply information is matched with a preset semantic information set, the reply information can be segmented, for the words obtained by segmentation, each word is respectively matched with each semantic information in the semantic set, if the matching is successful, whether the successfully matched words are class names or semantic associated words is determined, if the successfully matched words are class names, the successfully matched semantic information is directly adopted as second semantic information, if the successfully matched words are semantic associated words, the class names corresponding to the semantic associated words are obtained as second semantic information, and a weak label is generated based on the second semantic information.

It can be understood that, for the recognition accuracy of the semantic recognition model, a plurality of small sub-fields can be divided under a large field, each sub-field can train a corresponding semantic recognition model, or the medical field can be divided into more subdivided sub-fields such as surgery, internal medicine, digestive department, five sense organs department and the like, and each sub-field sets semantic information sets in a targeted manner, which is beneficial to improving the model accuracy.

The pseudo tag generation scheme in this embodiment is actually a tag reconstruction of the weakly supervised data, that is, a model (icon 3 in fig. 2) learned on the fine labeled data D is used to label D, and the labeled data after "reconstruction" can be obtained, which is called pseudo data D' (icon 4 in fig. 2). The pseudo label in D' contains accurate label information in a certain D. The learning objective in the above formula may be modified as:

wherein, P (D '| w) represents the semantic recognition accuracy of the pseudo-labeled data D' when the parameter of the semantic recognition model is w.

Accordingly, for the same sentence s in D and D', the classification layer in the semantic recognition model needs to learn from two kinds of labels: i.e. from weak labels from weak supervision data D and from dummy data D' containing accurate label information.

In the embodiment of the present application, the pseudo tag and the weak tag are fused to obtain an enhanced weak tag (icon 6 in fig. 2). The strengthened weak label contains accurate label information and weak label information, and the quantity of information is larger for training of a semantic recognition model.

The present application designs a label union algorithm to pre-train the model on D and D', with the goal called enhanced weakly supervised data (icon 6 in fig. 2). The specific method of the algorithm is that for the same sentence, the weak label and the pseudo label are combined to be used as the label of the enhanced weak supervision data.

Referring to fig. 2, it can be seen that the semantic recognition model of this embodiment is implemented by using a two-step training method, and aims to use data without artificial labeling, so that the depth model reduces dependence on artificially labeled data. The semantic recognition model input data are the fine-labeled data (icon 2 in fig. 2) and the strengthened weak supervision data (icon 6 in fig. 2). The semantic recognition model training is mainly carried out in two steps, namely a pre-training process (model pre-training) and a fine-tuning process (model fine-tuning).

The pre-training process and the fine tuning process have been described in detail in the previous examples, and the adjustment of the parameters of the model in the pre-training and fine tuning processes is explained herein.

In the third case that the label of the first user input information sample is the above-mentioned label, the pre-training process of the model is performed by using the first user input information sample with a pseudo label and a weak label, wherein the step "performing parameter adjustment on the semantic recognition model based on the label and the predicted semantic information of the first user input information sample" may include:

calculating the loss of the semantic recognition model based on the predicted semantic information of the first user input information sample and the semantic information in the merged label of the first user input information sample;

parameters of the semantic recognition model are adjusted based on the loss.

In the pre-training stage, the loss of the semantic recognition model is a classification loss, and the loss function for calculating the loss can be any available loss function, which is not limited by the embodiment.

In one example, the model parameter adjustment in the pre-training phase is described by taking a loss function as bcewithlogritsloss (Binary Cross entropy loss).

Let the parameter of the semantic recognition model be theta and the loss function be BCEWithLoitsLoss.

In the pre-training stage, the loss function (for distinction, denoted as the first loss function) of the semantic recognition model is expressed as:

L＝Loss(P(y_i|x_i,θ),y′_i)

wherein x is_iRepresenting a given first sample of user input information, y_iRepresenting semantic recognition model for sample x_iIn the predicted classification result of (2), the probability distribution, y ', of the semantic information is predicted'_iRepresenting a sample x of information input at a first user_iAnd in the combined label, the true probability distribution of the semantic information.

It will be appreciated that at y_iIn, including x_iAnd predicting the classification probability on each preset semantic information.

In the pre-training stage, the training goal of the semantic recognition model is to make the first loss function lower. Optionally, the training end condition of the pre-training stage includes, but is not limited to, that the training frequency of the model reaches a preset frequency threshold, or the loss of the model converges within a preset range, or the difference between the losses corresponding to two adjacent training of the model is smaller than a preset loss difference threshold.

In this embodiment, the model fine tuning process may be performed by using a second user input information sample with an artificial fine labeling tag for training, where the step "performing parameter adjustment on the semantic recognition model based on the tag and the predicted semantic information of the second user input information sample to obtain a trained semantic recognition model" may include:

calculating the loss of the semantic recognition model based on the predicted semantic information of the second user input information sample and the semantic information in the label of the second user input information sample;

and adjusting parameters of the semantic recognition model based on the loss to obtain the trained semantic recognition model.

In the fine tuning stage, the model loss is also the classification loss, and the loss calculation may use any available classification loss function, which is not limited by the embodiment.

Or taking the loss function as BCEWithLogitsLoss as an example, or taking the parameter of the classifier model as theta. In the fine tuning stage, the loss function (for distinction, denoted as the second loss function) of the semantic recognition model is expressed as:

L＝Loss(P(y_i|z_i,6),y″_i)

wherein z is_iRepresenting a given second sample of user input information, y_iRepresenting a semantic recognition model for a sample z_iIn the prediction classification result of (2), the probability distribution, y ″, of the semantic information is predicted_iRepresenting samples z of information input at a second user_iIn the fine label, the true probability distribution of semantic information. It will be appreciated that at y_iIn, including z_iAnd predicting the classification probability on each preset semantic information.

It can be understood that in this formula, the initial value of the parameter θ of the model is the optimal parameter obtained in the pre-training process.

In this embodiment, the semantic recognition model may be subjected to multiple rounds of training, that is, after fine tuning, the merged label of the first user input information sample may be updated based on the semantic recognition model, specifically, the first semantic information in the merged label may be updated based on the semantic recognition model, the mode of updating the first semantic information is not repeated here, then the semantic recognition model is trained based on the first user input information sample after updating the label, and then the semantic recognition model is fine tuned based on the second user input information sample. After multiple rounds of training, the recognition capability of the semantic recognition model can be further improved.

To compare the two-step training framework of the semantic recognition model proposed in this example laterally, comparative experiments were performed on four classes of typical classifiers.

1152 pieces of labeled data are selected as fine labeled training data, namely a second user input information sample, 500 pieces of labeled data are selected as development set data, 1000 pieces of labeled data are selected as test data, and 10000 pieces of unlabeled data are selected as sources of weak supervision data (a first user input information sample).

In the experiment, the feature extraction layer in the RNN, CNN, and RCNN may be implemented based on a trained BERT model, i.e., using a pre-trained BERT as a fixed text feature representation, and parameters of BERT may not be updated in the training process. When BERT is used as a separate classifier, its own parameters may be updated at the time of training.

In this experiment, the comparative scheme is as follows:

RNN, CNN, RCNN and BERT are classifiers: performing model training only on the fine labeling data;

RNN, CNN, RCNN or BERT + weak supervision: and (3) performing model pre-training by using weak supervision data (labels are weak labels), and performing fine adjustment on fine labeling data.

RNN, CNN, RCNN or BERT + weak supervision enhancement: and performing model pre-training by using the strengthened weak supervision data (the label is a combined label), and performing fine tuning on the fine labeling data.

The experimental results of the three schemes under the four classifiers are as follows:

scheme(s)	F1 value	Accuracy of round
			RNN	68.73	45.30
RNN + weak supervision	75.29	53.70
			RNN + Weak supervision enhancement	76.72	54.40
CNN	70.69	45.50
			CNN + weak supervision	74.18	51.90
CNN + weak supervision enhancement	77.89	54.50
			RCNN	72.36	49.70
RCNN + weak supervision	74.89	51.00
			RCNN + weak supervision enhancement	76.81	52.50
BERT	80.13	60.60
			BERT + Weak supervision	81.82	66.00
BERT + Weak supervision enhancement	88.59	70.00

The addition of the two-step training with weakly supervised data resulted in an increase in the recall and F1 scores of the model when compared to the results of a classifier trained on the fine labeled data only, indicating that weakly supervised data is expected to improve the performance of the classifier.

Furthermore, we can see that the average boost in F1 score for the classifier trained on the enhanced weakly supervised data is 3.46%, better than the classifier without data enhancement. It has been shown that the tag association method can benefit the proposed framework with better results, which means that adding pseudo-tags is effective.

105. And performing semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user.

Optionally, in this embodiment, the step "performing semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain semantic information of the input information of the target user" may include:

and identifying the semantic information of the target user input information based on the text characteristic information of the target user input information through the semantic identification model.

The step of performing text feature extraction on the target user input information by using the trained semantic recognition model to obtain text feature information of the target user input information may include:

and mapping the input information of the target user to a feature space through the trained semantic recognition model to obtain the text feature information of the input information of the target user in the feature space.

Specifically, the target user input information may be mapped to the feature space through a feature extraction layer of the semantic recognition model, so as to obtain text feature information of the target user input information in the feature space.

The text feature information in this embodiment may be formed by vectors, for example, the text feature information is word vectors, sentence vectors, and the like.

The step of identifying semantic information of the target user input information based on text feature information of the target user input information through a semantic identification model may include:

comparing the text characteristic information of the target user input information with the characteristic distance between the preset text characteristic information of at least one semantic information in the characteristic space through a semantic recognition model;

Specifically, the text feature information may be compared by a classification layer of a semantic recognition model, and then semantic information of the target user input information may be recognized based on the feature distance.

Each classification in the classification layer corresponds to a preset semantic information, each semantic recognition model outputs the semantic information including the input information of the target user, and the prediction probability of the preset semantic information corresponding to the output is obtained.

In this embodiment, the feature extraction layer may be implemented by using a trained model that can be used for text feature extraction, specifically, a trained word2vec or BERT model, and when extracting text feature information by using the trained model structure, in the training of the semantic recognition model, when adjusting parameters of the semantic recognition model based on loss, parameters of the feature extraction layer may be kept constant.

After the semantic information of the target user input information is obtained through the semantic recognition model, the semantic information can be input into a module of a downstream task for execution of the downstream task. In one example, the downstream task may be set by a developer, and the present embodiment is not limited thereto.

Optionally, the downstream service may be a user tag generation service, and after recognizing semantic information of target user input information based on the text feature information in the step, the method may further include:

generating an inquiry label of the current user based on the semantic information;

acquiring user information of a current user;

and sending the user information of the current user, the input information of the target user and the inquiry label to an inquiry object of the current user.

In different inquiry scenes, the inquiry objects can be different, and in the Yangtze river of the medical treatment session, the inquiry objects can be doctors, triage nurses and the like.

Generating intelligent reply information of the input information of the target user according to the current conversation scene;

and outputting the intelligent reply information.

In an example, the downstream service may be an intelligent question and answer service, and after recognizing semantic information of the target user input information based on the text feature information through a semantic recognition model, the method may further include:

generating intelligent reply information of the input information of the target user based on the semantic information and the current conversation scene;

and outputting the intelligent reply information.

The dialogue scenario may be associated with the recognition domain of the semantic recognition model, for example, the recognition domain of the semantic recognition model is a medical domain, and the dialogue scenario may be a disease inquiry scenario, a medication recommendation scenario, or the like.

In another example, if there is no semantic information in which the prediction probability of the semantic information is greater than the preset prediction probability in the semantic information of the target user input information identified by the model, the model may further guide the user to input target user input information with more accurate semantics in a guiding manner, so that the model can identify more accurate semantic information, and optionally after identifying the semantic information of the target user input information through the semantic identification model based on the text feature information, the method may further include:

if the semantic information identified by the semantic identification model from the target user input information does not have the semantic information with the prediction probability exceeding the preset probability threshold, acquiring guide semantic information from the identified semantic information;

generating user guide information associated with the guide semantic information based on the guide semantic information and the current conversation scene;

outputting user guide information;

acquiring user input information aiming at user guide information;

and taking the user input information as new target user input information, and identifying the semantic information of the new target user input information based on the semantic identification model.

The guidance semantic information may be guidance information with a preset number of the prediction probabilities arranged in front (for example, the prediction probability is arranged in front 3), or guidance information with a prediction probability exceeding a minimum preset probability threshold (for example, 30%).

The user guidance information in the present embodiment may be a detailed and accurate description for guidance semantic information. The user guidance information may include first guidance information for determining guidance voice information and second guidance information for excluding non-guidance semantic information.

For example, assuming that the guidance semantic information is "abdominal pain", the first guidance information in the user guidance information may include: specifically, which part feels pain, please refer to this description input: the intra-abdominal tissue below the ribs feels pain. The second guide information may include: the pain site should not be limited to the following sites, such as the rib cage, above the ribs.

By adopting the embodiment of the application, in view of the fact that the embodiment improves the labeling source of the model training sample, information can be extracted from the dialogue information and used as the weak label of the user input information in the dialogue information, and the requirement for manual labeling data is greatly reduced. And because of weak marking, the method and the device can introduce sample data containing more spoken expressions, thereby greatly improving the recall rate of the model of the scheme and having better performance in the actual application scene. The embodiment of the application aims at the problem that the weak supervision data is lack of fine label information, and the model obtained by training on the fine labeling data is used for labeling the non-labeling data to obtain the pseudo label. Therefore, only weak supervision data with weak labels are improved, the recognition accuracy of the semantic recognition model is enhanced, the structure of the semantic recognition model is not limited in the embodiment, as long as semantic recognition can be realized, the model is trained by adopting the embodiment, the parameter quantity of the model cannot be increased, and the running speed cannot be influenced in a real scene (particularly during online service).

In this embodiment, a dialog scenario is taken as a medical dialog scenario as an example, and a semantic recognition method is described in detail.

Referring to fig. 3, the semantic recognition method includes:

301. acquiring user input information from a medical conversation;

the medical session in this embodiment may be acquired from any existing platform related to medical treatment, for example, may be acquired from a customer service module of a shopping platform, may also be acquired from an intelligent robot in a hospital, and may also be acquired from a medical service client provided by a terminal, which is not limited in this embodiment.

The medical session of the embodiment includes user input information and reply information, the reply information is a reply to the user input information by professionals in the medical field, such as doctors, nurses, pharmacists, and multiple interactive sessions can be performed between the user and the doctor in one medical session.

For example, a medical session may include the following:

the user: what is the problem of pain in my belly?

A doctor: whether it is epigastric pain or hypogastric pain, and whether there is diarrhea.

The user: lower abdominal pain with diarrhea.

A doctor: it may be acute gastroenteritis.

The information input by the user in this embodiment may be information input by the user once in a medical session, such as "what is a problem about pain in the doctor and my belly? "or, alternatively, it may be all the information input by the user in one medical session, such as" doctor, what is my belly pain? "and" lower abdominal pain with diarrhea ".

302. Sending a part of user input information to the manual labeling platform, receiving labeled user input information returned by the manual labeling platform, and determining the labeled user input information as a second user input information sample;

303. performing semantic recognition training on the semantic recognition model based on the second user input information sample;

the semantic recognition model in this embodiment may be set according to the division of the hospital into the departments of the patient clinic, and each department may be correspondingly set with one semantic recognition model, or each department may also be divided into a plurality of smaller departments, and each small department is set with one semantic recognition model.

The n classifications set by the classification layer of each semantic recognition model may include medical vocabulary that may be used in a department corresponding to the semantic recognition model, where the medical vocabulary may be a vocabulary for describing disease conditions and disorders, such as department of gastroenterology, and the classifications that may be set for the semantic recognition model include: diarrhea, gastroenteritis, abdominal pain, abdominal distension, etc.

304. Taking part of the user input information as a pseudo-labeled first user input information sample, performing semantic recognition on the first user input information sample through a trained semantic recognition model to obtain first semantic information of the first user input information sample, and generating a pseudo label of the first user input information sample based on the first semantic information;

305. acquiring reply information corresponding to the user input information in the first user input information sample from the medical conversation;

306. matching reply information corresponding to a first user input information sample with a preset semantic information set, determining second semantic information of the first user input information sample based on the semantic information matched in the semantic information set, and generating a weak label of the first user input information sample based on the second semantic information;

the semantic information set in this embodiment may be set for a semantic recognition model in each department, where one semantic recognition model is set with one corresponding semantic information set, and the semantic information in the semantic information set includes n classification names of a classification layer of the semantic recognition model and semantic associated words semantically associated with the n classification names, where the semantic associated words may also be professional medical vocabularies.

The embodiment is based on the semantic information set, and the medical keywords can be extracted from the reply information quickly to serve as the weak labels of the first user input information sample.

307. Combining the weak label and the pseudo label of the first user input information sample to obtain a combined label of the first user input information sample;

308. extracting text characteristic information from the first user input information sample through a semantic identification model, and identifying predicted semantic information of the first user input information sample based on the text characteristic information of the first user input information sample;

309. calculating a first loss of a pre-training stage of the semantic recognition model based on the labels of the first user input information samples and the predicted semantic information;

310. adjusting parameters of the semantic recognition model based on the first loss;

311. extracting text characteristic information from the second user input information sample through the adjusted semantic identification model, and identifying predicted semantic information of the second user input information sample based on the text characteristic information of the second user input information sample;

312. calculating a second loss of the semantic recognition model in the refining stage based on the labels of the second user input information samples and the predicted semantic information;

313. performing parameter adjustment on the semantic recognition model based on the second loss to obtain a trained semantic recognition model;

314. and performing semantic recognition on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user.

By adopting the scheme of the embodiment, a professional in the medical field does not need to manually label the first user input information sample, the dependence of the training of the semantic recognition model on manual labeling can be reduced, the time required by model training is shortened, and after the semantic recognition model is trained based on the first user input information sample, the second user input information sample which is precisely labeled is adopted to further finely tune the semantic recognition model, so that the semantic recognition accuracy of the model can be effectively improved.

In order to better implement the method, correspondingly, the embodiment of the invention also provides a semantic recognition device which is specifically integrated in the terminal or the server.

Referring to fig. 4, the apparatus includes:

a first semantic prediction unit 401, configured to extract text feature information from the first user input information sample through a semantic recognition model, and recognize predicted semantic information of the first user input information sample based on the text feature information of the first user input information sample, where a tag of the first user input information sample includes semantic information obtained by performing semantic recognition on the first user input information sample, and/or information obtained from reply information of the first user input information sample;

a first parameter adjustment unit 402, configured to perform parameter adjustment on the semantic recognition model based on the label and the predicted semantic information of the first user input information sample;

a second semantic prediction unit 403, configured to extract text feature information from the second user input information sample through the adjusted semantic identification model, and identify predicted semantic information of the second user input information sample based on the text feature information of the second user input information sample, where a label of the second user input information sample includes semantic information of the second user input information sample;

a second parameter adjusting unit 404, configured to perform parameter adjustment on the semantic recognition model based on the label and the predicted semantic information of the second user input information sample, to obtain a trained semantic recognition model;

and the semantic recognition unit 405 is configured to perform semantic recognition on the target user input information based on the trained semantic recognition model to obtain semantic information of the target user input information.

In an alternative example, the semantic recognition unit is configured to:

performing text feature extraction on the input information of the target user through the trained semantic recognition model to obtain text feature information of the input information of the target user;

identifying semantic information of target user input information based on text characteristic information of the target user input information through a semantic identification model

In an alternative example, the semantic recognition unit is configured to:

In an optional example, when the tag of the first user input information sample includes semantic information obtained by performing semantic recognition on the first user input information sample, the apparatus further comprises: a sample acquisition unit for:

before text feature extraction is carried out on the first user input information sample through a semantic recognition model to obtain text feature information of the first user input information sample,

obtaining a first user input information sample without labels;

pseudo-labels for the first sample of user input information are generated based on the first semantic information.

before text feature extraction is carried out on the first user input information sample through a semantic recognition model to obtain text feature information of the first user input information sample, acquiring reply information of the first user input information sample in conversation information of the first user input information sample;

weak labels of the first user input information samples are generated based on the second semantic information.

In an optional example, the sample acquiring unit is configured to:

training the semantic recognition model through the labeled second user input information sample;

performing semantic recognition on the first user input information sample by adopting the trained semantic recognition model to obtain predicted semantic information of the first user input information sample;

In an optional example, the sample acquiring unit is further configured to:

generating a pseudo label of the first user input information sample based on the second semantic information;

In an optional example, the sample acquiring unit is configured to:

matching the reply information with a preset semantic information set;

In an optional example, the first parameter adjusting unit is configured to:

parameters of the semantic recognition model are adjusted based on the loss.

after semantic recognition is carried out on the input information of the target user based on the trained semantic recognition model to obtain the semantic information of the input information of the target user, intelligent reply information of the input information of the target user is generated based on the semantic information and the current conversation scene;

and outputting the intelligent reply information.

By adopting the embodiment of the application, the label of the first user input information sample of the semantic recognition model does not need manual marking, the dependence of the training of the semantic recognition model on the manual marking can be reduced, the training speed of the semantic recognition model is improved, and the semantic recognition training of the first user input information sample and the second user input information sample on the semantic recognition model can effectively realize the accuracy of semantic recognition.

In addition, an embodiment of the present invention further provides a computer device, where the computer device may be a terminal or a server, as shown in fig. 5, which shows a schematic structural diagram of the computer device according to the embodiment of the present invention, and specifically:

the computer device may include components such as a processor 501 of one or more processing cores, memory 502 of one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will appreciate that the computer device configuration illustrated in FIG. 5 does not constitute a limitation of computer devices, and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. Wherein:

the processor 501 is a control center of the computer device, connects various parts of the entire computer device by using various interfaces and lines, and performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 502 and calling data stored in the memory 502, thereby monitoring the computer device as a whole. Optionally, processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 501.

The memory 502 may be used to store software programs and modules, and the processor 501 executes various functional applications and data processing by operating the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

The computer device further comprises a power supply 503 for supplying power to the various components, and preferably, the power supply 503 may be logically connected to the processor 501 through a power management system, so that functions of managing charging, discharging, power consumption, and the like are realized through the power management system. The power supply 503 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The computer device may also include an input unit 504, and the input unit 504 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the computer device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 501 in the computer device loads the executable file corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application programs stored in the memory 502, so as to implement various functions as follows:

Therefore, in the embodiment, the first user of the semantic recognition model inputs the label of the information sample without manual labeling, so that the dependence of training of the semantic recognition model on manual labeling can be reduced, the training speed of the semantic recognition model is increased, and the accuracy of semantic recognition can be effectively realized by using two types of samples.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present invention further provides a storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the semantic recognition method provided by the embodiment of the present invention.

According to an aspect of the application, there is also provided a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternative implementations in the embodiments described above.

Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the storage medium can execute the steps in the semantic recognition method provided by the embodiment of the present invention, the beneficial effects that can be achieved by the semantic recognition method provided by the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

The semantic recognition method, the semantic recognition device, the computer device and the storage medium according to the embodiments of the present invention are described in detail, and the principles and embodiments of the present invention are described herein by applying specific embodiments, and the description of the embodiments is only used to help understanding the method and the core concept of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A method of semantic identification, comprising:

2. The semantic recognition method according to claim 1, wherein the semantic recognition of the target user input information based on the trained semantic recognition model to obtain the semantic information of the target user input information comprises:

3. The semantic recognition method according to claim 2, wherein the obtaining of the text feature information of the target user input information by performing text feature extraction on the target user input information through the trained semantic recognition model comprises:

the recognizing the semantic information of the target user input information based on the text characteristic information of the target user input information through the semantic recognition model comprises the following steps:

4. The semantic recognition method according to claim 1, wherein when the tag of the first user input information sample includes semantic information obtained by performing semantic recognition on the first user input information sample, before extracting text feature information from the first user input information sample by the semantic recognition model, the method further comprises:

obtaining a first user input information sample without labels;

5. The semantic recognition method according to claim 1, wherein when the tag of the first user input information sample comprises information obtained from reply information of the first user input information sample, before extracting text feature information from the first user input information sample by the semantic recognition model, the method further comprises:

acquiring reply information aiming at the first user input information sample in the dialog information where the first user input information sample is located;

6. The semantic recognition method according to claim 4, wherein the performing semantic recognition on the first user input information sample to obtain the first semantic information of the first user input information sample comprises:

7. The semantic recognition method according to claim 4, further comprising, before extracting text feature information from the first user input information sample by the semantic recognition model:

8. The semantic recognition method of claim 7, wherein the obtaining of the second semantic information of the first user input information sample from the reply information comprises:

matching the reply information with a preset semantic information set;

9. The semantic recognition method according to claim 7, wherein the parameter adjustment of the semantic recognition model based on the label and the predicted semantic information of the first user input information sample comprises:

adjusting parameters of the semantic recognition model based on the loss.

10. The semantic recognition method according to any one of claims 1 to 9, wherein the semantic recognition of the target user input information based on the trained semantic recognition model to obtain the semantic information of the target user input information further comprises:

and outputting the intelligent reply information.

11. A semantic recognition apparatus, comprising:

12. A memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of the method according to any of claims 1 to 10.

13. A storage medium having a computer program stored thereon, for causing a computer to perform the steps of the method according to any one of claims 1 to 10, when the computer program runs on the computer.