WO2021143211A1 - 句子对匹配方法、装置和计算机设备和存储介质 - Google Patents

句子对匹配方法、装置和计算机设备和存储介质 Download PDF

Info

Publication number
WO2021143211A1
WO2021143211A1 PCT/CN2020/119372 CN2020119372W WO2021143211A1 WO 2021143211 A1 WO2021143211 A1 WO 2021143211A1 CN 2020119372 W CN2020119372 W CN 2020119372W WO 2021143211 A1 WO2021143211 A1 WO 2021143211A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentence
parameter
attention mechanism
controller
matching model
Prior art date
Application number
PCT/CN2020/119372
Other languages
English (en)
French (fr)
Inventor
朱威
顾婷婷
李恬静
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021143211A1 publication Critical patent/WO2021143211A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • This application relates to the field of computer technology, in particular to a sentence pair matching method, device, computer equipment and storage medium.
  • natural language processing technology Based on natural language processing technology, natural language is used for effective communication between humans and computers.
  • the natural language understanding of natural language processing technology is based on the sentence-pair matching model as the basic model.
  • a question answering system is based on the sentence-pair matching model as the basic model to realize automatic question and answer.
  • the core part of the sentence-pair matching model is the inter-sentence attention mechanism.
  • the inter-sentence attention mechanism directly determines the accuracy and reliability of the sentence-pair matching model.
  • a sentence pair matching method includes:
  • a sentence pair matching device comprising:
  • the parameter value determination module is configured to initialize the parameter value of each parameter based on the optional parameter value of each parameter of the controller, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter;
  • the matching accuracy obtaining module is used to integrate the to-be-determined attention mechanism into the coding layer of the sentence-pair matching model, and input the sample data set for training and verification, to obtain the matching accuracy of the combined sentence-to-matching model;
  • the judgment module is used to determine whether the preset termination condition is satisfied according to the matching accuracy and the current iteration number
  • the parameter adjustment module is used to adjust the parameters of the controller when the preset termination condition is not met to generate a new attention mechanism to be determined;
  • the attention mechanism obtaining module is used to return to the coding layer of the sentence-pair matching model combining the attention mechanism to be determined, and input the sample data set for training and verification, to obtain the matching of the combined sentence-to-matching model Steps of accuracy until the preset iterative condition is met, and the final attention mechanism of the sentence-pair matching model is obtained;
  • the matching degree analysis module is used to input the acquired sentence pairs to be matched into the sentence pair matching model combined with the final attention mechanism for matching degree analysis to obtain sentence pair matching results.
  • a computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of implementing a sentence pair matching method are as follows:
  • the acquired sentence pairs to be matched are input into the sentence pair matching model combined with the final attention mechanism for matching degree analysis to obtain sentence pair matching results.
  • the acquired sentence pairs to be matched are input into the sentence pair matching model combined with the final attention mechanism for matching degree analysis to obtain sentence pair matching results.
  • the above sentence pair matching method, device, computer equipment and storage medium when it is necessary to construct an attention mechanism for the sentence pair matching model, initialize the parameter value of each parameter based on the optional parameter value of each parameter of the controller, and make the controller according to each parameter.
  • the parameter values of the parameters are automatically generated for the attention mechanism to be determined, and the input sample data set in the matching model is trained and verified by combining the attention mechanism to be determined to the sentence to obtain the matching accuracy of the combined sentence to the matching model ; Judging whether the generated attention mechanism to be determined meets the requirements according to the matching accuracy and the current number of iterations; does not meet the parameters of the update controller, generating a new attention mechanism to be determined, and continue to determine whether the generated attention mechanism to be determined meets the requirements, Until the final attention mechanism that meets the requirements is obtained, the final attention mechanism obtained is the most suitable sentence pair matching model.
  • the sentence pair matching model combined with the final attention mechanism can be considered from the attention mechanism.
  • the accuracy of the matching model is optimal.
  • the accuracy of the matching result of the sentence pair is also optimal, thereby improving the accuracy of the matching result of the sentence.
  • FIG. 1 is a schematic flowchart of a sentence pair matching method in an embodiment
  • Figure 2 is a schematic structural diagram of a sentence pair matching model in an embodiment
  • FIG. 3 is a schematic diagram of the structure of the combination of the attention mechanism and the coding layer in an embodiment
  • Figure 4 is a structural block diagram of a sentence pair matching device in an embodiment
  • Figure 5 is a structural block diagram of a sentence pair matching device in another embodiment
  • Fig. 6 is an internal structure diagram of a computer device in an embodiment.
  • a sentence pair matching method is provided.
  • This embodiment uses the method applied to a terminal for illustration. It is understood that the method can also be applied to a server, and can also be applied In a system including a terminal and a server, it is realized through the interaction between the terminal and the server.
  • the method includes the following steps:
  • step S220 the parameter value of each parameter is initialized based on the optional parameter value of each parameter of the controller, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter.
  • the controller is used to automatically construct the automatic modeling system of the attention mechanism between sentences.
  • the controller is composed of a cyclic neural network and is used to determine the algorithm logic of the attention mechanism.
  • Each parameter of the controller is a decision parameter that affects the algorithm logic of the attention mechanism.
  • the algorithm logic of the attention mechanism is determined, thereby generating the attention mechanism to be determined.
  • the attention mechanism to be determined is the attention mechanism generated by the controller according to the parameter values of the parameters after initialization.
  • the attention mechanism needs to be combined with the matching model of the sentence that needs to be combined with the attention mechanism to further determine whether the attention mechanism is It is the optimal attention mechanism used to combine the sentence to the matching model.
  • step S240 the attention mechanism to be determined is integrated into the sentence-pair matching model, and the sample data set is input for training and verification, and the matching accuracy of the combined sentence-pair matching model is obtained.
  • the sentence-pair matching model is a model for understanding sentence semantics and understanding the semantic relationship between sentences.
  • a pair of sentences (sentence A and sentence B) are input, and the embedding layer, coding layer and pooling are performed respectively.
  • feature extraction is performed, and the judgment of whether the semantics match is made according to the prediction score, and then semantic analysis is realized.
  • the attention mechanism between sentences occurs in the coding layer.
  • the coding layer is composed of one or more neural network layers with the same structure. Each layer contains multiple operators. The operators are divided into two categories: operators that only encode themselves (such as text-CNN and other classic networks), and attention operators that perform attention mechanisms.
  • the sample data set is a collection of sample data used to train the sentence-pair matching model.
  • the sample data is determined according to the application scenario of the sentence-pair matching model.
  • the sentence-pair matching model uses different application scenarios, and the sample data is used for training.
  • the data is different, the sample data is different, and the algorithm logic of the combined attention mechanism is different.
  • Matching accuracy is the accuracy of the sentence-to-sentence matching model after the sentence-to-matching model is combined with the attention mechanism to be determined.
  • the characterization of sentence 1 is q
  • Step S260 according to the matching accuracy and the current number of iterations, it is determined whether the preset termination condition is met.
  • the current number of iterations is the current number of times that the to-be-determined attention mechanism is cyclically generated according to the parameter value of each parameter and the sentence pair matching model is combined to train and verify the matching accuracy.
  • the preset termination condition is a preset condition for judging whether to obtain the optimal attention mechanism of the sentence pair matching model.
  • step S280 when the preset termination condition is not met, the parameters of the controller are adjusted to generate a new attention mechanism to be determined.
  • the parameter value of each parameter of the controller is adjusted, so that the controller generates a new attention mechanism to be determined according to the adjusted parameter value of each parameter.
  • the parameter value of one or more of the parameters is adjusted based on the optional parameter value of each parameter.
  • Step S300 returning to the step of combining the attention mechanism to be determined with the coding layer of the sentence pair matching model to obtain the sentence pair matching model to be determined, until the preset iterative condition is met, and the final attention mechanism of the sentence pair matching model is obtained.
  • the final attention mechanism is the attention mechanism that is finally combined into the sentence pair matching model.
  • the attention mechanism is the sentence pair matching model considered from the attention mechanism level to make the sentence pair matching model accurate in sentence pair matching The highest attention mechanism.
  • the new attention mechanism to be determined is integrated into the sentence pair matching model, and the sample data set is input for training and verification, and the combined sentence pair is obtained.
  • the matching accuracy of the matching model it is determined whether the preset termination condition is satisfied according to the matching accuracy and the current iteration number, until the preset iteration condition is satisfied, and the final attention mechanism of the sentence to the matching model is obtained.
  • Step S320 Input the acquired sentence pairs to be matched into the sentence pair matching model combined with the final attention mechanism to perform a matching degree analysis to obtain a sentence pair matching result.
  • the sentence pair to be matched is a sentence pair that requires matching degree analysis to determine whether there is a certain relationship between the two.
  • the sentence pair matching model is a model for understanding sentence semantics and understanding the semantic relationship between sentences. As shown in Figure 2, as soon as the matching sentence pair (sentence A and sentence B) is input, the embedding layer, coding layer and pooling are performed respectively. Finally, feature extraction is performed, and the judgment of whether the semantics match is made according to the prediction score, and then semantic analysis is realized.
  • the sentence pair matching model is applied to the question answering system to realize semantic analysis.
  • the sentence pair matching result is the prediction result obtained after the matching degree analysis is performed on the sentence pair to be matched.
  • the result can be a prediction score, such as a matching degree of 90 points. The higher the prediction score score, the more matching the sentence pair.
  • the parameter value of each parameter is initialized based on the optional parameter value of each parameter of the controller, so that the controller according to the parameter value of each parameter, Automatically generate the attention mechanism to be determined, train and verify the input sample data set in the matching model by combining the attention mechanism to be determined to the sentence, and obtain the matching accuracy of the combined sentence to the matching model; according to the matching accuracy and current
  • the number of iterations determines whether the generated attention mechanism to be determined meets the requirements; the parameters of the updated controller are not met, and a new attention mechanism to be determined is generated.
  • the final attention mechanism obtained is the most suitable sentence pair matching model, and the sentence pair matching model combined with the final attention mechanism, considering the attention mechanism, can optimize the accuracy of the sentence pair matching model .
  • the accuracy of the sentence-pair matching result is also optimal, thereby improving the accuracy of sentence-to-matching results.
  • the parameter value of each parameter is initialized based on the optional parameter value of each parameter of the controller, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter, including: the optional parameter value based on the controller Select the parameter value, randomly select the parameter value of each parameter of the controller from the optional parameter value of each parameter, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter.
  • the parameters of the controller may include: whether the sentence pair matching model requires an attention mechanism, whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, the interactive operation mode of the attention mechanism, and the attention of the attention mechanism
  • the attention head can be understood as the attention mechanism repeats in multiple different subspaces, which characterize the contextual characteristics of the input text from different angles to improve the effect.
  • the optional parameter values corresponding to whether the sentence pair matching model requires an attention mechanism can include: required and not required.
  • the optional parameter values corresponding to the interactive operation mode of the attention mechanism may include: Hadamard product, dot multiplication, splicing, addition, and subtraction.
  • Whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer and the corresponding optional parameter value may include: required and not required.
  • the optional parameter values corresponding to the number of attention heads of the attention mechanism may include: 1, 2, 4, 8, and 16.
  • the optional parameter values corresponding to the position of the attention mechanism in the coding layer may include: the first layer, the second layer, and the third layer.
  • the corresponding parameter value is required; the parameter of the interactive operation mode of the attention mechanism, the corresponding parameter value is the dot product; whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, the corresponding parameter value is Need; the parameter of the number of attention heads of the attention mechanism, the corresponding parameter value is: 1; the parameter of the position of the attention mechanism in the coding layer, the corresponding parameter value is: the second layer; the controller is based on each The parameter value of the parameter, the generated attention mechanism to be determined is: combined with the second layer of the coding layer, the interactive operation mode is dot multiplication, the number of attention heads is 1, and the output needs to be the same as the output of the coding module of the coding layer Increased attention mechanism.
  • the sentence-pair matching model requires the parameters of the attention mechanism, and the corresponding parameter values may be adjusted to be unnecessary when iterating or initializing.
  • the sentence-pair matching model requires the parameters of the attention mechanism, corresponding
  • the parameter value corresponding to the parameter of the interactive operation mode of the attention mechanism whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, the parameter value of the attention mechanism
  • the parameter value corresponding to the parameter of the number of attention heads and the parameter value corresponding to the parameter of the position of the attention mechanism in the coding layer are empty, that is, no need to generate the to-be-determined attention mechanism is empty, then the sentence pair matching model does not need to be combined with attention Mechanism, input the sample data set into the sentence without the attention mechanism to train and verify the matching model, obtain the matching accuracy of the sentence to the matching model, and determine whether the preset termination condition is met according to the matching accuracy and the current iteration number, When the preset termination conditions are not met, the
  • the parameter value of each parameter is initialized, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter, and further includes: The preset hidden state generated by the mechanism, the preset hidden state is input into the controller's classifier to form the parameters of the controller; the optional items corresponding to the preset hidden state are used as optional parameter values, and the controller Each parameter is correspondingly associated.
  • the preset hidden state is the state that needs to be selected when constructing the attention mechanism for the coding layer of the sentence pair matching model.
  • the classifier is a neural network used in the controller to determine the parameter values of each parameter of the controller.
  • the preset hidden states include: whether the sentence-pair matching model requires an attention mechanism, the interactive operation mode of the attention mechanism, the number of attention heads of the attention mechanism, and whether the output of the attention mechanism needs to be coded with the sentence-pair matching model At least one of combining the output of the coding module of the layer, the position of the attention mechanism in the coding layer of the sentence-pair matching model, and the position of the attention mechanism in the coding layer of the sentence-pair matching model.
  • the options for whether the sentence pair matching model requires an attention mechanism include: need and not; the options for the interactive operation of the attention mechanism include: at least one of Hadamard product, dot multiplication, splicing, addition, and subtraction ;
  • the available options for the number of attention heads of the attention mechanism include: at least one of 1, 2, 4, 8, and 16; the attention mechanism is at the position of the coding layer of the sentence pair matching model
  • the optional options include: at least one of the first layer, the second layer, and the third layer; whether the output of the attention mechanism needs to be combined with the output of the encoding module of the encoding layer of the sentence matching model.
  • the optional options include: need and unnecessary.
  • the classifier By inputting the preset implicit state into the classifier of the controller, the classifier automatically selects the algorithm logic for constructing the attention mechanism of the sentence-pair matching model, eliminating the need for the algorithm engineer to spend time on modeling the attention mechanism and improving the attention between sentences
  • the force mechanism builds efficiency and saves costs.
  • the attention mechanism to be determined is incorporated into the sentence pair matching model, and the sample data set is input for training and verification to obtain the matching accuracy of the sentence pair matching model, including: according to the attention mechanism in each parameter In the position of the coding layer, and whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, combine the attention mechanism to be determined with the coding module of the coding layer in the sentence pair matching model to obtain the combined sentence For the matching model; input the sample data set into the combined sentence to train and verify the matching model, and obtain the matching accuracy of the combined sentence to the matching model.
  • the encoding module is an operator that encodes the encoding layer in the matching model by the sentence itself (such as text-CNN and other classic networks).
  • the sentence-pair matching model needs to combine the attention mechanism, it is to combine the attention mechanism with the operator that encodes itself in the coding layer of the sentence-pair matching model.
  • the combination method is based on the attention mechanism in each parameter in the coding layer. Position determination, such as: assuming the parameters of the position of the attention mechanism in the coding layer, the corresponding parameter value is: the second layer, whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, the corresponding parameter The value is needed.
  • the first layer of the coding layer network is the coding module. 0 and coding module 1
  • the second layer of the coding layer network is coding module 4 and the attention mechanism
  • the third layer of the coding layer network is coding module 2 and coding module 3
  • the attention mechanism is combined with the second layer of the coding layer
  • attention The output of the force mechanism needs to be added to the output of the encoding module 4 of the encoding layer and then input to the next layer.
  • the output of the attention mechanism is added to the output of the encoding module 4 of the encoding layer by adding.
  • the sample data set includes training sample data and verification sample data.
  • the matching model is trained on the combined sentence through the training sample data, the trained sentence pair matching model is obtained, and the verification sample data is input to the trained sentence pair matching model Perform verification to determine the matching accuracy of the combined sentence to the matching model. According to the matching accuracy, it can be concluded whether the accuracy and reliability of the combined sentence-pair matching model meet the application requirements, and whether the sentence-pair matching can be performed accurately and robustly. If the sentence-pair matching model is applied to the question and answer system, whether it can be accurately analyzed The semantics of the received voice data or text data is revealed, and the corresponding content is further responded.
  • the parameters of the controller are adjusted to generate a new attention mechanism to be determined, including: when the preset termination conditions are not met, the control is controlled according to the policy gradient learning algorithm.
  • the parameters of the device are adjusted to generate a new attention mechanism to be determined.
  • the strategy gradient learning algorithm is to find an evaluation index (such as expected return), and then use the stochastic gradient ascent method to update the parameters, so as to continuously maximize the evaluation index algorithm, such as the REINFORCE algorithm.
  • the REINFORCE algorithm is used to adjust the parameters of the controller to generate a new attention mechanism to be determined, which can more accurately adjust the parameter values of the controller parameters, and construct the attention mechanism required by the sentence-pair matching model more quickly. The construction efficiency of the attention mechanism between sentences.
  • determining whether the preset termination condition is satisfied according to the matching accuracy and the current iteration number includes any one or more of the following: when the matching accuracy reaches a preset threshold, it is determined that the preset termination condition is satisfied; the current iteration number When the preset number of times is reached, it is determined that the preset termination condition is satisfied; when the matching accuracy continues to the preset number of iterations and does not change, it is determined that the preset termination condition is satisfied.
  • the preset threshold is that the matching accuracy of the sentence to the matching model reaches the required matching accuracy threshold, which can be set according to the actual situation.
  • the preset number of times is the number of cycles to adjust the parameters of the controller, such as 5 times, 10 times, and 20 times, etc., such as: the preset number is 10 times, adjust the parameters of the controller at the 10th time, and generate the attention mechanism to be determined. , Which is the final attention mechanism of the sentence-pair matching model.
  • the matching accuracy does not change continuously for the preset number of iterations, it means that the parameters of the controller are continuously adjusted, the number of continuous adjustments reaches the preset number of iterations, and the matching accuracy of the resulting combined sentence to the matching model does not change, then
  • the currently generated attention mechanism to be determined is the final attention mechanism of the sentence pair matching model.
  • steps in the flowchart of FIG. 1 are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in FIG. 1 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times, and the order of execution of these steps or stages is also It is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
  • a sentence pair matching device including: a parameter value determining module 310, a matching accuracy obtaining module 320, a judgment module 330, a parameter adjustment module 340, and an attention mechanism obtaining module 350 and matching degree analysis module 360, in which:
  • the parameter value determination module 310 is used to initialize the parameter value of each parameter based on the optional parameter value of each parameter of the controller, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter.
  • the matching accuracy obtaining module 320 is used to integrate the attention mechanism to be determined into the coding layer of the sentence pair matching model, and input the sample data set for training and verification, to obtain the matching accuracy of the combined sentence to the matching model.
  • the judging module 330 is configured to determine whether the preset termination condition is satisfied according to the matching accuracy and the current iteration number.
  • the parameter adjustment module 340 is configured to adjust the parameters of the controller when the preset termination condition is not met, and generate a new attention mechanism to be determined.
  • the attention mechanism obtaining module 350 is used to return the to-be-determined attention mechanism into the coding layer of the sentence pair matching model, and input the sample data set for training and verification, and obtain the matching accuracy of the combined sentence to the matching model Steps until the preset iterative conditions are met, and the final attention mechanism of the sentence-pair matching model is obtained.
  • the matching degree analysis module 360 is configured to input the acquired sentence pairs to be matched into the sentence pair matching model combined with the final attention mechanism to perform matching degree analysis to obtain sentence pair matching results.
  • the parameter value determining module 310 is also used to: randomly select the parameter value of each parameter of the controller from the optional parameter value of each parameter based on the optional parameter value of each parameter of the controller, so that the controller According to the parameter value of each parameter, the attention mechanism to be determined is generated.
  • the device further includes: a parameter determination module 370, configured to input the preset hidden state into the classifier of the controller according to the preset hidden state generated by the attention mechanism, Form the parameters of the controller; use the options corresponding to the preset hidden states as optional parameter values and associate them with the parameters of the controller.
  • a parameter determination module 370 configured to input the preset hidden state into the classifier of the controller according to the preset hidden state generated by the attention mechanism, Form the parameters of the controller; use the options corresponding to the preset hidden states as optional parameter values and associate them with the parameters of the controller.
  • the preset hidden state of the parameter determination module 370 and the options corresponding to the preset hidden state include any one or more of the following: whether the sentence pair matching model requires an attention mechanism, where the options include Needed and not needed; the interactive operation mode of the attention mechanism, where the options include at least one of Hadamard product, dot product, splicing, addition, and subtraction; the number of attention heads of the attention mechanism, where , The options include at least one of 1, 2, 4, 8, and 16; the position of the attention mechanism in the coding layer, where the options include the first, second, and third layers At least one; whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, where the optional options include required and not required.
  • the matching accuracy obtaining module 320 is also used to: according to the position of the attention mechanism in each parameter in the coding layer, and whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, add The attention mechanism to be determined is combined with the coding module of the coding layer in the sentence-pair matching model to obtain the combined sentence-pair matching model; input the sample data set into the combined sentence to train and verify the matching model, and obtain the combined sentence The matching accuracy of the sentence to the matching model.
  • the parameter adjustment module 340 is further configured to: when the preset termination condition is not met, adjust various parameters of the controller according to the strategy gradient learning algorithm to generate a new attention mechanism to be determined.
  • the judgment module 330 is further configured to: when the matching accuracy reaches a preset threshold, determine that the preset termination condition is satisfied; when the current iteration number reaches the preset number, determine that the preset termination condition is satisfied; when the matching accuracy When the number of continuous preset iterations does not change, it is determined that the preset termination condition is met.
  • each module in the above sentence pair matching device can be implemented in whole or in part by software, hardware and a combination thereof.
  • the above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.
  • a computer device is provided.
  • the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 6.
  • the computer equipment includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus.
  • the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and a computer program.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the communication interface of the computer device is used for wired or wireless communication with an external terminal.
  • the wireless method can be implemented through WIFI, operator network, NFC (Near Field Communication) or other technologies.
  • the display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, or it can be a button, a trackball or a touch pad set on the housing of the computer equipment , It can also be an external keyboard, touchpad, or mouse.
  • a computer program is stored in the memory of the computer device, and the processor implements the following steps when the processor executes the computer program:
  • each parameter of the controller Based on the optional parameter value of each parameter of the controller, initialize the parameter value of each parameter, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter; combines the attention mechanism to be determined into the coding layer of the sentence pair matching model , And input the sample data set for training and verification, and obtain the matching accuracy of the combined sentence to the matching model; according to the matching accuracy and the current iteration number, determine whether the preset termination condition is met; when the preset termination condition is not met , Adjust the parameters of the controller to generate a new attention mechanism to be determined; return to combine the attention mechanism to be determined into the coding layer of the sentence pair matching model, and input the sample data set for training and verification, and obtain the combined
  • the processor further implements the following steps when executing the computer program: based on the optional parameter value of each parameter of the controller, randomly selecting the parameter value of each parameter of the controller from the optional parameter value of each parameter, so that The controller generates the attention mechanism to be determined according to the parameter value of each parameter.
  • the processor further implements the following steps when executing the computer program: according to the preset hidden state generated by the attention mechanism, input the preset hidden state into the classifier of the controller to form various parameters of the controller;
  • the optional items corresponding to the preset hidden states are used as optional parameter values and are correspondingly associated with the parameters of the controller.
  • the processor further implements the following steps when executing the computer program: a preset hidden state, and the options corresponding to the preset hidden state, including any one or more of the following: whether the sentence pair matching model requires an attention mechanism , Among which, the options include need and not; the interactive operation mode of the attention mechanism, where the options include at least one of Hadamard product, dot multiplication, splicing, addition and subtraction; attention of the attention mechanism The number of heads, where the options include at least one of 1, 2, 4, 8, and 16; the position of the attention mechanism at the coding layer, where the options include the first layer and the second layer And at least one of the third layer; whether the output of the attention mechanism needs to be added to the output of the encoding module of the encoding layer, where the optional options include need and don't need.
  • the options include need and not
  • the interactive operation mode of the attention mechanism where the options include at least one of Hadamard product, dot multiplication, splicing, addition and subtraction
  • attention of the attention mechanism The number of
  • the processor further implements the following steps when executing the computer program: according to the position of the attention mechanism in each parameter in the coding layer, and whether the output of the attention mechanism needs to be added to the output of the coding module of the coding layer, Combine the to-be-determined attention mechanism with the coding module of the coding layer in the sentence pair matching model to obtain the combined sentence pair matching model; input the sample data set into the combined sentence to train and verify the matching model, and obtain the combined sentence The matching accuracy of the sentence to the matching model.
  • the processor executes the computer program, the following steps are also implemented: when the preset termination condition is not met, the parameters of the controller are adjusted according to the strategy gradient learning algorithm to generate a new attention mechanism to be determined.
  • the processor further implements the following steps when executing the computer program: when the matching accuracy reaches the preset threshold, determining that the preset termination condition is met; when the current iteration number reaches the preset number of times, determining that the preset termination condition is met; When the matching accuracy remains unchanged for the preset number of iterations, it is determined that the preset termination condition is met.
  • FIG. 6 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
  • the specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.
  • a computer-readable storage medium may be non-volatile or volatile, and has a computer program stored thereon.
  • the computer program When executed by a processor, Implement the following steps:
  • each parameter of the controller Based on the optional parameter value of each parameter of the controller, initialize the parameter value of each parameter, so that the controller generates the attention mechanism to be determined according to the parameter value of each parameter; combines the attention mechanism to be determined into the coding layer of the sentence pair matching model , And input the sample data set for training and verification, and obtain the matching accuracy of the combined sentence to the matching model; according to the matching accuracy and the current iteration number, determine whether the preset termination condition is met; when the preset termination condition is not met , Adjust the parameters of the controller to generate a new attention mechanism to be determined; return to combine the attention mechanism to be determined into the coding layer of the sentence pair matching model, and input the sample data set for training and verification, and obtain the combined
  • the following steps are further implemented: based on the optional parameter value of each parameter of the controller, randomly selecting the parameter value of each parameter of the controller from the optional parameter value of each parameter, Make the controller generate the attention mechanism to be determined according to the parameter value of each parameter.
  • the following steps are also implemented: according to the preset hidden state generated by the attention mechanism, the preset hidden state is input into the classifier of the controller to form the parameters of the controller ;
  • the optional items corresponding to the preset hidden states are used as optional parameter values and are associated with the controller parameters.
  • a preset hidden state and the options corresponding to the preset hidden state, including any one or more of the following: whether the sentence pair matching model requires attention Mechanism, among which the optional options include need and unnecessary; the interactive operation mode of the attention mechanism, where the optional options include at least one of Hadamard product, dot multiplication, splicing, addition and subtraction; attention of the attention mechanism
  • the number of power heads among which, the options include at least one of 1, 2, 4, 8, and 16; the position of the attention mechanism in the coding layer, where the options include the first layer and the second At least one of the layer and the third layer; whether the output of the attention mechanism needs to be added to the output of the encoding module of the encoding layer, where the optional options include need and need not.
  • the following steps are also implemented: according to the position of the attention mechanism in the coding layer in each parameter, and whether the output of the attention mechanism needs to be the same as the output of the coding module of the coding layer Add, combine the attention mechanism to be determined with the coding module of the coding layer in the sentence pair matching model to obtain the combined sentence pair matching model; input the sample data set into the combined sentence to train and verify the matching model, Obtain the matching accuracy of the combined sentence to the matching model.
  • the following steps are further implemented: when the preset termination condition is not met, the controller parameters are adjusted according to the strategy gradient learning algorithm to generate a new attention mechanism to be determined.
  • the following steps are further implemented: when the matching accuracy reaches the preset threshold, it is determined that the preset termination condition is satisfied; when the current iteration number reaches the preset number of times, the preset termination condition is determined to be satisfied ; When the matching accuracy does not change continuously for the preset number of iterations, it is determined that the preset termination condition is met.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Machine Translation (AREA)

Abstract

本申请涉及人工智能领域，提供一种句子对匹配方法，包括：基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制；将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；根据匹配准确度和当前迭代次数，确定不满足预设终止条件时，对控制器各参数进行调整，生成新的待确定注意力机制；返回结合到句子对匹配模型的编码层中的步骤，直至满足预设迭代条件，获得最终注意力机制；将待匹配句子对输入到结合了最终注意力机制的句子对匹配模型中进行匹配度分析，获得句子对匹配结果。采用本方法能够提高句子对匹配结果的准确率。

Description

句子对匹配方法、装置和计算机设备和存储介质
本申请要求于2020年07月27日提交中国专利局、申请号为202010732106.6，发明名称为“句子对匹配方法、装置和计算机设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别是涉及一种句子对匹配方法、装置、计算机设备和存储介质。
 背景技术
随着计算机技术的发展，出现了自然语言处理技术，基于自然语言处理技术使人与计算机之间用自然语言进行有效通信。而自然语言处理技术的自然语言理解，是基于句子对匹配模型为基础模型的，如：一个问答***，是基于句子对匹配模型为基础模型，实现自动问答的。而句子对匹配模型的最核心部分是句子间注意力机制，句子间注意力机制直接决定了句子对匹配模型的准确度和可靠性。
但是目前的句子间注意力机制建模都是需要算法工程师针对自己拥有的数据和经验多次手动调整模型，最终得到算法工程师认为最优的句子间注意力机制，发明人意识到，将该句子注意力机制应用到句子对匹配模型，并不一定能够提高句子对匹配模型的精度，因此，导致句子对匹配结果的准确率低。
 技术问题
基于此，有必要针对上述技术问题，提供一种能够提高句子对匹配结果的准确率的句子对匹配方法、装置、计算机设备和存储介质。
 技术解决方案
一种句子对匹配方法，所述方法包括：
基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中，获得句子对匹配结果。
一种句子对匹配装置，所述装置包括：
参数值确定模块，用于基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
匹配准确度获得模块，用于将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
判断模块，用于根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
参数调整模块，用于当不满足预设终止条件时，对所述控制器的参数进行调整，生成新的待确定注意力机制；
注意力机制获得模块，用于返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
匹配度分析模块，用于将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现一种句子对匹配方法的步骤：
基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现一种句子对匹配方法的步骤：
基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
 有益效果
上述句子对匹配方法、装置、计算机设备和存储介质，当需要为句子对匹配模型构建注意力机制时，基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各所述参数的参数值，自动生成待确定注意力机制，通过将待确定注意力机制结合到句子对匹配模型中输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；根据匹配准确度和当前迭代次数判断生成的待确定注意力机制是否满足要求；不满足更新控制器的参数，生成新的待确定注意力机制继续判断生成的待确定注意力机制是否满足要求，直至获得满足要求的最终注意力机制，获得的最终注意力机制是最适合句子对匹配模型，将结合了最终注意力机制的句子对匹配模型，从注意力机制的方面考虑，是可以使句子对匹配模型得精度达到最优，通过该句子对匹配模型对待匹配句子对进行匹配度分析，获得句子对匹配结果的精度也是最优的，从而提高了句子对匹配结果的准确率。
 附图说明
图1为一个实施例中句子对匹配方法的流程示意图；
图2为一个实施例中句子对匹配模型的结构示意图；
图3为一个实施例中注意力机制与编码层结合的结构示意图；
图4为一个实施例中句子对匹配装置的结构框图；
图5为另一个实施例中句子对匹配装置的结构框图；
图6为一个实施例中计算机设备的内部结构图。
 本发明的最佳实施方式
在一个实施例中，如图1所示，提供了一种句子对匹配方法，本实施例以该方法应用于终端进行举例说明，可以理解的是，该方法也可以应用于服务器，还可以应用于包括终端和服务器的***，并通过终端和服务器的交互实现。本实施例中，该方法包括以下步骤：
步骤S220，基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
其中，控制器是用于自动构建句子间注意力机制的自动建模***的控制器，该控制器由一个循环神经网络组成，用于决定注意力机制的算法逻辑的神经网络控制器。控制器各参数是影响注意力机制的算法逻辑的决定参数，根据控制器各参数对应的参数值，决定注意力机制的算法的逻辑，从而生成待确定注意力机制。待确定注意力机制是控制器根据初始化后的各参数的参数值生成的注意力机制，该注意力机制需要与当前需要结合注意力机制的句子对匹配模型进行结合，进一步确定该注意力机制是否是用于结合到该句子对匹配模型最优的注意力机制。
步骤S240，将待确定注意力机制结合到句子对匹配模型中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
其中，句子对匹配模型是理解句子语义，理解句子之间的语义关系的模型，如图2所示，一对句子（句子A和句子B）输入进来，分别进行嵌入层，编码层和池化层，最后进行特征提取，根据预测分数，做出语义是否匹配的判断，进而实现语义解析。其中，句子间注意力机制发生在编码层。编码层是由一个或者多个相同结构的神经网络层叠加组成。每层其中包含多个算子，算子分为两类，只进行自身编码的算子（如text-CNN等经典网络），和进行注意力机制的注意力算子。样本数据集是用于对句子对匹配模型进行训练的样本数据的集合，样本数据根据句子对匹配模型的应用场景确定，句子对匹配模型使用到不同的应用场景，对样本数据进行训练使用的样本数据不同，样本数据不同，结合的注意力机制的算法逻辑不同。匹配准确度是句子对匹配模型结合了待确定注意力机制后，句子对匹配模型进行句子匹配的结果的准确度。
注意力算子的内部计算一般流程如下（以句子1到句子2为例，句子2到句子1的是对称的）：
句子1的表征为q，句子2的表征为k，v(这里k==v，重复两次是为了数学符号方便表示)；这3个张量的形状都是[bsz，seq_len，hidden_dim](bsz表示一个batch的数据量，seq_len表示句子长度最大值，hidden_dim表示隐状态向量长度)。
句子1对句子2的自注意力，先将3个张量转到多个注意力头上：Q=W_q*q；K=W_k*k；V=W_v*v，这里Q，K，V形状分别为[bsz，eq_len，n_head*head_size]，进行改变形状操作后，形状改为[bsz，seq_len，n_head，head_size]，这里n_head是注意力头的个数，head_size是注意力头上面的隐状态向量长度。句子1对句子2的自注意力，先计算注意力权重，交互操作：attention_weight=f_n(Q，K)；然后计算注意力后的表征：attention_weight先归一化，即在seq_len维度，权重相加为1，然后attention_weight * V，将输出结果与句子对匹配模型中自身编码算子的其他部分结合起来，来共同表征句子1的特征。
步骤S260，根据匹配准确度和当前迭代次数，确定是否满足预设终止条件。
其中，当前迭代次数是当前是第几次根据各参数的参数值循环生成待确定注意力机制与句子对匹配模型结合，训练并验证获得匹配准确度。预设终止条件是预先设定用于判断是否得到句子对匹配模型的最优注意力机制的条件。
步骤S280，当不满足预设终止条件时，对控制器各参数进行调整，生成新的待确定注意力机制。
其中，当不满足预设终止条件时，调整控制器各参数的参数值，使控制器根据各参数调整后的参数值，生成新的待确定注意力机制。调整控制器各参数的参数值时，基于各参数的可选参数值，对各参数中的一项或多项的参数值进行调整。
步骤S300，返回将待确定注意力机制与句子对匹配模型的编码层进行结合，获得待确定句子对匹配模型的步骤，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制。
其中，最终注意力机制是最终用于结合到句子对匹配模型中的注意力机制，该注意力机制是句子对匹配模型从注意力机制层面考虑，使句子对匹配模型在句子对匹配的准确度最高的注意力机制。对控制器各参数进行调整，生成新的待确定注意力机制后，将新的待确定注意力机制结合到句子对匹配模型中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度，根据匹配准确度和当前迭代次数，确定是否满足预设终止条件，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制。
步骤S320，将获取的待匹配句子对输入到结合了最终注意力机制的句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
其中，待匹配句子对是需要进行匹配度分析来确定两者之间是否具备某种关系的句子对。句子对匹配模型是理解句子语义，理解句子之间的语义关系的模型，如图2所示，一对待匹配句子对（句子A和句子B）输入进来，分别进行嵌入层，编码层和池化层，最后进行特征提取，根据预测分数，做出语义是否匹配的判断，进而实现语义解析。该句子对匹配模型应用于问答***，实现语义解析。句子对匹配结果是对待匹配句子对进行匹配度分析后，获得的预测结果，该结果可以为预测分数，如：匹配度为90分等，预测分数分数越高，该句子对就越匹配。
上述句子对匹配方法中，当需要为句子对匹配模型构建注意力机制时，基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各所述参数的参数值，自动生成待确定注意力机制，通过将待确定注意力机制结合到句子对匹配模型中输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；根据匹配准确度和当前迭代次数判断生成的待确定注意力机制是否满足要求；不满足更新控制器的参数，生成新的待确定注意力机制继续判断生成的待确定注意力机制是否满足要求，直至获得满足要求的最终注意力机制，获得的最终注意力机制是最适合句子对匹配模型，将结合了最终注意力机制的句子对匹配模型，从注意力机制的方面考虑，是可以使句子对匹配模型得精度达到最优，通过该句子对匹配模型对待匹配句子对进行匹配度分析，获得句子对匹配结果的精度也是最优的，从而提高了句子对匹配结果的准确率。
在一个实施例中，基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制，包括：基于控制器各参数的可选参数值，随机从各参数的可选参数值中，选定控制器各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
其中，控制器各参数可以包括：句子对匹配模型是否需要注意力机制、注意力机制的输出是否需要与编码层的编码模块的输出相加、注意力机制的交互操作方式、注意力机制的注意力头的个数和注意力机制在编码层的位置，注意力头可以理解为，注意力机制在多个不同的子空间重复，通过不同角度表征输入文本的上下文特征，以提升效果。句子对匹配模型是否需要注意力机制对应的可选参数值可以包括：需要和不需要。注意力机制的交互操作方式对应的可选参数值可以包括：哈达玛积、点乘、拼接、相加和相减。注意力机制的输出是否需要与编码层的编码模块的输出相加对应的可选参数值可以包括：需要和不需要。注意力机制的注意力头的个数对应的可选参数值可以包括：1个、2个、4个、8个和16个。注意力机制在编码层的位置对应的可选参数值可以包括：第一层、第二层和第三层。
随机从各参数的可选参数值中选定控制器各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制，如：句子对匹配模型是否需要注意力机制的参数，对应参数值为需要；注意力机制的交互操作方式的参数，对应的参数值为点乘；注意力机制的输出是否需要与编码层的编码模块的输出相加的参数，对应的参数值为需要；注意力机制的注意力头的个数的参数，对应的参数值为：1个；注意力机制在编码层的位置的参数，对应的参数值为：第二层；则控制器根据各参数的参数值，生成的待确定注意力机制是：结合到编码层第二层，交互操作方式为点乘，注意力头的个数为1个，输出需要与编码层的编码模块的输出相加的注意力机制。
需要说明的是，句子对匹配模型是否需要注意力机制的参数，对应的参数值在迭代或初始化时，可能会调整为不需要的情况，当句子对匹配模型是否需要注意力机制的参数，对应的参数值为不需要时，注意力机制的交互操作方式的参数对应的参数值、注意力机制的输出是否需要与编码层的编码模块的输出相加的参数对应的参数值、注意力机制的注意力头的个数的参数对应的参数值和注意力机制在编码层的位置的参数对应的参数值为空，即无需生成待确定注意力机制为空，则句子对匹配模型无需结合注意力机制，将样本数据集输入没有结合注意力机制的句子对匹配模型进行训练并验证，获得该句子对匹配模型的匹配准确度，根据匹配准确度和当前迭代次数，确定是否满足预设终止条件，当不满足预设终止条件时，对控制器各参数进行调整，生成新的待确定注意力机制，返回将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制。有可能有的样本数据集训练出来的句子对匹配模型，无需结合注意力机制，或者不结合注意力机制训练出来的句子对匹配模型，匹配准确度更高。
在一个实施例中，基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制的步骤之前，还包括：根据注意力机制生成的预设隐含状态，将预设隐含状态输入到控制器的分类器中，形成控制器各参数；并将预设隐含状态对应的可选项作为可选参数值，与控制器各参数对应关联。
其中，预设隐含状态是为句子对匹配模型的编码层构建注意力机制时，需要选定的状态。分类器是控制器中用于决定控制器各参数的参数值的神经网络。
预设隐含状态包括：句子对匹配模型是否需要注意力机制、注意力机制的交互操作方式、注意力机制的注意力头的个数、注意力机制的输出是否需要与句子对匹配模型的编码层的编码模块的输出进行结合、注意力机制在句子对匹配模型的编码层的位置和注意力机制在句子对匹配模型的编码层的位置中的至少一种。
句子对匹配模型是否需要注意力机制的可选项包括：需要和不需要；注意力机制的交互操作方式的可选项包括：哈达玛积、点乘、拼接、相加和相减中的至少一种；注意力机制的注意力头的个数的可选项包括：1个、2个、4个、8个和16个中的至少一种；注意力机制在句子对匹配模型的编码层的位置的可选项包括：第一层、第二层和第三层中的至少一种；注意力机制的输出是否需要与句子对匹配模型的编码层的编码模块的输出进行结合的可选项包括：需要和不需要。通过将预设隐含状态输入到控制器的分类器中，分类器自动选定构建句子对匹配模型的注意力机制的算法逻辑，无需算法工程师耗费注意力机制建模时间，提高了句子间注意力机制构建效率，节省了成本。
在一个实施例中，将待确定注意力机制结合到句子对匹配模型中，并输入样本数据集进行训练并验证，获得句子对匹配模型的匹配准确度，包括：根据各参数中的注意力机制在编码层的位置，以及注意力机制的输出是否需要与编码层的编码模块的输出相加，将待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；将样本数据集输入到结合后的句子对匹配模型进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
其中，编码模块是句子对匹配模型中编码层进行自身编码的算子（如text-CNN等经典网络）。当句子对匹配模型需要结合注意力机制时，是将注意力机制与句子对匹配模型中编码层中进行自身编码的算子进行结合，结合的方式根据各参数中的注意力机制在编码层的位置确定，如：假设注意力机制在编码层的位置的参数，对应的参数值为：第二层，注意力机制的输出是否需要与编码层的编码模块的输出相加的参数，对应的参数值为需要，将待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，结合到编码层后，形成的编码层如图3所示，编码层网络的第一层是编码模块0和编码模块1，编码层网络的第二层是编码模块4和注意力机制，编码层网络的第三层是编码模块2和编码模块3，注意力机制结合到编码层第二层，注意力机制的输出需要与编码层的编码模块4的输出相加后输入到下一层，注意力机制的输出与编码层的编码模块4的输出相加的方式是相加。
样本数据集包括训练样本数据和验证样本数据，通过训练样本数据对输入到结合后的句子对匹配模型进行训练，获得训练后的句子对匹配模型，将验证样本数据输入训练后的句子对匹配模型进行验证，确定结合后的句子对匹配模型的匹配准确度。根据匹配准确度可以得出结合后的句子对匹配模型准确度和可靠性是否达到应用要求，是否可以精准稳健的进行句子对匹配，如将句子对匹配模型应用到问答***，是否可以准确的分析出接收的语音数据或文字数据的语义，进一步回复对应的内容。
在一个实施例中，当不满足预设终止条件时，对控制器各参数进行调整，生成新的待确定注意力机制，包括：当不满足预设终止条件时，根据策略梯度学习算法对控制器各参数进行调整，生成新的待确定注意力机制。
其中，策略梯度学习算法是先找到一个评价指标（比如期望回报），然后使用随机梯度上升法来更新参数，从而不断的最大化评价指标的算法，如REINFORCE算法。通过REINFORCE算法对控制器各参数进行调整，生成新的待确定注意力机制，可以更准确的调整控制器各参数的参数值，更快的构建出句子对匹配模型需要的注意力机制，提高了句子间注意力机制构建效率。
在一个实施例中，根据匹配准确度和当前迭代次数，确定是否满足预设终止条件，包括以下任意一种以上：当匹配准确度达到预设阈值时，确定满足预设终止条件；当前迭代次数达到预设次数时，确定满足预设终止条件；当匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。
其中，预设阈值是句子对匹配模型的匹配准确度达到要求的匹配准确度阈值，可根据实际情况设定。预设次数是循环调整控制器各参数的次数，如5次、10次和20次等，如：预设次数为10次，在第10次调整控制器各参数，生成的待确定注意力机制，即为句子对匹配模型的最终注意力机制。当匹配准确度持续预设迭代次数未发生变化时，是指连续调整控制器各参数，连续调整次数达到预设迭代次数，得到的结合后的句子对匹配模型的匹配准确度没有发生变化，则当前生成的待确定注意力机制，即为句子对匹配模型的最终注意力机制。
应该理解的是，虽然图1的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图1中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在一个实施例中，如图4所示，提供了一种句子对匹配装置，包括：参数值确定模块310、匹配准确度获得模块320、判断模块330、参数调整模块340、注意力机制获得模块350和匹配度分析模块360，其中：
参数值确定模块310，用于基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
匹配准确度获得模块320，用于将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
判断模块330，用于根据匹配准确度和当前迭代次数，确定是否满足预设终止条件。
参数调整模块340，用于当不满足预设终止条件时，对控制器的参数进行调整，生成新的待确定注意力机制。
注意力机制获得模块350，用于返回将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制。
匹配度分析模块360，用于将获取的待匹配句子对输入到结合了最终注意力机制的句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
在一个实施例中，参数值确定模块310还用于：基于控制器各参数的可选参数值，随机从各参数的可选参数值中，选定控制器各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
请参阅图5，在一个实施例中，该装置还包括：参数确定模块370，用于根据注意力机制生成的预设隐含状态，将预设隐含状态输入到控制器的分类器中，形成控制器各参数；并将预设隐含状态对应的可选项作为可选参数值，与控制器各参数对应关联。
在一个实施例中，参数确定模块370的预设隐含状态，以及预设隐含状态对应的可选项，包括以下任意一种以上：句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；注意力机制在编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；注意力机制的输出是否需要与编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
在一个实施例中，匹配准确度获得模块320还用于：根据各参数中的注意力机制在编码层的位置，以及注意力机制的输出是否需要与编码层的编码模块的输出相加，将待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；将样本数据集输入到结合后的句子对匹配模型进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
在一个实施例中，参数调整模块340还用于：当不满足预设终止条件时，根据策略梯度学习算法对控制器各参数进行调整，生成新的待确定注意力机制。
在一个实施例中，判断模块330还用于：当匹配准确度达到预设阈值时，确定满足预设终止条件；当前迭代次数达到预设次数时，确定满足预设终止条件；当匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。
关于句子对匹配装置的具体限定可以参见上文中对于句子对匹配方法的限定，在此不再赘述。上述句子对匹配装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图6所示。该计算机设备包括通过***总线连接的处理器、存储器、通信接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作***和计算机程序。该内存储器为非易失性存储介质中的操作***和计算机程序的运行提供环境。该计算机设备的通信接口用于与外部的终端进行有线或无线方式的通信，无线方式可通过WIFI、运营商网络、NFC（近场通信）或其他技术实现。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
在一个实施例中，该计算机设备的存储器中存储有计算机程序，该处理器执行计算机程序时实现以下步骤：
基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制；将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；根据匹配准确度和当前迭代次数，确定是否满足预设终止条件；当不满足预设终止条件时，对控制器的参数进行调整，生成新的待确定注意力机制；返回将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制；将获取的待匹配句子对输入到结合了最终注意力机制的句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：基于控制器各参数的可选参数值，随机从各参数的可选参数值中，选定控制器各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：根据注意力机制生成的预设隐含状态，将预设隐含状态输入到控制器的分类器中，形成控制器各参数；并将预设隐含状态对应的可选项作为可选参数值，与控制器各参数对应关联。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：预设隐含状态，以及预设隐含状态对应的可选项，包括以下任意一种以上：句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；注意力机制在编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；注意力机制的输出是否需要与编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：根据各参数中的注意力机制在编码层的位置，以及注意力机制的输出是否需要与编码层的编码模块的输出相加，将待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；将样本数据集输入到结合后的句子对匹配模型进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：当不满足预设终止条件时，根据策略梯度学习算法对控制器各参数进行调整，生成新的待确定注意力机制。
在一个实施例中，处理器执行计算机程序时还实现以下步骤：当匹配准确度达到预设阈值时，确定满足预设终止条件；当前迭代次数达到预设次数时，确定满足预设终止条件；当匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。
本领域技术人员可以理解，图6中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在一个实施例中，提供了一种计算机可读存储介质，所述计算机可读存储介质可以是非易失性，也可以是易失性，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：
基于控制器各参数的可选参数值，初始化各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制；将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；根据匹配准确度和当前迭代次数，确定是否满足预设终止条件；当不满足预设终止条件时，对控制器的参数进行调整，生成新的待确定注意力机制；返回将待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得句子对匹配模型的最终注意力机制；将获取的待匹配句子对输入到结合了最终注意力机制的句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：基于控制器各参数的可选参数值，随机从各参数的可选参数值中，选定控制器各参数的参数值，使控制器根据各参数的参数值，生成待确定注意力机制。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：根据注意力机制生成的预设隐含状态，将预设隐含状态输入到控制器的分类器中，形成控制器各参数；并将预设隐含状态对应的可选项作为可选参数值，与控制器各参数对应关联。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：预设隐含状态，以及预设隐含状态对应的可选项，包括以下任意一种以上：句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；注意力机制在编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；注意力机制的输出是否需要与编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：根据各参数中的所述注意力机制在编码层的位置，以及注意力机制的输出是否需要与编码层的编码模块的输出相加，将待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；将样本数据集输入到结合后的句子对匹配模型进行训练并验证，获得结合后的句子对匹配模型的匹配准确度。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：当不满足预设终止条件时，根据策略梯度学习算法对控制器各参数进行调整，生成新的待确定注意力机制。
在一个实施例中，计算机程序被处理器执行时还实现以下步骤：当匹配准确度达到预设阈值时，确定满足预设终止条件；当前迭代次数达到预设次数时，确定满足预设终止条件；当匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。

Claims (20)

  1. 一种句子对匹配方法，所述方法包括：
    基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
    将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
    根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
    当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
    返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
    将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
  2. 根据权利要求1所述的方法，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制，包括：
    基于控制器各参数的可选参数值，随机从各所述参数的可选参数值中，选定所述控制器各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制。
  3. 根据权利要求2所述的方法，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制的步骤之前，还包括：
    根据注意力机制生成的预设隐含状态，将所述预设隐含状态输入到所述控制器的分类器中，形成所述控制器各参数；
    并将所述预设隐含状态对应的可选项作为可选参数值，与所述控制器各参数对应关联。
  4. 根据权利要求3所述的方法，其中，所述预设隐含状态，以及所述预设隐含状态对应的可选项，包括以下任意一种以上：
    所述句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；
    所述注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；
    所述注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；
    所述注意力机制在所述编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；
    所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
  5. 根据权利要求4所述的方法，其中，所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得所述句子对匹配模型的匹配准确度，包括：
    根据各所述参数中的所述注意力机制在所述编码层的位置，以及所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，将所述待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；
    将样本数据集输入到所述结合后的句子对匹配模型进行训练并验证，获得所述结合后的句子对匹配模型的匹配准确度。
  6. 根据权利要求1所述的方法，其中，所述当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制，包括：
    当不满足预设终止条件时，根据策略梯度学习算法对所述控制器各所述参数进行调整，生成新的待确定注意力机制。
  7. 根据权利要求1所述的方法，其中，所述根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件，包括以下任意一种以上：
    当所述匹配准确度达到预设阈值时，确定满足预设终止条件；
    当前迭代次数达到预设次数时，确定满足预设终止条件；
    当所述匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。
  8. 一种句子对匹配装置，所述装置包括：
    参数值确定模块，用于基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
    匹配准确度获得模块，用于将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
    判断模块，用于根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
    参数调整模块，用于当不满足预设终止条件时，对所述控制器的参数进行调整，生成新的待确定注意力机制；
    注意力机制获得模块，用于返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
    匹配度分析模块，用于将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现一种句子对匹配方法的步骤：
    基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
    将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
    根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
    当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
    返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
    将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
  10. 根据权利要求9所述的计算机设备，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制，包括：
    基于控制器各参数的可选参数值，随机从各所述参数的可选参数值中，选定所述控制器各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制。
  11. 根据权利要求10所述的计算机设备，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制的步骤之前，还包括：
    根据注意力机制生成的预设隐含状态，将所述预设隐含状态输入到所述控制器的分类器中，形成所述控制器各参数；
    并将所述预设隐含状态对应的可选项作为可选参数值，与所述控制器各参数对应关联。
  12. 根据权利要求11所述的计算机设备，其中，所述预设隐含状态，以及所述预设隐含状态对应的可选项，包括以下任意一种以上：
    所述句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；
    所述注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；
    所述注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；
    所述注意力机制在所述编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；
    所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
  13. 根据权利要求12所述的计算机设备，其中，所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得所述句子对匹配模型的匹配准确度，包括：
    根据各所述参数中的所述注意力机制在所述编码层的位置，以及所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，将所述待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；
    将样本数据集输入到所述结合后的句子对匹配模型进行训练并验证，获得所述结合后的句子对匹配模型的匹配准确度。
  14. 根据权利要求9所述的计算机设备，其中，所述当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制，包括：
    当不满足预设终止条件时，根据策略梯度学习算法对所述控制器各所述参数进行调整，生成新的待确定注意力机制。
  15. 根据权利要求9所述的计算机设备，其中，所述根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件，包括以下任意一种以上：
    当所述匹配准确度达到预设阈值时，确定满足预设终止条件；
    当前迭代次数达到预设次数时，确定满足预设终止条件；
    当所述匹配准确度持续预设迭代次数未发生变化时，确定满足预设终止条件。
  16. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现一种句子对匹配方法的步骤：
    基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制；
    将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度；
    根据所述匹配准确度和当前迭代次数，确定是否满足预设终止条件；
    当不满足预设终止条件时，对所述控制器各所述参数进行调整，生成新的待确定注意力机制；
    返回所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得结合后的句子对匹配模型的匹配准确度的步骤，直至满足预设迭代条件，获得所述句子对匹配模型的最终注意力机制；
    将获取的待匹配句子对输入到结合了所述最终注意力机制的所述句子对匹配模型中进行匹配度分析，获得句子对匹配结果。
  17. 根据权利要求16所述的计算机可读存储介质，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制，包括：
    基于控制器各参数的可选参数值，随机从各所述参数的可选参数值中，选定所述控制器各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制。
  18. 根据权利要求17所述的计算机可读存储介质，其中，所述基于控制器各参数的可选参数值，初始化各所述参数的参数值，使所述控制器根据各所述参数的参数值，生成待确定注意力机制的步骤之前，还包括：
    根据注意力机制生成的预设隐含状态，将所述预设隐含状态输入到所述控制器的分类器中，形成所述控制器各参数；
    并将所述预设隐含状态对应的可选项作为可选参数值，与所述控制器各参数对应关联。
  19. 根据权利要求18所述的计算机可读存储介质，其中，所述预设隐含状态，以及所述预设隐含状态对应的可选项，包括以下任意一种以上：
    所述句子对匹配模型是否需要注意力机制，其中，可选项包括需要和不需要；
    所述注意力机制的交互操作方式，其中，可选项包括哈达玛积、点乘、拼接、相加和相减中的至少一种；
    所述注意力机制的注意力头的个数，其中，可选项包括1个、2个、4个、8个和16个中的至少一种；
    所述注意力机制在所述编码层的位置，其中可选项包括第一层、第二层和第三层中的至少一种；
    所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，其中，可选项包括需要和不需要。
  20. 根据权利要求19所述的计算机可读存储介质，其中，所述将所述待确定注意力机制结合到句子对匹配模型的编码层中，并输入样本数据集进行训练并验证，获得所述句子对匹配模型的匹配准确度，包括：
    根据各所述参数中的所述注意力机制在所述编码层的位置，以及所述注意力机制的输出是否需要与所述编码层的编码模块的输出相加，将所述待确定注意力机制与句子对匹配模型中编码层的编码模块进行结合，获得结合后的句子对匹配模型；
    将样本数据集输入到所述结合后的句子对匹配模型进行训练并验证，获得所述结合后的句子对匹配模型的匹配准确度。
PCT/CN2020/119372 2020-07-27 2020-09-30 句子对匹配方法、装置和计算机设备和存储介质 WO2021143211A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010732106.6 2020-07-27
CN202010732106.6A CN111858893B (zh) 2020-07-27 2020-07-27 句子对匹配方法、装置和计算机设备和存储介质

Publications (1)

Publication Number Publication Date
WO2021143211A1 true WO2021143211A1 (zh) 2021-07-22

Family

ID=72947401

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119372 WO2021143211A1 (zh) 2020-07-27 2020-09-30 句子对匹配方法、装置和计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN111858893B (zh)
WO (1) WO2021143211A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110134771A (zh) * 2019-04-09 2019-08-16 广东工业大学 一种基于多注意力机制融合网络问答***的实现方法
US20200097604A1 (en) * 2018-09-21 2020-03-26 Microsoft Technology Licensing, Llc Stacked cross-modal matching
CN111401375A (zh) * 2020-03-09 2020-07-10 苏宁云计算有限公司 文本识别模型训练方法、文本识别方法、装置及设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3593262A4 (en) * 2017-03-10 2020-12-09 Eduworks Corporation AUTOMATED TOOL FOR GENERATING QUESTIONS
JP6738769B2 (ja) * 2017-04-27 2020-08-12 日本電信電話株式会社 文ペア分類装置、文ペア分類学習装置、方法、及びプログラム
CN111400461B (zh) * 2019-01-02 2023-03-31 ***通信有限公司研究院 智能客服问题匹配方法及装置
CN109858032A (zh) * 2019-02-14 2019-06-07 程淑玉 融合Attention机制的多粒度句子交互自然语言推理模型
CN110765240B (zh) * 2019-10-31 2023-06-20 中国科学技术大学 多相关句子对的语义匹配评估方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200097604A1 (en) * 2018-09-21 2020-03-26 Microsoft Technology Licensing, Llc Stacked cross-modal matching
CN110134771A (zh) * 2019-04-09 2019-08-16 广东工业大学 一种基于多注意力机制融合网络问答***的实现方法
CN111401375A (zh) * 2020-03-09 2020-07-10 苏宁云计算有限公司 文本识别模型训练方法、文本识别方法、装置及设备

Also Published As

Publication number Publication date
CN111858893A (zh) 2020-10-30
CN111858893B (zh) 2022-06-03

Similar Documents

Publication Publication Date Title
US11790238B2 (en) Multi-task neural networks with task-specific paths
JP7273108B2 (ja) モデルトレーニング方法、装置、電子デバイス、記憶媒体、プログラム
CN107392255B (zh) 少数类图片样本的生成方法、装置、计算设备及存储介质
CN110929869B (zh) 序列数据处理方法、装置、设备及存储介质
CN113067873B (zh) 基于深度强化学习的边云协同优化方法
KR101899101B1 (ko) 인공 신경망 기반 예측 모델 생성 장치 및 방법
JP2020522817A (ja) 意味分析方法、装置、および記憶媒体
WO2017166449A1 (zh) 机器学习模型生成方法和装置
CN113361680B (zh) 一种神经网络架构搜索方法、装置、设备及介质
WO2020103721A1 (zh) 信息处理的方法、装置及存储介质
US11663479B2 (en) Apparatus and method of constructing neural network translation model
US11195098B2 (en) Method for generating neural network and electronic device
KR20190113928A (ko) 강화 학습을 통한 디바이스 배치 최적화
CN111667056B (zh) 用于搜索模型结构的方法和装置
CN113723613B (zh) 对量子电路进行模拟的方法及装置
CN110705273B (zh) 基于神经网络的信息处理方法及装置、介质和电子设备
WO2023150912A1 (zh) 算子的调度运行时间比较方法、装置及存储介质
CN110858226A (zh) 对话管理方法和装置
CN113392640B (zh) 一种标题确定方法、装置、设备及存储介质
CN108229640B (zh) 情绪表达的方法、装置和机器人
CN113965313A (zh) 基于同态加密的模型训练方法、装置、设备以及存储介质
CN115238883A (zh) 神经网络模型的训练方法、装置、设备及存储介质
WO2021143211A1 (zh) 句子对匹配方法、装置和计算机设备和存储介质
CN115795025A (zh) 一种摘要生成方法及其相关设备
KR102324634B1 (ko) 모바일 앱 사용자의 빅데이터 분석 모델 실시간 생성을 위한 딥러닝 오차 최소화 시스템 및 그 제어방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20914589

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20914589

Country of ref document: EP

Kind code of ref document: A1