WO2023045187A1

WO2023045187A1 - Semantic search method and apparatus based on neural network, device, and storage medium

Info

Publication number: WO2023045187A1
Application number: PCT/CN2022/071219
Authority: WO
Inventors: 刘波; 王彦; 马骏; 王少军
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-09-23
Filing date: 2022-01-11
Publication date: 2023-03-30
Also published as: CN113806487A; CN113806487B

Abstract

Provided are a semantic search method and apparatus based on a neural network, a device, and a storage medium. The method comprises: obtaining a corpus (201); inputting at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model (202); inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector (203); performing entity generation according to each training corpus in the at least one training corpus to obtain at least one entity, and storing the at least one entity and the at least one corpus semantic vector into a distributed search server (204); obtaining a search request, the search request comprising a text to be searched (205); inputting said text into the semantic extraction model to obtain a search semantic vector (206); and transmitting the search semantic vector into the distributed search server for semantic search to obtain a search result (207).

Description

基于神经网络的语义搜索方法、装置、设备和存储介质Semantic search method, device, device and storage medium based on neural network

本申请要求于2021年9月23日提交中国专利局、申请号为202111118260.5，发明名称为“基于神经网络的语义搜索方法、装置、设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202111118260.5 submitted to the China Patent Office on September 23, 2021, and the title of the invention is "Semantic Search Method, Device, Equipment and Storage Medium Based on Neural Network", the entire content of which Incorporated in this application by reference.

技术领域technical field

本申请涉及人工智能技术领域，具体涉及一种基于神经网络的语义搜索方法、装置、设备和存储介质。The present application relates to the technical field of artificial intelligence, in particular to a neural network-based semantic search method, device, equipment and storage medium.

背景技术Background technique

语义搜索作为信息搜索和自然语言处理领域的一个分支，受到的关注越来越多。具体而言，语义搜索引擎能从数亿条文本中，搜索出与用户输入的句子，语义相同或相近的语句，它作为计算机语义理解和人机交互的基础，已广泛应用到智能客服、智能问答、推荐***等领域，并在这些领域中发挥着重要作用。Semantic search, as a branch of information search and natural language processing, has received more and more attention. Specifically, the semantic search engine can search out sentences with the same or similar semantics to the sentences entered by users from hundreds of millions of texts. As the basis for computer semantic understanding and human-computer interaction, it has been widely used in intelligent customer service, intelligent Question answering, recommendation system and other fields, and play an important role in these fields.

发明人发现，目前，现有的语义搜索引擎多通过文本正则匹配等方式，或者使用词频-逆文本频率指数(Term Frequency–Inverse Document Frequency，TF-IDF)等简单算法计算输入语句与搜索库中的语句之间的文本相似度，从而实现语义搜索。The inventors found that, at present, the existing semantic search engines mostly use text regular matching, or use simple algorithms such as Term Frequency–Inverse Document Frequency (TF-IDF) to calculate the input sentence and search database. The textual similarity between the sentences of the sentence, so as to realize the semantic search.

但是，发明人意识到，上述语义搜索引擎所使用的语义分析方式无法真正理解文本语义，导致搜索准确率低。且现有的语义搜索引擎***将数亿条文本存储于搜索库中，致使搜索库的数据存储占用空间大，数据传输效率低，无法应对大数据多业务多任务场景的语义搜索。此外，现有的语义搜索引擎无法做到分布式计算，导致搜索算法效率低、数据传输效率低。总而言之，现有的语义搜索引擎存在搜索速度慢、搜索准确率低、应用场景窄、以及无法实时返回搜索结果的问题。However, the inventor realized that the semantic analysis method used by the above-mentioned semantic search engine cannot truly understand the semantics of the text, resulting in low search accuracy. Moreover, the existing semantic search engine system stores hundreds of millions of texts in the search database, resulting in a large space for data storage in the search database and low data transmission efficiency, which cannot cope with semantic search in big data, multi-service and multi-task scenarios. In addition, existing semantic search engines cannot achieve distributed computing, resulting in low efficiency of search algorithms and low efficiency of data transmission. All in all, the existing semantic search engines have the problems of slow search speed, low search accuracy, narrow application scenarios, and inability to return search results in real time.

发明内容Contents of the invention

为了解决现有技术中存在的上述问题，本申请实施方式提供了一种基于神经网络的语义搜索方法、装置、设备和存储介质，可以实现对文本语义的精准理解和分布式搜索，提升了搜索速度和搜索的准确性。In order to solve the above-mentioned problems in the prior art, the embodiment of the present application provides a neural network-based semantic search method, device, device and storage medium, which can realize precise understanding of text semantics and distributed search, and improve search Speed and accuracy of searches.

第一方面，本申请的实施方式提供了一种基于神经网络的语义搜索方法，包括：In the first aspect, the embodiment of the present application provides a semantic search method based on a neural network, including:

获取语料库，其中，语料库中包括至少一个训练语料，且至少一个训练语料中的每个训练语料对应的业务类型相同；Obtaining a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，组合神经网络由至少两个子神经网络组成，且至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，模型优化神经网络用于优化模型生成神经网络生成的模型；Input at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one model generation neural network and at least a model optimization neural network, the model optimization neural network is used to optimize the model generated by the model generation neural network;

将至少一个训练语料中的每个训练语料输入语义提取模型，得到至少一个语料语义向量，其中，至少一个语料语义向量和至少一个训练语料一一对应；Inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein at least one corpus semantic vector corresponds to at least one training corpus;

根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将至少一个实体和至少一个语料语义向量存入分布式搜索服务器，其中，至少一个实体和至少一个训练语料一一对应；Entity generation is performed according to each training corpus in at least one training corpus, at least one entity is obtained, and at least one entity and at least one corpus semantic vector are stored in a distributed search server, wherein at least one entity and at least one training corpus—— correspond;

获取检索请求，检索请求包括待检索文本；Obtain a retrieval request, the retrieval request includes the text to be retrieved;

将待检索文本输入语义提取模型，得到检索语义向量；Input the text to be retrieved into the semantic extraction model to obtain the retrieval semantic vector;

将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果。Pass the retrieval semantic vector to the distributed search server for semantic retrieval and obtain the retrieval results.

第二方面，本申请的实施方式提供了一种基于神经网络的语义搜索装置，包括：In a second aspect, the implementation of the present application provides a neural network-based semantic search device, including:

采集模块，用于获取语料库，其中，语料库中包括至少一个训练语料，且至少一个训练语料中的每个训练语料对应的业务类型相同；A collection module, configured to obtain a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

训练模块，用于将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，组合神经网络由至少两个子神经网络组成，且至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，模型优化神经网络用于优化模型生成神经网络生成的模型；The training module is used to input at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and at least two sub-neural networks include at least one model generating a neural network and at least one model optimizing neural network, the model optimizing neural network is used to optimize the model generated by the model generating neural network;

处理模块，用于将至少一个训练语料中的每个训练语料输入语义提取模型，得到至少一个语料语义向量，其中，至少一个语料语义向量和至少一个训练语料一一对应；A processing module, configured to input each training corpus in at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein at least one corpus semantic vector corresponds to at least one training corpus;

实体生成模块，用于根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将至少一个实体和至少一个语料语义向量存入分布式搜索服务器，其中，至少一个实体和至少一个训练语料一一对应；The entity generation module is used to generate entities according to each training corpus in at least one training corpus, obtain at least one entity, and store at least one entity and at least one corpus semantic vector in the distributed search server, wherein at least one entity and At least one training corpus has one-to-one correspondence;

接收模块，用于获取检索请求，检索请求包括待检索文本；A receiving module, configured to obtain a retrieval request, where the retrieval request includes text to be retrieved;

检索模块，用于将待检索文本输入语义提取模型，得到检索语义向量，并将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果。The retrieval module is used to input the text to be retrieved into the semantic extraction model to obtain the retrieval semantic vector, and transfer the retrieval semantic vector to the distributed search server for semantic retrieval to obtain the retrieval result.

第三方面，本申请实施方式提供一种电子设备，包括：处理器，处理器与存储器相连，存储器用于存储计算机程序，处理器用于执行存储器中存储的计算机程序，以使得电子设备执行如第一方面的方法，该方法包括：In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs as described in A method in one aspect, the method comprising:

获取语料库，其中，所述语料库中包括至少一个训练语料，且所述至少一个训练语料中的每个训练语料对应的业务类型相同；Acquiring a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，所述组合神经网络由至少两个子神经网络组成，且所述至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，所述模型优化神经网络用于优化所述模型生成神经网络生成的模型；The at least one training corpus is input into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one a model-generating neural network and at least one model-optimizing neural network for optimizing a model generated by the model-generating neural network;

将所述至少一个训练语料中的每个训练语料输入所述语义提取模型，得到至少一个语料语义向量，其中，所述至少一个语料语义向量和所述至少一个训练语料一一对应；Inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein the at least one corpus semantic vector corresponds to the at least one training corpus;

根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，其中，所述至少一个实体和所述至少一个训练语料一一对应；Entity generation is performed according to each training corpus in the at least one training corpus to obtain at least one entity, and the at least one entity and the at least one corpus semantic vector are stored in a distributed search server, wherein the at least one There is a one-to-one correspondence between the entity and the at least one training corpus;

获取搜索请求，所述搜索请求包括待搜索文本；Obtain a search request, the search request includes the text to be searched;

将所述待搜索文本输入所述语义提取模型，得到搜索语义向量；Inputting the text to be searched into the semantic extraction model to obtain a search semantic vector;

将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果。The search semantic vector is transmitted to the distributed search server to perform semantic search, and search results are obtained.

第四方面，本申请实施方式提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，计算机程序使得计算机执行如第一方面的方法，该方法包括：In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program causes the computer to execute the method according to the first aspect, the method comprising:

实施本申请实施方式，提升了后续语义搜索的精准度，提升了搜索速度，并拓宽了应用场景。The implementation of the embodiments of the present application improves the accuracy of the subsequent semantic search, improves the search speed, and broadens the application scenarios.

附图说明Description of drawings

图1为本申请实施方式提供的一种基于神经网络的语义搜索装置的硬件结构示意图；FIG. 1 is a schematic diagram of the hardware structure of a neural network-based semantic search device provided in an embodiment of the present application;

图2为本申请实施方式提供的一种基于神经网络的语义搜索方法的流程示意图；FIG. 2 is a schematic flow diagram of a neural network-based semantic search method provided in an embodiment of the present application;

图3为本申请实施方式提供的一种将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型的方法的流程示意图；3 is a schematic flow diagram of a method for inputting at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model provided by an embodiment of the present application;

图4为本申请实施方式提供的一种第二模型的总体架构图；FIG. 4 is an overall architecture diagram of a second model provided in an embodiment of the present application;

图5为本申请实施方式提供的一种第二模型融合卷积神经网络的架构示意图；FIG. 5 is a schematic diagram of the architecture of a second model fusion convolutional neural network provided in an embodiment of the present application;

图6为本申请实施方式提供的一种对第三模型进行白化处理，得到语义提取模型的方法的流程示意图；FIG. 6 is a schematic flowchart of a method for whitening a third model to obtain a semantic extraction model provided in an embodiment of the present application;

图7为本申请实施方式提供的一种根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体的方法的流程示意图；FIG. 7 is a schematic flowchart of a method for generating entities according to each training corpus in at least one training corpus to obtain at least one entity provided by an embodiment of the present application;

图8为本申请实施方式提供的一种不同实体的数据结构组成示意图；FIG. 8 is a schematic diagram of a data structure composition of a different entity provided in an embodiment of the present application;

图9为本申请实施方式提供的一种存储需要的格式的示意图；FIG. 9 is a schematic diagram of a storage required format provided by the embodiment of the present application;

图10为本申请实施方式提供的一种结合Elasticsearch、gRPC、HNSW和tensorflow serving的语义搜索引擎的结构示意图；Fig. 10 is a schematic structural diagram of a semantic search engine combining Elasticsearch, gRPC, HNSW and tensorflow serving provided by the embodiment of the present application;

图11为本申请实施方式提供的一种语义搜索引擎的总体结构图；FIG. 11 is an overall structural diagram of a semantic search engine provided in an embodiment of the present application;

图12为本申请实施方式提供的一种基于神经网络的语义搜索装置的功能模块组成框图；FIG. 12 is a block diagram of functional modules of a neural network-based semantic search device provided in an embodiment of the present application;

图13为本申请实施方式提供的一种电子设备的结构示意图。FIG. 13 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

具体实施方式Detailed ways

下面将结合本申请实施方式中的附图，对本申请实施方式中的技术方案进行描述。The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

首先，参阅图1，图1为本申请实施方式提供的一种基于神经网络的语义搜索装置的硬件结构示意图。该基于神经网络的语义搜索装置100包括至少一个处理器101，通信线路102，存储器103以及至少一个通信接口104。First, refer to FIG. 1 . FIG. 1 is a schematic diagram of a hardware structure of a neural network-based semantic search device provided in an embodiment of the present application. The neural network-based semantic search device 100 includes at least one processor 101 , a communication line 102 , a memory 103 and at least one communication interface 104 .

在本实施方式中，处理器101，可以是一个通用中央处理器(central processing unit，CPU)，微处理器，特定应用集成电路(application-specific integrated circuit，ASIC)，或一个或多个用于控制本申请方案程序执行的集成电路。In this embodiment, the processor 101 may be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, a specific application integrated circuit (application-specific integrated circuit, ASIC), or one or more An integrated circuit that controls the program execution of the program of this application.

通信线路102，可以包括一通路，在上述组件之间传送信息。 Communication line 102, which may include a path, transmits information between the aforementioned components.

通信接口104，可以是任何收发器一类的装置(如天线等)，用于与其他设备或通信网络通信，例如以太网，RAN，无线局域网(wireless local area networks，WLAN)等。The communication interface 104 may be any device such as a transceiver (such as an antenna) for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (wireless local area networks, WLAN) and the like.

存储器103，可以是只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、只读光盘(compact disc read-only memory，CD-ROM)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。 Memory 103 may be read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM) or other types that can store information and instructions Type of dynamic storage device, also can be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), read-only disc (compact disc read-only memory, CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be used by Any other medium accessed by a computer, but not limited to.

在本实施方式中，存储器103可以独立存在，通过通信线路102与处理器101相连接。存储器103也可以和处理器101集成在一起。本申请实施方式提供的存储器103通常可以具有非易失性。其中，存储器103用于存储执行本申请方案的计算机执行指令，并由处理器101来控制执行。处理器101用于执行存储器103中存储的计算机执行指令，从而实现本申请下述实施方式中提供的方法。In this embodiment, the memory 103 may exist independently and be connected to the processor 101 through the communication line 102 . The memory 103 can also be integrated with the processor 101 . The memory 103 provided in this embodiment of the present application may generally be non-volatile. Wherein, the memory 103 is used to store computer-executed instructions for implementing the solutions of the present application, and the execution is controlled by the processor 101 . The processor 101 is used to execute the computer-executed instructions stored in the memory 103, so as to implement the methods provided in the following embodiments of the present application.

在可选的实施方式中，计算机执行指令也可以称之为应用程序代码，本申请对此不作具体限定。In an optional implementation manner, computer-executed instructions may also be referred to as application code, which is not specifically limited in the present application.

在可选的实施方式中，处理器101可以包括一个或多个CPU，例如图1中的CPU0和CPU1。In an optional implementation manner, the processor 101 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 1 .

在可选的实施方式中，该基于神经网络的语义搜索装置100可以包括多个处理器，例如图1中的处理器101和处理器107。这些处理器中的每一个可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。In an optional implementation manner, the neural network-based semantic search apparatus 100 may include multiple processors, such as processor 101 and processor 107 in FIG. 1 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

在可选的实施方式中，若基于神经网络的语义搜索装置100为服务器，例如，可以是独立的服务器，也可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(ContentDelivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。则基于神经网络的语义搜索装置100还可以包括输出设备105和输入设备106。输出设备105和处理器101通信，可以以多种方式来显示信息。例如，输出设备105可以是液晶显示器(liquid crystal display，LCD)，发光二级管(light emitting diode，LED)显示设备，阴极射线管(cathode ray tube，CRT)显示设备，或投影仪(projector)等。输入设备106和处理器101通信，可以以多种方式接收用户的输入。例如，输入设备106可以是鼠标、键盘、触摸屏设备或传感设备等。In an optional embodiment, if the neural network-based semantic search device 100 is a server, for example, it may be an independent server, or it may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, Cloud servers for basic cloud computing services such as cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Then the neural network-based semantic search apparatus 100 may further include an output device 105 and an input device 106 . Output device 105 is in communication with processor 101 and may display information in a variety of ways. For example, the output device 105 may be a liquid crystal display (liquid crystal display, LCD), a light emitting diode (light emitting diode, LED) display device, a cathode ray tube (cathode ray tube, CRT) display device, or a projector (projector) wait. The input device 106 communicates with the processor 101 and can receive user input in various ways. For example, the input device 106 may be a mouse, a keyboard, a touch screen device, or a sensing device, among others.

上述的基于神经网络的语义搜索装置100可以是一个通用设备或者是一个专用设备。本申请实施方式不限定基于神经网络的语义搜索装置100的类型。The above-mentioned neural network-based semantic search apparatus 100 may be a general-purpose device or a special-purpose device. The embodiment of the present application does not limit the type of the neural network-based semantic search device 100 .

其次，需要说明的是，本申请所公开的实施方式可以基于人工智能技术对相关的数据进行获取和处理。其中，人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用***。Secondly, it should be noted that the embodiments disclosed in this application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. .

人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互***、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、机器人技术、生物识别技术、语音处理技术、自然语言处理技术以及机器学习/深度学习等几大方向。Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

以下，将对本申请所公开的基于神经网络的语义搜索方法进行说明：Below, the semantic search method based on neural network disclosed in this application will be described:

参阅图2，图2为本申请实施方式提供的一种基于神经网络的语义搜索方法的流程示意图。该基于神经网络的语义搜索方法包括以下步骤：Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a neural network-based semantic search method provided in an embodiment of the present application. The neural network-based semantic search method includes the following steps:

201：获取语料库。201: Obtain a corpus.

在本实施方式中，语料库中可以包括至少一个训练语料，且该至少一个训练语料中的每个训练语料对应的业务类型相同。简而言之，可以将同一种业务对应的语料存放于同一个语料库中，由此，可以生成针对某一业务的专用搜索库，提升后续语义搜索的效率。In this embodiment, the corpus may include at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same service type. In short, the corpus corresponding to the same business can be stored in the same corpus, so that a dedicated search library for a certain business can be generated to improve the efficiency of subsequent semantic searches.

202：将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型。202: Input at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model.

在本实施方式中，该组合神经网络可以由至少两个子神经网络组成，且该至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，模型优化神经网络用于优化模型生成神经网络生成的模型。示例性的，该至少两个子神经网络可以包括：老师神经网络(teacher network)、学生神经网络(student network)和卷积神经网络(Convolutional Neural Networks,CNN)。In this embodiment, the combined neural network can be composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one model generation neural network and at least one model optimization neural network, and the model optimization neural network is used to optimize the model. Generate the model generated by the neural network. Exemplarily, the at least two sub-neural networks may include: a teacher neural network (teacher network), a student neural network (student network), and a convolutional neural network (Convolutional Neural Networks, CNN).

具体而言，本申请中提供了一种将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型的方法，如图3所示，该方法包括：Specifically, the present application provides a method of inputting at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, as shown in Figure 3, the method includes:

301：将至少一个训练语料输入老师神经网络进行训练，得到第一模型。301: Input at least one training corpus into the teacher's neural network for training to obtain a first model.

在本实施方式中，可以通过将语料库在masked language model任务上对老师神经网络进行预训练，得到第一模型。In this embodiment, the first model can be obtained by pre-training the teacher's neural network on the masked language model task using the corpus.

302：根据学生神经网络，对第一模型进行蒸馏处理，得到第二模型。302: Perform distillation processing on the first model according to the student neural network to obtain a second model.

在本实施方式中，学生神经网络的结构与老师神经网络(BERT)的结构基本一致，差异在于学生神经网络将BERT的层数减少至原来的一半。且学生神经网络的每一层增加初始化，每一层的初始化为老师神经网络的参数。而蒸馏后的第一模型，即第二模型，使神经网络的编码速度提升了60％。如图4所示，图4示出了第二模型的总体架构图。In this embodiment, the structure of the student neural network is basically the same as that of the teacher neural network (BERT), the difference is that the student neural network reduces the number of layers of BERT to half of the original one. And each layer of the student's neural network is initialized, and the initialization of each layer is the parameter of the teacher's neural network. And the distilled first model, the second model, made the encoding speed of the neural network 60% faster. As shown in Fig. 4, Fig. 4 shows an overall architecture diagram of the second model.

303：将卷积神经网络与第二模型进行融合参数微调训练，得到第三模型。303: Perform fusion parameter fine-tuning training on the convolutional neural network and the second model to obtain a third model.

在本实施方式中，第三模型可以理解为参数微调后的第二模型。具体而言，可以将第二模型的输出向量输入卷积神经网络的一维卷积层，得到至少一个第一向量。然后，对至少一个第一向量中的每个第一向量进行最大池化，并将最大池化后的每个第一向量进行拼接，得到第二向量。再将第二向量输入卷积神经网络的全连接层，得到第一矩阵。最后，通过softmax对第一矩阵进行分类，并根据分类结果对第二模型进行优化调整，得到第三模型。In this embodiment, the third model can be understood as the second model after parameters are fine-tuned. Specifically, the output vector of the second model may be input into the one-dimensional convolution layer of the convolutional neural network to obtain at least one first vector. Then, perform maximum pooling on each first vector in the at least one first vector, and concatenate each first vector after maximum pooling to obtain a second vector. Then input the second vector into the fully connected layer of the convolutional neural network to obtain the first matrix. Finally, the first matrix is classified by softmax, and the second model is optimized and adjusted according to the classification result to obtain the third model.

具体而言，如图5所示，图5示出了一个第二模型融合卷积神经网络的架构示意图。其中，将第二模型(DistilBERT)的输出，输入一维卷积层convolutional layer，然后将卷积后得到的向量进行最大池化max-pooling并进行concatenate拼接，最后经过一个全连接层(full connected layer),输出一个形状为(batch_size，num_classes)的矩阵。最后利用回归模型，例如：softmax进行分类。由此，利用业务数据对DistilBERT在分类任务上进行参数微调，进一步的提高了语义编码的准确率。Specifically, as shown in FIG. 5 , FIG. 5 shows a schematic architecture diagram of a second model fusion convolutional neural network. Among them, the output of the second model (DistilBERT) is input into the one-dimensional convolutional layer, and then the vector obtained after convolution is subjected to max-pooling and concatenate splicing, and finally passes through a fully connected layer (full connected layer), outputs a matrix of shape (batch_size, num_classes). Finally, a regression model, such as softmax, is used for classification. As a result, the business data is used to fine-tune the parameters of DistilBERT on the classification task, which further improves the accuracy of semantic coding.

304：对第三模型进行白化处理，得到语义提取模型。304: Perform whitening processing on the third model to obtain a semantic extraction model.

在本实施方式中，考虑到输入向量如果为各项同性，再进行余弦相似度计算会提高相似度计算的精度。因此，可以对第三模型的输出向量进行白化。基于此，本实施方式提供了一种对第三模型进行白化处理的方法，如图6所示，该方法包括：In this embodiment, considering that the input vector is isotropic, performing cosine similarity calculation will improve the accuracy of similarity calculation. Therefore, the output vector of the third model can be whitened. Based on this, this embodiment provides a method for whitening the third model, as shown in FIG. 6 , the method includes:

601：根据第三模型的输出向量x _i，确定第一参数u。 601: Determine a first parameter u according to an output vector x _i of the third model.

在本实施方式中，第一参数u可以通过公式①进行表示：In this embodiment, the first parameter u can be expressed by formula ①:

其中，N为第三模型的输出向量x _i中元素的数量。 Wherein, N is the number of elements in the output vector _xi of the third model.

602：确定第三模型的输出向量x _i的协方差矩阵A。 602: Determine the covariance matrix A of the output vector _xi of the third model.

603：对第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C。603: Perform singular value decomposition on the covariance matrix A of the output vector of the third model to obtain a first characteristic matrix B and a second characteristic matrix C.

在本实施方式中，第三模型的输出向量的协方差矩阵A、第一特征矩阵B和第二特征矩阵C满足公式②：In this embodiment, the covariance matrix A, the first characteristic matrix B and the second characteristic matrix C of the output vector of the third model satisfy the formula ②:

A＝BCB ^T.........② A＝ ^BCBT ......②

其中，B ^T表示第一特征矩阵B的转置矩阵； Wherein, ^BT represents the transposition matrix of the first feature matrix B;

604：根据第一特征矩阵B和第二特征矩阵C，确定第二参数W。604: Determine a second parameter W according to the first characteristic matrix B and the second characteristic matrix C.

在本实施方式中，第二参数W、第一特征矩阵B和第二特征矩阵C满足公式③：In this embodiment, the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the formula ③:

605：根据第一参数u和第二参数W，对第三模型的输出向量x _i进行白化处理，得到白化向量

605: According to the first parameter u and the second parameter W, perform whitening processing on the output vector x _i of the third model to obtain a whitening vector

在本实施方式中，第一参数u、第二参数W、第三模型的输出向量x _i和白化向量

满足公式④： In this embodiment, the first parameter u, the second parameter W, the output vector x _i of the third model and the whitening vector

Satisfy the formula ④:

在本实施方式中，该白化向量

即为第三模型的白化结果。 In this embodiment, the whitening vector

It is the whitening result of the third model.

由此，在本实施方式中，采用DistilBERT、CNN、Whitening结合优化的编码方式，进一步加快了神经网络的语义编码速度和语义理解精度，从而使后续的相似度计算的任务的效率和准确率得到了提升，具体而言，准确率得到了5％的提升。Therefore, in this embodiment, the optimized encoding method of DistilBERT, CNN, and Whitening is adopted to further accelerate the semantic encoding speed and semantic understanding accuracy of the neural network, thereby improving the efficiency and accuracy of the subsequent similarity calculation tasks. There was an improvement, specifically, a 5% improvement in accuracy.

在可选的实施方式中，还可以对白化后的第三模型进行降维处理，再得到语义提取模型，从而进一步提升神经网络的语义编码速度和语义理解精度。In an optional implementation manner, dimensionality reduction processing may also be performed on the whitened third model to obtain a semantic extraction model, thereby further improving the semantic encoding speed and semantic understanding accuracy of the neural network.

203：将至少一个训练语料中的每个训练语料输入语义提取模型，得到至少一个语料语义向量。203: Input each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector.

在本实施方式中，该至少一个语料语义向量和至少一个训练语料一一对应。In this implementation manner, the at least one corpus semantic vector is in one-to-one correspondence with at least one training corpus.

204：根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将至少一个实体和至少一个语料语义向量存入分布式搜索服务器。204: Generate entities according to each training corpus in at least one training corpus to obtain at least one entity, and store at least one entity and at least one semantic vector of the corpus in the distributed search server.

在本实施方式中，该分布式搜索服务器可以是Elasticsearch搜索服务器，且该至少一个实体和至少一个训练语料一一对应。In this embodiment, the distributed search server may be an Elasticsearch search server, and the at least one entity corresponds to at least one training corpus.

在本实施方式中，通过引入实体生成器及实体序列化，且使不同实体对应于不同的业务，由此，能够对新增的业务进行快速扩展支持，以实现单语义搜索引擎对多任务多业务的支持。In this embodiment, by introducing the entity generator and entity serialization, and making different entities correspond to different services, it is possible to quickly expand and support the newly added services, so as to realize the single-semantic search engine's support for multiple tasks and multiple tasks. Business support.

示例性的，本实施方式提供了一种根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体的方法，如图7所示，该方法包括：Exemplarily, this embodiment provides a method for generating entities according to each training corpus in at least one training corpus to obtain at least one entity, as shown in FIG. 7 , the method includes:

701：根据每个训练语料的业务类型，获取实体模板。701: Obtain an entity template according to the business type of each training corpus.

在本实施方式中，不同业务类型的业务对应的实体的数据结构并不相同。示例性的，如图8所示，图8示出了本实施方式中包括的不同实体的数据结构组成示意图。具体而言，本实施方式所提供的实体可以包括：文本搜索实体、意图识别实体、问答实体、相似句查询实体、情绪识别实体(客户/座席)和文本质检实体(客户/座席)。实体可以根据其对应的业务类型的特性，选取不同的数据存储进实体中。例如，对于文本搜索实体，由于文本搜索任务的关注点在于搜索结果上，因此，该实体中可以包括：文本、搜索结果、搜索结果对应主题和搜索结果对应类别。In this embodiment, the data structures of entities corresponding to services of different service types are different. Exemplarily, as shown in FIG. 8 , FIG. 8 shows a schematic diagram of data structure composition of different entities included in this embodiment. Specifically, the entities provided in this embodiment may include: a text search entity, an intent recognition entity, a question-and-answer entity, a similar sentence query entity, an emotion recognition entity (customer/agent) and a text quality inspection entity (customer/agent). An entity can select different data to store in the entity according to the characteristics of its corresponding business type. For example, for a text search entity, since the text search task focuses on search results, the entity may include: text, search results, topics corresponding to search results, and categories corresponding to search results.

702：根据实体模型的数据结构，对每个训练语料进行数据采集，得到至少一个实体数据。702: According to the data structure of the entity model, perform data collection on each training corpus to obtain at least one entity data.

在本实施方式中，以情绪识别实体的实体模型为例，其数据结构为：角色、文本、文本对应的情绪和该情绪的程度。基于此，首先确定每个训练语料所对应的角色信息，再对该训练语料进行情绪识别和情绪程度确定。In this embodiment, the entity model of the emotion recognition entity is taken as an example, and its data structure is: character, text, emotion corresponding to the text, and the degree of the emotion. Based on this, first determine the role information corresponding to each training corpus, and then perform emotion recognition and emotion degree determination on the training corpus.

703：将至少一个实体数据中的每个实体数据填入实体模板中对应的位置，得到至少一个实体。703: Fill each entity data in the at least one entity data into a corresponding position in the entity template to obtain at least one entity.

在本实施方式中，沿用上述情绪识别实体的示例，可以将角色信息、训练语料、情绪识别结果和程度确定成果进行封装，形成情绪识别实体。In this embodiment, following the example of the above-mentioned emotion recognition entity, role information, training corpus, emotion recognition results and degree determination results can be packaged to form an emotion recognition entity.

此外，在本实施方式中，在将至少一个实体和至少一个语料语义向量存入Elasticsearch搜索服务器前，还可以确定该至少一个实体对应的索引名称，从而根据索引名称在Elasticsearch搜索服务器中确定一个存储位置，或者说使用该索引名称对该至少一个实体和至少一个语料语义向量在Elasticsearch搜索服务器中存储的位置进行标记，由此，在后续使用时，可以通过该索引名称对该至少一个实体和至少一个语料语义向量形成的检索库进行快速定位。In addition, in this embodiment, before at least one entity and at least one corpus semantic vector are stored in the Elasticsearch search server, the index name corresponding to the at least one entity can also be determined, so as to determine a storage in the Elasticsearch search server according to the index name location, or use the index name to mark the location where the at least one entity and at least one corpus semantic vector are stored in the Elasticsearch search server, so that in subsequent use, the at least one entity and at least one entity can be used by the index name A retrieval library formed by corpus semantic vectors is used for fast positioning.

同时，在本实施方式中，还可以对生成的至少一个实体进行序列化，例如，通过结构数据序列化工具protobuf将每个实体序列化，并将数据、序列化后的实体和该实体序列对应的语料语义向量组成的实体对转为存储需要的格式，并存入Elasticsearch搜索服务器。其中，存储需要的格式图9所示。由于序列化后的实体可以压缩并减少存储空间，获得更快的传输速率，从而支持更大规模的存储搜索，并提高安全性。At the same time, in this embodiment, at least one generated entity can also be serialized, for example, each entity is serialized through the structural data serialization tool protobuf, and the data, the serialized entity and the entity sequence are associated Entity pairs composed of corpus semantic vectors are converted to the format required for storage and stored in the Elasticsearch search server. Wherein, the format required for storage is shown in FIG. 9 . Since serialized entities can be compressed and reduce storage space, a faster transmission rate can be obtained, thereby supporting larger-scale storage searches and improving security.

205：获取检索请求，检索请求包括待检索文本。205: Obtain a retrieval request, where the retrieval request includes the text to be retrieved.

206：将待检索文本输入语义提取模型，得到检索语义向量。206: Input the text to be retrieved into the semantic extraction model to obtain the retrieval semantic vector.

207：将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果。207: Transfer the retrieval semantic vector to the distributed search server for semantic retrieval, and obtain retrieval results.

在本实施方式中，可以通过gRPC服务将检索语义向量传入Elasticsearch搜索服务器，再通过向量索引算法，例如：HNSW(Hierarchical Navigable Small World)算法，对检索语义向量进行语义检索，得到检索结果序列，其中，检索结果序列中可以包括至少一个候选结果，且至少一个候选结果在检索结果序列中，按照每个候选结果与检索语义向量之间的相似度从大到小的顺序进行排列。最后，根据检索请求，确定检索数量n，从而将检索结果序列中，前n个候选结果作为检索结果。In this embodiment, the retrieval semantic vector can be transmitted to the Elasticsearch search server through the gRPC service, and then the retrieval semantic vector can be semantically retrieved through a vector index algorithm, such as the HNSW (Hierarchical Navigable Small World) algorithm, to obtain a sequence of retrieval results. Wherein, the retrieval result sequence may include at least one candidate result, and the at least one candidate result is arranged in descending order of the similarity between each candidate result and the retrieval semantic vector in the retrieval result sequence. Finally, according to the retrieval request, the retrieval quantity n is determined, so that the first n candidate results in the retrieval result sequence are used as the retrieval results.

具体而言，本实施方式提供的语义搜索方法结合Elasticsearch、gRPC、HNSW等，实现了大数据分布式计算。其中，结合HNSW算法，可以实现在千万级别数据上的毫秒级的准确搜索，此外，本实施方式中的语义搜索方法还可以结合tensorflow serving实现多神经网络编码器同时在线以及热更新，从而实现多业务多任务支持，并通过gRPC服务提高传输速率，提供高效服务。Specifically, the semantic search method provided in this embodiment combines Elasticsearch, gRPC, HNSW, etc. to realize distributed computing of big data. Among them, combined with the HNSW algorithm, accurate search at the millisecond level on tens of millions of data can be realized. In addition, the semantic search method in this embodiment can also be combined with tensorflow serving to realize simultaneous online and hot update of multiple neural network encoders, thereby realizing Multi-service and multi-task support, and increase the transmission rate through gRPC services to provide efficient services.

以下，将通过一个具体的示例，对结合Elasticsearch、gRPC、HNSW和tensorflow serving的语义搜索方法进行说明：Below, a specific example will be used to illustrate the semantic search method combined with Elasticsearch, gRPC, HNSW and tensorflow serving:

如图10所示，图10示出了一种结合Elasticsearch、gRPC、HNSW和tensorflow serving的语义搜索引擎的结构示意图。基于图10中的语义搜索引擎，搜索方法如下：As shown in Figure 10, Figure 10 shows a schematic structural diagram of a semantic search engine that combines Elasticsearch, gRPC, HNSW and tensorflow serving. Based on the semantic search engine in Figure 10, the search method is as follows:

首先，用户发出一个搜索请求，该请求可以包括要检索的文本、索引名称以及需检索相似文本数量n。然后，将待检索文本通过tensorflow serving加载的神经网络编码器编码后得到文本向量。再将文本向量、索引名称以及需检索相似文本数量n，通过gRPC服务传入Elasticsearch中。最后，通过HNSW算法进行检索，返回与该文本相似的n个文本及实体，以及相似分数，并按相似分数大小进行排序。First, the user sends a search request, which may include the text to be retrieved, the index name, and the number n of similar texts to be retrieved. Then, encode the text to be retrieved through the neural network encoder loaded by tensorflow serving to obtain the text vector. Then pass the text vector, index name, and the number n of similar texts to be retrieved into Elasticsearch through the gRPC service. Finally, retrieve through the HNSW algorithm, return n texts and entities similar to the text, and the similarity scores, and sort them according to the size of the similarity scores.

在本实施方式中，Elasticsearch是一种高效的分布式架构，而HNSW算法是一种高效准确的图搜索算法。gRPC服务是一种快速安全的服务框架，gRPC服务与http服务相比有更快的传输速度。基于此，Elasticsearch分布式架构通过分布式计算，将数据备份到集群中的各个节点，并结合HNSW算法，进行多机联合检索，充分发挥大数据搜索能力，最后将搜索的结果通过gRPC服务进行交互，可实现千万级别数据上毫秒级的准确搜索。In this embodiment, Elasticsearch is an efficient distributed architecture, and the HNSW algorithm is an efficient and accurate graph search algorithm. gRPC service is a fast and secure service framework. Compared with http service, gRPC service has faster transmission speed. Based on this, the Elasticsearch distributed architecture backs up data to each node in the cluster through distributed computing, and combined with the HNSW algorithm, performs multi-machine joint retrieval to give full play to the big data search capability, and finally interacts the search results through the gRPC service , which can realize accurate search in milliseconds on tens of millions of data.

由此，本实施方式还提出了一种语义搜索引擎，如图11所示，图11示出了该语义搜索引擎的总体结构图。其中，图11左边部分为语料库编码、实体生成存储部分，各业务用户可通过该部分将自身业务对应的语料库通过神经网络进行编码，并由实体生成器生成相应的实体，并将生成的实体以及其他数据发送至Elasticsearch进行存储，建立对应索引。图11右边部分为文本实时搜索部分，用户输入需要搜索的文本，以及需要搜索的相似文本个数和索引名称。通过神经网络编码，HNSW算法搜索以及gRPC服务快速从Elasticsearch中找到相似的文本及实体并返回搜索结果。本实施方式所提供的的语义搜索引擎实现了高精度、高准确率、且支持分布式大数据的多任务多业务的实时语义搜索。Therefore, this embodiment also proposes a semantic search engine, as shown in FIG. 11 , which shows an overall structural diagram of the semantic search engine. Among them, the left part of Figure 11 is the corpus encoding and entity generation storage part. Through this part, each business user can encode the corpus corresponding to his own business through the neural network, and the corresponding entity is generated by the entity generator, and the generated entity and Other data are sent to Elasticsearch for storage and corresponding indexes are established. The right part of Figure 11 is the real-time text search part, where the user inputs the text to be searched, the number of similar texts to be searched and the index name. Through neural network coding, HNSW algorithm search and gRPC service, it can quickly find similar text and entities from Elasticsearch and return the search results. The semantic search engine provided by this embodiment realizes real-time semantic search of multi-task and multi-service with high precision and high accuracy, and supports distributed big data.

综上所述，本申请所提供的基于神经网络的语义搜索方法中，通过语料库对组合神经网络进行训练优化，生成一种基于神经网络的编码模型对文本进行语义提取，从而实现对文本语义的精准理解，并进一步提升了后续语义搜索的精准度。同时，通过将训练语料进行实体化，生成相应的实体进行存储，实现不同实体支持不同任务，并能对新增的任务进行扩展支持，可实现单语义搜索引擎多任务多业务支持。并进一步对实体序列化，序列化后的实体可以压缩减少存储空间，获得更快的传输速率，从而支持更大规模的存储搜索，并提高安全性。最后，通过Elasticsearch搜索服务器进行语义搜索，实现了大数据分布式计算，进一步的提升了搜索速度，并拓宽了应用场景。To sum up, in the neural network-based semantic search method provided by this application, the combined neural network is trained and optimized through the corpus, and a neural network-based coding model is generated to extract the semantics of the text, thereby realizing the semantic search of the text. Accurate understanding, and further improve the accuracy of subsequent semantic search. At the same time, by materializing the training corpus and generating corresponding entities for storage, different entities can support different tasks, and can extend support for newly added tasks, enabling multi-task and multi-service support for a single-semantic search engine. And further serialize the entity, the serialized entity can be compressed to reduce storage space and obtain faster transmission rate, thus supporting larger-scale storage search and improving security. Finally, semantic search is carried out through the Elasticsearch search server, which realizes distributed computing of big data, further improves the search speed, and broadens the application scenarios.

参阅图12，图12为本申请实施方式提供的一种基于神经网络的语义搜索装置的功能模块组成框图。如图12所示，该基于神经网络的语义搜索装置1200包括：Referring to FIG. 12 , FIG. 12 is a block diagram of functional modules of a neural network-based semantic search device provided in an embodiment of the present application. As shown in Figure 12, the neural network-based semantic search device 1200 includes:

采集模块1201，用于获取语料库，其中，语料库中包括至少一个训练语料，且至少一个训练语料中的每个训练语料对应的业务类型相同；The acquisition module 1201 is configured to acquire a corpus, wherein the corpus includes at least one training corpus, and each of the at least one training corpus corresponds to the same business type;

训练模块1202，用于将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，组合神经网络由至少两个子神经网络组成，且至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，模型优化神经网络用于优化模型生成神经网络生成的模型；The training module 1202 is used to input at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and at least one of the at least two sub-neural networks includes A model generation neural network and at least one model optimization neural network, the model optimization neural network is used to optimize the model generated by the model generation neural network;

处理模块1203，用于将至少一个训练语料中的每个训练语料输入语义提取模型，得到至少一个语料语义向量，其中，至少一个语料语义向量和至少一个训练语料一一对应；The processing module 1203 is configured to input each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein at least one corpus semantic vector corresponds to at least one training corpus;

实体生成模块1204，用于根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将至少一个实体和至少一个语料语义向量存入分布式搜索服务器，其中，至少一个实体和至少一个训练语料一一对应；The entity generation module 1204 is used to generate entities according to each training corpus in at least one training corpus, obtain at least one entity, and store at least one entity and at least one corpus semantic vector in the distributed search server, wherein at least one entity One-to-one correspondence with at least one training corpus;

接收模块1205，用于获取检索请求，检索请求包括待检索文本；A receiving module 1205, configured to obtain a retrieval request, where the retrieval request includes the text to be retrieved;

检索模块1206，用于将待检索文本输入语义提取模型，得到检索语义向量，并将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果。The retrieval module 1206 is used to input the text to be retrieved into the semantic extraction model to obtain the retrieval semantic vector, and transfer the retrieval semantic vector to the distributed search server for semantic retrieval to obtain the retrieval result.

在本申请的实施方式中，至少两个子神经网络包括：老师神经网络、学生神经网络、卷积神经网络，基于此，在将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型方面，训练模块1202，具体用于：In the embodiment of the present application, at least two sub-neural networks include: a teacher neural network, a student neural network, and a convolutional neural network. Based on this, at least one training corpus is input into a preset combined neural network for training to obtain semantic In terms of extracting models, the training module 1202 is specifically used for:

将至少一个训练语料输入老师神经网络进行训练，得到第一模型；Inputting at least one training corpus into the teacher's neural network for training to obtain the first model;

根据学生神经网络，对第一模型进行蒸馏处理，得到第二模型；Distilling the first model according to the student neural network to obtain the second model;

将卷积神经网络与第二模型进行融合参数微调训练，得到第三模型，其中，第三模型为参数微调后的第二模型；Performing fusion parameter fine-tuning training on the convolutional neural network and the second model to obtain a third model, wherein the third model is the second model after parameter fine-tuning;

对第三模型进行白化处理，得到语义提取模型。Perform whitening processing on the third model to obtain a semantic extraction model.

在本申请的实施方式中，在将卷积神经网络与第二模型进行融合参数微调训练，得到第三模型方面，训练模块1202，具体用于：In the embodiment of the present application, the training module 1202 is specifically used for:

将第二模型的输出向量输入卷积神经网络的一维卷积层，得到至少一个第一向量；Inputting the output vector of the second model into the one-dimensional convolutional layer of the convolutional neural network to obtain at least one first vector;

对至少一个第一向量中的每个第一向量进行最大池化，并将最大池化后的每个第一向量进行拼接，得到第二向量；performing maximum pooling on each first vector in at least one first vector, and splicing each first vector after maximum pooling to obtain a second vector;

将第二向量输入卷积神经网络的全连接层，得到第一矩阵；Input the second vector into the fully connected layer of the convolutional neural network to obtain the first matrix;

通过回归模型对第一矩阵进行分类，并根据分类结果对第二模型进行优化调整，得到第三模型。The first matrix is classified by the regression model, and the second model is optimized and adjusted according to the classification result to obtain the third model.

在本申请的实施方式中，在对第三模型进行白化处理方面，训练模块1202，具体用于：In the embodiment of the present application, in terms of whitening the third model, the training module 1202 is specifically used for:

根据第三模型的输出向量x _i，确定第一参数u，其中，第三模型的输出向量x _i和第一参数u满足公式⑤： According to the output vector x _i of the third model, determine the first parameter u, wherein, the output vector x _i of the third model and the first parameter u satisfy the formula ⑤:

其中，N为第三模型的输出向量x _i中元素的数量； Wherein, N is the number of elements in the output vector x _i of the third model;

确定第三模型的输出向量x _i的协方差矩阵A； determining the covariance matrix A of the output vector x _i of the third model;

对第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C，其中，第三模型的输出向量的协方差矩阵A、第一特征矩阵B和第二特征矩阵C满足公式⑥：Singular value decomposition is performed on the covariance matrix A of the output vector of the third model to obtain the first characteristic matrix B and the second characteristic matrix C, wherein the covariance matrix A of the output vector of the third model, the first characteristic matrix B and The second characteristic matrix C satisfies the formula ⑥:

A＝BCB ^T.........⑥ A＝ ^BCBT ......⑥

根据第一特征矩阵B和第二特征矩阵C，确定第二参数W，其中，第二参数W、第一特征矩阵B和第二特征矩阵C满足公式⑦：According to the first characteristic matrix B and the second characteristic matrix C, determine the second parameter W, wherein, the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the formula ⑦:

根据第一参数u和第二参数W，对第三模型的输出向量x _i进行白化处理，得到白化向量

其中，第一参数u、第二参数W、第三模型的输出向量x _i和白化向量

满足公式⑧： According to the first parameter u and the second parameter W, the output vector x _i of the third model is whitened to obtain a whitening vector

Among them, the first parameter u, the second parameter W, the output vector x _i of the third model and the whitening vector

Satisfy the formula ⑧:

其中，白化向量

为第三模型的白化结果。 Among them, the whitening vector

is the whitening result of the third model.

在本申请的实施方式中，在根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体方面，实体生成模块1204，具体用于：In the embodiment of the present application, in terms of generating entities according to each training corpus in at least one training corpus to obtain at least one entity, the entity generation module 1204 is specifically used for:

根据每个训练语料的业务类型，获取实体模板；Obtain entity templates according to the business type of each training corpus;

根据实体模型的数据结构，对每个训练语料进行数据采集，得到至少一个实体数据；According to the data structure of the entity model, data collection is performed on each training corpus to obtain at least one entity data;

将至少一个实体数据中的每个实体数据填入实体模板中对应的位置，得到至少一个实体。Filling each entity data in the at least one entity data into a corresponding position in the entity template to obtain at least one entity.

在本申请的实施方式中，在将至少一个实体和至少一个语料语义向量存入分布式搜索服务器方面，实体生成模块1204，具体用于：In the embodiment of the present application, in terms of storing at least one entity and at least one corpus semantic vector in the distributed search server, the entity generation module 1204 is specifically used for:

根据至少一个实体中每个实体的业务类型，获取索引名称；Obtain an index name according to the business type of each entity in at least one entity;

通过结构数据序列化工具将每个实体进行序列化，得到至少一个实体序列，其中，至少一个实体序列和至少一个实体一一对应；Serialize each entity through a structural data serialization tool to obtain at least one entity sequence, wherein at least one entity sequence corresponds to at least one entity;

根据索引名称在分布式搜索服务器中确定至少一个实体序列和至少一个语料语义向量的存储位置；Determine the storage location of at least one entity sequence and at least one corpus semantic vector in the distributed search server according to the index name;

根据至少一个实体序列和至少一个语料语义向量之间的对应关系，将至少一个实体序列和至少一个语料语义向量一一对应，得到至少一个组实体对；According to the correspondence between at least one entity sequence and at least one corpus semantic vector, at least one entity sequence and at least one corpus semantic vector are one-to-one corresponded to obtain at least one group entity pair;

将至少一组实体对传入存储位置进行存储。Store at least one set of entity pairs to the incoming storage location.

在本申请的实施方式中，在将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果方面，检索模块1206，具体用于：In the embodiment of the present application, the retrieval module 1206 is specifically used to:

通过向量索引算法，对检索语义向量进行语义检索，得到检索结果序列，其中，检索结果序列中包括至少一个候选结果，且至少一个候选结果在检索结果序列中，按照每个候选结果与检索语义向量之间的相似度从大到小的顺序进行排列；Perform semantic retrieval on the retrieval semantic vector through the vector indexing algorithm to obtain a retrieval result sequence, wherein the retrieval result sequence includes at least one candidate result, and at least one candidate result is in the retrieval result sequence, according to each candidate result and the retrieval semantic vector The similarity between them is arranged in order from large to small;

根据检索请求，确定检索数量n；Determine the retrieval quantity n according to the retrieval request;

将检索结果序列中，前n个候选结果作为检索结果。In the search result sequence, the first n candidate results are taken as the search results.

参阅图13，图13为本申请实施方式提供的一种电子设备的结构示意图。如图13所示，电子设备1300包括收发器1301、处理器1302和存储器1303。它们之间通过总线1304连接。存储器1303用于存储计算机程序和数据，并可以将存储器1303存储的数据传输给处理器1302。Referring to FIG. 13 , FIG. 13 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. As shown in FIG. 13 , an electronic device 1300 includes a transceiver 1301 , a processor 1302 and a memory 1303 . They are connected through a bus 1304 . The memory 1303 is used to store computer programs and data, and can transmit the data stored in the memory 1303 to the processor 1302 .

处理器1302用于读取存储器1303中的计算机程序执行以下操作：The processor 1302 is used to read the computer program in the memory 1303 to perform the following operations:

在本申请的实施方式中，至少两个子神经网络包括：老师神经网络、学生神经网络、卷积神经网络，基于此，在将至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, at least two sub-neural networks include: a teacher neural network, a student neural network, and a convolutional neural network. Based on this, at least one training corpus is input into a preset combined neural network for training to obtain semantic In terms of extracting models, the processor 1302 is specifically configured to perform the following operations:

在本申请的实施方式中，在将卷积神经网络与第二模型进行融合参数微调训练，得到第三模型方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, in terms of fine-tuning the fusion parameters of the convolutional neural network and the second model to obtain the third model, the processor 1302 is specifically configured to perform the following operations:

在本申请的实施方式中，在对第三模型进行白化处理方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, in terms of whitening the third model, the processor 1302 is specifically configured to perform the following operations:

根据第三模型的输出向量x _i，确定第一参数u，其中，第三模型的输出向量x _i和第一参数u满足公式⑨： According to the output vector _xi of the third model, the first parameter u is determined, wherein the output vector _xi of the third model and the first parameter u satisfy the formula ⑨:

对第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C，其中，第三模型的输出向量的协方差矩阵A、第一特征矩阵B和第二特征矩阵C满足公式⑩：Singular value decomposition is performed on the covariance matrix A of the output vector of the third model to obtain the first characteristic matrix B and the second characteristic matrix C, wherein the covariance matrix A of the output vector of the third model, the first characteristic matrix B and The second characteristic matrix C satisfies the formula ⑩:

A＝BCB ^T.........⑩ A＝ ^BCBT ......⑩

根据第一特征矩阵B和第二特征矩阵C，确定第二参数W，其中，第二参数W、第一特征矩阵B和第二特征矩阵C满足公式

According to the first characteristic matrix B and the second characteristic matrix C, determine the second parameter W, wherein, the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the formula

满足公式

According to the first parameter u and the second parameter W, the output vector x _i of the third model is whitened to obtain a whitening vector

satisfy the formula

其中，白化向量

为第三模型的白化结果。 Among them, the whitening vector

is the whitening result of the third model.

在本申请的实施方式中，在根据至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, in terms of generating entities according to each training corpus in at least one training corpus to obtain at least one entity, the processor 1302 is specifically configured to perform the following operations:

在本申请的实施方式中，在将至少一个实体和至少一个语料语义向量存入分布式搜索服务器方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, in terms of storing at least one entity and at least one corpus semantic vector in the distributed search server, the processor 1302 is specifically configured to perform the following operations:

将至少一个组实体对传入存储位置进行存储。Store at least one group entity pair to the incoming storage location.

在本申请的实施方式中，在将检索语义向量传入分布式搜索服务器进行语义检索，得到检索结果方面，处理器1302，具体用于执行以下操作：In the embodiment of the present application, the processor 1302 is specifically configured to perform the following operations in terms of transferring the retrieval semantic vector to the distributed search server for semantic retrieval and obtaining retrieval results:

应理解，本申请中的基于神经网络的语义搜索装置可以包括智能手机(如Android手机、iOS手机、Windows Phone手机等)、平板电脑、掌上电脑、笔记本电脑、移动互联网设备MID(Mobile Internet Devices，简称：MID)、机器人或穿戴式设备等。上述基于神经网络的语义搜索装置仅是举例，而非穷举，包含但不限于上述基于神经网络的语义搜索装置。在实际应用中，上述基于神经网络的语义搜索装置还可以包括：智能车载终端、计算机设备等等。It should be understood that the semantic search device based on the neural network in the present application can include smart phones (such as Android mobile phones, iOS mobile phones, Windows Phone mobile phones, etc.), tablet computers, palmtop computers, notebook computers, mobile Internet equipment MID (Mobile Internet Devices, Abbreviation: MID), robot or wearable device, etc. The above-mentioned semantic search device based on neural network is only an example, not exhaustive, including but not limited to the above-mentioned semantic search device based on neural network. In practical applications, the above-mentioned neural network-based semantic search device may also include: intelligent vehicle-mounted terminals, computer equipment, and the like.

通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到本申请可借助软件结合硬件平台的方式来实现。基于这样的理解，本申请的技术方案对背景技术做出贡献的全部或者部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施方式或者实施方式的某些部分所述的方法。Through the above description of the implementation manners, those skilled in the art can clearly understand that the present application can be implemented by means of software combined with a hardware platform. Based on this understanding, all or part of the contribution made by the technical solution of the present application to the background technology can be embodied in the form of software products, and the computer software products can be stored in storage media, such as ROM/RAM, magnetic disks, optical disks, etc. , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of this application.

因此，本申请实施方式还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行以实现如上述方法实施方式中记载的任何一种基于神经网络的语义搜索方法的部分或全部步骤。例如，所述存储介质可以包括硬盘、软盘、光盘、磁带、磁盘、优盘、闪存等。Therefore, the embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement any one of the above-mentioned method implementations based on Some or all of the steps of the semantic search method for neural networks. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a magnetic tape, a magnetic disk, a flash memory, and the like.

可选的，本申请涉及的存储介质如计算机可读存储介质可以是非易失性的，也可以是易失性的。Optionally, the storage medium involved in this application, such as a computer-readable storage medium, may be non-volatile or volatile.

本申请实施方式还提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行如上述方法实施方式中记载的任何一种基于神经网络的语义搜索方法的部分或全部步骤。The embodiments of the present application also provide a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause the computer to execute the method as described in the above-mentioned method embodiment. Some or all of the steps of any neural network-based semantic search method.

在上述实施方式中，对各个实施方式的描述都各有侧重，某个实施方式中没有详述的部分，可以参见其他实施方式的相关描述。In the foregoing implementation manners, the descriptions of each implementation manner have their own emphases, and for parts not described in detail in a certain implementation manner, reference may be made to relevant descriptions of other implementation manners.

在本申请所提供的几个实施方式中，应该理解到，所揭露的装置，可通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性或其它的形式。In the several implementation manners provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device implementation described above is only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical or other forms.

所述集成的单元如果以软件程序模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储器中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储器中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施方式所述方法的全部或部分步骤。而前述的存储器包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。The integrated units may be stored in a computer-readable memory if implemented in the form of a software program module and sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory. Several instructions are included to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned memory includes: various media that can store program codes such as U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk.

本领域普通技术人员可以理解上述实施方式的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储器中，存储器可以包括：闪存盘、只读存储器(英文：Read-Only Memory，简称：ROM)、随机存取器(英文：Random Access Memory，简称：RAM)、磁盘或光盘等。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, referred to as: ROM), random access device (English: Random Access Memory, referred to as: RAM), magnetic disk or optical disc, etc.

以上对本申请实施方式进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施方式的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。The implementation of the present application has been introduced in detail above, and the principle and implementation of the application have been explained by using specific examples in this paper. The description of the above implementation is only used to help understand the method and core idea of the application; at the same time, for Those skilled in the art will have changes in specific implementation methods and application scopes based on the ideas of the present application. In summary, the contents of this specification should not be construed as limiting the present application.

Claims

一种基于神经网络的语义搜索方法，其中，所述方法包括：A neural network-based semantic search method, wherein the method includes:

获取语料库，其中，所述语料库中包括至少一个训练语料，且所述至少一个训练语料中的每个训练语料对应的业务类型相同；Acquiring a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，所述组合神经网络由至少两个子神经网络组成，且所述至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，所述模型优化神经网络用于优化所述模型生成神经网络生成的模型；The at least one training corpus is input into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one a model-generating neural network and at least one model-optimizing neural network for optimizing a model generated by the model-generating neural network;

将所述至少一个训练语料中的每个训练语料输入所述语义提取模型，得到至少一个语料语义向量，其中，所述至少一个语料语义向量和所述至少一个训练语料一一对应；Inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein the at least one corpus semantic vector corresponds to the at least one training corpus;

根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，其中，所述至少一个实体和所述至少一个训练语料一一对应；Entity generation is performed according to each training corpus in the at least one training corpus to obtain at least one entity, and the at least one entity and the at least one corpus semantic vector are stored in a distributed search server, wherein the at least one There is a one-to-one correspondence between the entity and the at least one training corpus;

获取搜索请求，所述搜索请求包括待搜索文本；Obtain a search request, the search request includes the text to be searched;

将所述待搜索文本输入所述语义提取模型，得到搜索语义向量；Inputting the text to be searched into the semantic extraction model to obtain a search semantic vector;

将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果。The search semantic vector is transmitted to the distributed search server to perform semantic search, and search results are obtained.
根据权利要求1所述的方法，其中，The method according to claim 1, wherein,

所述至少两个子神经网络包括：老师神经网络、学生神经网络、卷积神经网络；The at least two sub-neural networks include: a teacher neural network, a student neural network, and a convolutional neural network;

所述将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，包括：The at least one training corpus is input into a preset combined neural network for training to obtain a semantic extraction model, including:

将所述至少一个训练语料输入所述老师神经网络进行训练，得到第一模型；Inputting the at least one training corpus into the teacher's neural network for training to obtain a first model;

根据所述学生神经网络，对所述第一模型进行蒸馏处理，得到第二模型；Distilling the first model according to the student neural network to obtain a second model;

将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，其中，所述第三模型为参数微调后的第二模型；Carrying out fusion parameter fine-tuning training of the convolutional neural network and the second model to obtain a third model, wherein the third model is the second model after parameter fine-tuning;

对所述第三模型进行白化处理，得到所述语义提取模型。Perform whitening processing on the third model to obtain the semantic extraction model.
根据权利要求2所述的方法，其中，所述将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，包括：The method according to claim 2, wherein said performing fusion parameter fine-tuning training on said convolutional neural network and said second model to obtain a third model, comprising:

将所述第二模型的输出向量输入所述卷积神经网络的一维卷积层，得到至少一个第一向量；Inputting the output vector of the second model into the one-dimensional convolutional layer of the convolutional neural network to obtain at least one first vector;

对所述至少一个第一向量中的每个第一向量进行最大池化，并将最大池化后的所述每个第一向量进行拼接，得到第二向量；performing maximum pooling on each first vector in the at least one first vector, and splicing each first vector after maximum pooling to obtain a second vector;

将所述第二向量输入所述卷积神经网络的全连接层，得到第一矩阵；Input the second vector into the fully connected layer of the convolutional neural network to obtain the first matrix;

通过回归模型对所述第一矩阵进行分类，并根据分类结果对所述第二模型进行优化调整，得到所述第三模型。The first matrix is classified by a regression model, and the second model is optimized and adjusted according to the classification result to obtain the third model.
根据权利要求2所述的方法，其中，所述对所述第三模型进行白化处理，包括：The method according to claim 2, wherein said whitening the third model comprises:

根据所述第三模型的输出向量x _i，确定第一参数u，其中，所述第三模型的输出向量x _i和所述第一参数u满足以下公式： According to the output vector x _i of the third model, the first parameter u is determined, wherein the output vector x _i of the third model and the first parameter u satisfy the following formula:

其中，所述N为所述第三模型的输出向量x _i中元素的数量； Wherein, the N is the number of elements in the output vector x _i of the third model;

确定所述第三模型的输出向量x _i的协方差矩阵A； determining the covariance matrix A of the output vector x _i of said third model;

对所述第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C，其中，所述第三模型的输出向量的协方差矩阵A、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：Singular value decomposition is performed on the covariance matrix A of the output vector of the third model to obtain the first characteristic matrix B and the second characteristic matrix C, wherein the covariance matrix A of the output vector of the third model, the The first characteristic matrix B and the second characteristic matrix C satisfy the following formula:

A＝BCB ^T A=BCB ^T

其中，B ^T表示所述第一特征矩阵B的转置矩阵； Wherein, ^BT represents the transpose matrix of the first feature matrix B;

根据所述第一特征矩阵B和所述第二特征矩阵C，确定第二参数W，其中，所述第二参数W、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：According to the first characteristic matrix B and the second characteristic matrix C, determine the second parameter W, wherein the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the following formula :

根据所述第一参数u和所述第二参数W，对所述第三模型的输出向量x _i进行白化处理，得到白化向量
其中，所述第一参数u、所述第二参数W、所述第三模型的输出向量x _i和所述白化向量
满足以下公式： According to the first parameter u and the second parameter W, the output vector x _i of the third model is whitened to obtain a whitened vector
Wherein, the first parameter u, the second parameter W, the output vector x _i of the third model and the whitening vector
satisfy the following formula:

其中，所述白化向量
为所述第三模型的白化结果。 Among them, the whitening vector
is the whitening result of the third model.
根据权利要求1所述的方法，其中，所述根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，包括：The method according to claim 1, wherein said performing entity generation according to each training corpus in said at least one training corpus to obtain at least one entity, comprising:

根据所述每个训练语料的业务类型，获取实体模板；Obtain an entity template according to the business type of each training corpus;

根据所述实体模型的数据结构，对所述每个训练语料进行数据采集，得到至少一个实体数据；According to the data structure of the entity model, data collection is performed on each of the training corpus to obtain at least one entity data;

将所述至少一个实体数据中的每个实体数据填入所述实体模板中对应的位置，得到所述至少一个实体。Filling each entity data in the at least one entity data into a corresponding position in the entity template to obtain the at least one entity.
根据权利要求1所述的方法，其中，所述将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，包括：The method according to claim 1, wherein said storing said at least one entity and said at least one corpus semantic vector into a distributed search server comprises:

根据所述至少一个实体中每个实体的业务类型，获取索引名称；Obtain an index name according to the business type of each entity in the at least one entity;

通过结构数据序列化工具将所述每个实体进行序列化，得到至少一个实体序列，其中，所述至少一个实体序列和所述至少一个实体一一对应；Serialize each entity by using a structural data serialization tool to obtain at least one entity sequence, wherein the at least one entity sequence corresponds to the at least one entity;

根据所述索引名称在所述分布式搜索服务器中确定所述至少一个实体序列和所述至少一个语料语义向量的存储位置；determining the storage location of the at least one entity sequence and the at least one corpus semantic vector in the distributed search server according to the index name;

根据所述至少一个实体序列和所述至少一个语料语义向量之间的对应关系，将所述至少一个实体序列和所述至少一个语料语义向量一一对应，得到至少一个组实体对；According to the correspondence between the at least one entity sequence and the at least one corpus semantic vector, the at least one entity sequence and the at least one corpus semantic vector are in one-to-one correspondence to obtain at least one group entity pair;

将所述至少一个组实体对传入所述存储位置进行存储。Passing the at least one group entity pair into the storage location for storage.
根据权利要求1-6中任意一项所述的方法，其中，所述将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果，包括：The method according to any one of claims 1-6, wherein said transferring said search semantic vector into said distributed search server to perform semantic search and obtaining search results includes:

通过向量索引算法，对所述搜索语义向量进行语义搜索，得到搜索结果序列，其中，所述搜索结果序列中包括至少一个候选结果，且所述至少一个候选结果在所述搜索结果序列中，按照每个候选结果与所述搜索语义向量之间的相似度从大到小的顺序进行排列；Perform a semantic search on the search semantic vector through a vector indexing algorithm to obtain a search result sequence, wherein the search result sequence includes at least one candidate result, and the at least one candidate result is in the search result sequence, according to The similarity between each candidate result and the search semantic vector is arranged in descending order;

根据所述搜索请求，确定搜索数量n；According to the search request, determine the search number n;

将所述搜索结果序列中，前n个所述候选结果作为所述搜索结果。Taking the first n candidate results in the search result sequence as the search results.
一种基于神经网络的语义搜索装置，其中，所述装置包括：A neural network-based semantic search device, wherein the device includes:

采集模块，用于获取语料库，其中，所述语料库中包括至少一个训练语料，且所述至少一个训练语料中的每个训练语料对应的业务类型相同；An acquisition module, configured to acquire a corpus, wherein the corpus includes at least one training corpus, and each of the at least one training corpus corresponds to the same business type;

训练模块，用于将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，所述组合神经网络由至少两个子神经网络组成，且所述至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，所述模型优化神经网络用于优化所述模型生成神经网络生成的模型；A training module, configured to input the at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks The network includes at least one model generation neural network and at least one model optimization neural network, and the model optimization neural network is used to optimize the model generated by the model generation neural network;

处理模块，用于将所述至少一个训练语料中的每个训练语料输入所述语义提取模型，得到至少一个语料语义向量，其中，所述至少一个语料语义向量和所述至少一个训练语料一一对应；A processing module, configured to input each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein the at least one corpus semantic vector and the at least one training corpus are one by one correspond;

实体生成模块，用于根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，其中，所述至少一个实体和所述至少一个训练语料一一对应；An entity generation module, configured to generate entities according to each training corpus in the at least one training corpus, obtain at least one entity, and store the at least one entity and the at least one corpus semantic vector in a distributed search server, Wherein, the at least one entity has a one-to-one correspondence with the at least one training corpus;

接收模块，用于获取搜索请求，所述搜索请求包括待搜索文本；A receiving module, configured to obtain a search request, the search request includes text to be searched;

搜索模块，用于将所述待搜索文本输入所述语义提取模型，得到搜索语义向量，并将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果。A search module is configured to input the text to be searched into the semantic extraction model to obtain a search semantic vector, and transfer the search semantic vector to the distributed search server for semantic search to obtain search results.
一种电子设备，其中，包括处理器、存储器、通信接口以及一个或多个程序，其中，所述一个或多个程序被存储在所述存储器中，并且被配置由所述处理器执行以实现以下方法：An electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor to implement The following methods:

获取语料库，其中，所述语料库中包括至少一个训练语料，且所述至少一个训练语料中的每个训练语料对应的业务类型相同；Acquiring a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，所述组合神经网络由至少两个子神经网络组成，且所述至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，所述模型优化神经网络用于优化所述模型生成神经网络生成的模型；The at least one training corpus is input into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one a model-generating neural network and at least one model-optimizing neural network for optimizing a model generated by the model-generating neural network;

将所述至少一个训练语料中的每个训练语料输入所述语义提取模型，得到至少一个语料语义向量，其中，所述至少一个语料语义向量和所述至少一个训练语料一一对应；Inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein the at least one corpus semantic vector corresponds to the at least one training corpus;

根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，其中，所述至少一个实体和所述至少一个训练语料一一对应；Entity generation is performed according to each training corpus in the at least one training corpus to obtain at least one entity, and the at least one entity and the at least one corpus semantic vector are stored in a distributed search server, wherein the at least one There is a one-to-one correspondence between the entity and the at least one training corpus;

获取搜索请求，所述搜索请求包括待搜索文本；Obtain a search request, the search request includes the text to be searched;

将所述待搜索文本输入所述语义提取模型，得到搜索语义向量；Inputting the text to be searched into the semantic extraction model to obtain a search semantic vector;

将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果。The search semantic vector is transmitted to the distributed search server to perform semantic search, and search results are obtained.
根据权利要求9所述的电子设备，其中，The electronic device according to claim 9, wherein,

所述至少两个子神经网络包括：老师神经网络、学生神经网络、卷积神经网络；The at least two sub-neural networks include: a teacher neural network, a student neural network, and a convolutional neural network;

执行所述将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，包括：Executing said inputting said at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, including:

将所述至少一个训练语料输入所述老师神经网络进行训练，得到第一模型；Inputting the at least one training corpus into the teacher's neural network for training to obtain a first model;

根据所述学生神经网络，对所述第一模型进行蒸馏处理，得到第二模型；Distilling the first model according to the student neural network to obtain a second model;

将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，其中，所述第三模型为参数微调后的第二模型；Carrying out fusion parameter fine-tuning training of the convolutional neural network and the second model to obtain a third model, wherein the third model is the second model after parameter fine-tuning;

对所述第三模型进行白化处理，得到所述语义提取模型。Perform whitening processing on the third model to obtain the semantic extraction model.
根据权利要求10所述的电子设备，其中，执行所述将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，包括：The electronic device according to claim 10, wherein performing the fusion parameter fine-tuning training of the convolutional neural network and the second model to obtain a third model includes:

将所述第二模型的输出向量输入所述卷积神经网络的一维卷积层，得到至少一个第一向量；Inputting the output vector of the second model into the one-dimensional convolutional layer of the convolutional neural network to obtain at least one first vector;

对所述至少一个第一向量中的每个第一向量进行最大池化，并将最大池化后的所述每个第一向量进行拼接，得到第二向量；performing maximum pooling on each first vector in the at least one first vector, and splicing each first vector after maximum pooling to obtain a second vector;

将所述第二向量输入所述卷积神经网络的全连接层，得到第一矩阵；Input the second vector into the fully connected layer of the convolutional neural network to obtain the first matrix;

通过回归模型对所述第一矩阵进行分类，并根据分类结果对所述第二模型进行优化调整，得到所述第三模型。The first matrix is classified by a regression model, and the second model is optimized and adjusted according to the classification result to obtain the third model.
根据权利要求10所述的电子设备，其中，执行所述对所述第三模型进行白化处理，包括：The electronic device according to claim 10, wherein performing the whitening process on the third model comprises:

根据所述第三模型的输出向量x _i，确定第一参数u，其中，所述第三模型的输出向量x _i和所述第一参数u满足以下公式： According to the output vector x _i of the third model, the first parameter u is determined, wherein the output vector x _i of the third model and the first parameter u satisfy the following formula:

其中，所述N为所述第三模型的输出向量x _i中元素的数量； Wherein, the N is the number of elements in the output vector x _i of the third model;

确定所述第三模型的输出向量x _i的协方差矩阵A； determining the covariance matrix A of the output vector x _i of said third model;

对所述第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C，其中，所述第三模型的输出向量的协方差矩阵A、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：Singular value decomposition is performed on the covariance matrix A of the output vector of the third model to obtain the first characteristic matrix B and the second characteristic matrix C, wherein the covariance matrix A of the output vector of the third model, the The first characteristic matrix B and the second characteristic matrix C satisfy the following formula:

A＝BCB ^T A=BCB ^T

其中，B ^T表示所述第一特征矩阵B的转置矩阵； Wherein, ^BT represents the transpose matrix of the first feature matrix B;

根据所述第一特征矩阵B和所述第二特征矩阵C，确定第二参数W，其中，所述第二参数W、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：According to the first characteristic matrix B and the second characteristic matrix C, determine the second parameter W, wherein the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the following formula :

根据所述第一参数u和所述第二参数W，对所述第三模型的输出向量x _i进行白化处理，得到白化向量
其中，所述第一参数u、所述第二参数W、所述第三模型的输出向量x _i和所述白化向量
满足以下公式： According to the first parameter u and the second parameter W, the output vector x _i of the third model is whitened to obtain a whitened vector
Wherein, the first parameter u, the second parameter W, the output vector x _i of the third model and the whitening vector
satisfy the following formula:

其中，所述白化向量
为所述第三模型的白化结果。 Among them, the whitening vector
is the whitening result of the third model.
根据权利要求9所述的电子设备，其中，执行所述根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，包括：The electronic device according to claim 9, wherein performing the entity generation according to each training corpus in the at least one training corpus to obtain at least one entity comprises:

根据所述每个训练语料的业务类型，获取实体模板；Obtain an entity template according to the business type of each training corpus;

根据所述实体模型的数据结构，对所述每个训练语料进行数据采集，得到至少一个实体数据；According to the data structure of the entity model, data collection is performed on each of the training corpus to obtain at least one entity data;

将所述至少一个实体数据中的每个实体数据填入所述实体模板中对应的位置，得到所述至少一个实体。Filling each entity data in the at least one entity data into a corresponding position in the entity template to obtain the at least one entity.
根据权利要求9所述的电子设备，其中，执行所述将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，包括：The electronic device according to claim 9, wherein performing the storing the at least one entity and the at least one corpus semantic vector into a distributed search server comprises:

根据所述至少一个实体中每个实体的业务类型，获取索引名称；Obtain an index name according to the business type of each entity in the at least one entity;

通过结构数据序列化工具将所述每个实体进行序列化，得到至少一个实体序列，其中，所述至少一个实体序列和所述至少一个实体一一对应；Serialize each entity by using a structural data serialization tool to obtain at least one entity sequence, wherein the at least one entity sequence corresponds to the at least one entity;

根据所述索引名称在所述分布式搜索服务器中确定所述至少一个实体序列和所述至少一个语料语义向量的存储位置；determining the storage location of the at least one entity sequence and the at least one corpus semantic vector in the distributed search server according to the index name;

根据所述至少一个实体序列和所述至少一个语料语义向量之间的对应关系，将所述至少一个实体序列和所述至少一个语料语义向量一一对应，得到至少一个组实体对；According to the correspondence between the at least one entity sequence and the at least one corpus semantic vector, the at least one entity sequence and the at least one corpus semantic vector are in one-to-one correspondence to obtain at least one group entity pair;

将所述至少一个组实体对传入所述存储位置进行存储。Passing the at least one group entity pair into the storage location for storage.
一种计算机可读存储介质，其中，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行以实现以下方法：A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following method:

获取语料库，其中，所述语料库中包括至少一个训练语料，且所述至少一个训练语料中的每个训练语料对应的业务类型相同；Acquiring a corpus, wherein the corpus includes at least one training corpus, and each training corpus in the at least one training corpus corresponds to the same business type;

将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，其中，所述组合神经网络由至少两个子神经网络组成，且所述至少两个子神经网络中包括至少一个模型生成神经网络和至少一个模型优化神经网络，所述模型优化神经网络用于优化所述模型生成神经网络生成的模型；The at least one training corpus is input into a preset combined neural network for training to obtain a semantic extraction model, wherein the combined neural network is composed of at least two sub-neural networks, and the at least two sub-neural networks include at least one a model-generating neural network and at least one model-optimizing neural network for optimizing a model generated by the model-generating neural network;

将所述至少一个训练语料中的每个训练语料输入所述语义提取模型，得到至少一个语料语义向量，其中，所述至少一个语料语义向量和所述至少一个训练语料一一对应；Inputting each training corpus in the at least one training corpus into the semantic extraction model to obtain at least one corpus semantic vector, wherein the at least one corpus semantic vector corresponds to the at least one training corpus;

根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，并将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，其中，所述至少一个实体和所述至少一个训练语料一一对应；Entity generation is performed according to each training corpus in the at least one training corpus to obtain at least one entity, and the at least one entity and the at least one corpus semantic vector are stored in a distributed search server, wherein the at least one There is a one-to-one correspondence between the entity and the at least one training corpus;

获取搜索请求，所述搜索请求包括待搜索文本；Obtain a search request, the search request includes the text to be searched;

将所述待搜索文本输入所述语义提取模型，得到搜索语义向量；Inputting the text to be searched into the semantic extraction model to obtain a search semantic vector;

将所述搜索语义向量传入所述分布式搜索服务器进行语义搜索，得到搜索结果。The search semantic vector is transmitted to the distributed search server to perform semantic search, and search results are obtained.
根据权利要求15所述的计算机可读存储介质，其中，The computer readable storage medium of claim 15, wherein:

所述至少两个子神经网络包括：老师神经网络、学生神经网络、卷积神经网络；The at least two sub-neural networks include: a teacher neural network, a student neural network, and a convolutional neural network;

执行所述将所述至少一个训练语料输入预先设置的组合神经网络中进行训练，得到语义提取模型，包括：Executing said inputting said at least one training corpus into a preset combined neural network for training to obtain a semantic extraction model, including:

将所述至少一个训练语料输入所述老师神经网络进行训练，得到第一模型；Inputting the at least one training corpus into the teacher's neural network for training to obtain a first model;

根据所述学生神经网络，对所述第一模型进行蒸馏处理，得到第二模型；Distilling the first model according to the student neural network to obtain a second model;

将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，其中，所述第三模型为参数微调后的第二模型；Carrying out fusion parameter fine-tuning training of the convolutional neural network and the second model to obtain a third model, wherein the third model is the second model after parameter fine-tuning;

对所述第三模型进行白化处理，得到所述语义提取模型。Perform whitening processing on the third model to obtain the semantic extraction model.
根据权利要求16所述的计算机可读存储介质，其中，执行所述将所述卷积神经网络与所述第二模型进行融合参数微调训练，得到第三模型，包括：The computer-readable storage medium according to claim 16, wherein performing the fusion parameter fine-tuning training of the convolutional neural network and the second model to obtain a third model includes:

将所述第二模型的输出向量输入所述卷积神经网络的一维卷积层，得到至少一个第一向量；Inputting the output vector of the second model into the one-dimensional convolutional layer of the convolutional neural network to obtain at least one first vector;

对所述至少一个第一向量中的每个第一向量进行最大池化，并将最大池化后的所述每个第一向量进行拼接，得到第二向量；performing maximum pooling on each first vector in the at least one first vector, and splicing each first vector after maximum pooling to obtain a second vector;

将所述第二向量输入所述卷积神经网络的全连接层，得到第一矩阵；Input the second vector into the fully connected layer of the convolutional neural network to obtain the first matrix;

通过回归模型对所述第一矩阵进行分类，并根据分类结果对所述第二模型进行优化调整，得到所述第三模型。The first matrix is classified by a regression model, and the second model is optimized and adjusted according to the classification result to obtain the third model.
根据权利要求16所述的计算机可读存储介质，其中，执行所述对所述第三模型进行白化处理，包括：The computer-readable storage medium according to claim 16, wherein performing the whitening process on the third model comprises:

根据所述第三模型的输出向量x _i，确定第一参数u，其中，所述第三模型的输出向量x _i和所述第一参数u满足以下公式： According to the output vector x _i of the third model, the first parameter u is determined, wherein the output vector x _i of the third model and the first parameter u satisfy the following formula:

其中，所述N为所述第三模型的输出向量x _i中元素的数量； Wherein, the N is the number of elements in the output vector x _i of the third model;

确定所述第三模型的输出向量x _i的协方差矩阵A； determining the covariance matrix A of the output vector x _i of said third model;

对所述第三模型的输出向量的协方差矩阵A进行奇异值分解，得到第一特征矩阵B和第二特征矩阵C，其中，所述第三模型的输出向量的协方差矩阵A、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：Singular value decomposition is performed on the covariance matrix A of the output vector of the third model to obtain the first characteristic matrix B and the second characteristic matrix C, wherein the covariance matrix A of the output vector of the third model, the The first characteristic matrix B and the second characteristic matrix C satisfy the following formula:

A＝BCB ^T A=BCB ^T

其中，B ^T表示所述第一特征矩阵B的转置矩阵； Wherein, ^BT represents the transpose matrix of the first feature matrix B;

根据所述第一特征矩阵B和所述第二特征矩阵C，确定第二参数W，其中，所述第二参数W、所述第一特征矩阵B和所述第二特征矩阵C满足以下公式：According to the first characteristic matrix B and the second characteristic matrix C, determine the second parameter W, wherein the second parameter W, the first characteristic matrix B and the second characteristic matrix C satisfy the following formula :

根据所述第一参数u和所述第二参数W，对所述第三模型的输出向量x _i进行白化处理，得到白化向量
其中，所述第一参数u、所述第二参数W、所述第三模型的输出向量x _i和所述白化向量
满足以下公式： According to the first parameter u and the second parameter W, the output vector x _i of the third model is whitened to obtain a whitened vector
Wherein, the first parameter u, the second parameter W, the output vector x _i of the third model and the whitening vector
satisfy the following formula:

其中，所述白化向量
为所述第三模型的白化结果。 Among them, the whitening vector
is the whitening result of the third model.
根据权利要求15所述的计算机可读存储介质，其中，执行所述根据所述至少一个训练语料中的每个训练语料进行实体生成，得到至少一个实体，包括：The computer-readable storage medium according to claim 15, wherein performing the entity generation according to each training corpus in the at least one training corpus to obtain at least one entity comprises:

根据所述每个训练语料的业务类型，获取实体模板；Obtain an entity template according to the business type of each training corpus;

根据所述实体模型的数据结构，对所述每个训练语料进行数据采集，得到至少一个实体数据；According to the data structure of the entity model, data collection is performed on each of the training corpus to obtain at least one entity data;

将所述至少一个实体数据中的每个实体数据填入所述实体模板中对应的位置，得到所述至少一个实体。Filling each entity data in the at least one entity data into a corresponding position in the entity template to obtain the at least one entity.
根据权利要求15所述的计算机可读存储介质，其中，执行所述将所述至少一个实体和所述至少一个语料语义向量存入分布式搜索服务器，包括：The computer-readable storage medium according to claim 15, wherein performing said storing said at least one entity and said at least one corpus semantic vector into a distributed search server comprises:

根据所述至少一个实体中每个实体的业务类型，获取索引名称；Obtain an index name according to the business type of each entity in the at least one entity;

通过结构数据序列化工具将所述每个实体进行序列化，得到至少一个实体序列，其中，所述至少一个实体序列和所述至少一个实体一一对应；Serialize each entity by using a structural data serialization tool to obtain at least one entity sequence, wherein the at least one entity sequence corresponds to the at least one entity;

根据所述索引名称在所述分布式搜索服务器中确定所述至少一个实体序列和所述至少一个语料语义向量的存储位置；determining the storage location of the at least one entity sequence and the at least one corpus semantic vector in the distributed search server according to the index name;

根据所述至少一个实体序列和所述至少一个语料语义向量之间的对应关系，将所述至少一个实体序列和所述至少一个语料语义向量一一对应，得到至少一个组实体对；According to the correspondence between the at least one entity sequence and the at least one corpus semantic vector, the at least one entity sequence and the at least one corpus semantic vector are in one-to-one correspondence to obtain at least one group entity pair;

将所述至少一个组实体对传入所述存储位置进行存储。Passing the at least one group entity pair into the storage location for storage.