WO2016110125A1

WO2016110125A1 - Hash method for high dimension vector, and vector quantization method and device

Info

Publication number: WO2016110125A1
Application number: PCT/CN2015/089607
Authority: WO
Inventors: 段凌宇; 王哲; 王潇放; 黄铁军; 高文
Original assignee: 北京大学
Priority date: 2015-01-09
Filing date: 2015-09-15
Publication date: 2016-07-14

Abstract

A hash method for a high dimension vector, and a vector quantization method and device. The hash method comprises: performing quantization processing on a compressed high dimension vector in a scalar quantization manner, to acquire quantization information of each element, each element being an element of each dimension in the compressed high dimension vector (S101); coding the quantization information of each element into binary code (S102); and combing all binary code according to a preset rule, to obtain a binary vector (S103). The method can reduce storage complexity in a neighbor query process in the prior art, and improve search accuracy and search efficiency.

Description

高维向量的哈希方法、向量量化方法及装置Hash method, vector quantization method and device for high dimensional vector

技术领域Technical field

本发明涉及计算机技术，具体涉及一种高维向量的哈希方法、向量量化方法及装置。The present invention relates to computer technology, and in particular to a high-dimensional vector hash method, vector quantization method and device.

背景技术Background technique

当前，高维向量的近邻查询在计算机的图像搜索中提供着关键性的技术支撑。高维向量的近邻查询是指：给定一个高维向量，从一个向量集中找出与其最近的若干个向量。高维向量的近邻查询可应用于人工智能、机器学习以及计算机视觉等方面。Currently, high-dimensional vector neighbor queries provide critical technical support in computer image search. A neighbor query of a high-dimensional vector means that given a high-dimensional vector, several vectors closest to it are found from a set of vectors. High-dimensional vector neighbor queries can be applied to artificial intelligence, machine learning, and computer vision.

为此，越来越多的业内人士研究高维向量的近邻查询的相关技术。现有技术中，解决近邻查询的方法主要分为两类。第一类方法是通过建立索引结构，离线将相似的向量集中到一起，在线查询时，从索引中快速的读取结果，从而获得查询向量的近邻。然而，该第一类方法最大的缺点是内存消耗大，该索引结构的内存占用量比原始数据的内存占用量还大，导致查询效率非常低。To this end, more and more people in the industry are studying the related techniques of high-dimensional vector neighbor query. In the prior art, methods for solving neighbor queries are mainly divided into two categories. The first method is to set up an index structure to bring together similar vectors offline. When online, the results are quickly read from the index, and the neighbors of the query vector are obtained. However, the biggest disadvantage of the first method is that the memory consumption is large, and the memory usage of the index structure is larger than the memory usage of the original data, resulting in very low query efficiency.

第二类方法是通过乘积量化的方式将高维向量分成多段，每段单独量化，并建立索引。以一个维度为8的向量为例Y＝{y₁，y₂，y₃，y₄，y₅，y₆，y₇，y₈}。常用的乘积量化会将这个8维向量划分成两段，分别是{y₁，y₂，y₃，y₄}以及{y₅，y₆，y₇，y₈}。然后每段的维度统一共同量化。该方法的缺点是需要将高维向量拆分成许多段，每段使用额外的查找表；当向量维度较高时，会破坏向量的空间结构，内存占用大，导致采用量化后的信息在图像检索中检索准确率降低，且检索速度慢，效率低。The second method is to divide the high-dimensional vector into multiple segments by means of product quantization, each segment is separately quantized and indexed. Take a vector with a dimension of 8 as an example Y={y ₁ , y ₂ , y ₃ , y ₄ , y ₅ , y ₆ , y ₇ , y ₈ }. The commonly used product quantization divides this 8-dimensional vector into two segments, {y ₁ , y ₂ , y ₃ , y ₄ } and {y ₅ , y ₆ , y ₇ , y ₈ }. Then the dimensions of each segment are unified and quantified together. The disadvantage of this method is that it needs to split the high-dimensional vector into many segments, each segment uses an additional lookup table; when the vector dimension is high, it will destroy the spatial structure of the vector, and the memory usage is large, resulting in the use of quantized information in the image. The retrieval accuracy in the search is reduced, and the retrieval speed is slow and the efficiency is low.

为此，如何降低近邻查询过程中的存储复杂度，并提升检索准确率和检索效率成为当前需要解决的技术问题。Therefore, how to reduce the storage complexity in the neighborhood query process and improve the retrieval accuracy and retrieval efficiency become the technical problems that need to be solved.

发明内容Summary of the invention

针对现有技术中的缺陷，本发明提供了一种高维向量的哈希方法及装置，能够降低现有技术中近邻查询过程中的存储复杂度，以及提升检索准确率和检索效率。 Aiming at the defects in the prior art, the present invention provides a high-dimensional vector hashing method and device, which can reduce the storage complexity in the prior neighbor query process, and improve the retrieval accuracy and retrieval efficiency.

第一方面，本发明提供一种高维向量的哈希方法，包括：In a first aspect, the present invention provides a hashing method for a high dimensional vector, comprising:

采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素；The compressed high-dimensional vector is quantized by using a scalar quantization method to obtain quantization information of each element, wherein each element is an element of each dimension in the compressed high-dimensional vector;

将所述每个元素的量化信息编码成二值码；Encoding the quantized information of each element into a binary code;

按照预设规则将所有二值码进行组合，获得二值向量。All binary codes are combined according to a preset rule to obtain a binary vector.

可选地，采用标量量化方式对压缩后的高维向量进行量化处理之前，所述方法还包括：Optionally, before performing the quantization process on the compressed high-dimensional vector by using a scalar quantization method, the method further includes:

对待处理的高维向量进行压缩处理，获取压缩后的高维向量。The processed high-dimensional vector is compressed to obtain a compressed high-dimensional vector.

可选地，采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，包括：Optionally, the compressed high-dimensional vector is quantized by using a scalar quantization method to obtain quantitative information of each element, including:

获取压缩后的高维向量中每个维度的元素；Obtaining the elements of each dimension in the compressed high-dimensional vector;

根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得每个元素的量化信息；And quantizing the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtaining quantization information of each element;

所述每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.

可选地，根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：Optionally, the method further includes: before the interval corresponding to the dimension to which the element belongs, according to the interval corresponding to each dimension acquired in advance, the method further includes:

获取所述压缩后高维向量的维度，根据所述维度获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。Acquiring the dimension of the compressed high-dimensional vector, and acquiring the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension of the dimension acquired in advance.

根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述高维向量中的每个维度；Obtaining, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the high-dimensional vector;

每一维度对应的区间均是不重叠的。The intervals corresponding to each dimension are not overlapping.

可选地，根据预设的训练数据集训练获取每个维度对应的区间，包括：Optionally, the training corresponding to the training data set is used to obtain the interval corresponding to each dimension, including:

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同；Obtaining a set of elements for each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all vectors in the training data set being the same;

根据预设距离差和量化误差，获取用于划分区间的目标函数；Obtaining an objective function for dividing the interval according to the preset distance difference and the quantization error;

根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。According to the objective function, the element set of each dimension is divided into sections to obtain an interval corresponding to the dimension.

可选地，根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间，包括：Optionally, according to the objective function, the element set of each dimension is divided into sections, and the interval corresponding to the dimension is obtained, including:

枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Enumerating a boundary value of the interval division in the set of elements, and dividing a boundary value of each interval of the group Determining and recording a value corresponding to the objective function as a parameter value of the objective function;

选择使目标函数取得极值的边界值划分该维度，获得该维度对应的区间。The boundary value of the extreme value of the objective function is selected to divide the dimension, and the interval corresponding to the dimension is obtained.

可选地，给每个维度对应的区间的元素进行编码，被量化到某个区间的元素将编码成该区间对应的二值码。编码的方式可以采用自然二进制码和一元码。Optionally, the elements of the interval corresponding to each dimension are encoded, and the elements quantized into a certain interval are encoded into the binary code corresponding to the interval. The encoding method can adopt a natural binary code and a unary code.

可选地，按照预设规则将所有二值码进行组合，获得二值向量，包括：Optionally, all the binary codes are combined according to a preset rule to obtain a binary vector, including:

将所有二值码按照每一二值码对应的维度序号顺序拼接，获得二值向量。All binary codes are spliced in the order of the dimension numbers corresponding to each binary code to obtain a binary vector.

可选地，所述对待处理的高维向量进行压缩处理，获取压缩后的高维向量，包括：Optionally, the high-dimensional vector to be processed is subjected to compression processing to obtain a compressed high-dimensional vector, including:

采用预设的线性变换矩阵对待处理的高维向量进行压缩处理，获得压缩后的高维向量；The high-dimensional vector to be processed is compressed by using a preset linear transformation matrix to obtain a compressed high-dimensional vector;

所述线性变换矩阵为通过训练样本预先训练获取的。The linear transformation matrix is obtained by pre-training through training samples.

采用预设的非线性变换函数对待处理的高维向量进行压缩处理，获得压缩后的高维向量；The high-dimensional vector to be processed is compressed by using a preset nonlinear transformation function to obtain a compressed high-dimensional vector;

所述非线性变换函数通过训练样本预先训练获取的。The nonlinear transformation function is pre-trained by training samples.

第二方面，本发明还提供一种高维向量的近邻查询方法，包括：In a second aspect, the present invention further provides a high-dimensional vector neighbor query method, including:

采用如上任一所述的方法获取待处理的高维向量的二值向量；Obtaining a binary vector of a high dimensional vector to be processed by using any of the methods described above;

获取所述二值向量的哈希二值码；Obtaining a hash binary code of the binary vector;

根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。利用二值向量的相似性/相似度表达高维向量之间的相似性。And according to the coding manner of the hash binary code, selecting a metric distance corresponding to the coding mode, and using the metric distance to measure a similarity of the binary vector corresponding to the other high-dimensional vectors. The similarity between high-dimensional vectors is expressed by the similarity/similarity of binary vectors.

第三方面，本发明还提供一种高维向量的哈希装置，包括：In a third aspect, the present invention also provides a high-dimensional vector hashing apparatus, including:

量化单元，用于采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素；And a quantization unit, configured to perform quantization processing on the compressed high-dimensional vector by using a scalar quantization method, to obtain quantization information of each element, wherein each of the elements is an element of each dimension in the compressed high-dimensional vector;

编码单元，用于将所述每个元素的量化信息编码成二值码；a coding unit, configured to encode the quantization information of each element into a binary code;

组合单元，用于按照预设规则将所有二值码进行组合，获得二值向量。A combination unit is configured to combine all the binary codes according to a preset rule to obtain a binary vector.

可选地，所述装置还包括：Optionally, the device further includes:

压缩单元，用于在量化单元量化之前对待处理的高维向量进行压缩处理，获取压缩后的高维向量。 And a compression unit, configured to perform compression processing on the high-dimensional vector to be processed before the quantization unit quantizes, to obtain the compressed high-dimensional vector.

可选地，量化单元，具体用于Optionally, the quantization unit is specifically used for

获取所述压缩后高维向量的维度，根据所述维度，获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间；Acquiring the dimension of the compressed high-dimensional vector, and acquiring the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension of the dimension acquired in advance;

根据获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得每个元素的量化信息；Obtaining, according to the interval corresponding to each dimension, the element to the interval corresponding to the dimension to which the element belongs, and obtaining the quantization information of each element;

可选地，所述量化单元，还用于Optionally, the quantization unit is further used to

可选地，所述量化单元，具体用于Optionally, the quantization unit is specifically configured to be used

枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Enumerating a boundary value of the interval division in the set of the elements, using a boundary value of each of the interval divisions as a parameter value of the objective function, determining and recording a value corresponding to the target function;

可选地，组合单元，具体用于Optionally, a combination unit, specifically for

可选地，所述编码单元，具体用于Optionally, the coding unit is specifically configured to be used

对每一维度的区间(即量化后的区间)中的元素编码成二值码，编码的方式可以是自然二进制码也可以是一元码。 The elements in the interval of each dimension (ie, the quantized interval) are encoded into a binary code, and the encoding may be a natural binary code or a unary code.

可选地，压缩单元，具体用于Optionally, the compression unit is specifically used for

所述非线性变换函数为通过训练样本预先训练获取的。The nonlinear transformation function is obtained by pre-training through training samples.

第四方面，本发明提供一种高维向量的近邻查询装置，包括：In a fourth aspect, the present invention provides a high-dimensional vector neighbor query device, including:

二值向量获取单元，用于采用上述任一所述的装置获取待处理的高维向量的二值向量；a binary vector obtaining unit, configured to acquire a binary vector of a high-dimensional vector to be processed by using any one of the foregoing devices;

哈希二值码获取单元，用于获取所述二值向量的哈希二值码；a hash binary value obtaining unit, configured to obtain a hash binary code of the binary vector;

相似度度量单元，用于根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。a similarity measure unit, configured to select a metric distance corresponding to the coding mode according to the coding manner of the hash binary code, and measure the binary vector corresponding to the other high-dimensional vector by using the metric distance Similarity.

第五方面，本发明提供一种向量量化方法，包括：In a fifth aspect, the present invention provides a vector quantization method, including:

获取待量化的向量中每个维度的元素；Get the elements of each dimension in the vector to be quantized;

根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息；And quantizing the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtaining the quantization information of the vector to be quantized;

可选地，所述根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：Optionally, the method further includes: before the interval corresponding to the dimension to which the element belongs, according to the interval corresponding to each dimension acquired in advance, the method further includes:

获取所述待量化向量的维度，根据所述维度获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。Acquiring the dimension of the vector to be quantized, and acquiring the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension of the dimension acquired in advance.

根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度；Obtaining, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the to-be quantized vector;

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Obtaining a set of elements of each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all the vectors in the training data set are the same, and waiting The dimensions of the quantization vector are the same;

第六方面，本发明还提供一种向量量化装置，包括：In a sixth aspect, the present invention also provides a vector quantization apparatus, including:

元素获取单元，用于获取待量化的向量中每个维度的元素；An element obtaining unit, configured to acquire an element of each dimension in the vector to be quantized;

量化单元，用于根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息；a quantization unit, configured to quantize the element to an interval corresponding to a dimension to which the element belongs according to an interval corresponding to each dimension acquired in advance, to obtain quantization information of the vector to be quantized;

其中，所述每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.

可选地，所述装置还包括：Optionally, the device further includes:

维度确定单元，用于获取所述待量化向量的维度，根据所述维度，获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。a dimension determining unit, configured to acquire a dimension of the to-be-quantized vector, and obtain, according to the dimension, interval information corresponding to the dimension, where the section information includes: a section corresponding to each dimension in the dimension acquired in advance.

可选地，所述装置还包括：Optionally, the device further includes:

维度划分单元，用于根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度；a dimension dividing unit, configured to acquire, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the to-be quantized vector;

可选地，维度划分单元，具体用于Optionally, the dimension dividing unit is specifically used for

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Obtaining a set of elements of each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all the vectors in the training data set being the same, and being the same as the dimension of the vector to be quantized;

由上述技术方案可知，本发明的高维向量的哈希方法及装置，通过获取高维向量中每个元素的量化信息，进而将量化信息编码成二值码，由此，可组合二值码，获得高维向量的二值向量，进而能够对降低现有技术中近邻查询过程中的存储复杂度，以及提升检索准确率和检索效率。According to the above technical solution, the high-dimensional vector hash method and apparatus of the present invention can obtain the quantized information of each element in the high-dimensional vector, and then encode the quantized information into a binary code, thereby combining the binary code. The binary vector of the high-dimensional vector is obtained, which can reduce the storage complexity in the prior-neighbor query process in the prior art, and improve the retrieval accuracy and retrieval efficiency.

本发明的向量量化方法及装置，通过获取待量化的向量中每个维度的元素，进而将元素量化到该元素所属维度对应的区间，获得待量化的向量的量化信息，由此，可实现对高维向量进行量化处理，且采用量化后的信息能够提高图像检索中的检索准确率。The vector quantization method and device of the present invention obtains the quantized information of the vector to be quantized by obtaining the element of each dimension in the vector to be quantized, and then quantizing the element to the interval corresponding to the dimension to which the element belongs, thereby realizing The high-dimensional vector is quantized, and the quantized information can improve the retrieval accuracy in image retrieval.

附图说明DRAWINGS

图1为本发明一实施例提供的高维向量的哈希方法的流程示意图；1 is a schematic flowchart of a hashing method for a high-dimensional vector according to an embodiment of the present invention;

图2为本发明一实施例提供的高维向量的哈希方法的流程示意图；2 is a schematic flowchart of a hashing method for a high-dimensional vector according to an embodiment of the present invention;

图3为本发明一实施例提供的高维向量的近邻查询方法的流程示意图；FIG. 3 is a schematic flowchart diagram of a high-dimensional vector neighbor query method according to an embodiment of the present invention;

图4为本发明一实施例提供的高维向量的哈希装置的结构示意图；4 is a schematic structural diagram of a high-dimensional vector hash device according to an embodiment of the present invention;

图5为本发明一实施例提供的高维向量的近邻查询装置的结构示意图；FIG. 5 is a schematic structural diagram of a high-dimensional vector neighbor query apparatus according to an embodiment of the present invention;

图6为本发明一实施例提供的向量量化方法的流程示意图；FIG. 6 is a schematic flowchart of a vector quantization method according to an embodiment of the present invention;

图7为本发明另一实施例提供的向量量化方法的流程示意图；FIG. 7 is a schematic flowchart diagram of a vector quantization method according to another embodiment of the present invention;

图8为本发明一实施例提供的向量量化装置的结构示意图。FIG. 8 is a schematic structural diagram of a vector quantization apparatus according to an embodiment of the present invention.

具体实施方式detailed description

下面结合附图，对发明的具体实施方式作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。本发明实施例中所使用的“第一”、“第二”仅为更清楚的说明本申请的内容，不具有特定含义，也不限定任何内容。The specific embodiments of the invention are further described below in conjunction with the accompanying drawings. The following examples are only intended to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of the present invention. The "first" and "second" used in the embodiments of the present invention are only for the purpose of clearly explaining the contents of the present application, and do not have a specific meaning and do not limit any content.

当前，业内还出现一种近邻查询的技术方案，例如，该技术方案是通过压缩高维向量降低高维向量的维度然后再将降维后的向量量化成二值码的哈希方法。不同于现有技术中提及的第一类和第二类方法，该技术方案中的哈希方法通常不需要额外的索引，直接利用量化后的二值码之间的距离判定向量之间的相似性。Currently, there is a technical solution for neighbor query in the industry. For example, the technical solution is a hash method for reducing a dimension of a high-dimensional vector by compressing a high-dimensional vector and then quantizing the reduced-dimensional vector into a binary code. Different from the first and second methods mentioned in the prior art, the hash method in the technical solution usually does not require an additional index, and directly utilizes the distance between the quantized binary codes. The similarity between the decision vectors.

然而，该近邻查询的技术方案还仅仅是简单地将压缩后向量的每个元素用一个预设的阈值量化成单个比特，这个过程会引入较大的量化误差。However, the technical solution of the neighbor query is simply to quantize each element of the compressed vector to a single bit with a preset threshold, which introduces a large quantization error.

为此，如何降低近邻查询过程中的存储复杂度，提升检索效率；以及解决上述方案中单个比特量化的误差问题，本发明实施例中引入更精确的量化的机制(比如多比特量化)进而可减少量化误差。To this end, how to reduce the storage complexity in the neighbor query process, improve the retrieval efficiency, and solve the error problem of single bit quantization in the above solution, and introduce a more accurate quantization mechanism (such as multi-bit quantization) in the embodiment of the present invention. Reduce quantization error.

图1示出了本发明一实施例提供的高维向量的哈希方法的流程示意图，如图1所示，本实施例中的高维向量的哈希方法如下所述。FIG. 1 is a schematic flowchart diagram of a hashing method for a high-dimensional vector according to an embodiment of the present invention. As shown in FIG. 1, the hashing method of the high-dimensional vector in this embodiment is as follows.

101、采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素。101. Perform quantization processing on the compressed high-dimensional vector by using a scalar quantization method to obtain quantization information of each element, where each element is an element of each dimension in the compressed high-dimensional vector.

举例来说，对于压缩后的高维向量X＝[x₁,x₂,…x_m]中的任何一个元素x_i(1≤i≤m)，x_i量化后的量化信息为Q(x_i)，其中，Q表示量化函数。For example, for any one of the compressed high-dimensional vectors X=[x ₁ , x ₂ , . . . x _m ] x _i (1≤i≤m), the quantized information after x _i is Q(x) _i ), where Q represents a quantization function.

102、将所述每个元素的量化信息编码成二值码；102. Encode the quantization information of each element into a binary code.

在具体应用中，每个维度的元素量化后的区间编码即为该元素的编码。对于元素x_i，若Q(x_i)＝S_ij,那么x_i将被编码成B(S_ij)。In a specific application, the interval coded element of each dimension is the code of the element. For element x _i , if Q(x _i )=S _ij , then x _i will be encoded as B(S _ij ).

举例来说，可在应用中，将每个维度的元素x_i划分到区间，将每个维度的元素划分后的区间S_ij，1≤i≤m，1≤j≤u，编码成二值码B(S_ij)。For example, in the application, the element x _{i of} each dimension is divided into intervals, and the interval S _ij , 1 ≤ _i ≤ m, 1 ≤ _j ≤ u, which is divided by the elements of each dimension, is encoded into two values. Code B (S _ij ).

其中，m为压缩后的高维向量的维度，u为非零自然数，u表示的是每个维度划分的区间数，比如u＝4，那么就是将每个维度划分成4个区间，此时若采用二进制编码，可以分别将每个区间段依次编码成00，01，10，11等。Where m is the dimension of the compressed high-dimensional vector, u is a non-zero natural number, and u represents the number of intervals divided by each dimension. For example, u=4, then each dimension is divided into 4 intervals. If binary coding is used, each interval segment can be sequentially encoded into 00, 01, 10, 11, and the like.

具体的，二值码包括但不限于自然二进制码，包括：Specifically, the binary code includes but is not limited to a natural binary code, including:

举例来说，设u＝4,如果将S_ij编码成2个比特，那么S_i1，S_i2，S_i3，S_i4对应的自然二进制码为00，01，10，11。对于二进制编码，采用的距离度量为二值码对应数值的差。比如00与01的距离|(00)₂-(11)₂|＝|0-3|＝3。For example, let u=4. If S _{ij is} encoded into 2 bits, the natural binary codes corresponding to S _i1 , S _i2 , S _i3 , S _i4 are 00, 01, 10, 11. For binary coding, the distance metric used is the difference between the values of the binary code. For example, the distance between 00 and 01|(00) ₂ -(11) ₂ |=|0-3|=3.

当然，在实际应用中，二值码包括但不限于上述的二进制码，还可为一元码，例如：设u＝4,如果将S_ij编码成3个比特，那么S_i1，S_i2，S_i3，S_i4对应的一元码为000，001，011，111。对于一元码，采用的距离度量可以为汉明距离。汉明距离的定义如下：给定两个二值码，其汉明距离等于异或以后数值为1的比特位的数目。比如010与011的抑或以后等于001，那么010与011的汉明距离为1.Of course, in practical applications, the binary code includes but is not limited to the above binary code, and may also be a unary code, for example: u=4, if S _{ij is} encoded into 3 bits, then S _i1 , S _i2 , S _The unary codes corresponding to _i3 and S _i4 are 000, 001, 011, and 111. For a unary code, the distance metric used can be Hamming distance. The Hamming distance is defined as follows: Given two binary codes, the Hamming distance is equal to the number of bits whose value is 1 after XOR. For example, if 010 and 011 are equal to or equal to 001, then the Hamming distance between 010 and 011 is 1.

其它编码方式在本实施例中也可使用，本实施例仅为举例说明，并不是限定性的内容。Other coding methods may also be used in this embodiment. This embodiment is merely illustrative and not limiting.

103、按照预设规则将所有二值码进行组合，获得二值向量。103. Combine all the binary codes according to a preset rule to obtain a binary vector.

例如，可将所有二值码按照每一二值码对应的维度序号顺序拼接，获得二值向量。For example, all the binary codes may be spliced in the order of the dimension numbers corresponding to each binary code to obtain a binary vector.

通常，每一二值码对应的维度序号即为该二值码的下标，进而可按照所有二值码的下标顺序拼接获得二值向量。举例来说，对于一个向量(y₁，y₂，y₃)，维度序号就是下标，该下标实质上就是1，2，3这些序号。其中y₁量化成00，y₂量化成01，y₃量化成11，那么拼接到一起获得二值向量就是000111。为此，可按维度序号从左至右拼接到一起。Generally, the dimension number corresponding to each binary code is the subscript of the binary code, and the binary vector can be obtained by splicing according to the subscript order of all the binary codes. For example, for a vector (y ₁ , y ₂ , y ₃ ), the dimension number is the subscript, and the subscript is essentially the sequence numbers ₁ , ₂ , and ₃ . Where y _{1 is} quantized to 00, y _{2 is} quantized to 01, and y _{3 is} quantized to 11, and then spliced together to obtain a binary vector is 000111. To do this, you can stitch together from left to right by dimension number.

首先，按顺序排列每个元素的量化信息编码后的二值码。对于X，按下标顺序排列X中各个元素的量化信息编码后的二值码B(Q(x₁))，B(Q(x₂))，…，B(Q(x_m))。First, the binary code encoded by the quantization information of each element is arranged in order. For X, the binary code B(Q(x ₁ )), B(Q(x ₂ )), ..., B(Q(x _m )) encoded by the quantization information of each element in the X is arranged in the following order.

接着，把各个元素的量化信息编码后的二值码组合到一起，构成二值向量。例如，将B(Q(x₁))，B(Q(x₂))，…，B(Q(x_m))组合，得到二值向量H，H＝[B(Q(x₁))，B(Q(x₂))，…，B(Q(x_m))]。Next, the binary codes encoded by the quantization information of the respective elements are combined to form a binary vector. For example, combining B(Q(x ₁ )), B(Q(x ₂ )),..., B(Q(x _m )) to obtain a binary vector H, H=[B(Q(x ₁ )) , B(Q(x ₂ )), ..., B(Q(x _m ))].

在本实施例中二值向量即为高维向量哈希后的结果。In this embodiment, the binary vector is the result of high-dimensional vector hashing.

本实施例的高维向量的哈希方法可通过获取高维向量中每个元素的量化信息，进而将量化信息编码成二值码，由此，可组合二值码，获得高维向量的二值向量，进而能够对降低现有技术中近邻查询过程中的存储复杂度，以及提升检索效率。The hash method of the high-dimensional vector of this embodiment can obtain the quantized information of each element in the high-dimensional vector, and then encode the quantized information into a binary code. Thus, the binary code can be combined to obtain the high-dimensional vector. The value vector can further reduce the storage complexity in the prior neighbor query process and improve the retrieval efficiency.

图2示出了本发明一实施例提供的高维向量的哈希方法的流程示意图，如图2所示，本实施例中的高维向量的哈希方法如下所述。FIG. 2 is a schematic flowchart diagram of a hashing method for a high-dimensional vector according to an embodiment of the present invention. As shown in FIG. 2, the hashing method of the high-dimensional vector in this embodiment is as follows.

201、对待处理的高维向量进行压缩处理，获取压缩后的高维向量。201. Perform compression processing on the processed high-dimensional vector to obtain a compressed high-dimensional vector.

例如，采用预设的线性变换矩阵对待处理的高维向量进行压缩处理，获得压缩后的高维向量；For example, the high-dimensional vector to be processed is compressed by using a preset linear transformation matrix to obtain a compressed high-dimensional vector;

所述线性变换矩阵为通过训练样本预先训练获取的。 The linear transformation matrix is obtained by pre-training through training samples.

举例来说，步骤201的具体实现过程如下：For example, the specific implementation process of step 201 is as follows:

A01、采集一个离线训练的数据集，组成训练样本S_Z，例如，采集一个n维向量集合S_Z＝{Z₁，Z₂，…，Z_t}，Z₁，Z₂，…,Z_t为S_Z中的元素。A01. Collect an offline training data set to form a training sample S _Z . For example, collect an n-dimensional vector set S _Z ={Z ₁ , Z ₂ , . . . , Z _t }, Z ₁ , Z ₂ , . . . , Z _t Is the element in S _Z.

其中，Z_i为n维向量，1≤i≤t，t为自然数。Where Z _i is an n-dimensional vector, 1 ≤ _i ≤ t, and t is a natural number.

A02、从训练样本S_Z中训练获取一个线性变换矩阵A。A02. Obtain a linear transformation matrix A from the training sample S _Z .

即，可采用主要成分分析方法((Principal Component Analysis，简称PCA)从向量集合S_Z中学习获得一个线性变换矩阵A，该线性变换矩阵A的维度为n*m，m为压缩后的高维向量的维度。That is, a Principal Component Analysis (PCA) can be used to learn from the vector set S _Z to obtain a linear transformation matrix A having a dimension of n*m and m being a compressed high dimension. The dimension of the vector.

A03、将待处理的高维向量乘以线性变换矩阵，得到压缩后的高维向量。A03. Multiply the high-dimensional vector to be processed by a linear transformation matrix to obtain a compressed high-dimensional vector.

例如，可对于待处理的高维向量Y＝[y₁，y₂，…，y_n]，乘以线性变换矩阵A，得到压缩后的高维向量X＝F(Y)＝(Y^TA)^T＝[x₁，x₂，…，x_m]，m为自然数，F代表压缩处理。For example, the high-dimensional vector Y=[y ₁ , y ₂ , . . . , y _n ] to be processed may be multiplied by the linear transformation matrix A to obtain a compressed high-dimensional vector X=F(Y)=(Y ^T A ^T = [x ₁ , x ₂ , ..., x _m ], m is a natural number, and F represents a compression process.

也可以将待处理的高维向量进行非线性变换，得到压缩后的高维向量。The high-dimensional vector to be processed can also be nonlinearly transformed to obtain a compressed high-dimensional vector.

例如可对于待处理的高维向量Y＝[y₁，y₂，…，y_n]，进行非线性函数cos()变换，得到压缩后的高维向量X＝K(Y)＝cos(YS)＝[cos(y₁s₁),cos(y₂s₂),…,cos(y_m s_m)]，m为自然数，K为非线性变换。非线性函数为预先通过训练样本获取的。For example, a nonlinear function cos() transformation can be performed on the high-dimensional vector Y=[y ₁ , y ₂ , . . . , y _n ] to be processed, and a compressed high-dimensional vector X=K(Y)=cos(YS) is obtained. ) = [cos(y ₁ s ₁ ), cos(y ₂ s ₂ ), ..., cos(y _m s _m )], m is a natural number, and K is a nonlinear transformation. The nonlinear function is obtained in advance through the training sample.

当然，在实际应用中，还可采用其他方式对待处理的高维向量进行压缩处理，不限定上述举例的步骤A01至步骤A03。Of course, in practical applications, the high-dimensional vectors to be processed may be compressed in other manners, and the steps A01 to A03 of the above examples are not limited.

202、获取压缩后的高维向量中每个维度的元素。202. Obtain an element of each dimension in the compressed high-dimensional vector.

203、根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得每个元素的量化信息。203. Quantify the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtain quantization information of each element.

例如，对于X中的任何一个元素x_i(1≤i≤m)，x_i量化后的量化信息为Q(x_i)，其中，Q表示量化函数。For example, for any element x in X (1 ≤ _i ≤ m), the quantized information after x _i is Q(x _i ), where Q represents a quantization function.

在具体实现过程中，预先将每个维度的元素取值范围划分成几个独立的区间。设X中每个维度的元素x_i取值范围为L_i＝[l_i，r_i]。将L_i划分成u个独立的区间，S_i1,S_i2,…,S_iu。In the specific implementation process, the range of elements of each dimension is divided into several independent intervals in advance. Let the element x _i of each dimension in X be in the range of L _i =[l _i ,r _i ]. L _{i is} divided into u independent intervals, S _i1 , S _i2 , ..., S _iu .

其中，区间之间不重叠。且L_i＝S_i1∪S_i2∪…∪S_iu。Among them, the intervals do not overlap. And L _i =S _i1 ∪S _i2 ∪...∪S _iu .

其次，把每个元素量化到该元素所属维度对应的区间。 Second, each element is quantized to the interval corresponding to the dimension to which the element belongs.

若元素x_i属于S_ij，那么x_i将被量化到S_ij＝Q(x_i)。If the element x _i belongs to S _ij , then x _i will be quantized to S _ij =Q(x _i ).

其中，每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.

举例来说，可预先离线获取所述压缩后高维向量的维度，根据所述维度获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间，进而再进一步执行步骤203。For example, the dimension of the compressed high-dimensional vector may be obtained in advance, and the section information corresponding to the dimension may be acquired according to the dimension, where the section information includes: each dimension of the dimension acquired in advance The interval is further performed in step 203.

例如，高维向量可为128维，则可获取与128维对应的每一维度的区间信息。在部分实施例中，不同维度的区间信息可不同。For example, if the high-dimensional vector can be 128-dimensional, the interval information of each dimension corresponding to 128-dimensional can be obtained. In some embodiments, the interval information for different dimensions may be different.

204、将所述每个元素的量化信息编码成二值码。204. Encode the quantized information of each element into a binary code.

例如，对于量化信息Q(x_i)编码成二值码B(Q(x_i))，B表示编码函数。For example, for the quantized information Q(x _i ) is encoded as a binary code B(Q(x _i )), and B represents an encoding function.

205、按照预设规则将所有二值码进行组合，获得二值向量。205. Combine all the binary codes according to a preset rule to obtain a binary vector.

例如，将编码后的二值码按照下标顺序拼接B(Q(x₁))，B(Q(x₂))，…，B(Q(x_m))，得到H＝[B(Q(x₁))，B(Q(x₂))，…，B(Q(x_m))]，H为二值向量。For example, the encoded binary code is spliced B(Q(x ₁ )), B(Q(x ₂ )),..., B(Q(x _m )) according to the subscript order, and H=[B(Q) is obtained. (x ₁ )), B(Q(x ₂ )), ..., B(Q(x _m ))], H is a binary vector.

本实施例中的H为紧凑的二值向量。H in this embodiment is a compact binary vector.

本实施例的高维向量的哈希方法能够对降低现有技术中近邻查询过程中的存储复杂度，以及提升检索效率。The hashing method of the high-dimensional vector of the embodiment can reduce the storage complexity in the prior-neighbor query process in the prior art, and improve the retrieval efficiency.

本实施例能够有效解决现有哈希方法中的单比特量化的性能损失问题。现有的哈希方法大多是简单地将压缩后向量的每个元素用一个预设的阈值量化成单个比特，这个过程会引入较大的量化误差。而本实施例则提供了一种更科学并且支持多比特量化的量化机制，能够大大降低量化误差，解决现有哈希方法中单比特量化的问题。This embodiment can effectively solve the performance loss problem of single bit quantization in the existing hash method. Most of the existing hashing methods simply quantize each element of the compressed vector to a single bit with a preset threshold, which introduces a large quantization error. The present embodiment provides a more scientific and multi-bit quantization quantization mechanism, which can greatly reduce the quantization error and solve the problem of single-bit quantization in the existing hash method.

下面对上述步骤203进行详细说明如下：The above step 203 will be described in detail as follows:

B01、根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述高维向量中的每个维度；每一维度对应的区间均是不重叠的。B01. The interval corresponding to each dimension is obtained according to a preset training data set, where each dimension is each dimension in the high-dimensional vector; and the intervals corresponding to each dimension are not overlapping.

B02、根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间。B02: Quantify the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance.

该步骤B02的具体实现过程如下：The specific implementation process of step B02 is as follows:

B021、获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同；B021. Obtain an element set of each dimension of all vectors in the training data set, where the element set includes elements of all vectors in the dimension, and all the vectors in the training data set have the same dimension;

B022、根据预设距离差和量化误差，获取用于划分区间的目标函数； B022. Obtain an objective function for dividing the interval according to a preset distance difference and a quantization error;

B023、根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。B023. Perform an interval division of the element set of each dimension according to the objective function, and obtain an interval corresponding to the dimension.

具体地，B0231、枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Specifically, B0231 enumerates a boundary value of the interval division in the set of elements, and uses a boundary value of each of the interval divisions as a parameter value of the objective function, and determines and records a value corresponding to the target function;

B0232、选择使目标函数取得极值(如最小值)的边界值划分该维度，获得该维度对应的区间。B0232: Select a boundary value that makes the target function obtain an extreme value (such as a minimum value) to divide the dimension, and obtain a section corresponding to the dimension.

可理解的是，在具体应用中，目标函数还可能是非连续而且不可导的函数，在应用中存在无法直接获取该目标函数的极值，也就是说，无法直接判断该目标函数在某个取值下是否是极大或者极小值。为此，可预先枚举足够多的边界值，从中选择使目标函数相对最大(或者最小)的边界值来划分区间。Understandably, in a specific application, the objective function may also be a non-continuous and non-conductible function. In the application, there is an extremum that cannot directly obtain the target function, that is, the objective function cannot be directly judged at a certain fetch. Whether the value is a maximum or a minimum value. To this end, a sufficient number of boundary values can be pre-enumerated, and a boundary value that makes the objective function relatively largest (or smallest) can be selected to divide the interval.

当然，可根据实际需要选择使目标函数取得最大值或最小值的方式，本实施例均为举例说明，不对其进行限定。Of course, the method of obtaining the maximum value or the minimum value of the target function may be selected according to actual needs. This embodiment is illustrative and not limited.

图3示出了本发明一实施例提供的高维向量的近邻查询方法的流程示意图，如图3所示，本实施例中的高维向量的近邻查询方法如下所述。FIG. 3 is a schematic flowchart diagram of a high-dimensional vector neighbor query method according to an embodiment of the present invention. As shown in FIG. 3, the high-dimensional vector neighbor query method in this embodiment is as follows.

301、获取待处理的高维向量的二值向量。301. Obtain a binary vector of a high-dimensional vector to be processed.

在本实施例中可采用上述任一实施例所述的方法获取待处理的高维向量的二值向量。In this embodiment, the method described in any of the above embodiments may be used to obtain a binary vector of a high-dimensional vector to be processed.

302、获取所述二值向量的哈希二值码；302. Acquire a hash binary code of the binary vector.

303、根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。303. Select, according to the coding manner of the hash binary code, a metric distance corresponding to the coding mode, and measure, by using the metric distance, a similarity of the binary vector corresponding to the other high-dimensional vectors.

例如，对于两个紧凑的二值向量H1和H2，选择与所用二值码相匹配的距离D，来度量二值向量之间的相似性。For example, for two compact binary vectors H1 and H2, the distance D matching the binary code used is selected to measure the similarity between the binary vectors.

即两个二值向量H1和H2的相似性用D(H1，H2)来表示。That is, the similarity between the two binary vectors H1 and H2 is represented by D(H1, H2).

对于两个二值向量H1和H2，其相似性的度量举例可以为汉明距离，也可以为曼哈顿举例度量，其他举例度量方法本实施例不对其进行限定。For the two binary vectors H1 and H2, the metric of the similarity may be a Hamming distance or a Manhattan example metric. Other example metrics are not limited in this embodiment.

上述近邻查询方法可降低现有技术中高额的存储复杂度，并且大幅度地提升检索的效率。The above neighbor query method can reduce the high storage complexity in the prior art and greatly improve the retrieval efficiency.

图4示出了本发明一实施例提供的高维向量的哈希装置的结构示意图，如图4所示，本实施例的高维向量的哈希装置包括：量化单元41、编码单元42和组合单元43； FIG. 4 is a schematic structural diagram of a high-dimensional vector hashing apparatus according to an embodiment of the present invention. As shown in FIG. 4, the high-dimensional vector hashing apparatus of this embodiment includes: a quantization unit 41, an encoding unit 42, and Combination unit 43;

其中，量化单元41用于采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素；The quantization unit 41 is configured to perform quantization processing on the compressed high-dimensional vector by using a scalar quantization method to obtain quantization information of each element, where each element is an element of each dimension in the compressed high-dimensional vector. ;

编码单元42用于将所述每个元素的量化信息编码成二值码；The encoding unit 42 is configured to encode the quantization information of each element into a binary code;

组合单元43用于按照预设规则将所有二值码进行组合，获得二值向量。The combining unit 43 is configured to combine all the binary codes according to a preset rule to obtain a binary vector.

在具体应用中，前述的装置还可包括图中未示出的压缩单元40，该压缩单元40用于在量化单元41量化之前对待处理的高维向量进行压缩处理，获取压缩后的高维向量。例如，采用预设的线性变换矩阵对待处理的高维向量进行压缩处理，获得压缩后的高维向量；所述线性变换矩阵为通过训练样本预先训练获取的。In a specific application, the foregoing apparatus may further include a compression unit 40 not shown in the figure, and the compression unit 40 is configured to perform compression processing on the high-dimensional vector to be processed before the quantization unit 41 quantizes, and obtain the compressed high-dimensional vector. . For example, the high-dimensional vector to be processed is compressed by using a preset linear transformation matrix to obtain a compressed high-dimensional vector; the linear transformation matrix is obtained by pre-training through training samples.

在一种可能的应用中，量化单元41可具体用于，获取压缩后的高维向量中每个维度的元素；In one possible application, the quantization unit 41 may be specifically configured to obtain an element of each dimension in the compressed high-dimensional vector;

在另一种具体的应用中，量化单元41可具体用于，获取压缩后的高维向量中每个维度的元素；In another specific application, the quantization unit 41 may be specifically configured to obtain an element of each dimension in the compressed high-dimensional vector;

举例来说，所述量化单元41还可用于，根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述高维向量中的每个维度；For example, the quantization unit 41 is further configured to: acquire, according to a preset training data set, an interval corresponding to each dimension, where each dimension is each dimension in the high-dimensional vector;

例如，获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同；For example, obtaining a set of elements for each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all vectors in the training data set being the same;

根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。每一维度对应的区间均是不重叠的。According to the objective function, the element set of each dimension is divided into sections to obtain an interval corresponding to the dimension. The intervals corresponding to each dimension are not overlapping.

在实际应用中，根据所述目标函数，将每一维度的元素集合进行区间划分，可为：枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值； In an actual application, according to the objective function, the element set of each dimension is divided into sections, which may be: enumerating the boundary values of the interval division in the set of the elements, and using the boundary value of each section of the set as the Determining and recording a value corresponding to the objective function;

在更具体的例子中，组合单元43可具体用于，将所有二值码按照每一二值码对应的维度序号顺序拼接，获得二值向量。In a more specific example, the combining unit 43 may be specifically configured to splicing all the binary codes in order of the dimension numbers corresponding to each binary code to obtain a binary vector.

本实施例的高维向量的哈希装置能够对降低现有技术中近邻查询过程中的存储复杂度，以及提升检索效率。The high-dimensional vector hashing device of this embodiment can reduce the storage complexity in the prior art neighboring query process and improve the retrieval efficiency.

图5示出了本发明一实施例提供的高维向量的近邻查询装置的结构示意图，如图5所示，本实施例的高维向量的近邻查询装置包括：二值向量获取单元51、哈希二值码获取单元52和相似度度量单元53；FIG. 5 is a schematic structural diagram of a high-dimensional vector neighbor query apparatus according to an embodiment of the present invention. As shown in FIG. 5, the high-dimensional vector neighbor query apparatus of this embodiment includes: a binary vector acquisition unit 51, a binary binary code obtaining unit 52 and a similarity measuring unit 53;

其中，二值向量获取单元51，用于获取待处理的高维向量的二值向量；例如，可采用上述任意实施例所述的高维向量的哈希装置来获取二值向量；The binary vector obtaining unit 51 is configured to obtain a binary vector of the high-dimensional vector to be processed; for example, the hash device of the high-dimensional vector described in any of the foregoing embodiments may be used to obtain the binary vector;

哈希二值码获取单元52用于获取所述二值向量的哈希二值码；The hash binary code obtaining unit 52 is configured to obtain a hash binary code of the binary vector;

相似度度量单元53用于根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。The similarity measure unit 53 is configured to select a metric distance corresponding to the coding mode according to the coding manner of the hash binary code, and measure the binary vector corresponding to the other high-dimensional vectors by using the metric distance Similarity.

上述近邻查询装置可降低现有技术中高额的存储复杂度，并且大幅度地提升检索的效率。The above neighbor query device can reduce the high storage complexity in the prior art and greatly improve the efficiency of retrieval.

可理解的是，本实施例中的哈希装置和近邻查询装置均在计算设备中实现，如终端、移动终端或服务器等实现向量查询或检索，可应用于图像检索，或向量学习、机器学习等方面。It can be understood that the hash device and the neighbor query device in this embodiment are all implemented in a computing device, such as a terminal, a mobile terminal, or a server to implement vector query or retrieval, and can be applied to image retrieval, or vector learning, machine learning. etc.

下面结合附图，对发明的向量量化方法及装置进行详细说明。The vector quantization method and apparatus of the invention will be described in detail below with reference to the accompanying drawings.

图6示出了本发明一实施例提供的向量量化方法的流程示意图，如图6所示，本实施例的向量量化方法如下所述。FIG. 6 is a schematic flowchart diagram of a vector quantization method according to an embodiment of the present invention. As shown in FIG. 6, the vector quantization method in this embodiment is as follows.

601、获取待量化的向量中每个维度的元素。601. Obtain an element of each dimension in the vector to be quantized.

举例来说，待量化的n维向量为X＝[x₁，x₂，…，x_n]，获取待量化的向量中每个维度的元素x_i，其中，1≤i≤n。For example, the n-dimensional vector to be quantized is X=[x ₁ , x ₂ , . . . , x _n ], and the element x _i of each dimension in the vector to be quantized is obtained, where 1≤i≤n.

602、根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息。602. Quantify, according to the interval corresponding to each dimension acquired in advance, the element to the interval corresponding to the dimension to which the element belongs, and obtain the quantization information of the vector to be quantized.

对于每个维度的元素集合中的元素x_i，将元素集合中的所有元素量化到该维度对应的u个独立的区间S_i1，S_i2，…，S_iu。其中，该u个区间之间不重叠(即区间是独立的,对于一个维度的任意两个区间，它们的交集是空集)。即，若元素x_i属于区间S_ij，则将元素x_i量化到区间S_ij。For the element x _i in the set of elements of each dimension, all elements in the set of elements are quantized to the u independent intervals S _i1 , S _i2 , . . . , S _iu corresponding to the dimension. Wherein, the u intervals do not overlap (ie, the intervals are independent, and for any two intervals of one dimension, their intersection is an empty set). That is, if the element x _i belongs to the interval S _ij , the element x _{i is} quantized into the interval S _ij .

其中，u表示的是每个维度划分的区间数，比如u＝4，那么就是将该维度划分成4个区间。Where u denotes the number of intervals divided by each dimension, such as u=4, then the dimension is divided into 4 intervals.

举例来说，在步骤602之前，图6所示的方法还可包括下述的步骤601a；For example, before step 602, the method shown in FIG. 6 may further include the following step 601a;

601a、获取所述待量化向量的维度，根据所述维度获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。601a. Obtain a dimension of the to-be-quantized vector, and obtain the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension in the dimension acquired in advance.

例如，待量化向量的维度为64维，则可根据64维获取与64维度对应的区间信息；或者，待量化向量的维度为128维，则可根据128维获取与128维度对应的区间信息。For example, if the dimension of the vector to be quantized is 64 dimensions, the section information corresponding to the 64 dimension may be acquired according to the 64-dimensional; or the dimension of the vector to be quantized is 128-dimensional, and the section information corresponding to the 128 dimension may be acquired according to the 128-dimensional.

本实施例的向量量化方法，通过获取待量化的向量中每个维度的元素，进而将元素量化到该元素所属维度对应的区间，获得待量化的向量的量化信息，由此，可实现对高维向量进行量化处理，且采用量化后的信息能够提高图像检索中的检索准确率。The vector quantization method of the embodiment obtains the quantization information of the vector to be quantized by obtaining the element of each dimension in the vector to be quantized, and then quantizing the element to the interval corresponding to the dimension to which the element belongs, thereby achieving high The dimensional vector is quantized, and the quantized information can improve the retrieval accuracy in image retrieval.

图7示出了本发明一实施例提供的向量量化方法的流程示意图，如图7所示，本实施例的向量量化方法如下所述。FIG. 7 is a schematic flowchart diagram of a vector quantization method according to an embodiment of the present invention. As shown in FIG. 7, the vector quantization method in this embodiment is as follows.

701、获取待量化的向量中每个维度的元素。701. Obtain an element of each dimension in the vector to be quantized.

702、根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度。702. Obtain an interval corresponding to each dimension according to a preset training data set, where each dimension is each dimension in the to-be quantized vector.

在本实施例中，每一维度对应的区间均是不重叠的。In this embodiment, the intervals corresponding to each dimension are not overlapping.

举例来说，步骤701可包括下述的图中未示出的子步骤：For example, step 701 can include sub-steps not shown in the following figures:

7021、获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化的向量的维度相同。7021. Acquire an element set of each dimension of all vectors in the training data set, where the element set includes elements of all vectors in the dimension, and all the vectors in the training data set have the same dimension and the same dimension as the vector to be quantized.

例如，选t个n维向量X₁，X₂，…，X_t作为训练数据集。对于第i个维度，统计这t个向量在此维度上所有的取值，不妨设为，A＝{a₁，a₂，…，a_t}。其中， a₁≤a₂≤…≤a_t。For example, t n-dimensional vectors X ₁ , X ₂ , . . . , X _{t are selected} as training data sets. For the i-th dimension, count all the values of the t vectors in this dimension, and set it to A={a ₁ , a ₂ ,..., a _t }. Where a ₁ ≤ a ₂ ≤ ... ≤ a _t .

7022、根据预设距离差和量化误差，获取用于划分区间的目标函数。7022. Acquire an objective function for dividing the interval according to the preset distance difference and the quantization error.

7023、根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。7023. Perform an interval division of the element set of each dimension according to the objective function, and obtain an interval corresponding to the dimension.

具体地，枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Specifically, the boundary value of the interval division in the set of elements is enumerated, and the boundary value of each of the interval divisions is used as a parameter value of the objective function, and the value corresponding to the objective function is determined and recorded;

选择使目标函数取得极值(如最小值)的边界值划分该维度，获得该维度对应的区间。Selecting the boundary value that makes the objective function obtain the extreme value (such as the minimum value) divides the dimension to obtain the interval corresponding to the dimension.

在一个具体的例子中，对于一个维度的元素集合A的区间划分，用一个目标函数来判定该维度的元素集合A的区间划分的好坏。In a specific example, for the interval division of the element set A of one dimension, an objective function is used to determine the interval division of the element set A of the dimension.

具体中，假定将元素集合A划分成3个区间。Specifically, it is assumed that the element set A is divided into three sections.

对于区间划分如下说明：For the interval division as follows:

A₁＝{a₁，a₂，…，a(l₁)}A ₁ ={a ₁ ,a ₂ ,...,a(l ₁ )}

A₂＝{a(l₁₊₁)，a(l₁₊₂)，…，a(l₂)}A ₂ ={a(l ₁₊₁ ), a(l ₁₊₂ ),...,a(l ₂ )}

A₃＝{a(l₂₊₁)，…，a_t}。A ₃ ={a(l ₂₊₁ ),...,a _t }.

本实施例中采用一个目标函数F判定维度中区间划分质量的好坏。In this embodiment, an objective function F is used to determine the quality of the interval division in the dimension.

目标函数的信息如下：The information of the objective function is as follows:

当前，current,

距离差定义成：The distance difference is defined as:

量化误差定义成：MSE＝Σ_i(a_i-Q(a_i))²； The quantization error is defined as: MSE = Σ _i (a _{i -} Q(a _i )) ² ;

其中，a_i被量化到的区间的均值为Q(a_i)。即，如果a_i对应的区间为A_j，Q(a_i)代表A_j中所有元素的平均值。Wherein, the mean of the interval to which a _i is quantized is Q(a _i ). That is, if the interval corresponding to a _i is A _j , Q(a _i ) represents the average of all elements in A _j .

t是一个放缩因子，调整d(Q(a_i)，Q(a_j))和d(V(a_i)，V(a_j))的差值的绝对值大小。t is a scaling factor that adjusts the absolute value of the difference between d(Q(a _i ), Q(a _j )) and d(V(a _i ), V(a _j )).

V(a_i)代表元素a_i所属的向量。即，如果a_i属于X_j,那么V(a_i)＝X_j。V(a _i ) represents the vector to which the element a _i belongs. That is, if a _i belongs to X _j , then V(a _i )=X _j .

最后，目标函数由MSDE和MSE的一个组合构成，即：Finally, the objective function consists of a combination of MSDE and MSE, namely:

F＝λ*MSDE+μ*MSEF=λ*MSDE+μ*MSE

λ和μ的取值可根据实际需要选择。举例来说，(λ＝1，μ＝0)，(λ＝0，μ＝1)，(λ＝0.5，μ＝0.5)等。The values of λ and μ can be selected according to actual needs. For example, (λ=1, μ=0), (λ=0, μ=1), (λ=0.5, μ=0.5), and the like.

寻找能够使上述定义的目标函数取得极值的区间的边界值。即，寻找一个使F取得极小值的划分方式。Find the boundary value of the interval in which the objective function defined above can obtain the extreme value. That is, look for a division method that makes F take a minimum value.

实际中，如果A₁＝{a₁，a₂，…，a(s₁)},A₂＝{a(s₁₊₁)，a(s₁₊₂)，…，a(s₂)}，A3＝{a(s₂₊₁)，…，a_t}对应的F最小，那么选a(s₁)和a(s₂)对应的划分方式。In practice, if A ₁ ={a ₁ , a ₂ ,..., a(s ₁ )}, A ₂ ={a(s ₁₊₁ ), a(s ₁₊₂ ),...,a(s ₂ ) }, A3={a(s ₂₊₁ ),..., a _t } The corresponding F is the smallest, then the division manner corresponding to a(s ₁ ) and a(s ₂ ) is selected.

选用上述使目标函数取得极值的划分方案来确定维度的区间。The division of the dimension is determined by using the above-described division scheme in which the objective function obtains the extreme value.

使用上述的a(s₁)和a(s₂)将维度划分成3段。The dimensions are divided into 3 segments using a(s ₁ ) and a(s ₂ ) described above.

在本实施例中，查找所述元素集合中区间划分的边界值，将所述区间划分的边界值作为所述目标函数的参数值，确定所述目标函数的取值是否为极值；如果所述目标函数的取值为极值，则采用该区间划分的边界值划分该维度的元素集合中的元素，获得该维度对应的区间，举例如下：In this embodiment, searching for a boundary value of the interval division in the set of elements, and using the boundary value of the interval division as a parameter value of the objective function, determining whether the value of the objective function is an extreme value; If the value of the objective function is an extreme value, the boundary value of the interval is used to divide the elements in the element set of the dimension to obtain the interval corresponding to the dimension, as follows:

可枚举每个划分边界的取值，具体中，假定将A＝{a₁，a₂，…，a_t}划分成3个区间A₁＝{a₁，a₂，…，a_(l1)}The value of each partition boundary can be enumerated. Specifically, it is assumed that A={a ₁ , a ₂ , . . . , a _t } is divided into three intervals A ₁ ={a ₁ , a ₂ , . . . , a _{(l1 )} }

A₂＝{a_(l1+1)，a_(l1+2)，…，a_(l2)}A ₂ ={a _(l1+1) , a _(l1+2) ,...,a _(l2) }

A₃＝{a_(l2+1)，…，a_t}。A ₃ ={a _(l2+1) ,...,a _t }.

那么，枚举l1和l2的取值。1≤l1≤t，l1<l2≤t。Then, enumerate the values of l1 and l2. 1 ≤ l1 ≤ t, l1 < l2 ≤ t.

对于每一组取值，带入目标函数计算函数值。当l1和l2确定后，计算对应F的取值。For each set of values, the objective function is calculated to calculate the function value. After l1 and l2 are determined, the value of the corresponding F is calculated.

枚举过程中，记录下所有l1和l2组合对应的F的取值。During the enumeration process, record the values of F corresponding to all l1 and l2 combinations.

选所有取值中最优的一组来划分维度。假如某组l1＝p，l2＝q使F最小。那么选用p和q将维度划分成三部分，Select the best of all the values to divide the dimension. If a group l1=p, l2=q minimizes F. Then use p and q to divide the dimension into three parts.

A₁＝{a₁，a₂，…，a_p}A ₁ ={a ₁ ,a ₂ ,...,a _p }

A₂＝{a_(p+1)，a_(p+2)，…，a_q}A ₂ ={a _(p+1) , a _(p+2) ,...,a _q }

A₃＝{a_(q+1)，a_(q+2)，…，a_t}。A ₃ ={a _(q+1) , a _(q+2) ,..., a _t }.

703、根据获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息。703. Quantify, according to the interval corresponding to each dimension, the element to the interval corresponding to the dimension to which the element belongs, and obtain the quantization information of the vector to be quantized.

本实施例中的量化可理解为是一种分组，量化后也就是意味着被分到了特定的组，而量化信息就是所属组的编号或者用来表示该组的符号。The quantization in this embodiment can be understood as a grouping, and after quantization, it means that it is assigned to a specific group, and the quantization information is the number of the group to which it belongs or the symbol used to represent the group.

例如，假设是量化第i维的元素yi,将第i维的元素分为三组，(-∞,-1]，(-1，1]，(1，∞)，每个组编号分别是1，2，3。对于一个yi＝0.5，它被分到(-1，1]，这个过程是量化。它被编码成2，这就是量化信息。For example, suppose that the element i of the i-th dimension is quantized, and the elements of the i-th dimension are divided into three groups, (-∞, -1], (-1, 1], (1, ∞), and each group number is 1, 2, 3. For a yi = 0.5, it is assigned to (-1, 1), the process is quantization. It is encoded as 2, which is the quantitative information.

另外，需要说明的是，原始向量之间的相关性体现在目标函数中的

这一部分中。在寻找目标函数的极值时，不仅仅依赖对应维度的元素集合中元素的分布，还会利用维度对应的向量之间的距离关系，为此本实施例中可称之为原始向量之间的相关性。In addition, it should be noted that the correlation between the original vectors is reflected in the objective function.

In this part. When searching for the extremum of the objective function, it not only depends on the distribution of the elements in the element set of the corresponding dimension, but also uses the distance relationship between the vectors corresponding to the dimension. For this reason, in this embodiment, it can be called between the original vectors. Correlation.

由此，本实施例中的向量量化方法能够结合原始向量之间的相关性，进而对每一维度的每一元素进行量化，进而在向量的维度很高时也不会破坏向量的空间结构，由此，采用量化后的信息在图像检索中检索准确率提高。Therefore, the vector quantization method in this embodiment can combine the correlation between the original vectors, and then quantize each element of each dimension, so that the spatial structure of the vector is not destroyed when the dimension of the vector is high. Thus, the accuracy of the search in the image search is improved by using the quantized information.

另外，本发明实施例还提供一种使用上述图6或图7所示的向量量化方法在图像检索中使用的方法如下：In addition, an embodiment of the present invention further provides a method for using the vector quantization method shown in FIG. 6 or FIG. 7 in image retrieval as follows:

M01、获取一幅图像，并提取一个描述图像视觉特性的高维向量。M01, acquire an image, and extract a high-dimensional vector describing the visual characteristics of the image.

M02、统计高维向量每个维度的数值分布，基于分布信息给每个维度量化。M02. The numerical distribution of each dimension of the statistical high-dimensional vector is quantized for each dimension based on the distribution information.

M03、将每个维度量化后的信息进行二值编码，从而得到高维向量的量化信息。M03: Perform binary coding on the quantized information of each dimension to obtain quantized information of the high-dimensional vector.

M04、选择与二值编码对应的距离度量方法度量量化后的向量之间的相似性。M04, selecting a distance metric corresponding to the binary coding to measure the phase between the quantized vectors Similarity.

上述方法在图像检索过程中可提高图像检索中的检索准确率。The above method can improve the retrieval accuracy in image retrieval in the image retrieval process.

图8示出了本发明一实施例提供的向量量化装置的结构示意图，如图8所示，本实施例的向量量化装置包括：元素获取单元81、量化单元82；FIG. 8 is a schematic structural diagram of a vector quantization apparatus according to an embodiment of the present invention. As shown in FIG. 8, the vector quantization apparatus of this embodiment includes: an element obtaining unit 81 and a quantization unit 82;

元素获取单元81用于获取待量化的向量中每个维度的元素；The element obtaining unit 81 is configured to acquire an element of each dimension in the vector to be quantized;

量化单元82用于根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息；The quantization unit 82 is configured to quantize the element to the interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtain the quantization information of the vector to be quantized;

在一种具体的例子中，上述向量量化装置还可包括下述的图中未示出的维度确定单元，该维度确定单元用于获取所述待量化向量的维度，根据所述维度，获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。In a specific example, the vector quantization apparatus may further include a dimension determining unit not shown in the following figure, the dimension determining unit is configured to acquire a dimension of the to-be-quantized vector, according to the dimension, acquiring and The section information corresponding to the dimension, the section information includes: a section corresponding to each dimension in the dimension acquired in advance.

在另一种具体的例子中，向量量化装置还可包括下述的图中未示出的维度划分单元，该维度划分单元可为离线完成过程，用于根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度；In another specific example, the vector quantization apparatus may further include a dimension division unit not shown in the following figure, and the dimension division unit may be an offline completion process for training each acquisition according to a preset training data set. Intervals corresponding to the dimensions, each of the dimensions being each of the dimensions to be quantized;

例如，维度划分单元具体用于，获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化的向量的维度相同；For example, the dimension dividing unit is specifically configured to acquire a set of elements of each dimension of all the vectors in the training data set, where the set of elements includes elements of all the vectors in the dimension, and the dimensions of all the vectors in the training data set are the same, and The dimensions of the quantized vectors are the same;

或者，维度划分单元具体用于，获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Or the dimension dividing unit is specifically configured to: acquire an element set of each dimension of all the vectors in the training data set, where the element set includes elements of all the vectors in the dimension, and all the vectors in the training data set have the same dimension, and The dimensions of the quantization vector are the same;

本实施例的向量量化装置，可实现对高维向量进行量化处理，且采用量化后的信息能够提高图像检索中的检索准确率。The vector quantization apparatus of this embodiment can implement quantization processing on high-dimensional vectors, and the quantized information can improve the retrieval accuracy in image retrieval.

本发明的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。In the description of the invention, numerous specific details are illustrated. However, it is understood that the embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of the description.

类似地，应当理解，为了精简本发明公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释呈反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。Similarly, the various features of the invention are sometimes grouped together into a single embodiment in the above description of the exemplary embodiments of the invention in order to the , diagram, or description of it. However, the method of the present disclosure should not be construed as reflecting that the claimed invention requires more features than those specifically recited in the appended claims. Rather, as the following claims reflect, inventive aspects reside in less than all features of the single embodiments disclosed herein. Therefore, the claims following the specific embodiments are hereby explicitly incorporated into the embodiments, and each of the claims as a separate embodiment of the invention.

本领域技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在于该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是互相排斥之处，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and placed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components. In addition to the fact that at least some of such features and/or processes or units are mutually exclusive, any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed or All processes or units of the device are combined. Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.

此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features that are included in other embodiments and not in other features, combinations of features of different embodiments are intended to be within the scope of the present invention. Different embodiments are formed and formed. For example, in the following claims, any one of the claimed embodiments may be use.

本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。The various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It is to be noted that the above-described embodiments are illustrative of the invention and are not intended to be limiting, and that the invention may be devised without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as a limitation. The word "comprising" does not exclude the presence of the elements or steps that are not recited in the claims. The word "a" or "an" The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by the same hardware item. The use of the words first, second, and third does not indicate any order. These words can be interpreted as names.

最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围，其均应涵盖在本发明的权利要求和说明书的范围当中。 Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. The scope is intended to be included within the scope of the claims and the description of the invention.

Claims

一种高维向量的哈希方法，其特征在于，包括：A high-dimensional vector hashing method, comprising:

采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素；The compressed high-dimensional vector is quantized by using a scalar quantization method to obtain quantization information of each element, wherein each element is an element of each dimension in the compressed high-dimensional vector;

将所述每个元素的量化信息编码成二值码；Encoding the quantized information of each element into a binary code;

按照预设规则将所有二值码进行组合，获得二值向量。All binary codes are combined according to a preset rule to obtain a binary vector.
根据权利要求1所述的方法，其特征在于，采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，包括：The method according to claim 1, wherein the compressed high-dimensional vector is quantized by using a scalar quantization method to obtain quantitative information of each element, including:

获取压缩后的高维向量中每个维度的元素；Obtaining the elements of each dimension in the compressed high-dimensional vector;

根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得每个元素的量化信息；And quantizing the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtaining quantization information of each element;

所述每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.
根据权利要求2所述的方法，其特征在于，根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：The method according to claim 2, wherein the method further comprises: before the interval corresponding to the dimension to which the element belongs, before the interval corresponding to each dimension of the pre-acquisition, the method further comprises:

获取所述压缩后高维向量的维度，根据所述维度获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。Acquiring the dimension of the compressed high-dimensional vector, and acquiring the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension of the dimension acquired in advance.
根据权利要求2或3所述的方法，其特征在于，根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：The method according to claim 2 or 3, wherein the method further comprises: before the interval corresponding to the dimension to which the element belongs, before the interval corresponding to each dimension of the pre-acquisition, the method further comprises:

根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述高维向量中的每个维度；Obtaining, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the high-dimensional vector;

每一维度对应的区间均是不重叠的。The intervals corresponding to each dimension are not overlapping.
根据权利要求4所述的方法，其特征在于，根据预设的训练数据集训练获取每个维度对应的区间，包括：The method according to claim 4, wherein the obtaining the interval corresponding to each dimension according to the preset training data set comprises:

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，均等于所述压缩后的高维向量的维度；Obtaining a set of elements for each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, all of the vectors in the training data set having the same dimension, equal to The dimension of the compressed high dimensional vector;

根据预设距离差和量化误差，获取用于划分区间的目标函数；Obtaining an objective function for dividing the interval according to the preset distance difference and the quantization error;

根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。According to the objective function, the element set of each dimension is divided into sections to obtain an interval corresponding to the dimension.
根据权利要求5所述的方法，其特征在于，根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间，包括：The method according to claim 5, wherein the element set of each dimension is divided into sections according to the objective function, and the interval corresponding to the dimension is obtained, including:

枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Enumerating a boundary value of the interval division in the set of the elements, using a boundary value of each of the interval divisions as a parameter value of the objective function, determining and recording a value corresponding to the target function;

选择使目标函数取得极值的边界值划分该维度，获得该维度对应的区间。The boundary value of the extreme value of the objective function is selected to divide the dimension, and the interval corresponding to the dimension is obtained.
根据权利要求1至6任一所述的方法，其特征在于，按照预设规则将所有二值码进行组合，获得二值向量，包括：The method according to any one of claims 1 to 6, wherein all binary codes are combined according to a preset rule to obtain a binary vector, including:

将所有二值码按照每一二值码对应的维度序号顺序拼接，获得二值向量。All binary codes are spliced in the order of the dimension numbers corresponding to each binary code to obtain a binary vector.
一种高维向量的近邻查询方法，其特征在于，包括：A high-dimensional vector neighbor query method, which comprises:

采用权利要求1至7任一所述的方法获取待处理的高维向量的二值向量；Acquiring a binary vector of a high dimensional vector to be processed by using the method according to any one of claims 1 to 7;

获取所述二值向量的哈希二值码；Obtaining a hash binary code of the binary vector;

根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。And according to the coding manner of the hash binary code, selecting a metric distance corresponding to the coding mode, and using the metric distance to measure a similarity of the binary vector corresponding to the other high-dimensional vectors.
一种高维向量的哈希装置，其特征在于，包括：A high-dimensional vector hashing device, comprising:

量化单元，用于采用标量量化方式对压缩后的高维向量进行量化处理，获取每个元素的量化信息，其中，所述每个元素为压缩后的高维向量中每个维度的元素；And a quantization unit, configured to perform quantization processing on the compressed high-dimensional vector by using a scalar quantization method, to obtain quantization information of each element, wherein each of the elements is an element of each dimension in the compressed high-dimensional vector;

编码单元，用于将所述每个元素的量化信息编码成二值码；a coding unit, configured to encode the quantization information of each element into a binary code;

组合单元，用于按照预设规则将所有二值码进行组合，获得二值向量。A combination unit is configured to combine all the binary codes according to a preset rule to obtain a binary vector.
一种高维向量的近邻查询装置，其特征在于，包括：A high-dimensional vector neighbor query device, comprising:

二值向量获取单元，用于采用上述权利要求9所述的装置获取待处理的高维向量的二值向量；a binary vector obtaining unit, configured to acquire a binary vector of a high-dimensional vector to be processed by using the apparatus according to claim 9;

哈希二值码获取单元，用于获取所述二值向量的哈希二值码； a hash binary value obtaining unit, configured to obtain a hash binary code of the binary vector;

相似度度量单元，用于根据所述哈希二值码的编码方式，选择与该编码方式对应的度量距离，采用所述度量距离度量所述二值向量与其他高维向量对应的二值向量的相似度。a similarity measure unit, configured to select a metric distance corresponding to the coding mode according to the coding manner of the hash binary code, and measure the binary vector corresponding to the other high-dimensional vector by using the metric distance Similarity.
一种向量量化方法，其特征在于，包括：A vector quantization method, comprising:

获取待量化的向量中每个维度的元素；Get the elements of each dimension in the vector to be quantized;

根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息；And quantizing the element to an interval corresponding to the dimension to which the element belongs according to the interval corresponding to each dimension acquired in advance, and obtaining the quantization information of the vector to be quantized;

其中，所述每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.
根据权利要求11所述的方法，其特征在于，所述根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：The method according to claim 11, wherein the method further comprises: before the interval corresponding to the dimension to which the element belongs, according to the interval corresponding to each dimension acquired in advance, the method further includes:

获取所述待量化向量的维度，根据所述维度，获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。Acquiring the dimension of the vector to be quantized, and acquiring the section information corresponding to the dimension according to the dimension, where the section information includes: a section corresponding to each dimension of the dimension acquired in advance.
根据权利要求11或12所述的方法，其特征在于，所述根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间之前，所述方法还包括：The method according to claim 11 or 12, wherein the method further comprises: before the interval corresponding to the dimension to which the element belongs, according to the interval corresponding to each dimension acquired in advance, the method further comprises:

根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度；Obtaining, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the to-be quantized vector;

每一维度对应的区间均是不重叠的。The intervals corresponding to each dimension are not overlapping.
根据权利要求13所述的方法，其特征在于，根据预设的训练数据集训练获取每个维度对应的区间，包括：The method according to claim 13, wherein the obtaining the interval corresponding to each dimension according to the preset training data set comprises:

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Obtaining a set of elements of each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all the vectors in the training data set being the same, and being the same as the dimension of the vector to be quantized;

根据预设距离差和量化误差，获取用于划分区间的目标函数；Obtaining an objective function for dividing the interval according to the preset distance difference and the quantization error;

根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。According to the objective function, the element set of each dimension is divided into sections to obtain an interval corresponding to the dimension.
根据权利要求14所述的方法，其特征在于，根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间，包括：The method of claim 14 wherein, based on said objective function, The element set of each dimension is divided into sections, and the corresponding interval of the dimension is obtained, including:

枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Enumerating a boundary value of the interval division in the set of the elements, using a boundary value of each of the interval divisions as a parameter value of the objective function, determining and recording a value corresponding to the target function;

选择使目标函数取得极值的边界值划分该维度，获得该维度对应的区间。The boundary value of the extreme value of the objective function is selected to divide the dimension, and the interval corresponding to the dimension is obtained.
一种向量量化装置，其特征在于，包括：A vector quantization device, comprising:

元素获取单元，用于获取待量化的向量中每个维度的元素；An element obtaining unit, configured to acquire an element of each dimension in the vector to be quantized;

量化单元，用于根据预先获取的每个维度对应的区间，将所述元素量化到该元素所属维度对应的区间，获得所述待量化的向量的量化信息；a quantization unit, configured to quantize the element to an interval corresponding to a dimension to which the element belongs according to an interval corresponding to each dimension acquired in advance, to obtain quantization information of the vector to be quantized;

其中，所述每个维度对应的区间是根据预设的训练数据集训练获取的。The interval corresponding to each dimension is obtained according to a preset training data set.
根据权利要求16所述的装置，其特征在于，所述装置还包括：The device according to claim 16, wherein the device further comprises:

维度确定单元，用于获取所述待量化向量的维度，根据所述维度，获取与所述维度对应的区间信息，所述区间信息包括：预先获取的所述维度中每个维度对应的区间。a dimension determining unit, configured to acquire a dimension of the to-be-quantized vector, and obtain, according to the dimension, interval information corresponding to the dimension, where the section information includes: a section corresponding to each dimension in the dimension acquired in advance.
根据权利要求16或17所述的装置，其特征在于，所述装置还包括：The device according to claim 16 or 17, wherein the device further comprises:

维度划分单元，用于根据预设的训练数据集训练获取每个维度对应的区间，所述每个维度为所述待量化向量中的每个维度；a dimension dividing unit, configured to acquire, according to a preset training data set, a section corresponding to each dimension, where each dimension is each dimension in the to-be quantized vector;

每一维度对应的区间均是不重叠的。The intervals corresponding to each dimension are not overlapping.
根据权利要求18所述的装置，其特征在于，维度划分单元，具体用于The device according to claim 18, wherein the dimension dividing unit is specifically configured to

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Obtaining a set of elements of each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all the vectors in the training data set being the same, and being the same as the dimension of the vector to be quantized;

根据预设距离差和量化误差，获取用于划分区间的目标函数；Obtaining an objective function for dividing the interval according to the preset distance difference and the quantization error;

根据所述目标函数，将每一维度的元素集合进行区间划分，获得该维度对应的区间。According to the objective function, the element set of each dimension is divided into sections to obtain an interval corresponding to the dimension.
根据权利要求18所述的装置，其特征在于，维度划分单元，具体用于 The device according to claim 18, wherein the dimension dividing unit is specifically configured to

获取训练数据集中所有向量的每一维度的元素集合，所述元素集合包括该维度上所有向量的元素，所述训练数据集中所有向量的维度相同，且与待量化向量的维度相同；Obtaining a set of elements of each dimension of all vectors in the training data set, the set of elements including elements of all vectors in the dimension, the dimensions of all the vectors in the training data set being the same, and being the same as the dimension of the vector to be quantized;

根据预设距离差和量化误差，获取用于划分区间的目标函数；Obtaining an objective function for dividing the interval according to the preset distance difference and the quantization error;

枚举所述元素集合中区间划分的边界值，将每组所述区间划分的边界值作为所述目标函数的参数值，确定并记录所述目标函数对应的取值；Enumerating a boundary value of the interval division in the set of the elements, using a boundary value of each of the interval divisions as a parameter value of the objective function, determining and recording a value corresponding to the target function;

选择使目标函数取得极值的边界值划分该维度，获得该维度对应的区间。 The boundary value of the extreme value of the objective function is selected to divide the dimension, and the interval corresponding to the dimension is obtained.