CN111143510A - Searching method based on latent semantic analysis model - Google Patents

Abstract

The invention discloses a searching method based on a potential semantic analysis model, which comprises the following steps that character contents to be searched are received in an input frame, the searched contents are two or more than two sentences, adjectives in the searched contents are removed, then the searched contents are subjected to characteristic division, possibly-related characteristics in the contexts are captured by a GloVe model, designated meanings are juxtaposed, and the searched contents input by a user are subjected to group division by adopting a collaborative filtering mode; the method and the device can perform search to assist in judgment by judging the context, so that the accuracy of the search is improved.

Description

Searching method based on latent semantic analysis model

Technical Field

The invention belongs to the technical field of search, and particularly relates to a search method based on a potential semantic analysis model.

Background

The existing search algorithm decomposes sentences into 'a series of words', analyzes meanings of important words, and accordingly provides simple local search results, and because the model generates a single 'word embedding' representation for each word in a vocabulary table without auxiliary judgment according to context, the 'bank' may have the same representations in the 'bank destination' and the 'river bank', and the search results are confused, so that a search method based on a potential semantic analysis model is provided.

Disclosure of Invention

The invention aims to provide a searching method based on a latent semantic analysis model so as to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme: the searching method based on the latent semantic analysis model comprises the following steps:

s1, receiving the text content to be searched in the input box, wherein the searched content is two or more sentences;

s2, removing adjectives in the search content;

s3, carrying out feature division on the search content, capturing possibly related features in the context by the GloVe model, and juxtaposing the specified meanings;

and S4, performing group division on the search content input by the user by adopting a collaborative filtering mode.

Preferably, the GloVe model is a logarithmic bilinear model with a weighted least squares objective, the main intuition of which is a simple observation, i.e. the ratio of word-word co-occurrence probabilities is presented in coded form.

Preferably, in step S4, the collaborative filtering discovers the preferences of the user by mining the historical search behavior data of the user, groups the user based on different preferences and recommends the content with similar tastes, and the collaborative filtering recommendation algorithm is divided into two categories, which are a collaborative filtering algorithm based on the user and a collaborative filtering algorithm based on the goods.

Preferably, after the above steps are completed, SVD singular value decomposition is performed.

Preferably, the SVD singular value decomposition is matrix decomposition of the search inner diameter, and the erection matrix a is an m × n matrix, so we define the SVD of the matrix a as:

A＝U∑V^T

where U is a matrix of m x m and is a matrix of m x n, all 0 except the elements on the main diagonal, each element on the main diagonal being called a singular value, and V is a matrix of n x n. U and V are both west matrices, i.e. UTU ═ I and VTV ═ I are satisfied.

Compared with the prior art, the invention has the beneficial effects that:

the method and the device can perform search to assist in judgment by judging the context, so that the accuracy of the search is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention.

FIG. 1 is a schematic view of the present invention;

FIG. 2 is a graph of the results of an overview of the model calculations of the present invention;

FIG. 3 is a schematic diagram of the relationship of characters of the present invention;

FIG. 4 is a schematic diagram of the relationship between a city and a city zip code of the present invention;

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the description of the present invention, it should be noted that the terms "vertical", "upper", "lower", "horizontal", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention.

In the description of the present invention, it should also be noted that, unless otherwise explicitly specified or limited, the terms "disposed," "mounted," "connected," and "connected" are to be construed broadly and may, for example, be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; either directly or indirectly through intervening media, or through the communication between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

Referring to fig. 1-2, the present invention provides a technical solution: the searching method based on the latent semantic analysis model comprises the following steps:

s1, receiving the text content to be searched in the input box, wherein the searched content is two or more sentences;

s2, removing adjectives in the search content;

s3, then performing feature segmentation on the search content, and the GloVe model captures features that may be associated in context, juxtaposing the assigned meanings, for contextual understanding, e.g., considering co-occurrence probabilities of the target words "ice" and "steam" with various probe words in the vocabulary, ice appears more frequently in solid and steam in gas than in solid, both words often appear with water they share, while both words appear less frequently with irrelevant words, as shown in fig. 2, only in the ratio of probabilities noise from non-discriminative words such as water and fashion can cancel out, so that large values (much larger than 1) correlate well with ice-specific attributes and small values (much smaller than 1) correlate well with ice-specific attributes, characteristic of steam, so that probability ratio encodes some rough form of meaning associated with abstract concepts of thermodynamic phase, when features with relationships, such as "ice" and "fixed" or city name and corresponding city zip code, appear in the context, it may be convenient to further narrow the search results;

and S4, performing group division on the search content input by the user by adopting a collaborative filtering mode.

In this embodiment, it is preferable that the GloVe model is a logarithmic bilinear model with weighted least squares objective, the main intuition of the model is simple observation, i.e. the ratio of word-word co-occurrence probability is presented in the form of coded form, GloVe is for capturing subtle differences that distinguish related words in a quantitative way, such as men and women, for the model it is necessary to associate more than one number to word pairs, the vector difference between two word vectors is a natural and simple candidate for a larger set of discriminators, the GloVe is designed to capture as many meanings as possible as two words are specified in parallel, as shown in fig. 3 and 4;

the basic concept that can distinguish men from women, i.e. gender or gender, can be equivalently specified in various other word pairs, such as king and queen or siblings, and in order to mathematically state this observation we can expect that the vector differences between men (women), queen (queen) and siblings (sisters) may be roughly equal, but contextual decisions can be made by the relationship between them.

In this embodiment, preferably, in the step S4, the collaborative filtering discovers the preference of the user by mining the historical search behavior data of the user, groups the user based on different preferences and recommends the content with similar taste, and the collaborative filtering recommendation algorithm is divided into two categories, which are respectively a collaborative filtering algorithm based on the user and a collaborative filtering algorithm based on the goods.

The collaborative filtering algorithm based on the users finds out the likes (such as release, praise, content comment or share) of the articles by the users through the historical behavior data of the users, and measures and scores the likes. And calculating the relationship between the users according to attitudes and preference degrees of different users to the same article. Recommendations are made among users having the same preferences.

The relationship between the articles is obtained by calculating the scores of different users to different articles. And recommending similar articles for the user based on the relationship among the articles.

In this embodiment, after the above steps are completed, SVD singular value decomposition is preferably performed.

In this embodiment, preferably, the SVD singular value decomposition is a matrix decomposition of the search inner diameter, and the erection matrix a is an m × n matrix, so we define the SVD of the matrix a as:

A＝U∑V^T

where U is a matrix of m x m and is a matrix of m x n, all 0 except the elements on the main diagonal, each element on the main diagonal being called a singular value, and V is a matrix of n x n. U and V are western matrixes, namely UTU is equal to I, and VTV is equal to I;

the three matrices have very clear physical meanings, each row in the first matrix X represents a class of words with related meanings, each non-zero element of the matrix X represents the importance (or relevance) of each word in the class, and the larger the value, the more relevant the value. Each column in the last matrix Y represents the same topic type of articles, wherein each element represents the correlation of each article in the articles, and the middle matrix represents the correlation between the class words and the article types, so that only one singular value decomposition is carried out on the incidence matrix A, W can simultaneously complete the classification of the near meaning words and the classification of the articles.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (5)

1. The searching method based on the latent semantic analysis model is characterized by comprising the following steps of:

s1, receiving the text content to be searched in the input box, wherein the searched content is two or more sentences;

s2, removing adjectives in the search content;

s3, carrying out feature division on the search content, capturing possibly related features in the context by the GloVe model, and juxtaposing the specified meanings;

and S4, performing group division on the search content input by the user by adopting a collaborative filtering mode.

2. The latent semantic analysis model-based search method of claim 1, wherein: the GloVe model is a logarithmic bilinear model with a weighted least squares objective, the main intuition of which is a simple observation, i.e. the ratio of word-word co-occurrence probabilities is presented in coded form.

3. The latent semantic analysis model-based search method of claim 1, wherein: in step S4, collaborative filtering finds preferences of users by mining historical search behavior data of the users, groups the users are divided based on different preferences and content with similar tastes is recommended, and collaborative filtering recommendation algorithms are divided into two categories, namely, a collaborative filtering algorithm based on users and a collaborative filtering algorithm based on articles.

4. The latent semantic analysis model-based search method of claim 1, wherein: after the steps are completed, SVD singular value decomposition is carried out.

5. The latent semantic analysis model-based search method of claim 4, wherein: the SVD is to perform matrix decomposition on the inner diameter of the search, the erection matrix A is an m x n matrix, and then the SVD of the matrix A is defined as:

A＝U∑V^T

where U is a matrix of m x m and is a matrix of m x n, all 0 except the elements on the main diagonal, each element on the main diagonal being called a singular value, and V is a matrix of n x n. U and V are both west matrices, i.e. UTU ═ I and VTV ═ I are satisfied.

Images