TWI512502B - Method and system for generating custom language models and related computer program product - Google Patents
Method and system for generating custom language models and related computer program product Download PDFInfo
- Publication number
- TWI512502B TWI512502B TW098137647A TW98137647A TWI512502B TW I512502 B TWI512502 B TW I512502B TW 098137647 A TW098137647 A TW 098137647A TW 98137647 A TW98137647 A TW 98137647A TW I512502 B TWI512502 B TW I512502B
- Authority
- TW
- Taiwan
- Prior art keywords
- cluster
- clusters
- files
- user
- generating
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Description
本發明係關於語言模式。The present invention relates to language patterns.
語言模式係用於模式化一概率:在一給定詞組內的一串記號(例如字元(word)或字元(character))將出現在一語言中。舉例而言,語言模式係用於輸入方法,諸如但不限於輸入方法編輯器(IME)、自動語音辨識(ASR)、機器翻譯、手寫辨識及光學字元辨識(OCR)應用。通常使用一連鎖定律且計算一給定記號在w 在一給定串語境中的概率p(w |語境)來執行對一串記號在該詞組中之概率的模式化,其中該語境為在該給定記號w 之前之該串中的該等記號。The language pattern is used to pattern a probability: a string of tokens (such as a word or character) within a given phrase will appear in a language. For example, language mode is used for input methods such as, but not limited to, Input Method Editor (IME), Automatic Speech Recognition (ASR), Machine Translation, Handwriting Recognition, and Optical Character Recognition (OCR) applications. Law and usually a chain of a given computing w mark in a given sequence context probability p | performs schematically a string of tokens in the probability of a phrase (w context), wherein the context These are the tokens in the string before the given token w .
在一n元語言模式中,文本中的n個連續記號形成為n元,且一當前字元z的概率(例如)取決於n-1個先前字元的概率,例如p(zi |語境)=p(zi |zi-n+1 ,zi-n+2 ,...,zi-1 )。一n元具有一階數,其為在該n元中的記號數目。舉例而言,一個一元(或一元)包含一個記號;一二元(或雙元)包含兩個記號。In an n-gram language mode, n consecutive tokens in the text are formed as n-ary, and the probability of a current character z depends, for example, on the probability of n-1 previous characters, such as p(z i |境) = p(z i |z i-n+1 , z i-n+2 ,..., z i-1 ). An n-ary has a first order, which is the number of tokens in the n-ary. For example, a unary (or unary) contains a token; a binary (or binary) contains two tokens.
n元在文本(例如在一句子中的字元)中的概率分佈很大程度上取決於語境,亦可以一更一般的意義檢視該分佈。舉例而言,特定n元在文本中的概率分佈可取決於該文本將表達之一主題或該文本出現之一領域。「籃球」出現在一體育文章中的概率係大於「籃球」出現在一金融文章中的概率。另外,不同的使用者可使用(例如青睞)不同的字元(例如)來表達相同的想法。西班牙的使用者可能使用「足球(football)」,而美國的使用者可能使用「足球(soccer)」。因此,n元在文本中的概率分佈可為依賴於使用者且依賴於領域兩者。The probability distribution of an n-ary in a text (such as a character in a sentence) is largely dependent on the context, and the distribution can also be viewed in a more general sense. For example, the probability distribution of a particular n-ary in the text may depend on the field in which the text will express one of the topics or the text appears. The probability that "basketball" appears in a sports article is greater than the probability that "basketball" will appear in a financial article. In addition, different users can use (eg, favor) different characters (for example) to express the same idea. Spanish users may use "football", while American users may use "soccer". Thus, the probability distribution of n-arys in text can be both user-dependent and dependent on the domain.
習知的輸入方法使用一般語言模式。舉例而言,一單一語言模式可為所有使用者使用。作為另一實例,其可使用相同的語言模式或自所有領域(例如一電腦系統領域、地理領域)之訓練資料而產生相同的語言模式。一個一般語言模式不可為所有輸入方法使用最佳化。The conventional input method uses a general language mode. For example, a single language mode is available to all users. As another example, the same language pattern can be generated using the same language pattern or training material from all fields (eg, a computer system domain, a geographic domain). A general language mode cannot be optimized for all input methods.
本說明書描述關於習慣語言模式之技術。This specification describes techniques for custom language patterns.
一般而言,描述於本說明書中之本發明之一態樣可體現在諸多方法中,該等方法包含動作:接收一文件集合;將該等文件叢集化為一或多個叢集;為該一個或多個叢集之各叢集產生一叢集向量;產生一與一目標設定檔相關的目標向量;比較該目標向量與該等叢集向量之各者;基於該比較選擇該一個或多個叢集之一者或多者;及使用來自該一個或多個所選叢集之文件產生一語言模式。此態樣之其他實施例包含對應的系統、裝置及電腦程式產品。In general, one aspect of the invention described in this specification can be embodied in a number of methods, the methods comprising the acts of: receiving a collection of files; clustering the files into one or more clusters; Or each cluster of the plurality of clusters generates a cluster vector; generates a target vector associated with a target profile; compares the target vector with each of the cluster vectors; and selects one of the one or more clusters based on the comparison Or more; and generating a language pattern using files from the one or more selected clusters. Other embodiments of this aspect include corresponding systems, devices, and computer program products.
此等及其他實施例可視情況包含以下特徵之一者或多者。產生一語言模式包含識別來自該一個或多個所選叢集之該等文件之一n元集合。各n元具有出現在來自該一個或多個所選叢集之該等文件中之一對應的相對頻率。各叢集係與一主題或領域相關。各叢集向量表示一對應叢集之一質心。叢集化該等文件包含隨機地指派各文件至一叢集,並為各叢集迭代計算一質心,且基於一最近質心重新指派各文件至一叢集直到收斂。該目標向量包含一詞頻向量。該目標向量係自一使用者詞典或使用者輸入之至少一者產生。該比較包含計算介於該目標向量與該等叢集向量各者之間之一餘弦相似性。該方法進一步包含接收一個或多個額外文件,且基於該一個或多個額外文件更新該語言模式。These and other embodiments may optionally include one or more of the following features. Generating a language pattern includes identifying an n-ary set of one of the files from the one or more selected clusters. Each n-ary has a relative frequency that corresponds to one of the files from the one or more selected clusters. Each cluster is associated with a topic or field. Each cluster vector represents one of the centroids of a corresponding cluster. Clustering the files involves randomly assigning each file to a cluster and calculating a centroid for each cluster iteration, and reassigning each file to a cluster based on a recent centroid until convergence. The target vector contains a word frequency vector. The target vector is generated from at least one of a user dictionary or user input. The comparison includes calculating a cosine similarity between the target vector and each of the cluster vectors. The method further includes receiving one or more additional files and updating the language mode based on the one or more additional files.
一般而言,描述於本說明書中之本發明的另一態樣可體現在諸多方法中,該等方法包含動作:接收一文件集合;將該等文件叢集化為一或多個叢集;為該一個或多個叢集之各叢集產生一叢集向量;產生一與一目標設定檔相關的目標向量;比較該目標向量與該等叢集向量之各者;及基於該比較而選擇該一個或多個一般叢集之一者或多者。此態樣之其他實施例包含對應的系統、裝置及電腦程式產品。In general, another aspect of the invention described in this specification can be embodied in a number of methods, the methods comprising the acts of: receiving a collection of files; clustering the files into one or more clusters; Each cluster of one or more clusters generates a cluster vector; generates a target vector associated with a target profile; compares the target vector with each of the cluster vectors; and selects the one or more based on the comparison One or more of the clusters. Other embodiments of this aspect include corresponding systems, devices, and computer program products.
此等及其他實施例可視情況包含以下特徵之一者或多者。該方法進一步包含使用來自該一個或多個所選一般叢集之文件來產生一語言模式。These and other embodiments may optionally include one or more of the following features. The method further includes generating a language pattern using the files from the one or more selected general clusters.
一般地,描述於本說明書中之本發明的另一態樣可體現在諸多方法中,該等方法包含動作:接收識別該使用者之一使用者輸入;識別對應於該使用者之一使用者設定檔;使用該經識別設定檔來產生一使用者特定語言模式;及發送該使用者特定語言模式至一第一客戶端。此態樣之其他實施例包含對應的系統、裝置及電腦程式產品。In general, another aspect of the invention described in this specification can be embodied in a number of methods, the methods comprising: receiving a user input identifying one of the users; identifying a user corresponding to the user a profile; using the identified profile to generate a user-specific language mode; and transmitting the user-specific language mode to a first client. Other embodiments of this aspect include corresponding systems, devices, and computer program products.
此等及其他實施例可視情況包含以下特徵之一者或多者。該方法進一步包含發送該使用者特定語言模式至一第二客戶端。該方法進一步包含接收一文件集合;將該等文件叢集化為一或多個叢集;為該一個或多個叢集之各叢集產生一叢集向量;產生與該使用者設定檔相關的一目標向量;比較該目標向量與該等叢集向量之各者;及基於該比較選擇該一個或多個叢集之一者或多者。產生一使用者特定語言模式包含識別來自該一個或多個所選叢集之該等文件的一n元集合。各n元具有出現在來自該一個或多個所選叢集之該等文件中的一對應相對頻率。These and other embodiments may optionally include one or more of the following features. The method further includes transmitting the user specific language mode to a second client. The method further includes receiving a set of files; clustering the files into one or more clusters; generating a cluster vector for each of the one or more clusters; generating a target vector associated with the user profile; Comparing the target vector to each of the cluster vectors; and selecting one or more of the one or more clusters based on the comparison. Generating a user-specific language pattern includes identifying an n-ary set of files from the one or more selected clusters. Each n-ary has a corresponding relative frequency that appears in the files from the one or more selected clusters.
一般地,描述於本說明書中之本發明的另一態樣可體現在諸多方法中,該等方法包含動作:接收一個或多個文件之一第一集合;基於一個或多個文件之該第一集合產生一設定檔;接收一個或多個文件之一第二集合;基於一個或多個文件之該第二集合及該設定檔產生一習慣語言模式;及發送該習慣語言模式至一客戶端。此態樣之其他實施例包含對應的系統、裝置及電腦程式產品。In general, another aspect of the invention described in this specification can be embodied in a number of methods that include an action of receiving a first set of one or more files; based on the one or more files a set of generating a profile; receiving a second set of one or more files; generating a custom language mode based on the second set of the one or more files and the profile; and transmitting the custom language mode to a client . Other embodiments of this aspect include corresponding systems, devices, and computer program products.
此等及其他實施例可視情況包含以下特徵之一者或多者。一個或多個文件之該第一集合包含一使用者詞典或表示該使用者輸入行為的使用者輸入之至少一者。These and other embodiments may optionally include one or more of the following features. The first set of one or more files includes at least one of a user dictionary or user input indicating the user input behavior.
描述於本說明書中之本發明的特定實施例可經實施以實現以下優點之一者或多者。將訓練資料非監督分類為諸多子集可(i)增加可考慮用於訓練一習慣語言模式之資料量(例如一語料庫的大小);及(ii)減少使用者干預量,藉此減少使用者錯誤。Particular embodiments of the invention described in this specification can be implemented to realize one or more of the following advantages. Unsupervised classification of training data into a number of subsets may (i) increase the amount of data that can be considered for training a customary language model (eg, the size of a corpus); and (ii) reduce the amount of user intervention, thereby reducing users error.
可產生用於特定輸入方法使用(例如對於特定使用者或領域)之一習慣語言模式以增加該使用方法之準確性、精確性及對該輸入方法之調用,藉此降低一輸入錯誤率。另外,一習慣語言模式可經調適於一使用者行為(例如基於由該使用者輸入、表示該使用者行為的文件或一使用者詞典而產生/改進),藉此進一步增加準確性、精確性及調用,並且降低該輸入錯誤率。此外,習慣語言模式可為依賴於主題且適合於一使用者當前輸入之一主題。舉例而言,一使用者可基於感興趣的一主題從儲存在一伺服器上的複數個不同主題的習慣語言模式中選擇。因為一習慣語言模式可自所有可用訓練資料之一子集得到訓練,所以該習慣語言模式可具有小於一通用語言模式的一記憶體大小。因此,使用該習慣語言模式減小(i)使用該習慣語言模式之一輸入方法的記憶體佔據;(ii)用於儲存該習慣語言模式之記憶體數量;及(iii)提供該習慣語言模式至一使用者需要的頻寬/時間數量。One of the custom language patterns used for a particular input method (eg, for a particular user or domain) can be generated to increase the accuracy, accuracy, and call to the input method, thereby reducing an input error rate. In addition, a custom language mode can be adapted to a user behavior (eg, generated/improved based on a file entered by the user, representing the user's behavior, or a user dictionary), thereby further increasing accuracy and accuracy. And call, and reduce the input error rate. In addition, the custom language mode can be subject-dependent and suitable for a user to currently input one of the themes. For example, a user may select from a plurality of differently-language custom language patterns stored on a server based on a topic of interest. Because a custom language mode can be trained from a subset of all available training materials, the custom language mode can have a memory size that is less than a common language mode. Therefore, using the custom language mode reduces (i) memory occupancy using one of the custom language modes of input methods; (ii) the number of memories used to store the custom language mode; and (iii) providing the custom language mode The number of bandwidth/times required by a user.
描述於本說明書中之本發明的一個或多個實施例細節係在隨附圖式及以下描述中提出。由該描述、該等圖式及該等技術方案,本發明之其他特徵、態樣及優點將變得顯然。The details of one or more embodiments of the invention described in the specification are set forth Other features, aspects, and advantages of the present invention will be apparent from the description, the drawings and the claims.
圖1為繪示用於產生習慣語言模式124之一實例系統100的一圖表。該系統100包含一第一客戶端112(例如一桌面式電腦)、一第二客戶端114(例如一蜂巢式電話)及一伺服器120。該伺服器120可包含目標設定檔122、習慣語言模式124及訓練資料126(例如一註釋的語料庫)。FIG. 1 is a diagram showing an example system 100 for generating a custom language pattern 124. The system 100 includes a first client 112 (e.g., a desktop computer), a second client 114 (e.g., a cellular phone), and a server 120. The server 120 can include a target profile 122, a custom language mode 124, and a training profile 126 (eg, a corpus of annotations).
作為產生習慣語言模式之一概述,該訓練資料126可分類為該訓練資料126之諸多集合(例如一個或多個文件集合)。該伺服器120可接收識別該使用者(例如在該第一客戶端112上的一第一使用者、在該第一客戶端112上的一第二使用者、在該第二客戶端114上的該第一使用者)的一使用者輸入。舉例而言,該伺服器120可使用一使用者登錄或一小型文字檔來識別該使用者。該伺服器120可產生對應於各使用者之一目標設定檔122(例如使用者設定檔)。As an overview of generating a custom language pattern, the training material 126 can be classified into a plurality of sets of the training material 126 (eg, one or more sets of files). The server 120 can receive the user (eg, a first user on the first client 112, a second user on the first client 112, and the second client 114) A user input of the first user). For example, the server 120 can identify the user using a user login or a small text file. The server 120 can generate a target profile 122 (e.g., a user profile) corresponding to each user.
在一些實施方案中,可預定該目標設定檔122。舉例而言,該伺服器120可識別對應於該使用者之先前產生的一目標設定檔122。在一些實施方案中,該目標設定檔122可基於該使用者之一詞典或使用者輸入(例如由該使用者提供表示該使用者輸入行為的資料)產生。此外,該目標設定檔122可在該伺服器120與一客戶端(例如該第一客戶端112)之間自動或手動同步。該目標設定檔122可與該訓練資料126之該等子集之各者的設定檔做比較。可基於一相似性(例如)在該目標設定檔122與各子集之設定檔之間選擇該訓練資料126之子集之一者或多者。In some embodiments, the target profile 122 can be predetermined. For example, the server 120 can identify a previously generated target profile 122 corresponding to the user. In some embodiments, the target profile 122 can be generated based on a dictionary or user input of the user (eg, by the user providing information indicative of the user's input behavior). Additionally, the target profile 122 can be automatically or manually synchronized between the server 120 and a client (eg, the first client 112). The target profile 122 can be compared to a profile of each of the subsets of the training profile 126. One or more of the subset of training profiles 126 may be selected based on a similarity, for example, between the target profile 122 and the profile of each subset.
訓練資料126之所選子集可用於產生一習慣語言模式124(例如一使用者特定語言模式、一領域特定語言模式)。該習慣語言模式可透過一網路發送(例如)至該第一客戶端112。該第一客戶端112可將該習慣語言模式用於該使用者所使用的輸入方法。在一些實施方案中,該習慣語言模式之一複本係儲存在該伺服器120上。若該使用者在該第二客戶端114上使用一輸入方法,則(例如)可將該習慣語言模式複本發送至(及/或更新於)該第二客戶端114。The selected subset of training materials 126 can be used to generate a custom language pattern 124 (e.g., a user specific language mode, a domain specific language mode). The custom language mode can be sent to the first client 112 via a network, for example. The first client 112 can use the custom language mode for the input method used by the user. In some embodiments, one of the custom language patterns is stored on the server 120. If the user uses an input method on the second client 114, the custom language mode replica can be sent (and/or updated to) the second client 114, for example.
在一些實施方案中,該(等)目標設定檔122、習慣語言模式124及/或訓練資料126係儲存在複數個伺服器上或其他位置。舉例而言,該訓練資料126可包含該伺服器120可存取之任意文件。特定言之,該訓練資料126可包含但不限於輸入方法編輯器(IME)使用者輸入、IME使用者詞典、網頁、搜索查詢記錄、電子郵件、博客、即時訊息(IM)腳本及新聞文章,可用於訓練一習慣語言模式124。In some embodiments, the (etc.) target profile 122, the custom language mode 124, and/or the training material 126 are stored on a plurality of servers or other locations. For example, the training material 126 can include any file accessible by the server 120. In particular, the training material 126 may include, but is not limited to, an input method editor (IME) user input, an IME user dictionary, a web page, a search query record, an email, a blog, an instant message (IM) script, and a news article. Can be used to train a custom language mode 124.
可為各使用者(或領域)產生一目標設定檔122。在一些實施方案中,該目標設定檔122係為一識別使用者產生。舉例而言,該使用者可藉由提供登錄資訊識別他自己/她自己,且一目標設定檔122可基於與該使用者相關的資料(例如與該識別使用者相關的一使用者詞典)產生。一使用者詞典可包含來自一IME及/或來自(例如)使用者輸入之預定義字元。另外,一使用者詞典可包含與字元使用相關的統計,例如在一使用者輸入歷史(例如談話文本、搜索查詢)中的字元總數或頻率。其他統計資料(諸如出現在使用者輸入中的n元)亦可儲存在該目標設定檔122中。在一些實施方案中,基於該使用者提供的資料為該使用者產生一目標設定檔122。舉例而言,一使用者可提交表示他的/她的輸入行為之文件,且一目標設定檔122可基於該等提交的文件產生。在一些實施方案中,該目標設定檔122係儲存在該伺服器120(或客戶端)上且重使用。A target profile 122 can be generated for each user (or domain). In some embodiments, the target profile 122 is generated by an identification user. For example, the user can identify himself/herself by providing login information, and a target profile 122 can be generated based on information related to the user (eg, a user dictionary associated with the identified user). . A user dictionary can contain predefined characters from an IME and/or from, for example, user input. Additionally, a user dictionary can contain statistics related to character usage, such as the total number or frequency of characters in a user input history (eg, conversation text, search query). Other statistics, such as n-numerics that appear in user input, may also be stored in the target profile 122. In some embodiments, a target profile 122 is generated for the user based on the information provided by the user. For example, a user may submit a file indicating his/her input behavior, and a target profile 122 may be generated based on the submitted documents. In some embodiments, the target profile 122 is stored on the server 120 (or client) and reused.
該目標設定檔122可表示特定記號對該使用者之相對重要性。在一些實施方案中,使用一向量表示該目標設定檔122。舉例而言,一特徵向量可包含一個或多個特徵(例如詞條)及各特徵之一對應的詞頻(tf)權重。該等tf權重可用作為一特徵對於該使用者重要性的統計量測。舉例而言,一特徵對於該使用者之重要性可根據該特徵出現在該使用者輸入之一文件集合或該使用者詞典中之頻率(例如一詞頻)而成比例地增加。The target profile 122 can indicate the relative importance of a particular token to the user. In some embodiments, a vector is used to represent the target profile 122. For example, a feature vector may include one or more features (eg, terms) and a word frequency (tf) weight corresponding to one of the features. These tf weights can be used as a statistical measure of the importance of the feature to the user. For example, the importance of a feature to the user may be proportionally increased based on the frequency at which the feature appears in one of the user input files or in the user dictionary (eg, a word frequency).
在一文件中的詞頻為一特定詞條出現在該文件中的一相對頻率,且可表示為:The word frequency in a file is a relative frequency in which a particular entry appears in the file and can be expressed as:
其中該詞頻為該特定詞條在一文件(d j )中的一出現次數(n i , j )除以所有詞條在d j 中的出現次數。Wherein the word frequency for a particular term appears in a number of documents (d j) in the (n i, j) divided by the number of occurrences of the term in d j.
在一些實施方案中,tf-idf權重係用作為該等特徵對於該使用者重要性的統計量測。一tf-idf權重可藉由用一文件頻率倒數(idf)乘以一詞頻而計算。In some embodiments, the tf-idf weight is used as a statistical measure of the importance of the features for the user. A tf-idf weight can be calculated by multiplying a word frequency inverse (idf) by a word frequency.
該文件頻率倒數(idf)可表示為:
一tf-idf權重可表示為:tf _idf i,j =tf i,j .idf i,j 。A tf-idf weight can be expressed as: tf _ idf i,j = tf i,j . Idf i,j .
該目標設定檔122可與訓練資料126子集之設定檔(例如特徵向量)比較以判定用於訓練該習慣語言模式124之該訓練資料126之該等子集。The target profile 122 can be compared to a profile of the training profile 126 (e.g., feature vector) to determine the subset of the training material 126 for training the custom language mode 124.
可使用各種分類方法將該訓練資料126分類為諸多子集。在一些實施方案中,使用叢集化來將該訓練資料126分類為諸多子集。舉例而言,可使用分割式叢集化(例如k均值叢集化、地點敏感散列、圖論方法)或階層式叢集化(例如凝聚層次叢集化、概念叢集化)來分類該訓練資料126。作為其他實施例,可使用奇異值分解法(SVD)、潛在語義分析(LSA)及神經網路使文件與叢集相關。可能存在其他實施方案。舉例而言,可使用二次分類器或一k近鄰演算法來將該訓練資料126分類為諸多子集。The training data 126 can be classified into a number of subsets using various classification methods. In some embodiments, clustering is used to classify the training material 126 into a number of subsets. For example, the training data 126 can be classified using partitioned clustering (eg, k-means clustering, location-sensitive hashing, graph theory methods) or hierarchical clustering (eg, condensed hierarchical clustering, concept clustering). As other embodiments, singular value decomposition (SVD), latent semantic analysis (LSA), and neural networks can be used to correlate files with clusters. Other embodiments are possible. For example, the training data 126 can be classified into a number of subsets using a secondary classifier or a k-nearest neighbor algorithm.
圖2為包含實例文件叢集(210、220、230、240及250)之一圖表。該等文件由圖2中之諸多點表示。一k 均值叢集化技術可用於產生該等叢集。特定言之,可選擇一叢集數目k 。舉例而言,圖2顯示五個(k =5)叢集。可隨機產生該等叢 集210、220、230、240及250。特定言之,可隨機指派各文件至一叢集。在一些實施方案中,可初始地指派一文件至一個以上叢集。如下文所述,可為該等叢集210、220、230、240及250之各者確定一質心(或叢集中心)。2 is a diagram including one of the example file clusters (210, 220, 230, 240, and 250). These documents are represented by the many points in Figure 2. A k- means clustering technique can be used to generate the clusters. In particular, a cluster number k can be selected. For example, Figure 2 shows five ( k = 5) clusters. The clusters 210, 220, 230, 240, and 250 can be randomly generated. In particular, each file can be randomly assigned to a cluster. In some embodiments, one file can be initially assigned to more than one cluster. As described below, a centroid (or cluster center) can be determined for each of the clusters 210, 220, 230, 240, and 250.
圖3為包含圖2之該等實例叢集及對應質心(215、225、235、245及255)之一圖表。在一些實施方案中,一叢集質心係藉由為在該叢集內之該等文件各者計算一tf-idf(詞頻逆文件頻率)向量而判定。該tf-idf向量可包含諸特徵及其等之對應特徵權重(例如tf-idf權重)。一叢集質心可由對應於該叢集內之文件之所有該等tf-idf向量之平均值表示。可以先前描述之方式計算該等tf-idf權重。3 is a chart including one of the example clusters and corresponding centroids (215, 225, 235, 245, and 255) of FIG. 2. In some embodiments, a cluster of mass centers is determined by computing a tf-idf (word frequency inverse file frequency) vector for each of the files within the cluster. The tf-idf vector may contain features and their corresponding feature weights (eg, tf-idf weights). A cluster centroid can be represented by the average of all of the tf-idf vectors corresponding to the files within the cluster. The tf-idf weights can be calculated in the manner previously described.
作為一實例,一特定文件(例如圖3之叢集250中的一點)可含有5,000字元。在該文件中,字元「籃球」可能出現200次。「籃球」的詞頻將為0.04(200/5,000)。叢集250中的文件集合(例如叢集250中的所有點)包含九個文件。假設「籃球」僅出現在該九個文件之一者中,該逆向文件頻率為log(9/1)0.95。該文件之tf-idf向量可包含特徵「籃球」且一對應的tf-idf權重為(0.04.0.95)0.04。可使用類似的計算為在一特定叢集之該等文件中的額外詞條產生額外特徵及權重。As an example, a particular file (e.g., a point in cluster 250 of Figure 3) may contain 5,000 characters. In this document, the character "basketball" may appear 200 times. The word frequency of "Basketball" will be 0.04 (200/5,000). The set of files in cluster 250 (e.g., all points in cluster 250) contains nine files. Assume that "basketball" only appears in one of the nine files. The reverse file frequency is log(9/1). 0.95. The tf-idf vector of the file may contain the feature "basketball" and a corresponding tf-idf weight is (0.04.0.95) 0.04. Similar calculations can be used to generate additional features and weights for additional terms in such files in a particular cluster.
該叢集質心可計算為在該叢集中之所有該等文件的tf-idf向量的平均數。舉例而言,對於具有兩個文件X與Y的一叢集(X與Y各具有tf-idf向量,各包含三個特徵),文件X與Y可分別表示為:X=(x1 ,x2 ,x3 )及Y=(y1 ,y2 ,y3 )。該 叢集的質心Z可表示為:Z=(z1 ,z2 ,z3 ),其中z1 =(x1 +y1 )/2;z2 =(x2 +y2 )/2;且z3 =(x3 +y3 )/2。The cluster centroid can be calculated as the average of the tf-idf vectors for all of the files in the cluster. For example, for a cluster with two files X and Y (X and Y each have a tf-idf vector, each containing three features), the files X and Y can be represented as: X = (x 1 , x 2 , x 3) and Y = (y 1, y 2 , y 3). The centroid Z of the cluster can be expressed as: Z = (z 1 , z 2 , z 3 ), where z 1 = (x 1 + y 1 )/2; z 2 = (x 2 + y 2 )/2; And z 3 = (x 3 + y 3 )/2.
在為該等初始叢集計算該等質心後,重新指派各文件至最近的質心。圖4為繪示圖2文件之一些者之一實例重新分類的一圖表。特定言之,圖4顯示基於一對應的最近質心重新指派至一不同叢集之該等文件之一些者。舉例而言,叢集250中的文件410可重新指派至叢集220,如由箭頭415顯示。藉由計算介於一文件與該等叢集質心各者之間的距離可判定一最近質心。特定言之,可計算介於表示該文件之tf-idf向量與表示各質心之tf-idf向量之間的距離。返回該先前實例,介於質心Z與文件X之間的距離可表示為一餘弦距離:
該文件可指派至距離該文件最近的叢集。在該等文件之各者重新指派至其之最近質心後,可計算新的質心。This file can be assigned to the cluster closest to the file. The new centroid can be calculated after each of the documents has been reassigned to its nearest centroid.
圖5為繪示基於圖4中的重新分類之該等實例叢集510、520、530、540及550之一實例重新調整的一圖表。可確定該等新質心,如先前關於圖2及圖3之描述。特定言之,可分別為叢集510、520、530、540及550判定質心515、525、535、545及555。FIG. 5 is a diagram illustrating an example re-adjustment of one of the example clusters 510, 520, 530, 540, and 550 based on the reclassification in FIG. These new centroids can be determined as previously described with respect to Figures 2 and 3. In particular, centroids 515, 525, 535, 545, and 555 can be determined for clusters 510, 520, 530, 540, and 550, respectively.
可執行對基於最近質心之重新分類該等文件及基於該重新分類重新調整該等叢集之多重迭代,直到滿足一個或多個準則。舉例而言,可執行該等重新分類及重新調整直到 滿足一收斂準則(例如無文件被重新指派)。Multiple iterations of reclassifying the files based on the most recent centroid and re-adjusting the clusters based on the reclassification may be performed until one or more criteria are met. For example, such reclassifications and re-adjustments can be performed until A convergence criterion is met (eg no file is reassigned).
在分類完成後,可產生該等叢集之各者的設定檔。圖6為繪示圖5之該等實例叢集510、520、530、540及550之各者之設定檔610、620、630、640及650的一實例產生的一圖表。舉例而言,該等設定檔可由tf-idf向量表示,該tf-idf向量係根據在該等各自叢集中之該等文件為該等叢集之各者計算(如上所述)。After the classification is completed, profiles for each of the clusters can be generated. FIG. 6 is a diagram showing an example of profiles 610, 620, 630, 640, and 650 of each of the example clusters 510, 520, 530, 540, and 550 of FIG. For example, the profiles can be represented by a tf-idf vector that is calculated for each of the clusters (as described above) based on the files in the respective clusters.
因為得到的叢集依賴初始隨機指派該等文件至各叢集,所有在一些實施方案中,重複上文描述的程序以自該訓練資料126建立額外叢集。Because the resulting cluster relies on initially randomly assigning the files to the clusters, all in some embodiments, the procedures described above are repeated to establish additional clusters from the training material 126.
此外,由於產生訓練資料126子集之方式,所以不必為各訓練資料子集預定一主題或領域。舉例而言,可認為所產生之該等叢集為一般叢集。特定言之,並非為表示任一特定主題/領域而初始地建立各叢集,但是作為叢集化類似文件之結果,各叢集可固有地表示一主題或領域。因此,不必預定主題、關鍵字元或領域(例如)以標記或識別該等叢集。然而,為一使用者產生之一習慣語言模式可基於該等叢集產生,且該習慣語言模式亦可固有地包含該等叢集表示的一個或多個主題/領域。Moreover, since a subset of the training materials 126 is generated, it is not necessary to reserve a subject or field for each subset of training materials. For example, the clusters produced can be considered to be a general cluster. In particular, the clusters are not initially built to represent any particular topic/area, but as a result of clustering similar files, each cluster may inherently represent a topic or domain. Therefore, it is not necessary to pre-book topics, key elements or fields (for example) to mark or identify the clusters. However, one of the custom language patterns generated for a user may be generated based on the clusters, and the custom language pattern may also inherently include one or more topics/areas of the cluster representations.
圖7為繪示圖6之該等實例叢集之一者或多者之一實例選擇的一圖表。特定言之,叢集510及520係基於來自一使用者詞典710之一目標設定檔而選擇。如先前描述,該目標設定檔可與各叢集之該等設定檔比較以判定介於該目標設定檔與各叢集設定檔之間的相似性。7 is a diagram showing an example selection of one or more of the example clusters of FIG. 6. In particular, clusters 510 and 520 are selected based on a target profile from a user dictionary 710. As previously described, the target profile can be compared to the profiles of each cluster to determine a similarity between the target profile and each cluster profile.
在一些實施方案中,一餘弦相似性係於該目標設定檔與各叢集設定檔之間計算。該餘弦相似性可用於量測介於兩個向量之間的相似性。該餘弦相似性可表達為:In some embodiments, a cosine similarity is calculated between the target profile and each cluster profile. This cosine similarity can be used to measure the similarity between two vectors. The cosine similarity can be expressed as:
其中xi 為表示該目標設定檔之一向量且yi 為表示一叢集設定檔之一向量。隨著介於兩個向量之間之餘弦相似性接近1,該兩個向量變得更接近或為更類似。該餘弦相似性可與一臨限值比較以判定是否選擇該叢集。舉例而言,若該餘弦相似性大於一臨限值0.8,則可認為x 與y 關於一主題高度相關,且可選擇對應於y 之該叢集。Where x i is a vector representing the target profile and y i is a vector representing a cluster profile. As the cosine similarity between the two vectors approaches 1, the two vectors become closer or more similar. The cosine similarity can be compared to a threshold to determine whether to select the cluster. For example, if the cosine similarity is greater than a threshold of 0.8, then x and y can be considered highly correlated with respect to a subject, and the cluster corresponding to y can be selected.
圖8為繪示一習慣語言模式810之一實例產生的一圖表。該習慣語言模式可基於在該等所選叢集中之該等文件而產生。該習慣語言模式可包含n元及該n元出現於在該等所選叢集中之該等文件中的對應概率。FIG. 8 is a diagram showing an example of an example of a custom language mode 810. The custom language mode can be generated based on the files in the selected clusters. The custom language pattern can include n elements and corresponding probabilities that the n elements appear in the files in the selected cluster.
根據一n元語言模式,一特定串將出現的概率可使用連鎖定律判定。該連鎖定律判定一串之一概率為各自概率之一乘積。因此,對於一給定串「e1 ,e2 ,...,ek 」,該串之概率p (e1 ,e2 ,...ek )等於:According to an n-gram language mode, the probability that a particular string will appear can be determined using a conjunction law. The linkage law determines that one of the probabilities of a string is the product of one of the respective probabilities. Thus, for a given string "e 1 , e 2 , ..., e k ", the probability p (e 1 , e 2 , ... e k ) of the string is equal to:
該n元語言模式可限於一特定最大尺寸n元,例如限於1元、2元、3元等。舉例而言,對於一給定串「NASA officials say they hope,」,其中該最大n元階數限於3元,該串之概率可判定為諸條件概率之一乘積,如下:p (NASA officials say they hope)=p (NASA)‧p (officials|NASA)‧p (say|NASA officials)‧p (they|officials say)‧p (hope|say they)。此可一般化為:The n-gram language mode may be limited to a specific maximum size n-ary, for example, limited to 1 yuan, 2 yuan, 3 yuan, and the like. For example, for a given string "NASA officials say they hope," where the maximum n-order is limited to 3 yuan, the probability of the string can be determined as the product of the conditional probabilities, as follows: p (NASA means say they hope) = p (NASA) ‧ p (officials | NASA) ‧ p (say | NASA officials) ‧ p (they | officials say) ‧ p (hope | say they). This can be generalized as:
其中n 為該語言模式允許之最大n元的階數。Where n is the maximum n-ary order allowed for the language mode.
通常根據在該等所選叢集之該等文件中的相對頻率憑經驗判定該條件概率。舉例而言,在以上實例中,字元「say」給予語境「NASA officials」之概率係給出為:The conditional probability is typically determined empirically based on the relative frequencies in the documents of the selected clusters. For example, in the above example, the probability that the character "say" gives the context "NASA officials" is given as:
其中f (NASA officials say )為該串「NASA officials say」在該等所選叢集之該等文件中的一出現頻率或一出現總數。在該n元語言模式中之該最大n元階數內之諸串的條件概率對應於儲存在該n元語言模式中的概率,例如p (say |NASA officials )為儲存在該3元條目「NASA officials say」之語言模式中的條件概率。Where f (NASA officials say) strings, "NASA officials say" in such documents selected in such a cluster of frequency of occurrence or the total appeared for. The conditional probability of the strings in the largest n-ary order in the n-gram language mode corresponds to the probability stored in the n-gram language mode, for example, p ( say | NASA officials ) is stored in the 3-ary entry " Conditional probability in the language mode of NASA stores say.
圖9A為顯示用於產生一叢集向量之一實例程序的一流程圖。為了方便,將關於執行該產生之一系統描述該叢集向量之產生。一系統接收910一文件集合。舉例而言,該伺服器120可接收該訓練資料126。該系統將該等文件叢集化920為一個或多個叢集。舉例而言,在該伺服器120中的一分類引擎(未顯示)可叢集化該等文件。該系統為該一個或多個叢集之各叢集產生930一叢集向量。舉例而言,在該伺服器120中之該分類引擎亦可為各叢集產生一叢集向量。Figure 9A is a flow chart showing an example program for generating a cluster vector. For convenience, the generation of the cluster vector will be described with respect to one of the systems performing the generation. A system receives 910 a set of files. For example, the server 120 can receive the training material 126. The system clusters 920 the files into one or more clusters. For example, a classification engine (not shown) in the server 120 can cluster the files. The system generates 930 a cluster vector for each cluster of the one or more clusters. For example, the classification engine in the server 120 can also generate a cluster vector for each cluster.
圖9B為顯示用於產生一習慣語言模式之一實例程序之一流程圖。為了方便,將相對於執行該產生之一系統來描述該習慣語言模式的產生。該系統產生940一與一目標設定檔相關的目標向量。舉例而言,該分類引擎可產生一與該目標設定檔相關的目標向量。該系統比較950該目標向量與該等叢集向量之各者。舉例而言,在該伺服器120中之一比較引擎(未顯示)可比較該目標向量與該等叢集向量之各者。該系統基於該比較選擇960該一個或多個叢集之一者或多者,且使用來自一個或多個所選叢集之文件來產生970一語言模式(例如一習慣語言模式)。Figure 9B is a flow chart showing one of the example programs for generating a custom language pattern. For convenience, the generation of the custom language pattern will be described with respect to a system that performs the generation. The system generates 940 a target vector associated with a target profile. For example, the classification engine can generate a target vector associated with the target profile. The system compares 950 the target vector to each of the cluster vectors. For example, one of the comparison engines (not shown) in the server 120 can compare the target vector to each of the cluster vectors. The system selects 960 one or more of the one or more clusters based on the comparison and uses a file from one or more selected clusters to generate a 970 language pattern (eg, a custom language pattern).
雖然本說明書以英語字元繪示許多實例n元,但是可對其他語言(例如漢語、日語及韓語)實施所描述之本發明的實施例及功能操作。舉例而言,一n元可包含一個或多個漢字。Although the present specification depicts many example n-arys in English characters, the described embodiments of the invention and functional operations can be implemented in other languages, such as Chinese, Japanese, and Korean. For example, an n-ary can contain one or more Chinese characters.
一電子文件(為簡潔起見稱其為一文件)不一定要對應於一檔案。一文件可儲存在保存其他文件之一檔案的一部分中,儲存在致力於討論中之文件之一單一檔案中或在多重協調的檔案中。An electronic file (called a file for the sake of brevity) does not necessarily correspond to a file. A file can be stored in a portion of a file that holds one of the other files, stored in a single file of one of the documents in question, or in a multi-coordinated file.
本說明書中描述之本發明的實施例及該等功能操作可以數位電子電路實施,或以電腦軟體、韌體或硬體(包含本說明書中揭示之該等結構及其等之結構等效物)實施,或以其等之一者或多者的組合實施。本說明書中描述之本發明實施例可實施為一個或多個電腦程式產品,例如在一實體程式載體上編碼、用於為資料處理裝置執行或用以控制資料處理裝置之操作之電腦程式指令的一個或多個模組。該實體程式載體可為一電腦可讀媒體。該電腦可讀媒體可為一機器可讀儲存裝置、一機器可讀儲存基板、一記憶體裝置、一產生一機器可讀傳播信號之物質的組成,或其等之一者或多者之一組合。Embodiments of the invention and such functional operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware or hardware (including such structures as disclosed in the specification and their equivalents) Implemented, or implemented in a combination of one or more of them. The embodiments of the invention described in this specification can be implemented as one or more computer program products, such as computer program instructions for encoding on a physical program carrier for execution of a data processing device or for controlling the operation of a data processing device. One or more modules. The physical program carrier can be a computer readable medium. The computer readable medium can be a machine readable storage device, a machine readable storage substrate, a memory device, a component that produces a machine readable propagated signal, or one or more of combination.
術語「資料處理裝置」涵蓋用於處理資料的所有裝置、器件及機器,包含(藉由實例)一可程式化處理器、一電腦或多重處理器或多重電腦。除硬體之外,該裝置可包含為討論中之電腦程式建立一執行環境的程式碼,例如構成處理器韌體、一協定堆疊、一資料庫管理系統、一操作系統或其等之一者或多者之一組合的程式碼。The term "data processing device" encompasses all devices, devices and machines for processing data, including (by way of example) a programmable processor, a computer or multiple processors or multiple computers. In addition to hardware, the apparatus can include code for creating an execution environment for the computer program in question, such as one that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or the like. A combination of one or more of the code.
一電腦程式(亦稱為一程式、軟體、軟體應用、腳本或程式碼)可以任一編程語言形式書寫,包含編譯或解譯語言,或宣告或程序語言,且可以任意形式使用,包含作為一獨立的程式或作為一模組、組件、子程式或適合用於一計算環境中的其他單元。一電腦程式不必對應於一檔案系統中之一檔案。一程式可儲存在保存其他程式或資料(例如儲存在一標記語言文件中的一個或多個腳本)之一檔案的一部分中,儲存在致力於討論中之程式之一單一檔案中或在多重協調檔案(例如儲存一個或多個模組、子程式或部分程式碼的檔案)中。可利用一電腦程式在一電腦上執行,或在位於一站點或跨多重站點而分散且藉由一通信網路互連的多重電腦上執行。A computer program (also known as a program, software, software application, script or code) can be written in any programming language, including a compiled or interpreted language, or an announcement or programming language, and can be used in any form, including as a A stand-alone program can be used as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program does not have to correspond to one file in a file system. A program can be stored in a portion of an archive that holds other programs or materials (such as one or more scripts stored in a markup language file), stored in a single file of a program dedicated to the discussion, or in multiple coordination Files (such as files that store one or more modules, subprograms, or parts of code). It can be executed on a computer using a computer program, or on multiple computers located at a site or across multiple sites and interconnected by a communication network.
藉由操作輸入資料且產生輸出,執行一個或多個電腦程式以執行功能之一個或多個可程式化處理器可執行描述於本說明書中程序及邏輯流程。該等程序及邏輯流程亦可由專用邏輯電路執行,且裝置亦可實施為專用邏輯電路,例如一FPGA(場可程式化閘陣列)或一ASIC(特定應用積體電路)。One or more programmable processors executing one or more computer programs to perform functions can perform the procedures and logic flows described in this specification by operating input data and generating output. The programs and logic flows may also be performed by dedicated logic circuits, and the devices may also be implemented as dedicated logic circuits, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).
適用於執行一電腦程式的處理器包含(藉由實例)通用及專用微處理器兩者及任意種類數位電腦之任意一個或多個處理器。通常,一處理器將接收來自一唯讀記憶體或一隨機存取記憶體或兩者的指令及資料。一電腦的重要元件為用於執行指令之一處理器及用於儲存指令及資料的一個或多個記憶體裝置。通常,一電腦亦將包含用於儲存資料的一個或多個大容量儲存裝置(例如磁碟、磁光碟或光碟),或可操作耦合以自該一個或多個大容量儲存裝置接收資料或傳遞資料至該一個或多個大容量儲存裝置或兩者。然而,一電腦不需要具有此等裝置。此外,一電腦可嵌入另一裝置(例如一行動電話、一個人數位助理(PDA)、一行動音訊或視訊播放器、一數位圖像框、一遊戲主機、一全球定位系統(GPS)接收器,僅列出該一些)中。A processor suitable for executing a computer program includes (by way of example) any one or more processors of both general and special purpose microprocessors and any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory or both. An important component of a computer is a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also contain one or more mass storage devices (eg, magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operatively coupled to receive data from or deliver the data from the one or more mass storage devices. Data to the one or more mass storage devices or both. However, a computer does not need to have such a device. In addition, a computer can be embedded in another device (such as a mobile phone, a PDA, a mobile audio or video player, a digital image frame, a game console, a global positioning system (GPS) receiver, Only listed in some of them).
適合儲存電腦程式指令及資料的電腦可讀媒體包含所有形式的非揮發性記憶體、媒體及記憶體裝置,包含(藉由實例)半導體記憶體裝置(例如EPROM、EEPROM及快閃記憶體裝置)、磁碟(例如內部硬碟或可移除式碟)、磁光碟及CD-ROM與DVD-ROM碟。該處理器及該記憶體可由專用邏輯電路補充或嵌入專用邏輯電路中。Computer-readable media suitable for storing computer program instructions and data includes all forms of non-volatile memory, media, and memory devices, including (by way of example) semiconductor memory devices (eg, EPROM, EEPROM, and flash memory devices) , disk (such as internal hard drive or removable disc), magneto-optical disc and CD-ROM and DVD-ROM disc. The processor and the memory can be supplemented by or embedded in dedicated logic circuitry.
為提供與一使用者的交互,描述於本說明書中之本發明之實施例可在具有用於顯示資訊給該使用者的一顯示裝置(例如一CRT(陰極射線管)或LCD(液晶顯示器)監視器)及一鍵盤及一指向裝置(例如一滑鼠或一軌跡球,藉由其等該使用者可提供輸入至該電腦)的一電腦上實施。亦可使用其他種類的裝置以提供與一使用者的交互,例如提供至該使用者的反饋可為任意形式的感測反饋(例如視覺反饋、音訊反饋或觸覺反饋)且來自該使用者的輸入可以任意形式(包含聽覺,語音,或觸覺輸入)得到接收。In order to provide interaction with a user, embodiments of the invention described in this specification can have a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display)) for displaying information to the user. The monitor) and a keyboard and a pointing device (for example, a mouse or a trackball, by which the user can provide input to the computer) are implemented on a computer. Other types of devices may be used to provide interaction with a user, for example, feedback provided to the user may be any form of sensory feedback (eg, visual feedback, audio feedback, or tactile feedback) and input from the user. It can be received in any form (including auditory, speech, or tactile input).
描述於本說明書中之本發明的實施例可在一計算系統中實施,該計算系統包含一後端組件(例如一資料伺服器),或包含一中間件組件(例如一應用伺服器),或包含一前端組件(例如具有一圖形使用者介面或一網頁瀏覽器的一客戶端電腦,一使用者可透過該圖形使用者介面或網頁瀏覽器與描述於本說明書中之本發明的一實施方案交互),或一個或多個此種後端、中間件或前端組件之任意組合。該系統之該等組件可藉由任意數位資料通信形式或媒體(例如一通信網路)而互連。通信網路實例包含一區域網路(「LAN」)及一廣域網路(「WAN」),例如網際網路。Embodiments of the invention described in this specification can be implemented in a computing system that includes a backend component (eg, a data server) or a middleware component (eg, an application server), or Including a front-end component (for example, a client computer having a graphical user interface or a web browser, and a user can use the graphical user interface or web browser to describe an embodiment of the present invention described in the specification Interactive), or any combination of one or more such backend, middleware, or front-end components. The components of the system can be interconnected by any number of data communication forms or media (e.g., a communication network). The communication network instance includes a local area network ("LAN") and a wide area network ("WAN"), such as the Internet.
該計算系統可包含客戶端及伺服器。一客戶端與伺服器通常相互遠離且通常透過一通信網路交互。客戶端與伺服器的關係憑藉運行在各自電腦上且具有相對彼此之一客戶端-伺服器關係的電腦程式而發生。The computing system can include a client and a server. A client and server are typically remote from each other and typically interact through a communication network. The relationship between the client and the server occurs with computer programs running on their respective computers and having a client-server relationship with respect to each other.
雖然本說明書含有許多特定實施方案細節,但是不應認為此等為對任意發明或可申明之發明之範圍的限制,而是作為可對特定發明之特定實施例特定之特徵的描述。描述於單獨實施例內文之本說明書中的某些特徵亦可組合在一單一實施例中實施。相反地,描述於一單一實施例內文中之各種特徵亦可單獨地在多重實施例中實施,或在任意適合的次組合中實施。此外,雖然諸特徵可在上文中被描述為以某些組合起作用及甚至初始地聲明如此,但是來自一聲明組合之一個或多個特徵在一些情況下可自該組合刪去,且該申明組合可關於一次組合或一次組合之變更。The description contains a number of specific embodiments of the invention, and is not to be construed as limiting the scope of the invention or the invention. Some of the features in this specification, which are described in the context of separate embodiments, can also be implemented in a single embodiment. Conversely, various features that are described in the context of a single embodiment can be implemented in various embodiments, or in any suitable sub-combination. Moreover, although the features may be described above as acting in certain combinations and even initially declaring such, one or more features from a claimed combination may in some cases be deleted from the combination, and the claim Combinations can be made about one combination or one combination.
類似地,雖然以一特定順序在圖式中描繪操作,但是不應將此理解為需要以所顯示之該特定順序或以循序順序執行此等操作,或執行所有繪示之操作以得到所要的結果。在某些環境下,多任務及並行處理可為有利。此外,不應將在該等實施例中之各種系統組件的分離理解為在所有實施例中需要此分離,且應瞭解該等描述之程式組件及系統可通常在一單一軟體產品中整合在一起或封裝為多重軟體產品。Similarly, although the operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or all illustrated operations are performed to obtain the desired result. In some environments, multitasking and parallel processing can be advantageous. In addition, the separation of the various system components in the embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can be generally integrated in a single software product. Or packaged as a multiple software product.
已描述於本說明書中描述之本發明的特定實施例。其他實施例係在以下請求項之範圍內。舉例而言,可以一不同順序執行列舉於該等請求項中之動作且仍得到所要的結果。作為一實例,描繪於隨附圖式中的程序不必需要所顯示之該特定順序或循序順序以得到所要的結果。在某些實施方案中,多任務及並行處理可為有利。Specific embodiments of the invention described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions listed in the request items can be performed in a different order and still achieve the desired result. As an example, a program depicted in the accompanying drawings does not necessarily require that the particular order or sequence of the sequence shown in order to obtain the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.
100...系統100. . . system
112...第一客戶端112. . . First client
114...第二客戶端114. . . Second client
120...伺服器120. . . server
122...目標設定檔122. . . Target profile
124...習慣語言模式124. . . Custom language mode
126...訓練資料126. . . Training materials
210...文件叢集210. . . File cluster
220...文件叢集220. . . File cluster
230...文件叢集230. . . File cluster
240...文件叢集240. . . File cluster
250...文件叢集250. . . File cluster
410...文件410. . . file
415...箭頭415. . . arrow
510...叢集510. . . Cluster
515...質心515. . . Centroid
520...叢集520. . . Cluster
525...質心525. . . Centroid
530...叢集530. . . Cluster
535...質心535. . . Centroid
540...叢集540. . . Cluster
545...質心545. . . Centroid
550...叢集550. . . Cluster
555...質心555. . . Centroid
610...設定檔610. . . Profile
620...設定檔620. . . Profile
630...設定檔630. . . Profile
640...設定檔640. . . Profile
650...設定檔650. . . Profile
710...使用者詞典710. . . User dictionary
810...習慣語言模式810. . . Custom language mode
圖1為繪示用於產生習慣語言模式之一實例系統的一圖表;1 is a diagram showing an example system for generating a custom language pattern;
圖2為包含實例文件叢集之一圖表;Figure 2 is a chart containing a cluster of example files;
圖3為包含圖2之該等實例叢集及對應質心之一圖表;3 is a chart including the example clusters and corresponding centroids of FIG. 2;
圖4為繪示圖2之該等文件之一些者之一實例重新分類的一圖表;4 is a diagram showing an example of reclassification of one of the files of FIG. 2;
圖5為繪示基於圖4中的重新分類之該等實例叢集之一實例重新調整的一圖表;5 is a diagram showing an example re-adjustment of one of the example clusters based on the reclassification in FIG. 4;
圖6為繪示圖5之該等實例叢集之各者設定檔之一實例產生的一圖表;6 is a diagram showing an example of an example of each of the profile sets of the example clusters of FIG. 5;
圖7為繪示圖6之該等實例叢集之一者或多者之一實例選擇的一圖表;7 is a diagram showing an example selection of one or more of the example clusters of FIG. 6;
圖8為繪示一習慣語言模式之一實例產生的一圖表;Figure 8 is a diagram showing an example of an example of a custom language mode;
圖9A為顯示用於產生一叢集向量之一實例程序的一流程圖;及9A is a flow chart showing an example program for generating a cluster vector; and
圖9B為顯示用於產生一習慣語言模式之一實例程序之一流程圖。Figure 9B is a flow chart showing one of the example programs for generating a custom language pattern.
100‧‧‧系統100‧‧‧ system
112‧‧‧第一客戶端112‧‧‧First client
114‧‧‧第二客戶端114‧‧‧Second client
120‧‧‧伺服器120‧‧‧Server
122‧‧‧目標設定檔122‧‧‧ Target profile
124‧‧‧習慣語言模式124‧‧‧ Custom language mode
126‧‧‧訓練資料126‧‧‧ Training materials
Claims (16)
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2008/001845 WO2010051654A1 (en) | 2008-11-05 | 2008-11-05 | Custom language models |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201022964A TW201022964A (en) | 2010-06-16 |
TWI512502B true TWI512502B (en) | 2015-12-11 |
Family
ID=42152444
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW098137647A TWI512502B (en) | 2008-11-05 | 2009-11-05 | Method and system for generating custom language models and related computer program product |
Country Status (6)
Country | Link |
---|---|
US (1) | US8826226B2 (en) |
JP (1) | JP5475795B2 (en) |
KR (1) | KR101537078B1 (en) |
CN (1) | CN102272754B (en) |
TW (1) | TWI512502B (en) |
WO (1) | WO2010051654A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI771468B (en) * | 2017-08-10 | 2022-07-21 | 美商鄧白氏公司 | System and method for dynamic synthesis and transient clustering of semantic attributions for feedback and adjudication |
TWI833072B (en) * | 2021-03-30 | 2024-02-21 | 緯創資通股份有限公司 | Speech recognition system and speech recognition method |
Families Citing this family (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8510109B2 (en) | 2007-08-22 | 2013-08-13 | Canyon Ip Holdings Llc | Continuous speech transcription performance indication |
US9973450B2 (en) | 2007-09-17 | 2018-05-15 | Amazon Technologies, Inc. | Methods and systems for dynamically updating web service profile information by parsing transcribed message strings |
TWI506019B (en) * | 2008-12-08 | 2015-11-01 | Basf Se | Process for manufacturing substituted 5-methoxymethylpyridine-2,3-dicarboxylic acid derivatives |
EP2982673B1 (en) * | 2008-12-09 | 2018-02-21 | Basf Se | Process for manufacturing 5-chloromethyl-2,3-dicarboxylic anhydride |
US8447608B1 (en) * | 2008-12-10 | 2013-05-21 | Adobe Systems Incorporated | Custom language models for audio content |
US20110071817A1 (en) * | 2009-09-24 | 2011-03-24 | Vesa Siivola | System and Method for Language Identification |
US8589163B2 (en) * | 2009-12-04 | 2013-11-19 | At&T Intellectual Property I, L.P. | Adapting language models with a bit mask for a subset of related words |
JP5504097B2 (en) * | 2010-08-20 | 2014-05-28 | Kddi株式会社 | Binary relation classification program, method and apparatus for classifying semantically similar word pairs into binary relation |
US8880403B2 (en) * | 2010-09-03 | 2014-11-04 | Canyon Ip Holdings Llc | Methods and systems for obtaining language models for transcribing communications |
US9484018B2 (en) * | 2010-11-23 | 2016-11-01 | At&T Intellectual Property I, L.P. | System and method for building and evaluating automatic speech recognition via an application programmer interface |
US9679561B2 (en) | 2011-03-28 | 2017-06-13 | Nuance Communications, Inc. | System and method for rapid customization of speech recognition models |
US9176941B2 (en) * | 2011-07-14 | 2015-11-03 | Tencent Technology (Shenzhen) Company Limited | Text inputting method, apparatus and system based on a cache-based language model and a universal language model |
US9324323B1 (en) * | 2012-01-13 | 2016-04-26 | Google Inc. | Speech recognition using topic-specific language models |
US8775177B1 (en) | 2012-03-08 | 2014-07-08 | Google Inc. | Speech recognition process |
US20140129221A1 (en) * | 2012-03-23 | 2014-05-08 | Dwango Co., Ltd. | Sound recognition device, non-transitory computer readable storage medium stored threreof sound recognition program, and sound recognition method |
EP2845147B1 (en) * | 2012-04-29 | 2018-06-06 | Hewlett-Packard Development Company, L.P. | Re-digitization and error correction of electronic documents |
US9620111B1 (en) * | 2012-05-01 | 2017-04-11 | Amazon Technologies, Inc. | Generation and maintenance of language model |
GB201208373D0 (en) | 2012-05-14 | 2012-06-27 | Touchtype Ltd | Mechanism for synchronising devices,system and method |
US9035884B2 (en) | 2012-10-17 | 2015-05-19 | Nuance Communications, Inc. | Subscription updates in multiple device language models |
US20150278194A1 (en) * | 2012-11-07 | 2015-10-01 | Nec Corporation | Information processing device, information processing method and medium |
US20140278349A1 (en) * | 2013-03-14 | 2014-09-18 | Microsoft Corporation | Language Model Dictionaries for Text Predictions |
US9672818B2 (en) * | 2013-04-18 | 2017-06-06 | Nuance Communications, Inc. | Updating population language models based on changes made by user clusters |
CN104166455B (en) * | 2013-05-16 | 2018-11-13 | 百度在线网络技术(北京)有限公司 | Method and apparatus for determining the input model corresponding to target user |
US9728184B2 (en) | 2013-06-18 | 2017-08-08 | Microsoft Technology Licensing, Llc | Restructuring deep neural network acoustic models |
US9311298B2 (en) | 2013-06-21 | 2016-04-12 | Microsoft Technology Licensing, Llc | Building conversational understanding systems using a toolset |
US9589565B2 (en) | 2013-06-21 | 2017-03-07 | Microsoft Technology Licensing, Llc | Environmentally aware dialog policies and response generation |
TWI506458B (en) | 2013-12-24 | 2015-11-01 | Ind Tech Res Inst | Apparatus and method for generating recognition network |
US9552408B2 (en) * | 2014-01-24 | 2017-01-24 | Facebook, Inc. | Nearest neighbor clustering determination and estimation algorithm that hashes centroids into buckets and redistributes vectors between clusters |
US9626426B2 (en) | 2014-01-24 | 2017-04-18 | Facebook, Inc. | Clustering using locality-sensitive hashing with improved cost model |
US20150254233A1 (en) * | 2014-03-06 | 2015-09-10 | Nice-Systems Ltd | Text-based unsupervised learning of language models |
JP6165657B2 (en) * | 2014-03-20 | 2017-07-19 | 株式会社東芝 | Information processing apparatus, information processing method, and program |
US9564122B2 (en) * | 2014-03-25 | 2017-02-07 | Nice Ltd. | Language model adaptation based on filtered data |
US9529794B2 (en) | 2014-03-27 | 2016-12-27 | Microsoft Technology Licensing, Llc | Flexible schema for language model customization |
WO2015166508A1 (en) * | 2014-04-30 | 2015-11-05 | Hewlett-Packard Development Company, L.P. | Correlation based instruments discovery |
US9678822B2 (en) * | 2015-01-02 | 2017-06-13 | Tata Consultancy Services Limited | Real-time categorization of log events |
KR102450853B1 (en) * | 2015-11-30 | 2022-10-04 | 삼성전자주식회사 | Apparatus and method for speech recognition |
US10032463B1 (en) * | 2015-12-29 | 2018-07-24 | Amazon Technologies, Inc. | Speech processing with learned representation of user interaction history |
RU2634180C1 (en) * | 2016-06-24 | 2017-10-24 | Акционерное общество "Лаборатория Касперского" | System and method for determining spam-containing message by topic of message sent via e-mail |
US11205103B2 (en) | 2016-12-09 | 2021-12-21 | The Research Foundation for the State University | Semisupervised autoencoder for sentiment analysis |
US10586528B2 (en) * | 2017-02-02 | 2020-03-10 | Adobe Inc. | Domain-specific speech recognizers in a digital medium environment |
JPWO2019021804A1 (en) * | 2017-07-24 | 2020-05-28 | ソニー株式会社 | Information processing apparatus, information processing method, and program |
US10515637B1 (en) * | 2017-09-19 | 2019-12-24 | Amazon Technologies, Inc. | Dynamic speech processing |
US10685183B1 (en) * | 2018-01-04 | 2020-06-16 | Facebook, Inc. | Consumer insights analysis using word embeddings |
US10732952B1 (en) * | 2018-02-06 | 2020-08-04 | Intuit, Inc. | Deployment and customization of applications at the widget level |
US11488602B2 (en) | 2018-02-20 | 2022-11-01 | Dropbox, Inc. | Meeting transcription using custom lexicons based on document history |
US10657954B2 (en) * | 2018-02-20 | 2020-05-19 | Dropbox, Inc. | Meeting audio capture and transcription in a collaborative document context |
US10467335B2 (en) | 2018-02-20 | 2019-11-05 | Dropbox, Inc. | Automated outline generation of captured meeting audio in a collaborative document context |
US11106868B2 (en) * | 2018-03-06 | 2021-08-31 | Samsung Electronics Co., Ltd. | System and method for language model personalization |
KR20190126734A (en) | 2018-05-02 | 2019-11-12 | 삼성전자주식회사 | Contextual recommendation |
CN110968246A (en) * | 2018-09-28 | 2020-04-07 | 北京搜狗科技发展有限公司 | Intelligent Chinese handwriting input recognition method and device |
US11595484B2 (en) * | 2019-05-03 | 2023-02-28 | Servicenow, Inc. | Centralized machine learning predictor for a remote network management platform |
US11689379B2 (en) | 2019-06-24 | 2023-06-27 | Dropbox, Inc. | Generating customized meeting insights based on user interactions and meeting media |
CN110349569B (en) * | 2019-07-02 | 2022-04-15 | 思必驰科技股份有限公司 | Method and device for training and identifying customized product language model |
US11257486B2 (en) * | 2020-02-28 | 2022-02-22 | Intuit Inc. | Machine learning to propose actions in response to natural language questions |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050273462A1 (en) * | 2002-11-22 | 2005-12-08 | Accenture Global Services Gmbh | Standardized customer application and record for inputting customer data into analytic models |
US7283997B1 (en) * | 2003-05-14 | 2007-10-16 | Apple Inc. | System and method for ranking the relevance of documents retrieved by a query |
US20080215314A1 (en) * | 2000-09-26 | 2008-09-04 | International Business Machines Corporation | Method for adapting a k-means text clustering to emerging data |
TW200839542A (en) * | 2007-01-08 | 2008-10-01 | Microsoft Corp | Document clustering based on entity association rules |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5675801A (en) * | 1994-09-30 | 1997-10-07 | International Business Machines Corporation | Object oriented system and method for generating target language code |
US5828999A (en) * | 1996-05-06 | 1998-10-27 | Apple Computer, Inc. | Method and system for deriving a large-span semantic language model for large-vocabulary recognition systems |
US6418431B1 (en) | 1998-03-30 | 2002-07-09 | Microsoft Corporation | Information retrieval and speech recognition based on language models |
JP2001134285A (en) * | 1999-11-01 | 2001-05-18 | Matsushita Electric Ind Co Ltd | Speech recognition device |
US6934683B2 (en) * | 2001-01-31 | 2005-08-23 | Microsoft Corporation | Disambiguation language model |
JP4067776B2 (en) * | 2001-03-13 | 2008-03-26 | 三菱電機株式会社 | Language model construction device and speech recognition device |
US7418386B2 (en) * | 2001-04-03 | 2008-08-26 | Intel Corporation | Method, apparatus and system for building a compact language model for large vocabulary continuous speech recognition (LVCSR) system |
US7134075B2 (en) * | 2001-04-26 | 2006-11-07 | International Business Machines Corporation | Conversion of documents between XML and processor efficient MXML in content based routing networks |
JP2003255985A (en) * | 2002-02-28 | 2003-09-10 | Toshiba Corp | Method, device, and program for statistical language model generation |
JP2004109906A (en) * | 2002-09-20 | 2004-04-08 | Advanced Telecommunication Research Institute International | Text clustering method and speech recognizing method |
CN100380373C (en) * | 2002-10-29 | 2008-04-09 | 埃里·阿博 | Knowledge system method and apparatus |
US7406416B2 (en) * | 2004-03-26 | 2008-07-29 | Microsoft Corporation | Representation of a deleted interpolation N-gram language model in ARPA standard format |
KR100816934B1 (en) | 2006-04-13 | 2008-03-26 | 엘지전자 주식회사 | Clustering system and method using search result document |
US7856350B2 (en) * | 2006-08-11 | 2010-12-21 | Microsoft Corporation | Reranking QA answers using language modeling |
US20090299822A1 (en) * | 2006-11-08 | 2009-12-03 | P C Grocery Ltd. | System and method for optimized shopping transactions |
JP2008226104A (en) * | 2007-03-15 | 2008-09-25 | National Institute Of Information & Communication Technology | Information processor and processing method, and program |
US8127270B1 (en) * | 2007-04-09 | 2012-02-28 | Cisco Technology, Inc. | Abstracting transformation for model driven architecture |
US7720870B2 (en) * | 2007-12-18 | 2010-05-18 | Yahoo! Inc. | Method and system for quantifying the quality of search results based on cohesion |
CN101226557B (en) * | 2008-02-22 | 2010-07-14 | 中国科学院软件研究所 | Method for processing efficient relating subject model data |
US20140059514A1 (en) * | 2008-11-14 | 2014-02-27 | Adobe Systems Incorporated | Methods and Systems Utilizing Behavioral Data Models |
US8972927B2 (en) * | 2009-09-25 | 2015-03-03 | Adobe Systems Incorporated | Method and system for providing modeled components |
US8949773B2 (en) * | 2010-03-25 | 2015-02-03 | International Business Machines Corporation | Deriving process models from natural language use case models |
-
2008
- 2008-11-05 CN CN200880132611.4A patent/CN102272754B/en active Active
- 2008-11-05 JP JP2011534984A patent/JP5475795B2/en active Active
- 2008-11-05 KR KR1020117010447A patent/KR101537078B1/en active IP Right Grant
- 2008-11-05 US US13/127,417 patent/US8826226B2/en active Active
- 2008-11-05 WO PCT/CN2008/001845 patent/WO2010051654A1/en active Application Filing
-
2009
- 2009-11-05 TW TW098137647A patent/TWI512502B/en not_active IP Right Cessation
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080215314A1 (en) * | 2000-09-26 | 2008-09-04 | International Business Machines Corporation | Method for adapting a k-means text clustering to emerging data |
US20050273462A1 (en) * | 2002-11-22 | 2005-12-08 | Accenture Global Services Gmbh | Standardized customer application and record for inputting customer data into analytic models |
US7283997B1 (en) * | 2003-05-14 | 2007-10-16 | Apple Inc. | System and method for ranking the relevance of documents retrieved by a query |
TW200839542A (en) * | 2007-01-08 | 2008-10-01 | Microsoft Corp | Document clustering based on entity association rules |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI771468B (en) * | 2017-08-10 | 2022-07-21 | 美商鄧白氏公司 | System and method for dynamic synthesis and transient clustering of semantic attributions for feedback and adjudication |
TWI833072B (en) * | 2021-03-30 | 2024-02-21 | 緯創資通股份有限公司 | Speech recognition system and speech recognition method |
Also Published As
Publication number | Publication date |
---|---|
JP5475795B2 (en) | 2014-04-16 |
CN102272754B (en) | 2015-04-01 |
TW201022964A (en) | 2010-06-16 |
JP2012507809A (en) | 2012-03-29 |
US8826226B2 (en) | 2014-09-02 |
WO2010051654A1 (en) | 2010-05-14 |
KR20110093785A (en) | 2011-08-18 |
US20110296374A1 (en) | 2011-12-01 |
CN102272754A (en) | 2011-12-07 |
KR101537078B1 (en) | 2015-07-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI512502B (en) | Method and system for generating custom language models and related computer program product | |
US10489399B2 (en) | Query language identification | |
CN106575503B (en) | Method and system for session context modeling for dialog understanding systems | |
US11693894B2 (en) | Conversation oriented machine-user interaction | |
US10282419B2 (en) | Multi-domain natural language processing architecture | |
US8386240B2 (en) | Domain dictionary creation by detection of new topic words using divergence value comparison | |
US9146987B2 (en) | Clustering based question set generation for training and testing of a question and answer system | |
JP5379138B2 (en) | Creating an area dictionary | |
US9336298B2 (en) | Dialog-enhanced contextual search query analysis | |
US8688727B1 (en) | Generating query refinements | |
US11861319B2 (en) | Chatbot conducting a virtual social dialogue | |
US9436681B1 (en) | Natural language translation techniques | |
US20130060769A1 (en) | System and method for identifying social media interactions | |
JP5710581B2 (en) | Question answering apparatus, method, and program | |
CN114429133A (en) | Relying on speech analysis to answer complex questions through neuro-machine reading understanding | |
CN110717038B (en) | Object classification method and device | |
US11983502B2 (en) | Extracting fine-grained topics from text content | |
Huang et al. | Multi-task learning for entity recommendation and document ranking in web search | |
US9223833B2 (en) | Method for in-loop human validation of disambiguated features | |
US11847424B1 (en) | Natural language generation | |
EP3660698A1 (en) | Self-learning and adaptable mechanism for tagging documents | |
EP3161666A1 (en) | Semantic re-ranking of nlu results in conversational dialogue applications | |
Naderi et al. | Topic modeling for entity linking using keyphrase | |
Mehdizadeh Naderi et al. | Topic modeling for entity linking using keyphrase |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
MM4A | Annulment or lapse of patent due to non-payment of fees |