US20120254190A1 - Extracting method, computer product, extracting system, information generating method, and information contents - Google Patents
Extracting method, computer product, extracting system, information generating method, and information contents Download PDFInfo
- Publication number
- US20120254190A1 US20120254190A1 US13/423,319 US201213423319A US2012254190A1 US 20120254190 A1 US20120254190 A1 US 20120254190A1 US 201213423319 A US201213423319 A US 201213423319A US 2012254190 A1 US2012254190 A1 US 2012254190A1
- Authority
- US
- United States
- Prior art keywords
- character
- code
- files
- computer
- gram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2237—Vectors, bitmaps or matrices
Definitions
- a technique has been also disclosed that equivalently realizes a high speed full text search by a concatenated character component table searching unit capable of executing sufficient narrowing-down from a given retrieval term in the case of a document consisting of phonograms of a small number of character types such as English (see, e.g., Japanese Patent No. 3497243).
- Retrieval performed based on a map created by using a portion of character codes leads to extraction of a file not including target characters and including non-target characters having the portion of character codes same as those used for creating the map.
- an extracting method is executed by a computer.
- the extracting method includes storing to a storage device, a plurality of files respectively including character units; storing to the storage device, first index information indicative of which file includes at least any one of the character units included in a character unit group having a usage frequency that is less than a predetermined frequency and among a plurality of character units that include common information in a predetermined portion, the usage frequency being indicative of, among the plurality of the files, the extent of files that include a given character unit; storing to the storage device, second index information indicative of which file among the plurality of the files, includes a first character unit that has a usage frequency at least equal to the predetermined frequency and is among the character units that include common information in a predetermined portion, the usage frequency being indicative of, among the plurality of the files, the extent of files that include a given character unit; and referring to the first index information and the second index information to extract from the plurality of the files, a file indicated to include character units in the first index
- FIGS. 1A , 1 B, and 1 C are explanatory diagrams of size reduction of a compression code map of bi-gram character strings.
- FIG. 2 is a chart of sizes related to a division of the compression code map of bi-gram character strings.
- FIG. 3 is an explanatory diagram of an example of division of a compression code map of bi-gram character strings.
- FIG. 4 is an explanatory diagram of a compression code map of bi-gram character strings.
- FIG. 5 is an explanatory diagram of an example of a character string present in a data file group.
- FIG. 6 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 5 .
- FIG. 7 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 5 .
- FIG. 8 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 4 .
- FIG. 9 is an explanatory diagram of an example of a first component compression code map M 4 a.
- FIG. 10 is an explanatory diagram of an example of a second component compression code map M 4 b.
- FIG. 11 is an explanatory diagram of an example of a third component compression code map M 4 c.
- FIG. 12 is an explanatory diagram of an example of a fourth component compression code map M 4 d.
- FIG. 13 is an explanatory diagram of an AND result of “ ”.
- FIG. 14 is an explanatory diagram of a modification example of the second component compression code map.
- FIG. 15 is an explanatory diagram of a modification example of the fourth component compression code map.
- FIG. 16 is an explanatory diagrams of an AND result when the modified appearance maps of FIGS. 14 and 15 are used for “ ”.
- FIG. 17 is an explanatory diagram of a preprocessing example of a compression code map generation.
- FIG. 18 is an explanatory diagram of details of (1) Count of the Number of Appearances.
- FIG. 21 is an explanatory diagram of a correction result for each of character unit.
- FIG. 24 is an explanatory diagram of a leaf structure.
- FIG. 25 is an explanatory diagram of a specified single character structure.
- FIG. 26 is an explanatory diagram of a character code component structure.
- FIG. 27 is an explanatory diagram of a fundamental word structure.
- FIG. 28 is an explanatory diagram of a generation example of the compression code maps.
- FIG. 29 is a block diagram of a hardware configuration of a computer according to an embodiment.
- FIG. 30 is a flowchart of an example of a compression code map generation process procedure executed by a computer using a generation program.
- FIG. 31 is a flowchart of a detailed process procedure example of a count process (step S 3001 ) depicted in FIG. 30 .
- FIG. 32 is a flowchart of a detailed process procedure example of a count process of a data file Fi (step S 3103 ) depicted in FIG. 31 .
- FIG. 33 is an explanatory diagram of a character appearance frequency count table.
- FIG. 34 is a flowchart of a detailed process procedure example of a fundamental word count process (step S 3202 ) depicted in FIG. 32 .
- FIG. 35 is an explanatory diagram of a fundamental word appearance frequency count table.
- FIG. 36 is a flowchart of a detailed process procedure of a longest match retrieval process (step S 3401 ) depicted in FIG. 34 .
- FIG. 37 is a flowchart of a detailed process procedure example of a map assignment number determination process (step S 3002 ) depicted in FIG. 30 .
- FIG. 38 is a flowchart of a detailed process procedure example of a re-count process (step S 3003 ) depicted in FIG. 30 .
- FIG. 39 is a flowchart of a detailed process procedure example of a re-count process of the data file Fi (step S 3803 ).
- FIG. 40 is an explanatory diagram of an upper character code component appearance frequency count table.
- FIG. 41 is an explanatory diagram of a lower character code component appearance frequency count table.
- FIG. 42 is a flowchart of a detailed process procedure of a bi-gram character string identification process (step S 3906 ) depicted in FIG. 39 .
- FIG. 43 is an explanatory diagram of a bi-gram character string appearance frequency count table.
- FIG. 44 is a flowchart of a detailed process procedure example of a Huffman tree generation process (step S 3004 ) depicted in FIG. 30 .
- FIG. 45 is a flowchart of a detailed process procedure example of a branch number specification process (step S 4404 ) depicted in FIG. 44 .
- FIG. 46 is a flowchart of a detailed process procedure of a construction process (step S 4405 ) depicted in FIG. 44 .
- FIG. 47 is a flowchart of a detailed process procedure of a pointer-to-leaf generation process (step S 4603 ) depicted in FIG. 46 .
- FIG. 48 is a flowchart of a detailed process procedure example of a division rank determination process (step S 3005 ) depicted in FIG. 30 .
- FIG. 49 is a flowchart of a detailed process procedure example of a map generation process (step S 3006 ) depicted in FIG. 30 .
- FIG. 50 is a flowchart of a detailed process procedure of a map generation process of the data file Fi (step S 4903 ) depicted in FIG. 49 .
- FIG. 51 is a flowchart of a detailed process procedure example of a fundamental word appearance map generation process (step S 5002 ) depicted in FIG. 50 .
- FIG. 52 is a flowchart of a detailed process procedure example of a specified single character appearance map generation process (step S 5003 ) depicted in FIG. 50 .
- FIG. 53 is a flowchart of a detailed process procedure example of a character code component appearance map generation process (step S 5203 ) depicted in FIG. 52 .
- FIG. 54 is a flowchart of a detailed process procedure example of a bi-gram character string appearance map generation process (step S 5004 ) depicted in FIG. 50 .
- FIG. 55 is a flowchart of a detailed process procedure example of a bi-gram character string appearance map (non-divided) generation process (step S 5404 ).
- FIG. 56 is a flowchart of a detailed process procedure example (part 1 ) of a bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 57 is a flowchart of a detailed process procedure example (part 2 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 58 is a flowchart (part 1 ) of a detailed process procedure example (part 3 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 59 is a flowchart (part 2 ) of a detailed process procedure example (part 3 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 60 is an explanatory diagram of a specific example of a compression process using a 2 N -branch nodeless Huffman tree.
- FIG. 61 is a flowchart of a compression process procedure example of a data file group Fs using the 2 N -branch nodeless Huffman tree automatically executed by a computer.
- FIG. 62 is a flowchart (part 1 ) of a detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- FIG. 63 is a flowchart (part 2 ) of the detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- FIG. 64 is a flowchart (part 3 ) of the detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- FIG. 65 is an explanatory diagram of a flow (A) of a retrieval process using the compression code maps.
- FIG. 66 is an explanatory diagram of a flow (B) of the retrieval process using the compression code maps.
- FIG. 67 is an explanatory diagram of flows (C) and (D) of the retrieval process using the compression code maps.
- FIG. 68 is an explanatory diagram (part 1 ) of collation/expansion according to (D) depicted in FIG. 67 .
- FIG. 69 is an explanatory diagram (part 2 ) of collation/expansion according to (D) depicted in FIG. 67 .
- FIG. 70 is an explanatory diagram (part 3 ) of collation/expansion according to (D) depicted in FIG. 67 .
- FIG. 71 is a flowchart of a retrieval process procedure of an embodiment.
- FIG. 72 is a flowchart (part 1 ) of a detailed process procedure of a file narrowing-down process (step S 7102 ) depicted in FIG. 71 .
- FIG. 73 is a flowchart (part 2 ) of a detailed process procedure of a file narrowing-down process (step S 7102 ) depicted in FIG. 71 .
- FIG. 74 is a flowchart (part 1 ) of a detailed example of an expansion process (step S 7103 ) using a 2 N -branch nodeless Huffman tree depicted in FIG. 71 .
- FIG. 75 is a flowchart (part 2 ) of the detailed example of the expansion process (step S 7103 ) using the 2 N -branch nodeless Huffman tree depicted in FIG. 71 .
- FIG. 76 is an explanatory diagram of a configuration example of an extracting system.
- character unit means a single character, a fundamental word, a character code component, etc., making up text data.
- a data file group is electronic data such as document files, web pages, and e-mails, for example, and is electric data in text format, HTML (HyperText Markup Language) format, and XML (Extensible Markup Language) format, for example.
- a “single character” means a character represented by one character code.
- the length of a character code for a single character differs according to character code type.
- the character code is 16-bit code in the case of Unicode Transformation Format (UTF) 16, 8-bit code in the case of American Standard Code for Information Interchange (ASCII) code, and 8-bit code in the case of shift Japanese Industrial Standard (JIS) code. If a Japanese character is represented by the shift JIS code, two 8-bit codes are combined.
- UTF Unicode Transformation Format
- ASCII American Standard Code for Information Interchange
- JIS shift Japanese Industrial Standard
- “Fundamental words” mean fundamental words taught, for example, in elementary school or junior high school, and reserved words that are represented by particular character strings. Taking an English sentence “This is a . . . ” as an example, the fundamental words are words such as “This”, “is”, and “a” and are classified into a 1000-word level, a 2000-word level, and a several-thousand-word level, to which marks “***”, “**”, and “*” are respectively added in English-Japanese dictionaries.
- the reserved words are predetermined character strings and include, for example, HTML tags (e.g., ⁇ br>).
- a “character code component” means each code obtained by dividing a single character into an upper code and a lower code.
- a single character may be divided into an upper code and a lower code.
- the character code of a single character “ ” is represented as “9D82” in the case of UTF16 and is divided into an upper character code component “0x9D” and a lower character code component “0x82”.
- a “gram” is a character unit.
- one character is a uni-gram.
- each character code component individually makes up a uni-gram. Therefore, a single character formed by an upper character code component and a lower character code component, such as “ ”, is a bi-gram. This embodiment will be described by taking UTF16 as an example of a character, code.
- a “compression code map” is a bit map that integrates appearance maps of respective character units.
- An “appearance map” is a bit string obtained by combining a pointer that specifies a character unit and a bit string indicative of the presence of the character unit in each data file. At the time of a retrieval process, this bit string can be used as an index that indicates whether the target character unit is included, depending on the ON/OFF state of bits.
- a character unit compression code is employed as a pointer that specifies the character unit.
- a compression code map of a string of bi-gram characters is a compression code string obtained by combining the compression code of the first gram and the compression code of the second gram.
- a “bi-gram character string” is a string of uni-gram characters whose character codes are concatenated.
- a character string “ ” includes double concatenated characters “ ”, and “ ”, and “ ”.
- Each of “ ” and “ ” of the double concatenated characters “ ” is a single character that is not divided and therefore, the double concatenated characters “ ” is a bi-gram character string by itself.
- FIGS. 1A , 1 B, and 1 C are explanatory diagrams of size reduction of a compression code map of bi-gram character strings.
- FIG. 1A depicts the size of the compression code map of bi-gram character strings before division.
- FIG. 1A depicts an example when 10 bits are used as a pointer to a compression code map of a first gram (head gram) and 10 bits are used as a pointer to a compression code map of a second gram (tail gram).
- the pointer to the compression code map of the second gram is divided by 5 bits.
- an AND operation of two appearance maps respectively identified from two divided compression code maps enables narrowing-down of the data files in which a bi-gram character string is present.
- an AND operation of four appearance maps respectively identified from four divided compression code maps enables narrowing-down of data files where a bi-gram character string is present.
- the division method that is to be used among the division methods depicted in FIG. 1B and FIG. 10 is determined in advance.
- whether the first gram or the second gram is divided is also determined in advance.
- FIG. 2 is a chart of sizes related to the division of the compression code map of bi-gram character strings.
- the total size of combinations of a bi-gram character string is 1.64 M.
- the total memory size of combinations of bi-gram character strings is 144 K.
- the total memory size of combinations of bi-gram character strings is 10 K.
- FIG. 3 is an explanatory diagram of an example of division of a compression code map of bi-gram character strings.
- a “bi-gram character string group” is a set of bi-gram character strings identified from a data file group.
- a compression code map M of bi-gram character strings is divided according to the appearance frequency of bi-gram character strings. For example, for a bi-gram character string group having a higher appearance frequency (hereinafter, a “higher-order bi-gram character string group”), the two-part or four-part division as depicted in FIG. 1B or 1 C is not performed. On the other hand, for a bi-gram character string group having a lower appearance frequency (hereinafter, a “lower-order bi-gram character string group”), the two-part or four-part division is performed.
- a first component compression code map M 2 a and a second component compression code map M 2 b are generated for the lower-order bi-gram character string group.
- the first component compression code map M 2 a is a set of first component appearance maps specified by a concatenated code of an upper code of the compression code of the first gram (hereinafter, a “first-gram upper code”) and the compression code of the second gram.
- the second component compression code map M 2 b is a set of second component appearance maps specified by a concatenated code of a lower code of the compression code of the first gram (hereinafter, a “first-gram lower code”) and the compression code of the second gram.
- the first component compression code map M 2 a is a set of the first component appearance maps specified by a concatenated code of the compression code of the first gram and an upper code of the compression code of the second gram (hereinafter, a “second-gram upper code”).
- the second component compression code map M 2 b is a set of the second component appearance maps specified by a concatenated code of the compression code of the first gram and a lower code of the compression code of the second gram (hereinafter, a “second-gram lower code”).
- a first component compression code map M 4 a to a fourth divided compression code map M 4 d are generated.
- the first component compression code map M 4 a is a set of first component appearance maps specified by a concatenated code of the first-gram upper code and the second-gram upper code.
- the third component compression code map M 4 c is a set of third component appearance maps specified by a concatenated code of the first-gram lower code and the second-gram upper code.
- the fourth component compression code map M 4 d is a set of fourth component appearance maps specified by a concatenated code of the first-gram lower code and the second-gram lower code.
- FIG. 4 is an explanatory diagram of a compression code map of bi-gram character strings.
- a compression code string e.g., 20 bits
- FIG. 4 it is assumed that “ ” and “ ” are higher-order bi-gram character strings and that “ ” is a lower-order bi-gram character string. Since the division according to appearance frequency is not performed in FIG. 4 , higher-order bi-gram character strings and lower-order bi-gram character strings are mixed in the compression code map of bi-gram character strings.
- FIG. 5 is an explanatory diagram of an example of a character string present in a data file group.
- a character string “ ” includes a higher-order bi-gram character string “ ” and a lower-order bi-gram character string “ ”.
- FIG. 6 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 5 .
- the compression code of the first gram “ ” and the compression code of the second gram “ ” are respectively divided into upper and lower parts and four concatenated codes are generated in a cross-coupled manner.
- FIG. 7 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 5 .
- the compression code of the first gram “ ” and the compression code of the second gram “ ” are respectively divided into two upper and lower parts and four concatenated codes are generated in a cross-coupled manner.
- FIG. 8 is an explanatory diagram of an example of four-part division of the bi-gram character string “ ” depicted in FIG. 4 .
- “ ” is a bi-gram character string making up a portion of “ ”, and is a higher-order bi-gram character string having a higher appearance frequency.
- the compression code of the first gram “ ” and the compression code of a second gram “ ” are respectively divided into two upper and lower parts and four concatenated codes are generated in a cross-coupled manner.
- FIG. 9 is an explanatory diagram of an example of the first component compression code map M 4 a .
- the first component compression code map M 4 a is a compression code map using a concatenated code obtained by concatenating the first-gram upper code and the second-gram upper code (hereinafter, “first concatenated code”) as a pointer.
- FIG. 10 is an explanatory diagram of an example of the second component compression code map M 4 b .
- the second component compression code map M 4 b is a compression code map using a concatenated code obtained by concatenating the first-gram upper code and the second-gram lower code (hereinafter, “second concatenated code”) as a pointer.
- second concatenated code concatenated code from “0x8B” and “0x30” used as a pointer is the second concatenated code of the lower-order bi-gram character string “ ” and the second concatenated code of the higher-order bi-gram character string “ ” and therefore, the codes are the same.
- the bit of the data file is set to ON (“1”) if the higher-order bi-gram character string “ ” appears in the data file.
- a combination of the upper code of “ ” and the lower code of “ ” in the lower-order bi-gram character string “ ” is present only in data files F 1 , F 3 , and Fn, a combination of the upper code of “ ” and the lower code of “ ” in the higher-order bi-gram character string “ ” is present in data files F 1 to Fn.
- the appearance map of the second concatenated code (concatenated code from “0x8B” and “0x30”) in the second component compression code map M 4 b is a bit string obtained by an OR operation of the appearance map of the second concatenated code of the lower-order bi-gram character string “ ” and the appearance map of the second concatenated code of the higher-order bi-gram character string “ ”.
- a data file without the lower-order bi-gram character string “ ” may be retrieved.
- FIG. 11 is an explanatory diagram of an example of the third component compression code map M 4 c .
- the third component compression code map M 4 c is a compression code map using a concatenated code obtained by concatenating the first-gram lower code and the second-gram upper code (hereinafter, “third concatenated code”) as a pointer.
- FIG. 12 is an explanatory diagram of an example of the fourth component compression code map M 4 d .
- the fourth component compression code map M 4 d is a compression code map using a compression code string of a concatenated code obtained by concatenating the first-gram lower code and the second-gram lower code (hereinafter, “fourth concatenated code”) as a pointer.
- fourth concatenated code a concatenated code obtained by concatenating the first-gram lower code and the second-gram lower code
- a fourth concatenated code (concatenated code from “0x30” and “0x30”) used as a pointer is the fourth concatenated code of the lower-order bi-gram character string “ ” and the fourth concatenated code of the higher-order bi-gram character string “ ” and therefore, the codes are the same.
- the bit of the data file is set to ON (“1”) if the higher-order bi-gram character string “ ” appears in the data file.
- a combination of the lower code of “ ” and the lower code of “ ” in the lower-order bi-gram character string “ ” is present only in data files F 2 and F 3
- a combination of the lower code of “ ” and the lower code of “ ” in the higher-order bi-gram character string “ ” is present in data files F 1 to Fn.
- the appearance map of the fourth concatenated code (concatenated code from “0x30” and “0x30”) in the fourth component compression code map M 4 d is a bit string obtained by an OR operation of the appearance map of the fourth concatenated code of the lower-order bi-gram character string “ ” and the appearance map of the fourth concatenated code of the higher-order bi-gram character string “ ”.
- a data file without the lower-order bi-gram character string “ ” may be retrieved.
- FIG. 13 is an explanatory diagram of an AND result of “ ”.
- (A) depicts an AND result 1301 of four appearance maps of “ ” and
- (B) depicts an AND result 1302 of four appearance maps of “ ”.
- (C) depicts an AND result 1303 from an AND operation of both of the AND results 1301 and 1302 of (A) and (B).
- the AND result of (C) reveals that “ ” is possibly present in the data files F 1 to F 3 and F(n ⁇ 1) and is not present in other data files.
- FIG. 14 is an explanatory diagram of a modification example of the second component compression code map.
- the appearance map of the upper and lower code string of the higher-order bi-gram character string “ ” is subjected to NULL bit (all zero) conversion.
- NULL bit all zero
- FIG. 15 is an explanatory diagram of a modification example of the fourth component compression code map.
- the appearance map of the fourth concatenated code of the higher-order bi-gram character string “ ” is subjected to NULL bit (all zero) conversion.
- NULL bit all zero
- FIG. 16 is an explanatory diagrams of an AND result when the modified appearance maps of FIGS. 14 and 15 are used for “ ”.
- (A) depicts the same contents as the AND operation of (A) in FIG. 13 and therefore, the AND results 1301 and 1601 are the same.
- (B) of FIG. 16 among four appearance maps, the appearance maps of FIGS. 14 and 15 are employed as the appearance map in the second component compression code map M 4 b and the appearance map in the fourth divided compression code map M 4 d.
- an AND result 1602 of (B) in FIG. 16 is different from the AND result 1302 of (B) in FIG. 13 .
- (C) depicts an AND result 1603 from an AND operation of both of the AND results 1601 and 1602 in (A) and (B) of FIG. 16 .
- the AND result 1603 of (C) in FIG. 16 reveals that “ ” is present in the data file F 3 and is not present in the other data files.
- the AND result 1303 in (C) of FIG. 13C it is known that accuracy of narrowing-down of data files is improved.
- the 2 N -branch nodeless Huffman tree is a Huffman tree having 2 N branches branching from a root to directly point a leaf with one or multiple branches. No node (inner node) exists. Since no node exists and a leaf is directly hit, and the expansion rate can be accelerated as compared to a normal Huffman tree having nodes.
- a leaf is a structure that includes the corresponding character unit and the compression code thereof. A leaf is also referred to as a leaf structure. The number of branches assigned to a leaf depends on the compression code length of the compression code present in the leaf to which the branches are assigned. A preprocessing example will hereinafter be described.
- FIG. 17 is an explanatory diagram of the preprocessing example of the compression code map generation.
- the computer calculates a compression code length for each of the character units, based on the count result acquired at (1). For example, the computer calculates an appearance rate for each of the character units. The appearance rate can be obtained by dividing the number of appearances of a character unit by the total number of appearances of all the character units. The computer obtains an occurrence probability corresponding to the appearance rate and derives a compression code length from the occurrence probability.
- the occurrence probability is expressed by 1/2 x .
- X is an exponent.
- a compression code length is the exponent X of the occurrence probability.
- the compression code length is determined according to which of the following ranges the occurrence probability falls within.
- AR denotes an appearance rate.
- compression code length is 1 bit.
- 1/2 1 >AR ⁇ 1/2 2 compression code length is 2 bits.
- 1/2 2 >AR ⁇ 1/2 3 compression code length is 3 bits.
- the computer counts the number of leaves for each compression code length to specify the number of leaves for each compression code length.
- the maximum compression code length is 17 bits.
- the number of leaves is the number of types of character units. Therefore, if the number of leaves at the compression code length of 5 bits is two, this indicates that two character units assigned with a five-bit compression code are present.
- the computer assigns the number of branches per leaf for each compression code length. For example, the number of branches per leaf is determined as 2 0 , 2 1 , 2 2 , 2 3 , 2 4 , 2 5 , 2 6 , and 2 7 for the compression code lengths after the correction in descending order.
- the number of branches per leaf is 1. To each of the character units assigned with a compression code having the compression code length of 11 bits, only one branch is assigned. On the other hand, while the total number of the character units (the number of leaves) assigned with a compression code having the compression code length of 6 bits is 6, the number of branches per leaf is 32. To each of the character units assigned with a compression code having the compression code length of 6 bits, 32 branches are assigned. The correction of the number of leaves of (4) may be performed as needed and may be omitted.
- the leaf structure is a data structure formed by correlating a character unit, a compression code length thereof, and a compression code of the compression code length. For example, a compression code length of a character “0” ranked in the first place of the appearance ranking is 6 bits and the compression code is “000000”.
- the computer then generates for each leaf structure, a pointer to the leaf.
- the pointer to a given leaf is a bit string obtained by concatenating the compression code in the given leaf structure and a bit string that corresponds to one number among numbers equivalent in quantity to the number of branches per leaf. For example, since the compression code length of the compression code “000000” assigned to the character “0” of the leaf L 1 is 6 bits, the number of branches of the leaf L 1 is 32.
- the leading six bits of the pointers to the leaf L 1 is the compression code “000000”.
- 32 types of 5-bit bit strings are subsequent bit strings of the compression code “000000”. Therefore, the pointers to the leaf L 1 are 32 types of 11-bit bit strings having a leading six bits fixed to “000000”. If the number of branches per leaf is one, one pointer to leaf exists, and the compression code and the pointer to leaf are the same bit strings. Details of (6) Generation of Pointer to Leaf will be described with reference to FIG. 22 .
- FIG. 18 is an explanatory diagram of details of (1) Count of the Number of Appearances.
- the computer executes three phases, i.e., (A) count from the data file group Fs, (B) sort in descending order of appearance frequency, and (C) extract up to the rank of the target appearance rate.
- the three phases will hereinafter be described separately for fundamental words and single characters.
- the computer reads the data file group Fs to count the appearance frequency (number of appearances) of fundamental words.
- the computer refers to a fundamental word structure and, if a character string identical to a fundamental word in the fundamental word structure is present in the data files, the computer adds one to the appearance frequency of the fundamental word (default value is zero).
- the fundamental word structure is a data structure having descriptions of fundamental words.
- the computer sorts a fundamental word appearance frequency count table in descending order of the appearance frequency. In other words, the table is sorted in the order from the highest appearance frequency and the fundamental words are ranked in the order from the highest appearance frequency.
- the computer reads the data file group Fs to count the appearance frequency of single characters. For example, the computer adds one to the appearance frequency of the single characters (default value is zero).
- the computer sorts a single-character appearance frequency count table in descending order of the appearance frequency. In other words, the table is sorted in the order form the highest appearance frequency and the single characters are ranked in the order from the highest appearance frequency.
- the computer then refers to the fundamental word appearance frequency count table after the sorting at (B1) to extract the fundamental words ranked within a target appearance rate Pw. For example, the computer calculates the appearance rate Pw to each rank by using the sum of appearance frequencies (the total appearance frequency) of all the fundamental words as a denominator and successively summing the appearance frequencies in descending order from the fundamental word ranked in the first place to obtain a numerator.
- the computer then refers to the single-character appearance frequency count table after the sorting at (B2) to extract the single characters ranked within a target appearance rate Pc. For example, the computer calculates an appearance rate to each rank by using the sum of appearance frequencies (the total appearance frequency) of all the single characters as a denominator and successively summing the appearance frequencies in descending order from the single character ranked in the first place to obtain a numerator.
- a single character extracted at (C21) is referred to as a “specified single character” so as to distinguish the character from original single characters.
- a single character excluded from among the specified single characters (hereinafter, a “nonspecified single character”) has an appearance frequency that is lower than each of the specified single characters and therefore, the character code thereof is divided.
- a character code of a nonspecified single character is divided into a character code of upper bits and a character code of lower bits.
- the character code is divided into a character code of an upper 8 bits and a character code of a lower 8 bits.
- Each of the character code components is represented by a code from 0x00 to 0xFF.
- the character code of the upper bits is an upper character code component and the character code of the lower bits is a lower character code component.
- a character unit table of FIG. 19 is a table reflecting the count result at (1) of FIG. 17 and includes a rank column, a expansion type column, a code column, a character column, an appearance count column, a total appearance count column, an appearance rate column, an uncorrected occurrence probability column, and a compression code length column set for each character unit.
- the rank column to the total appearance count column store information obtained as a result of re-sorting.
- the rank column In the rank column, the rank (in ascending order) is written in descending order of the number of appearances of character unit.
- the type of character unit In the expansion type column in character unit column, the type of character unit is written.
- a single character of a 16-bit code is denoted by “16”.
- An 8-bit character code component is denoted by “8”. “FW” indicates a fundamental word.
- the code column in the character unit column the code of the specified single character or the character code component is written. In the case of a fundamental word, this column is left blank.
- the character column in the character unit column the character or the fundamental word is written.
- the appearance count column the number of appearances of character unit in the data file group Fs is written.
- the total appearance count column the total number of appearances of all the character units is written.
- the appearance rate column the value obtained by dividing the number of appearances by the total number is written as an appearance rate.
- the occurrence probability corresponding to the appearance rate is written.
- the compression code length column the compression code length corresponding to the occurrence probability, i.e., an exponent y of the occurrence probability 1/2 y is written as the compression code length.
- the number of leaves (the total number of types of character units) counted according the compression code lengths in the character unit table depicted in FIG. 19 is the uncorrected number of leaves in FIG. 20 .
- Correction A is correction that consolidates the number of leaves assigned to compression code lengths equal to or greater than the upper limit length N of the compression code length (i.e., the exponent N of the maximum branch number 2 N of the 2 N -branch nodeless Huffman tree) to the upper limit length N of the compression code length.
- the maximum compression code length before the correction is 17 bits
- the computer determines whether the sum of occurrence probabilities is not greater than one.
- the correction B is correction to update the number of leaves without changing the compression code lengths (5 bits to 12 bits) in the correction A. For example, this is correction performed if the sum of occurrence probabilities in the correction A is not equal to or greater than the threshold value t, or not equal to or less than 1. For example, two types of the correction B exist.
- correction B if the sum of occurrence probabilities is less than the threshold value t, the sum of occurrence probabilities is increased until the maximum value of the sum of occurrence probabilities equal to or less than 1 is acquired, e.g., until the sum of occurrence probabilities converges to a maximum asymptotic value (hereinafter, correction B + ).
- correction B + if the sum of occurrence probabilities is greater than 1, the sum of occurrence probabilities is reduced until the maximum value equal to or less than 1 is acquired after the sum of occurrence probabilities turns to be less than 1, for example, until the sum of occurrence probabilities converges to a maximum asymptotic value (hereinafter, correction B ⁇ ).
- the correction B is performed. The same correction is performed by dividing the number of leaves by the sum of occurrence probabilities in the correction B regardless of whether the correction B + or correction B ⁇ .
- the number of leaves at each compression code length in the correction A is divided by the sum of occurrence probabilities (1.146) for the previous correction (the correction A in this case) to update the number of leaves. Digits after the decimal point may be omitted or rounded off.
- the number of leaves at the upper limit N of the compression code length in the correction A is obtained by subtracting the total number of leaves at the compression code lengths in the correction B ⁇ 1 (excluding the number of leaves at the upper limit length N of the compression code length) from the total number of leaves (1305), rather than dividing by the sum of occurrence probabilities (1.146) for the previous correction (the correction A in this case). In this case, the number of leaves is 1208.
- the computer subsequently obtains the sum of occurrence probabilities in the correction B ⁇ 1 from a computing process identical to the case of the correction A.
- the computer determines whether the sum of occurrence probabilities in the correction B ⁇ 1 converges to the maximum asymptotic value less than or equal to 1. If the sum of occurrence probabilities in the correction B ⁇ 1 does not converge to the maximum asymptotic value less than or equal to 1, transition to a second execution of the correction B ⁇ (correction B ⁇ 2) is made. In the case of convergence to the maximum asymptotic value, the number of leaves at each compression code length at this point is fixed without transitioning to the correction B ⁇ 2. Since the sum of occurrence probabilities “1.042” updated in the correction B ⁇ 1 is greater than 1 and does not converge to the maximum asymptotic value, transition to the correction B ⁇ 2 is made.
- the number of leaves at each compression code length in the correction B ⁇ 1 is divided by the sum of occurrence probabilities (1.042) for the previous correction (the correction B ⁇ 1 in this case) to update the number of leaves. Digits after the decimal point may be omitted or rounded off.
- the number of leaves at the upper limit N of the compression code length in the correction B ⁇ 1 is obtained by subtracting the total number of leaves at the compression code lengths in the correction B ⁇ 2 (excluding the number of leaves at the upper limit length N of the compression code length) from the total number of leaves (1305), rather than dividing by the sum of occurrence probabilities (1.042) for the previous correction (the correction B ⁇ 1 in this case). In this case, the number of leaves is 1215.
- the computer subsequently obtains the sum of occurrence probabilities in the correction B ⁇ 2 from a computing process identical to the case of the correction B ⁇ 1.
- the computer determines whether the sum of occurrence probabilities in the correction B ⁇ 2 converges to the maximum asymptotic value less than or equal to 1. If the sum of occurrence probabilities in the correction B ⁇ 2 does not converge to the maximum asymptotic value less than or equal to 1, transition to a third execution of the correction B ⁇ (correction B ⁇ 3) is made.
- the number of leaves at each compression code length at this point is fixed without transitioning to the correction B ⁇ 3. Since the sum of occurrence probabilities “0.982” updated in the correction B ⁇ 2 is smaller than 1 but it is not clear whether the sum converges to the maximum asymptotic value, transition to the correction B ⁇ 3 is made.
- the number of leaves at each compression code length in the correction B ⁇ 2 is divided by the sum of occurrence probabilities (0.982) for the previous correction (the correction B ⁇ 2 in this case) to update the number of leaves. Digits after the decimal point may be omitted or rounded off.
- the number of leaves at the upper limit N of the compression code length in the correction B ⁇ 2 is obtained by subtracting the total number of leaves at the compression code lengths in the correction B ⁇ 3 (excluding the number of leaves at the upper limit length N of the compression code length) from the total number of leaves (1305), rather than dividing by the sum of occurrence probabilities (0.982) for the previous correction (the correction B ⁇ 2 in this case). In this case, the number of leaves is 1215.
- the computer subsequently obtains the sum of occurrence probabilities in the correction B ⁇ 3 from a computing process identical to the case of the correction B ⁇ 2.
- the computer determines whether the sum of occurrence probabilities in the correction B ⁇ 3 converges to the maximum asymptotic value less than or equal to 1. If the sum of occurrence probabilities in the correction B ⁇ 3 does not converge to the maximum asymptotic value less than or equal to 1, transition to a fourth execution of the correction B ⁇ (correction B ⁇ 4) is made. In the case of convergence to the maximum asymptotic value, the number of leaves at each compression code length at this point is fixed without transitioning to the correction B ⁇ 4.
- the sum of occurrence probabilities “0.982” updated in the correction B ⁇ 3 is the same value as the sum of occurrence probabilities “0.982” updated in the correction B ⁇ 2.
- the number of leaves at the compression code lengths in the correction B ⁇ 3 is the same as the number of leaves at the compression code lengths in the correction B ⁇ 2.
- the computer determines that the sum of occurrence probabilities converges to the maximum asymptotic value and the number of leaves is fixed.
- the correction B ⁇ is continued until the number of leaves is fixed.
- the number of leaves at each compression code length is fixed in the correction B ⁇ 3.
- the computer calculates the number of branches per leaf for each compression code length.
- a subtotal of the number of branches is the product of multiplying the number of branches per leaf by the fixed number of leaves for each compression code length.
- FIG. 21 is an explanatory diagram of a correction result for each of character unit.
- the correction results for the correction A and the corrections B ⁇ 1 to B ⁇ 2 are added to the character unit table.
- shorter compression code lengths are assigned sequentially from the character unit ranked in the first place of the rank column.
- the number of leaves is 6 when the compression code length is 6 bits; the number of leaves is 18 when the compression code length is 7 bits; . . . ; and the number of leaves is 1215 when the compression code length is 11 bits. Therefore, the compression code length of 6 bits is assigned to the character units ranked in the first to sixth places (corresponding to 6 leaves); the compression code length of 7 bits is assigned to the character units ranked in the 7th to 24th places (corresponding to 18 leaves); . . . ; and the compression code length of 11 bits is assigned to the character units ranked in the 91st to 1305th places (corresponding to 1215 leaves).
- the computer assigns a compression code to each of the character units to generate the leaf structures, based on the character unit, the compression code length assigned to the character unit, and the number of leaves at each compression code length. For example, since the compression code length of 5 bits is assigned to the single character “0” ranked in the first place for the appearance rate, the compression code is “000000”. Therefore, a structure of a leaf L 1 is generated that includes the compression code “000000”, the compression code length “6”, and the character unit “0”.
- the compression code length is 5 bits to 11 bits in the correction process described above, a compression code map of bi-gram character strings may be divided in some cases and therefore, the compression code length may be corrected to be an even number of bits.
- the character unit of the compression code length of 5 bits and 7 bits is corrected to be 6 bits; the character unit of 9 bits is corrected to be 8 bits; and the character unit of 11 bits is corrected to be 10 bits.
- FIG. 22 depicts a pointer to a leaf when the upper limit N of the compression code length is 11 bits.
- N the number of leaves
- compression codes “000000” to “000101” are assigned.
- the leading 6 bits of the pointers to the leaf are identical to the compression codes and the subsequent 5 bits are 32 types of bit strings. Therefore, 32 types of the pointers to the leaf are generated for each of the compression codes having the compression code length of 6 bits.
- a root structure stores the pointers to leaf. The pointers to leaf can specify the leaf structures to be pointed to.
- 32 pointers to the leaf are generated for a leaf structure that stores a compression code having the compression code length of 6 bits as depicted in FIG. 22 . Therefore, for the structure of the leaf L 1 , 32 pointers L 1 P( 1 ) to L 1 P( 32 ) to the leaf L 1 are stored in the root structure. The same applies to the structure of a leaf L 2 to the structure of a leaf L 6 . The structures of a leaf L 7 and thereafter are as depicted in FIG. 23 .
- FIG. 24 is an explanatory diagram of the leaf structure.
- the leaf structure is a data structure having first to fourth areas.
- the first area stores a compression code and a compression code length thereof.
- the second area stores the leaf label and expansion type (see FIG. 19 ).
- the third area stores a 16-bit character code that is a specified single character, an 8-bit character code component obtained by dividing a character code of a nonspecified single character, or a pointer to fundamental word depending on the expansion type.
- the pointer to fundamental word specifies a fundamental word within the fundamental word structure.
- a collation flag is also stored.
- the collation flag is “0” by default. In the case of “0”, a character to be expanded is directly written in an expansion buffer and, in the case of “1”, the character is interposed between a ⁇ color> tag and a ⁇ /color> tag and written in the expansion buffer.
- the fourth area stores a code type and a code category.
- the code type is information that identifies which among a numeric character, an alphabetic character, a special symbol, katakana, hiragana, or kanji a character code corresponds to, or whether a character code is a pointer to a reserved word.
- the code category is information that identifies whether the character code is 16-bit or 8-bit. In the case of 16-bit character code or in the case of a reserved word, “1” is assigned as the code category and, in the case of 8-bit character code component, “0” is assigned as the code category.
- FIG. 25 is an explanatory diagram of a specified single character structure.
- a specified single character structure 2500 is a data structure that stores specified single character codes e# and pointers to leaves L# thereof. For example, for example, when the computer acquires the count result from the data file group Fs, the computer stores the specified single character codes e# to the specified single character structure 2500 . When the 2 N -branch nodeless Huffman tree is constructed, the computer stores pointers to the specific character codes e# into the specified single character structure 2500 , which correspond to compression codes stored in the structures of leaves in the 2 N -branch nodeless Huffman tree.
- the computer stores pointers to the leaves corresponding to the specified single character codes e# in the 2 N -branch nodeless Huffman tree in a manner correlated with the corresponding specified single character codes e# in the specified single character structure 2500 .
- the specified single character structure 2500 is generated.
- FIG. 26 is an explanatory diagram of a character code component structure.
- a character code component structure 2600 stores character code components and pointers to leaves L# thereof. For example, for example, when the computer acquires the count result from the data file group Fs, the computer stores the character code components into the character code component structure 2600 . When the 2 N -branch nodeless Huffman tree is constructed, the computer stores pointers to the character code components in the character code component structure 2600 , which correspond to compression codes stored in the structures of leaves in the 2 N -branch nodeless Huffman tree.
- the computer stores pointers to the leaves corresponding to the character code components in the 2 N -branch nodeless Huffman tree in a manner correlated with the corresponding character code components in the character code component structure 2600 .
- the character code component structure 2600 is generated.
- FIG. 27 is an explanatory diagram of a fundamental word structure.
- a fundamental word structure 2700 is a data structure that stores fundamental words and pointers to leaves L# thereof.
- the fundamental word structure 2700 stores the fundamental words in advance.
- the computer stores pointers to the fundamental words in the fundamental word structure 2700 , which correspond to compression codes stored in the structures of leaves in the 2 N -branch nodeless Huffman tree.
- the computer stores pointers to the leaves corresponding to the fundamental words in the 2 N -branch nodeless Huffman tree in a manner correlated with the corresponding fundamental words in the fundamental word structure 2700 .
- a compression code map of single characters a compression code map of upper character code components, a compression code map of lower character code components, a compression code map of fundamental words, and a compression code map of bi-gram character strings can be generated.
- a detailed generation example of the compression code map of single characters, the compression code map of upper character code components, the compression code map of lower character code components, and the compression code map of bi-gram character strings will hereinafter be described.
- the compression code map of fundamental words is generated in the same way as the compression code map of single characters and will not be described.
- FIG. 28 is an explanatory diagram of a generation example of the compression code maps. In FIG. 28 , it is assumed that a character string “ ” is described in a data file Fi.
- the head character “ ” is the target character. Since the target character “ ” is a specified single character, the compression code of the specified single character “ ” is obtained by accessing the 2 N -branch nodeless Huffman tree, to identify the appearance map of the specified single character “ ”. If not generated, an appearance map for the specified single character “ ” is generated that has the compression code of the specified single character “ ” as a pointer and a bit string indicative of the presence in data files, each bit being set to zero. In the appearance map of the specified single character “ ”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the target character is shifted by one gram to define “ ” as the target character. Since the target character “ ” is a specified single character, the compression code of the specified single character “ ” is obtained by accessing the 2 N -branch nodeless Huffman tree, to identify the appearance map of the specified single character “ ”. If not generated, an appearance map for the specified single character “ ” is generated that has the compression code of the specified single character “ ” as a pointer and a bit string indicative of the presence in data files, each bit being set to zero. In the appearance map of the specified single character “ ”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the appearance map of the bi-gram character string “ ” is identified by the compression code string of “ ” obtained by connecting the compression code of “ ” and the compression code of “ ”. If not generated, an appearance map for the bi-gram character string “ ” is generated that has the compression code of “ ” as a pointer and a bit string indicative of the presence in data files, each bit being set to zero. In the appearance map of the bi-gram character string “ ”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the target character is shifted by one gram to define “ ” as the target character.
- the target character “ ” is processed in the same way as (B) and, in the appearance map of the specified single character “ ”, the bit of the data file Fi is turned on (“0” ⁇ “1”). Similarly, in the appearance map of the bi-gram character string “ ”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the target character is shifted by one gram to define “ ” as the target character. Since the target character “ ” is not a specified single character, the character code “0x8131” of the target character “ ” is divided into the upper character code component “0x81” and the lower character code component “0x31”. The target character is then defined as the upper character code component “0x81”.
- the upper character code component “0x81” is processed in the same way as a specified single character and, in the appearance map of the upper character code component “0x81”, the bit of the data file Fi is turned on (“0” ⁇ “1”). Similarly, in the appearance map of the bi-gram character string “ 0x81”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the target character is shifted by one gram to define the lower character code component “0x31” of the character “ ” as the target character.
- the lower character code component “0x31” is processed in the same way and in the appearance map of the lower character code component “0x31”, the bit of the data file Fi is turned on (“0” ⁇ “1”). Similarly, in the appearance map of the bi-gram character string “0x81 0x31”, the bit of the data file Fi is turned on (“0” ⁇ “1”).
- the same process is executed at (F) to (I) and the completion of the process of the last data file Fn leads to the generation of the compression code map of single characters, the compression code map of upper character code components, the compression code map of lower character code components, and the compression code map of bi-gram character strings.
- a first concatenated code to a fourth concatenated code are generated for the bi-gram character string (in the case of four-part division).
- An appearance map group having the first concatenated code to the fourth concatenated code as pointers is generated and the bit of the data file Fi is turned on (“0” ⁇ “1”).
- FIG. 29 is a block diagram of a hardware configuration of the computer according to the embodiment.
- the computer includes a central processing unit (CPU) 2901 , a read-only memory (ROM) 2902 , a random access memory (RAM) 2903 , a magnetic disk drive 2904 , a magnetic disk 2905 , an optical disk drive 2906 , an optical disk 2907 , a display 2908 , an interface (I/F) 2909 , a keyboard 2910 , a mouse 2911 , a scanner 2912 , and a printer 2913 , respectively connected by a bus 2900 .
- CPU central processing unit
- ROM read-only memory
- RAM random access memory
- I/F interface
- the CPU 2901 governs overall control of the computer.
- the ROM 2902 stores therein programs such as a boot program.
- the ROM 2902 also stores therein a generation program of the compression code map and a search program using the compression code map or the code map.
- the RAM 2903 is used as a work area of the CPU 2901 .
- the CPU 2901 can read out the programs stored in the ROM 2902 into the RAM 2903 for execution.
- the magnetic disk drive 2904 under the control of the CPU 2901 , controls the reading and writing of data with respect to the magnetic disk 2905 .
- the magnetic disk 2905 stores therein data written under control of the magnetic disk drive 2904 .
- the optical disk drive 2906 under the control of the CPU 2901 , controls the reading and writing of data with respect to the optical disk 2907 .
- the optical disk 2907 stores therein data written under control of the optical disk drive 2906 , the data being read by a computer.
- the display 2908 displays data such as text, images, functional information, etc., in addition to a cursor, icons, and/or tool boxes.
- a cathode ray tube (CRT), a thin-film-transistor (TFT) liquid crystal display, a plasma display, etc., may be employed as the display 2908 .
- the I/F 2909 is connected to a network 2914 such as the local area network (LAN), the wide area network (WAN), and the Internet via a communication line, and to other apparatuses through the network 2914 .
- the I/F 2909 administers an internal interface with the network 2914 and controls the input/output of data from/to external apparatuses.
- a modem or a LAN adaptor may be employed as the I/F 2909 .
- the keyboard 2910 includes keys for inputting letters, numerals, and various instructions and performs the input of data. Alternatively, a touch-panel-type input pad or numeric keypad, etc. may be adopted.
- the mouse 2911 is used to move the cursor, select a region, or move and change the size of windows.
- a track ball or a joy stick may be adopted provided each respectively has a function similar to a pointing device.
- the scanner 2912 optically reads an image and takes in the image data into the computer.
- the scanner 2912 may have an optical character reader (OCR) function as well.
- OCR optical character reader
- the printer 2913 prints image data and text data.
- the printer 2913 may be, for example, a laser printer or an ink jet printer.
- the computer may be a server or a desktop personal computer as well as a portable terminal such as a portable telephone, a smartphone, an e-book device, and a notebook personal computer.
- FIG. 76 this embodiment may be implemented for multiple computers.
- a system depicted in FIG. 76 includes computers 7601 to 7603 that may include the hardware depicted in FIG. 29 , a network 7604 , a switch 7605 , and a radio base station 7607 .
- I/F included in the computer 7603 has a radio communication function.
- a process of generating compression code maps or code maps for contents including files may be executed by the computer 7601 for distribution to the computer 7602 and the computer 7603 , and the computer 7602 and the computer 7603 may each execute a retrieval process for the distributed contents.
- a process of generating compression code maps or code maps for contents including files may be executed by the computer 7601 , and the computer 7601 may accept a retrieval request for the contents from the computer 7602 or the computer 7603 and may execute a retrieval process to return a result of the executed retrieval process to each of the computer 7602 and the computer 7603 .
- each of the computers 7601 to 7603 may be a server or a desktop personal computer as well as a portable terminal such as a portable telephone, a smartphone, an e-book device, and a notebook personal computer.
- FIG. 30 is a flowchart of the example of the compression code map generation process procedure executed by the computer using the generation program.
- the computer executes a count process (step S 3001 ), a map assignment number determination process (step S 3002 ), a re-count process (step S 3003 ), a Huffman tree generation process (step S 3004 ), a division rank determination process (step S 3005 ), and a map generation process (step S 3006 ).
- the count process is a process of counting the numbers of appearances (also called appearance frequencies) of single characters and fundamental words in the data file group Fs.
- the map assignment number determination process is a process of determining the map assignment numbers of the single characters and the fundamental words counted at the count process (step S 3001 ).
- Single characters and fundamental words in the appearance ranks corresponding to the map assignment numbers are respectively defined as the specified single characters and the fundamental words.
- the re-count process is a process of dividing non-specific characters other than the specified single characters among the single characters into upper character code components and lower character code components, and counting the respective numbers of appearances. In the re-count process (step S 3003 ), the numbers of appearances of bi-gram character strings are also counted.
- the Huffman tree generation process (step S 3004 ) is a process of generating a 2 N -branch nodeless Huffman tree as depicted in FIGS. 19 to 24 .
- the division rank determination process (step S 3005 ) is a process of determining the division rank.
- the division rank is the appearance rank of bi-gram character strings that divide the compression code maps of bi-gram character strings re-counted in the re-count process (step S 3003 ). As a result, the compression code maps of the bi-gram character strings below the division rank are divided.
- the map generation process (step S 3006 ) is a process of generating the compression code maps of specified single characters, fundamental words, upper character code components, lower character code components, and bi-gram character strings.
- the compression code maps of bi-gram character strings are divided. As a result, the reduction of retrieval noise can be achieved.
- FIG. 31 is a flowchart of a detailed process procedure example of the count process (step S 3001 ) depicted in FIG. 30 .
- the computer executes the count process on the data file Fi (step S 3103 ). Details of the count process of the data file Fi (step S 3103 ) will be described with reference to FIG. 32 .
- the computer determines whether the file number i satisfies i>n (where n is the total number of data files F 1 to Fn) (step S 3104 ).
- step S 3104 If i>n is not satisfied (step S 3104 : NO), the computer increments i (step S 3105 ) and returns to step S 3102 . On the other hand, if i>n is satisfied (step S 3104 : YES), the computer goes to the map assignment number determination process (step S 3002 ) depicted in FIG. 30 and terminates the count process (step S 3001 ). With this count process (step S 3001 ), the count process of the data file Fi (step S 3103 ) can be executed for each of the data files Fi.
- FIG. 32 is a flowchart of a detailed process procedure example of the count process of the data file Fi (step S 3103 ) depicted in FIG. 31 .
- the computer defines the head character of the data file Fi as the target character (step S 3201 ) and executes a fundamental word count process (step S 3202 ). Details of the fundamental word count process (step S 3202 ) will be described with reference to FIG. 34 .
- the computer then increments the number of appearances of the target character by one in the character appearance frequency count table (step S 3203 ).
- FIG. 33 is an explanatory diagram of the character appearance frequency count table.
- a character appearance frequency count table 3300 is stored in a storage device such as the RAM 2903 or the magnetic disk 2905 and the number of appearances is incremented by one each time a corresponding character appears.
- the computer determines whether the target character is the tail character of the data file Fi (step S 3204 ). If the target character is not the tail character of the data file Fi (step S 3204 : NO), the computer shifts the target character by one character toward the tail (step S 3205 ) and returns to step S 3202 .
- step S 3204 if the target character is the tail character of the data file Fi (step S 3204 : YES), the computer goes to step S 3104 and terminates the count process of the data file Fi (step S 3103 ). With this count process of the data file Fi (step S 3103 ), the appearance frequencies of the fundamental words and the single characters present in the data file group Fs can be counted.
- FIG. 34 is a flowchart of a detailed process procedure example of the fundamental word count process (step S 3202 ) depicted in FIG. 32 .
- the computer executes a longest match retrieval process (step S 3401 ) and determines whether a longest matching fundamental word is present (step S 3402 ). Details of the longest match retrieval process (step S 3401 ) will be described with reference to FIG. 36 . If the longest matching fundamental word is present (step S 3402 ; YES), the computer increments the number of appearances of the longest matching fundamental word by one in the fundamental word appearance frequency count table (step S 3403 ) and goes to step S 3203 .
- FIG. 35 is an explanatory diagram of the fundamental word appearance frequency count table.
- a fundamental word appearance frequency count table 3500 is stored in the storage device such as the RAM 2903 or the magnetic disk 2905 and the number of appearances is incremented by one each time a corresponding fundamental word appears.
- step S 3402 if no longest matching fundamental word is present (step S 3402 ; NO), the computer goes to step S 3203 . This leads to the termination of the fundamental word count process (step S 3202 ). With the fundamental word count process (step S 3202 ), fundamental words can be counted by the longest match retrieval process (step S 3401 ) and therefore, fundamental words of longer character strings can preferentially be counted.
- FIG. 36 is a flowchart of a detailed process procedure of the longest match retrieval process (step S 3401 ) depicted in FIG. 34 .
- the computer then performs binary search for a fundamental word that forward-matches an target character string of c characters from the target character (step S 3602 ).
- the computer determines whether the fundamental word is present as a result of the search (step S 3603 ). If no fundamental word is hit by the binary search (step S 3603 : NO), the computer goes to step S 3606 .
- step S 3603 if a fundamental word is hit by the binary search (step S 3603 : YES), the computer determines whether the hit fundamental word perfectly matches the target character string (step S 3604 ). If not (step S 3604 : NO), the computer goes to step S 3606 . On the other hand, if the hit fundamental word perfectly matches (step S 3604 : YES), the computer retains in a storage device, the fundamental word as a longest match candidate (step S 3605 ) and goes to step S 3606 .
- step S 3606 the computer determines whether the binary search has been completed for the target character string (step S 3606 ). For example, the computer determines whether the binary search has been performed for the ending fundamental word. If the binary search is not finished (step S 3606 : NO), the computer goes to step S 3602 to continue until the binary search is completed.
- step S 3606 determines whether a cth character is the tail character of the data file Fi (step S 3607 ). If the cth character is the tail character of the data file Fi (step S 3607 : YES), the computer goes to step S 3610 . On the other hand, if the cth character is not the tail character of the data file Fi (step S 3607 : NO), the computer determines whether c>cmax is satisfied (step S 3608 ). A preset value is denoted by cmax and, as a result, the upper limit number of characters of the target character string is set.
- step S 3608 NO
- the computer increments c (step S 3609 ) and returns to step S 3602 .
- the computer determines whether a longest match candidate is present (step S 3610 ). For example, the computer determines whether at least one longest match candidate is retained in the memory at step S 3605 .
- step S 3610 determines the longest character string of the longest match candidates as the longest matching fundamental word (step S 3611 ). The computer then goes to step S 3402 . On the other hand, if no longest match candidate is present at step S 3610 (step S 3610 : NO), the computer goes to step S 3402 . This leads to the termination of the longest match retrieval process (step S 3401 ). With this longest match retrieval process (step S 3401 ), the longest character string that the perfectly matches the character string can be retrieved as the fundamental word among fundamental words within the fundamental word structure.
- FIG. 37 is a flowchart of a detailed process procedure example of the map assignment number determination process (step S 3002 ) depicted in FIG. 30 .
- the computer sorts in descending order of appearance frequency, the fundamental word appearance frequency count table 3500 indicative of the appearance frequency of each fundamental word and the character appearance frequency count table 3300 indicative of the appearance frequency of each single character obtained by the count process (step S 3001 ) (step S 3701 ).
- the computer determines whether expression (1) is satisfied (step S 3704 ).
- Aw is the total number of appearances of the counted fundamental words.
- step S 3704 If expression (1) is not satisfied (step S 3704 : NO), the computer increments the appearance rank Rw (step S 3705 ), and returns to step S 3703 . Therefore, the appearance rank Rw is continuously lowered until expression (1) is satisfied.
- the map assignment number Nw is the number of fundamental words assigned to the fundamental word appearance map generated in the map generation process (step S 3006 ) and means the number of records (lines) of the fundamental word appearance map.
- the computer determines whether the following expression (2) is satisfied (step S 3709 ).
- step S 3709 NO
- the computer increments the appearance rank Rc (step S 3710 ) and returns to step S 3708 . Therefore, the appearance rank Rc is continuously lowered until expression (2) is satisfied.
- the map assignment number Nc is the number of specified single characters assigned to the specified single character appearance map generated in the map generation process (step S 3006 ) and means the number of records (lines) of the specified single character appearance map. Subsequently, the computer goes to the re-count process (step S 3003 ) and terminates the map assignment number determination process (step S 3002 ).
- the fundamental word appearance map can be generated for the number of the fundamental words corresponding to the target appearance rate Pw in the map generation process (step S 3006 ). Therefore, since it is not necessary to assign all the fundamental words to the map and the assignment is determined in accordance with the target appearance rate Pw, the map size can be optimized.
- the compression code map of specified single characters can be generated for the number of the single characters corresponding to the target appearance rate Pc in the map generation process (step S 3006 ). Therefore, since it is not necessary to assign all the single characters to the map and the assignment is determined in accordance with the target appearance rate Pc, the map size can be optimized.
- FIG. 38 is a flowchart of a detailed process procedure example of the re-count process (step S 3003 ) depicted in FIG. 30 .
- the computer executes the re-count process of the data file Fi (step S 3803 ). Details of the re-count process of the data file Fi (step S 3803 ) will be described with reference to FIG. 39 .
- the computer determines whether the file number i satisfies i>n (where n is the total number of the data files F 1 to Fn) (step S 3804 ).
- step S 3804 If i>n is not satisfied (step S 3804 : NO), the computer increments i (step S 3805 ) and returns to step S 3802 . On the other hand, if i>n is satisfied (step S 3804 : YES), the computer goes to the Huffman tree generation process (step S 3004 ) depicted in FIG. 30 and terminates the re-count process (step S 3003 ). With this re-count process (step S 3003 ), the re-count process of the data file Fi (step S 3803 ) can be executed for each of the data files Fi.
- FIG. 39 is a flowchart of a detailed process procedure example of the re-count process of the data file Fi (step S 3803 ).
- the computer defines the head character of the data file Fi as the target character (step S 3901 ) and determines whether the target character is a specified single character (step S 3902 ). If the target character is a specified single character (step S 3902 : YES), the computer goes to step S 3904 without dividing the character.
- step S 3902 the computer divides the character code of the target character into the upper character code component and the lower character code component (step S 3903 ). The computer goes to step S 3904 .
- step S 3904 the computer adds one to the number of appearances of the same character code component as the upper character code component acquired at step S 3903 in an upper character code component appearance frequency count table (step S 3904 ).
- FIG. 40 is an explanatory diagram of the upper character code component appearance frequency count table.
- An upper character code component appearance frequency count table 4000 is stored in the storage device such as the RAM 2903 or the magnetic disk 2905 and the number of appearances is incremented by one each time a corresponding upper character code component appears.
- the computer adds one to the number of appearances of the same character code component as the lower character code component acquired at step S 3903 in a lower character code component appearance frequency count table (step S 3905 ).
- FIG. 41 is an explanatory diagram of the lower character code component appearance frequency count table.
- An lower character code component appearance frequency count table 4100 is stored in the storage device such as the RAM 2903 or the magnetic disk 2905 and the number of appearances is incremented by one each time a corresponding lower character code component appears.
- the computer executes a bi-gram character string identification process (step S 3906 ).
- a bi-gram character string identification process step S 3906
- a bigram character string starting from the target character is identified. Details of the bi-gram character string identification process (step S 3906 ) will be described with reference to FIG. 42 .
- the computer adds one to the number of appearances of the bi-gram character string identified in the bi-gram character string identification process (step S 3906 ) in a bi-gram character string appearance frequency count table (step S 3907 ).
- FIG. 42 is a flowchart of a detailed process procedure of the bi-gram character string identification process (step S 3906 ) depicted in FIG. 39 .
- the computer determines whether the target character has been divided (step S 4201 ). In other words, the computer determines whether the target character is a character code component. If not (step S 4201 : NO), i.e., in the case of a single character, the computer determines whether the previous character is present (step S 4202 ).
- step S 4202 determines whether the previous character has been divided (step S 4203 ). In other words, the computer determines whether the previous character is a character code component. If not (step S 4203 : NO), i.e., in the case of a single character, the computer determines a character string consisting of the previous single character before the target character and the target character (single character) as a bi-gram character string (step S 4204 ). The computer goes to step S 3907 .
- step S 4203 if the previous character has been divided (step S 4203 : YES), i.e., in the case of a character code component, the character code component, i.e., the previous character is a lower character code component. Therefore, the computer determines a character string consisting of the lower character code component, which is the previous character, and the target character as a bi-gram character string (step S 4205 ). The computer goes to step S 3907 .
- step S 4202 if no previous character is present (step S 4202 : NO), since only the target character is left, the computer goes to step S 3907 without determining a bi-gram character string.
- step S 4201 if the target character has been divided (step S 4201 : YES), i.e., in the case of a character code component, the computer determines whether the character code component is an upper character code component or a lower character code component (step S 4206 ).
- step S 4206 the computer determines whether the previous character has been divided (step S 4207 ). In other words, it is determined whether the previous character is a character code component. If not (step S 4207 : NO), i.e., in the case of a single character, the computer determines a character string consisting of the previous single character before the target character and the upper character code component divided from the target character as a bi-gram character string (step S 4208 ). The computer goes to step S 3907 .
- step S 4207 if the previous character has been divided (step S 4207 : YES), i.e., in the case of a character code component, the character code component, i.e., the previous character is a lower character code component. Therefore, the computer determines a character string consisting of the lower character code component, which is the previous character, and the upper character code component divided from the target character as a bi-gram character string (step S 4209 ). The computer goes to step S 3907 .
- step S 4206 in the case of the lower character code component (step S 4206 : lower), the computer determines a character string consisting of the upper character code component and the lower character code component divided from the target character as a bi-gram character string (step S 4210 ). The computer goes to step S 3907 .
- a bi-gram character string can be identified even if the target character has been divided. Since the bi-gram character strings are identified by shifting characters one-by-one, the map can simultaneously be generated in parallel with the compression code map of fundamental words and the compression code map of specified single characters.
- FIG. 43 is an explanatory diagram of the bi-gram character string appearance frequency count table.
- a bi-gram character string appearance frequency count table 4300 is stored in the storage device such as the RAM 2903 or the magnetic disk 2905 and the number of appearances is incremented by one each time a corresponding bi-gram character string appears.
- the computer determines whether the subsequent character of the target character is present within the data file Fi (step S 3908 ), if the subsequent character is present (step S 3908 : YES), the subsequent character is set as the target character (step S 3909 ) and the computer returns to step S 3902 . On the other hand, if no subsequent character is present (step S 3908 : NO), the computer terminates the re-count process of the data file Fi (step S 3803 ) and goes to step S 3804 .
- the number of appearances of the upper character code components, the lower character code components, and the bi-gram character strings present within the data files Fi can be counted for each of the data files Fi.
- FIG. 44 is a flowchart of a detailed process procedure example of the Huffman tree generation process (step S 3004 ) depicted in FIG. 30 .
- the computer determines the upper limit length N of the compression code length (step S 4401 ).
- the computer then executes a correction process (step S 4402 ).
- the correction process is a process of correcting the occurrence probability and the compression code length for each character unit by using the upper limit length N of the compression code length as described with reference to FIGS. 19 to 21 .
- the computer generates a leaf structure for each of the character units (step S 4403 ).
- the computer executes a branch number specification process (step S 4404 ).
- the branch number specification process step S 4404
- the number of branches per leaf is specified for each compression code length. Details of the branch number specification process (step S 4404 ) will be described with reference to FIG. 45 .
- the computer executes a construction process (step S 4405 ). Since the number of branches of each leaf structure is specified by the branch number specification process (step S 4404 ), the computer first generates pointers to the leaves by the number of branches for each leaf structure. The computer integrates the generated pointers to leaves for the leaf structures to form a root structure. As a result, a 2 N -branch nodeless Huffman tree is generated. The generated 2 N -branch nodeless Huffman tree is stored in a storage device (such as the RAM 2903 or the magnetic disk 2905 ) in the computer. The computer then goes to the division rank determination process (step S 3005 ) of FIG. 30 .
- FIG. 45 is a flowchart of a detailed process procedure example of the branch number specification process (step S 4404 ) depicted in FIG. 44 .
- the computer determines whether j>D is satisfied (step S 4503 ). If j>D is not satisfied (step S 4503 : NO), the computer calculates the count b(CL) of branches per leaf at the compression code length CL (step S 4504 ).
- the computer calculates the total count B(L) of branches at the compression code length CL (step S 4505 ).
- step S 4506 the computer increments j and decrements the compression code length CL (step S 4506 ) and returns to step S 4503 to determine whether j after the increment satisfies j>D.
- FIG. 46 is a flowchart of a detailed process procedure of the construction process (step S 4405 ) depicted in FIG. 44 .
- the computer determines whether an unselected leaf exists at the compression code length CL (step S 4602 ). If an unselected leaf exists (step S 4602 : YES), the computer executes a pointer-to-leaf generation process (step S 4603 ) and returns to step S 4602 .
- pointers to the leaf are generated by the number of branches corresponding to the compression code length CL for each leaf structure. Details of the pointer-to-leaf generation process (step S 4603 ) will be described with reference to FIG. 47 .
- step S 4602 if no unselected leaf exists (step S 4602 : NO), the computer determines whether CL>N is satisfied (step S 4604 ). If CL>N is not satisfied (step S 4604 : NO), the computer increments CL (step S 4605 ) and returns to step S 4602 . On the other hand, if CL>N is satisfied (step S 4604 : YES), this means that the 2 N -branch nodeless Huffman tree is constructed, and the computer goes to the division rank determination process (step S 3005 ) of FIG. 30 .
- FIG. 47 is a flowchart of a detailed process procedure of the pointer-to-leaf generation process (step S 4603 ) depicted in FIG. 46 .
- the computer stores the pointer PL(k) to the selected leaf into the root structure (step S 4705 ). Subsequently, the computer determines whether k>b(CL) is satisfied (step S 4706 ), where b(CL) is the number of branches per leaf of the selected leaf at the compression code length CL. If k>b(CL) is not satisfied (step S 4706 : NO), since pointers to leaf have not been generated for all the branches assigned to the selected leaf, the computer increments k (step S 4707 ).
- the computer increments the current subsequent bit string and concatenates the incremented subsequent bit string to the end of the preceding bit string to newly generate the pointer PL(k) to the selected leaf (step S 4708 ).
- the computer stores the pointer PL(k) to the selected leaf into the root structure (step S 4709 ) and returns to step S 4706 .
- the pointers to leaf are generated by the number of branches per leaf.
- step S 4706 if k>b(CL) is satisfied (step S 4706 : YES), the computer goes to step S 4602 .
- the computer mutually correlates the leaf structures in the 2 N -branch nodeless Huffman tree with the fundamental word structure, the specific character code structure, and the character code component structure by reference to the character unit table depicted in FIG. 21 .
- the leaf structures store pointers to the specific characters, the character code components, and the fundamental words corresponding to the compression codes stored in the corresponding leaves.
- the computer stores a pointer to a leaf that stores a corresponding compression code for each fundamental word of the fundamental word structure.
- the computer stores a pointer to a leaf that stores a corresponding compression code for each specific character of the specific character code structure.
- the computer stores a pointer to a leaf that stores a corresponding compression code for each character code component of the character code component structure.
- FIG. 48 is a flowchart of a detailed process procedure example of the division rank determination process (step S 3005 ) depicted in FIG. 30 .
- the computer sorts the bi-gram character string appearance frequency count table 4300 in descending order of the number of appearances (step S 4801 ).
- the computer refers to the bi-gram character string appearance frequency count table 4300 to count the cumulative appearance number Ar2g up to the appearance rank R2g (step S 4803 ).
- the computer determines whether expression (3) is satisfied (step S 4804 ).
- P2g is the target appearance rate of the bi-gram character strings and A2g is the total number of appearances of the counted bi-gram character strings.
- step S 4804 If expression (3) is not satisfied (step S 4804 : NO), the computer increments the appearance rank R2g (step S 4805 ) and returns to step S 4803 . Therefore, the appearance rank R2g is continuously lowered until expression (3) is satisfied.
- step S 4804 When expression (3) is satisfied (step S 4804 : YES), the computer transitions to the map generation process (step S 3006 ) depicted in FIG. 30 .
- the appearance rank R2g is the division rank.
- the bi-gram character strings after the division rank R2g are divided.
- FIG. 49 is a flowchart of a detailed process procedure example of the map generation process (step S 3006 ) depicted in FIG. 30 .
- the computer executes the map generation process of the data file Fi (step S 4903 ). Details of the map generation process of the data file Fi (step S 4903 ) will be described with reference to FIG. 50 .
- the computer determines whether the file number i satisfies i>n (where n is the total number of the data files F 1 to Fn) (step S 4904 ).
- step S 4904 If i>n is not satisfied (step S 4904 : NO), the computer increments i (step S 4905 ) and returns to step S 4902 . On the other hand, if i>n is satisfied (step S 4904 : YES), the map generation process (step S 3006 ) is terminated. With this map generation process (step S 3006 ), the map generation process of the data file Fi (step S 4903 ) can be executed for each of the data files Fi.
- FIG. 50 is a flowchart of a detailed process procedure of the map generation process of the data file Fi (step S 4903 ) depicted in FIG. 49 .
- the computer defines the head character of the data file Fi as the target character (step S 5001 ) and executes a fundamental word appearance map generation process (step S 5002 ), a specified single character appearance map generation process (step S 5003 ), and a bi-gram character string appearance map generation process (step S 5004 ).
- step S 5002 Details of the fundamental word appearance map generation process (step S 5002 ) will be described with reference to FIG. 51 . Details of the specified single character appearance map generation process (step S 5003 ) will be described with reference to FIG. 52 . Details of the bi-gram character string appearance map generation process (step S 5004 ) will be described with reference to FIG. 54 .
- the computer determines whether the target character is the tail character of the data file Fi (step S 5005 ). If the target character is not the tail character of the data file Fi (step S 5005 : NO), the computer shifts the target character by one character toward the end (step S 5006 ) and returns to step S 5002 . On the other hand, if the target character is the tail character of the data file Fi (step S 5005 : YES), the computer goes to step S 4904 and terminates the map generation process of the data file Fi (step S 4903 ).
- the fundamental word appearance map, the specified single character appearance map, and the bi-gram character string appearance map can simultaneously be generated in parallel while shifting the target character one-by-one.
- FIG. 51 is a flowchart of a detailed process procedure example of the fundamental word appearance map generation process (step S 5002 ) depicted in FIG. 50 .
- the computer executes a longest match retrieval process (step S 5101 ).
- a detailed process procedure of the longest match retrieval process (step S 5101 ) has the same process contents as the longest match retrieval process (step S 3401 ) depicted in FIG. 36 and will not be described.
- the computer determines whether a longest matching fundamental word, i.e., a fundamental word, is present (step S 5102 ). If no longest matching fundamental word is present (step S 5102 : NO), the computer goes to the specified single character appearance map generation process (step S 5003 ). On the other hand, if a longest matching fundamental word is present (step S 5102 : YES), the computer determines whether the fundamental word appearance map is already set in terms of the longest matching fundamental word (step S 5103 ).
- step S 5103 If already set (step S 5103 : YES), the computer goes to step S 5106 .
- step S 5103 the computer accesses the leaf of the longest matching fundamental word in the 2 N -branch nodeless Huffman tree to acquire the compression code thereof (step S 5104 ).
- the computer sets the acquired compression code as a pointer to the fundamental word appearance map for the longest matching fundamental word (step S 5105 ) and goes to step S 5106 .
- step S 5106 the computer turns on the bit of the data file Fi in the fundamental word appearance map for the longest matching fundamental word (step S 5106 ).
- the computer then terminates the fundamental word appearance map generation process (step S 5002 ) and goes to the specified single character appearance map generation process (step S 5003 ).
- the map generation can be performed with the longest matching fundamental word defined as a fundamental word for each target character.
- FIG. 52 is a flowchart of a detailed process procedure example of the specified single character appearance map generation process (step S 5003 ) depicted in FIG. 50 .
- the computer performs binary search of the specified single character structure for the target character (step S 5201 ) and determines whether a coinciding character is found (S 5202 ). If no coinciding single character is present (step S 5202 : NO), the computer executes a character code component appearance map generation process (step S 5203 ) and goes to the bi-gram character string appearance map generation process (step S 5004 ). Details of the character code component appearance map generation process (step S 5203 ) will be described with reference to FIG. 53 .
- step S 5202 if a single character coinciding with the target character is present as a result of the binary search (step S 5202 : YES), the computer accesses the leaf of the binary-searched single character in the 2 N -branch nodeless Huffman tree to acquire the compression code thereof (step S 5204 ). The computer determines whether the specified single character appearance map is already set in terms of the acquired compression code (step S 5205 ). If already set (step S 5205 : YES), the computer goes to step S 5207 .
- step S 5205 the computer sets the acquired compression code as a pointer to the specified single character appearance map for the binary-searched single character (step S 5206 ) and goes to step S 5207 .
- step S 5207 the computer turns on the bit of the data file Fi in the specified single character appearance map for the binary-searched single character (step S 5207 ).
- the computer then terminates the specified single character appearance map generation process (step S 5003 ) and goes to the bi-gram character string appearance map generation process (step S 5004 ).
- the map generation can be performed with the binary-searched target character defined as a specified single character.
- FIG. 53 is a flowchart of a detailed process procedure example of the character code component appearance map generation process (step S 5203 ) depicted in FIG. 52 .
- the computer divides the target character (step S 5301 ) and accesses the leaf of the upper character code component in the 2 N -branch nodeless Huffman tree to acquire the compression code thereof (step S 5302 ).
- the computer determines whether the appearance map of the upper character code component is already set in terms of the acquired compression code (step S 5303 ).
- step S 5303 If already set (step S 5303 : YES), the computer goes to step S 5305 . On the other hand, if not already set (step S 5303 : NO), the computer sets the acquired compression code as a pointer to the appearance map of the upper character code component (step S 5304 ) and goes to step S 5305 . At step S 5305 , the computer turns on the bit of the data file Fi in the appearance map of the upper character code component divided from the target character (step S 5305 ).
- the computer accesses the leaf of the lower character code component in the 2 N -branch nodeless Huffman tree to acquire the compression code thereof (step S 5306 ).
- the computer determines whether the appearance map of the lower character code component is already set in terms of the acquired compression code (step S 5307 ). If already set (step S 5307 : YES), the computer goes to step S 5309 .
- step S 5307 the computer sets the acquired compression code as a pointer to the appearance map of the lower character code component (step S 5308 ) and goes to step S 5309 .
- step S 5309 the computer turns on the bit of the data file Fi in the appearance map of the lower character code component separated from the target character (step S 5309 ).
- the computer then terminates the character code component appearance map generation process (step S 5203 ) and goes to the bi-gram character string appearance map generation process (step S 5004 ).
- this character code component appearance map generation process for single characters ranked lower than the rank corresponding to the target appearance rate Pc, a multiplicity of OFF bits appears due to lower appearance frequency.
- the map size of the compression code map of the specified single characters can be optimized.
- the single characters ranked lower than the rank corresponding to the target appearance rate Pc are set in maps having fixed map sizes such as the compression code map of the upper character code components and the compression code map of the lower character code components. Therefore, the map sizes can be prevented from increasing and memory can be saved regardless of an appearance rate set as the target appearance rate Pc.
- FIG. 54 is a flowchart of a detailed process procedure example of the bi-gram character string appearance map generation process (step S 5004 ) depicted in FIG. 50 .
- the computer executes a bi-gram character string identification process (step S 5401 ).
- a detailed process procedure of the bi-gram character string identification process (step S 5401 ) has the same process contents as the bi-gram character string identification process (step S 3906 ) depicted in FIG. 42 and will not be described.
- the computer determines whether a bi-gram character string is identified by the bi-gram character string identification process (step S 5401 ) (step S 5402 ). If no bi-gram character string is identified (step S 5402 : NO), the computer goes to step S 5005 of FIG. 50 .
- step S 5402 determines whether the appearance rank of the identified bi-gram character string is higher than the division rank R2g determined by the division rank determination process (step S 3005 ), by reference to the sorted bi-gram character string appearance frequency count table 4300 (step S 5403 ).
- step S 5404 the computer executes a bi-gram character string appearance map (non-divided) generation process (step S 5404 ) and goes to step S 5005 .
- the bi-gram character string appearance map (non-divided) generation process step S 5404 ) the division as depicted in FIGS. 1B and 1C is not performed.
- step S 5405 the computer executes a bi-gram character string appearance map (divided) generation process (step S 5405 ) and goes to step S 5005 .
- the bi-gram character string appearance map (divided) generation process step S 5405 ) the division as depicted in FIGS. 1B and 1C is performed.
- FIG. 55 is a flowchart of a detailed process procedure example of the bi-gram character string appearance map (non-divided) generation process (step S 5404 ).
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for a first gram (specified single character or character code component) of the bi-gram character string identified by the bi-gram character string identification process (step S 5401 ) depicted in FIG. 54 to acquire a compression code (step S 5501 ).
- the computer also accesses a leaf of the 2 N -branch nodeless Huffman tree for a second gram (specified single character or character code component) to acquire a compression code (step S 5502 ).
- the computer concatenates the compression code of the first gram and the compression code of the second gram (step S 5503 ).
- the computer determines whether an appearance map having the concatenated compression code as a pointer is already set (step S 5504 ). If already set (step S 5504 : YES), the computer goes to step S 5506 .
- step S 5504 the computer sets the concatenated compression code as the pointer to the appearance map of the identified bi-gram character string (step S 5505 ).
- step S 5506 the computer turns on the bit of the data file Fi in the appearance map for the identified bi-gram character string (step S 5506 ).
- the computer then terminates the bi-gram character string appearance map (non-divided) generation process (step S 5404 ) and proceeds to step S 5005 .
- the bi-gram character string appearance map (non-divided) generation process step S 5404
- the appearance map is generated without dividing the characters. Therefore, no retrieval noise is superimposed on the appearance map of divided bi-grams.
- step S 5405 A detailed process procedure example of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 will be described.
- FIG. 56 is a flowchart of a detailed process procedure example (part 1 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 56 depicts an example of dividing the first gram (specified single character or character code component) and not dividing the second gram (specified single character or character code component) of a bi-gram character string after the division rank R2g.
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for the first gram (specified single character or character code component) of the bi-gram character string identified by the bi-gram character string identification process (step S 5401 ) depicted in FIG. 54 to acquire a compression code (step S 5601 ).
- the computer divides the acquired compression code of the first gram into an upper code and a lower code (step S 5602 ).
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for the second gram (specified single character or character code component) to acquire a compression code (step S 5603 ).
- the computer concatenates the upper code (first-gram upper code) of the compression code of the first gram and the compression code of the second gram (step S 5604 ).
- the computer determines whether an appearance map having the concatenated code of step S 5604 as a pointer is already set (step S 5605 ). If already set (step S 5605 : YES), the computer goes to step S 5607 .
- step S 5605 the computer sets the concatenated code as the pointer (step S 5606 ).
- step S 5507 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5607 ).
- the computer concatenates the lower code (first-gram lower code) of the compression code of the first gram and the compression code of the second gram (step S 5608 ).
- the computer determines whether an appearance map having the concatenated code of step S 5608 as a pointer is already set (step S 5609 ). If already set (step S 5609 : YES), the computer goes to step S 5611 .
- step S 5609 the computer sets the concatenated code as the pointer (step S 5610 ).
- step S 5611 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5611 ).
- the computer then terminates the bi-gram character string appearance map (divided) generation process (step S 5405 ) and proceeds to step S 5005 .
- step S 5405 since the compression code map of bi-gram character strings after the rank corresponding to the target appearance rate Pc is divided into two parts, no retrieval noise is superimposed and the reduction of memory size can be achieved.
- FIG. 57 is a flowchart of a detailed process procedure example (part 2 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- FIG. 57 depicts an example of dividing the second gram (specified single character or character code component) and not dividing the first gram (specified single character or character code component) of a bi-gram character string after the division rank R2g.
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for the first gram (specified single character or character code component) of the bi-gram character string identified by the bi-gram character string identification process (step S 5401 ) depicted in FIG. 54 to acquire a compression code (step S 5701 ).
- the computer also accesses a leaf of the 2 N -branch nodeless Huffman tree for the second gram (specified single character or character code component) to acquire a compression code (step S 5702 ).
- the computer divides the acquired compression code of the second gram into an upper code and a lower code (step S 5703 ).
- the computer concatenates the compression code of the first gram and the upper code (second-gram upper code) of the compression code of the second gram (step S 5704 ).
- the computer determines whether an appearance map having the concatenated code of step S 5704 as a pointer is already set (step S 5705 ). If already set (step S 5705 : YES), the computer goes to step S 5707 .
- step S 5705 the computer sets the concatenated code as the pointer (step S 5706 ).
- step S 5707 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5707 ).
- the computer concatenates the compression code of the first gram and the lower code (second-gram lower code) of the compression code of the second gram (step S 5708 ).
- the computer determines whether an appearance map having the concatenated code of step S 5708 as a pointer is already set (step S 5709 ). If already set (step S 5709 : YES), the computer goes to step S 5711 .
- step S 5709 the computer sets the concatenated code as the pointer (step S 5710 ).
- step S 5711 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5711 ).
- the computer then terminates the bi-gram character string appearance map (divided) generation process (step S 5405 ) and proceeds to step S 5005 .
- step S 5405 since the compression code map of bi-gram character strings after the rank corresponding to the target appearance rate Pc is divided into two parts, no retrieval noise is superimposed and the reduction of memory size can be achieved.
- FIGS. 58 and 59 are flowcharts of a detailed process procedure example (part 3 ) of the bi-gram character string appearance map (divided) generation process (step S 5405 ) depicted in FIG. 54 .
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for the first gram (specified single character or character code component) of the bi-gram character string identified by the bi-gram character string identification process (step S 5401 ) depicted in FIG. 54 to acquire a compression code (step S 5801 ).
- the computer divides the acquired compression code of the first gram into an upper code and a lower code (step S 5802 ).
- the computer accesses a leaf of the 2 N -branch nodeless Huffman tree for the second gram (specified single character or character code component) to acquire a compression code (step S 5803 ).
- the computer divides the acquired compression code of the second gram into an upper code and a lower code (step S 5804 ).
- the computer then concatenates the upper code (first-gram upper code) of the compression code of the first gram and the upper code (second-gram upper code) of the compression code of the second gram (step S 5805 ).
- the computer determines whether an appearance map having the concatenated code of step S 5805 as a pointer is already set (step S 5806 ). If already set (step S 5806 : YES), the computer goes to step S 5808 .
- step S 5806 the computer sets the concatenated code as the pointer (step S 5807 ).
- step S 5808 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5808 ). The computer goes to step S 5901 of FIG. 59 .
- step S 5901 the computer concatenates the first-gram upper code and the lower code (second-gram lower code) of the compression code of the second gram (step S 5901 ).
- the computer determines whether an appearance map having the concatenated code of step S 5901 as a pointer is already set (step S 5902 ). If already set (step S 5902 : YES), the computer goes to step S 5904 .
- step S 5902 determines whether the pointer has been set. If not already set (step S 5902 : NO), the computer sets the concatenated code as the pointer (step S 5903 ). At step S 5904 , the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5904 ).
- the computer concatenates the lower code (first-gram lower code) of the compression code of the first gram and the second-gram upper code (step S 5905 ).
- the computer determines whether an appearance map having the concatenated code of step S 5905 as a pointer is already set (step S 5906 ). If already set (step S 5906 : YES), the computer goes to step S 5908 .
- step S 5906 the computer sets the concatenated code as the pointer (step S 5907 ).
- step S 5908 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5908 ).
- step S 5909 the computer concatenates the first-gram lower code and the second-gram lower code.
- the computer determines whether an appearance map having the concatenated code of step S 5909 as a pointer is already set (step S 5910 ). If already set (step S 5910 : YES), the computer goes to step S 5912 .
- step S 5910 the computer sets the concatenated code as the pointer (step S 5911 ).
- step S 5912 the computer turns on the bit of the data file Fi in the appearance map for the concatenated code (step S 5912 ).
- the computer then terminates the bi-gram character string appearance map (divided) generation process (step S 5405 ) and proceeds to step S 5005 .
- step S 5405 since the compression code map of bi-gram character strings after the rank corresponding to the target appearance rate Pc is divided into four parts, no retrieval noise is superimposed and the reduction of memory size can be achieved.
- FIG. 60 is an explanatory diagram of a specific example of the compression process using a 2 N -branch nodeless Huffman tree.
- the computer acquires a compression target character code of a first character from the data file group Fs and retains a position on the data file Fi.
- the computer performs binary tree search of the fundamental word structure. Since a fundamental word is a character code string of two or more characters, if the compression target character code of the first character is hit, a character code of a second character is acquired as the compression target character code.
- the character code of the second character is searched from the position where the compression target character code of the first character is hit.
- the binary tree search is performed for a third or later character until a compression target character code that does not coincide appears. If a coinciding fundamental word ra (“a” is a number of a leaf) is retrieved, a pointer to the leaf La correlated in the fundamental word structure is used to access a structure of the leaf La.
- the computer searches for the compression code of the fundamental word ra stored in the accessed structure of the leaf La and stores the compression code into a compression buffer 6000 .
- the binary tree search of the fundamental word structure is terminated (proceeds to End Of Transmission (EOT)).
- EOT End Of Transmission
- the computer sets the compression target character code of the first character into a register again and performs the binary tree search of the specific single character structure.
- the computer uses a pointer to the leaf Lb to access a structure of the leaf Lb.
- the computer searches for the compression code of the character code eb stored in the accessed structure of the leaf Lb and stores the compression code into the compression buffer 6000 .
- the computer divides the compression target character code into upper eight bits and lower eight bits. For the character code component of the upper eight bits, the computer performs binary tree search of the character code component structure. If a coinciding character code component Dc 1 (“c 1 ” is a number of a leaf) is retrieved, the computer uses a pointer to the leaf Lc 1 to access a structure of the leaf Lc 1 . The computer searches for the compression code of the character code component Dc 1 stored in the accessed structure of the leaf Lc 1 and stores the compression code into the compression buffer 6000 .
- the computer continues the binary tree search of the character code component structure. If a coinciding character code component Dc 2 (“c 2 ” is a number of a leaf) is retrieved, the computer uses a pointer to the leaf Lc 2 to access a structure of the leaf Lc 2 . The computer searches for the compression code of the character code component Dc 2 stored in the accessed structure of the leaf Lc 2 and stores the compression code into the compression buffer 6000 . As a result, the data file Fi is compressed.
- FIG. 61 is a flowchart of the compression process procedure example of the data file group Fs using the 2 N -branch nodeless Huffman tree automatically executed by the computer.
- the computer executes the compression process (step S 6103 ) and increments the file number: p (step S 6104 ). Details of the compression process (step S 6103 ) will be described with reference to FIG. 62 .
- step S 6105 The computer determines whether p>n is satisfied (step S 6105 ), where n is the total number of the data files Fs. If p>n is not satisfied (step S 6105 : NO), the computer returns to step S 6102 . On the other hand, if p>n is satisfied (step S 6105 : YES), the computer terminates the compression process of the data file group Fs.
- FIG. 62 is a flowchart (part 1 ) of a detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- the computer determines whether a compression target character code is present in the data file group Fs (step S 6201 ). If present (step S 6201 : YES), the computer acquires and sets the compression target character code in a register (step S 6202 ). The computer determines whether the compression target character code is the head compression target character code (step S 6203 ).
- the head compression target character code is uncompressed character code of the first character. If the compression target character code is the head compression target character code (step S 6203 : YES), the computer acquires a pointer of the position (head position) of the compression target character code on the data file group Fs (step S 6204 ) and goes to step S 6205 . On the other hand, if the compression target character code is not the head compression target character code (step S 6203 : NO), the computer goes to step S 6205 without acquiring the head position.
- the computer performs the binary tree search of the fundamental word structure (step S 6205 ). If code coinciding with the compression target character code is present (step S 6206 : YES), the computer determines whether successive coinciding character code strings correspond to (the character code string of) a fundamental word (step S 6207 ). If not (step S 6207 : NO), the computer returns to step S 6202 and acquires the subsequent character code as the compression target character code. In this case, since the subsequent character code is not the head code, the head position is not acquired.
- step S 6207 if successive coinciding character code strings correspond to a fundamental word (step S 6207 : YES), the computer uses a pointer to the leaf L# of the corresponding fundamental word to access a structure of the leaf L# (step S 6208 ). The computer extracts the compression code of the fundamental word stored in the structure of the leaf L# that is pointed to (step S 6209 ).
- step S 6210 the computer stores the extracted compression code into the compression buffer 6000 (step S 6210 ) and returns to step S 6201 .
- This loop makes a flow of the compression process of fundamental words.
- step S 6201 if no compression target character code is present (step S 6201 : NO), the computer performs file output to store a compressed file fp obtained by compressing the data file Fp, from the compression buffer 6000 (step S 6211 ). The computer goes to step S 6104 .
- step S 6206 if coinciding code is not present at step S 6206 (step S 6206 : NO), the computer enters a loop of the compression process of 16-bit character codes.
- FIG. 63 is a flowchart (part 2 ) of the detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- the computer refers to the pointer of the head position acquired at step S 6204 to acquire and set the compression target character code from the data file group Fs into the register (step S 6301 ).
- the computer performs the binary tree search of the specified single character code structure for the compression target character code (step S 6302 ). If coinciding code is present (step S 6303 : YES), the computer uses a pointer to the leaf L# of the corresponding character to access the structure of the leaf L# (step S 6304 ). The computer extracts the compression code of the compression target character code stored in the structure of the leaf L# (step S 6305 ) pointed to.
- step S 6306 the computer stores the retrieved compression code into the compression buffer 6000 (step S 6306 ) and returns to step S 6201 .
- This loop makes a flow of the compression process of 16-bit character codes.
- step S 6303 if no coinciding character code is present at step S 6303 (step S 6303 : NO), the computer enters a loop of the compression process of character code components.
- FIG. 64 is a flowchart (part 3 ) of the detailed process procedure of the compression process (step S 6103 ) depicted in FIG. 61 .
- the computer divides the compression target character code into upper eight bits and lower eight bits (step S 6401 ) and extracts the character code component of the upper eight bits (step S 6402 ).
- the computer performs a binary tree search of the character code component structure (step S 6403 ).
- the computer uses a pointer to the leaf L# of the retrieved character code component to access the structure of the leaf L# (step S 6404 ).
- the computer extracts the compression code of the character code component stored in the structure of the leaf L# pointed to (step S 6405 ). Subsequently, the computer stores the retrieved compression code into the compression buffer 6000 (step S 6406 ).
- the computer determines whether the lower eight bits have been retrieved (step S 6407 ) and if not (step S 6407 : NO), the computer extracts the character code component of the lower eight bits (step S 6408 ) and executes steps S 6403 to S 6406 . On the other hand, if the lower eight bits have been retrieved (step S 6407 : YES), the computer returns to step S 6201 and enters the loop of the compression process of fundamental words.
- the structure of the leaf L# storing the compression target character code can be immediately identified from the fundamental word structure, the specified single character code structure, and the character code component structure. Therefore, it is not necessary to search the leaves of the 2 N -branch nodeless Huffman tree and the compression process can be accelerated.
- By dividing a lower-order character code into an upper bit code and a lower bit code nonspecified single characters can be compressed into compression codes of 256 types of character code components. Therefore, the compression rate can be improved.
- FIG. 65 is an explanatory diagram of the flow (A) of the retrieval process using the compression code maps.
- a compression code of each character unit “ ”, “ ”, and “ ” making up the retrieval character string is obtained by accessing the 2 N -branch nodeless Huffman tree through a process identical to the compression process.
- the computer For each of the character units “ ”, “ ”, and “ ” making up the retrieval character string, the computer then acquires the appearance map of “ ”, the appearance map of “ ”, and the appearance map of “ ”, from the compression code map of specified single characters.
- FIG. 66 is an explanatory diagram of the flow (B) of the retrieval process using the compression code maps.
- (A) depicted in FIG. 65 it is determined whether bi-gram strings “ ” and “ ” making up the retrieval character string “ ” are ranked below the division rank. Since “ ” is ranked higher than the division rank, the computer acquires from the compression code map of bi-gram character strings, the appearance map of “ ” pointed to by the compression code string obtained by concatenating the compression code of “ ” and the compression code of “ ”.
- the computer uses the concatenated code of the upper code (first-gram upper code) of the compression code of the first gram “ ” and the upper code (second-gram upper code) of the compression code of the second gram “ ” for the pointing.
- the computer acquires from the compression code map of the first-gram upper codes and the second-gram upper codes, the appearance map of a bi-gram character string having the upper code “0x8B” of the first gram “ ” and the upper code “0x81” of the second gram “ ”.
- the computer uses the concatenated code of the upper code (first-gram upper code) of the compression code of the first gram “ ” and the lower code (second-gram lower code) of the compression code of the second gram “ ” for the pointing.
- the computer acquires from the compression code map of the first-gram upper codes and the second-gram lower codes, the appearance map of a bi-gram character string having the upper code “0x8B” of the first gram “ ” and the lower code “0x30” of the second gram “ ”.
- the computer uses the concatenated code of the lower code (first-gram lower code) of the compression code of the first gram “ ” and the upper code (second-gram upper code) of the compression code of the second gram “ ” for the pointing.
- the computer acquires from the compression code map of the first-gram lower codes and the second-gram upper codes, the appearance map of a bi-gram character string having the lower code “0x30” of the first gram “ ” and the upper code “0x81” of the second gram “ ”.
- the computer uses the concatenated code of the lower code (first-gram lower code) of the compression code of the first gram “ ” and the lower code (second-gram lower code) of the compression code of the second gram “ ” for the pointing.
- the computer acquires from the compression code map of the first-gram lower codes and the second-gram lower codes, the appearance map of a bi-gram character string having the lower code “0x30” of the first gram “ ” and the lower code “0x30” of the second gram “ ”.
- FIG. 67 is an explanatory diagram of the flows (C) and (D) of the retrieval process using the compression code maps.
- the computer can narrow down from the compressed file group fs, compressed files fi potentially having “ ”, by performing the AND operation of the appearance maps acquired at (B) depicted in FIG. 66 .
- the computer uses a compression character string obtained by compressing the retrieval character string with the 2 N -branch nodeless Huffman tree to collate the compressed files fi narrowed down at (C) still in the compressed state with the compression character string and expand the compressed files fi while performing replacement (e.g., sandwiching between tags for color display).
- the compression code string of the retrieval character string “ ” is used to expand compressed files fi while performing the collation.
- the compression code of the specified single character “ ” is “1100010011” (10 bits); the compression code of the specified single character “ ” is “0100010010” (8 bits); and the compression code of the specified single character “ ” is “1010010100” (10 bits).
- the compression code string is set in the register and a compression code is extracted through a mask pattern.
- the extracted compression code is searched for from the root of the 2 N -branch nodeless Huffman tree by one pass (access through one branch).
- a character code stored in the accessed structure of the leaf L# is read and stored in the expansion buffer.
- the mask position of the mask pattern is offset.
- the initial value of the mask pattern is set to “0xFFF00000”.
- This mask pattern is a bit string whose head 12 bits are “1” and the subsequent 20 bits are “0”.
- FIGS. 68 to 70 are explanatory diagrams of collation/expansion according to (D) depicted in FIG. 67 .
- FIG. 68 depicts an example (A) of collating/expanding the specified single character “ ”.
- the CPU calculates a bit address abi, a byte offset byos, and a bit offset bios.
- a block in the memory indicates a one-byte bit string and a numerical character inside indicates a byte position serving as a byte boundary.
- the mask pattern is “0xFFF00000”. Therefore, an AND result is acquired from the logical product (AND) operation of the compression code string set in the register and the mask pattern “0xFFF00000”.
- the pointer (branch number) to the leaf L# coinciding with this target bit string is searched.
- the pointer to the leaf L# coinciding with this target bit string is searched.
- the corresponding pointer to the leaf L 691 is read to access the structure of the leaf L 691 .
- FIG. 69 depicts an example (B) of collating/expanding the specified single character “ ”
- the mask pattern is “0x3FFC0000”. Therefore, an AND result is acquired from the logical product (AND) operation of the compression code string set in the register and the mask pattern “0x3FFC0000”.
- the pointer (branch number) to the leaf L# coinciding with this bit string is searched.
- the target bit string “0100010010” coincides with one of the pointers to a leaf L 24
- the corresponding pointer to the leaf L 24 is read to access the structure of the leaf L 24 .
- the mask pattern is “0x3FFC0000”. Therefore, an AND result is acquired from the logical product (AND) operation of the compression code string set in the register and the mask pattern “0x3FFC0000”.
- the pointer (branch number) to the leaf L# coinciding with this bit string is searched.
- the target bit string “1010010000” coincides with one of the pointers to a leaf L 167
- the corresponding pointer to the leaf L 167 is read to access the structure of the leaf L 167 .
- FIG. 71 is a flowchart of a retrieval process procedure of this embodiment.
- the computer waits for input of a retrieval character string (step S 7101 : NO) and, when a retrieval character string is input (step S 7101 : YES), the computer executes a file narrowing-down process (step S 7102 ) and an expansion process (step S 7103 ).
- the file narrowing-down process step S 7102
- the compressed files fi of the data files Fi having the character unit making up the retrieval character string are narrowed down from the compressed file group fs. Details of the file narrowing-down process (step S 7102 ) will be described with reference to FIGS. 72 and 73 .
- step S 7103 the compression code string to be expanded is collated with the compression character string of the retrieval character string in the course of expanding the compressed files fi narrowed down by the file narrowing-down process (step S 7102 ). Details of the expansion process (step S 7103 ) will be described with reference to FIGS. 74 and 75 .
- FIG. 72 is a flowchart (part 1 ) of a detailed process procedure of the file narrowing-down process (step S 7102 ) depicted in FIG. 71 .
- the computer sets the retrieval character string as the target character string (step S 7201 ) and executes the longest match retrieval process (step S 7202 ).
- the longest match retrieval process (step S 7202 ) is the same process as the longest match retrieval process (step S 3401 ) depicted in FIG. 36 and will not be described.
- the computer performs binary search of the fundamental word structure for the longest match retrieval result obtained by the longest match retrieval process (step S 7202 ) (step S 7203 ). If the longest match retrieval result is retrieved from the fundamental word structure (step S 7203 : YES), for the fundamental word that is the target character string, the computer acquires the appearance map of the fundamental word from the appearance map group of fundamental words (step S 7204 ).
- the computer determines whether the target character string has a subsequent character string (step S 7205 ). If a subsequent character string is present (step S 7205 : YES), the computer sets the subsequent character string as the target character string (step S 7206 ) and returns to the longest match retrieval process (step S 7202 ). If no subsequent character string is present (step S 7205 : NO), the data files are narrowed down through the AND operation of the acquired appearance map group (step S 7207 ). The computer then terminates the file narrowing-down process (step S 7102 ) and transitions to the expansion process (step S 7103 ).
- step S 7203 if the longest match retrieval result is not retrieved from the fundamental word structure (step S 7203 : NO), the computer proceeds to step S 7301 of FIG. 73 . For example, if the longest match retrieval result is not registered in the fundamental word structure or if no longest match candidate is present consequent to the longest match retrieval (step S 7203 : NO), the computer proceeds to step S 7301 depicted in FIG. 73 .
- FIG. 73 is a flowchart (part 2 ) of the detailed process procedure of the file narrowing-down process (step S 7102 ) depicted in FIG. 71 .
- FIG. 73 depicts a process of acquiring an appearance map for each character making up the target character string.
- the computer sets the head character of the target character string as the target character (step S 7301 ).
- the computer performs a binary search of the specified single character structure for the target character (step S 7302 ). If the target character is retrieved (step S 7303 : YES), the computer acquires the appearance map of the target character from the compression code map of specified single characters (step S 7304 ). The computer returns to step S 7205 of FIG. 72 .
- step S 7303 the computer divides the target character into an upper 8 bits and a lower 8 bits (step S 7305 ).
- the computer acquires from the compression code map of upper character code components, the appearance map of the upper character code component obtained by the division at step S 7305 (step S 7306 ).
- step S 7309 the computer executes a bi-gram character string identification process (step S 7309 ).
- the bi-gram character string identification process (step S 7309 ) is the same process as the bi-gram character string identification process (step S 3906 ) depicted in FIG. 42 and will not be described.
- the computer also acquires from the compression code map of lower character code components, the appearance map of the lower character code component obtained by the division at step S 7305 (step S 7307 ).
- the computer accesses the leaves of the 2 N -branch nodeless Huffman tree to turn on collation flags (step S 7308 ).
- the computer executes the bi-gram character string identification process (step S 7309 ).
- step S 7310 NO
- the computer returns to step S 7205 of FIG. 72 .
- step S 7310 YES
- the computer acquires an appearance map of a non-divided bi-gram character string (step S 7312 ). For example, the computer accesses the 2 N -branch nodeless Huffman tree to acquire and concatenate the compression code of the first gram and the compression code of the second gram, and acquires the appearance map identified by the concatenated compression code from the compression code map of bi-gram character strings.
- the computer determines whether the rank is equal to or lower than the division rank R2g (step S 7311 : NO). If the rank is equal to or lower than the division rank R2g (step S 7311 : NO), the computer generates a concatenated code group (step S 7313 ). For example, for example, the computer accesses the 2 N -branch nodeless Huffman tree to acquire the compression code of the first gram and the compression code of the second gram. In the case of four-part division, the computer divides the compression code of the first gram and the compression code of the second gram, respectively, into an upper code and a lower code.
- the computer generates a first concatenated code obtained by concatenating the first-gram upper code and the second-gram upper code, a second concatenated code obtained by concatenating the first-gram upper code and the second-gram lower code, a third concatenated code obtained by concatenating the first-gram lower code and the second-gram upper code, and a fourth concatenated code obtained by concatenating the first-gram lower code and the second-gram lower code.
- the computer acquires the appearance map group of the bi-gram character strings (step S 7314 ). For example, the computer acquires from the first component compression code map, the appearance map identified by the first concatenated code generated at step S 7313 . The computer acquires from the second component compression code map, the appearance map identified by the second concatenated code generated at step S 7313 . The computer acquires from the third divided compression code map, the appearance map identified by the third concatenated code generated at step S 7313 . The computer acquires from the fourth divided compression code map, the appearance map identified by the fourth concatenated code generated at step S 7313 . The computer returns to step S 7205 of FIG. 72 .
- the appearance map group for the target character and the appearance map group for the bi-gram character strings can be acquired. Therefore, the compressed files fi can be narrowed down through the AND operation at step S 7207 depicted in FIG. 72 .
- FIG. 74 is a flowchart (part 1 ) of a detailed example of the expansion process (step S 7103 ) using the 2 N -branch nodeless Huffman tree depicted in FIG. 71 .
- the computer sets a compression code string from the position of the byte offset byos into a register r 1 (step S 7404 ).
- the computer shifts toward the tail and by the bit offset bios, a mask pattern set in a register r 2 (step S 7405 ) and performs an AND operation with the compression code string set in the register r 1 (step S 7406 ).
- the computer subsequently calculates the register shift number rs (step S 7407 ) and after the AND operation, shifts the register r 2 toward the tail, by the register shift number rs (step S 7408 ).
- FIG. 75 is a flowchart (part 2 ) of the detailed example of the expansion process (step S 7103 ) using the 2 N -branch nodeless Huffman tree depicted in FIG. 71 .
- the computer extracts the tail N bits as a target bit string, from the register r 2 after the shift (step S 7501 ).
- the computer identifies the pointer to the leaf L# from the root structure of the 2 N -branch nodeless Huffman tree (step S 7502 ) and accesses the structure of the leaf L# pointed to by one pass (S 7503 ).
- the computer determines whether the collation flag of the accessed structure of the leaf L# is ON (step S 7504 ).
- step S 7504 If the collation flag is ON (step S 7504 : YES), a replacement character of the character unit in the accessed structure of the leaf L# is written into the expansion buffer (step S 7505 ) and the computer proceeds to step S 7507 .
- step S 7504 if the collation flag is OFF (step S 7504 : NO), the character unit (expanded character) in the accessed structure of the leaf L# is written into the expansion buffer (step S 7506 ) and the computer proceeds to step S 7507 .
- step S 7507 the computer extracts the compression code length leg from the accessed structure of the leaf L# (step S 7507 ) and updates the bit address abi (step S 7508 ).
- the computer determines whether a compression code string is present in the memory, for example, whether a compression code string not subjected to the mask process using the mask pattern is present (step S 7509 ). For example, this is determined based on whether a byte position corresponding to the byte offset byos is present. If the compression code string is present (step S 7509 : YES), the computer returns to step S 7402 of FIG. 74 . On the other hand, if no compression code string is present (step S 7509 : NO), the expansion process (step S 7103 ) is terminated.
- step S 7103 the collation/expansion can be performed while the compressed state is maintained and the expansion rate can be accelerated.
- the two-part or four-part divided appearance map group can be generated for a bi-gram character string lower than or equal to the division rank to achieve a reduction in the size of the compression code map and a reduction of retrieval noise.
- bit strings indicative of the presence of bi-gram character strings in the data file group Fs can be specified by concatenated codes to achieve the efficient size reduction of the compression code map. If the data file group Fs is not compressed, the character codes can directly be utilized instead of the compression codes to make the division process of the compression code map more efficient.
- the division rank is determined depending on the target appearance rate, the extent of the number of appearances can easily be determined in terms of the appearance maps of the bi-gram character strings excluded from the division, within the total number of appearances of bi-gram character strings.
- the target appearance rate can operationally be input from an input apparatus to determine the division rank depending on a request from a user. Therefore, the determination can freely be made depending on the intent of the user such as “retrieval noise is acceptable if the target appearance rate is set to this extent”, and the degree of freedom of the division of the compression code map can be increased.
- the division rank may be determined simply depending on ranking in descending order of the number of appearances of bi-gram character strings, instead of the target appearance rate. In this case, the calculation of the division rank using the target appearance rate can be omitted and the division process of the compression code map can be accelerated.
- a series of the processes depicted in FIG. 30 can be executed for the data file group Fs after the addition to optimize the compression code map. For example, if the data file Fi is e-mail, since e-mail is stored each time e-mail is transmitted or received, the series of the processes depicted in FIG. 30 is executed for a stored e-mail group to which e-mail is newly added.
- the 2 N -branch nodeless Huffman tree is employed in this embodiment. Since the employment of the 2 N -branch nodeless Huffman tree enables one-pass access to a compression code of each character unit, the generation of concatenated codes can be accelerated. Since the 2 N -branch nodeless Huffman tree is used, even if an uncompressed retrieval character string is input, the retrieval character string can be compressed to a compression code string by one pass.
- an appearance map in a divided compression code map can directly be identified. Therefore, the corresponding appearance map can rapidly be extracted, and the compressed files fi including a compression code string of a bi-gram character string can be rapidly and accurately narrowed down from the compressed file group fs obtained by compressing the data file group Fs.
- the retrieval process can be realized at high speed with memory usage suppressed to the minimum.
- the generation and the retrieval may be performed by separate computers.
- a computer generating the 2 N -branch nodeless Huffman tree and the compression code maps transmits the data file group Fs (or compressed file group fs), the 2 N -branch nodeless Huffman tree, and the compression code maps to another computer.
- Another computer executes a retrieval process by using the transmitted data file group Fs (or compressed file group fs), the 2 N -branch nodeless Huffman tree, and the compression code maps.
- Each process described in the flowcharts may be implemented by executing a prepared program on a computer such as a personal computer and a workstation.
- the program is stored on a computer-readable medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, read out from the recording medium, and executed by the computer.
- the program may be distributed through a network such as the Internet.
- the computer-readable medium does not include a transitory medium such as a propagation signal.
- retrieval noise generated by reducing map size can be suppressed.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/243,173 US9720976B2 (en) | 2011-03-31 | 2014-04-02 | Extracting method, computer product, extracting system, information generating method, and information contents |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011081105A JP5605288B2 (ja) | 2011-03-31 | 2011-03-31 | 出現マップ生成方法、ファイル抽出方法、出現マップ生成プログラム、ファイル抽出プログラム、出現マップ生成装置、およびファイル抽出装置 |
JP2011-081105 | 2011-03-31 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/243,173 Continuation US9720976B2 (en) | 2011-03-31 | 2014-04-02 | Extracting method, computer product, extracting system, information generating method, and information contents |
Publications (1)
Publication Number | Publication Date |
---|---|
US20120254190A1 true US20120254190A1 (en) | 2012-10-04 |
Family
ID=46928651
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/423,319 Abandoned US20120254190A1 (en) | 2011-03-31 | 2012-03-19 | Extracting method, computer product, extracting system, information generating method, and information contents |
US14/243,173 Active 2033-07-19 US9720976B2 (en) | 2011-03-31 | 2014-04-02 | Extracting method, computer product, extracting system, information generating method, and information contents |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/243,173 Active 2033-07-19 US9720976B2 (en) | 2011-03-31 | 2014-04-02 | Extracting method, computer product, extracting system, information generating method, and information contents |
Country Status (2)
Country | Link |
---|---|
US (2) | US20120254190A1 (ja) |
JP (1) | JP5605288B2 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104765737A (zh) * | 2014-01-03 | 2015-07-08 | 易搜比控股公司 | 撷取超文字标签语言文件内容的方法 |
US9450601B1 (en) * | 2015-04-02 | 2016-09-20 | Microsoft Technology Licensing, Llc | Continuous rounding of differing bit lengths |
CN109558731A (zh) * | 2017-09-26 | 2019-04-02 | 腾讯科技(深圳)有限公司 | 特征码处理方法、装置及存储介质 |
US10324963B2 (en) | 2015-10-09 | 2019-06-18 | Fujitsu Limited | Index creating device, index creating method, search device, search method, and computer-readable recording medium |
US10747725B2 (en) | 2015-07-14 | 2020-08-18 | Fujitsu Limited | Compressing method, compressing apparatus, and computer-readable recording medium |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10387492B2 (en) * | 2014-05-12 | 2019-08-20 | Rakuten, Inc. | Information processing system, information processing method, and program |
US20160259857A1 (en) * | 2015-03-06 | 2016-09-08 | Microsoft Technology Licensing, Llc | User recommendation using a multi-view deep learning framework |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5706496A (en) * | 1995-03-15 | 1998-01-06 | Matsushita Electric Industrial Co., Ltd. | Full-text search apparatus utilizing two-stage index file to achieve high speed and reliability of searching a text which is a continuous sequence of characters |
US20030177116A1 (en) * | 2002-02-28 | 2003-09-18 | Yasushi Ogawa | System and method for document retrieval |
US7644076B1 (en) * | 2003-09-12 | 2010-01-05 | Teradata Us, Inc. | Clustering strings using N-grams |
US8160402B2 (en) * | 2007-07-23 | 2012-04-17 | Sharp Kabushiki Kaisha | Document image processing apparatus |
US8412796B2 (en) * | 2009-07-31 | 2013-04-02 | University College Dublin—National University of Ireland, Dublin | Real time information feed processing |
Family Cites Families (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS5218303B2 (ja) | 1972-11-22 | 1977-05-20 | ||
JP3497243B2 (ja) | 1994-05-24 | 2004-02-16 | 株式会社日立製作所 | 文書検索方法及び装置 |
JP3253657B2 (ja) * | 1991-11-11 | 2002-02-04 | 株式会社リコー | 文書検索方法 |
JP3333549B2 (ja) | 1992-03-24 | 2002-10-15 | 株式会社リコー | 文書検索方式 |
JPH05266080A (ja) * | 1992-03-24 | 1993-10-15 | Matsushita Electric Ind Co Ltd | 検索装置 |
JPH0668159A (ja) | 1992-08-14 | 1994-03-11 | Matsushita Electric Ind Co Ltd | 検索装置 |
JP3563823B2 (ja) * | 1994-06-02 | 2004-09-08 | 株式会社リコー | 文書管理装置 |
JP3489237B2 (ja) | 1995-01-11 | 2004-01-19 | 株式会社日立製作所 | 文書検索方法 |
JP3335498B2 (ja) | 1995-03-29 | 2002-10-15 | 能美防災株式会社 | 火災報知設備 |
US7269548B2 (en) * | 2002-07-03 | 2007-09-11 | Research In Motion Ltd | System and method of creating and using compact linguistic data |
US7164802B2 (en) * | 2002-11-14 | 2007-01-16 | Zoran Corporation | Method for image compression by modified Huffman coding |
US7305385B1 (en) * | 2004-09-10 | 2007-12-04 | Aol Llc | N-gram based text searching |
US7561738B2 (en) * | 2004-09-22 | 2009-07-14 | Microsoft Corporation | Symbol grouping and recognition in expression recognition |
CN100530171C (zh) * | 2005-01-31 | 2009-08-19 | 日电(中国)有限公司 | 字典学习方法和字典学习装置 |
US7493293B2 (en) * | 2006-05-31 | 2009-02-17 | International Business Machines Corporation | System and method for extracting entities of interest from text using n-gram models |
US7877258B1 (en) * | 2007-03-29 | 2011-01-25 | Google Inc. | Representing n-gram language models for compact storage and fast retrieval |
JP5007743B2 (ja) * | 2007-05-24 | 2012-08-22 | 富士通株式会社 | 情報検索プログラム、該プログラムを記録した記録媒体、情報検索装置、および情報検索方法 |
US8359326B1 (en) * | 2008-04-02 | 2013-01-22 | Google Inc. | Contextual n-gram analysis |
JP5062131B2 (ja) * | 2008-10-06 | 2012-10-31 | 富士通株式会社 | 情報処理プログラム、情報処理装置、および情報処理方法 |
JP5418218B2 (ja) * | 2009-12-25 | 2014-02-19 | 富士通株式会社 | 情報処理プログラム、情報検索プログラム、情報処理装置、および情報検索装置 |
JP5524144B2 (ja) * | 2011-08-08 | 2014-06-18 | 株式会社東芝 | key−valueストア方式を有するメモリシステム |
-
2011
- 2011-03-31 JP JP2011081105A patent/JP5605288B2/ja active Active
-
2012
- 2012-03-19 US US13/423,319 patent/US20120254190A1/en not_active Abandoned
-
2014
- 2014-04-02 US US14/243,173 patent/US9720976B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5706496A (en) * | 1995-03-15 | 1998-01-06 | Matsushita Electric Industrial Co., Ltd. | Full-text search apparatus utilizing two-stage index file to achieve high speed and reliability of searching a text which is a continuous sequence of characters |
US20030177116A1 (en) * | 2002-02-28 | 2003-09-18 | Yasushi Ogawa | System and method for document retrieval |
US7644076B1 (en) * | 2003-09-12 | 2010-01-05 | Teradata Us, Inc. | Clustering strings using N-grams |
US8160402B2 (en) * | 2007-07-23 | 2012-04-17 | Sharp Kabushiki Kaisha | Document image processing apparatus |
US8412796B2 (en) * | 2009-07-31 | 2013-04-02 | University College Dublin—National University of Ireland, Dublin | Real time information feed processing |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104765737A (zh) * | 2014-01-03 | 2015-07-08 | 易搜比控股公司 | 撷取超文字标签语言文件内容的方法 |
US9450601B1 (en) * | 2015-04-02 | 2016-09-20 | Microsoft Technology Licensing, Llc | Continuous rounding of differing bit lengths |
US10747725B2 (en) | 2015-07-14 | 2020-08-18 | Fujitsu Limited | Compressing method, compressing apparatus, and computer-readable recording medium |
US10324963B2 (en) | 2015-10-09 | 2019-06-18 | Fujitsu Limited | Index creating device, index creating method, search device, search method, and computer-readable recording medium |
CN109558731A (zh) * | 2017-09-26 | 2019-04-02 | 腾讯科技(深圳)有限公司 | 特征码处理方法、装置及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
JP2012216088A (ja) | 2012-11-08 |
US9720976B2 (en) | 2017-08-01 |
JP5605288B2 (ja) | 2014-10-15 |
US20140214854A1 (en) | 2014-07-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9720976B2 (en) | Extracting method, computer product, extracting system, information generating method, and information contents | |
US20160321282A1 (en) | Extracting method, information processing method, computer product, extracting apparatus, and information processing apparatus | |
US9916314B2 (en) | File extraction method, computer product, file extracting apparatus, and file extracting system | |
CN106649783B (zh) | 一种同义词挖掘方法和装置 | |
US10389378B2 (en) | Computer product, information processing apparatus, and information search apparatus | |
US5649023A (en) | Method and apparatus for indexing a plurality of handwritten objects | |
CN100483417C (zh) | 获取限制词信息的方法、优化输出的方法和输入法*** | |
US8866647B2 (en) | Computer product, information processing apparatus, and information search apparatus | |
US9501557B2 (en) | Information generating computer product, apparatus, and method; and information search computer product, apparatus, and method | |
CN106557777B (zh) | 一种基于SimHash改进的Kmeans文档聚类方法 | |
JP6447161B2 (ja) | 意味構造検索プログラム、意味構造検索装置、及び意味構造検索方法 | |
EP3032439B1 (en) | Compression computer program and search computer program | |
US9542427B2 (en) | Computer product, generating apparatus, and generating method for generating Huffman tree, and computer product for file compression using Huffman tree | |
EP3635586A1 (en) | Computerized methods of data compression and analysis | |
CN113901214B (zh) | 表格信息的提取方法、装置、电子设备及存储介质 | |
US9501558B2 (en) | Computer product, searching apparatus, and searching method | |
CN111753534A (zh) | 标识文档中的序列标题 | |
US6470362B1 (en) | Extracting ordered list of words from documents comprising text and code fragments, without interpreting the code fragments | |
JP4208326B2 (ja) | 情報索引装置 | |
JPH11328318A (ja) | 確率テーブル作成装置、確率方式言語処理装置、認識装置、及び、記録媒体 | |
JP2016149160A (ja) | 情報生成方法、およびインデックス情報 | |
JP2016149160A5 (ja) | ||
JP2003331214A (ja) | 文字認識誤り訂正方法、装置及びプログラム | |
JP4618083B2 (ja) | 文書処理装置および文書処理方法 | |
CN117290523B (zh) | 基于动态索引表的全文检索方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FUJITSU LIMITED, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KATAOKA, MASAHIRO;MURATA, TAKAHIRO;OHTA, TAKAFUMI;REEL/FRAME:027936/0334 Effective date: 20111212 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |