CN112347146B - Index recommendation method and device - Google Patents
Index recommendation method and device Download PDFInfo
- Publication number
- CN112347146B CN112347146B CN202011141173.7A CN202011141173A CN112347146B CN 112347146 B CN112347146 B CN 112347146B CN 202011141173 A CN202011141173 A CN 202011141173A CN 112347146 B CN112347146 B CN 112347146B
- Authority
- CN
- China
- Prior art keywords
- execution
- database
- statement
- field
- priority
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005457 optimization Methods 0.000 claims abstract description 17
- 238000011156 evaluation Methods 0.000 claims description 98
- 238000012545 processing Methods 0.000 claims description 22
- 230000000875 corresponding effect Effects 0.000 claims description 14
- 230000002596 correlated effect Effects 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000014509 gene expression Effects 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000002354 daily effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an index recommendation method and device, wherein the method comprises the following steps: extracting each searched first field from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; for any one of the second fields, determining the index recommendation priority of the second field at least according to the execution information of each second database execution statement of the second field; and determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field. When the method is applied to the financial technology (Fintech), the index recommendation priority of the second field is obtained more reasonably, so that the recommendation index of the target database is obtained more reasonably.
Description
Technical Field
The invention relates to the field of databases in the field of financial science and technology (Fintech), in particular to an index recommendation method and an index recommendation device.
Background
With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changed to the financial technology (Fintech), but due to the requirements of safety and real-time performance of the financial industry, the requirements of the technology are also higher. Financial institutions process large amounts of data daily and frequently access databases. Compared with direct search, the database has obviously improved access performance by setting a proper index for searching.
At present, no automatic index recommendation technical scheme is available, and indexes are manually added through manually analyzing slow SQL sentences after slower structured query language (Structured Query Language, SQL) sentences are captured. However, this ignores a large number of optimization points and may repeat the optimization. For example, setting the execution duration threshold of the slow SQL statement to be 5 seconds makes the best sense for an SQL statement that is executed 1000 times per day but has an execution duration of 4.9 seconds more than an SQL statement that is executed 10 times per day but has an execution duration of 5.1 seconds but has been set with an index. But the SQL sentences are not captured because of being not slow, and particularly, the SQL sentences are more in scenes and easier to miss when being uniformly analyzed in a single person or a short time. Therefore, the optimization effect of the adding index method is not good, and an automatic index recommendation method is needed.
Disclosure of Invention
The invention provides an index recommendation method and device, which solve the problem of poor optimization effect of an index adding mode in the prior art.
In a first aspect, the present invention provides an index recommendation method, including: extracting each searched first field from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; for any one of the second fields, determining the index recommendation priority of the second field at least according to the execution information of each second database execution statement of the second field; the execution information of each second database execution statement characterizes the influence degree of the second field on the execution time of the database execution statement; and determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field.
In the method, after each first database execution statement of the target database is acquired, each first field which is searched is extracted from each first database execution statement, each second field is screened out through a preset optimization standard, only specific second fields are inspected, and for any second field of each second field, the index recommendation priority of each second field is determined according to the execution information of each second database execution statement of the second field, the execution information of each second database execution statement characterizes the influence degree of the second field on the execution time of the database execution statement, the influence degree is more recommended, the index recommendation priority of the second field is more reasonable, and the recommendation index of the target database is more reasonable.
Optionally, the determining the index recommendation priority of the second field at least according to the execution information of each second database execution statement of the second field includes: determining a first priority evaluation value of the second field and a second priority evaluation value of the second field at least according to execution information of each second database execution statement of the second field; the first priority evaluation value characterizes the resource consumption state of each second database execution statement; the second priority evaluation value characterizes the influence degree of the second field on the resource consumption status of each second database execution statement; and determining the index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value.
In the method, the first priority evaluation value and the second priority evaluation value are comprehensively considered, and the first priority evaluation value characterizes the resource consumption condition of each second database execution statement, and the second priority evaluation value characterizes the influence degree of the second field on the resource consumption condition of each second database execution statement, so that the index recommendation priority of the second field is more comprehensively determined.
Optionally, the execution information of each second database execution statement includes at least one of the following: average execution time length of the same database execution statement, execution times of the same database execution statement, average execution time length of each second database execution statement and standard deviation of the execution time length of each second database execution statement; the determining the first priority evaluation value of the second field and the second priority evaluation value of the second field at least according to the execution information of each second database execution statement of the second field includes: determining the first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement; the average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; the average execution duration of each first database execution statement is inversely related to the first priority evaluation value; determining the second priority evaluation value according to the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement; wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; and the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value.
In the method, the execution time and statistics of the database execution statement are relatively visual and simple quantization indexes reflecting the resource consumption state, so that the first priority evaluation value and the second priority evaluation value can be accurately and efficiently described.
Optionally, the index recommendation priority of the second field is determined according to the first priority evaluation value and the second priority evaluation value specifically in the following manner: wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences,/> A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>And evaluating the value for the second priority.
In the above manner, the method, on the one hand,The comprehensive execution duration of each second database execution statement is characterized, and/>Is the operation result of the high power, the result of the first power is obtained after the addition of ln operation for order reduction, and further, the result of the/>, is obtained by increasing TThereby comparing with the average execution time length T of each first database execution statement, and on the other hand,/>For the second priority evaluation value, it is obvious that the larger σs, the more the second field is dragged, and the average execution time length S of each second database execution statement is compared relatively, so that the sum can be obtainedAnd performing peer comparison, thereby comprehensively considering the factors of the whole sentence of the database execution sentence and the influence factors of the second field on the database execution sentence, and more precisely recommending the priority of the index of the second field.
Optionally, the determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field includes: merging all second database execution sentences of the second fields meeting the preset merging rules to obtain database execution sentences of the group fields, and determining the index recommendation priority of the group fields according to the index recommendation priority of the merged second fields; wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field; and determining the recommendation index of the target database according to the index recommendation priority of each group of fields and the index recommendation priority of each second field which is not combined.
In the above manner, each second database execution statement of the second field meeting the preset merging rule is merged to obtain the database execution statement of the group field, and the index recommendation priority of the group field is redetermined, so that the indexes are recommended by taking the group field as a unit, and the number of indexes is reduced.
Optionally, the merging the second database execution statements of the second fields meeting the preset merging rule to obtain a database execution statement of the group field, and determining the index recommendation priority of the group field according to the index recommendation priority of each merged second field, including: each second database execution statement of any second field forms a database execution statement set; if any first database execution statement set is a subset of a second database execution statement set, merging the first database execution statement set and the second database execution statement set to be used as the database execution statements of the group of fields; taking the highest index recommendation priority of the first index recommendation priority and the second index recommendation priority as the index recommendation priority of the group field; the first index recommendation priority is the index recommendation priority of the second field corresponding to the first database execution statement set, and the second index recommendation priority is the index recommendation priority of the second field corresponding to the second database execution statement set.
In the above manner, if any first database execution statement set is a subset of the second database execution statement set, merging the first database execution statement set and the second database execution statement set to obtain the database execution statement of the group field, and resetting the index recommendation priority of the group field, so that the index of the second field of the more database execution statements is covered by the index of the second field of the more database execution statement, and the index recommendation priority of the highest index of the first index recommendation priority and the second index recommendation priority is used as the index recommendation priority of the group field, so that the due recommendation state of the index of the original second field before merging is reserved.
Optionally, the determining each second field from each first field according to a preset optimization criterion includes: for any one of the first fields, if the data table of the first field satisfies the following chebyshev inequality, the first field is taken as a second field: x is less than mu-epsilon-delta; wherein X is the number of data records in the data table where the first field is located, μ is the average number of data records in the data table in the target database, σ is the standard deviation of the number of data records in the data table in the target database, and ε is a positive number greater than 1.
In the above manner, the first field of the smaller-scale data table can be removed through the chebyshev inequality, so that the operand of the subsequent step is reduced.
Optionally, the extracting each searched first field from each first database execution statement of the target database includes: acquiring an execution plan of each first database execution statement; and taking the fields in the selected search condition in the execution plan of each first database execution statement as the first fields.
In the above manner, after the execution plan of each first database execution statement is acquired, the field in the selection search condition is the query object of the main consumption resource of the entire database execution statement, and therefore, the field in the execution plan of each first database execution statement in the selection search condition is used as each first field, and each first field is a field that needs to be optimized.
In a second aspect, the present invention provides an index recommendation apparatus, including: the acquisition module is used for extracting searched first fields from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; the processing module is used for determining the index recommendation priority of any second field of the second fields at least according to the execution information of each second database execution statement of the second field; the execution information of each second database execution statement characterizes the influence degree of the second field on the execution time of the database execution statement; and determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field.
Optionally, the processing module is specifically configured to: determining a first priority evaluation value of the second field and a second priority evaluation value of the second field at least according to execution information of each second database execution statement of the second field; the first priority evaluation value characterizes the resource consumption state of each second database execution statement; the second priority evaluation value characterizes the influence degree of the second field on the resource consumption status of each second database execution statement; and determining the index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value.
Optionally, the execution information of each second database execution statement includes at least one of the following: average execution time length of the same database execution statement, execution times of the same database execution statement, average execution time length of each second database execution statement and standard deviation of the execution time length of each second database execution statement; the processing module is specifically configured to: determining the first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement; the average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; the average execution duration of each first database execution statement is inversely related to the first priority evaluation value; determining the second priority evaluation value according to the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement; wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; and the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value.
Optionally, the processing module is specifically configured to: determining an index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value in the following manner:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>And evaluating the value for the second priority.
Optionally, the processing module is specifically configured to: merging all second database execution sentences of the second fields meeting the preset merging rules to obtain database execution sentences of the group fields, and determining the index recommendation priority of the group fields according to the index recommendation priority of the merged second fields; wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field; and determining the recommendation index of the target database according to the index recommendation priority of each group of fields and the index recommendation priority of each second field which is not combined.
Optionally, the processing module is specifically configured to: each second database execution statement of any second field forms a database execution statement set; if any first database execution statement set is a subset of a second database execution statement set, merging the first database execution statement set and the second database execution statement set to be used as the database execution statements of the group of fields; taking the highest index recommendation priority of the first index recommendation priority and the second index recommendation priority as the index recommendation priority of the group field; the first index recommendation priority is the index recommendation priority of the second field corresponding to the first database execution statement set, and the second index recommendation priority is the index recommendation priority of the second field corresponding to the second database execution statement set.
Optionally, the processing module is specifically configured to: for any one of the first fields, if the data table of the first field satisfies the following chebyshev inequality, the first field is taken as a second field: x is less than mu-epsilon-delta; wherein X is the number of data records in the data table where the first field is located, μ is the average number of data records in the data table in the target database, σ is the standard deviation of the number of data records in the data table in the target database, and ε is a positive number greater than 1.
Optionally, the processing module is specifically configured to: acquiring an execution plan of each first database execution statement; and taking the fields in the selected search condition in the execution plan of each first database execution statement as the first fields.
The advantages of the foregoing second aspect and the advantages of the foregoing optional apparatuses of the second aspect may refer to the advantages of the foregoing first aspect and the advantages of the foregoing optional methods of the first aspect, and will not be described herein.
In a third aspect, the present invention provides a computer device comprising a program or instructions which, when executed, is operable to perform the above-described first aspect and the respective alternative methods of the first aspect.
In a fourth aspect, the present invention provides a storage medium comprising a program or instructions which, when executed, is adapted to carry out the above-described first aspect and the respective alternative methods of the first aspect.
These and other aspects of the invention will be more readily apparent from the following description of the embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it will be apparent that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart corresponding to an index recommendation method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a system architecture of an index recommendation system according to an embodiment of the present invention;
fig. 3 is a schematic flowchart corresponding to an index recommendation method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an index recommendation device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms appearing in the embodiments of the present application will be described first.
The dynamic weighting is that n evaluated objects (or systems) are provided, each of which is marked as S1, S2, … … Sn (n > 1), each system has m attributes (or evaluation indexes), each system has x1, x2, … … xm (m > 1), each attribute xi has K grades, each grade is marked as p1, p2, … … pK (K > 1), each grade pK contains a range of intervals, each grade is marked as [ ak (i), bk (i)), and ak (i) < bk (i) (i=1, 2,3 … … m, k=1, 2,3 … … K), namely when the attribute xi is [ ak (i), bk (i)), the attribute xi belongs to the kth class pK (1 is less than or equal to K), namely, each attribute has differences of different types and different magnitudes, and the conventional "difference" is a reasonable and comprehensive weighting method is used.
Chebyshev inequality the chebyshev inequality is an inequality demonstrated by russian mathematician chebyshev expressed in standard deviation, which has a statistically common meaning, called chebyshev's theorem, and is the following: assuming that the random variable X has a mathematical expectation E (X) =μ, the variance D (X) =σ 2 is for any positive number ε, the inequalityOr/>This is true.
Regular expressions, also known as regular expressions, (English: regular Expression, often abbreviated as regex, regexp, or RE in code), a concept of computer science. Regular expressions are typically used to retrieve, replace, text that meets a certain pattern (rule). A regular expression is a logical formula that operates on strings (including common characters (e.g., letters between a and z) and special characters (called "meta-characters")) by forming a "regular string" with specific characters defined in advance, and combinations of the specific characters, and this "regular string" is used to express a filtering logic for the string. A regular expression is a text pattern that describes one or more strings to be matched when searching text.
During the operation of a financial institution (banking institution, insurance institution or securities institution) in doing business (e.g., loan business, deposit business, etc. of a bank), the financial institution processes a large amount of data every day, and frequently accesses the database. Compared with direct search, the database has obviously improved access performance by setting a proper index for searching. At present, no automatic index recommendation technical scheme is available, and indexes are manually added through manually analyzing slow SQL sentences after slower structured query language (Structured Query Language, SQL) sentences are captured. However, a large number of optimization points can be ignored, and particularly, the SQL statement is more in scene, and is easier to miss when uniformly analyzed in a single person or in a short time. This situation does not meet the requirements of financial institutions such as banks, and cannot guarantee efficient operation of various businesses of the financial institutions. Therefore, the optimization effect of the adding index method is not good at present, and an automatic index recommendation method is needed. The embodiment of the application provides an index recommendation method, which is shown in fig. 1.
Step 101: extracting each searched first field from each first database execution statement of the target database; and determining each second field from the first fields according to a preset optimization standard.
Step 102: and determining the index recommendation priority of any second field of the second fields at least according to the execution information of each second database execution statement of the second field.
The execution information of each second database execution statement characterizes the influence degree of the second field on the execution duration of the database execution statement.
Step 103: and determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field.
In an alternative embodiment, the specific manner of extracting the searched first fields from the first database execution statements in the target database in step 101 may be as follows:
Acquiring an execution plan of each first database execution statement; and taking the fields in the selected search condition in the execution plan of each first database execution statement as the first fields.
It should be noted that, the execution plan is a common command in the databases, and the execution statements of each first database may be parsed. There are also various ways of extracting the searched first fields from the first database execution sentences of the target database, such as matching the text of the first database execution sentences by a regular expression.
In an alternative embodiment, the specific manner of determining each second field from each first field in step 101 may be as follows:
for any one of the first fields, if the data table of the first field satisfies the following chebyshev inequality, the first field is taken as a second field:
X＜μ-ε·δ。
wherein X is the number of data records in the data table where the first field is located, μ is the average number of data records in the data table in the target database, σ is the standard deviation of the number of data records in the data table in the target database, and ε is a positive number greater than 1.
It should be noted that, according to chebyshev's theorem, the ratio (or fraction) of any one data set lying within the range of its mean k standard deviations is always at least 1-1/k 2, where k is any positive number greater than 1. For k=2, k=3 and k=5 have the following results: of all the data, at least 3/4 (or 75%) of the data lie within 2 standard deviations of the mean. Of all the data, at least 8/9 (or 88.9%) of the data lie within 3 standard deviations of the mean. Of all the data, at least 24/25 (or 96%) of the data lie within 5 standard deviations of the mean.
If the data table of the first field is located outside the range of one or more standard deviations of the average number, the data table of the first field is smaller than the data record number of most data tables of the target database, and can be identified as a smaller-scale table in the target database, and the smaller-scale table can be ignored, so that the execution efficiency of the whole database statement is not greatly influenced, but the number of fields is reduced; if the provincial data table is used, the optimization is not optimized, and the influence on the execution efficiency of the whole database statement is small.
In an alternative embodiment (hereinafter referred to as an embodiment of the priority evaluation value), step 102 may specifically be as follows:
step (2-1): and determining a first priority evaluation value of the second field and a second priority evaluation value of the second field at least according to the execution information of each second database execution statement of the second field.
Step (2-2): and determining the index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value.
Wherein the first priority evaluation value characterizes resource consumption status of each second database execution statement; the second priority evaluation value characterizes the extent to which the second field affects the resource consumption status of the respective second database execution statement.
It should be noted that, the form of the first priority evaluation value may be various, as long as the resource consumption condition of each second database execution statement can be represented, for example, statistics of the execution time durations of the second database execution statements remaining after each second database execution statement eliminates a plurality of second database execution statements with the shortest execution time duration, for example, average execution time durations, and the like.
The second priority evaluation value may have various forms, as long as the degree of influence of the second field on the resource consumption condition of each second database execution statement can be represented, for example, the standard deviation of each second database execution statement, or the standard deviation of the execution duration of the second database execution statement remaining after each second database execution statement eliminates a plurality of second database execution statements with the shortest execution duration.
In an embodiment of the priority evaluation value, specifically, the execution information of each second database execution statement may include at least one of the following: the average execution time length of the same database execution statement, the execution times of the same database execution statement, the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement.
Based on this, step (2-1) may be as follows:
Step (2-1-1): and determining the first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement.
The average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; and the average execution duration of each first database execution statement is inversely related to the first priority evaluation value.
Step (2-1-2): and determining the second priority evaluation value according to the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement.
Wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; and the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value.
Based on the steps (2-1-1) to (2-1-2), the step (2-2) can be specifically realized by the following formula:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>And evaluating the value for the second priority.
In an alternative embodiment, step 103 may specifically be as follows:
And (3-1) merging the second database execution sentences of the second fields meeting the preset merging rules to obtain database execution sentences of the group fields, and determining the index recommendation priority of the group fields according to the index recommendation priority of the merged second fields.
And (3-2) determining the recommended index of the target database according to the index recommended priority of each group of fields and the index recommended priority of each second field which is not combined.
Wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field.
It should be noted that, the preset merging rules may be various, for example, for any first database execution statement set and second database execution statement set, when the first database execution statement set is a subset of the second database execution statement set, the first database execution statement set and the second database execution statement set are merged; for another example, when the same database execution statement in the first database execution statement set and the second database execution statement set is greater than a preset proportion, the first database execution statement set and the second database execution statement set are combined, and specific preset combining rules can be flexibly set according to specific scenes, which is not limited herein.
More specifically, step (3-1) may be performed as follows:
Step (3-1-1): each second database execution statement of any second field constitutes a set of database execution statements.
Step (3-1-2): and if any first database execution statement set is a subset of the second database execution statement set, merging the first database execution statement set and the second database execution statement set to be used as the database execution statements of the group of fields.
Step (3-1-3): and taking the highest index recommendation priority in the first index recommendation priority and the second index recommendation priority as the index recommendation priority of the group field.
The first index recommendation priority is the index recommendation priority of the second field corresponding to the first database execution statement set, and the second index recommendation priority is the index recommendation priority of the second field corresponding to the second database execution statement set.
An index recommendation method provided by the embodiment of the application is described in detail below with reference to fig. 2 and 3. Fig. 2 is a system architecture diagram applicable to the index recommendation method shown in fig. 1. Fig. 2 includes two parts, a main program and an external module. The external module is used for collecting execution information of the database execution statement from the outside. The main program part comprises a calculation module and a control module; the computing module is used for realizing the index recommendation method shown in fig. 1, obtaining each recommendation index and index recommendation priority, the control module is used for configuring a main program, providing a basic operation environment and outputting a final index recommendation report.
Fig. 3 is a specific flow chart of the index recommendation method shown in fig. 1, and it should be noted that fig. 3 is a more detailed explanation of fig. 1, but is not limited to fig. 1, for example, the database execution statement in fig. 3 is an SQL statement.
Step 301: and recording the execution information of the SQL statement.
Step 302: and collecting logs to obtain the execution information of the SQL sentences.
Step 303: an execution plan of the SQL statement is obtained.
Step 304: each first field is extracted using a regular expression resolution execution plan.
Step 305: the execution plan extraction index is parsed using regular expressions.
Step 306: the data is flushed, excluding the first field of the used index.
Step 307: the chebyshev's theorem is used to exclude the first field in the low-scale data table.
Step 308: an index recommendation priority for each second field is determined.
Step 309: a set of database execution statements for each second field is generated.
Step 310: it is determined whether the database execution statement sets can be merged.
Step 311: as a joint recommendation index.
Step 312: and regenerating a database execution statement set for the joint recommendation index.
Step 313: as individual recommendation indexes.
Step 314: a recommendation index report is issued.
In step 301, this may be implemented by means of druid tools. Capturing SQL fingerprints, SQL sentence texts and execution time of each SQL sentence executed by an application system in a database, execution times of the SQL sentences and other execution information through druid tools, and recording the execution information in a system log of an application server; it should be noted that the application system may be distributed, possibly deployed on a plurality of application servers, where each application server may execute an SQL statement and have corresponding execution information on the application server. Step 301 may be performed by an information acquisition module.
In step 302, specifically, the logs recorded in step 301 may be collected from different application servers by remote replication, and the log is parsed to obtain execution information of the SQL statement. Step 301 may be performed by a data collection module in a computing module.
In step 303, specifically, the execution plan of the SQL statement may be obtained by means of explain. explain is a key word of a command in the database, and does not actually execute the SQL statement, but can predict the execution process of the SQL statement.
In step 304, specifically, a regular expression may be used to extract, as each first field, a field used by a selected search condition (i.e., a where condition) in the execution plan of the SQL statement. In step 301, the execution information of the SQL statement is already recorded, and after each first field is extracted, the execution information of the SQL statement corresponding to each first field, such as the execution duration and the execution times of the SQL statement, can be obtained. It should be noted that, the fields in different data tables are distinguished in each first field, for example, the "name" field in the a data table is not identical to the "name" field in the B data table.
In step 305, the execution plan of the SQL statement may be parsed, specifically using a regular expression, with the primary key or index of the execution plan that is used by the selected search condition (i.e., the where condition) being marked.
In step 306, specifically, data cleansing may be performed on each first field, excluding the first fields that have used the primary key or index.
In step 307, specifically, using the chebyshev inequalityThe scale distribution of each data table in the target database is counted, and the low-scale data table is eliminated, so that the calculation amount of the subsequent steps is reduced, and the effect of taking the epsilon value of 2 or 3 in the inequality is better according to the practical experience.
It should be noted that, after step 307 is executed, granularity may be directly coarsened to a table level, so that the calculation amount may be greatly reduced, that is, a data table with a larger scale may be directly output, and the first field in the data table is used as the recommendation index.
It should be noted that, steps 303 to 307 may be performed by the data processing module.
In step 308, specifically, the index recommendation priority of each second field may be determined by the following formula:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each database execution statement in each second database execution statement, S represents the average execution time length of each database execution statement in each second database execution statement,/>And evaluating the value for the second priority.
It should be noted that, the formula extracts and quantifies the influence degree (also referred to as contribution) of different fields on the resource consumption by using a dynamic weighted evaluation method from the execution information of the SQL statement, and further gives priority to index recommendation. Wherein,For the total resource consumption of the whole sentence of the SQL sentence, taking natural logarithm reduction after weighted summation of the ratio of the sentence execution time length of the query x field to the average time length as the first half section of the formula, so that the second half section/>, of the sum formulaForming a peer operation. /(I)It is possible to check the degree of influence of other fields on the database execution statement when the query of the second field is in the database execution statement having a longer execution duration. If/>If the database execution statement containing the second field is larger, that is, if the second field itself does not cause that the execution time of the database execution statement is longer in some cases, the index recommendation priority of the second field passes through the second half of the formula/>, if the database execution statement containing the second field is slow by other fieldsTo decrease.
It should be noted that, step 308 may be performed by the data calculation module.
In step 309, specifically, if there are multiple second fields xi belonging to the same table, the number of indexes can be reduced by setting joint indexes for these second fields, and in order to evaluate this possibility, it is necessary to first count the SQL statement data execution statement Set (xi) using each second field xi, and it is to be noted that the specific database execution statement Set form of SQL may be composed of a Set of message digests (MD 5) of data execution statements.
In step 310, specifically, a Set operation is performed between the database execution statement sets Set (xi) generated in the previous step for each second field xi, so as to determine whether a certain database execution statement Set is a subset of other database execution statement sets. As occursI.e., set (xb) is a subset of Set (xa), then step 311 may be performed; otherwise, step 313 is performed.
It should be noted that, step 310 is a step that may be performed circularly, for example, until no database execution statement set is a subset of other database execution statement sets, so far, all individual primary keys that may be replaced by a joint primary key in the data table have been covered.
In step 311, specifically, the field xb is excluded from the recommendation index list, and the recommendation index of the field xa is changed to a joint recommendation index based on the field (xa, xb), and step 312 is performed.
In step 312, the joint recommendation index non-index recommendation priority is specifically the same as the index recommendation priority of xa.
It should be noted that, steps 309 to 313 may be performed by the aggregate calculating module.
In step 314, the recommendation index report may specifically include information such as a recommendation index and an index recommendation priority, and other information may also be added, such as the number of executions, a standard deviation of the average execution duration and the execution duration, and the like.
As shown in fig. 4, the present invention provides an index recommendation device, including: an obtaining module 401, configured to extract each first field searched from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; a processing module 402, configured to determine, for any second field of the second fields, an index recommendation priority of the second field at least according to execution information of each second database execution statement of the second field; the execution information of each second database execution statement characterizes the influence degree of the second field on the execution time of the database execution statement; and determining the recommendation index of the target database according to the second database execution statement of each second field and the index recommendation priority of each second field.
Optionally, the processing module 402 is specifically configured to: determining a first priority evaluation value of the second field and a second priority evaluation value of the second field at least according to execution information of each second database execution statement of the second field; the first priority evaluation value characterizes the resource consumption state of each second database execution statement; the second priority evaluation value characterizes the influence degree of the second field on the resource consumption status of each second database execution statement; and determining the index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value.
Optionally, the execution information of each second database execution statement includes at least one of the following: average execution time length of the same database execution statement, execution times of the same database execution statement, average execution time length of each second database execution statement and standard deviation of the execution time length of each second database execution statement; the processing module 402 is specifically configured to: determining the first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement; the average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; the average execution duration of each first database execution statement is inversely related to the first priority evaluation value; determining the second priority evaluation value according to the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement; wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; and the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value.
Optionally, the processing module 402 is specifically configured to: determining an index recommendation priority of the second field according to the first priority evaluation value and the second priority evaluation value in the following manner:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>And evaluating the value for the second priority.
Optionally, the processing module 402 is specifically configured to: merging all second database execution sentences of the second fields meeting the preset merging rules to obtain database execution sentences of the group fields, and determining the index recommendation priority of the group fields according to the index recommendation priority of the merged second fields; wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field; and determining the recommendation index of the target database according to the index recommendation priority of each group of fields and the index recommendation priority of each second field which is not combined.
Optionally, the processing module 402 is specifically configured to: each second database execution statement of any second field forms a database execution statement set; if any first database execution statement set is a subset of a second database execution statement set, merging the first database execution statement set and the second database execution statement set to be used as the database execution statements of the group of fields; taking the highest index recommendation priority of the first index recommendation priority and the second index recommendation priority as the index recommendation priority of the group field; the first index recommendation priority is the index recommendation priority of the second field corresponding to the first database execution statement set, and the second index recommendation priority is the index recommendation priority of the second field corresponding to the second database execution statement set.
Optionally, the processing module 402 is specifically configured to: for any one of the first fields, if the data table of the first field satisfies the following chebyshev inequality, the first field is taken as a second field: x is less than mu-epsilon-delta; wherein X is the number of data records in the data table where the first field is located, μ is the average number of data records in the data table in the target database, σ is the standard deviation of the number of data records in the data table in the target database, and ε is a positive number greater than 1.
Optionally, the processing module 402 is specifically configured to: acquiring an execution plan of each first database execution statement; and taking the fields in the selected search condition in the execution plan of each first database execution statement as the first fields.
Based on the same inventive concept, the embodiments of the present invention also provide a computer device including a program or an instruction, when the program or the instruction is executed, the index recommendation method and any optional method provided in the embodiments of the present invention are executed.
Based on the same inventive concept, the embodiments of the present invention also provide a computer readable storage medium including a program or instructions, which when executed, perform the index recommendation method and any optional method as provided in the embodiments of the present invention.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (6)
1. An index recommendation method, comprising:
Extracting each searched first field from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; wherein the execution information of each second database execution statement of any one of the second fields includes at least one of the following: average execution time length of the same database execution statement, execution times of the same database execution statement, average execution time length of each second database execution statement and standard deviation of the execution time length of each second database execution statement;
Determining a first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement; the first priority evaluation value characterizes the resource consumption state of each second database execution statement; the average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; the average execution duration of each first database execution statement is inversely related to the first priority evaluation value;
Determining a second priority evaluation value according to the average execution time length of each second database execution statement and the standard deviation of the execution time length of each second database execution statement; the second priority evaluation value characterizes the influence degree of the second field on the resource consumption status of each second database execution statement; wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value;
Determining an index recommendation priority of the second field based on the first priority evaluation value and the second priority evaluation value by the following formula:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>For the second priority evaluation value;
Merging all second database execution sentences of the second fields meeting the preset merging rules to obtain database execution sentences of the group fields, and determining the index recommendation priority of the group fields according to the index recommendation priority of the merged second fields; wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field;
And determining the recommendation index of the target database according to the index recommendation priority of each group of fields and the index recommendation priority of each second field which is not combined.
2. The method of claim 1, wherein merging each of the second database execution statements of the second fields satisfying the preset merge rule to obtain a database execution statement of the group field, and determining the index recommendation priority of the group field according to the index recommendation priority of each of the merged second fields, comprises:
Each second database execution statement of any second field forms a database execution statement set;
If any first database execution statement set is a subset of a second database execution statement set, merging the first database execution statement set and the second database execution statement set to be used as the database execution statements of the group of fields;
Taking the highest index recommendation priority of the first index recommendation priority and the second index recommendation priority as the index recommendation priority of the group field; the first index recommendation priority is the index recommendation priority of the second field corresponding to the first database execution statement set, and the second index recommendation priority is the index recommendation priority of the second field corresponding to the second database execution statement set.
3. The method of claim 1, wherein determining each second field from each first field according to a preset optimization criteria comprises:
for any one of the first fields, if the data table of the first field satisfies the following chebyshev inequality, the first field is taken as a second field:
X＜μ-ε·δ；
wherein X is the number of data records in the data table where the first field is located, μ is the average number of data records in the data table in the target database, σ is the standard deviation of the number of data records in the data table in the target database, and ε is a positive number greater than 1.
4. The method of claim 1, wherein extracting the searched first fields from the first database execution statements of the target database comprises:
acquiring an execution plan of each first database execution statement;
And taking the fields in the selected search condition in the execution plan of each first database execution statement as the first fields.
5. An index recommendation device, comprising:
The acquisition module is used for extracting searched first fields from each first database execution statement of the target database; determining each second field from each first field according to a preset optimization standard; wherein the execution information of each second database execution statement of any one of the second fields includes at least one of the following: average execution time length of the same database execution statement, execution times of the same database execution statement, average execution time length of each second database execution statement and standard deviation of the execution time length of each second database execution statement;
The processing module is used for determining a first priority evaluation value according to the average execution time length of the same database execution statement, the execution times of the database execution statement and the average execution time length of each first database execution statement; the first priority evaluation value characterizes the resource consumption state of each second database execution statement; the average execution time length of the same database execution statement and the execution times of the database execution statement are positively correlated with the first priority evaluation value; the average execution duration of each first database execution statement is inversely related to the first priority evaluation value;
the processing module is further configured to determine a second priority evaluation value according to the average execution duration of each second database execution statement and the standard deviation of the execution duration of each second database execution statement; the second priority evaluation value characterizes the influence degree of the second field on the resource consumption status of each second database execution statement; wherein the average execution duration of each second database execution statement is inversely related to the second priority evaluation value; the standard deviation of the execution time length of each second database execution statement is positively correlated with the second priority evaluation value;
The processing module is further configured to determine, based on the first priority evaluation value and the second priority evaluation value, an index recommendation priority of the second field according to the following formula:
Wherein P represents the index recommendation priority of the second field, ln represents a logarithmic operation based on natural logarithms, m represents the total number of the second database execution sentences, n i represents the execution times of the ith database execution sentence in the second database execution sentences, T i represents the average execution time of the ith database execution sentence in the second database execution sentences, T represents the average execution time of the first database execution sentences, A first priority evaluation value; sigma represents the standard deviation of the execution time length of each second database execution statement, S represents the average execution time length of each second database execution statement,/>For the second priority evaluation value;
The processing module is further configured to combine the second database execution statements of the second field that satisfy the preset combining rule to obtain a database execution statement of the group field, and determine an index recommendation priority of the group field according to the index recommendation priority of the combined second field; wherein the database execution statements of any one set of fields comprise a second database execution statement of at least one second field;
the processing module is further configured to determine a recommendation index of the target database according to the index recommendation priority of each group of fields and the index recommendation priority of each second field that is not combined.
6. A computer device comprising a program or instructions which, when executed, performs the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011141173.7A CN112347146B (en) | 2020-10-22 | 2020-10-22 | Index recommendation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011141173.7A CN112347146B (en) | 2020-10-22 | 2020-10-22 | Index recommendation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112347146A CN112347146A (en) | 2021-02-09 |
| CN112347146B true CN112347146B (en) | 2024-05-14 |
Family
ID=74359788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011141173.7A Active CN112347146B (en) | 2020-10-22 | 2020-10-22 | Index recommendation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347146B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0352050A2 (en) * | 1988-07-18 | 1990-01-24 | Digital Equipment Corporation | Single-keyed indexed file for TP queue repository |
| CN101102357A (en) * | 2006-07-05 | 2008-01-09 | 英业达股份有限公司 | Automatic sorting system and its method |
| KR20080087356A (en) * | 2007-03-26 | 2008-10-01 | 엔에이치엔(주) | Method and system for adding automatic indexing word to search database |
| CN103810212A (en) * | 2012-11-14 | 2014-05-21 | 阿里巴巴集团控股有限公司 | Automated database index creation method and system |
| KR101485940B1 (en) * | 2013-08-23 | 2015-01-27 | 네이버 주식회사 | Presenting System of Keyword Using depth of semantic Method Thereof |
| GB201511380D0 (en) * | 2015-06-29 | 2015-08-12 | Broadridge Financial Solutions Ltd | A data handling method |
| CN110807041A (en) * | 2019-11-01 | 2020-02-18 | 广州华多网络科技有限公司 | Index recommendation method and device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010081382A (en) * | 2008-09-26 | 2010-04-08 | Toshiba Corp | Device and method for selecting video telop |
| US9298761B2 (en) * | 2009-04-30 | 2016-03-29 | Hewlett Packard Enterprise Development Lp | Adaptive merging in database indexes |
-
2020
- 2020-10-22 CN CN202011141173.7A patent/CN112347146B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0352050A2 (en) * | 1988-07-18 | 1990-01-24 | Digital Equipment Corporation | Single-keyed indexed file for TP queue repository |
| CN101102357A (en) * | 2006-07-05 | 2008-01-09 | 英业达股份有限公司 | Automatic sorting system and its method |
| KR20080087356A (en) * | 2007-03-26 | 2008-10-01 | 엔에이치엔(주) | Method and system for adding automatic indexing word to search database |
| CN103810212A (en) * | 2012-11-14 | 2014-05-21 | 阿里巴巴集团控股有限公司 | Automated database index creation method and system |
| KR101485940B1 (en) * | 2013-08-23 | 2015-01-27 | 네이버 주식회사 | Presenting System of Keyword Using depth of semantic Method Thereof |
| GB201511380D0 (en) * | 2015-06-29 | 2015-08-12 | Broadridge Financial Solutions Ltd | A data handling method |
| CN110807041A (en) * | 2019-11-01 | 2020-02-18 | 广州华多网络科技有限公司 | Index recommendation method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于关系型数据库的SQL检索优化研究;徐昂;成科扬;;电子设计工程(第11期);第57-61页 * |
| 基于索引的SQL语句查询优化方法的研究;刘晓平;;韶关学院学报(第12期);第32-35页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112347146A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Virtucio et al. | Predicting decisions of the philippine supreme court using natural language processing and machine learning | |
| Choi et al. | Using machine learning to revisit the diversification–performance relationship | |
| Natt och Dag et al. | A feasibility study of automated natural language requirements analysis in market-driven development | |
| Yang et al. | Fairness-Aware Instrumentation of Preprocessing~ Pipelines for Machine Learning | |
| CN111767716B (en) | Method and device for determining enterprise multi-level industry information and computer equipment | |
| Bowen et al. | Generalized SHAP: Generating multiple types of explanations in machine learning | |
| CN108563783B (en) | Financial analysis management system and method based on big data | |
| CN112581006A (en) | Public opinion engine and method for screening public opinion information and monitoring enterprise main body risk level | |
| CN111967761A (en) | Monitoring and early warning method and device based on knowledge graph and electronic equipment | |
| CN107844533A (en) | A kind of intelligent Answer System and analysis method | |
| US20210141822A1 (en) | Systems and methods for identifying latent themes in textual data | |
| US20180336250A1 (en) | Apparatus and method for generation of a financial event database | |
| CN110544035A (en) | internal control detection method, system and computer readable storage medium | |
| Zhou et al. | EDM-JBW: A novel event detection model based on JS-ID′ Forder and Bikmeans with word embedding for news streams | |
| Yanti et al. | Application of named entity recognition via Twitter on SpaCy in Indonesian (case study: Power failure in the Special Region of Yogyakarta) | |
| CN111241288A (en) | Emergency sensing system of large centralized power customer service center and construction method | |
| CN111221873A (en) | Inter-enterprise homonym identification method and system based on associated network | |
| US11880394B2 (en) | System and method for machine learning architecture for interdependence detection | |
| US11562264B2 (en) | System and method for using machine learning to select one or more submissions from a plurality of submissions | |
| CN112347146B (en) | Index recommendation method and device | |
| Xu et al. | Probabilistic financial community models with latent dirichlet allocation for financial supply chains | |
| Li et al. | A commonsense knowledge-enabled textual analysis approach for financial market surveillance | |
| AlKhatib et al. | On the use of arabic tweets to predict stock market changes in the arab world | |
| CN113642321A (en) | Financial field-oriented causal relationship extraction method and system | |
| CN112347102A (en) | Multi-table splicing method and multi-table splicing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |