CN105630789A - Query plan converting method and device - Google Patents

Query plan converting method and device Download PDF

Info

Publication number
CN105630789A
CN105630789A CN201410588240.8A CN201410588240A CN105630789A CN 105630789 A CN105630789 A CN 105630789A CN 201410588240 A CN201410588240 A CN 201410588240A CN 105630789 A CN105630789 A CN 105630789A
Authority
CN
China
Prior art keywords
query
query operator
operator
plan
zone attribute
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410588240.8A
Other languages
Chinese (zh)
Other versions
CN105630789B (en
Inventor
李丰
康炎丽
冯晓兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Institute of Computing Technology of CAS
Original Assignee
Huawei Technologies Co Ltd
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, Institute of Computing Technology of CAS filed Critical Huawei Technologies Co Ltd
Priority to CN201410588240.8A priority Critical patent/CN105630789B/en
Publication of CN105630789A publication Critical patent/CN105630789A/en
Application granted granted Critical
Publication of CN105630789B publication Critical patent/CN105630789B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a query plant converting method and a query plan converting device, relates to the field of computers, which can greatly reduce the quantity of physical query tasks constituting a physical query plan. The query plan converting method specifically comprises the steps of extracting a first query operator and a second query operator from a logic query plan; if the partition property of the first query operator is the prefix of the partition property of the second query operator, rewriting the partition property of the second query operator in the logic query plan, thus allowing the partition property of the second query operator to be the same as the partition property of the first query operator; deleting the partition operator of the second query operator in the logic query plan, and generating a physical query task according to the second query operator, the first query operator and the partition operator of the first query operator to constitute the physical query plan. The query plant converting method is applied to the process of converting the logic query plan into the physical query plan.

Description

A kind of inquiry plan method for transformation and device
Technical field
The present invention relates to computer realm, particularly relate to a kind of inquiry plan method for transformation and device.
Background technology
Logical query plan is that one is resolved by query statement, the tree-shaped query structure being node with the query operator in query statement, each query operator in logical query plan has zone attribute and division operation symbol, and zone attribute is the property value that the required all data operated of this query operator are total. Division operation accords with between this query operator and another query operator, in order to two, interval query operator. Logical query plan can be converted into physical query plan.
Logical query plan is converted into the detailed process of physical query plan: accord with as starting to generate the mark of a physical queries task using a division operation, query operator between this division operation symbol and next division operation symbol is generated a physical queries task, using next division operation symbol as the mark generating next physical queries task, start to generate next physical queries task. Physical query plan is a kind of tree-shaped query structure being node with multiple physical queries tasks of these generations.
When constituting the physical queries number of tasks quantitative change needed for physical query plan and being few, reading data manipulation and data writing operation that execution physical queries required by task carries out also can tail off, the time overhead introduced owing to reading data manipulation and data writing operation also will diminish, and so the time performed used by physical query plan will diminish.
Prior art is in order to reach to reduce the purpose of the quantity constituting the physical queries task needed for physical query plan, search one, interval division operation symbol and identical two query operators of zone attribute in logical query plan, delete the division operation symbol between above-mentioned two query operator, make to be converted in the process of physical query plan at logical query plan, become generating a physical queries task from being originally generated as two different physical queries tasks by above-mentioned two query operator, reduce the quantity of physical queries task with this.
But this technology requires that two query operators have identical zone attribute, and has the follow-up relation of direct precursor, so can cause that the application scenarios of this technology is very restricted.
Summary of the invention
Embodiments of the invention provide a kind of inquiry plan method for transformation and device, it is possible to reduce the quantity of the physical queries task constituting physical query plan greatly.
For reaching above-mentioned purpose, embodiments of the invention adopt the following technical scheme that
The first aspect of the embodiment of the present invention, it is provided that a kind of inquiry plan method for transformation, including:
Extracting the first query operator and the second query operator from logical query plan, described first query operator is forerunner's operator of described second query operator;
If the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute physical query plan.
In conjunction with first aspect, in the implementation that the first is possible, if the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator, including:
If the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
Rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan, including:
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
In conjunction with first aspect, in the implementation that the second is possible, if the zone attribute of described first query operator is identical with the zone attribute of described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
It is likely to implementation in conjunction with first aspect, implementation that the first is possible and the second, in the implementation that the third is possible, before the described division operation symbol deleting described second query operator from described logical query plan, described method also includes:
Rewrite the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator;
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting;
Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
It is likely to implementation in conjunction with first aspect, implementation that the first is possible and the second, in the 4th kind of possible implementation, the described division operation according to described second query operator, described first query operator and described first query operator accords with and generates a physical queries task, to constitute described physical query plan, including:
Adopt task flow relatedness to optimize JFC technology, accord with according to the division operation of described second query operator, described first query operator and described first query operator and generate a physical queries task, to constitute described physical query plan.
The second aspect of the embodiment of the present invention, also provides for a kind of inquiry plan reforming unit, including:
Extraction unit, for extracting the first query operator and the second query operator from logical query plan, described first query operator is forerunner's operator of described second query operator;
First rewrites unit, if the prefix for the zone attribute that the zone attribute of described first query operator is described second query operator that described extraction unit extracts that described extraction unit extracts, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
Delete unit, after rewriting the zone attribute of the second query operator described in the unit described logical query plan of rewriting described first, from described logical query plan, delete the division operation symbol of described second query operator;
Generate unit, for after the division operation of the second query operator described in described deletion element deletion accords with, division operation symbol according to described second query operator, described first query operator and described first query operator generates a physical queries task, to constitute physical query plan.
In conjunction with second aspect, in the implementation that the first is possible, described first rewrites unit, including:
First rewrites module, if the prefix of the zone attribute that zone attribute is described second query operator for described first query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
Second rewrites module, for rewriting the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
Described generation unit, it is additionally operable to the division operation symbol according to described second query operator, described broadcast query operator, described first query operator and described first query operator and generates a physical queries task, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
In conjunction with second aspect, in the implementation that the second is possible, second rewrites unit, if the zone attribute of described second query operator that the zone attribute of described first query operator extracted for described extraction unit extracts with described extraction unit is identical, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator is rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with,
Described deletion unit, is additionally operable to, after described 3rd query operator is rewritten as described broadcast query operator by described second rewriting unit, delete the division operation symbol of described second query operator from described logical query plan;
Described generation unit, it is additionally operable to after the division operation of the second query operator described in described deletion element deletion accords with, division operation symbol according to described second query operator, described broadcast query operator, described first query operator and described first query operator generates a physical queries task, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
In conjunction with the implementation that second aspect, implementation that the first is possible and the second are possible, in the implementation that the third is possible, 3rd rewrites unit, before division operation for deleting described second query operator from described logical query plan at described deletion unit accords with, rewrite the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator;
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting;
Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
In conjunction with the implementation that second aspect, implementation that the first is possible and the second are possible, in the 4th kind of possible implementation, described generation unit, specifically for adopting task flow relatedness to optimize JFC technology, division operation symbol according to described second query operator, described first query operator and described first query operator generates a physical queries task, to constitute described physical query plan.
The inquiry plan method for transformation of embodiment of the present invention offer and device, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, just can pass through to rewrite the zone attribute of the second query operator in logical query plan, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, to generate a physical queries task according to the first query operator and the second query operator, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the zone attribute of the zone attribute of the first query operator and the second query operator be identical, the division operation symbol that can delete the second query operator is compared, when the prefix of the zone attribute that zone attribute is the second query operator of the first query operator, rewrite the zone attribute of the second query operator and can obtain the identical query operator meeting forerunner-follow-up relation of more zone attribute to (such as the first query operator and the second query operator), and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, the accompanying drawing used required in embodiment or description of the prior art will be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the premise not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of a kind of inquiry plan method for transformation in the embodiment of the present invention;
Fig. 2 is the example schematic of a kind of logical query plan in the embodiment of the present invention;
Fig. 3 is the schematic flow sheet of the another kind of inquiry plan method for transformation in the embodiment of the present invention;
Fig. 4 is the example schematic of the another kind of logical query plan in the embodiment of the present invention;
Fig. 5 is the example schematic of the another kind of logical query plan in the embodiment of the present invention;
Fig. 6 is the example schematic of the another kind of logical query plan in the embodiment of the present invention;
Fig. 7 is the schematic flow sheet of the another kind of inquiry plan method for transformation in the embodiment of the present invention;
Fig. 8 is the structural representation of a kind of inquiry plan reforming unit in the embodiment of the present invention;
Fig. 9 is the structural representation of the another kind of inquiry plan reforming unit in the embodiment of the present invention;
Figure 10 is the structural representation of the another kind of inquiry plan reforming unit in the embodiment of the present invention;
Figure 11 is the structural representation of the another kind of inquiry plan reforming unit in the embodiment of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only a part of embodiment of the present invention, rather than whole embodiments. Based on the embodiment in the present invention, the every other embodiment that those of ordinary skill in the art obtain under not making creative work premise, broadly fall into the scope of protection of the invention.
It addition, the terms " system " and " network " are often used interchangeably in this article. The terms "and/or", is only a kind of incidence relation describing affiliated partner, and expression can exist three kinds of relations, for instance, A and/or B, it is possible to represent: individualism A, there is A and B, individualism B these three situation simultaneously. It addition, character "/" herein, typically represent forward-backward correlation to as if the relation of a kind of "or".
Embodiment 1
The embodiment of the present invention provides a kind of inquiry plan method for transformation, it is possible to be applied to be converted in the process of physical query plan logical query plan, as it is shown in figure 1, this inquiry plan method for transformation includes:
S101, inquiry plan reforming unit extract the first query operator and the second query operator from logical query plan, and the first query operator is forerunner's operator of the second query operator.
Wherein, inquiry plan reforming unit extracts the method for the first query operator and the second query operator from logical query plan and may include that two query operators that there is forerunner-follow-up relation searched by inquiry plan reforming unit from logical query plan, and determines the zone attribute of two query operators that there is forerunner-follow-up relation respectively; Prefix when the zone attribute that zone attribute is follow-up query operator of the forerunner's query operator in two query operators that there is forerunner-follow-up relation, or when the zone attribute of forerunner's query operator is identical with the zone attribute of follow-up query operator, then two query operators that this can exist forerunner-follow-up relation are defined as the first query operator and the second query operator. Concrete, existing in two query operators of forerunner-follow-up relation, forerunner's query operator is as the first query operator, and follow-up query operator is as the second query operator.
It should be noted that in embodiments of the present invention, there is forerunner-follow-up relation and may include that the two query operator exists direct precursor-follow-up relation in two query operators in logical query plan.
Concrete, exist between two query operators in logical query plan and directly rely on relation, namely when the data of a query operator action required in the two query operator are the results of data operated by another query operator, then it is believed that the two query operator exists direct precursor-follow-up relation. Such as, assume that two query operators are query operator 1 and query operator 2, when the data of query operator 2 action required are the results that query operator 1 operates data, then it is believed that query operator 1 and query operator 2 exist direct precursor-follow-up relation, and the direct precursor operator that query operator 1 is query operator 2, query operator 2 is the immediate successor operator of query operator 1.
Further, there is forerunner-follow-up relation and can also include in two query operators in logical query plan: the two query operator exists indirect forerunner-follow-up relation.
Concrete, exist between two query operators in logical query plan and indirectly rely on relation, when being separated with at least one other query operator between namely between two query operators, if the data of a query operator action required in the two query operator are the results of data operated by these other query operators, and during result that data operated by these other query operators are data operated by another query operator in the two query operator, then it is believed that the two query operator exists indirect forerunner-follow-up relation. such as, assume that two query operators are query operator 3 and query operator 4, query operator 5 it is separated with between query operator 3 and query operator 4, when the data of query operator 4 action required are the results that query operator 5 operates data, and the data of query operator 5 action required are when being the result that query operator 3 operates data, then it is believed that query operator 3 and query operator 4 exist indirect forerunner-follow-up relation, and indirect forerunner's operator that query operator 3 is query operator 4, query operator 4 is the indirect successor operation symbol of query operator 3.
Accordingly, when between the first query operator and the second query operator, existence directly relies on relation, the first query operator is then the direct precursor operator of the second query operator; Exist between the first query operator and the second query operator and indirectly rely on relation, when being separated with three query operators between namely between the first query operator and the second query operator, the first query operator is then indirect forerunner's operator of the second query operator.
Exemplary, in the logical query plan shown in Fig. 2, the data of query operator 2 action required are the results that query operator 1 operates data, then may determine that between query operator 1 and query operator 2, there is direct precursor-follow-up relation, query operator 1 is that (zone attribute " zone attribute 1 " of query operator 1 and the zone attribute " zone attribute 1 " of query operator 2 are identical for the direct precursor operator of query operator 2, then may determine that query operator 1 is as the first query operator, query operator 2 is as the second query operator); The data of query operator 5 action required are the results that query operator 4 operates data, then may determine that between query operator 5 and query operator 4, there is direct precursor-follow-up relation, query operator 4 is the direct precursor operator (prefix of the zone attribute " zone attribute 3; zone attribute 5 " that the zone attribute " zone attribute 3 " of query operator 4 is query operator 5 of query operator 5, then may determine that query operator 4 is as the first query operator, query operator 5 is as the second query operator).
Exemplary, in the logical query plan shown in Fig. 2, the data of query operator 5 action required are the results that query operator 4 operates data, the data of query operator 4 action required are the results that query operator 3 operates data, then may determine that between query operator 3 and query operator 5, there is indirect forerunner-follow-up relation, query operator 3 is indirect forerunner's operator (zone attribute " zone attribute 3 that the zone attribute " zone attribute 3 " of query operator 3 is query operator 5 of query operator 5, zone attribute 5 " prefix, then may determine that query operator 3 is as the first query operator, query operator 5 is as the second query operator, query operator 4 is as the 3rd query operator).
It should be noted that, the inquiry plan method for transformation that the embodiment of the present invention provides can apply to have in the equipment (background server/storage device such as communication system or data handling system) of data query function, inquiry plan reforming unit in the embodiment of the present invention can be the processor of the above-mentioned equipment with data query function, such as central processing unit (CentralProcessingUnit, CPU), or inquiry plan reforming unit can also be a functional module in the processor of the above-mentioned equipment with data query function, the concrete form of inquiry plan reforming unit is not limited by the embodiment of the present invention.
If the prefix of the zone attribute that zone attribute is the second query operator of S102 the first query operator, then the zone attribute of the second query operator in logical query plan rewritten by inquiry plan reforming unit, so that the zone attribute of the zone attribute of the second query operator and the first query operator is identical.
Exemplary, the zone attribute assuming the first query operator is: " zone attribute 1 ", the zone attribute of the second query operator is: " zone attribute 1; zone attribute 2 ", then draw the prefix of the zone attribute that zone attribute " zone attribute 1 " is the second query operator " zone attribute 1, zone attribute 2 " of the first query operator.
Wherein, the zone attribute of the first query operator is the prefix of the zone attribute of the second query operator, it is possible to represent that namely carry out subregion by the zone attribute of the first query operator is that in the tables of data by the first query operator action required, all row containing zone attribute 1 are divided into a region; Namely carry out subregion by the zone attribute of the second query operator is all in the tables of data operated by the second query operator be divided into a region containing zone attribute 1 and containing the row of zone attribute 2, this shows, when the prefix of the zone attribute that the zone attribute of the first query operator extracted is the second query operator extracted, by the packet in the region that the zone attribute of the first query operator divides containing the data in the region divided by the zone attribute of the second query operator.
It should be noted that, rewrite the zone attribute of the second query operator in logical query plan, the zone attribute of the first query operator it is rewritten as by the zone attribute of the second query operator, the zone attribute keeping the first query operator is constant, and the zone attribute of zone attribute and the first query operator to ensure revised second query operator is identical. So, the zone attribute of the first query operator and the zone attribute of the second query operator are identical, just meet the identical condition of zone attribute of the query operator pair that prior art requires, in order to the first query operator and the second query operator are generated a physical queries task.
The division operation symbol of the second query operator deleted by S103, inquiry plan reforming unit from logical query plan, and the division operation according to the second query operator, the first query operator and the first query operator accords with one physical queries task of generation, to constitute physical query plan.
Wherein, identical through the zone attribute of revised second query operator and the zone attribute of the first query operator, now the first query operator and the second query operator meet the query operator of the prior art requirement condition to having identical zone attribute, after deleting the division operation symbol of the second query operator, owing to there is forerunner-follow-up relation between the first query operator and the second query operator, and first no longer have division operation to accord with between query operator and the second query operator, accord with as the mark generating new physical queries task using the division operation of the first query operator, first query operator and the second query operator can generate a physical queries task. inquiry plan reforming unit with generate multiple physical queries tasks for one physical query plan of joint structure.
The inquiry plan method for transformation that the embodiment of the present invention provides, pass through this programme, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, just can pass through to rewrite the zone attribute of the second query operator in logical query plan, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, to generate a physical queries task according to the first query operator and the second query operator, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the zone attribute of the zone attribute of the first query operator and the second query operator be identical, the division operation symbol that can delete the second query operator is compared, when the prefix of the zone attribute that zone attribute is the second query operator of the first query operator, rewrite the zone attribute of the second query operator and can obtain the identical query operator meeting forerunner-follow-up relation of more zone attribute to (such as the first query operator and the second query operator), and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
Embodiment 2
The embodiment of the present invention provides a kind of inquiry plan method for transformation, it is possible to being applied in the process that logical query plan converts to physical query plan, this inquiry plan method for transformation includes:
S201, inquiry plan reforming unit extract the first query operator and the second query operator from logical query plan, and the first query operator is forerunner's operator of the second query operator.
First query operator is be separated with the 3rd query operator between forerunner's operator of the second query operator specifically may include that between direct precursor operator, the first query operator and the second query operator that the first query operator is the second query operator.
In the first application scenarios of the embodiment of the present invention, as shown in Figure 3, extract the first query operator and the second query operator at inquiry plan reforming unit after, inquiry plan reforming unit is separated with the 3rd query operator between whether can first judging between the first query operator and the second query operator, namely performs S202:
S202, inquiry plan reforming unit are separated with the 3rd query operator between whether judging between the first query operator and the second query operator.
Wherein, between the first query operator and the second query operator, it is separated with the 3rd query operator, refers to that the first query operator is indirect forerunner's operator of the second query operator.
Concrete, in the first application scenarios of the embodiment of the present invention, if the direct precursor operator that the first query operator is the second query operator, namely there is no interval the 3rd query operator between the first query operator and the second query operator, then continue executing with S204 and follow-up flow process; If indirect forerunner's operator that the first query operator is the second query operator, it is separated with the 3rd query operator between namely between the first query operator and the second query operator, then continues executing with S203 and follow-up flow process.
S203, inquiry plan reforming unit judge whether the 3rd query operator can use broadcast query algorithm to realize.
Concrete, if the 3rd query operator can use broadcast query algorithm to realize, inquiry plan reforming unit then continues executing with S204; If the 3rd query operator can not use broadcast query algorithm to realize, 3rd query operator then can not be rewritten as broadcast query operator by inquiry plan reforming unit, and then the division operation that can not delete the second query operator accords with, to merge the first query operator and the second query operator.
3rd query operator is rewritten as broadcast query operator by S204, inquiry plan reforming unit, and broadcast query operator does not have division operation to accord with.
Wherein, when being separated with three query operators between the first query operator and the second query operator, if then representing, logical query plan is being rewritten as in the process of physical query plan by inquiry plan reforming unit, a physical queries task is only generated according to the first query operator and the second query operator, then before generating physics query task, need the 3rd query operator at interval between the first query operator and the second query operator is rewritten, otherwise when generating physics query task, the 3rd query operator owing to being spaced between the first query operator and the second query operator has division operation symbol, namely represent that there is between the first query operator and the second query operator a division operation symbol, inquiry plan reforming unit is when generating physics query task, two physical queries tasks then at least can be generated according to the first query operator and the second query operator, then can not reach to reduce the purpose of the quantity of the physical queries task constituting physical query plan.
Based on foregoing description, when being separated with three query operators between the first query operator and the second query operator, 3rd query operator then can be rewritten as broadcast query operator by inquiry plan reforming unit, and broadcast query operator does not have division operation to accord with, therefore, owing to now broadcast query operator between the first query operator and the second query operator does not have division operation to accord with, after deleting the division operation symbol of the second query operator, division operation is not had to accord with between first query operator and the second query operator, then inquiry plan reforming unit is when generating physics query task, a physical queries task only can be generated according to the first query operator and the second query operator, then can reach to reduce the purpose of the quantity of the physical queries task constituting physical query plan.
Exemplary, as shown in Fig. 4 or Fig. 5, the data of query operator 2 action required are the results that query operator 3 operates data, the data of query operator 3 action required are the results that query operator 1 operates data, it is determined that query operator 1 and query operator 2 exist indirect forerunner-follow-up relation. Wherein, the prefix of the zone attribute that zone attribute is query operator 2 of query operator 1 in logical query plan before rewriting in Fig. 4, in the logical query plan before rewriting in Figure 5, the zone attribute of query operator 1 is identical with the zone attribute of query operator 2.
S205, inquiry plan reforming unit determine the zone attribute of the first query operator and the relation of the zone attribute of the second query operator.
Wherein, the relation of the zone attribute of the zone attribute of the first query operator and the second query operator includes: the zone attribute that the zone attribute of the first query operator is the prefix of the zone attribute of the second query operator, the zone attribute of the first query operator and the second query operator is identical.
Concrete, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, then continue executing with S206 and follow-up flow process; If the zone attribute of the zone attribute of the first query operator and the second query operator is identical, then continue executing with S207 and follow-up flow process.
The zone attribute of the second query operator in logical query plan rewritten by S206, inquiry plan reforming unit, so that the zone attribute of the zone attribute of the second query operator and the first query operator is identical.
Wherein, inquiry plan reforming unit can rewrite the zone attribute of the second query operator in logical query plan, the zone attribute of the first query operator it is rewritten as by the zone attribute of the second query operator, the zone attribute being maintained with the first query operator is constant, the zone attribute of zone attribute and the first query operator to ensure revised second query operator is identical, after so rewriting, first query operator and the second query operator have identical zone attribute, meet prior art and generate the condition to having identical zone attribute of the query operator required by physics query task, so that when being converted into physical query plan by logical query plan, first query operator and the second query operator are generated a physical queries task.
Exemplary, as shown in Figure 4, the zone attribute of the query operator 2 in " rewriting the logical query plan after broadcast query operator " can be rewritten by inquiry plan reforming unit, by the zone attribute " zone attribute 1 of query operator 2, zone attribute 2 " it is rewritten as the zone attribute " zone attribute 1 " of query operator 1, the zone attribute " zone attribute 1 " being maintained with query operator 1 is constant, so that the zone attribute of query operator 1 is identical with the zone attribute of query operator 2, then can obtain " rewriting the logical query plan after zone attribute " as shown in Figure 4.
The division operation symbol of the second query operator deleted by S207, inquiry plan reforming unit from logical query plan.
It should be noted that, now the zone attribute of the second query operator and the zone attribute of the first query operator are identical, meet the identical requirement of zone attribute of the query operator that prior art requires, the division operation of the second query operator in logical query plan is accorded with after deleting, owing to there is forerunner-follow-up relation between the first query operator and the second query operator, and no longer there is division operation to accord with between the first query operator and the second query operator, therefore, accord with as the mark generating new physical queries task using the division operation of the first query operator, first query operator and the second query operator can generate a physical queries task.
Exemplary, inquiry plan reforming unit can using " rewrite zone attribute after logical query plan " as shown in Figure 4 or " revised logical query plan " as shown in Figure 5 as " deleting the logical query plan before division operation accords with " as shown in Figure 6, and delete the division operation symbol of query operator 2 in " deleting the logical query plan before division operation symbol ", to obtain " deleting the logical query plan after division operation symbol " as shown in Figure 6.
S208, inquiry plan reforming unit accord with according to the division operation of the second query operator, the first query operator and the first query operator and generate a physical queries task, to constitute physical query plan.
Wherein, when being separated with three query operators between the first query operator and the second query operator, inquiry plan reforming unit accords with, according to the division operation of the second query operator, the first query operator and the first query operator, the method generating a physical queries task and is specifically as follows: accords with according to the division operation of the second query operator, broadcast query operator, the first query operator and the first query operator and generates a physical queries task.
Exemplary, inquiry plan reforming unit can adopt task flow relatedness to optimize (Job-flowCorrelation, JFC) technology, division operation symbol according to the second query operator, the first query operator and the first query operator generates a physical queries task, to constitute physical query plan. Inquiry plan reforming unit adopts JFC technology to generate physics query task, concrete grammar to constitute physical query plan generates physics query task with employing JFC technology in prior art, method to constitute physical query plan is similar, and the embodiment of the present invention repeats no more here.
In the second application scenarios of the embodiment of the present invention, as shown in Figure 7, extract the first query operator and the second query operator at inquiry plan reforming unit after, inquiry plan reforming unit can first determine the zone attribute of the first query operator and the relation of the zone attribute of the second query operator, namely performs S208.
In the second application scenarios, as shown in Figure 7, after performing S205, if the prefix of the zone attribute that zone attribute is the second query operator of the first query operator determined by inquiry plan reforming unit, then can continue executing with S206, namely the zone attribute of the second query operator in logical query plan rewritten by inquiry plan reforming unit, so that the zone attribute of the zone attribute of the second query operator and the first query operator is identical; If the zone attribute that inquiry plan reforming unit determines the zone attribute of the first query operator and the second query operator is identical, then directly performing S207, namely the division operation symbol of the second query operator deleted by inquiry plan reforming unit from logical query plan; Then inquiry plan reforming unit is separated with the 3rd query operator between whether determining between the first query operator and the second query operator, namely performs S202; If inquiry plan reforming unit is separated with the 3rd query operator between determining between the first query operator and the second query operator, then continue executing with S204, if inquiry plan reforming unit is separated with the 3rd query operator between determining between the first query operator and the second query operator not, then directly perform S208.
The embodiment of the present invention utilizes below equation in order to calculate the beneficial effect adopting the inquiry plan method for transformation that the embodiment of the present invention provides to bring.
The time spent by a physical queries task of performing by the time read spent by data, write time four part spent by the time spent by data, the time spent by transmitted data on network, CPU computing and form, by detecting the time used by the data reading unit data quantity, the size of the data volume of input before operating with query operator in current physical queries task, calculates the time read spent by data; The time used by data of unit data quantity and the size of the data volume exported after query operator operation in current physical queries task are write in detection, calculate the time write spent by data; The data of unit data quantity are carried out the time used by computing by detection CPU, in percentage ratio input data used with query operator in current physical queries task and current physical queries task, the size of the data volume of input before query operator operation, calculates the time spent by CPU computing; The data of unit data quantity are transmitted the time used by detection network, in percentage ratio input data used with query operator in current physical queries task and current physical queries task, the size of the data volume of input before query operator operation, carrys out the computing network transmission time spent by data.
For three query operator OP in a logical query plani��OPjAnd OPk, satisfy condition OPiIt is OPjDirect precursor operator, OPjIt is OPkDirect precursor operator, and OPiZone attribute be OPkThe prefix of zone attribute, OPjCan be completed by broadcast query operator.
Before adopting the inquiry plan method for transformation that the embodiment of the present invention provides, these three query operator generates three physical queries tasks, the time spent by three physical queries tasks that performs is to perform the time sum spent by these three physical queries task respectively, it is possible to calculate according to below equation one:
Costbefore=Hr��Ii+v��Ii+�̡�ai��Ii+v��ai��Ii+Hw��Mi
+Hr��Ij+v��Ij+�̡�aj��Ij+v��aj��Ij+Hw��Mj
+Hr��Ik+v��Ik+�̡�ak��Ik+v��ak��Ik+Hw��Mk
Formula one
Wherein CostbeforeBefore representing the inquiry plan method for transformation that embodiment of the present invention offer is provided, perform the time spent by three physical queries tasks, HrRepresent the time used by data reading unit data quantity, HwRepresenting the time used by data writing unit data quantity, v represents that the data of unit data quantity are carried out the time used by computing by CPU, and �� represents that in network, the data of unit data quantity are transmitted time used, Ii��IjAnd IkRepresent the query operator OP in current three physical queries tasks respectivelyi��OPjAnd OPkThe size of the data volume of input, M before operationi��MjAnd MkRepresent the query operator OP in current three physical queries tasks respectivelyi��OPjAnd OPkThe size of the data volume of output, a after operationi��ajAnd akRepresent the query operator OP in current three the physical queries tasks performed respectivelyi��OPjAnd OPkThe percentage ratio that input data are used.
After adopting the inquiry plan method for transformation that the embodiment of the present invention provides, query operator OPkWith query operator OPiZone attribute identical, query operator OPjIt is rewritten as broadcast query operator, three query operators can be generated as a physical queries task, owing to three query operators are in same physical queries task, when then performing physical queries task, only carry out single reading according to operation and a data writing operation, although the broadcast query operator in this physical queries task need not to query operator OPiOutput data carry out read data manipulation, but still need to other input source data carry out read data manipulation. Therefore system performs this time spent by physical queries task and can calculate according to below equation two:
Costafter=Hr��Ii+v��Ii+�̡�ai��Ii+v��ai��Ii+Hr��(Ij-Mi)+v��r��Ij+v��aj��Ij+
v��ak��Ik+Hw��Mk
Formula two
Wherein, CostafterAfter representing the inquiry plan method for transformation that embodiment of the present invention offer is provided, perform three query operator OPi��OPjAnd OPkThe time spent by a physical queries task generated, r represents MiNumber of processes, HrRepresent the time used by data reading unit data quantity, HwRepresenting the time used by data writing unit data quantity, v represents that the data of unit data quantity are carried out the time used by computing by CPU, and �� represents that in network, the data of unit data quantity are transmitted time used, Ii��IjAnd IkRepresent the query operator OP in current physical queries task respectivelyi��OPjAnd OPkThe size of the data volume of input, M before operationiAnd MkRepresent the query operator OP in current physical queries task respectivelyiAnd OPkThe size of the data volume of output, a after operationi��ajAnd akRepresent the query operator OP in the current physical queries task performed respectivelyi��OPjAnd OPkThe percentage ratio that input data are used.
Adopt the inquiry plan method for transformation savable time that the embodiment of the present invention provides, for the difference of time spent after time spent before adopting the inquiry plan method for transformation and device that the embodiment of the present invention provides and the inquiry method for transformation adopting the embodiment of the present invention to provide and device. The inquiry plan method for transformation and the device savable time that adopt embodiment of the present invention offer can be passed through below equation three and be calculated:
Costsaved=Costbefore-Costafter
=Hw��Mi+Hr��Mi-v��(r-1)��Ij+�̡�aj��Ij+Hw��Mj+Hr��Ik+v��Ik+
�̡�ak��Ik
=(Hw+Hr)��Mi+(�̡�aj-v��(r-1))��Ij+Hw��Mj+(Hr+v+�̡�ak)��Ik
Formula three
Wherein, CostsavedFor represent adopt the embodiment of the present invention provide inquiry plan method for transformation after the savable time, r represents MiNumber of processes, HrRepresent the time used by data reading unit data quantity, HwRepresenting the time used by data writing unit data quantity, v represents that the data of unit data quantity are carried out the time used by computing by CPU, and �� represents that in network, the data of unit data quantity are transmitted time used, IjAnd IkRepresent query operator OP respectivelyjAnd OPkThe size of the data volume of input, M before operationiAnd MjRepresent query operator OP respectivelyiAnd OPjThe size of the data volume of output, a after operationjAnd akRepresent query operator OP respectivelyjAnd OPkThe percentage ratio that input data are used.
When in system, reading data manipulation and data writing operation account for the leading of time overhead, namely when the ratio that reading data manipulation and data writing operation account in time overhead is more than other times expense, all negligible for the coefficient �� (data of unit data quantity are transmitted the time used by network) in formula three and coefficient v (data of unit data quantity are carried out the time used by computing by CPU), then after adopting the inquiry plan method for transformation of embodiment of the present invention offer, the savable time is:
Costsaved=(Hw+Hr)��Mi+Hw��Mj+Hr��Ik
Wherein, CostsavedFor represent adopt the embodiment of the present invention provide inquiry plan method for transformation after the savable time, HrRepresent the time used by data reading unit data quantity, HwRepresent the time used by data writing unit data quantity, IkRepresent query operator OPkThe size of the data volume of input, M before operationiAnd MjRepresent query operator OP respectivelyiAnd OPjThe size of the data volume of output after operation.
When in system, network transmission accounts for the leading of time overhead, when the ratio that namely network transmission accounts in time overhead is more than other times expense, for the coefficient H in formula threer(reading the time used by data of unit data quantity), coefficient Hw(writing the time used by the data of unit data quantity) and coefficient v (data of unit data quantity are carried out the time used by computing by CPU) are all negligible, then after adopting the inquiry plan method for transformation of embodiment of the present invention offer, the savable time is:
Costsaved=�� �� aj��Ij+�̡�ak��Ik
Wherein, CostsavedFor represent adopt the embodiment of the present invention provide inquiry plan method for transformation after the savable time, �� represents that in network, the data of unit data quantity are transmitted time used, IjAnd IkRepresent query operator OP respectivelyjAnd OPkThe size of the data volume of input, a before operationjAnd akRepresent query operator OP respectivelyjAnd OPkThe percentage ratio that input data are used.
When system being read data manipulation and data writing operation and network and transmitting that proportion is suitable on time overhead, during the ratio that the ratio that namely reading data manipulation and data writing operation account in time overhead accounts in time overhead equal to network transmission, only coefficient v (data of unit data quantity are carried out the time used by computing by CPU) is ignored in formula three, then after adopting the inquiry plan method for transformation of embodiment of the present invention offer, the savable time is:
Costsaved=(Hw+Hr)��Mi+�̡�aj��Ij+Hw��Mj+(Hr+��+ak)��Ik
Wherein, CostsavedFor represent adopt the embodiment of the present invention provide inquiry plan method for transformation after the savable time, HrRepresent the time used by data reading unit data quantity, HwRepresenting the time used by data writing unit data quantity, �� represents that in network, the data of unit data quantity are transmitted time used, IjAnd IkRepresent query operator OP respectivelyjAnd OPkThe size of the data volume of input, M before operationiAnd MjRepresent query operator OP respectivelyiAnd OPjThe size of the data volume of output, a after operationjAnd akRepresent query operator OP respectivelyjAnd OPkThe percentage ratio that input data are used.
Further alternative, in order to improve the search efficiency of data query system, before the division operation deleting the second query operator from logical query plan accords with, this inquiry plan method for transformation can also include: rewrites the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator.
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting; Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
It should be noted that the data in the tables of data operated by the query operator of logical query plan are ranked up by division operation symbol according to the ordering attribute of query operator.
When the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator before rewriting, namely the ordering attribute of the first query operator is the prefix of the ordering attribute of the second query operator. assume that the ordering attribute of the first query operator is for " ordering attribute 1 ", the ordering attribute of the second query operator is " ordering attribute 1, ordering attribute 2 " time, when tables of data operated by first query operator and the second query operator is ranked up by division operation symbol according to the ordering attribute of the first query operator, namely it is to together by line orderings containing identical ordering attribute 1 all in tables of data, when tables of data operated by first query operator and the second query operator is carried out subregion sequence according to the ordering attribute of the second query operator by division operation symbol, namely be by tables of data all containing identical ordering attribute 1 and containing the line ordering of identical ordering attribute 2 to together.
Under scene ordering attribute not rewritten, when being converted into physical query plan by logical query plan, delete owing to the division operation of the second query operator accords with, first query operator and the second query operator will generate a physical queries task, the tables of data of the first query operator and the second query operator action required is ranked up by division operation symbol by the ordering attribute according to the first query operator, owing to zone attribute and the ordering attribute of the first query operator are identical, therefore it is identical carry out the result of subregion sequence and subregion sequence according to the ordering attribute of the first query operator before. when tables of data after sorting according to the ordering attribute of the first query operator is operated by the second query operator, it is still necessary to this tables of data is comprehensively scanned, all row containing ordering attribute 2 are found out, so, can compare and expend time in.
The ordering attribute of the first query operator is rewritten, namely it is the ordering attribute that the ordering attribute of the first query operator is rewritten as the second query operator, so the tables of data of the first query operator and the second query operator action required is carried out subregion sequence by the ordering attribute according to the second query operator by division operation symbol, by in tables of data all containing identical ordering attribute 1 and containing the line ordering of identical ordering attribute 2 to together, tables of data need not comprehensively be scanned by the second query operator again, just can find the data needing to be operated, relative to scene ordering attribute not rewritten, save the time needed for scanning data.
The inquiry plan method for transformation that the embodiment of the present invention provides, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, just the zone attribute of the second query operator in logical query plan can be rewritten, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, to generate a physical queries task according to the first query operator and the second query operator, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the zone attribute of the zone attribute of the first query operator and the second query operator be identical, the division operation symbol that can delete the second query operator is compared, when the prefix of the zone attribute that zone attribute is the second query operator of the first query operator, rewrite the zone attribute of the second query operator and can obtain the identical query operator meeting forerunner-follow-up relation of more zone attribute to (such as the first query operator and the second query operator), and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
And pass through this programme, when the first query operator is indirect forerunner's operator of the second query operator, namely it is separated with the 3rd query operator between the first query operator and the second query operator, when 3rd query operator can use broadcast query algorithm to realize, 3rd query operator can be rewritten as broadcast query operator, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, by the first query operator, second query operator and broadcast query operator generate a physical queries task, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the direct precursor operator that the first query operator is the second query operator, the division operation symbol of the second query operator can be deleted, make identical first query operator of zone attribute and the second query operator generate a physical queries task to compare, when being separated with three query operators between the first query operator and the second query operator, 3rd query operator is rewritten as broadcast query operator, so that broadcast query operator first query operator identical with zone attribute and the second query operator can generate a physical queries task, and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
It should be noted that the logical query plan example provided in the embodiment of the present invention is illustrative of several, plays the effect of explanation, do not represent the embodiment of the present invention and may only be applied in these several logical query plan examples.
Embodiment 3
The embodiment of the present invention provides a kind of inquiry plan reforming unit, as shown in Figure 8, including: extraction unit 31, first is rewritten unit 32, is deleted unit 33 and generate unit 34.
Extraction unit 31, for extracting the first query operator and the second query operator from logical query plan, the first query operator is forerunner's operator of the second query operator.
First rewrites unit 32, if the prefix for the zone attribute that the zone attribute of the first query operator is the second query operator that extraction unit extracts that extraction unit 31 extracts, then rewrite the zone attribute of the second query operator in logical query plan, so that the zone attribute of the zone attribute of the second query operator and the first query operator is identical.
Deleting unit 33, for rewriting in logical query plan described in unit 32 after the zone attribute of the second query operator first, the division operation deleting the second query operator from logical query plan accords with.
Generate unit 34, after division operation for deleting the second query operator at deletion unit 33 accords with, division operation symbol according to the second query operator, the first query operator and the first query operator generates a physical queries task, to constitute physical query plan.
Further, as it is shown in figure 9, described first rewrites unit 32, it is possible to including: first rewrites module 321 and second rewrites module 322.
First rewrites module 321, if the prefix of the zone attribute that zone attribute is the second query operator for the first query operator, and first between query operator and the second query operator between be separated with the 3rd query operator, described 3rd query operator can use broadcast query algorithm to realize, then the 3rd query operator being rewritten as broadcast query operator, broadcast query operator does not have division operation to accord with.
Second rewrites module 322, for rewriting the zone attribute of the second query operator in logical query plan, so that the zone attribute of the zone attribute of the second query operator and the first query operator is identical.
Described generation unit 34, is additionally operable to the division operation symbol according to the second query operator, broadcast query operator, described first query operator and the first query operator and generates a physical queries task, to constitute physical query plan.
Wherein, the first query operator is the direct precursor operator of the 3rd query operator, and the direct precursor operator that the 3rd query operator is the second query operator.
Further, as shown in Figure 10, the inquiry plan reforming unit that the embodiment of the present invention provides, it is also possible to including: second rewrites unit 35.
Second rewrites unit 35, if the zone attribute for the first query operator of extraction unit 31 extraction is identical with the zone attribute of the second query operator that extraction unit 31 extracts, and first between query operator and the second query operator between be separated with the 3rd query operator, described 3rd query operator can use broadcast query algorithm to realize, then the 3rd query operator being rewritten as broadcast query operator, broadcast query operator does not have division operation to accord with.
Described deletion unit 33, is additionally operable to, after the 3rd query operator is rewritten as broadcast query operator by the second rewriting unit 35, delete the division operation symbol of the second query operator from logical query plan.
Described generation unit 34, it is additionally operable to after described deletion unit 33 deletes the division operation symbol of the second query operator, division operation symbol according to the second query operator, broadcast query operator, the first query operator and the first query operator generates a physical queries task, to constitute physical query plan.
Wherein, the first query operator is the direct precursor operator of the 3rd query operator, and the direct precursor operator that the 3rd query operator is the second query operator.
Further, generate unit 34, specifically for adopting task flow relatedness to optimize JFC technology, accord with according to the division operation of the second query operator, the first query operator and the first query operator and generate a physical queries task, to constitute physical query plan.
The inquiry plan reforming unit that the embodiment of the present invention provides, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, just can pass through to rewrite the zone attribute of the second query operator in logical query plan, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, to generate a physical queries task according to the first query operator and the second query operator, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the zone attribute of the zone attribute of the first query operator and the second query operator be identical, the division operation symbol that can delete the second query operator is compared, when the prefix of the zone attribute that zone attribute is the second query operator of the first query operator, rewrite the zone attribute of the second query operator and can obtain the identical query operator meeting forerunner-follow-up relation of more zone attribute to (such as the first query operator and the second query operator), and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
And pass through this programme, when the first query operator is indirect forerunner's operator of the second query operator, namely it is separated with the 3rd query operator between the first query operator and the second query operator, when 3rd query operator can use broadcast query algorithm to realize, 3rd query operator can be rewritten as broadcast query operator, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, by the first query operator, second query operator and broadcast query operator generate a physical queries task, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the direct precursor operator that the first query operator is the second query operator, the division operation symbol of the second query operator can be deleted, make identical first query operator of zone attribute and the second query operator generate a physical queries task to compare, when being separated with three query operators between the first query operator and the second query operator, 3rd query operator is rewritten as broadcast query operator, so that broadcast query operator first query operator identical with zone attribute and the second query operator can generate a physical queries task, and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
Embodiment 4
The embodiment of the present invention provides a kind of inquiry plan reforming unit, as shown in figure 11, including: memorizer 41 and processor 42; Described memorizer 41 is connected with described processor 42.
Described memorizer 41, is used for storing batch processing code, and described memorizer 41 is the computer-readable storage medium of described inquiry plan reforming unit, and described computer-readable storage medium includes: non-volatile memory medium.
Described processor 42, for performing described memorizer 41 storing said program code, and specifically for performing following operation: extracting the first query operator and the second query operator from logical query plan, described first query operator is forerunner's operator of described second query operator; If the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator; The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute physical query plan.
Described memorizer 51 and described processor 52 are connected by bus and complete mutual communicating.
Wherein, described bus can be industry standard architecture (IndustryStandardArchitecture, ISA) bus, external equipment interconnect (PeripheralComponentInterconnect, PCI) bus or extended industry-standard architecture (ExtendedIndustryStandardArchitecture, EISA) bus etc. This bus can be divided into address bus, data/address bus, control bus etc. For ease of representing, Figure 11 only represents with a thick line, it is not intended that only have a bus or a type of bus.
Further, described processor 42, if the prefix of the zone attribute that the zone attribute being additionally operable to described first query operator is described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with; Rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator; The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan.
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
Further, described processor 42, if the zone attribute being additionally operable to described first query operator is identical with the zone attribute of described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with; The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan.
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
Further, described processor 42, it is additionally operable to before the division operation deleting described second query operator from described logical query plan accords with, rewrite the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator.
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting; Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
Further, described processor 42, it is additionally operable to adopt task flow relatedness to optimize JFC technology, accords with according to the division operation of described second query operator, described first query operator and described first query operator and generate a physical queries task, to constitute described physical query plan.
It should be noted that the specific descriptions of part functional module are referred to the corresponding content in embodiment of the method in the inquiry plan reforming unit of embodiment of the present invention offer, the present embodiment is no longer described in detail here.
The inquiry plan reforming unit that the embodiment of the present invention provides, if the prefix of the zone attribute that the zone attribute of the first query operator is the second query operator, just can pass through to rewrite the zone attribute of the second query operator in logical query plan, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, to generate a physical queries task according to the first query operator and the second query operator, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the zone attribute of the zone attribute of the first query operator and the second query operator be identical, the division operation symbol that can delete the second query operator is compared, when the prefix of the zone attribute that zone attribute is the second query operator of the first query operator, rewrite the zone attribute of the second query operator and can obtain the identical query operator meeting forerunner-follow-up relation of more zone attribute to (such as the first query operator and the second query operator), and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
And pass through this programme, when the first query operator is indirect forerunner's operator of the second query operator, namely it is separated with the 3rd query operator between the first query operator and the second query operator, when 3rd query operator can use broadcast query algorithm to realize, 3rd query operator can be rewritten as broadcast query operator, so that can when the zone attribute of the zone attribute of the second query operator and the first query operator is identical, delete the division operation symbol of the second query operator, by the first query operator, second query operator and broadcast query operator generate a physical queries task, reduce the quantity of the physical queries task constituting physical query plan.
With in prior art, only can when the direct precursor operator that the first query operator is the second query operator, the division operation symbol of the second query operator can be deleted, make identical first query operator of zone attribute and the second query operator generate a physical queries task to compare, when being separated with three query operators between the first query operator and the second query operator, 3rd query operator is rewritten as broadcast query operator, so that broadcast query operator first query operator identical with zone attribute and the second query operator can generate a physical queries task, and then the quantity of the physical queries task constituting physical query plan can be reduced greatly.
Through the above description of the embodiments, those skilled in the art is it can be understood that arrive, for convenience and simplicity of description, only it is illustrated with the division of above-mentioned each functional module, in practical application, as desired above-mentioned functions distribution can be completed by different functional modules, be divided into different functional modules by the internal structure of device, to complete all or part of function described above. The specific works process of the system of foregoing description, device and unit, it is possible to reference to the corresponding process in preceding method embodiment, do not repeat them here.
In several embodiments provided herein, it should be understood that disclosed system, apparatus and method, it is possible to realize by another way. Such as, device embodiment described above is merely schematic, such as, the division of described module or unit, being only a kind of logic function to divide, actual can have other dividing mode when realizing, for instance multiple unit or assembly can in conjunction with or be desirably integrated into another system, or some features can ignore, or do not perform. Another point, shown or discussed coupling each other or direct-coupling or communication connection can be through INDIRECT COUPLING or the communication connection of some interfaces, device or unit, it is possible to be electrical, machinery or other form.
The described unit illustrated as separating component can be or may not be physically separate, and the parts shown as unit can be or may not be physical location, namely may be located at a place, or can also be distributed on multiple NE. Some or all of unit therein can be selected according to the actual needs to realize the purpose of the present embodiment scheme.
It addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it is also possible to be that unit is individually physically present, it is also possible to two or more unit are integrated in a unit. Above-mentioned integrated unit both can adopt the form of hardware to realize, it would however also be possible to employ the form of SFU software functional unit realizes.
If described integrated unit is using the form realization of SFU software functional unit and as independent production marketing or use, it is possible to be stored in a computer read/write memory medium. Based on such understanding, part or all or part of of this technical scheme that prior art is contributed by technical scheme substantially in other words can embody with the form of software product, this computer software product is stored in a storage medium, including some instructions with so that a computer equipment (can be personal computer, server, or the network equipment etc.) or processor (processor) perform all or part of step of method described in each embodiment of the present invention. And aforesaid storage medium includes: USB flash disk, portable hard drive, read only memory (ROM, Read-OnlyMemory), the various media that can store program code such as random access memory (RAM, RandomAccessMemory), magnetic disc or CD.
The above; being only the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, any those familiar with the art is in the technical scope that the invention discloses; change can be readily occurred in or replace, all should be encompassed within protection scope of the present invention. Therefore, protection scope of the present invention should be as the criterion with described scope of the claims.

Claims (10)

1. an inquiry plan method for transformation, it is characterised in that including:
Extracting the first query operator and the second query operator from logical query plan, described first query operator is forerunner's operator of described second query operator;
If the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute physical query plan.
2. method according to claim 1, it is characterized in that, if the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator, including:
If the prefix of the zone attribute that the zone attribute of described first query operator is described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
Rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
The described division operation symbol deleting described second query operator from described logical query plan, and the division operation according to described second query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan, including:
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
3. method according to claim 1, it is characterised in that also include:
If the zone attribute of described first query operator is identical with the zone attribute of described second query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
The division operation symbol of described second query operator is deleted from described logical query plan, and the division operation according to described second query operator, described broadcast query operator, described first query operator and described first query operator accords with one physical queries task of generation, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
4. the method according to any one of claim 1-3, it is characterised in that before the described division operation symbol deleting described second query operator from described logical query plan, described method also includes:
Rewrite the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator;
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting;
Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
5. the method according to any one of claim 1-3, it is characterized in that, the described division operation according to described second query operator, described first query operator and described first query operator accords with and generates a physical queries task, to constitute described physical query plan, including:
Adopt task flow relatedness to optimize JFC technology, accord with according to the division operation of described second query operator, described first query operator and described first query operator and generate a physical queries task, to constitute described physical query plan.
6. an inquiry plan reforming unit, it is characterised in that including:
Extraction unit, for extracting the first query operator and the second query operator from logical query plan, described first query operator is forerunner's operator of described second query operator;
First rewrites unit, if the prefix for the zone attribute that the zone attribute of described first query operator is described second query operator that described extraction unit extracts that described extraction unit extracts, then rewrite the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
Delete unit, after rewriting the zone attribute of the second query operator described in the unit described logical query plan of rewriting described first, from described logical query plan, delete the division operation symbol of described second query operator;
Generate unit, for after the division operation of the second query operator described in described deletion element deletion accords with, division operation symbol according to described second query operator, described first query operator and described first query operator generates a physical queries task, to constitute physical query plan.
7. device according to claim 6, it is characterised in that described first rewrites unit, including:
First rewrites module, if the prefix of the zone attribute that zone attribute is described second query operator for described first query operator, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
Second rewrites module, for rewriting the zone attribute of the second query operator described in described logical query plan, so that the zone attribute of described second query operator is identical with the zone attribute of described first query operator;
Described generation unit, it is additionally operable to the division operation symbol according to described second query operator, described broadcast query operator, described first query operator and described first query operator and generates a physical queries task, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
8. device according to claim 6, it is characterised in that also include:
Second rewrites unit, if the zone attribute of described second query operator that the zone attribute of described first query operator extracted for described extraction unit extracts with described extraction unit is identical, and it is separated with the 3rd query operator between described first query operator and described second query operator, described 3rd query operator can use broadcast query algorithm to realize, then described 3rd query operator being rewritten as broadcast query operator, described broadcast query operator does not have division operation to accord with;
Described deletion unit, is additionally operable to, after described 3rd query operator is rewritten as described broadcast query operator by described second rewriting unit, delete the division operation symbol of described second query operator from described logical query plan;
Described generation unit, it is additionally operable to after the division operation of the second query operator described in described deletion element deletion accords with, division operation symbol according to described second query operator, described broadcast query operator, described first query operator and described first query operator generates a physical queries task, to constitute described physical query plan;
Wherein, described first query operator is forerunner's operator of described 3rd query operator, and forerunner's operator that described 3rd query operator is described second query operator.
9. the device according to any one of claim 6-8, it is characterised in that also include:
3rd rewrites unit, before division operation for deleting described second query operator from described logical query plan at described deletion unit accords with, rewrite the ordering attribute of described first query operator, so that the ordering attribute of described first query operator is identical with the ordering attribute of described second query operator;
Wherein, the ordering attribute of described first query operator before rewriting is identical with the zone attribute of the first query operator, and the ordering attribute of described second query operator is identical with the zone attribute of described second query operator before rewriting;
Described ordering attribute is for carrying out subregion sequence to the data in the tables of data operated by the described query operator of described logical query plan.
10. the device according to any one of claim 6-8, it is characterized in that, described generation unit, specifically for adopting task flow relatedness to optimize JFC technology, division operation symbol according to described second query operator, described first query operator and described first query operator generates a physical queries task, to constitute described physical query plan.
CN201410588240.8A 2014-10-28 2014-10-28 A kind of inquiry plan method for transformation and device Expired - Fee Related CN105630789B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410588240.8A CN105630789B (en) 2014-10-28 2014-10-28 A kind of inquiry plan method for transformation and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410588240.8A CN105630789B (en) 2014-10-28 2014-10-28 A kind of inquiry plan method for transformation and device

Publications (2)

Publication Number Publication Date
CN105630789A true CN105630789A (en) 2016-06-01
CN105630789B CN105630789B (en) 2019-07-12

Family

ID=56045743

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410588240.8A Expired - Fee Related CN105630789B (en) 2014-10-28 2014-10-28 A kind of inquiry plan method for transformation and device

Country Status (1)

Country Link
CN (1) CN105630789B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109902101A (en) * 2019-02-18 2019-06-18 国家计算机网络与信息安全管理中心 Transparent partition method and device based on SparkSQL
US20200210427A1 (en) * 2018-12-26 2020-07-02 Palantir Technologies Inc. Column lineage and metadata propagation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101436192A (en) * 2007-11-16 2009-05-20 国际商业机器公司 Method and apparatus for optimizing inquiry aiming at vertical storage type database
US20090144346A1 (en) * 2007-11-29 2009-06-04 Microsoft Corporation Partitioning and repartitioning for data parallel operations
CN102323946A (en) * 2011-09-05 2012-01-18 天津神舟通用数据技术有限公司 Implementation method for operator reuse in parallel database
CN102831139A (en) * 2011-03-25 2012-12-19 微软公司 Co-range partition for query plan optimization and data-parallel programming model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101436192A (en) * 2007-11-16 2009-05-20 国际商业机器公司 Method and apparatus for optimizing inquiry aiming at vertical storage type database
US20090144346A1 (en) * 2007-11-29 2009-06-04 Microsoft Corporation Partitioning and repartitioning for data parallel operations
CN102831139A (en) * 2011-03-25 2012-12-19 微软公司 Co-range partition for query plan optimization and data-parallel programming model
CN102323946A (en) * 2011-09-05 2012-01-18 天津神舟通用数据技术有限公司 Implementation method for operator reuse in parallel database

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200210427A1 (en) * 2018-12-26 2020-07-02 Palantir Technologies Inc. Column lineage and metadata propagation
US11599539B2 (en) * 2018-12-26 2023-03-07 Palantir Technologies Inc. Column lineage and metadata propagation
CN109902101A (en) * 2019-02-18 2019-06-18 国家计算机网络与信息安全管理中心 Transparent partition method and device based on SparkSQL

Also Published As

Publication number Publication date
CN105630789B (en) 2019-07-12

Similar Documents

Publication Publication Date Title
CN102707990B (en) Container based processing method and device
CN105868166B (en) Regular expression generation method and system
KR101617696B1 (en) Method and device for mining data regular expression
CN105095287A (en) LSM (Log Structured Merge) data compact method and device
US20230117105A1 (en) On-demand loading of dynamic scripting language code for reduced memory usage
CN105550225A (en) Index construction method and query method and apparatus
CN105022757A (en) Webpage revision method and webpage revision device
JP7047228B2 (en) Data query methods, devices, electronic devices, readable storage media, and computer programs
CN104881466A (en) Method and device for processing data fragments and deleting garbage files
CN103617226A (en) Regular expression matching method and device
CN104965826A (en) Search method and search apparatus based on a browser
CN102915344B (en) SQL (structured query language) statement processing method and device
CN105760289A (en) Debugging method and device of embedded system
US10482087B2 (en) Storage system and method of operating the same
CN110704139B (en) Icon classification method and device
CN105630789A (en) Query plan converting method and device
CN105701128A (en) Query statement optimization method and apparatus
US20140172897A1 (en) Device, method, and program for processing data with tree structure
CN105790967A (en) Weblog processing method and device
CN103092955B (en) Checkpointed method, Apparatus and system
CN117171296A (en) Information acquisition method and device and electronic equipment
CN109697234B (en) Multi-attribute information query method, device, server and medium for entity
CN104572638A (en) Data reading and writing method and device
CN109918074B (en) Compiling link optimization method
CN104750846A (en) Method and device for finding substring

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20190712

Termination date: 20201028