CN101231614A - Method for locating software unsoundness base on execution track block semblance - Google Patents

Abstract

The invention discloses a software defect locating method which is based on the lump similarity of execution traces, and the method comprises the following steps that: firstly, the executive information of test cases are collected and arranged, and the execution traces are generated; secondly, a test case set which has defects is positioned according to the lump similarity selection of the execution traces; thirdly, the system comparison is carried out to the execution traces of the selected test cases, and the suspecting rate of the operation part is calculated; fourthly, the suspecting rate of the operation part is mapped to the source program to generate a defect locating report. The invention can be widely applied to the automatic test environment of a large-scale software system, and relates to the stage and the work of an integrated test, a system test, an acceptance test, a user problem report analysis, etc.; the existing test data are fully utilized, and the lump similarity of the execution traces is based on to carry out the position to the software defect, thereby effectively reducing the searching range and improving the defect positioning efficiency. The invention can be used in each period and stage of the test, the selected test case set has higher pertinence, the calculation method of the suspecting rate of the operation part is simple and effective, and the expansibility is better.

Description

A kind of based on the software defect positioning method of carrying out the track block similarity

Technical field

The present invention relates to the defect location in the software automated testing, particularly exist under the situation of substantive test use-case, effectively utilize that pass through and unsanctioned test case, it is carried out track compare and come defective in the positioning software.

Background technology

Software defect positioning method is widely used in stage such as integration testing, system testing, Acceptance Test and the customer problem report analysis of large software system and the work at present.Test is a requisite link in the software development process, and its final goal is the quality that guarantees software.Along with software becomes increasingly complex, the use of automation software testing more and more widely.Automatic test can produce a large amount of test cases, and can write down the execution information of test case, and therefore existing a large amount of test datas before repair-deficiency can utilize.Traditional adjustment method is just carried out defective to unsanctioned single test case and is followed the tracks of, under the environment of automatic test, be difficult on the one hand so simultaneously a plurality of unsanctioned test cases be followed the tracks of, only consider unsanctioned test case on the other hand and ignored the information that the test case passed through can provide.Software defect positioning method can fully utilize test execution information and come the positioning software defective under the automatic test environment, improve the quality of software.

Can be based on the software defect positioning method of carrying out track by test data being analyzed the defective that exists in the automatic positioning software.This method need be collected and put in order test case and be carried out information, degree under a cloud (suspicion rate) by Accounting Legend Code that the execution track is compared, the developer can examine code under a cloud according to suspicion rate order from big to small, reduce code quantity and the scope that defective must be examined that remove, improve the efficient of searching defective.Existent method is considered separately to pass through and unsanctioned test case usually, fails fine comprehensive; And only compare, be unsuitable for the defect location of large software system at statement level.

Summary of the invention

Fundamental purpose of the present invention is the problem that makes full use of existing test data at traditional software defect positioning method workload greatly and not, propose a kind of based on the software defect positioning method of carrying out the track block similarity, dwindle the hunting zone of defective in developer's positioning software, improve the efficient of defect location.

For realizing purpose of the present invention, the present invention adopts following step:

1) at first to source program plug-in mounting collected metadata, generate the driving file of each test case, specified configuration information hereof, the execution information of test case is collected in implementation of test cases then, and to test case according to by whether dividing into groups;

2) the execution track of test case is assembled, scale and complexity according to software can arrive method layer or class layer by selective aggregation, calculate the similarity between unsanctioned test case and all the other test cases then respectively, select a plurality of test cases the most similar according to similarity with unsanctioned test case, test case is divided into several groups according to similarity like this, according to the test case in every group the implicit defective of this group is positioned respectively;

3) test case of selecting is carried out system's comparison, the suspicion rate of Accounting Legend Code piece;

4) be mapped to source program according to metadata at last and generate the defect location report.

Above-mentioned steps 2) similarity in is with the included angle cosine value representation between two vectors, and whether test case is selected is determined by threshold alpha, for test case t _FiIf, use-case t _jWith its similarity greater than α, test case t then _jSelected and test case t _FiOne group.

Above-mentioned steps 2) test case is divided into 3 kinds of situations according to 3 kinds of different grouping types of similarity, wherein:

1,1 unsanctioned test case of situation and the test case of passing through more than 1 and 1;

Situation is unsanctioned test case and 0 test case of passing through more than 2,1;

Situation is unsanctioned test case and the test case passed through more than 1 and 1 more than 3,1.

Suspicion rate calculation process by the test case of situation 1 grouping is: at first at unsanctioned test case t _Fi, same t _FiThe similar test use cases TpList that passes through _Fi, statistics TpList _FiIn the coverage condition of test case, then according to code block s whether by t _FiCovering checks that it is by TpList _FiIn each test case t _PkThe ratio that covers.If code block s is by t _FiCover.Its suspicion rate is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}$

Wherein NTp represents TpList _FiThe quantity of middle test case, NTp ₀(s) expression TpList _FiIn do not cover the test case quantity of s;

If code block s is not by t _FiCover, its suspicion rate sus (s) is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}$

NTp wherein ₁(s) expression TpList _FiThe middle test case quantity that covers s.

Suspicion rate calculation process by the test case of situation 2 grouping is: consider same t _FiSimilar unsanctioned test use cases TfList _Fi, to code block s statistics TfList _FiIn covered the test case number NTf of s ₁(s), the suspicion rate sus (s) of code block s is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}$

Wherein NTf represents TfList _FiThe number of middle test case.

Suspicion rate calculation process by the test case of situation 3 grouping is: if code block s is by TfList _FiIn each test case all covered, then in the calculating of s rate under a cloud and the situation 1 s by t _FiThe calculating that covers is identical.If code block s is not by TfList _FiIn any one test case cover, then in the calculating of s rate under a cloud and the situation 1 s not by t _FiThe calculating that covers is identical.If s is by TfList _FiIn the partial test use-case cover, then calculate the suspicion rate sus of s respectively by following two formula ₁(s) and sus ₂(s), NTf wherein ₀(s) be TfList _FiIn do not cover the test case number of s, other variable-definitions are the same, get two the greater in the value then, i.e. sus (s)=Max (sus1 (s), sus2 (s)); Formula is as follows:

${\mathrm{sus}}_1\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}$

${\mathrm{sus}}_2\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_0\left(s\right)}{\mathrm{NTf}}$

The present invention can make full use of existing test data, based on carrying out the track block similarity software defect is positioned, and can effectively dwindle the hunting zone, improves the efficient of defect location.The present invention can be widely used in the automatic test environment of large software system, relates to stage and work such as integration testing, system testing, Acceptance Test and customer problem report analysis.With traditional method relatively, at given defective (or inefficacy), the this patent method can be found out similar passing through and unsanctioned test use cases fast and accurately, and the execution track of each test case is carried out system's comparison, and may there be the code range of defective in the location automatically.Its advantage comprises: each period and the stage that can be used for testing; Make the set of uses case of selection have higher specific aim by the calculating of piece similarity, effectively dwindle the defective seek scope; Has extendability preferably, by aggregation algorithms being improved and replaced test applicable to dissimilar systems; The suspicion rate computing method are simple, be convenient to implement with lower time and space cost, and validity are better than existing defect positioning method through the example proof.

Be elaborated below in conjunction with accompanying drawing.

Description of drawings

Fig. 1 is based on the structural drawing of the software defect positioning method of carrying out the track block similarity;

Fig. 2 is that test case is carried out the track organization chart;

Fig. 3 is based on the workflow diagram of the software defect positioning method of carrying out the track block similarity;

Fig. 4 is the process flow diagram of test case selection algorithm;

Fig. 5 is the algorithm flow chart of the coverage condition of statistical test use-case;

Fig. 6 is the suspicion rate calculation flow chart of situation 1;

Fig. 7 is the suspicion rate calculation flow chart of situation 2;

Fig. 8 is the suspicion rate calculation flow chart of situation 3;

Fig. 9 is based on the instrument master interface of the software defect positioning method of carrying out the track block similarity.

Embodiment

As shown in Figure 1, software defect location by detecting information compile, test case is selected, suspicion rate calculates and generate the architecture tissue of four modules of defect location report according to pipeline or stream.The detecting information collection module is collected and the arrangement test case carries out information and track is carried out in generation; Test case selects module to select to be used to locate the test use cases of defective according to the execution track block similarity of test case.The suspicion rate computing module is compared to the execution track of selected test case, the suspicion rate of Accounting Legend Code piece.The generation reporting modules is mapped to the report of source program generation defect location according to the suspicion rate of code block.Fig. 2 is that test case is carried out the track organization chart.

General flow chart of the present invention as shown in Figure 3, in the detecting information collection module, the 1.1st step was at first carried out plug-in mounting to source program, collect the source program metadata, promptly about the data of source program structure: comprise which class in the source program, which method is each class comprise, each method which code block is made of etc.The execution that the 1.2nd step generated each test case drives file, represents with XML document, and some of collecting comprising detecting information are provided with as execution environment, carry out information storing path etc.The 1.3rd step carry out to drive file, and the execution track of test case, information such as whether pass through are kept under the designated directory.The execution track of test case is organized according to the level of class, method, code block as shown in Figure 2.Wherein code block refers under any input, the code collection that implementation status is all identical.A class comprises a plurality of methods, and a method comprises a plurality of code blocks.Whether the 1.4th go on foot according to test case by test use cases T being divided into test use cases Tp and the unsanctioned test use cases Tf that passes through.For example existing test use cases T={t ₁... t ₁₅, carry out back test case t ₃, t ₅, t ₈, t ₉, t ₁₀Do not pass through, remaining has all passed through, so Tf={t ₃, t ₅, t ₈, t ₉, t ₁₀, Tp={t ₁, t ₂, t ₄, t ₆, t ₇, t ₁₁, t ₁₂, t ₁₃, t ₁₄, t ₁₅.

Test case selects module to select to be used for the test case of defect location, and this part is that the present invention is peculiar.The 2.1st step was assembled the execution track of test case, different according to scale and complexity, can selective aggregation to method layer or class layer.Fig. 5 is the coverage condition of statistical test use-case and the algorithm flow chart that the execution track is assembled.If gather the method layer, then total code block is counted totalNum (m) in the number coverNum (m) of the code block that at first tested use-case covers in the statistical method and the method, then by formula the ratio cover (m) that is capped of (1) computing method; If gather the class layer, then at first add up in the number coverNum (c) of the code block that tested use-case covers in the class and the class total code block and count totalNum (c), then by formula the ratio cover (c) that is capped of (2) compute classes.The execution track of assembling each test case of back is expressed as a many-valued vector

$\mathrm{cover}\left(m\right)=\frac{\mathrm{coverNum}\left(m\right)}{\mathrm{totalNum}\left(m\right)}---\left(1\right)$

$\mathrm{cover}\left(c\right)=\frac{\mathrm{coverNum}\left(c\right)}{\mathrm{totalNum}\left(c\right)}---\left(2\right)$

The 2.2nd step computing block similarity, at certain unsanctioned test case with calculate it with the similarity between all the other test cases, and according to threshold value select with this by the similar test case of test case.The test case selection algorithm is as shown in Figure 4: the 2.2.1 step judges whether unsanctioned test case is arranged among the Tf, if for sky then finish, otherwise from Tf certain unsanctioned test case of selection

Put into TfList _FiIn and with t _FiFrom Tf, remove.The 2.2.2 step is taken out remaining unsanctioned test case t among the Tf respectively _Fi, calculate t _FiAnd t _FiSimilarity sim (t _Fi, t _Fi).If sim is (t _Fi, t _Fi)＞α (α is a threshold value) is then t _FiPut into TfList _FiIn, and with t _FiFrom Tf, remove.The 2.2.3 step is taken out each the test case t that passes through among the Tp respectively _Pk, calculate t _FiWith t _PkBetween similarity sim (t _Fi, t _Pk), if sim is (t _Fi, t _Pk)＞α is then t _PkPut into TpList _FiIn.2.2.4 goes on foot output＜TfList _Fi, TpList _FiAnd jump to the 2.2.1 step.Similarity is calculated by formula (3).This formula calculates under polar coordinates the cosine value of angle between two vectors, and two vectors of the big more expression of included angle cosine value are similar more.Wherein vectorial

Be respectively test case t _i, t _jCarry out the vector representation of track. With

Expression is vectorial respectively Mould,

The expression vector

Scalar product.

$\mathrm{sim}(t_i,t_j)=\cos ({\stackrel{\→}{t}}_i,{\stackrel{\→}{t}}_j)=\frac{{\stackrel{\→}{t}}_i\·{\stackrel{\→}{t}}_j}{\left|{\stackrel{\→}{t}}_i\right|\×\left|{\stackrel{\→}{t}}_j\right|}=\frac{\underset{k=0}{\overset{n}{\mathrm{\Σ}}}(t_{\mathrm{ik}}\×t_{\mathrm{jk}})}{\sqrt{\underset{k=0}{\overset{n}{\mathrm{\Σ}}}{t_{\mathrm{ik}}}^2\×\underset{k=0}{\overset{n}{\mathrm{\Σ}}}{t_{\mathrm{jk}}}^2}}---\left(3\right)$

For example calculate according to similarity, the possible outcome that test use cases is selected in the last example is as follows:

＜TfList _F3, TpList _F3, TfList wherein _F3={ t ₃, t ₅, TpList _F3={ t ₁, t ₂, t ₄, t ₆, t ₇;

＜TfList _F8, TpList _F8, TfList wherein _F8={ t ₈, TpList _F8={ t ₁₁, t ₁₂, t ₁₅;

＜TfList _F9, TpList _F9, TfList wherein _F9={ t ₉, t ₁₀, TpList _F9=Φ;

The 3rd step suspected to the suspicion rate of the test case systematic calculation code block of selection that considering calculating section also was that the present invention is peculiar.The main thought of the computing method that the present invention adopts be judge unsanctioned test case and the test case passed through between the consistance that covers, if unsanctioned test case t _FiCovered code block s, but the similar test case of passing through does not all have covering, s causes t so _FiUnsanctioned reason has higher suspicion rate.On the other hand, if t _FiDo not cover code block s, but the similar test case of passing through covered all, code block s is likely the protection code, but t _FiDo not cause it not pass through thereby carry out, have higher suspicion rate equally.The result who selects according to the 2nd step can distinguish three kinds of situations:

1,1 unsanctioned test case of situation and a plurality of test case of passing through (comprising 1);

Situation 2, a plurality of unsanctioned test cases (greater than 1) and 0 test case of passing through;

Situation 3, a plurality of unsanctioned test cases (greater than 1) and a plurality of test case of passing through (comprising 1).Introduce the computation model of three kinds of situations below in detail.

1, one unsanctioned test case t of situation _FiWith a plurality of test case TpList that pass through _FiAccording to code block s whether by t _FiCover and calculate it by TpList _FiIn each test case t _PkThe ratio that covers: if code block s is by t _FiCover, then add up TpList _FiIn do not cover the number NTp of the test case of s ₀(s), calculate the suspicion rate sus (s) of s by formula (4).If code block s is not by t _FiCover, then add up TpList _FiIn covered the number NTp of the test case of s ₁(s), calculate the suspicion rate sus (s) of s by formula (5).Wherein the NTp in formula (4) and (5) represents TpList _FiThe quantity of middle test case.Computation process can be divided into for two steps and finishes as shown in Figure 6: 3.1.1 step statistics TpList _FiIn the coverage condition of test case, 3.1.2 is according to t _FiCoverage condition calculate the suspicion rate of each code block.

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}---\left(4\right)$

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}---\left(5\right)$

Situation 2, a plurality of unsanctioned test case TfList _FiBut the similar test case of not passing through.The Accounting Legend Code piece is by the ratio of these unsanctioned test cases coverings in such cases, and the high more suspicion rate of ratio is big more.To code block s statistics TfList _FiIn covered the number NTf of the test case of s ₁(s), by the suspicion rate sus (s) of formula (6) calculating s, wherein NTf is TfList _FiThe number of middle test case.As shown in Figure 7, computation process can be finished in two steps, 3.2.1 step, statistics TfList _FiThe coverage condition of middle test case, 3.2.2 step, the suspicion rate of Accounting Legend Code piece.

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}---\left(6\right)$

Situation 3, a plurality of unsanctioned test case TfList _FiWith a plurality of test case TpList that pass through _FiThis kind situation is the comprehensive of situation 1 and situation 2, if code block s is by TfList _FiIn each test case all covered, then calculate the suspicion rate of s by formula (4) in the situation 1.If s is not by TfList _FiIn any one test case cover, then calculate the suspicion rate of s, if s is by TfList by the formula (5) in the situation 1 _FiIn the partial test use-case cover the suspicion rate sus by formula (7) and formula (8) calculating s respectively then ₁(s) and sus ₂(s), get two the greater between value then, i.e. sus (s)=Max (sus ₁(s), sus ₂(s)), wherein NTf is TfList _FiThe number of middle test case, NTp is TpList _FiThe number of middle test case, NTf ₀(s), NTf ₁(s) NTp of the same joint ₀(s), NTp ₁(s) definition is identical.As shown in Figure 8, computation process can be divided into for 3 steps, 3.3.1 step statistics TfList _FiThe coverage condition of middle test case, concrete operations are identical with the 3.3.1 step in the situation 2,3.3.2 step statistics TpList _FiIn the coverage condition of test case, concrete operations are identical with the 3.1.1 step in the situation 1,3.3.3 step Accounting Legend Code suspicion rate.

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}---\left(7\right)$

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_0\left(s\right)}{\mathrm{NTf}}---\left(8\right)$

The 4th step metadata of collecting according to the 1st step is mapped to source program with the suspicion rate of code block, generates the defect location report of html format.

In sum, the selection of test case and suspicion rate computation model are cores of the present invention. Fig. 9 uses The software defect orientation tool interface of this patented method. Realize based on Java language and XML technology. The Java language Speech has complete object-oriented, the portable characteristics such as strong, by corresponding Java Virtual Machine is installed, and can Guarantee that the method may operate on the operating system platform of present main flow. The XML technology then has extensibility , the characteristics such as flexibility, self descriptiveness. The user can make amendment to realize to the XML document of describing operation Different detecting information collecting functions.

Claims (6)

1. one kind based on the software defect positioning method of carrying out the track block similarity, it is characterized in that may further comprise the steps:

1) at first to source program plug-in mounting collected metadata, generate the driving file of each test case, specified configuration information hereof, the execution information of test case is collected in implementation of test cases then, and to test case according to by whether dividing into groups;

2) the execution track of test case is assembled, scale and complexity selective aggregation according to software arrive method layer or class layer, calculate the similarity between unsanctioned test case and all the other test cases then respectively, select a plurality of test cases the most similar according to similarity with unsanctioned test case, test case is divided into several groups according to similarity, according to the test case in every group the implicit defective of this group is positioned respectively;

3) test case of selecting is carried out system's comparison, the suspicion rate of Accounting Legend Code piece;

4) be mapped to source program according to metadata at last and generate the defect location report.

2. according to claim 1 based on the software defect positioning method of carrying out the track block similarity, it is characterized in that step 2) in similarity with the included angle cosine value representation between two vectors, whether test case selected definite by threshold alpha, for test case t _FiIf, use-case t _jWith its similarity greater than α, test case t then _jSelected and test case t _FiOne group.

3. according to claim 1 and 2 based on the software defect positioning method of carrying out the track block similarity, it is characterized in that step 2) test case is divided into 3 kinds of situations according to 3 kinds of different grouping types of similarity, wherein:

1,1 unsanctioned test case of situation and the test case of passing through more than 1 and 1;

Situation is unsanctioned test case and 0 test case of passing through more than 2,1;

Situation is unsanctioned test case and the test case passed through more than 1 and 1 more than 3,1.

4. according to claim 3 based on the software defect positioning method of carrying out the track block similarity, it is characterized in that in the step 3) by the suspicion rate calculation process of the test case of situation 1 grouping being: at first at unsanctioned test case t _Fi, same t _FiThe similar test use cases TpList that passes through _Fi, statistics TpList _FiIn the coverage condition of test case, then according to code block s whether by t _FiCovering checks that it is by TpList _FiIn each test case t _PkThe ratio that covers is if code block s is by t _FiCover, its suspicion rate sus (s) is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}$

Wherein NTp represents TpList _FiThe quantity of middle test case, NTp ₀(s) expression TpList _FiIn do not cover the test case quantity of s;

If code block s is not by t _FiCover, its suspicion rate sus (s) is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}$

NTp wherein ₁(s) expression TpList _FiThe middle test case quantity that covers s.

5. according to claim 3 based on the software defect positioning method of carrying out the track block similarity, it is characterized in that in the step 3) by the suspicion rate calculation process of the test case of situation 2 groupings being: consider same t _FiSimilar unsanctioned test use cases TfList _Fi, to code block s statistics TfList _FiIn covered the test case number NTf of s ₁(s), the suspicion rate sus (s) of code block s is tried to achieve by following formula:

$\mathrm{sus}\left(s\right)=\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}$

Wherein NTf represents TfList _FiThe number of middle test case.

6. according to claim 3 based on the software defect positioning method of carrying out the track block similarity, it is characterized in that in the step 3) by the suspicion rate calculation process of the test case of situation 3 groupings being: if code block s is by TfList _FiIn each test case all covered, then in the calculating of s rate under a cloud and the situation 1 s by t _FiThe calculating that covers is identical; If code block s is not by TfList _FiIn any one test case cover, then in the calculating of s rate under a cloud and the situation 1 s not by t _FiThe calculating that covers is identical; If s is by TfList _FiIn the partial test use-case cover, then calculate the suspicion rate sus of s respectively by following two formula ₁(s) and sus ₂(s), NTf wherein ₀(s) be TfList _FiIn do not cover the test case number of s, other variable-definitions are the same, get two the greater in the value then, i.e. sus (s)=Max (sus1 (s), sus2 (s)); Formula is as follows:

${\mathrm{sus}}_1\left(s\right)=\frac{{\mathrm{NTp}}_0\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_1\left(s\right)}{\mathrm{NTf}}$

${\mathrm{sus}}_2\left(s\right)=\frac{{\mathrm{NTp}}_1\left(s\right)}{\mathrm{NTp}}\×\frac{{\mathrm{NTf}}_0\left(s\right)}{\mathrm{NTf}}.$

Images