CN105224460A - Method for counting test coverage rate based on plug-in technology - Google Patents

Method for counting test coverage rate based on plug-in technology Download PDF

Info

Publication number
CN105224460A
CN105224460A CN201510694988.0A CN201510694988A CN105224460A CN 105224460 A CN105224460 A CN 105224460A CN 201510694988 A CN201510694988 A CN 201510694988A CN 105224460 A CN105224460 A CN 105224460A
Authority
CN
China
Prior art keywords
test
coverage rate
coverage
program
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510694988.0A
Other languages
Chinese (zh)
Inventor
赵霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Electronic Information Industry Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN201510694988.0A priority Critical patent/CN105224460A/en
Publication of CN105224460A publication Critical patent/CN105224460A/en
Pending legal-status Critical Current

Links

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a method for counting test coverage rate based on an insertion technology, which inserts a plurality of probes into a program on the basis of ensuring the original logic integrity of the tested program, obtains control flow and data flow information of the program by executing the probes and throwing out characteristic data of program operation and analyzing the data, and further obtains dynamic information test coverage rate so as to realize the test purpose; the test coverage rate adopts a layer coverage rate and is divided into three layers according to program granularity: the method comprises the following steps of api coverage, function coverage and code conditional branch coverage, wherein the coverage of three layers is related from coarse to fine. According to the invention, the influence of excessive centralized calling on data is reduced to a certain extent by calculating the test coverage of three levels and combining the three coverage; according to the project test stage and the coverage rate data, the validity of the test is judged reasonably and flexibly, mutual deniability and invisibility facts are prevented, and the scientificity of project management is improved.

Description

A kind of method based on Program instrumentation statistical test coverage rate
Technical field
The present invention relates to software test and project management techniques field, be specifically related to a kind of method based on Program instrumentation statistical test coverage rate, mainly for quality in software development process repeatedly, poor controllability, the problems such as accountability difficulty, by inserting detective pole in program code, monitoring this code and whether being performed in certain test, statistical test coverage rate, reduce randomness, the randomness of test, improve the science of project management, thus shorten the project testing time, reduce testing cost.
Background technology
When being engaged in software project management and test and management, we can find in project in the face of concluding a research item time pressure, now software quality repeatedly, the bug degree of convergence not problem such as expected, the often greatest problem that faces of project concluding a research item.The way that project manager takes usually first is that accountability is followed the trail of in bug classification: classified by this bug, new discovery, amendment cause, unmodified etc., to investigate the responsibility of tester and developer, expect that developer revises bug quality and tester tests comprehensive raising, second is that overtime work is overworked.QA Manager, in order to reduce the quantity of new discovery bug, often adopts the way of stuck-module tester.
Undeniable, these ways can obtain certain effect.But the statistic processes of bug classification, lack science, the standard of system, classify for previous test impression according to exploitation and test, this often causes exploitation and tests the phenomenon of shift onto each other.
Sum up the problem being at all test coverage of these contradictions, how to ensure just comprehensively to cover at the test initial stage? this comprehensive for program test of test how is added up after a test as managerial personnel?
At present, software quality tight at project duration repeatedly, the bug degree of convergence as expected in situation, often do not lack science and reliability when project manager sums up reason, lacks data foundation and cogency to this conclusion project other staff.
Summary of the invention
The technical problem to be solved in the present invention is: based on above-mentioned technology, the invention provides a kind of method based on Program instrumentation statistical test coverage rate.
The technical solution adopted in the present invention is:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
Described code condition branch covering rate, to branch into unit in function, inserts probe when each branch performs, and detecting probe information comprises branch's numbering, in coding stage, and record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether judge that this takes turns test is effective test, for the relation of coverage rate and Validity Test, different phase according to test specifically defines, and the phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Described system testing needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Beneficial effect of the present invention is:
The present invention, by the technology of plug-in mounting in code, records test trails; Probe after taking turns test by one exports, and calculate the test coverage of three levels, three kinds of coverage rates combine, and to a certain degree minimizing concentrations calls the impact on data; According to project testing stage and coverage data, reasonable, judge flexibly to prevent from the validity of testing shift onto each other, ignoring the fact, improve the science of project management.
Accompanying drawing explanation
Fig. 1 calculates test coverage method figure for the present invention is based on Program instrumentation.
Embodiment
Below according to Figure of description, in conjunction with embodiment, the present invention is further described:
Embodiment 1:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Embodiment 2:
On the basis of embodiment 1, api coverage rate described in the present embodiment, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Embodiment 3:
On the basis of embodiment 1, function coverage rate described in the present embodiment, in units of function, before each function starts to perform, insert probe, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, and initial value is 0), when program detailed design, record number is the invoked number of times of function of i, counts total A (i), function coverage rate=X(i)/A (i).
Embodiment 4:
On the basis of embodiment 1, code condition branch covering rate described in the present embodiment, to branch into unit in function, insert probe when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Embodiment 5:
On the basis of embodiment 1-4, method described in the present embodiment is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Embodiment 6:
As shown in Figure 1, on the basis of embodiment 5, described in the present embodiment, method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Embodiment 7:
On the basis of embodiment 6, system testing described in the present embodiment needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Above embodiment is only for illustration of the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all equivalent technical schemes also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.

Claims (7)

1. the method based on Program instrumentation statistical test coverage rate, it is characterized in that: described method inserts some probes in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
2. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, performs degree variables X(i), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered depending on this interface and reject the calculating of i interface in total api coverage rate
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)].
3. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
4. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described code condition branch covering rate, to branch into unit in function, probe is inserted when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
5. according to the arbitrary a kind of described method based on Program instrumentation statistical test coverage rate of claim 1-4, it is characterized in that: described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
6. a kind of method based on Program instrumentation statistical test coverage rate according to claim 5, is characterized in that, described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and branch covering rate respectively, the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, whether test is effectively judged.
7. a kind of method based on Program instrumentation statistical test coverage rate according to claim 6, it is characterized in that: described system testing lasting 5 is taken turns above, test coverage relatively between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
CN201510694988.0A 2015-10-23 2015-10-23 Method for counting test coverage rate based on plug-in technology Pending CN105224460A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510694988.0A CN105224460A (en) 2015-10-23 2015-10-23 Method for counting test coverage rate based on plug-in technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510694988.0A CN105224460A (en) 2015-10-23 2015-10-23 Method for counting test coverage rate based on plug-in technology

Publications (1)

Publication Number Publication Date
CN105224460A true CN105224460A (en) 2016-01-06

Family

ID=54993445

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510694988.0A Pending CN105224460A (en) 2015-10-23 2015-10-23 Method for counting test coverage rate based on plug-in technology

Country Status (1)

Country Link
CN (1) CN105224460A (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106294163A (en) * 2016-08-12 2017-01-04 北京航空航天大学 A kind of code branch covering rate detection method of lightweight
CN109542789A (en) * 2018-11-26 2019-03-29 泰康保险集团股份有限公司 A kind of code coverage statistical method and device
WO2019071929A1 (en) * 2017-10-09 2019-04-18 武汉斗鱼网络科技有限公司 Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device
CN110580214A (en) * 2018-06-07 2019-12-17 百度在线网络技术(北京)有限公司 application program test evaluation method and device, storage medium and terminal equipment
CN112597064A (en) * 2021-03-02 2021-04-02 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN112597007A (en) * 2020-12-14 2021-04-02 中国航发控制***研究所 Embedded software integration test integrity analysis method
CN113094289A (en) * 2021-05-18 2021-07-09 绿漫科技有限公司 Method for realizing interface coverage rate statistics through python language
US11086759B2 (en) 2018-09-27 2021-08-10 SeaLights Technologies LTD System and method for probe injection for code coverage
US11093374B2 (en) 2016-08-09 2021-08-17 SeaLights Technologies LTD System and method for continuous testing and delivery of software
CN113392033A (en) * 2021-08-17 2021-09-14 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate
CN113535559A (en) * 2021-07-14 2021-10-22 杭州默安科技有限公司 Application interface coverage rate statistical method and system
US11573885B1 (en) 2019-09-26 2023-02-07 SeaLights Technologies LTD System and method for test selection according to test impact analytics

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7334161B2 (en) * 2004-04-30 2008-02-19 Arm Limited Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus
CN102103538A (en) * 2011-02-22 2011-06-22 南京航空航天大学 Method for testing palletizing robot control software based on Agent
CN102419728A (en) * 2011-11-01 2012-04-18 北京邮电大学 Method for determining software test process sufficiency based on coverage rate quantitative indicators

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7334161B2 (en) * 2004-04-30 2008-02-19 Arm Limited Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus
CN102103538A (en) * 2011-02-22 2011-06-22 南京航空航天大学 Method for testing palletizing robot control software based on Agent
CN102419728A (en) * 2011-11-01 2012-04-18 北京邮电大学 Method for determining software test process sufficiency based on coverage rate quantitative indicators

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11093374B2 (en) 2016-08-09 2021-08-17 SeaLights Technologies LTD System and method for continuous testing and delivery of software
US11775416B2 (en) 2016-08-09 2023-10-03 SeaLights Technologies LTD System and method for continuous testing and delivery of software
CN106294163A (en) * 2016-08-12 2017-01-04 北京航空航天大学 A kind of code branch covering rate detection method of lightweight
WO2019071929A1 (en) * 2017-10-09 2019-04-18 武汉斗鱼网络科技有限公司 Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device
CN110580214A (en) * 2018-06-07 2019-12-17 百度在线网络技术(北京)有限公司 application program test evaluation method and device, storage medium and terminal equipment
US11847041B2 (en) 2018-09-27 2023-12-19 Sealights Technologies Ltd. System and method for probe injection for code coverage
US11086759B2 (en) 2018-09-27 2021-08-10 SeaLights Technologies LTD System and method for probe injection for code coverage
CN109542789B (en) * 2018-11-26 2022-03-25 泰康保险集团股份有限公司 Code coverage rate statistical method and device
CN109542789A (en) * 2018-11-26 2019-03-29 泰康保险集团股份有限公司 A kind of code coverage statistical method and device
US11573885B1 (en) 2019-09-26 2023-02-07 SeaLights Technologies LTD System and method for test selection according to test impact analytics
CN112597007A (en) * 2020-12-14 2021-04-02 中国航发控制***研究所 Embedded software integration test integrity analysis method
CN112597007B (en) * 2020-12-14 2023-10-03 中国航发控制***研究所 Integrated test integrity analysis method for embedded software
CN112597064A (en) * 2021-03-02 2021-04-02 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN112597064B (en) * 2021-03-02 2021-06-29 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN113094289A (en) * 2021-05-18 2021-07-09 绿漫科技有限公司 Method for realizing interface coverage rate statistics through python language
CN113535559A (en) * 2021-07-14 2021-10-22 杭州默安科技有限公司 Application interface coverage rate statistical method and system
CN113392033B (en) * 2021-08-17 2022-03-08 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate
CN113392033A (en) * 2021-08-17 2021-09-14 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate

Similar Documents

Publication Publication Date Title
CN105224460A (en) Method for counting test coverage rate based on plug-in technology
EP1899876B1 (en) System and method for using model analysis to generate directed test vectors
CN107301119B (en) Method and device for analyzing IT fault root cause by utilizing time sequence correlation
US5655074A (en) Method and system for conducting statistical quality analysis of a complex system
US5500941A (en) Optimum functional test method to determine the quality of a software system embedded in a large electronic system
Tahat et al. Regression test suite prioritization using system models
CN107193732B (en) Verification function positioning method based on path comparison
CN101697144A (en) Software test method and software test device
CN102445941B (en) Method for automatically determining and analyzing interoperability test results of on-board equipment of train control system
US9891281B1 (en) Method and system for automatically identifying test runs contributing to coverage events of interest in verification test data
CN102999417B (en) Automatic test management system and method
CN109858097A (en) A kind of spacecraft single machine test assessment methods of sampling
CN110059010A (en) The buffer overflow detection method with fuzz testing is executed based on dynamic symbol
CN113127314B (en) Method and device for detecting program performance bottleneck and computer equipment
CN109885500A (en) A kind of method and system improving software test coverage rate
CN115576738A (en) Method and system for realizing equipment fault determination based on chip analysis
Khalilian et al. Bi-criteria test suite reduction by cluster analysis of execution profiles
CN111522725A (en) SSD performance automatic evaluation method, device, equipment and medium
CN106598763A (en) Method for testing BI report
CN116136950B (en) Chip verification method, device, system, electronic equipment and storage medium
CN113079061A (en) Internet of things performance testing method and system
CN112416733B (en) Memory detection method, memory detection device and intelligent device
CN112506802A (en) Test data management method and system
Savor et al. Toward automatic detection of software failures
CN115129706B (en) Soil moisture observation data quality evaluation method considering periodic characteristics

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20160106