US20180081792A1 - Code coverage through overlay hooks - Google Patents
Code coverage through overlay hooks Download PDFInfo
- Publication number
- US20180081792A1 US20180081792A1 US15/671,392 US201715671392A US2018081792A1 US 20180081792 A1 US20180081792 A1 US 20180081792A1 US 201715671392 A US201715671392 A US 201715671392A US 2018081792 A1 US2018081792 A1 US 2018081792A1
- Authority
- US
- United States
- Prior art keywords
- condition
- program
- overlay
- overlay hook
- program code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3624—Software debugging by performing operations on the source code, e.g. via a compiler
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3644—Software debugging by instrumenting at runtime
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Definitions
- the present invention relates generally to the field of code coverage, and more particularly to runtime-directed overlay hooks being used to track code coverage.
- Code coverage is a measure used to describe the degree to which the program code of a program is tested by a particular test suite. A program with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a program with low code coverage. To measure what percentage of code has been exercised by a test suite, one or more coverage criteria are used: function coverage, statement coverage, branch coverage, and condition coverage.
- Function coverage requires that each function in the program be called.
- Statement coverage requires each statement in the program to be executed.
- Branch coverage requires each branch of each control structure to be executed.
- Condition coverage requires each Boolean sub-expression to be evaluated both to true and false.
- a combination of function coverage and branch coverage is sometimes also called decision coverage. This criterion requires that every point of entry and exit in the program have been invoked at least once, and every decision in the program have taken on all possible outcomes at least once.
- Modified condition/decision coverage MC/DC requires that both decision and condition coverage be satisfied and that each condition should affect the decision outcome independently.
- aspects of an embodiment of the present invention disclose a method for utilizing overlay hooks to increase code coverage.
- a processor inserts an overlay hook in program code at a location within the program code corresponding to a condition statement.
- a processor executes the program code.
- Upon reaching the overlay hook a processor branches to a set of instructions operative to document an outcome of the condition statement.
- a processor executes the condition statement.
- a processor records the outcome of the condition statement.
- FIG. 1 is a functional block diagram illustrating a code testing environment, in accordance with an embodiment of the present invention
- FIG. 2 is a flowchart depicting operational steps of an overlay hook program, on a computing device within the environment of FIG. 1 , in accordance with an embodiment of the present invention.
- FIG. 3 depicts a block diagram of components of the computing device executing the overlay hook program, in accordance with an embodiment of the present invention.
- Embodiments of the present invention recognize the importance of achieving high code coverage when testing the program code of a program.
- MC/DC modified condition/decision coverage
- Embodiments of the present invention provide solutions for this inability to determine whether each condition has actually been tested. In this manner, as discussed in greater detail herein, embodiments of the present invention can provide finer-grained coverage of conditional paths through code by tracking all possible condition codes through the use of overlay hooks.
- FIG. 1 depicts a diagram of code testing environment 10 , in accordance with an embodiment of the present invention.
- FIG. 1 provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented.
- code testing environment 10 includes computing device 100 .
- Code testing environment 10 may include additional computing devices, servers, computers, mobile devices, or other devices not shown.
- Computing device 100 may be a management server, a web server, laptop computer, tablet computer, netbook computer, personal computer (PC), a desktop computer, a smart phone, or any programmable electronic device or computing system capable of running a program and receiving and sending data.
- computing device 100 may represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment.
- computing device 100 contains overlay hook program 110 , condition table 120 , and debug information 130 .
- Computing device 100 may include components, as depicted and described in further detail with respect to FIG. 3 .
- Overlay hook program 110 operates as a runtime debugger that executes overlay hooks inserted in the program code to track code coverage by recording each condition outcome as the condition outcome is set.
- An overlay hook is a segment of code that places a hook into a program to transfer control to a debugger and overlays the current instruction with a new instruction.
- overlay hook program 110 creates a location that can be branched to, and at that location, overlay hook program 110 saves the condition statement that is overlaid by the overlay hook.
- the overlay hook lines of code are inserted directly on top of where the condition statement was in the program code. For example, overlay hook program 110 would replace a condition statement instruction “Compare A & B” with “Branch to comparison test,” which is the overlay hook.
- the “comparison test” has instructions to execute the condition, record the condition outcome, check if all condition outcomes have been recorded, and return to the program code to continue executing. It should be appreciated that the actual overlay hook sequence will vary by operating system, compiled language, hardware instruction set, and assembler.
- overlay hook program 110 runs alongside a debugger executing the program code.
- overlay hook program 110 initiates running alongside the debugger.
- overlay hook program 110 executes the overlay hook in the program code.
- Overlay hook program 110 determines whether to remove the overlay hook by checking condition table 120 to see if each possible condition outcome has executed. If each possible condition outcome has executed, overlay hook program 110 removes the overlay hook, reinserts the original condition statement, and continues executing the program code. If each possible condition outcome has not executed, overlay hook program 110 leaves the overlay hook within the program code and continues executing the program code.
- overlay hook program 110 When the program code has been executed completely, overlay hook program 110 generates a report from condition table 120 .
- Overlay hook program 110 has access to condition table 120 and updates condition table 120 with table entries of the location and outcome of a condition statement.
- overlay hook program 110 resides on computing device 100 .
- overlay hook program 110 may reside elsewhere within code testing environment 10 provided overlay hook program 110 has access to debug information 130 and condition table 120 .
- Overlay hook program 110 is described in further detail with respect to FIG. 2 .
- Condition table 120 is updated by overlay hook program 110 with table entries of each possible condition outcome from a condition statement in program code as each possible condition outcome is set.
- Condition table 120 is a mapping table in which each entry includes a map to the location of the condition statement in the program code and the state of the condition once set.
- the state of the condition is the condition code register, or its equivalent depending on the instruction set architecture (ISA), such as high, low, less than or equal to, less than, equal to, greater than, greater than or equal to, etc.
- ISA instruction set architecture
- condition table 120 resides on computing device 100 . In another embodiment, condition table 120 may reside elsewhere within code testing environment 10 provided overlay hook program 110 has access to condition table 120 .
- Debug information 130 is used by overlay hook program 110 to locate where conditions are in the program code allowing overlay hook program 110 to insert an overlay hook at each condition.
- Debug information 130 is generated as an object file when compiling code with a static compiler.
- Debug information 130 includes side information associated with the program code that gives mapping information from actual individual instructions back to the program code. For example, “instruction A” was generated to support the original source level statement on line 57 of “source file X.”
- debug information 130 resides on computing device 100 . In another embodiment, debug information 130 may reside elsewhere within code testing environment 10 provided overlay hook program 110 has access to debug information 130 .
- FIG. 2 depicts flowchart 200 of the steps of overlay hook program 110 , executing within code testing environment 10 of FIG. 1 , in accordance with an embodiment of the present invention.
- Overlay hook program 110 prepares a program code and then executes inserted overlay hooks until each possible condition outcome has executed.
- overlay hook program 110 prepares the program code. Before the program code is executed, overlay hook program 110 prepares the program code as described below according to different embodiments.
- overlay hook program 110 analyzes debug information 130 to identify where condition statements are located in the program code and inserts an overlay hook at each condition statement.
- overlay hook program 110 compiles the program code for use in MC/DC code coverage analysis.
- These broken apart branches create simple condition statements in which each condition statement in the program code only has one possible condition outcome, i.e., one condition in a condition code register.
- Overlay hook program 110 analyzes debug information 130 for where condition statements are in the program code.
- Overlay hook program 110 inserts a simple overlay hook at each simple condition statement that includes instructions to update condition table 120 with the condition statement's location when the condition statement executes.
- overlay hook program 110 executes the overlay hook in the program code, which involves overlay hook program 110 recording the location of the condition statement in condition table 120 .
- Overlay hook program 110 removes the overlay hook and continues executing the program code.
- overlay hook program 110 generates a report from condition table 120 .
- overlay hook program 110 is a combination of the first and second embodiments.
- Overlay hook program 110 analyzes debug information 130 for where condition statements are in the program code.
- Overlay hook program 110 inserts a simple overlay hook for a simple condition statement or inserts a complex overlay hook for a complex condition statement.
- a simple condition statement only has one possible condition outcome, i.e. one condition in a condition code register.
- a complex condition statement has multiple possible outcomes, i.e. multiple possible conditions in a condition code register and perhaps multiple condition code registers, if the instruction set architecture supports multiple condition code registers.
- a simple overlay hook records its location in condition table 120 and then the simple overlay hook is eliminated.
- a complex overlay hook operates the same as an overlay hook described in paragraphs [0011-0012].
- overlay hook program 110 reaches an overlay hook.
- overlay hook program 110 runs alongside a debugger executing the program code. When the debugger begins executing the program code, overlay hook program 110 initiates running alongside the debugger. In an embodiment, overlay hook program 110 runs alongside the debugger and reaches an overlay hook in the program code and executes the overlay hook.
- overlay hook program 110 records a condition outcome.
- overlay hook program 110 records the outcome of the condition and the location of the condition as table entries in condition table 120 .
- overlay hook program 110 records the location of the condition as a table entry in condition table 120 .
- overlay hook program 110 records the outcome of the condition and the location of the condition as table entries in condition table 120 .
- overlay hook program 110 records the location of the condition as a table entry in condition table 120 .
- overlay hook program 110 determines whether each possible condition outcome has been recorded. For example, if a condition statement is “less than or equal to,” the condition statement has multiple possible outcomes. Overlay hook program 110 cross-references with condition table 120 to determine if all the possible condition outcomes have been recorded. If overlay hook program 110 determines that all the possible condition outcomes have been recorded (decision 240 , yes branch), then overlay hook program 110 eliminates the overlay hook and continues executing the program code (see step 260 ). If overlay hook program 110 determines that not all the possible condition outcomes have been recorded (decision 240 , no branch), then overlay hook program 110 returns to the program code (see step 250 ).
- overlay hook program 110 continues executing the program code.
- overlay hook program 110 determines that not all the possible condition outcomes have been recorded, therefore overlay hook program 110 keeps the overlay hook at the condition statement and continues executing the program code after the overlay hook.
- overlay hook program 110 removes the overlay hook and continues executing the program code.
- overlay hook program 110 determines that all the possible condition outcomes have been recorded, therefore, overlay hook program 110 removes the overlay hook from the condition statement and continues executing the program code after the overlay hook.
- overlay hook program 110 generates a report.
- overlay hook program 110 generates a report of the condition states and condition statement locations recorded in condition table 120 .
- FIG. 3 is a block diagram depicting components of a computer 300 suitable for executing the overlay hook program 110 .
- FIG. 3 displays the computer 300 , the one or more processor(s) 304 (including one or more computer processors), the communications fabric 302 , the memory 306 , the cache 316 , the persistent storage 308 , the communications unit 310 , the I/O interfaces 312 , the display 320 , and the external devices 318 .
- FIG. 3 provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.
- the computer 300 operates over a communications fabric 302 , which provides communications between the cache 316 , the computer processor(s) 304 , the memory 306 , the persistent storage 308 , the communications unit 310 , and the input/output (I/O) interface(s) 312 .
- the communications fabric 302 may be implemented with any architecture suitable for passing data and/or control information between the processors 304 (e.g. microprocessors, communications processors, and network processors, etc.), the memory 306 , the external devices 318 , and any other hardware components within a system.
- the communications fabric 302 may be implemented with one or more buses or a crossbar switch.
- the memory 306 and persistent storage 308 are computer readable storage media.
- the memory 306 includes a random access memory (RAM).
- the memory 306 may include any suitable volatile or non-volatile implementations of one or more computer readable storage media.
- the cache 316 is a fast memory that enhances the performance of computer processor(s) 304 by holding recently accessed data, and data near accessed data, from memory 306 .
- Program instructions for overlay hook program 110 may be stored in the persistent storage 308 or in memory 306 , or more generally, any computer readable storage media, for execution by one or more of the respective computer processors 304 via the cache 316 .
- the persistent storage 308 may include a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, the persistent storage 308 may include, a solid state hard disk drive, a semiconductor storage device, read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information.
- the media used by the persistent storage 308 may also be removable.
- a removable hard drive may be used for persistent storage 308 .
- Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer readable storage medium that is also part of the persistent storage 308 .
- the communications unit 310 provides for communications with other data processing systems or devices.
- the communications unit 310 may include one or more network interface cards.
- the communications unit 310 may provide communications through the use of either or both physical and wireless communications links.
- Overlay hook program 110 may be downloaded to the persistent storage 308 through the communications unit 310 .
- the source of the various input data may be physically remote to the computer 300 such that the input data may be received and the output similarly transmitted via the communications unit 310 .
- the I/O interface(s) 312 allows for input and output of data with other devices that may operate in conjunction with the computer 300 .
- the I/O interface 312 may provide a connection to the external devices 318 , which may include a keyboard, keypad, a touch screen, and/or some other suitable input devices.
- External devices 318 may also include portable computer readable storage media, for example, thumb drives, portable optical or magnetic disks, and memory cards.
- Software and data used to practice embodiments of the present invention may be stored on such portable computer readable storage media and may be loaded onto the persistent storage 308 via the I/O interface(s) 312 .
- the I/O interface(s) 312 may similarly connect to a display 320 .
- the display 320 provides a mechanism to display data to a user and may be, for example, a computer monitor.
- the present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration
- the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention
- the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
- the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
- a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- SRAM static random access memory
- CD-ROM compact disc read-only memory
- DVD digital versatile disk
- memory stick a floppy disk
- a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon
- a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
- Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
- the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.
- a network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
- Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages.
- the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
- These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
- the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
- each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s).
- the functions noted in the blocks may occur out of the order noted in the Figures.
- two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
- The present invention relates generally to the field of code coverage, and more particularly to runtime-directed overlay hooks being used to track code coverage.
- Code coverage is a measure used to describe the degree to which the program code of a program is tested by a particular test suite. A program with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a program with low code coverage. To measure what percentage of code has been exercised by a test suite, one or more coverage criteria are used: function coverage, statement coverage, branch coverage, and condition coverage. Function coverage requires that each function in the program be called. Statement coverage requires each statement in the program to be executed. Branch coverage requires each branch of each control structure to be executed. Condition coverage requires each Boolean sub-expression to be evaluated both to true and false.
- A combination of function coverage and branch coverage is sometimes also called decision coverage. This criterion requires that every point of entry and exit in the program have been invoked at least once, and every decision in the program have taken on all possible outcomes at least once. Modified condition/decision coverage (MC/DC) requires that both decision and condition coverage be satisfied and that each condition should affect the decision outcome independently.
- Aspects of an embodiment of the present invention disclose a method for utilizing overlay hooks to increase code coverage. A processor inserts an overlay hook in program code at a location within the program code corresponding to a condition statement. A processor executes the program code. Upon reaching the overlay hook, a processor branches to a set of instructions operative to document an outcome of the condition statement. A processor executes the condition statement. A processor records the outcome of the condition statement.
-
FIG. 1 is a functional block diagram illustrating a code testing environment, in accordance with an embodiment of the present invention; -
FIG. 2 is a flowchart depicting operational steps of an overlay hook program, on a computing device within the environment ofFIG. 1 , in accordance with an embodiment of the present invention; and -
FIG. 3 depicts a block diagram of components of the computing device executing the overlay hook program, in accordance with an embodiment of the present invention. - Embodiments of the present invention recognize the importance of achieving high code coverage when testing the program code of a program. Using modified condition/decision coverage (MC/DC), it is impossible to tell if a test is covering all the possible conditions for a section of code. For example, if a line of code reads—if (a<=b) then {b=5} else {b=1}, MC/DC code coverage will only show that a<=b and a>b were tested. However, it cannot be determined whether a<b or a=b caused the ‘if’ to execute for the a<=b instance. Thus, there is a need for a code coverage method that covers these edge conditions in testing. Embodiments of the present invention provide solutions for this inability to determine whether each condition has actually been tested. In this manner, as discussed in greater detail herein, embodiments of the present invention can provide finer-grained coverage of conditional paths through code by tracking all possible condition codes through the use of overlay hooks.
- The present invention will now be described in detail with reference to the Figures.
-
FIG. 1 depicts a diagram ofcode testing environment 10, in accordance with an embodiment of the present invention.FIG. 1 provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented. - In the depicted embodiment,
code testing environment 10 includes computing device 100.Code testing environment 10 may include additional computing devices, servers, computers, mobile devices, or other devices not shown. Computing device 100 may be a management server, a web server, laptop computer, tablet computer, netbook computer, personal computer (PC), a desktop computer, a smart phone, or any programmable electronic device or computing system capable of running a program and receiving and sending data. In other embodiments, computing device 100 may represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In the depicted embodiment, computing device 100 containsoverlay hook program 110, condition table 120, anddebug information 130. Computing device 100 may include components, as depicted and described in further detail with respect toFIG. 3 . -
Overlay hook program 110 operates as a runtime debugger that executes overlay hooks inserted in the program code to track code coverage by recording each condition outcome as the condition outcome is set. An overlay hook is a segment of code that places a hook into a program to transfer control to a debugger and overlays the current instruction with a new instruction. When inserting an overlay hook,overlay hook program 110 creates a location that can be branched to, and at that location,overlay hook program 110 saves the condition statement that is overlaid by the overlay hook. The overlay hook lines of code are inserted directly on top of where the condition statement was in the program code. For example,overlay hook program 110 would replace a condition statement instruction “Compare A & B” with “Branch to comparison test,” which is the overlay hook. The “comparison test” has instructions to execute the condition, record the condition outcome, check if all condition outcomes have been recorded, and return to the program code to continue executing. It should be appreciated that the actual overlay hook sequence will vary by operating system, compiled language, hardware instruction set, and assembler. - Generally,
overlay hook program 110 runs alongside a debugger executing the program code. When the debugger begins executing the program code,overlay hook program 110 initiates running alongside the debugger. When an overlay hook is reached in the program code,overlay hook program 110 executes the overlay hook in the program code.Overlay hook program 110 determines whether to remove the overlay hook by checking condition table 120 to see if each possible condition outcome has executed. If each possible condition outcome has executed,overlay hook program 110 removes the overlay hook, reinserts the original condition statement, and continues executing the program code. If each possible condition outcome has not executed,overlay hook program 110 leaves the overlay hook within the program code and continues executing the program code. When the program code has been executed completely,overlay hook program 110 generates a report from condition table 120.Overlay hook program 110 has access to condition table 120 and updates condition table 120 with table entries of the location and outcome of a condition statement. In the depicted embodiment,overlay hook program 110 resides on computing device 100. In another embodiment,overlay hook program 110 may reside elsewhere withincode testing environment 10 providedoverlay hook program 110 has access todebug information 130 and condition table 120.Overlay hook program 110 is described in further detail with respect toFIG. 2 . - Condition table 120 is updated by
overlay hook program 110 with table entries of each possible condition outcome from a condition statement in program code as each possible condition outcome is set. Condition table 120 is a mapping table in which each entry includes a map to the location of the condition statement in the program code and the state of the condition once set. The state of the condition is the condition code register, or its equivalent depending on the instruction set architecture (ISA), such as high, low, less than or equal to, less than, equal to, greater than, greater than or equal to, etc. In the depicted embodiment, condition table 120 resides on computing device 100. In another embodiment, condition table 120 may reside elsewhere withincode testing environment 10 providedoverlay hook program 110 has access to condition table 120. -
Debug information 130 is used byoverlay hook program 110 to locate where conditions are in the program code allowingoverlay hook program 110 to insert an overlay hook at each condition.Debug information 130 is generated as an object file when compiling code with a static compiler.Debug information 130 includes side information associated with the program code that gives mapping information from actual individual instructions back to the program code. For example, “instruction A” was generated to support the original source level statement on line 57 of “source file X.” In the depicted embodiment,debug information 130 resides on computing device 100. In another embodiment,debug information 130 may reside elsewhere withincode testing environment 10 providedoverlay hook program 110 has access to debuginformation 130. -
FIG. 2 depictsflowchart 200 of the steps ofoverlay hook program 110, executing withincode testing environment 10 ofFIG. 1 , in accordance with an embodiment of the present invention.Overlay hook program 110 prepares a program code and then executes inserted overlay hooks until each possible condition outcome has executed. - In
step 210,overlay hook program 110 prepares the program code. Before the program code is executed,overlay hook program 110 prepares the program code as described below according to different embodiments. - In a first embodiment,
overlay hook program 110 analyzesdebug information 130 to identify where condition statements are located in the program code and inserts an overlay hook at each condition statement. - In a second embodiment,
overlay hook program 110 compiles the program code for use in MC/DC code coverage analysis.Overlay hook program 110 breaks apart branches to test only one possible condition outcome at a time. For example, the condition statement “if (a<=b) then {b=5} else {b=1}” would be broken apart to “if (a<b) then {b=5} else {b=1}” and “if (a=b) then {b=5} else {b=1}.” These broken apart branches create simple condition statements in which each condition statement in the program code only has one possible condition outcome, i.e., one condition in a condition code register.Overlay hook program 110 analyzesdebug information 130 for where condition statements are in the program code.Overlay hook program 110 inserts a simple overlay hook at each simple condition statement that includes instructions to update condition table 120 with the condition statement's location when the condition statement executes. When an overlay hook is reached in the program code,overlay hook program 110 executes the overlay hook in the program code, which involvesoverlay hook program 110 recording the location of the condition statement in condition table 120.Overlay hook program 110 removes the overlay hook and continues executing the program code. When the program code has been executed completely,overlay hook program 110 generates a report from condition table 120. - In a third embodiment,
overlay hook program 110 is a combination of the first and second embodiments.Overlay hook program 110 analyzesdebug information 130 for where condition statements are in the program code.Overlay hook program 110 inserts a simple overlay hook for a simple condition statement or inserts a complex overlay hook for a complex condition statement. A simple condition statement only has one possible condition outcome, i.e. one condition in a condition code register. A complex condition statement has multiple possible outcomes, i.e. multiple possible conditions in a condition code register and perhaps multiple condition code registers, if the instruction set architecture supports multiple condition code registers. A simple overlay hook records its location in condition table 120 and then the simple overlay hook is eliminated. A complex overlay hook operates the same as an overlay hook described in paragraphs [0011-0012]. - In
step 220,overlay hook program 110 reaches an overlay hook. In an embodiment,overlay hook program 110 runs alongside a debugger executing the program code. When the debugger begins executing the program code,overlay hook program 110 initiates running alongside the debugger. In an embodiment,overlay hook program 110 runs alongside the debugger and reaches an overlay hook in the program code and executes the overlay hook. - In
step 230,overlay hook program 110 records a condition outcome. In the first embodiment, once the condition statement has executed,overlay hook program 110 records the outcome of the condition and the location of the condition as table entries in condition table 120. In the second embodiment, once the simple condition statement has executed,overlay hook program 110 records the location of the condition as a table entry in condition table 120. In the third embodiment, for a complex condition statement that is executed,overlay hook program 110 records the outcome of the condition and the location of the condition as table entries in condition table 120. In the third embodiment, for a simple condition statement that is executed,overlay hook program 110 records the location of the condition as a table entry in condition table 120. - In
decision 240,overlay hook program 110 determines whether each possible condition outcome has been recorded. For example, if a condition statement is “less than or equal to,” the condition statement has multiple possible outcomes.Overlay hook program 110 cross-references with condition table 120 to determine if all the possible condition outcomes have been recorded. Ifoverlay hook program 110 determines that all the possible condition outcomes have been recorded (decision 240, yes branch), thenoverlay hook program 110 eliminates the overlay hook and continues executing the program code (see step 260). Ifoverlay hook program 110 determines that not all the possible condition outcomes have been recorded (decision 240, no branch), thenoverlay hook program 110 returns to the program code (see step 250). - In
step 250,overlay hook program 110 continues executing the program code. In an embodiment,overlay hook program 110 determines that not all the possible condition outcomes have been recorded, thereforeoverlay hook program 110 keeps the overlay hook at the condition statement and continues executing the program code after the overlay hook. - In
step 260,overlay hook program 110 removes the overlay hook and continues executing the program code. In an embodiment,overlay hook program 110 determines that all the possible condition outcomes have been recorded, therefore,overlay hook program 110 removes the overlay hook from the condition statement and continues executing the program code after the overlay hook. - In
step 270,overlay hook program 110 generates a report. In an embodiment,overlay hook program 110 generates a report of the condition states and condition statement locations recorded in condition table 120. -
FIG. 3 is a block diagram depicting components of acomputer 300 suitable for executing theoverlay hook program 110.FIG. 3 displays thecomputer 300, the one or more processor(s) 304 (including one or more computer processors), the communications fabric 302, the memory 306, the cache 316, the persistent storage 308, thecommunications unit 310, the I/O interfaces 312, thedisplay 320, and the external devices 318. It should be appreciated thatFIG. 3 provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made. - As depicted, the
computer 300 operates over a communications fabric 302, which provides communications between the cache 316, the computer processor(s) 304, the memory 306, the persistent storage 308, thecommunications unit 310, and the input/output (I/O) interface(s) 312. The communications fabric 302 may be implemented with any architecture suitable for passing data and/or control information between the processors 304 (e.g. microprocessors, communications processors, and network processors, etc.), the memory 306, the external devices 318, and any other hardware components within a system. For example, the communications fabric 302 may be implemented with one or more buses or a crossbar switch. - The memory 306 and persistent storage 308 are computer readable storage media. In the depicted embodiment, the memory 306 includes a random access memory (RAM). In general, the memory 306 may include any suitable volatile or non-volatile implementations of one or more computer readable storage media. The cache 316 is a fast memory that enhances the performance of computer processor(s) 304 by holding recently accessed data, and data near accessed data, from memory 306.
- Program instructions for
overlay hook program 110 may be stored in the persistent storage 308 or in memory 306, or more generally, any computer readable storage media, for execution by one or more of the respective computer processors 304 via the cache 316. The persistent storage 308 may include a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, the persistent storage 308 may include, a solid state hard disk drive, a semiconductor storage device, read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information. - The media used by the persistent storage 308 may also be removable. For example, a removable hard drive may be used for persistent storage 308. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer readable storage medium that is also part of the persistent storage 308.
- The
communications unit 310, in these examples, provides for communications with other data processing systems or devices. In these examples, thecommunications unit 310 may include one or more network interface cards. Thecommunications unit 310 may provide communications through the use of either or both physical and wireless communications links.Overlay hook program 110 may be downloaded to the persistent storage 308 through thecommunications unit 310. In the context of some embodiments of the present invention, the source of the various input data may be physically remote to thecomputer 300 such that the input data may be received and the output similarly transmitted via thecommunications unit 310. - The I/O interface(s) 312 allows for input and output of data with other devices that may operate in conjunction with the
computer 300. For example, the I/O interface 312 may provide a connection to the external devices 318, which may include a keyboard, keypad, a touch screen, and/or some other suitable input devices. External devices 318 may also include portable computer readable storage media, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention may be stored on such portable computer readable storage media and may be loaded onto the persistent storage 308 via the I/O interface(s) 312. The I/O interface(s) 312 may similarly connect to adisplay 320. Thedisplay 320 provides a mechanism to display data to a user and may be, for example, a computer monitor. - The programs described herein are identified based upon the application for which they are implemented in a specific embodiment of the invention. However, it should be appreciated that any particular program nomenclature herein is used merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature.
- The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
- The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
- Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
- Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
- Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
- These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
- The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
- The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
Claims (1)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/671,392 US9934133B1 (en) | 2016-09-19 | 2017-08-08 | Code coverage through overlay hooks |
US15/876,450 US10176083B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
US15/876,487 US10169212B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/268,670 US10417112B2 (en) | 2016-09-19 | 2016-09-19 | Code coverage through overlay hooks |
US15/671,392 US9934133B1 (en) | 2016-09-19 | 2017-08-08 | Code coverage through overlay hooks |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/268,670 Continuation US10417112B2 (en) | 2016-09-19 | 2016-09-19 | Code coverage through overlay hooks |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/876,487 Continuation US10169212B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
US15/876,450 Continuation US10176083B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180081792A1 true US20180081792A1 (en) | 2018-03-22 |
US9934133B1 US9934133B1 (en) | 2018-04-03 |
Family
ID=61620397
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/268,670 Active US10417112B2 (en) | 2016-09-19 | 2016-09-19 | Code coverage through overlay hooks |
US15/671,392 Expired - Fee Related US9934133B1 (en) | 2016-09-19 | 2017-08-08 | Code coverage through overlay hooks |
US15/876,487 Expired - Fee Related US10169212B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
US15/876,450 Expired - Fee Related US10176083B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/268,670 Active US10417112B2 (en) | 2016-09-19 | 2016-09-19 | Code coverage through overlay hooks |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/876,487 Expired - Fee Related US10169212B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
US15/876,450 Expired - Fee Related US10176083B2 (en) | 2016-09-19 | 2018-01-22 | Code coverage through overlay hooks |
Country Status (1)
Country | Link |
---|---|
US (4) | US10417112B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110765017A (en) * | 2019-10-24 | 2020-02-07 | 上海创景信息科技有限公司 | RISC-V extended instruction-based source code coverage rate test method and system |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020045929A1 (en) * | 2018-08-27 | 2020-03-05 | Samsung Electronics Co., Ltd. | Code coverage method for embedded system on chip |
GB2577102B (en) * | 2018-09-14 | 2021-03-03 | Advanced Risc Mach Ltd | Generation of code coverage information during testing of a code sequence |
US11200151B1 (en) | 2021-03-12 | 2021-12-14 | Sas Institute Inc. | Graphical user interface and debugger system for selecting and testing alterations to source code for software applications |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5761408A (en) * | 1996-01-16 | 1998-06-02 | Parasoft Corporation | Method and system for generating a computer program test suite using dynamic symbolic execution |
US5784553A (en) * | 1996-01-16 | 1998-07-21 | Parasoft Corporation | Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs |
US6023580A (en) * | 1996-07-03 | 2000-02-08 | Objectswitch Corporation | Apparatus and method for testing computer systems |
US20100175052A1 (en) * | 2009-01-05 | 2010-07-08 | Tata Consultancy Services Limited | System and method for automatic generation of test data to satisfy modified condition decision coverage |
US20160321586A1 (en) * | 2015-04-29 | 2016-11-03 | Microsoft Technology Licensing, Llc | Selecting tests for execution on a software product |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4987249A (en) * | 1989-09-28 | 1991-01-22 | Atochem North America, Inc. | 2,4-pentanedione-1,5-disulfonic acid and method for preparing the same |
US5909577A (en) * | 1994-04-18 | 1999-06-01 | Lucent Technologies Inc. | Determining dynamic properties of programs |
US5758061A (en) * | 1995-12-15 | 1998-05-26 | Plum; Thomas S. | Computer software testing method and apparatus |
US6990438B1 (en) * | 2000-09-12 | 2006-01-24 | Synopsys, Inc. | Method and apparatus for observability-based code coverage |
US20050081104A1 (en) * | 2003-09-25 | 2005-04-14 | Borislav Nikolik | Test diversity software testing method and apparatus |
US20060253739A1 (en) * | 2005-05-03 | 2006-11-09 | Godefroid Patrice I | Method and apparatus for performing unit testing of software modules with use of directed automated random testing |
US7861226B1 (en) | 2006-03-16 | 2010-12-28 | Avaya Inc. | Constraint solver to code based test data generation for improving software reliability and security |
US7873945B2 (en) | 2007-06-29 | 2011-01-18 | Microsoft Corporation | Automatically generating test cases for binary code |
US8473915B2 (en) * | 2007-08-03 | 2013-06-25 | International Business Machines Corporation | Coverage analysis tool for testing database-aware software applications |
US8302080B2 (en) | 2007-11-08 | 2012-10-30 | Ntt Docomo, Inc. | Automated test input generation for web applications |
US8898647B2 (en) | 2008-10-09 | 2014-11-25 | Siemens Aktiengesellschaft | Method and apparatus for test coverage analysis |
US8914879B2 (en) * | 2010-06-11 | 2014-12-16 | Trustwave Holdings, Inc. | System and method for improving coverage for web code |
US8769512B2 (en) * | 2011-03-24 | 2014-07-01 | International Business Machines Corporation | Adding instrumentation to a body of code to enable generation of code coverage data |
US8839203B2 (en) | 2011-05-25 | 2014-09-16 | Microsoft Corporation | Code coverage-based taint perimeter detection |
US8954926B2 (en) | 2012-03-05 | 2015-02-10 | Microsoft Corporation | Code coverage detection with scriptable language unmodified source |
-
2016
- 2016-09-19 US US15/268,670 patent/US10417112B2/en active Active
-
2017
- 2017-08-08 US US15/671,392 patent/US9934133B1/en not_active Expired - Fee Related
-
2018
- 2018-01-22 US US15/876,487 patent/US10169212B2/en not_active Expired - Fee Related
- 2018-01-22 US US15/876,450 patent/US10176083B2/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5761408A (en) * | 1996-01-16 | 1998-06-02 | Parasoft Corporation | Method and system for generating a computer program test suite using dynamic symbolic execution |
US5784553A (en) * | 1996-01-16 | 1998-07-21 | Parasoft Corporation | Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs |
US6023580A (en) * | 1996-07-03 | 2000-02-08 | Objectswitch Corporation | Apparatus and method for testing computer systems |
US20100175052A1 (en) * | 2009-01-05 | 2010-07-08 | Tata Consultancy Services Limited | System and method for automatic generation of test data to satisfy modified condition decision coverage |
US20160321586A1 (en) * | 2015-04-29 | 2016-11-03 | Microsoft Technology Licensing, Llc | Selecting tests for execution on a software product |
Non-Patent Citations (2)
Title |
---|
Fu et al., "Compiler-directed Program-fault Coverage for Highly Available Java Internet Services", 2003 * |
Shahid et al., "An Evaluation of Test Coverage Tools in Software Testing", 2011 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110765017A (en) * | 2019-10-24 | 2020-02-07 | 上海创景信息科技有限公司 | RISC-V extended instruction-based source code coverage rate test method and system |
Also Published As
Publication number | Publication date |
---|---|
US20180081782A1 (en) | 2018-03-22 |
US20180129591A1 (en) | 2018-05-10 |
US10169212B2 (en) | 2019-01-01 |
US10176083B2 (en) | 2019-01-08 |
US20180121334A1 (en) | 2018-05-03 |
US10417112B2 (en) | 2019-09-17 |
US9934133B1 (en) | 2018-04-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10198345B2 (en) | Test case generation for uncovered code paths | |
US10169212B2 (en) | Code coverage through overlay hooks | |
US9952965B2 (en) | Test self-verification with integrated transparent self-diagnose | |
US20170364331A1 (en) | Method to support multiple versions of apis and to control run-time execution of an api | |
US10162732B2 (en) | Managing logger source code segments | |
US10216609B2 (en) | Exception prediction before an actual exception during debugging | |
US20140258785A1 (en) | Identifying a storage location for a storage address requested during debugging | |
US20160321168A1 (en) | Unified processing test structure | |
US10120666B2 (en) | Conditional branch instruction compaction for regional code size reduction | |
US10599554B2 (en) | Dynamic instrumentation based on detected errors | |
US10248534B2 (en) | Template-based methodology for validating hardware features | |
US20160350155A1 (en) | Synthesizing inputs to preserve functionality | |
US10078572B1 (en) | Abnormal timing breakpoints | |
US20200319994A1 (en) | Generating representative microbenchmarks | |
US10353766B2 (en) | Managing execution of computer tasks under time constraints | |
US10275248B2 (en) | Testing computer software using tracking bits |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FULTON, MICHAEL S.;STOODLEY, KEVIN A.;SIGNING DATES FROM 20160831 TO 20160911;REEL/FRAME:043228/0603 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20220403 |