Disclosure of Invention
Embodiments of the present specification aim to provide a more efficient automatic test scheme to address the deficiencies in the prior art.
To achieve the above object, an aspect of the present specification provides an automatic test method, including:
acquiring a first test configuration file corresponding to a first object to be tested, wherein the first test configuration file comprises at least one test case which is sequentially arranged, each test case comprises a call for one of a plurality of prefabricated functional modules and configuration for input data and output data of the functional module, and each functional module corresponds to one operation in a test; and
and sequentially calling the function modules in each test case according to the arrangement sequence of the at least one test case and the configuration of the input data and the output data of the corresponding function modules in each test case aiming at the first object to be tested, so as to obtain a test result corresponding to the first test configuration file.
In one embodiment, the first object to be tested is a first node in a blockchain.
In one embodiment, each of the functional modules corresponds to one of the following operations: creating an account, transferring accounts, sending transactions, verifying transactions, querying messages, querying account balances, comparing values, creating smart contracts, invoking smart contracts, executing smart contracts, generating blocks, sending blocks, storing blocks.
In one embodiment, the method is performed in a test server, and the method further comprises, after acquiring the first test configuration file corresponding to the first object to be tested, acquiring connection configuration information of the first node, and connecting the first node based on the connection configuration information.
In one embodiment, the plurality of functional modules includes a first functional module, and the operations included in the first functional module include: and indicating the first node to perform a preset operation in the blockchain, receiving an execution result of the preset operation from the first node and outputting the execution result.
In one embodiment, the method further comprises, after obtaining a first test profile corresponding to the first object to be tested, converting the first test profile into a program readable form.
In one embodiment, the method further comprises, after obtaining the test result corresponding to the first test configuration file, performing result statistics based on the test result.
In one embodiment, in the sequentially arranged at least one test case, for two test cases in which the test cases are sequentially arranged, the output data of the preceding test case is configured as the input data of the following test case.
Another aspect of the present specification provides an automatic test equipment, comprising:
an obtaining unit, configured to obtain a first test configuration file corresponding to a first object to be tested, where the first test configuration file includes at least one test case arranged in sequence, each test case includes a call to one of a plurality of prefabricated functional modules and a configuration of input data and output data of the functional module, and each functional module corresponds to one operation in a test; and
the calling unit is configured to sequentially call the functional modules in each test case according to the arrangement sequence of the at least one test case and the configuration of the input data and the output data of the corresponding functional module in each test case aiming at the first object to be tested, so as to obtain a test result corresponding to the first test configuration file.
In one embodiment, the first object to be tested is a first node in a blockchain, where the apparatus is deployed in a test server, and the apparatus further includes a connection unit configured to obtain connection configuration information of the first node after obtaining a first test configuration file corresponding to the first object to be tested, and connect the first node based on the connection configuration information.
In an embodiment, the apparatus further comprises an interface unit configured to convert the first test profile into a program readable form after acquiring the first test profile corresponding to the first object to be tested.
In one embodiment, the apparatus further includes a statistics unit configured to, after acquiring the test result corresponding to the first test configuration file, perform result statistics based on the test result.
Another aspect of the present description provides a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform any of the methods described above.
Another aspect of the present specification provides a computing device comprising a memory and a processor, wherein the memory has executable code stored therein, and wherein the processor, when executing the executable code, performs any of the methods described above.
By the automatic test scheme according to the embodiment of the specification, a configurable and pluggable data-driven automatic test framework is realized, and a tester only needs to configure corresponding test files when writing test cases, so that the tester does not need to be familiar with specific development languages, the writing cost of the test cases is reduced, and the test efficiency is improved.
Detailed Description
Embodiments of the present specification will be described below with reference to the accompanying drawings.
Fig. 1 shows a schematic diagram of an automated test framework 100 according to an embodiment of the present description. As shown in fig. 1, the framework 100 includes an input module 11, a calling module 12, a function module 13, and an output module 14. The test framework 100 may be deployed, for example, at a test server that tests corresponding services in a blockchain through respective nodes, for example, by connecting the respective nodes in the blockchain. For example, when a first node in a blockchain is to be tested, connection configuration information of the first node and a test configuration file corresponding to the first node are first obtained through the input module 11. For example, the connection configuration information of the first node and the file path of the test configuration file corresponding to the first node may be input in the form of an input command line through the input module 11, so that the test configuration file may be acquired through the file path, and a connection may be established with the first node through the connection configuration information of the first node. Wherein the test profile comprises calls to at least one functional module 13. The input module 11 may further comprise an interface module (not shown) for converting the data form of the test configuration file. The call modules 12 are generated by the framework 100 based on the respective test profiles after the respective test profiles are acquired, one call module corresponding to each test profile, only one call module being shown here for illustration. In the calling module 12, a call to the at least one (3 are schematically shown in the figure) functional module 13 is performed based on the test configuration file, and a test result is obtained, wherein each functional module corresponds to a specific functional operation under test. The calling module 12 then sends the test results to the output module 14 for processing of statistics, report generation, configuration log, etc. of the test results. The foregoing is illustrative only and is not intended to limit the scope of the embodiments of the present disclosure. For example, in the case where the above-described test framework is deployed on a non-node test server, at the input module 11, a plurality of (node connection configuration information, at least one test configuration file corresponding to the node) information sets arranged in sequence may be acquired, so that the test on a plurality of nodes, which are connected by connection configuration information corresponding to the node before the test on each node is performed, may be sequentially performed by the plurality of calling modules 12; the test framework can also be deployed locally to the node to be tested, so that the step of connecting the nodes is not required; in addition, the test framework is not limited to use in testing blockchain systems, but may be used to test other software, software systems, and the like.
The above-described test procedure through the automatic test framework is described in detail below.
Fig. 2 shows a flow chart of an automatic test method according to an embodiment of the present specification, including:
in step S202, a first test configuration file corresponding to a first object to be tested is obtained, where the first test configuration file includes at least one test case arranged in sequence, and each test case includes a call to one of a plurality of prefabricated functional modules and a configuration of input data and output data of the functional module, where each functional module corresponds to one operation in the test;
in step S204, for the first object to be tested, according to the arrangement sequence of the at least one test case, based on the configuration of the input data and the output data of the corresponding functional module in each test case, the functional modules in each test case are sequentially called, so as to obtain a test result corresponding to the first test configuration file.
The method is performed for a particular object to be tested, such as any node in the blockchain, such as the first node. As shown in fig. 1, the automatic test framework includes a plurality of functional modules, each corresponding to an operation under test, that are prefabricated. The embodiment is not limited to use in testing blockchain systems. When the systems to which the automatic test framework is applied are different, the prefabricated functional modules included therein are correspondingly different, i.e. the functional modules correspond to the operations that may be required to be used when testing the system to be tested. For example, when the automatic test framework is used for testing a blockchain system, the functional operations corresponding to the functional modules include: creating an account, transferring accounts, sending transactions, verifying transactions, querying messages, querying account balances, comparing values, creating smart contracts, invoking smart contracts, executing smart contracts, generating blocks, sending blocks, storing blocks, etc.
In one embodiment, the method may be performed locally at the first node. In one embodiment, the method is performed at a test server that performs a test for a first node by connecting the first node. It will be appreciated that, according to the embodiments of the present disclosure, for an automatic test framework deployed in a test server, multiple sets of information may be input at one time, each set of information including connection configuration information corresponding to one node and at least one test configuration file, so that the multiple nodes may be automatically connected in succession, and the method shown in fig. 2 may be performed on the multiple nodes to perform at least one test on each of them. Each step of the method is described in detail below.
First, in step S202, a first test configuration file corresponding to a first object to be tested is obtained, where the first test configuration file includes at least one test case arranged in sequence, and each test case includes a call to one of a plurality of prefabricated function modules and a configuration of input data and output data of the function module, where each function module corresponds to one operation in a test.
As described above, the automated test framework may read the corresponding test profile by retrieving the test profile path. The first test configuration file corresponds to a test of the first object to be tested (for example, the first node), and the test configuration file includes at least one test case arranged in sequence, that is, at least one test case arranged in sequence corresponds to at least one operation sequentially executed in the test, and each operation is implemented by calling one functional module shown in fig. 1, so that each test case includes a call to one functional module and configuration of input data and output data of the functional module. As described above, in the case where the automatic test framework is applied to a test blockchain, the functional modules respectively correspond to any one of the following operations: creating an account, transferring accounts, sending transactions, verifying transactions, querying messages, querying account balances, comparing values, creating smart contracts, invoking smart contracts, executing smart contracts, generating blocks, sending blocks, storing blocks.
In the case that the automatic test framework is deployed on a test server, the method further includes, after acquiring a first test configuration file corresponding to a first node, acquiring connection configuration information of the first node, and connecting the first node based on the connection configuration information. The connection configuration information includes, for example, an IP address and port of the first node, key information of the first node, and the like. The test server may connect to the first node through the IP address and the port number, and perform encrypted communication with the first node through key information provided by the first node, and so on.
In the case of the above test server, when the first operation corresponding to the functional module needs to be performed by a node in the blockchain, all operations included in the functional module include: and indicating the node (namely the first node) connected with the server to execute the first operation, receiving the execution result of the first operation from the first node, and outputting the execution result, thereby realizing the remote control of the node connected with the test server to complete the test.
The test configuration file may be implemented in various configuration file formats, such as Json file, XML file, etc., as long as the configuration file includes the above items and can be parsed by the program. One test case included in the test configuration file in the form of a Json file is shown below.
Wherein "Test-N-001" is the number of the Test case, which corresponds to an "account query" operation, where "type" represents a call to a function module "query account", where "input" represents a configuration of input data of the function module "query account", that is, input is an account address "c60a9d48 …", where "output" represents a configuration of output data of the function module, that is, a result of querying an account is assigned to a parameter "{ abcde }", and so on. Accordingly, the Test configuration file may further include a Test case "Test-N-002" arranged after the Test case "Test-N-001", where the Test case "Test-N-002" may call a function module (assertion function module) corresponding to the numerical comparison, for example, with the output (i.e., the actual account balance) of the Test case "Test-N-001" and the desired account balance as inputs, and compare them and output the comparison result as outputs. Therefore, the Test of the service for inquiring the account balance of the first node can be completed through the Test case 'Test-N-001' and the Test case 'Test-N-002'.
In step S204, for the first object to be tested, according to the arrangement sequence of the at least one test case, based on the configuration of the input data and the output data of the corresponding functional module in each test case, the functional modules in each test case are sequentially called, so as to obtain a test result corresponding to the first test configuration file.
For example, for the first node in the blockchain, when the first node is tested by using the Test configuration file including two Test cases (Test-N-001 and Test-N-002), corresponding calls are executed for the first node according to the arrangement order of the Test cases, that is, the account query function module in Test-N-001 is called first, so that the first node uses the service of the account query in the blockchain, and then the assertion function module in Test-N-002 is called to determine whether the account balance queried by the first node matches the expected account balance. Before each functional module is invoked, pre-processing of test cases may be performed, such as object construction based on configuration files (e.g., json files described above), type conversion of input data, and the like. After obtaining the output value of the calling function module each time, post processing of the test case can be performed, such as formatting the output data, i.e. packaging the output data into a specific format. After each test case in the test configuration file is executed sequentially, finally, a test result of the query balance test of the first node corresponding to the test configuration file is obtained, so that whether the query account balance service of the test blockchain is normal or not can be tested.
After the test results are obtained, result statistics may be performed based on the test results.
The test result is only a test result of one test on the first object to be tested (for example, the first node), for example, in the case of testing various functions and performances of the blockchain through an automatic test framework deployed in the test server, the test framework may test at least one node in the blockchain and at least one test is performed on each node, so the statistics on the test result may be that the test result is added in the statistics on the test result of a single node, or that the test result is added in the statistics on the test results of a plurality of nodes in the blockchain.
FIG. 3 schematically illustrates a loop diagram of the method of FIG. 2 performed multiple times in testing a blockchain. As shown in fig. 3, after the test framework acquires connection configuration information of at least one node and at least one test file path corresponding to each node, the test framework performs an outermost loop for at least one node, a middle loop for each test profile of each node, and an innermost loop for each test case in each test profile. Specifically, in step S302, the test framework first obtains connection configuration information of each node, and at least one file path corresponding to each node, where each file path corresponds to one test configuration file for the node. In step S304, a loop is started to be executed with respect to each node, and for one of the nodes, the node is connected based on the connection configuration information of the node, and at least one test configuration file corresponding to the node is acquired based on the corresponding at least one file path. In step S306, the middle tier loop is started to be executed, and the innermost tier loop is executed under the middle tier loop, that is, for one of the test profiles, a sequential call is made to each functional module in the innermost tier loop based on each test case of the test profile. In the figure, "yes" indicates the end of the cycle, and "no" indicates that the cycle is not ended. When the innermost loop is over, step S308 is performed, i.e., for the node for which the loop is intended, result statistics for that node are performed. After step S308, it is determined whether the middle tier cycle is ended, that is, whether the inner tier cycle of step S306 is performed for each test profile of the node, and in the case that the middle tier cycle is not ended, returning to step S306, the function module call for each test case in the next test profile is continued, and in the case that the middle tier cycle is ended, the result statistics for the blockchain may be performed in step S310, that is, the statistics for the node may be increased in the statistics for the blockchain. After that, it is determined whether or not the outermost loop is ended, that is, whether or not the above steps have been performed for each node corresponding to each node connection information, and in the case of not ending, returning to step S304, the above steps are re-performed, and in the case of ending, the automatic test procedure is ended.
After the node-level result statistics and the blockchain-level result statistics are performed as shown in fig. 3, a test report is also generated based on the statistics, and/or configuration management of test logs is performed based on the statistics, such as adjusting parameters of whether each level of test log is visible, and the like.
Fig. 4 shows a flowchart of the execution of an automatic test framework according to an embodiment of the present description. The flow diagrams correspond to examples of multiple tests performed on nodes in a blockchain, and it is to be understood that the flow diagrams are merely illustrative and are not intended to limit the scope of the embodiments of the present disclosure. Specifically, in step S401, the frame execution subject generates a frame instance, and the frame execution subject, that is, the main program of the frame. In step S402, the tester may input a file path listed as each test profile through the command line, so that the framework may acquire each test profile based on the file path, and in the case of testing each node through the framework deployed at the test server (which is not any node in the blockchain), the framework may further acquire connection configuration information, such as an IP address, a port number, etc., of each node input through the command line by the tester, and each test profile corresponds to each node, respectively, wherein one node may correspond to at least one test profile, i.e., at least one test is performed on one node. In step S403, the framework instance loads and registers the existing prefabricated plurality of function modules. In step S404, the framework execution body sends the acquired test configuration file to the interface module, so as to perform data conversion processing on the test configuration file, so as to convert the test configuration file into program readable data. In step S405, the interface module transmits the converted test profile to the execution body.
From steps S406-S412, cycle 1, the number of times of cycle 1 corresponds to the number of test profiles. Where the example tests multiple nodes, cycle 1 includes the outer and middle layer cycles of fig. 3, where the steps of connecting nodes (not shown) are also included for different nodes. From step S407 to step S410, cycle 2 is performed, and the number of cycles 2 corresponds to the number of test cases included in one test configuration file. Specifically, in step S406, the execution body creates a corresponding calling module for one of the plurality of test profiles based on the test profile for executing the subsequent calling step. In step S407, for one test case in the test configuration file, the calling module executes a call pre-processing. In step S408, the calling module calls the corresponding function module based on the configuration of the function module to be called in the test case, and inputs the input data configured in the test case into the corresponding function module in the calling process. In step S409, the functional module outputs an execution result based on the input data, processes the execution result based on the configuration of the input data in the test case, and returns a call result. In step S410, the calling module calls the call result for post-processing.
In step S411, after the cycle 2 of the plurality of test cases included in the test configuration file is completed, the node-level result statistics is performed for the corresponding node. In step S412, the calling module sends the obtained node-level result statistics to the execution body. After the end of the loop 1 for each test profile, in step S413, in the case where the loop 1 includes loops for a plurality of nodes, the result statistics at the blockchain level are performed. The part of the execution body where step S411 and step S413 are performed corresponds to a part of the output module in fig. 1.
Fig. 5 shows an automatic test equipment 500 according to an embodiment of the present specification, including:
an obtaining unit 51, configured to obtain a first test configuration file corresponding to a first object to be tested, where the first test configuration file includes at least one test case arranged in sequence, and each test case includes a call to one of a plurality of prefabricated function modules and a configuration of input data and output data of the function module, where each function module corresponds to one operation in a test; and
the calling unit 52 is configured to sequentially call the functional modules in each test case according to the arrangement sequence of the at least one test case and based on the configuration of the input data and the output data of the corresponding functional module in each test case, so as to obtain a test result corresponding to the first test configuration file.
In one embodiment, the first object to be tested is a first node in a blockchain, where the apparatus is deployed in a test server, and the apparatus further includes a connection unit 53 configured to obtain connection configuration information of the first node after obtaining a first test configuration file corresponding to the first object to be tested, and connect the first node based on the connection configuration information.
In an embodiment, the apparatus further comprises an interface unit 54 configured to convert the first test profile corresponding to the first object to be tested into a program readable form after acquiring the first test profile.
In an embodiment, the apparatus further comprises a statistics unit 55 configured to, after obtaining the test result corresponding to the first test configuration file, perform result statistics based on the test result.
Another aspect of the present description provides a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform any of the methods described above.
Another aspect of the present specification provides a computing device comprising a memory and a processor, wherein the memory has executable code stored therein, and wherein the processor, when executing the executable code, performs any of the methods described above.
By the automatic test scheme according to the embodiment of the specification, a configurable and pluggable data-driven automatic test framework is realized, and a tester only needs to configure corresponding test files when writing test cases, so that the tester does not need to be familiar with specific development languages, the writing cost of the test cases is reduced, and the test efficiency is improved.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Those of ordinary skill would further appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those of ordinary skill in the art may implement the described functionality using different approaches for each particular application, but such implementation is not considered to be beyond the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the application, and is not meant to limit the scope of the application, but to limit the application to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the application are intended to be included within the scope of the application.