CN111176989A - Fault injection test scenario case model and framework based on service arrangement - Google Patents
Fault injection test scenario case model and framework based on service arrangement Download PDFInfo
- Publication number
- CN111176989A CN111176989A CN201911332361.5A CN201911332361A CN111176989A CN 111176989 A CN111176989 A CN 111176989A CN 201911332361 A CN201911332361 A CN 201911332361A CN 111176989 A CN111176989 A CN 111176989A
- Authority
- CN
- China
- Prior art keywords
- test
- service
- component
- case
- fault injection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a fault injection test scene case model and a frame based on service arrangement, wherein the case arrangement model describes a test case into a plurality of test flows consisting of a plurality of test steps, the test steps and the test flows containing special test logics are executed by calling a servitized test component, and the frame integrates the existing micro-service arrangement component and a servitized test tool on the basis of the arrangement model. After a user transmits a test case file described by the arrangement model to the test framework, an analyzer in the test framework analyzes the test case into a workflow file and transmits the workflow file to the micro-service arrangement component, and the micro-service arrangement component calls each test service according to the workflow file to execute the test case. Compared with the prior art, the invention has the advantages of improving the testing efficiency, supporting special testing logic service and the like.
Description
Technical Field
The invention relates to the technical field of computer testing, in particular to a fault injection test scenario case model and a fault injection test scenario framework based on service arrangement.
Background
With the development of network function virtualization, containerization and other technologies, the topology of the present system depending on the cloud environment (such system is usually called cloud native system) is more complex, and its reliability faces a serious challenge. The traditional system reliability test only comprises pure fault injection, however, in order to verify the reliability of the cloud native system, a complex test scenario needs to be designed: in a test scene, not only a fault needs to be injected into a target system, but also a large amount of user pressure needs to be simulated for the target system, and meanwhile, performance monitoring data and log data of the target system also need to be continuously collected. In the process of using the existing testing tools (such as a meter, LoadRunner, WebLoad, Yardstick, and the like) to realize the testing scenario, the following problems exist:
1. different test tools are different in emphasis (such as a test tool only used for fault injection and a test tool only used for user pressure simulation), and the test tools can only be used one by one to test a target system in the test process, so that the test tools cannot be integrated together for common use, and a more comprehensive test scene is realized;
2. the existing test tool cannot embody flow logics of branching, parallel, waiting and the like, and cannot meet the requirements of a more complex test scene.
The microservice architecture is a new software architecture. Compared with the application adopting the traditional monomer architecture, the application built based on the micro-service architecture has the advantages of easiness in expansion, capability of automatic deployment and the like. By utilizing the micro-service architecture, the existing test tool can be serviced, and the complex fault injection test case is realized through service arrangement. However, in the process of implementing the servitization test by using the micro-service orchestration component (such as a conductor), the service orchestration file for executing the test case often contains a large amount of configuration information about the service, which is difficult for the tester to interpret. Furthermore, existing microservice orchestration components also fail to implement specific test logic (e.g., cyclically increasing traffic pressure during stress testing until the system fails).
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a fault injection test scenario model and a framework based on service arrangement.
The purpose of the invention can be realized by the following technical scheme:
a fault injection test scenario case model based on service arrangement comprises a test case, wherein the test case comprises a plurality of test flows, each test flow comprises a plurality of test steps, and each test step calls corresponding test component services.
Further, all the test flows can be nested and combined with each other.
Further, the types of the test flow include a general execution flow for simple sequential execution and a test logic flow for running a special test logic.
Further, the test logic flow calls a corresponding test logic service.
Further, the test cases are packaged into separate test services, and each test service presents the test function of the test service in the form of an interface.
Further, all the interfaces are uniformly called or programmed.
The invention also provides a service arrangement testing framework of the micro-service framework based on the fault injection testing scenario case model based on the service arrangement, which comprises the following steps:
the testing server is used as a front-end interactive interface based on Web, and testers can establish, edit and execute test cases through the testing server;
the case analysis module is used for converting the test case into a workflow file which can be identified by the micro-service arrangement component and sending the workflow file to the micro-service arrangement component;
the micro-service arranging component is used for receiving the workflow file sent by the case analysis module, calling the testing component service according to the execution sequence of the workflow, and sending the testing logic flow which is not supported by the testing component service to the testing logic service;
the test logic service is used for executing the specified test logic, extracting the sub test flow which can be executed by the arranging component in the test logic flow, and sending the sub test flow to the test service end to trigger the execution of the sub test flow;
the test component services are used as the serviced test tools, and each test component service provides a single test function required in the test process.
Further, the test functions include fault injection, pressure simulation and monitoring.
Compared with the prior art, the invention has the following advantages:
(1) modeling a serviced test scenario. Configuration information in the service arrangement test case is simplified through the test case arrangement model, the execution process of the test case is clearer in the test case file, and the readability of the test case is enhanced.
(2) The test logic service is utilized to strengthen the arrangement function, so that the test framework can execute the test logic which is not supported in the micro-service arrangement component at present.
(3) The test component, the test logic, the micro-service orchestration component are decoupled from the test framework. When a test frame needs to replace a certain test service, other test services can be directly reused without being customized again, so that the test cost is saved, and the test efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of a test case orchestration model according to the present invention;
FIG. 2 is a diagram of a test framework corresponding to a trial layout model according to the present invention;
fig. 3 is a schematic diagram of a test case execution scheme of test logic servitization according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
The invention provides a test tool and a test logic service test mode:
(a) the test tool required to be used is packaged into separate test services in the process of the cloud native system fault injection test, each test service exposes the test function of the test tool in the form of an interface, and the test tool is called or arranged by the test framework in a unified mode through the interfaces.
(b) For special test logic that is not supported by the existing micro-service orchestration framework (e.g., increasing user pressure cyclically until the target system fails), the test logic is implemented by code independent of the orchestration framework, encapsulating it into a test logic service and exposing a test logic call interface. Test logic may be nested into the normal service call execution flow to implement a special test scenario.
Based on the service test assembly and the test logic, a test case arrangement model is provided. The tester can use the model to describe test cases containing complex fault injection test scenarios. The composition of the layout model is shown in FIG. 1:
(a) the test process of a test case is divided into a plurality of test flows, wherein the test flows are divided into two types, namely, a common execution flow which is simply executed in sequence, and a test logic flow which comprises complex test logic.
(b) The test flows can be nested and combined with each other.
(c) The test flow contains specific test steps, each of which will invoke a test component service.
(d) The execution of the test logic flow will invoke a test logic service that specifically implements the specific test step execution logic in the test logic flow.
Based on the test case layout model, a service layout test framework of the micro-service architecture is proposed, as shown in fig. 2. The test framework comprises the following components:
testing the service end: the front-end interactive interface based on Web is characterized in that a tester can realize the functions of creating a test case, editing the test case, executing the test case and the like through a test server;
a use case analysis module: for a test case to be executed sent by a test server, a case analysis module converts the test case into a workflow file which can be identified by the micro-service arrangement component and sends the workflow file to the micro-service arrangement component;
a microservice orchestration component: the micro-service arranging component receives the workflow file sent by the case analyzing module, calls the testing component service according to the execution sequence of the workflow, and sends the testing logic flow which is not supported by the testing component service to the testing logic service;
testing the logic service: the test logic service is used for executing the specified test logic, extracting the sub-test flow which can be executed by the arranging component in the logic test flow, and sending the sub-test flow to the test service terminal to trigger the execution of the sub-test flow.
Testing component service: the serviced test tool, a test component service, provides certain test functions required in the test process, such as fault injection, pressure simulation, monitoring, etc.
Practical examples:
a complex fault injection test scene in a cloud native system is described through a test case arrangement model based on service arrangement, and a corresponding test framework is provided to realize the execution process of the test case. FIG. 3 is a process of a test framework executing a test case, where the test case describes a test scenario including a normal stream (task1) and a special test logic stream (loop: task2), and the specific steps of the test framework executing the test case are described as follows:
(1) a user requests to execute a test case describing a complex fault injection test scene;
(2) the test server sends the test case file to a case analysis module;
(3) the case analysis module converts the test cases into workflow files and sends the workflow files to the microservice arrangement component;
(4) the micro-service arranging component reads the step of task1 in the test process and calls the corresponding test component to execute the service;
(5) the micro-service arranging component reads the test flow (the circular execution task2) containing the special test logic and calls the corresponding test logic service;
(6) reading the test stream by a test stream analysis file in the test logic service and registering a sub-test stream (task2) which can be executed by the micro-service orchestration component in the test stream as a sub-test case into a test framework;
(7) the test logic service executes the sub-test flow according to the test logic, and executes task2 circularly;
(8) the test server sends the sub-test case file to the case analysis module;
(9) the case analysis module converts the sub-test cases into workflow files and sends the workflow files to the micro-service arrangement component;
(10) the microservice orchestration component reads the step of task2 in the workflow file and invokes the corresponding test component service execution.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A fault injection test scenario case model based on service orchestration is characterized by comprising a test case, wherein the test case comprises a plurality of test flows, each test flow comprises a plurality of test steps, and each test step calls a corresponding test component service.
2. The service orchestration based fault injection test scenario case model according to claim 1, wherein all the test flows can be nested and combined with each other.
3. The service orchestration based fault injection test scenario case model according to claim 1, wherein the types of test flows comprise a normal execution flow for simple sequential execution and a test logic flow for running special test logic.
4. The service orchestration based fault injection test scenario case model according to claim 3, wherein the test logic flow invokes a corresponding test logic service.
5. The service orchestration based fault injection test scenario case model according to claim 1, wherein the test cases are packaged into separate test services, and each test service exposes its own test function in the form of an interface.
6. The service orchestration based fault injection test scenario case model according to claim 5, wherein all the interfaces are uniformly invoked or orchestrated.
7. A service orchestration testing framework based on the micro service architecture of the service orchestration based fault injection test scenario case model according to any one of claims 1-6, the testing framework comprising:
the testing server is used as a front-end interactive interface based on Web, and testers can establish, edit and execute test cases through the testing server;
the case analysis module is used for converting the test case into a workflow file which can be identified by the micro-service arrangement component and sending the workflow file to the micro-service arrangement component;
the micro-service arranging component is used for receiving the workflow file sent by the case analysis module, calling the testing component service according to the execution sequence of the workflow, and sending the testing logic flow which is not supported by the testing component service to the testing logic service;
the test logic service is used for executing the specified test logic, extracting the sub test flow which can be executed by the arranging component in the test logic flow, and sending the sub test flow to the test service end to trigger the execution of the sub test flow;
the test component services are used as the serviced test tools, and each test component service provides a single test function required in the test process.
8. The service orchestration testing framework according to the micro service architecture of the service orchestration-based fault injection test scenario case model of claim 7, wherein the testing functions include fault injection, pressure simulation, and monitoring.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332361.5A CN111176989A (en) | 2019-12-22 | 2019-12-22 | Fault injection test scenario case model and framework based on service arrangement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911332361.5A CN111176989A (en) | 2019-12-22 | 2019-12-22 | Fault injection test scenario case model and framework based on service arrangement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111176989A true CN111176989A (en) | 2020-05-19 |
Family
ID=70657461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911332361.5A Pending CN111176989A (en) | 2019-12-22 | 2019-12-22 | Fault injection test scenario case model and framework based on service arrangement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111176989A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905486A (en) * | 2021-03-26 | 2021-06-04 | 建信金融科技有限责任公司 | Service integration test method, device and system |
| CN113285849A (en) * | 2021-04-06 | 2021-08-20 | 武汉虹信科技发展有限责任公司 | Equipment testing method and device based on microservice |
| CN117271359A (en) * | 2023-10-12 | 2023-12-22 | 润芯微软件(南京)有限公司 | Automatic test system and method for application scenes of various clients |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101212362A (en) * | 2006-12-26 | 2008-07-02 | 中兴通讯股份有限公司 | Automatic testing device and method incorporating a variety of testing tools |
| CN101739334A (en) * | 2008-11-21 | 2010-06-16 | 上海科梁信息工程有限公司 | Automatic testing method of embedded software |
| CN108874650A (en) * | 2017-05-09 | 2018-11-23 | 上海秦苍信息科技有限公司 | A kind of continuous integrating automated testing method |
| CN109559583A (en) * | 2017-09-27 | 2019-04-02 | 华为技术有限公司 | Failure simulation method and its device |
-
2019
- 2019-12-22 CN CN201911332361.5A patent/CN111176989A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101212362A (en) * | 2006-12-26 | 2008-07-02 | 中兴通讯股份有限公司 | Automatic testing device and method incorporating a variety of testing tools |
| CN101739334A (en) * | 2008-11-21 | 2010-06-16 | 上海科梁信息工程有限公司 | Automatic testing method of embedded software |
| CN108874650A (en) * | 2017-05-09 | 2018-11-23 | 上海秦苍信息科技有限公司 | A kind of continuous integrating automated testing method |
| CN109559583A (en) * | 2017-09-27 | 2019-04-02 | 华为技术有限公司 | Failure simulation method and its device |
Non-Patent Citations (1)
| Title |
|---|
| 王奇等: "基于SaaS的自动化测试云平台研究", 《电信工程技术与标准化》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905486A (en) * | 2021-03-26 | 2021-06-04 | 建信金融科技有限责任公司 | Service integration test method, device and system |
| CN113285849A (en) * | 2021-04-06 | 2021-08-20 | 武汉虹信科技发展有限责任公司 | Equipment testing method and device based on microservice |
| CN117271359A (en) * | 2023-10-12 | 2023-12-22 | 润芯微软件(南京)有限公司 | Automatic test system and method for application scenes of various clients |
| CN117271359B (en) * | 2023-10-12 | 2024-03-19 | 润芯微软件(南京)有限公司 | Automatic test system and method for application scenes of various clients |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109933522B (en) | Test method, test system and storage medium for automatic case | |
| CN107704392B (en) | Test case processing method and server | |
| CN111176989A (en) | Fault injection test scenario case model and framework based on service arrangement | |
| CN102141962B (en) | Safety distributed test framework system and test method thereof | |
| CN107506303A (en) | Method, apparatus and system for automatic test | |
| CN101382915B (en) | Software debugging system and debugging method | |
| CN102752770B (en) | Method and device for polling service system | |
| CN104462943A (en) | Non-intrusive performance monitoring device and method for service system | |
| KR20210042713A (en) | Test apparatus to test interoperability of nfv system | |
| CN107122307B (en) | Internet of things execution system | |
| CN113360386B (en) | Method and device for testing driving of switching chip, electronic equipment and storage medium | |
| CN112749083A (en) | Test script generation method and device | |
| CN109522181A (en) | A kind of performance test methods of distributed memory system, device and equipment | |
| CN105550103B (en) | A kind of automated testing method based on self-defined test script | |
| CN101609427B (en) | Programmable network service automation test system and programmable network service automation method | |
| CN105760300B (en) | A kind of automated testing method and test macro of STK/UTK business | |
| CN112583663B (en) | Automatic network performance testing method based on domestic operating system | |
| WO2005082072A2 (en) | Testing web services workflow using web service tester | |
| CN109902010A (en) | The method and system that API is tested under a kind of KVM framework | |
| CN101916221A (en) | Method and system for performing unit testing on equipment | |
| CN115668895A (en) | Resource configuration determining method and device of cloud service system | |
| CN101227349A (en) | Device and method for automatically testing system change | |
| CN109857576A (en) | RPC interface call method and device | |
| CN116450234A (en) | C++ model library service packaging framework system and method | |
| CN101876936A (en) | Service test method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |