CN114896108A - Test verification system - Google Patents
Test verification system Download PDFInfo
- Publication number
- CN114896108A CN114896108A CN202210534702.2A CN202210534702A CN114896108A CN 114896108 A CN114896108 A CN 114896108A CN 202210534702 A CN202210534702 A CN 202210534702A CN 114896108 A CN114896108 A CN 114896108A
- Authority
- CN
- China
- Prior art keywords
- test
- simulation
- service
- data
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 373
- 238000012795 verification Methods 0.000 title claims abstract description 72
- 238000004088 simulation Methods 0.000 claims abstract description 217
- 238000000034 method Methods 0.000 claims abstract description 23
- 230000008569 process Effects 0.000 claims abstract description 13
- 238000007726 management method Methods 0.000 claims description 69
- 238000010200 validation analysis Methods 0.000 claims description 20
- 230000002567 autonomic effect Effects 0.000 claims description 14
- 238000011161 development Methods 0.000 claims description 12
- 238000013523 data management Methods 0.000 claims description 10
- 238000012423 maintenance Methods 0.000 claims description 5
- 238000007667 floating Methods 0.000 claims description 4
- 238000012546 transfer Methods 0.000 claims description 4
- 230000007613 environmental effect Effects 0.000 claims description 3
- 238000011056 performance test Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 238000013439 planning Methods 0.000 description 10
- 238000013461 design Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 238000004806 packaging method and process Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011990 functional testing Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 235000019580 granularity Nutrition 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005191 phase separation Methods 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/26—Functional testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3447—Performance evaluation by modeling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3457—Performance evaluation by simulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Environmental & Geological Engineering (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a test verification system, which at least comprises: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem; the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement; the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result; the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results, verifying technical indexes of hardware and software in the test process, and flexibly assembling models with different resolutions, so that vivid service scenes meeting different types such as function tests, performance tests, service capability tests and the like can be quickly constructed, and the accuracy of test results is greatly improved.
Description
Technical Field
The invention relates to the technical field of communication, in particular to a test verification system.
Background
The complex information system test usually designs many service fields, and needs to be closely matched and cooperatively designed among multiple domains. At present, the test design software in a single field is very mature, but is difficult to get through and cooperate with other fields.
Complex information system testing requires many resources involved and is largely a continuation of previously developed assets. Software and hardware developed at different stages are different in aspects such as technical systems, system architectures and implementation modes, and generally need to be individually adapted and developed for each resource, so that the workload is very large, and interfaces for managing and controlling heterogeneous resources are not uniform, so that the requirement on a scheduling center is very high, and the communication requirements of different resources are difficult to meet. In addition, in the test development process, fine-grained instruction level management and control needs to be performed on various resources, and the management and control difficulty is increased.
Disclosure of Invention
The invention aims to provide a test verification system to solve the defects in the prior art, and the technical problem to be solved by the invention is realized by the following technical scheme.
The embodiment of the invention provides a test verification system, which at least comprises: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem;
the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement;
the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result;
the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results and verifying technical indexes of hardware and software in the test process.
Optionally, the universal business simulation system is further configured to collect test data and send the test data to a test management control system.
Optionally, the simulation service application and support subsystem at least comprises a central task simulation unit, a daily application simulation unit and a test resource scheduling management tool unit; wherein,
the central task simulation unit is used for simulating various scheduling services and control tasks of the test management control system;
the daily application simulation unit is used for simulating daily business generated by daily development of each post, and the daily business at least comprises one or more of file transfer, mail sending or information query;
the test resource scheduling management tool unit is used for determining test verification service information according to the load degree of the simulation system, wherein the verification service information at least comprises the test scheme and the set-up function of the test environment, and managing and scheduling various resources to be tested.
Optionally, the simulated data environment and test validation subsystem includes a simulated data environment module and a test validation module.
Optionally, the simulation data environment module at least includes a simulation base environment data management unit and a test environment data access unit, where:
the simulation basic environment data management system unit is used for managing simulation basic environment data, wherein the simulation basic environment data at least comprises one or more of sites, organizations, personnel, facilities, devices, equipment and key objects;
the test environment data access unit is used for providing data unified access service.
Optionally, the test validation module includes at least: the system comprises a service capability test verification unit, an autonomous computer comprehensive test tool set unit and an autonomous computing service test tool unit, wherein:
the service capability test verification unit is used for determining a service capability test verification software toolset according to typical service capability verification requirements of systems under different application modes;
the integrated test tool set unit of the self-host computer is used for testing the delay and bandwidth, the fixed point and floating point capability, the input and output port performance, the network performance and the data access bandwidth of the computer system and evaluating the device level, the board level and the performance of the whole computer of different types;
the autonomic computing service test tool unit is used for testing and verifying the resource virtualization management scheduling capability, the damage-resistant disaster-tolerant capability, the demand deployment capability, the opening and withdrawing capability and the maintenance and guarantee capability of the computing service guarantee system based on the hardware platform.
Optionally, the environmental data of the simulation operation includes a test network and a service network, where the service network at least includes a physical network and a simulation network;
the test resource scheduling management tool unit, the simulation data environment module and the test verification module are respectively interconnected through a test network;
the general service simulation subsystem, the central task simulation unit, the daily application simulation unit and the situation handling simulation unit are at least provided with two network ports which are respectively interconnected with the test network and the service network.
Optionally, the test verification system at least includes a network simulation and capability test layer, an autonomic computing storage platform test layer, a business application simulation layer, and a business capability and basic service test layer.
Optionally, the autonomic computing storage platform test layer at least comprises commercial computer/communication equipment, autonomic computer/communication equipment and computing storage equipment of different systems of the autonomic computing service platform.
Optionally, the test verification system at least comprises a system under test, an accompanying system and a test system; wherein,
the system to be tested comprises a real service system;
the accompanying and testing system at least comprises a general service simulation subsystem, a simulation data basic environment module, a central task simulation unit, a daily application simulation unit and a situation handling simulation unit;
the test system comprises a service capability test verification system.
The embodiment of the invention has the following advantages:
the test verification system provided by the embodiment of the invention at least comprises: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem; the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement; the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result; the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results, verifying technical indexes of hardware and software in the test process, and flexibly assembling models with different resolutions, so that different types of vivid service scenes meeting the requirements of functional tests, performance tests, service capability tests and the like can be quickly constructed, and the accuracy of test results is greatly improved.
Drawings
FIG. 1 is a schematic structural diagram of a test verification system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a complex information system comprehensive test verification system provided in an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a command simulation and command test system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a typical service capability test deployment structure provided in the embodiment of the present invention;
FIG. 5 is a diagram illustrating an exemplary deployment structure of a service capability test provided in an embodiment of the present invention;
fig. 6 is a schematic diagram of an exemplary workflow provided by an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
Referring to fig. 1, a schematic structural diagram of an embodiment of a test verification system of the present invention is shown, where the test verification system at least includes: a general service simulation subsystem 101, a simulation service application and support subsystem 102, a simulation data environment and a test verification subsystem 103;
specifically, the embodiment of the present invention is applied to a complex information system, where a complex information system test needs many resources, and most of the complex information system tests use assets developed before, software and hardware developed at different stages, which are different in aspects such as technical system, system architecture, implementation manner, etc., and usually need to be adapted and developed individually for each resource, and the workload is very large, and interfaces for controlling these heterogeneous resources are not uniform, so that the requirement on the center is very high, it is difficult to meet communication requirements of different resources, and it is difficult to perform service capability test verification quickly and accurately, that is, problems of difficult interconnection and intercommunication interoperation, weak collaborative planning design capability, low level of process hierarchical fine-grained control, poor fidelity of service scene simulation structure, large data acquisition interference loss, and the like in a process of testing and verifying service capability of a complex information system under an autonomous environment, it is difficult to complete the comprehensive test of the function performance index, boundary condition, system conformity and service capability of the complex information system under the environment close to the real service operation environment.
In view of the above, embodiments of the present invention provide a test verification system, that is, in a complex information system, a flexible communication middleware with a multi-mode hybrid architecture is added, a multi-domain collaborative test planning technique based on workflow and model-driven ideas, a service modeling and uniform interface packaging of test resources, a service scene supporting multi-resolution may assemble a simulation framework, and a multi-source data lossless low-interference acquisition technique are supported, so as to effectively solve the above technical problems, where the middleware is a key part for solving heterogeneous resource access and eliminating resource interconnection and intercommunication performance bottlenecks. The middleware is based on a cloud test platform, is designed based on a multi-mode mixed flexible architecture, provides an advanced resource service modeling and packaging mode, and can effectively improve the interconnection, intercommunication and interoperation capabilities of various LVC heterogeneous test resources. The method specifically comprises the following steps:
the universal service simulation system 101 is used for determining the service load of the universal service flow with regular generating capacity according to the network test requirement;
specifically, the universal business simulation system is further used for collecting test data and sending the test data to the test management control system.
The universal service simulation system mainly aims at network testing, provides the generation capacity of universal service flow, can directly receive the control of a test management control system, and generates service load meeting the specified distribution rule according to the description of a test scheme. In addition, the universal business simulation system also has the functions of acquiring and reporting experimental data.
The simulation service application and support subsystem 102 is used for simulating the operation of the test management control system and the daily service operation, and acquiring a simulation test result;
optionally, the simulation service application and support subsystem at least comprises a central task simulation unit, a daily application simulation unit and a test resource scheduling management tool unit; wherein,
the central task simulation unit is used for simulating various scheduling services and control tasks of the test management control system;
the daily application simulation unit is used for simulating daily business generated by daily development of each post, and the daily business at least comprises one or more of file transmission, mail sending or information query;
the test resource scheduling management tool unit is used for determining test verification service information according to the load degree of the simulation system, wherein the verification service information at least comprises the set-up functions of a test scheme and a test environment, and managing and scheduling various resources to be tested.
Specifically, the simulation service application and support system mainly comprises a central task simulation tool, a daily application simulation tool and a test resource scheduling management tool. Wherein:
the central task simulator is mainly responsible for simulating various scheduling, control tasks and other related services of the management center;
the daily application simulator is mainly responsible for simulating relevant daily services such as file transfer, mail sending, information query and the like generated by the development of daily services of each post;
the test management tool mainly provides test verification common basic service aiming at the characteristics of a complex information system test, on the basis, provides the functions of design and refinement of a test scheme and establishment of a test environment, provides various professional test scheduling engines, and can receive a control command of a test management control system, so that the unified management and scheduling of various business simulators, data collectors and other reference resources are realized, and the collection, analysis and display of test data are finally completed.
The simulation data environment and test verification subsystem 103 is used for managing environment data for executing simulation operation and simulation test results, and verifying technical indexes of hardware and software in the test process.
Specifically, the simulated data environment and test validation subsystem comprises a simulated data environment module and a test validation module.
Optionally, the simulation data environment module at least includes a simulation base environment data management unit and a test environment data access unit, wherein:
the simulation basic environment data management system unit is used for managing simulation basic environment data, wherein the simulation basic environment data at least comprises one or more of sites, organizations, personnel, facilities, devices, equipment and key objects;
the test environment data access unit is used for providing data unified access service.
Optionally, the test validation module comprises at least: the system comprises a service capability test verification unit, an autonomous computer comprehensive test tool set unit and an autonomous computing service test tool unit, wherein:
the service capability test verification unit is used for determining a service capability test verification software toolset according to typical service capability verification requirements of systems in different application modes;
the self-host computer comprehensive test tool set unit is used for testing the delay and bandwidth, the fixed point and floating point capability, the input and output port performance, the network performance and the data access bandwidth of a computer system and evaluating the performance of a device level, a board level and a complete machine of computers of different types;
the autonomic computing service test tool unit is used for testing and verifying the resource virtualization management scheduling capability, the damage-resistant disaster-tolerant capability, the demand deployment capability, the opening and withdrawing capability and the maintenance support capability of the computing service support system based on the hardware platform.
The test verification system provided by the embodiment of the invention at least comprises: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem; the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement; the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result; the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results, verifying technical indexes of hardware and software in the test process, and flexibly assembling models with different resolutions, so that different types of vivid service scenes meeting the requirements of functional tests, performance tests, service capability tests and the like can be quickly constructed, and the accuracy of test results is greatly improved.
Fig. 2 is a schematic structural diagram of a complex information system comprehensive test verification system according to an embodiment of the present invention, and according to a construction target of the complex information system test verification system, a test experiment capable of supporting three layers of a service capability and basic service test, a network-to-service carrying capability test, and an autonomic computing storage platform test is required. In order to better support the development of the test by comprehensively considering the characteristics of the three different types of test tests, a system architecture as shown in fig. 2 is adopted.
The complex information system test and verification system architecture comprises a network simulation and capability test layer, an autonomic computing storage platform test layer, a business application simulation layer, a business capability and basic service test layer, a test management center for providing unified management and control for the layers and a simulated data environment for providing data guarantee support for the layers.
The core of the network simulation and capability test layer is a simulation network and related systems for providing simulation test support for the simulation network. The simulation network mainly provides a basic network bearing environment for test development, and various simulated service applications run on the simulation network and aim at simulating an application system of the network.
The test layer of the autonomous computing storage platform mainly comprises a hardware bearing platform consisting of computing storage devices of different systems, such as a commercial computer/communication device, an autonomous computing service platform and the like, and on the basis, the test layer provides an autonomous computer comprehensive test tool and an autonomous computing service test tool to complete comprehensive tests on the device level, the board level and the overall machine performance of the autonomous computer and the autonomous computing service.
The service application simulation layer mainly comprises various simulated service applications and a service support system, wherein the general service simulation system mainly provides general service load for network test and simulates the traffic load operation condition borne by a network in coarse granularity; the simulation service application and support system mainly provides simulation of services such as a central task, daily services, situation handling and the like. The two methods provide simulation service support for network test and system test together, thereby generating service data according with the operation characteristics of a real service system. In addition, the layer also provides a test management tool which can receive a control command of a test management center and carry out unified deployment, management and scheduling on various levels of simulation service applications.
The business capability and basic service testing layer mainly comprises a business capability comprehensive test verification system, so that a unified design and planning function of a test is provided for testers, and the acquisition, evaluation and test result display of test data are completed.
The simulation data environment comprises a simulation data environment data management system and a test environment access service system, wherein the simulation data environment data management system mainly provides a unified management function and a simulation service for simulation data environment data, and the test environment access service system mainly provides a unified access service for test data, so that environment data support is provided for normal operation of other subsystems such as a simulation business application and support system, a simulation network environment and the like.
Fig. 3 is a schematic structural diagram of a command simulation and command test system according to an embodiment of the present invention, and as shown in fig. 3, the command simulation and command test system mainly includes three subsystems, namely, a general service simulation system (general service simulation subsystem), a simulation service application and support system (simulation service application and support subsystem), and a simulation data environment and a test verification system (simulation data environment and test verification subsystem).
The universal service simulation system mainly aims at network testing, provides the generation capacity of universal service flow, can directly receive the control of a test management control system, and generates service load meeting the specified distribution rule according to the description of a test scheme. In addition, the universal business simulation system also has the functions of acquiring and reporting experimental data.
The simulation service application and support system mainly comprises a central task simulation unit, a daily application simulation unit and a test resource scheduling management tool unit. Wherein:
the central task simulator (central task simulation unit) is mainly responsible for simulating various scheduling, control tasks and other related services of the management center;
the daily application simulator (daily application simulation unit) is mainly responsible for simulating relevant daily services such as file transfer, mail sending, information query and the like generated by the development of daily services of each post;
the test management tool (test resource scheduling management tool unit) mainly provides test verification common basic service aiming at the characteristics of the test of the complex information system, on the basis, provides the design and refinement of a test scheme and the establishment function of a test environment, provides various professional test scheduling engines, and can receive the control command of a test management control system, thereby realizing the unified management and scheduling of various business simulators, data collectors and other reference resources, and finally completing the collection, analysis and display of test data.
The simulation data environment and test verification system comprises a simulation data environment module and a test verification module, wherein the simulation data environment module mainly comprises a simulation basic environment data management system (unit) and a test environment data access system (unit). Wherein:
the simulation basic environment data management system (unit) provides unified management of basic simulation basic environment data such as sites, organizations, personnel, facilities, devices, equipment, key objects and the like, and real-time calculation and on-demand simulation generation of dynamic simulation basic environment data.
The test environment data access system (unit) comprises a data unified access service, and mainly provides a unified access and acquisition means for test data for other systems in a target range.
The test and verification module mainly comprises a service capability test and verification system (unit), an autonomous computer comprehensive test tool set (unit) and an autonomous computing service test tool (unit). Wherein:
the service capability test verification system (unit) mainly meets the typical service capability verification requirement of the system in a typical application mode, constructs a reconfigurable and extensible service capability test verification software toolset, and supports the development of comprehensive test and verification of typical service capabilities such as the opening, operation and maintenance capability, management and scheduling capability, condition handling capability, information sharing service capability and the like of the system under complex conditions, so that the overall capability of the system expressed under the condition close to real operation is verified;
the self-host computer comprehensive test tool set (unit) mainly completes the comprehensive test of the aspects of computer system delay and bandwidth, fixed point and floating point capability, I/O performance, Web performance, data access bandwidth and the like, and realizes the performance evaluation of the device level, the board level and the whole machine of a commercial/domestic computer;
the autonomous computing service testing tool (unit) mainly realizes the testing and verification of core technical indexes such as resource virtualization management scheduling capability, damage resistance and disaster tolerance capability, on-demand deployment capability, opening and withdrawing capability, maintenance and guarantee capability and the like of a computing service guarantee system based on a domestic hardware platform.
Fig. 4 is a schematic diagram of a typical service capability test deployment structure provided in the embodiment of the present invention; in order to test the service capability, firstly, a typical application scene with a certain scale and meeting typical application characteristics of the service field is constructed based on a general service simulation system, a simulation data base environment, a central task simulation, a daily application simulation, a situation handling simulation and a related tested real system, a typical service environment of a multi-stage mechanism from a center to a department to a working post is simulated, on the basis, a semi-physical simulation test under the typical application scene is designed and developed, a field typical application is loaded, a typical service flow is executed, and test data required by service capability judgment is collected according to the requirement of a test task; and finally, selecting a corresponding service capability test model and operator according to the characteristics of the service capability index system, loading corresponding test data for analysis and judgment, and finally obtaining a test result, thereby completing the comprehensive test of the typical service capability of the information system and further verifying the overall capability of the system expressed under the condition close to the real service.
Typical service capability test deployment structures are shown in fig. 4 and 5.
In this embodiment, the environmental data of the simulated operation includes a test network and a service network, wherein the service network at least includes a physical network and a simulation network; the test resource scheduling management tool unit, the simulation data environment module and the test verification module are respectively interconnected through a test network; the general service simulation subsystem, the central task simulation unit, the daily application simulation unit and the situation handling simulation unit are at least provided with two network ports which are respectively interconnected with the test network and the service network.
The test verification system at least comprises a network simulation and capability test layer, an autonomic computing storage platform test layer, a business application simulation layer and a business capability and basic service test layer.
The test layer of the autonomous computing storage platform at least comprises commercial computers/communication equipment, autonomous computers/communication equipment and computing storage equipment of different systems of the autonomous computing service platform.
Specifically, the test verification system at least comprises a tested system, an accompanying test system and a test system; wherein,
the tested system comprises a real service system;
the accompanying and testing system at least comprises a general service simulation subsystem, a simulation data basic environment module, a central task simulation unit, a daily application simulation unit and a situation handling simulation unit;
the test system comprises a service capability test verification system.
The test environment consists of a test network and a service network, wherein the service network consists of a physical network and a simulation network in order to really build a huge service operation support network supporting service operation. The test management control system, the test resource scheduling management tool, the simulation data environment and the service capability test system are interconnected through a test network, and the universal service simulation system, the central task simulation system, the daily application simulation system, the situation handling simulation system and the related tested real system are required to be provided with at least two network ports which are respectively interconnected with the test network and the service network. The above systems participating in the test can be divided into three major categories, namely, a system under test, an accompanying test system and a test system. The tested system is mainly a real service system which is operated on line, and the test accompanying system mainly comprises a general service simulation system, a simulated data base environment, a central task simulation system, a daily application simulation system, a situation handling simulation system and the like, which jointly construct a typical application scene for testing the capability of the tested system. The test system is mainly a service capability test verification system, and forms a service system test environment together with the tested system and the accompanying system. By deploying the test data acquisition tool of the service capability test verification system to the terminal where the tested system is located, the test data can be acquired, processed, analyzed and judged, so that the typical service capability of the service system can be comprehensively tested.
For the problem that a complex information system is difficult to test and verify service capability quickly and accurately in the prior art, the embodiment of the invention provides a flexible communication middleware with a multi-mode mixed architecture, a multi-domain collaborative test planning technology based on workflow and model driving ideas, a service modeling and unified interface packaging of test resources, a service scene supporting multi-resolution ratio, an assembled simulation framework and a multi-source data nondestructive low-interference acquisition technology, and the technical problems are effectively solved.
The heterogeneous resources provided by the embodiment of the invention can be rapidly accessed to the test environment, and can be automatically and dynamically adapted to the optimal communication channel, so that the efficient communication among the resources is realized; by providing meta-model definitions of a unified task scene model, a test flow model and a resource deployment model, multi-domain collaborative design based on workflow is realized, and a planned model can be automatically generated into an executable script; the method realizes the full-autonomous on-demand set-up, on-demand elastic expansion and resource recovery of the complex test, greatly improves the test development efficiency and saves resources; by flexibly assembling the models with different resolutions, vivid service scenes meeting different types such as function tests, performance tests, service capability tests and the like can be quickly constructed, and the accuracy of test results is greatly improved; the data acquisition can be carried out under the condition of not modifying the system, the acquisition and the calculation and the phase separation are realized by the semantic extraction of the acquired data, and the resource overhead and the influence are reduced.
FIG. 6 is a schematic diagram of an exemplary workflow provided by an embodiment of the present invention, including scenario formulation, environment configuration, execution and monitoring, and result analysis;
the test management and control system, the test resource scheduling management system and the simulation resource respectively execute the steps; the simulation resources comprise a general service simulator, a central service simulation and the like.
The test management and control system executes a scheme making step, a command control test scheme is sent to the test resource scheduling management system, and the test resource scheduling management system carries out test scheme refinement according to the command control test scheme to generate a command control test script;
the test management and control system executes a test environment configuration step, and sends the instruction control test script to the test resource scheduling management system, the test resource scheduling management system automatically configures the environment according to the instruction control test script, sends the obtained working parameters to the simulation resources, the simulation purple cloud configures the parameters according to the working parameters, and returns the configuration result to the test resource scheduling management system, and the test resource scheduling management system then configures the configuration result, namely, the configuration result return value test management and control system;
the test management and control system performs an execution monitoring step, and sends the command test script to the test resource scheduling management system, the test resource scheduling management system performs test scheduling management according to the command test script, namely, performs activity coordination on each simulation resource to enable each simulation resource to generate a simulation service, and returns a test state to the test resource scheduling management system, and the test resource scheduling management system returns the test state to the test management and control system;
in the last step, when each simulation resource performs service simulation, index data and collected data are generated and sent to a test resource scheduling management system, the test resource scheduling management system performs cross-node index calculation on the data, for example, performance index calculation, consistency index calculation and flow index calculation are included to obtain index data, the index data are sent to the test management control system, and the test management control system performs result analysis and display after receiving the index data.
The embodiment of the invention designs a flexible communication middleware with a multimode hybrid architecture, which comprises the following steps: the middleware is a key part for solving heterogeneous resource access and eliminating resource interconnection performance bottleneck. The middleware is based on a cloud test platform, is designed based on a multi-mode mixed flexible architecture, provides an advanced resource service modeling and packaging mode, and can effectively improve the interconnection, intercommunication and interoperation capabilities of various LVC heterogeneous test resources.
A multi-domain collaborative test planning technology based on workflow and model driving ideas: by adopting the workflow idea, the multi-field collaborative planning and activity scheduling can be flexibly arranged by utilizing the workflow modeling; by adopting a model driving idea, a professional model and a shared model are communicated in the whole planning process, the meta-model definitions of a task scene model, a test flow model and a resource deployment model with unified standards are provided, and the planning reusability is improved. And the planning model automatically generates an executable script by supporting, so that the planning and design capability of the multi-domain collaborative test scheme based on the workflow is realized.
The method comprises the following steps of testing the service modeling and uniform interface packaging of resources: a unified service modeling and packaging mode for heterogeneous resources is designed, unified type definition and resource service modeling can be provided, automatic generation and standardized packaging of resource access codes are supported, and resource development integration efficiency and interoperability can be greatly improved; and (3) according to main activities of each stage of test development, defining a unified API interface set, and providing six standard services of node registration, resource deployment, activity coordination, data interaction, state query and clock management.
A service scene supporting multiple resolutions can assemble a simulation framework: the method realizes the separation simulation of the application layer and the transmission layer, supports different granularities, supports the flexible assembly of models, and can quickly meet the requirements of different types of test tasks.
The non-destructive low-interference acquisition technology of multi-source data comprises the following steps: the method supports acquisition without system modification, realizes acquisition and calculation and phase separation by semantic extraction of acquired data, realizes network transmission based on data compression, and reduces resource overhead.
The test verification system provided by the embodiment of the invention at least comprises: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem; the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement; the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result; the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results, verifying technical indexes of hardware and software in the test process, and flexibly assembling models with different resolutions, so that different types of vivid service scenes meeting the requirements of functional tests, performance tests, service capability tests and the like can be quickly constructed, and the accuracy of test results is greatly improved.
It should be noted that the above detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than those illustrated or otherwise described herein.
Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.
Spatially relative terms, such as "above … …," "above … …," "above … … surface," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may also be oriented in other different ways, such as by rotating it 90 degrees or at other orientations, and the spatially relative descriptors used herein interpreted accordingly.
In the foregoing detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like numerals typically identify like components, unless context dictates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A test validation system, the test validation system comprising at least: the system comprises a general service simulation subsystem, a simulation service application and support subsystem, a simulation data environment and a test verification subsystem;
the universal service simulation system is used for determining the service load of the universal service flow generating capacity rule according to the network test requirement;
the simulation service application and support subsystem is used for simulating the operation of the test management control system and the daily service operation and acquiring a simulation test result;
the simulation data environment and test verification subsystem is used for managing environment data for executing simulation operation and simulation test results and verifying technical indexes of hardware and software in the test process.
2. The test validation system of claim 1, wherein the generic traffic simulation system is further configured to collect test data and send the test data to a test management control system.
3. The test validation system of claim 1, wherein the simulation business application and support subsystem comprises at least a central task simulation unit, a daily application simulation unit, and a trial resource scheduling management tool unit; wherein,
the central task simulation unit is used for simulating various scheduling services and control tasks of the test management control system;
the daily application simulation unit is used for simulating daily business generated by daily development of each post, and the daily business at least comprises one or more of file transfer, mail sending or information query;
the test resource scheduling management tool unit is used for determining test verification service information according to the load degree of the simulation system, wherein the verification service information at least comprises the test scheme and the set-up function of the test environment, and managing and scheduling various resources to be tested.
4. The test validation system of claim 1, wherein the simulated data environment and test validation subsystem comprises a simulated data environment module and a test validation module.
5. The test validation system of claim 4, wherein the simulated data environment module comprises at least a simulated base environment data management unit and a test environment data access unit, wherein:
the simulation basic environment data management system unit is used for managing simulation basic environment data, wherein the simulation basic environment data at least comprises one or more of sites, organizations, personnel, facilities, devices, equipment and key objects;
the test environment data access unit is used for providing data unified access service.
6. The test validation system of claim 4, wherein the test validation module comprises at least: the system comprises a service capability test verification unit, an autonomous computer comprehensive test tool set unit and an autonomous computing service test tool unit, wherein:
the service capability test verification unit is used for determining a service capability test verification software toolset according to typical service capability verification requirements of systems under different application modes;
the integrated test tool set unit of the self-host computer is used for testing the delay and bandwidth, the fixed point and floating point capability, the input and output port performance, the network performance and the data access bandwidth of the computer system and evaluating the device level, the board level and the performance of the whole computer of different types;
the autonomic computing service test tool unit is used for testing and verifying the resource virtualization management scheduling capability, the damage-resistant disaster-tolerant capability, the demand deployment capability, the opening and withdrawing capability and the maintenance and guarantee capability of the computing service guarantee system based on the hardware platform.
7. The test validation system of claim 1, wherein the environmental data of the simulated operation comprises a test net and a service net, wherein the service net comprises at least a physical network and a simulation network;
the test resource scheduling management tool unit, the simulation data environment module and the test verification module are respectively interconnected through a test network;
the general service simulation subsystem, the central task simulation unit, the daily application simulation unit and the situation handling simulation unit are at least provided with two network ports which are respectively interconnected with the test network and the service network.
8. The system of claim 1, wherein the system comprises at least a network simulation and capability test layer, an autonomic computing storage platform test layer, a business application simulation layer, a business capability and basic service test layer.
9. The test validation system of claim 8, wherein the autonomic computing storage platform test layer comprises at least computing storage devices of different systems of the commercial computer, the autonomic computer, and the autonomic computing services platform.
10. The test validation system of claim 1, wherein the test validation system comprises at least a system under test, a companion system, and a test system; wherein,
the system to be tested comprises a real service system;
the accompanying and testing system at least comprises a general service simulation subsystem, a simulation data basic environment module, a central task simulation unit, a daily application simulation unit and a situation handling simulation unit;
the test system comprises a service capability test verification system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210534702.2A CN114896108A (en) | 2022-05-17 | 2022-05-17 | Test verification system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210534702.2A CN114896108A (en) | 2022-05-17 | 2022-05-17 | Test verification system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114896108A true CN114896108A (en) | 2022-08-12 |
Family
ID=82723695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210534702.2A Pending CN114896108A (en) | 2022-05-17 | 2022-05-17 | Test verification system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114896108A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116684301A (en) * | 2023-06-26 | 2023-09-01 | 北京永信至诚科技股份有限公司 | Method, system, equipment and storage medium for realizing cross-range task collaboration |
| CN117592311A (en) * | 2024-01-17 | 2024-02-23 | 上海朋熙半导体有限公司 | Multi-level simulation method, device and equipment for workflow and readable medium |
-
2022
- 2022-05-17 CN CN202210534702.2A patent/CN114896108A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116684301A (en) * | 2023-06-26 | 2023-09-01 | 北京永信至诚科技股份有限公司 | Method, system, equipment and storage medium for realizing cross-range task collaboration |
| CN116684301B (en) * | 2023-06-26 | 2024-01-30 | 北京永信至诚科技股份有限公司 | Method, system, equipment and storage medium for realizing cross-range task collaboration |
| CN117592311A (en) * | 2024-01-17 | 2024-02-23 | 上海朋熙半导体有限公司 | Multi-level simulation method, device and equipment for workflow and readable medium |
| CN117592311B (en) * | 2024-01-17 | 2024-04-16 | 上海朋熙半导体有限公司 | Multi-level simulation method, device and equipment for workflow and readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210311858A1 (en) | System and method for providing a test manager for use with a mainframe rehosting platform | |
| KR101132560B1 (en) | System and method for automatic interface testing based on simulation for robot software components | |
| CN102141962B (en) | Safety distributed test framework system and test method thereof | |
| CN114896108A (en) | Test verification system | |
| CN109948228A (en) | A kind of confronting simulation and Effectiveness Evaluation System based on equipment parametrization | |
| CN102523030B (en) | Communication satellite effective load test system simulation platform | |
| D'Angelo et al. | Modeling the Internet of Things: a simulation perspective | |
| KR101056682B1 (en) | A weapon simulation system and the same method based on the component | |
| CN114661622B (en) | Test middleware and test management system | |
| CN110991040A (en) | Complex product collaborative simulation environment construction method | |
| CN107193741A (en) | A kind of General Aviation electronic tasks system product tests verification environment | |
| CN104331530A (en) | An electronic warfare visual simulation platform based on XML description and working method thereof | |
| CN114707236A (en) | Model-based virtual-real combined simulation test method | |
| CN113837908A (en) | Course-based network training system and method | |
| Rak | Performance modeling using queueing Petri nets | |
| Truong | Using IoTCloudSamples as a software framework for simulations of edge computing scenarios | |
| CN113704095A (en) | Automatic generation method and device for automatic interface test case | |
| Li et al. | Architecture of collaborative design grid and its application based on LAN | |
| CN110650063A (en) | Centralized bank third-party software simulation system and method | |
| CN103176904B (en) | A kind of simulation test platform software architecture and the method for testing based on this framework | |
| CN113094266B (en) | Fault testing method, platform and equipment for container database | |
| Tsoutsa et al. | Service Simulation in Industry 4.0: A Comparison of Simulators | |
| CN115801599B (en) | Evaluation model construction method based on using section | |
| CN110414097A (en) | IMA system resource configuration verification method and system, computer readable storage medium | |
| CN115407321A (en) | Vehicle-mounted radar testing continuous integration platform constructed based on real dynamic scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |